license:mit arxiv:2303.17557 region:us
---
license: mit
---
This repository contains the results of all experiments (inlcuding every single hyperparameter run) reported in the following paper:

Orhan AE (2023) [Recognition, recall, and retention of few-shot memories in large language models.](https://arxiv.org/abs/2303.17557) arXiv:2303.17557.

A brief description of the directories included in this repository:

* [`evals`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/evals): contains the results of all recognition experiments
* [`recalls`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/recalls): contains the results of all recall experiments
* [`re-evals`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/re-evals): contains the results of all recognition experiments during the retention phase
* [`re-recalls`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/re-recalls): contains the results of all recall experiments during the retention phase
* [`scratch-evals`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/scratch-evals), [`scratch-recalls`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/scratch-recalls), [`scratch-re-evals`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/scratch-re-evals), [`scratch-re-recalls`](https://huggingface.co/datasets/eminorhan/llm-memory/tree/main/scratch-re-recalls): similar to the above, but the results are for the `gpt-j-6B-st` model trained from scratch on [`wikitext-103-raw-v1`](https://huggingface.co/datasets/wikitext). 