@inproceedings{
bigelow2023subjective,
title={Subjective Randomness and In-Context Learning},
author={Eric J Bigelow and Ekdeep Singh Lubana and Robert P. Dick and Hidenori Tanaka and Tomer Ullman},
booktitle={UniReps:  the First Workshop on Unifying Representations in Neural Models},
year={2023},
url={https://openreview.net/forum?id=JFnjdl2Vc2}
}
Large language models (LLMs) exhibit intricate capabilities, often achieving high performance on tasks they were not explicitly trained for. The precise nature of LLM capabilities is often unclear, with different prompts eliciting different capabilities, especially when used with in-context learning (ICL). We propose a "Cognitive Interpretability" framework that enables us to analyze ICL dynamics to understand latent concepts underlying LLMs' behavioral patterns. This provides a more nuanced understanding than posthoc evaluation benchmarks, but does not require observing model internals as a mechanistic interpretation would require. Inspired by the cognitive science of human randomness perception, we use random binary sequences as context and study dynamics of ICL by manipulating properties of context data, such as sequence length. In the latest GPT-3.5+ models, we find emergent abilities to generate pseudo-random numbers and learn basic formal languages, with striking ICL dynamics where model outputs transition sharply from pseudo-random behaviors to deterministic repetition.
Subjective Randomness and In-Context Learning
Eric J Bigelow Ekdeep Singh Lubana Robert P. Dick Hidenori Tanaka Tomer Ullman