2 20 73

neuralink

AI & ML interests

Don't bet against deep learning

Articles

A failed experiment: Infini-Attention, and why we should keep trying?

Aug 14

• 42

Organizations

neuralink's activity

upvoted a paper 21 days ago

Small-scale proxies for large-scale Transformer training instabilities

Paper • 2309.14322 • Published Sep 25, 2023 • 19

upvoted an article 25 days ago

Article

How NuminaMath Won the 1st AIMO Progress Prize

Jul 11

• 93

upvoted a paper about 1 month ago

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Paper • 2201.02177 • Published Jan 6, 2022 • 2

upvoted an article about 2 months ago

Article

A failed experiment: Infini-Attention, and why we should keep trying?

Aug 14

• 42

upvoted 2 papers about 2 months ago

Grokfast: Accelerated Grokking by Amplifying Slow Gradients

Paper • 2405.20233 • Published May 30 • 5

Transformer Explainer: Interactive Learning of Text-Generative Models

Paper • 2408.04619 • Published Aug 8 • 152

upvoted a paper 3 months ago

What matters when building vision-language models?

Paper • 2405.02246 • Published May 3 • 98

upvoted a paper 4 months ago

DiPaCo: Distributed Path Composition

Paper • 2403.10616 • Published Mar 15 • 12

upvoted 2 articles 4 months ago

Article

Putting RL back in RLHF

Jun 12

• 60

Article

Let's talk about LLM evaluation

•

May 23

• 107

upvoted 3 papers 6 months ago

upvoted a paper 7 months ago

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Paper • 2402.15627 • Published Feb 23 • 33

upvoted 2 papers 8 months ago

FP8-LM: Training FP8 Large Language Models

Paper • 2310.18313 • Published Oct 27, 2023 • 31

Zero Bubble Pipeline Parallelism

Paper • 2401.10241 • Published Nov 30, 2023 • 22

upvoted a paper 9 months ago

Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

Paper • 2312.17172 • Published Dec 28, 2023 • 26

upvoted 3 papers 11 months ago

OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents

Paper • 2306.16527 • Published Jun 21, 2023 • 47

RWKV: Reinventing RNNs for the Transformer Era

Paper • 2305.13048 • Published May 22, 2023 • 13

Llemma: An Open Language Model For Mathematics

Paper • 2310.10631 • Published Oct 16, 2023 • 49