MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Paper
•
2402.15627
•
Published
•
32
LLM
-
-
Beyond Language Models: Byte Models are Digital World Simulators
Paper • 2402.19155 • Published • 47 -
VisionLLaMA: A Unified LLaMA Interface for Vision Tasks
Paper • 2403.00522 • Published • 40 -
Stealing Part of a Production Language Model
Paper • 2403.06634 • Published • 86 -
Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU
Paper • 2403.06504 • Published • 52 -
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models
Paper • 2403.06764 • Published • 24 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 62 -
Layer Skip: Enabling Early Exit Inference and Self-Speculative Decoding
Paper • 2404.16710 • Published • 56 -
Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
Paper • 2406.18629 • Published • 36 -
Simulating Classroom Education with LLM-Empowered Agents
Paper • 2406.19226 • Published • 27 -
SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation
Paper • 2406.19215 • Published • 26 -
Aligning Teacher with Student Preferences for Tailored Training Data Generation
Paper • 2406.19227 • Published • 22 -
T-FREE: Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings
Paper • 2406.19223 • Published • 8 -
Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
Paper • 2406.18676 • Published • 5 -
Direct Preference Knowledge Distillation for Large Language Models
Paper • 2406.19774 • Published • 17 -
We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?
Paper • 2407.01284 • Published • 64