stereoplegic
's Collections
Distributed
updated
A Unified View of Long-Sequence Models towards Modeling Million-Scale
Dependencies
Paper
•
2302.06218
•
Published
•
1
ZeRO++: Extremely Efficient Collective Communication for Giant Model
Training
Paper
•
2306.10209
•
Published
•
2
SE-MoE: A Scalable and Efficient Mixture-of-Experts Distributed Training
and Inference System
Paper
•
2205.10034
•
Published
•
1
A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize
Mixture-of-Experts Training
Paper
•
2303.06318
•
Published
•
1
Ultra-Long Sequence Distributed Transformer
Paper
•
2311.02382
•
Published
•
2
HetuMoE: An Efficient Trillion-scale Mixture-of-Expert Distributed
Training System
Paper
•
2203.14685
•
Published
•
1
Pathways: Asynchronous Distributed Dataflow for ML
Paper
•
2203.12533
•
Published
•
2
Distributed Pruning Towards Tiny Neural Networks in Federated Learning
Paper
•
2212.01977
•
Published
•
1
Punica: Multi-Tenant LoRA Serving
Paper
•
2310.18547
•
Published
•
2
Distributed Inference and Fine-tuning of Large Language Models Over The
Internet
Paper
•
2312.08361
•
Published
•
25
Training Transformers Together
Paper
•
2207.03481
•
Published
•
5