Shears: Unstructured Sparsity with Neural Low-rank Adapter Search
Paper
•
2404.10934
•
Published
Shears Models (Shears: Unstructured Sparsity with Neural Low-rank Adapter Search)
Note Shears fine-tuned models for Llama-7B on Math Instruction Tuning (50% sparsity)
Note Shears fine-tuned models for Llama-7B on CS Instruction Tuning (50% sparsity)
Note Shears fine-tuned models for Llama-13B on Math Instruction Tuning (50% sparsity)
Note Shears base models (sparse via Wanda) for MPT-7B
Note Shears fine-tuned models for MPT-7B on GSM8K (50% sparsity)