princeton-nlp

/

Sheared-LLaMA-1.3B

like 91

Very interested in your research on pruning and dynamic batch training and to see where it evolves. We wanted to share with you that we are seeing some of the best RAG instruct fine-tuning (for a small model) built on top of the Sheared-LLama-1.3B, in particular, and would welcome you to check it out (llmware/bling-sheared-llama-1.3-0.1) - we just posted the RAG finetuned model and will be publishing some benchmark "RAG-instruct" test evaluations in the next couple of weeks. Would look forward to chances to collaborate in the future.