Hugging Face's logo
Jim Lai's picture
42

Jim Lai

grimjim
kamazon's profile picture Tanvir1337's profile picture smsaurabhv's profile picture
·

AI & ML interests

Experimenting primarily with 7B-12B parameter text completion models. Not all models are intended for direct use, but aim for educational and/or merge purposes.

Recent Activity

New activity 23 days ago
anthracite-org/magnum-v4-27b
New activity 23 days ago
anthracite-org/magnum-v4-12b
updated a model 27 days ago
grimjim/Magnolia-v1-12B-GGUF

Organizations

Social Post Explorers's profile picture Debased AI's profile picture Anthracite's profile picture Anthracite Core's profile picture

Posts 14

view post
Post
1950
To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
grimjim/Llama-Nephilim-Metamorphosis-v2-8B
view post
Post
1923
I was reading through an abstract and found myself wondering how much LLM performance is being left on the table due to insufficient curation of training datasets: "Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning" by Kaur, Park, Goyal, Arora.
https://arxiv.org/abs/2408.14774
In particular, the observation that "Introducing low quality answers ("shirkers") in 20% of Instruct-SkillMix examples causes performance to plummet..." had me wondering how many ostensibly good datasets out there are in fact populated with a significant number of "shirkers".
View all posts

Collections 5