Effect of the fine-tuning

#12
by mmbela - opened

Without quantization, testing ep2 and ep3 under the same conditions, they seem to have the same knowledge of the new feature (reflection), but the overall, original knowledge of ep3 is weaker, which is not unusual for finetuning. I think ep2 performs better, it would be worth trying ep1 as well.

That is funny, because the model files of ep2 and ep3 are exactly the same. You can see that here in the community discussion "mattshumer/ref_70_e3 and mattshumer/Reflection-Llama-3.1-70B-ep2-working are the SAME." and you can even compare the hash values (SHA256) of all the model files uploaded. The hashes prove that the uploaded files of ep2 and ep3 are exactly the same files, identical in every single bit.
You just had more "luck" testing one over the other. Given that and the other technical fails on this repository here, I don't think the models uploaded here deserve the attention they are getting.

You are right.

mmbela changed discussion status to closed

Sign up or log in to comment