| base_model: | |
| - aaditya/Llama3-OpenBioLLM-70B | |
| library_name: transformers | |
| tags: | |
| - mergekit | |
| - merge | |
| license: llama3 | |
| language: | |
| - en | |
| # Llama-3-Galen-70B-v1 | |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/60c8619d95d852a24572b025/R73wGdZE3GWeF9QZPvruG.jpeg" width="600" /> | |
| This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). | |
| ## Merge Details | |
| ### Merge Method | |
| This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [aaditya/Llama3-OpenBioLLM-70B](https://huggingface.co/aaditya/Llama3-OpenBioLLM-70B) as a base. | |
| ### Evaluation | |
| | Tasks |Version|Filter|n-shot| Metric |Value | |Stderr| | |
| |---------------------|-------|------|-----:|--------|-----:|---|-----:| | |
| |pubmedqa | 1|none | 0|acc |0.7820|± |0.0185| | |
| |professional_medicine| 0|none | 0|acc |0.9375|± |0.0147| | |
| |medical_genetics | 0|none | 0|acc |0.9300|± |0.0256| | |
| |college_medicine | 0|none | 0|acc |0.8555|± |0.0268| | |
| |college_biology | 0|none | 0|acc |0.9375|± |0.0202| | |
| |clinical_knowledge | 0|none | 0|acc |0.9283|± |0.0159| | |
| |anatomy | 0|none | 0|acc |0.8444|± |0.0313| | |
| |medqa_4options |Yaml |none | 0|acc |0.7777|± |0.0117| | |
| | | |none | 0|acc_norm|0.7777|± |0.0117| | |
| |medmcqa |Yaml |none | 0|acc |0.7423|± |0.0068| | |
| | | |none | 0|acc_norm|0.7423|± |0.0068| | |
| **Average:** 0.8594 | |