You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
You agree not to use the model for healthcare decision-making or commercial use
Log in or Sign Up to review the conditions and access this model content.
Clinical Camel
Model Description
Clinical Camel is an open large language model (LLM), fine-tuned on the LLaMA-2 70B architecture using QLoRA. It is tailored for the medical and clinical research, capable of processing and generating relevant content.
Review our pre-print for more details: Clinical Camel - Pre-print
Performance
Clinical Camel demonstrates competitive performance on medical benchmarks.
Table: Five-Shot Performance of Clinical Camel-70B (C70), GPT3.5, GPT4, and Med-PaLM 2 on Various Medical Datasets
| Dataset | ClinicalCamel-70B | GPT3.5 | GPT4 | Med-PaLM 2 |
|---|---|---|---|---|
| MMLU Anatomy | 65.2 | 60.7 | 80.0 | 77.8 |
| MMLU Clinical Knowledge | 72.8 | 68.7 | 86.4 | 88.3 |
| MMLU College Biology | 81.2 | 72.9 | 93.8 | 94.4 |
| MMLU College Medicine | 68.2 | 63.6 | 76.3 | 80.9 |
| MMLU Medical Genetics | 69.0 | 68.0 | 92.0 | 90.0 |
| MMLU Professional Medicine | 75.0 | 69.8 | 93.8 | 95.2 |
| MedMCQA | 54.2 | 51.0 | 72.4 | 71.3 |
| MedQA (USMLE) | 60.7 | 53.6 | 81.4 | 79.7 |
| PubMedQA | 77.9 | 60.2 | 74.4 | 79.2 |
| USMLE Sample Exam | 64.3 | 58.5 | 86.6 | - |
Evaluation Datasets:
The performance of Clinical Camel was benchmarked across several datasets, including:
Evaluation Reproduction:
To reproduce the evaluations with lm-evaluation-harness see the 'TaskFiles' folder
- Downloads last month
- 1,607