File size: 4,510 Bytes
6c6f774 9098f0f 6c6f774 b586d57 a9ef974 b586d57 6c6f774 b586d57 9098f0f b586d57 9098f0f b586d57 6c6f774 9098f0f 6c6f774 9098f0f b586d57 9098f0f 6c6f774 9098f0f 6c6f774 9098f0f 6c6f774 9098f0f 6c6f774 b586d57 9098f0f 6c6f774 9098f0f 6c6f774 b586d57 9098f0f b586d57 9098f0f 6c6f774 9098f0f 6c6f774 9098f0f 6c6f774 b586d57 6c6f774 b586d57 9098f0f 6c6f774 9098f0f 6c6f774 9098f0f b586d57 9098f0f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 |
---
library_name: peft
base_model: mistralai/Mistral-7B-v0.1
license: apache-2.0
---
# Model Card for MedMistral-7B
## Model Details
### Model Description
MedMistral-7B is a Mistral fine tune 180993 samples from the [medmcqa](https://huggingface.co/datasets/medmcqa)
dataset.
- **Developed by:** [segmed.ai](https://segmed.ai)
- **Model type:** QLoRA Fine tune Mistral 7B
- **License:** Apache 2.0
- **Finetuned from model :** [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)
### Model Sources [optional]
<!-- Provide the basic links for the model. -->
- **Repository:** https://github.com/segmed/med_mistral
## Uses
This model is to demonstrate medical domain knowledge, but is not intended for medical advise.
### Direct Use
Prompts used in training followed this format:
```
You are a helpful medical assistant. Your task is to answer the following question one of the options and explain why.
### Question: Turban epiglottitis is a clinical finding in - [0] Tubercular laryngitis [1] Tubercular pharyngitis [2] Polypoid degeneration of vocal cord [3] Subglottic hemangioma
### Answer: 0
### Explanation: Ans. is 4a' i.e.. Tubercular laryngitis Laryngeal examination in TB lary ngitiso Hyperaemia of the vocal cord in its whole extent or confined to posterior part with impairment of adduction is the first sign.o Swelling in the interarytenoid region giving a mammilated appearance,o Ulceration of vocal cord giving mouse-nibbled appearance.o Superficial ragged ulceration on the arytenoids and interarytenoid region,o Granulation tissue in interarvtenoid region or vocal process of arytenoid,o Pseudoedema of the epiglottis 'Turban epiglottis",o Swelling of ventricular bands and aryepiglottic folds,o Marked pallor of surrounding mucosa.
```
And for inference, you would need to remove the answer.
```
You are a helpful medical assistant. Your task is to answer the following question one of the options and explain why.
### Question: Turban epiglottitis is a clinical finding in - [0] Tubercular laryngitis [1] Tubercular pharyngitis [2] Polypoid degeneration of vocal cord [3] Subglottic hemangioma
### Answer:
```
### Downstream Use
This model could be further fine tuned on your specific medical dataset.
## Bias, Risks, and Limitations
This model is not intended for medical use.
## How to Get Started with the Model
Download the model and call generate_tokens using the prompt format defined above.
```
def generate_tokens(m, prompt, max_new_tokens=32):
model_input = tokenizer(prompt, return_tensors="pt").to("cuda")
m.eval()
with torch.no_grad():
return tokenizer.decode(m.generate(**model_input, max_new_tokens=max_new_tokens, do_sample=True, top_k=0, num_return_sequences=1, temperature=0.1, eos_token_id=tokenizer.eos_token_id)[0].cuda())
```
These are the hyper-parameters which yielded the best results when experimenting.
## Training Details
### Training Data
180k samples from the [medmcqa](https://huggingface.co/datasets/medmcqa) were used for training. 914 samples were reserved for test and eval.
A small number of samples over 512 tokens were removed to simplify training and to keep the max tokens size small.
### Training Procedure
Training
#### Training Hyperparameters
- **Training regime:** This used the standard qLoRA peft parameters as defined by
```
peft_config = LoraConfig(
r=16,
lora_alpha=16,
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM",
target_modules=[
"q_proj",
"k_proj",
"v_proj",
"o_proj",
"gate_proj",
"up_proj",
"down_proj",
"lm_head",
]
)
```
#### Speeds, Sizes, Times [optional]
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
[More Information Needed]
## Evaluation
Evaluation was performed on the holdout of 914 samples from the medmcqa dataset. Given the multiple choice nature of the data, the F1 was computed and explanations thrown away.
F1 medmcqa : 66%
Evaluation was also performed on the [pubmed_qa](https://huggingface.co/datasets/pubmed_qa) labeled dataset. Since Yes/No answers were provided, an F1 was calculated. I waant to point out this evaluation is on a completely different dataset with different prompt format.
F1 pubmed_qa:
|