cgus
/

Question Answering
GGUF
biology
medical
Inference Endpoints
imatrix
conversational
Edit model card

Apollo2-7B-GGUF

Original model: Apollo2-7B
Made by: FreedomIntelligence

Quantization notes

Made with llama.cpp-b3938 with imatrix file based on Exllamav2 callibration dataset.
This model is meant to run with llama.cpp-compatible apps such as Text-Generation-WebUI, KoboldCpp, Jan, LM Studio and many many others.

Original model card

Democratizing Medical LLMs For Much More Languages

Covering 12 Major Languages including English, Chinese, French, Hindi, Spanish, Arabic, Russian, Japanese, Korean, German, Italian, Portuguese and 38 Minor Languages So far.

๐Ÿ“ƒ Paper โ€ข ๐ŸŒ Demo โ€ข ๐Ÿค— ApolloMoEDataset โ€ข ๐Ÿค— ApolloMoEBench โ€ข ๐Ÿค— Models โ€ข๐ŸŒ Apollo โ€ข ๐ŸŒ ApolloMoE

Apollo

๐ŸŒˆ Update

  • [2024.10.15] ApolloMoE repo is published๏ผ๐ŸŽ‰

Languages Coverage

12 Major Languages and 38 Minor Languages

Click to view the Languages Coverage

ApolloMoE

Architecture

Click to view the MoE routing image

ApolloMoE

Results

Dense

๐Ÿค— Apollo2-0.5B โ€ข ๐Ÿค— Apollo2-1.5B โ€ข ๐Ÿค— Apollo2-2B

๐Ÿค— Apollo2-3.8B โ€ข ๐Ÿค— Apollo2-7B โ€ข ๐Ÿค— Apollo2-9B

Click to view the Dense Models Results

ApolloMoE

Post-MoE

๐Ÿค— Apollo-MoE-0.5B โ€ข ๐Ÿค— Apollo-MoE-1.5B โ€ข ๐Ÿค— Apollo-MoE-7B

Click to view the Post-MoE Models Results

ApolloMoE

Usage Format

Apollo2
  • 0.5B, 1.5B, 7B: User:{query}\nAssistant:{response}<|endoftext|>
  • 2B, 9B: User:{query}\nAssistant:{response}<eos>
  • 3.8B: <|user|>\n{query}<|end|><|assisitant|>\n{response}<|end|>
Apollo-MoE
  • 0.5B, 1.5B, 7B: User:{query}\nAssistant:{response}<|endoftext|>

Dataset & Evaluation

  • Dataset ๐Ÿค— ApolloMoEDataset

    Click to expand

    ApolloMoE

  • Evaluation ๐Ÿค— ApolloMoEBench

    Click to expand
    • EN:

      • MedQA-USMLE
      • MedMCQA
      • PubMedQA: Because the results fluctuated too much, they were not used in the paper.
      • MMLU-Medical
        • Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
    • ZH:

      • MedQA-MCMLE
      • CMB-single: Not used in the paper
        • Randomly sample 2,000 multiple-choice questions with single answer.
      • CMMLU-Medical
        • Anatomy, Clinical_knowledge, College_medicine, Genetics, Nutrition, Traditional_chinese_medicine, Virology
      • CExam: Not used in the paper
        • Randomly sample 2,000 multiple-choice questions
    • ES: Head_qa

    • FR:

      • Frenchmedmcqa
      • [MMLU_FR]
        • Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
    • HI: MMLU_HI

      • Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
    • AR: MMLU_AR

      • Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
    • JA: IgakuQA

    • KO: KorMedMCQA

    • IT:

      • MedExpQA
      • [MMLU_IT]
        • Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
    • DE: BioInstructQA: German part

    • PT: BioInstructQA: Portuguese part

    • RU: RuMedBench

Results reproduction

Click to expand

We take Apollo2-7B or Apollo-MoE-0.5B as example

  1. Download Dataset for project:

    bash 0.download_data.sh  
    
  2. Prepare test and dev data for specific model:

    • Create test data for with special token
    bash 1.data_process_test&dev.sh
    
  3. Prepare train data for specific model (Create tokenized data in advance):

    • You can adjust data Training order and Training Epoch in this step
    bash 2.data_process_train.sh
    
  4. Train the model

    • If you want to train in Multi Nodes please refer to ./src/sft/training_config/zero_multi.yaml
    bash 3.single_node_train.sh
    
  5. Evaluate your model: Generate score for benchmark

    bash 4.eval.sh
    

Citation

Please use the following citation if you intend to use our dataset for training or evaluation:

@misc{zheng2024efficientlydemocratizingmedicalllms,
      title={Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts}, 
      author={Guorui Zheng and Xidong Wang and Juhao Liang and Nuo Chen and Yuping Zheng and Benyou Wang},
      year={2024},
      eprint={2410.10626},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.10626}, 
}
Downloads last month
214
GGUF
Model size
7.62B params
Architecture
qwen2

4-bit

5-bit

6-bit

8-bit

Inference Examples
Unable to determine this model's library. Check the docs .

Model tree for cgus/Apollo2-7B-iMat-GGUF

Base model

Qwen/Qwen2-7B
Quantized
(2)
this model

Dataset used to train cgus/Apollo2-7B-iMat-GGUF