Text Generation
Transformers
Safetensors
imp_phi3
conversational
custom_code
Oyoy1235 commited on
Commit
b134a69
1 Parent(s): a3cfd11

update readme

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -12,7 +12,7 @@ datasets:
12
 
13
  ## Introduction
14
 
15
- The Imp project aims to provide a family of a strong multimodal `small` language models (MSLMs). Our `Imp-v1.5-4B-Phi3` is a strong MSLM with only **4B** parameters, which is build upon a small yet powerful SLM [Phi-3 ](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct)(3.8B) and a powerful visual encoder [SigLIP ](https://huggingface.co/google/siglip-so400m-patch14-384)(0.4B), and trained on 1M mixed dataset.
16
 
17
 
18
  We release our model weights and provide an example below to run our model . Detailed technical report and corresponding training/evaluation code will be released soon on our [GitHub repo](https://github.com/MILVLG/imp). We will persistently improve our model and release the next versions to further improve model performance :)
@@ -27,7 +27,7 @@ pip install transformers # latest version is ok, but we recommend v4.36.0
27
  pip install -q pillow accelerate einops
28
  ```
29
 
30
- You can use the following code for model inference. The format of text instruction is similar to [LLaVA](https://github.com/haotian-liu/LLaVA). A Colab page to run this example is provided [here](https://colab.research.google.com/drive/1EBYky6xIPjnlPppo2gZaiNK6gEsjXgom?usp=drive_link#scrollTo=2-VpU6QzWCVZ). Note that the example can only be run on GPUs currently.
31
 
32
  ```Python
33
  import torch
@@ -65,7 +65,8 @@ We conduct evaluation on 9 commonly-used benchmarks, including 5 academic VQA be
65
 
66
  | Models | Size | VQAv2 | GQA | SQA(IMG) | TextVQA | POPE | MME(P) | MMB |MMB_CN|MM-Vet|
67
  |:--------:|:-----:|:----:|:-------------:|:--------:|:-----:|:----:|:-------:|:-------:|:-------:|:-------:|
68
- | Imp-v1.5-4B-Phi3| 4B | 81.5 | 63.5 | 78.0|60.2 | 86.9| 1507.7 |73.3 |61.1|44.6|
 
69
 
70
 
71
 
 
12
 
13
  ## Introduction
14
 
15
+ The Imp project aims to provide a family of strong multimodal lightweight LMMs. Our `Imp-v1.5-4B-Phi3` is a strong MSLM with only **4B** parameters, which is build upon a small yet powerful SLM [Phi-3 ](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct)(3.8B) and a powerful visual encoder [SigLIP ](https://huggingface.co/google/siglip-so400m-patch14-384)(0.4B), and trained on 1M mixed dataset.
16
 
17
 
18
  We release our model weights and provide an example below to run our model . Detailed technical report and corresponding training/evaluation code will be released soon on our [GitHub repo](https://github.com/MILVLG/imp). We will persistently improve our model and release the next versions to further improve model performance :)
 
27
  pip install -q pillow accelerate einops
28
  ```
29
 
30
+ You can use the following code for model inference. The format of text instruction is similar to [LLaVA](https://github.com/haotian-liu/LLaVA). Note that the example can only be run on GPUs currently.
31
 
32
  ```Python
33
  import torch
 
65
 
66
  | Models | Size | VQAv2 | GQA | SQA(IMG) | TextVQA | POPE | MME(P) | MMB |MMB_CN|MM-Vet|
67
  |:--------:|:-----:|:----:|:-------------:|:--------:|:-----:|:----:|:-------:|:-------:|:-------:|:-------:|
68
+ | Imp-v1.5-4B-Phi3| 4B | **81.7** | 63.4 | 76.3|- | **87.0**| 1503.9 |**74.1** |**64.8**|-|
69
+ | Imp-v1.5-4B-Phi3| 4B | 81.5 | **63.5** | **78.0**|60.2 | 86.9| **1507.7** |73.3 |61.1|44.6|
70
 
71
 
72