File size: 1,087 Bytes
256a159 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
# InstructBLIP
### Prepare the environment
```sh
git clone https://github.com/salesforce/LAVIS.git
cd ./LAVIS
pip install -e .
```
### Modify the config
Modify the config of InstructBlip, like model path of LLM and Qformer.
Then update `tasks.py` like the following code snippet.
```python
from mmengine.config import read_base
with read_base():
from .instructblip.instructblip_mmbench import (instruct_blip_dataloader,
instruct_blip_evaluator,
instruct_blip_load_from,
instruct_blip_model)
models = [instruct_blip_model]
datasets = [instruct_blip_dataloader]
evaluators = [instruct_blip_evaluator]
load_froms = [instruct_blip_load_from]
num_gpus = 8
num_procs = 8
launcher = 'pytorch' # or 'slurm'
```
### Start evaluation
#### Slurm
```sh
cd $root
python run.py configs/multimodal/tasks.py --mm-eval --slurm -p $PARTITION
```
#### PyTorch
```sh
cd $root
python run.py configs/multimodal/tasks.py --mm-eval
```
|