JustinLin610 commited on
Commit
fe49e57
1 Parent(s): 2f13d6c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -3,10 +3,14 @@ license: apache-2.0
3
  ---
4
 
5
  # OFA-large-caption
 
 
6
  This is the **large** version of OFA model finetuned for **image captioning**. OFA is a unified multimodal pretrained model that unifies modalities (i.e., cross-modality, vision, language) and tasks (e.g., image generation, visual grounding, image captioning, image classification, text generation, etc.) to a simple sequence-to-sequence learning framework.
7
 
8
  The directory includes 4 files, namely `config.json` which consists of model configuration, `vocab.json` and `merge.txt` for our OFA tokenizer, and lastly `pytorch_model.bin` which consists of model weights. There is no need to worry about the mismatch between Fairseq and transformers, since we have addressed the issue yet.
9
 
 
 
10
  To use it in transformers, please refer to https://github.com/OFA-Sys/OFA/tree/feature/add_transformers. Install the transformers and download the models as shown below.
11
  ```
12
  git clone --single-branch --branch feature/add_transformers https://github.com/OFA-Sys/OFA.git
@@ -15,7 +19,7 @@ git clone https://huggingface.co/OFA-Sys/OFA-large-caption
15
  ```
16
  After, refer the path to OFA-large to `ckpt_dir`, and prepare an image for the testing example below. Also, ensure that you have pillow and torchvision in your environment.
17
 
18
- ```
19
  >>> from PIL import Image
20
  >>> from torchvision import transforms
21
  >>> from transformers import OFATokenizer, OFAModel
 
3
  ---
4
 
5
  # OFA-large-caption
6
+
7
+ ## Introduction
8
  This is the **large** version of OFA model finetuned for **image captioning**. OFA is a unified multimodal pretrained model that unifies modalities (i.e., cross-modality, vision, language) and tasks (e.g., image generation, visual grounding, image captioning, image classification, text generation, etc.) to a simple sequence-to-sequence learning framework.
9
 
10
  The directory includes 4 files, namely `config.json` which consists of model configuration, `vocab.json` and `merge.txt` for our OFA tokenizer, and lastly `pytorch_model.bin` which consists of model weights. There is no need to worry about the mismatch between Fairseq and transformers, since we have addressed the issue yet.
11
 
12
+
13
+ ## How to use
14
  To use it in transformers, please refer to https://github.com/OFA-Sys/OFA/tree/feature/add_transformers. Install the transformers and download the models as shown below.
15
  ```
16
  git clone --single-branch --branch feature/add_transformers https://github.com/OFA-Sys/OFA.git
 
19
  ```
20
  After, refer the path to OFA-large to `ckpt_dir`, and prepare an image for the testing example below. Also, ensure that you have pillow and torchvision in your environment.
21
 
22
+ ```python
23
  >>> from PIL import Image
24
  >>> from torchvision import transforms
25
  >>> from transformers import OFATokenizer, OFAModel