metadata
datasets:
- axiong/pmc_oa_beta
language:
- en
library_name: open_clip
pipeline_tag: text-to-image
tags:
- biology
- chemistry
- medical
Model Description
The model is fine-tuned from openai's ViT-L-14 using PMC_OA_beta and roco's data sets, using the tool open_clip（https://github.com/mlfoundations/open_clip）.
Training
python -m training.main \
--save-frequency 2 \
--zeroshot-frequency 1 \
--report-to tensorboard \
--train-data="/home/data1/ryanyip/huggingface-models/pmc_oa_beta/train.csv" \
--val-data="/home/data1/ryanyip/huggingface-models/pmc_oa_beta/sample_valid.csv" \
--csv-separator "," \
--csv-img-key image \
--csv-caption-key caption \
--warmup 10000 \
--batch-size=128 \
--lr=1e-5 \
--wd=0.2 \
--epochs=30 \
--workers=8 \
--model "ViT-L-14" \
--name "pmc_vit_l_14" \
--pretrained "ViT-L-14_state_dict.pt" \
--save-most-recent
ViT-L-14_state_dict.pt is the pretrained weight from openai/ViT-L-14