vinid commited on
Commit
80200b5
1 Parent(s): 7369efb

adding readme.md

Browse files
Files changed (2) hide show
  1. app.py +8 -0
  2. readme.md +27 -0
app.py CHANGED
@@ -1,6 +1,7 @@
1
  import streamlit as st
2
  import os
3
  import torch
 
4
  from transformers import AutoTokenizer
5
  from jax import numpy as jnp
6
  import json
@@ -89,3 +90,10 @@ if query:
89
  )
90
 
91
  st.image(image_paths)
 
 
 
 
 
 
 
 
1
  import streamlit as st
2
  import os
3
  import torch
4
+ from pathlib import Path
5
  from transformers import AutoTokenizer
6
  from jax import numpy as jnp
7
  import json
 
90
  )
91
 
92
  st.image(image_paths)
93
+
94
+
95
+ def read_markdown_file(markdown_file):
96
+ return Path(markdown_file).read_text()
97
+
98
+ intro_markdown = read_markdown_file("readme.md")
99
+ st.markdown(intro_markdown, unsafe_allow_html=True)
readme.md ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Italian CLIP
2
+
3
+ ....
4
+
5
+ # Novel Contributions
6
+ The original CLIP model was trained on 400millions text-image pairs; this amount of data is not available for Italian and the only datasets for captioning in the literature are MSCOCO-IT (translated version of MSCOCO) and WIT. To get competitive results we follewed three directions: 1) more data 2) better augmentation and 3) better training.
7
+
8
+ ## More Data
9
+
10
+ ## Better Augmentations
11
+
12
+ ## Better Training
13
+
14
+ different optimizer and backbone freezing
15
+
16
+ # Scientific Validity
17
+ To better understand how well our clip-italian model works we run an experimental evaluation. Since this is the first clip-based model in Italian, we used the multilingual CLIP model as a comparison baseline.
18
+
19
+ We selected two different tasks:
20
+ + image-retrieval
21
+ + zero-shot classification
22
+
23
+ ## Image Retrieval
24
+
25
+ ## Zero-shot classification
26
+
27
+ # Broader Outlook