doc: update README

Former-commit-id: ed59fc60cf2ffe870f02931c96dc114a7c87737b

README.md

@@ -1,42 +1,76 @@
-## DALL-E Mini - Generate image from text
+---
+title: Dalle Mini
+emoji: 🎨
+colorFrom: red
+colorTo: blue
+sdk: streamlit
+app_file: app/app.py
+pinned: false
+---
 
-## Tentative Strategy of training (proposed by Luke and Suraj)
+# DALL-E Mini
 
-### Data: 
-* [Conceptual 12M](https://github.com/google-research-datasets/conceptual-12m) Dataset (already loaded and preprocessed in TPU VM by Luke).
-* [YFCC100M Subset](https://github.com/openai/CLIP/blob/main/data/yfcc100m.md)
-* [Coneptual Captions 3M](https://github.com/google-research-datasets/conceptual-captions)
+_Generate images from a text prompt_
 
-### Architecture: 
-  * Use the Taming Transformers VQ-GAN (with 16384 tokens)
-  * Use a seq2seq (language encoder --> image decoder) model with a pretrained non-autoregressive encoder (e.g. BERT) and an autoregressive decoder (like GPT). 
+TODO: add some cool example
 
-### Remaining Architecture Questions: 
-  * Whether to freeze the text encoder?
-  * Whether to finetune the VQ-GAN?
-  * Which text encoder to use (e.g. BERT, RoBERTa, etc.)?
-  * Hyperparameter choices for the decoder (e.g. positional embedding, initialization, etc.)
+## [Create my own images with the demo →](TODO)
 
-## TODO
+## How does it work?
 
-* experiment with flax/jax and setup of the TPU instance that we should get shortly
-* work on dataset loading - [see suggested datasets](https://discuss.huggingface.co/t/dall-e-mini-version/7324/4)
-* Optionally create the OpenAI YFCC100M subset (see [this post](https://discuss.huggingface.co/t/dall-e-mini-version/7324/30?u=boris))
-* work on text/image encoding
-* concatenate inputs (not sure if we need fixed length for text or use a special token separating text & image)
-* adapt training script
-* create inference function
-* integrate CLIP for better results (only if we have the time)
-* work on a demo (streamlit or colab or maybe just HF widget)
-* document (set up repo on model hub per instructions, start on README writeup…)
-* help with coordinating activities & progress
+Refer to [our report](TODO).
 
+## Development
 
-## Dependencies Installation
-You should create a new python virtual environment and install the project dependencies inside the virtual env. You need to use the `-f` (`--find-links`) option for `pip` to be able to find the appropriate `libtpu` required for the TPU hardware:
+This section is for the adventurous people wanting to look into the code.
+
+### Dependencies Installation
+
+The root folder and associated `requirements.txt` is only for the app.
+
+You will find necessary requirements in each sub-section.
+
+You should create a new python virtual environment and install the project dependencies inside the virtual env. You need to use the `-f` (`--find-links`) option for `pip` to be able to find the appropriate `libtpu` required for the TPU hardware.
+
+Adapt the installation to your own hardware and follow library installation instructions.
 
 ```
 $ pip install -r requirements.txt -f https://storage.googleapis.com/jax-releases/libtpu_releases.html
 ```
 
 If you use `conda`, you can create the virtual env and install everything using: `conda env update -f environments.yaml`
+
+### Training of VQGAN
+
+The VQGAN was trained using [taming-transformers](https://github.com/CompVis/taming-transformers).
+
+We recommend using the latest version available.
+
+### Conversion of VQGAN to JAX
+
+Use [patil-suraj/vqgan-jax](https://github.com/patil-suraj/vqgan-jax).
+
+### Training of Seq2Seq
+
+Refer to `seq2seq` folder (some parameters may have been hardcoded for convenience when training on our TPU VM).
+
+### Inference
+
+Refer to the demo notebooks.
+TODO: add links
+
+## Authors
+
+- [Boris Dayma](https://github.com/borisdayma)
+- [Suraj Patil](https://github.com/patil-suraj)
+- [Pedro Cuenca](https://github.com/pcuenca)
+- [Khalid Saifullah](https://github.com/khalidsaifullaah)
+- [Tanishq Abraham](https://github.com/tmabraham)
+- [Phúc Lê Khắc](https://github.com/lkhphuc)
+- [Luke Melas](https://github.com/lukemelas)
+- [Ritobrata Ghosh](https://github.com/ghosh-r)
+
+## Acknowledgements
+
+- 🤗 Hugging Face for organizing [the FLAX/JAX community week](https://github.com/huggingface/transformers/tree/master/examples/research_projects/jax-projects)
+- Google Cloud team for providing access to TPU's