Update README.md

README.md

@@ -1,11 +1,3 @@
-<!-- ## **HunyuanDiT** -->
-
-<p align="center">
-  <img src="./asset/logo.png"  height=100>
-</p>
-
-# Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
-
 -----
 
 This repo contains PyTorch model definitions, pre-trained weights and inference/sampling code for our paper exploring Hunyuan-DiT. You can find more visualizations on our [project page](https://dit.hunyuan.tencent.com/).
@@ -14,7 +6,7 @@ This repo contains PyTorch model definitions, pre-trained weights and inference/
 > Zhimin Li*, Jianwei Zhang*, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiaoxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang, Meng Chen, Jie Liu, Zheng Fang, Weiyan Wang, Jinbao Xue, Yangyu Tao, JianChen Zhu, Kai Liu, Sihuan Lin, Yifu Sun, Yun Li, Dongdong Wang, Zhichao Hu, Xiao Xiao, Yan Chen, Yuhong Liu, Wei Liu, Di Wang, Yong Yang, Jie Jiang, Qinglin Lu‡
 > <br>Tencent Hunyuan<br>
 
-> [**DialogGen:Multi-modal Interactive Dialogue System for Multi-turn Text-to-Image Generation**](https://hunyuan-dialoggen.github.io/)<br>
+> [**DialogGen:Multi-modal Interactive Dialogue System for Multi-turn Text-to-Image Generation**](https://arxiv.org/abs/2403.08857)<br>
 > Minbin Huang*, Yanxin Long*, Xinchi Deng,  Ruihang Chu, Jiangfeng Xiong, Xiaodan Liang, Hong Cheng, Qinglin Lu&#8224;, Wei Liu
 > <br>Chinese University of Hong Kong, Tencent Hunyuan, Shenzhen Campus of Sun Yat-sen University<br>
 
@@ -23,32 +15,32 @@ This repo contains PyTorch model definitions, pre-trained weights and inference/
 ## 📑 Open-source Plan
 
 - Hunyuan-DiT (Text-to-Image Model)
-  - [x] Inference ✅
-  - [x] Checkpoints ✅
+  - [x] Inference 
+  - [x] Checkpoints 
   - [ ] Distillation Version (Coming soon ⏩️)
   - [ ] TensorRT Version (Coming soon ⏩️)
   - [ ] Training (Coming later ⏩️)
 - DialogGen (Prompt Enhancement Model)
-  - [x] Inference ✅
-- [X] Web Demo (Gradio) ✅
-- [X] Cli Demo ✅
+  - [x] Inference 
+- [X] Web Demo (Gradio) 
+- [X] Cli Demo 
 
 ## Contents
-- [Hunyuan-DiT](#hunyuan-dit-a-powerful-multi-resolution-diffusion-transformer-with-fine-grained-chinese-understanding)
+- [Hunyuan-DiT](#hunyuan-dit--a-powerful-multi-resolution-diffusion-transformer-with-fine-grained-chinese-understanding)
   - [Abstract](#abstract)
-  - [🎉 Hunyuan-DiT Key Features](#hunyuan-dit-key-features)
+  - [🎉 Hunyuan-DiT Key Features](#-hunyuan-dit-key-features)
     - [Chinese-English Bilingual DiT Architecture](#chinese-english-bilingual-dit-architecture)
     - [Multi-turn Text2Image Generation](#multi-turn-text2image-generation)
   - [📈 Comparisons](#comparisons)
   - [🎥 Visualization](#visualization)
   - [📜 Requirements](#requirements)
-  - [🛠 Dependencies and Installation](#dependencies-and-installation)
+  - [🛠 Dependencies and Installation](#%EF%B8%8Fdependencies-and-installation)
   - [🧱 Download Pretrained Models](#download-pretrained-models)
-  - [🔑 Inference](#inference)
+  - [🔑 Inference](#-inference)
     - [Using Gradio](#using-gradio)
     - [Using Command Line](#using-command-line)
     - [More Configurations](#more-configurations)
-  - [🔗 BibTeX](#bibtex)
+  - [🔗 BibTeX](#-bibtex)
 
 ## **Abstract**
 
@@ -60,7 +52,7 @@ Through our carefully designed holistic human evaluation protocol with more than
 ### **Chinese-English Bilingual DiT Architecture**
 Hunyuan-DiT is a diffusion model in the latent space, as depicted in figure below. Following the Latent Diffusion Model, we use a pre-trained Variational Autoencoder (VAE) to compress the images into low-dimensional latent spaces and train a diffusion model to learn the data distribution with diffusion models. Our diffusion model is parameterized with a transformer. To encode the text prompts, we leverage a combination of pre-trained bilingual (English and Chinese) CLIP and multilingual T5 encoder.
 <p align="center">
-  <img src="./asset/framework.png"  height=500>
+  <img src="https://raw.githubusercontent.com/Tencent/HunyuanDiT/main/asset/framework.png"  height=450>
 </p>
 
 ### Multi-turn Text2Image Generation
@@ -70,7 +62,7 @@ step by step. In this section, we will detail how we empower Hunyuan-DiT with th
 conversations and image generation. We train MLLM to understand the multi-round user dialogue
 and output the new text prompt for image generation.
 <p align="center">
-  <img src="./asset/mllm.png"  height=300>
+  <img src="https://raw.githubusercontent.com/Tencent/HunyuanDiT/main/asset/mllm.png"  height=300>
 </p>
 
 ## Comparisons
@@ -85,52 +77,53 @@ In order to comprehensively compare the generation capabilities of HunyuanDiT an
 </thead> 
 <tbody> 
 <tr> 
-    <td>SDXL</td> <td>&#10004</td> <td>64.3</td> <td>60.6</td> <td>91.1</td> <td>76.3</td> <td>42.7</td> 
+    <td>SDXL</td> <td> ✔ </td> <td>64.3</td> <td>60.6</td> <td>91.1</td> <td>76.3</td> <td>42.7</td> 
 </tr> 
 <tr> 
-    <td>PixArt-α</td> <td>&#10004</td> <td>68.3</td> <td>60.9</td> <td>93.2</td> <td>77.5</td> <td>45.5</td> 
+    <td>PixArt-α</td> <td> ✔</td> <td>68.3</td> <td>60.9</td> <td>93.2</td> <td>77.5</td> <td>45.5</td> 
 </tr> 
 <tr> 
-    <td>Playground 2.5</td> <td>&#10004</td> <td>71.9</td> <td>70.8</td> <td>94.9</td> <td>83.3</td> <td>54.3</td> 
+    <td>Playground 2.5</td> <td>✔</td> <td>71.9</td> <td>70.8</td> <td>94.9</td> <td>83.3</td> <td>54.3</td> 
 </tr> 
 
 <tr> 
     <td>SD 3</td> <td>&#10008</td> <td>77.1</td> <td>69.3</td> <td>94.6</td> <td>82.5</td> <td>56.7</td> 
     
 </tr> 
-<tr style="font-weight: bold; background-color: #f2f2f2;"> 
-    <td>Hunyuan-DiT</td><td>&#10004</td> <td>74.2</td> <td>74.3</td> <td>95.4</td> <td>86.6</td> <td>59.0</td> 
-</tr>
 <tr> 
     <td>MidJourney v6</td><td>&#10008</td> <td>73.5</td> <td>80.2</td> <td>93.5</td> <td>87.2</td> <td>63.3</td> 
 </tr> 
 <tr> 
     <td>DALL-E 3</td><td>&#10008</td> <td>83.9</td> <td>80.3</td> <td>96.5</td> <td>89.4</td> <td>71.0</td> 
 </tr> 
+<tr style="font-weight: bold; background-color: #f2f2f2;"> 
+    <td>Hunyuan-DiT</td><td>✔</td> <td>74.2</td> <td>74.3</td> <td>95.4</td> <td>86.6</td> <td>59.0</td> 
+</tr>
 </table> 
 </p>
 
-## 🎥Visualization
+## 🎥 Visualization
 
 * **Chinese Elements**
 <p align="center">
-  <img src="./asset/chinese elements understanding.png"  height=220>
+  <img src="https://raw.githubusercontent.com/Tencent/HunyuanDiT/main/asset/chinese elements understanding.png"  height=220>
 </p>
 
 * **Long Text Input**
 
 
 <p align="center">
-  <img src="./asset/long text understanding.png"  height=310>
+  <img src="https://raw.githubusercontent.com/Tencent/HunyuanDiT/main/asset/long text understanding.png"  height=310>
 </p>
 
 * **Multi-turn Text2Image Generation**
 
+
 [demo video](https://youtu.be/4AaHrYnuIcE)
 
 ---
 
-## 📜Requirements
+## 📜 Requirements
 
 This repo consists of DialogGen (a prompt enhancement model) and Hunyuan-DiT (a text-to-image model).
 
@@ -150,7 +143,7 @@ The following table shows the requirements for running the models (The TensorRT
   * **Recommended**: We recommend using a GPU with 32GB of memory for better generation quality.
 * Tested operating system: Linux
 
-## 🛠️Dependencies and Installation
+## 🛠️ Dependencies and Installation
 
 Begin by cloning the repository:
 ```bash
@@ -175,7 +168,7 @@ python -m pip install -r requirements.txt
 python -m pip install git+https://github.com/Dao-AILab/[email protected]
 ```
 
-## 🧱Download Pretrained Models
+## 🧱 Download Pretrained Models
 To download the model, first install the huggingface-cli. (Detailed instructions are available [here](https://huggingface.co/docs/huggingface_hub/guides/cli).)
 
 ```bash
@@ -193,9 +186,6 @@ huggingface-cli download Tencent-Hunyuan/HunyuanDiT --local-dir ./ckpts
 ```
 Note：If an `No such file or directory: 'ckpts/.huggingface/.gitignore.lock'` like error occurs during the download process, you can ignore the error and retry the command by executing `huggingface-cli download Tencent-Hunyuan/HunyuanDiT --local-dir ./ckpts`
 
-For more information about the model, visit the Hugging Face repository [here](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT).
-
-
 All models will be automatically downloaded. For more information about the model, visit the Hugging Face repository [here](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT).
 
 |       Model        | #Params |                                              Download URL                                               |
@@ -246,6 +236,8 @@ python sample_t2i.py --infer-mode fa --prompt "渔舟唱晚"
 python sample_t2i.py --prompt "渔舟唱晚" --image-size 1280 768
 ```
 
+More example prompts can be found in [example_prompts.txt](example_prompts.txt)
+
 ### More Configurations
 
 We list some more useful configurations for easy usage:
@@ -273,4 +265,4 @@ If you find Hunyuan-DiT useful for your research and applications, please cite u
       author={Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiaoxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang, Meng Chen, Jie Liu, Zheng Fang, Weiyan Wang, Jinbao Xue, Yangyu Tao, JianChen Zhu, Kai Liu, Sihuan Lin, Yifu Sun, Yun Li, Dongdong Wang, Zhichao Hu, Xiao Xiao, Yan Chen, Yuhong Liu, Wei Liu, Di Wang, Yong Yang, Jie Jiang, Qinglin Lu},
       year={2024},
 }
-```
+```