Update README.md

README.md

@@ -2,35 +2,50 @@
 license: mit
 language:
 - en
+pipeline_tag: robotics
+library_name: diffusers
+tags:
+- robotics
+- multimodal
+- pretraining
+- vla
+- diffusion
 ---
 # RDT-1B
 
-RDT-1B is a 1B-parameter imitation learning Diffusion Transformer pre-trained on 1M+ multi-robot episodes. Given a language instruction and 3-view RGB image observations, RDT can predict the next 
-64 robot actions. RDT is inherently compatible with almost all kinds of modern mobile manipulators, from single-arm to dual-arm, joint to EEF, pos. to vel., and even with a mobile chassis.
+RDT-1B is a 1B-parameter imitation learning Diffusion Transformer pre-trained on 1M+ multi-robot episodes. Given language instruction and RGB images of up to three views, RDT can predict the next 
+64 robot actions. RDT is compatible with almost all modern mobile manipulators, from single-arm to dual-arm, joint to EEF, pos. to vel., and even with a mobile chassis.
 
-All the [code]() and pretrained model weights are licensed under MIT license.
+All the [code](https://github.com/GeneralEmbodiedSystem/RoboticsDiffusionTransformer/tree/main?tab=readme-ov-file) and pre-trained model weights are licensed under the MIT license.
 
 Please refer to our [project page](https://rdt-robotics.github.io/rdt-robotics/) and [paper]() for more information.
 
 ## Model Details
 
-- **Developed by** RDT team from Tsinghua University
+- **Developed by:** The RDT team consisting of researchers from the [TSAIL group](https://ml.cs.tsinghua.edu.cn/) at Tsinghua University
+- **Task Type:** Vision-Language-Action (language, image => robot actions)
+- **Modle Type:** Diffusion Policy with Transformers
 - **License:** MIT
 - **Language(s) (NLP):** en
-- **Model Architecture:** Diffusion Transformer
-- **Pretrain dataset:** Curated pretrain dataset collected from 46 datasets. Please see [here]() for detail
+- **Multi-Modal Encoders:**
+  - **Vision Backbone:** [siglip-so400m-patch14-384](https://huggingface.co/google/siglip-so400m-patch14-384)
+  - **Language Model:** [t5-v1_1-xxl](https://huggingface.co/google/t5-v1_1-xxl)
+- **Pre-Training Datasets:** 46 datasets consisting of [RT-1 Dataset](https://robotics-transformer1.github.io/), [RH20T](https://rh20t.github.io/), [DROID](https://droid-dataset.github.io/), [BridgeData V2](https://rail-berkeley.github.io/bridgedata/), [RoboSet](https://robopen.github.io/roboset/), and a subset of [Open X-Embodiment](https://robotics-transformer-x.github.io/). See [todo]() for a detailed list.
 - **Repository:** [repo_url]
 - **Paper :** [paper_url]
 - **Project Page:** https://rdt-robotics.github.io/rdt-robotics/
 
 ## Uses
 
-RDT takes language instruction, image observations and proprioception as input, and predicts the next 64 robot actions in the form of unified action space vector.
-The unified action space vector includes all the main physical quantities of robots (e.g. the end-effector and joint, position and velocity, base movement, etc.) and can be applied to a wide range of robotic embodiments.
+RDT takes language instruction, RGB image (of up to three views), control frequency (if any), and proprioception as input and predicts the next 64 robot actions in the form of the unified action space vector.
+The unified action space vector includes all the main physical quantities of the robot manipulator (e.g., the end-effector and joint, position and velocity, and base movement). 
+To deploy on your robot platform, you need to pick the relevant quantities from the unified vector. See our repository for more information.
 
-The pre-trained RDT model can be fine-tuned for specific robotic embodiment and deployed on real-world robots. 
-Here's an example of how to use the RDT-1B model for inference on a Mobile-ALOHA robot:
+**Out-of-Scope**: Due to the embodiment gap, RDT cannot yet generalize to new robot platforms (not seen in the pre-training datasets). 
+In this case, we recommend collecting a small dataset of the target robot and then using it to fine-tune RDT.
+See our repository for a tutorial.
 
+Here's an example of how to use the RDT-1B model for inference on a robot:
 ```python
 # Clone the repository and install dependencies
 from scripts.agilex_model import create_model
@@ -43,7 +58,7 @@ config = {
     'camera_names': CAMERA_NAMES,
 }
 pretrained_vision_encoder_name_or_path = "google/siglip-so400m-patch14-384" 
-# Create the model with specified configuration
+# Create the model with the specified configuration
 model = create_model(
     args=config,
     dtype=torch.bfloat16, 
@@ -64,8 +79,8 @@ actions = policy.step(
 )
 ```
 
-RDT-1B supports finetuning on custom dataset, deploying and inferencing on real-robots, as well as pretraining the model.
-Please refer to [our repository](https://github.com/GeneralEmbodiedSystem/RoboticsDiffusionTransformer/blob/main/docs/pretrain.md) for all the above guides.
+<!-- RDT-1B supports finetuning on custom datasets, deploying and inferencing on real robots, as well as retraining the model.
+Please refer to [our repository](https://github.com/GeneralEmbodiedSystem/RoboticsDiffusionTransformer/blob/main/docs/pretrain.md) for all the above guides. -->
 
 
 ## Citation