|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
# LMDrive Model Card |
|
|
|
## Model details |
|
|
|
**Model type:** |
|
LMDrive is an end-to-end, closed-loop, language-based autonomous driving framework, which interacts with the dynamic environment via multi-modal multi-view sensor data and natural language instructions. |
|
|
|
**Model date:** |
|
LMDrive-1.0 (based on LLaVA-v1.5-7B) was trained in November 2023. The original LLaVA-v1.5 also needs to be downloaded. |
|
|
|
**Paper or resources for more information:** |
|
|
|
Github: https://github.com/opendilab/LMDrive/README.md |
|
|
|
Paper: https://arxiv.org/abs/2312.07488 |
|
|
|
**Related weights for the vision encoder** |
|
|
|
https://huggingface.co/deepcs233/LMDrive-vision-encoder-r50-v1.0 |
|
|
|
**Where to send questions or comments about the model:** |
|
|
|
https://github.com/opendilab/LMDrive/issues |
|
|
|
|
|
|
|
## Intended use |
|
**Primary intended uses:** |
|
|
|
The primary use of LMDrive is research on large multimodal models for autonomous driving. |
|
|
|
**Primary intended users:** |
|
|
|
The primary intended users of the model are researchers and hobbyists in computer vision, large multimodal model, autonomous driving, and artificial intelligence. |
|
|
|
## Training dataset |
|
|
|
- 64K instruction-sensor-control data clips collected in the CARLA simulator. [dataset_webpage](https://huggingface.co/datasets/deepcs233/LMDrive) |
|
- where each clip includes one navigation instruction, several notice instructions, a sequence of multi-modal multi-view sensor data, and control signals. The duration of the clip spans from 2 to 20 seconds |
|
|
|
## Evaluation benchmark |
|
LangAuto, LangAuto-short, LangAuto-tiny, LangAuto-notice |
