@inproceedings{
jiang2024scenediffuser,
title={SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout},
author={Chiyu Max Jiang and Yijing Bai and Andre Cornman and Christopher Davis and Xiukun Huang and Hong Jeon and Sakshum Kulshrestha and John Wheatley Lambert and Shuangyu Li and Xuanyu Zhou and Carlos Fuertes and Chang Yuan and Mingxing Tan and Yin Zhou and Dragomir Anguelov},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=a4qT29Levh}
}
Simulation with realistic and interactive agents represents a key task for autonomous vehicle (AV) software development in order to test AV performance in prescribed, often long-tail scenarios. In this work, we propose SceneDiffuser, a scene-level diffusion prior for traffic simulation. We present a singular framework that unifies two key stages of simulation: scene initialization and scene rollout. Scene initialization refers to generating the initial layout for the traffic in a scene, and scene rollout refers to closed-loop simulation for the behaviors of the agents. While diffusion has been demonstrated to be effective in learning realistic, multimodal agent distributions, two open challenges remain: controllability and closed-loop inference efficiency and realism. To this end, to address controllability challenges, we propose generalized hard constraints, a generalized inference-time constraint mechanism that is simple yet effective. To improve closed-loop inference quality and efficiency, we propose amortized diffusion, a novel diffusion denoising paradigm that amortizes the physical cost of denoising over future simulation rollout steps, reducing the cost of per physical rollout step to a single denoising function evaluation, while dramatically reducing closed-loop errors. We demonstrate the effectiveness of our approach on the Waymo Open Dataset, where we are able to generate distributionally realistic scenes, while obtaining competitive performance in the Sim Agents Challenge, surpassing the state-of-the-art in many realism attributes.
SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout
Chiyu Max Jiang Yijing Bai Andre Cornman Christopher Davis Xiukun Huang Hong Jeon Sakshum Kulshrestha John Wheatley Lambert Shuangyu Li Xuanyu Zhou Carlos Fuertes Chang Yuan Mingxing Tan Yin Zhou Dragomir Anguelov