Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking
Wei Zhang*, Pengfei Li*, Junli Wang, Bingchuan Sun, Qihao Jin, Guangjun Bao, Shibo Rui, Yang Yu, Wenchao Ding, Peng Li📧, Yilun Chen📧
* indicates equal contribution, and 📧 indicates corresponding authors.
AIR, Tsinghua University | Lenovo Research | HIT | CASIA | Fudan University
Automatic Emergency Braking (AEB) systems are a crucial component in ensuring the safety of passengers in autonomous vehicles. Conventional AEB systems primarily rely on closed-set perception modules to recognize traffic conditions and assess collision risks. To enhance the adaptability of AEB systems in open scenarios, we propose Dual-AEB, a system combines an advanced multimodal large language model (MLLM) for comprehensive scene understanding and a conventional rule-based rapid AEB to ensure quick response times. To the best of our knowledge, Dual-AEB is the first method to incorporate MLLMs within AEB systems. Through extensive experimentation, we have validated the effectiveness of our method.
[Coming Soon]: We will release full code after paper acceptance! Stay tuned for updates on this page.
If you find this work useful in your research, please consider cite:
@misc{zhang2024dualaebsynergizingrulebasedmultimodal,
title={Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking},
author={Wei Zhang and Pengfei Li and Junli Wang and Bingchuan Sun and Qihao Jin and Guangjun Bao and Shibo Rui and Yang Yu and Wenchao Ding and Peng Li and Yilun Chen},
year={2024},
eprint={2410.08616},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2410.08616},
}
We thank all the authors who made their codes and datasets public, which tremendously accelerates our project progress. If you find these works helpful, please consider citing them as well.