2024 · Sim2Real
WARM-3D
Weakly-supervised Sim2Real for roadside 3D detection
- Sim2Real
- 3D Detection
- PyTorch
- ITSC 2024
Overview
Roadside infrastructure cameras are critical for cooperative driving, but high-quality monocular 3D labels at scale are prohibitively expensive. WARM-3D leans on a synthetic twin of the TUM Traffic intersection (the TUMTraf Synthetic Dataset) and an off-the-shelf 2D detector to supervise a 3D student model in the real domain.
The framework yields +12.40 % mAP3D over the baseline and reaches near-Oracle performance using only 2D ground truth as weak supervision. Built on top of MonoDETR; trained and evaluated on the real TUM Traffic deployment.
Highlights
- Synthetic dataset generation pipeline (TUMTraf Synthetic)
- Weak supervision from off-the-shelf 2D detectors
- Teacher–student domain adaptation loop
- +12.40 % mAP3D vs. baseline, near-Oracle accuracy
- Built on MonoDETR, evaluated on real roadside cameras
Install
git clone https://github.com/WARM-3D/WARM-3D && cd WARM-3D && pip install -r requirements.txtGallery
