DeeperRadar: End-to-End MIMO Radar Design and Multi-Modal Fusion for Autonomous Vehicle Perception

IROS 2026

Radar-centric sensor-stack-conditioned MIMO design for autonomous vehicle perception

Eli Goldenshluger¹ Barak Pinkovich¹ Chaim Baskin²

¹ Technion-Israel Institute of Technology

² INSIGHT Lab, Ben-Gurion University of the Negev

Code Paper Citation

DeeperRadar training and inference overview — High-level overview of our radar-centric pipeline: during training, LiDAR, radar, and camera supervise a learnable MIMO design module that learns a subsampling mask on raw radar data; at deployment, the learned mask is applied to the radar, which is fused with LiDAR and camera features for 3D detection.

Abstract

DeeperRadar is a radar-centric, sensor-stack-conditioned framework that co-designs radar sensing and multi-modal 3D detection for autonomous mobility by learning a sparse acquisition pattern end-to-end with the fusion model. A learnable MIMO design module is trained end-to-end within a fusion network that operates directly on raw radar ADC data together with camera images and LiDAR point clouds. During training, the design module is supervised by the other sensors, enabling the system to learn both which receiver antennas to activate and the effective number of them. At deployment, the design module is removed and replaced by the learned sparse subsampling mask, leaving the downstream model architecture unchanged. Evaluated on the RADIal dataset, DeeperRadar discovers sparse, task-aware radar configurations that match or exceed full-array baselines while using fewer receivers, potentially reducing radar cost and integration complexity. These results show that learned optimal MIMO radar design depends on the fusion stack and the downstream perception task.

Method

Experimental Results

Performance Across Receiver Budgets

Radar-only AP across receiver budgets — Overall 3D detection accuracy (AP@0.5) versus radar receiver budget n_R for three configurations: Radar, Radar+Camera, and Radar+LiDAR. Each panel compares the mean performance of heuristic subsampling with the mean performance of learned antenna selection across random seeds. The marker denotes the learned optimal budget n^*_R predicted by the budget-regularized objective. The radar-only panel serves as a complementary baseline evaluated without the fusion module.

Radar and camera AP across receiver budgets — Overall 3D detection accuracy (AP@0.5) versus radar receiver budget n_R for three configurations: Radar, Radar+Camera, and Radar+LiDAR. Each panel compares the mean performance of heuristic subsampling with the mean performance of learned antenna selection across random seeds. The marker denotes the learned optimal budget n^*_R predicted by the budget-regularized objective. The radar-only panel serves as a complementary baseline evaluated without the fusion module.

Per Experiment Comparison of Learned and Fixed Layouts

Radar-only learned configuration scatter plot — Comparison of learned layouts ψ̂ versus fixed layouts ψ̃ across different sensor stacks. Each point represents one random seed per configuration. The diagonal indicates parity between fixed and learned antenna selections in terms of 3D AP.

Radar and camera learned configuration scatter plot — Comparison of learned layouts ψ̂ versus fixed layouts ψ̃ across different sensor stacks. Each point represents one random seed per configuration. The diagonal indicates parity between fixed and learned antenna selections in terms of 3D AP.

BibTeX

@misc{goldenshluger2026deeperradar,
  title         = {DeeperRadar: End-to-End MIMO Radar Design and Multi-Modal Fusion for Autonomous Vehicle Perception},
  author        = {Eli Goldenshluger and Barak Pinkovich and Chaim Baskin},
  year          = {2026},
  eprint        = {2607.17351},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2607.17351}
}