原始笔记链接:https://mp.weixin.qq.com/s?__biz=Mzg4MjgxMjgyMg==&mid=2247486680&idx=1&sn=edf41d4f95395d7294bc958ea68d3a68&chksm=cf51be21f826373790bc6d79bcea6eb2cb3d09bb1860bba0af0fd5e60c448ca006976503e460#rd
↑ \uparrow ↑点击上述链接即可阅读全文
IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar
毫米波雷达成像论文阅读笔记: IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar
Abstract
-
背景
- mmWave radars suffer from low angular resolution due to small apertures and conventional signal processing
- 稀疏阵列雷达 can increase aperture size while minimizing power consumption and readout bandwidth
-
方法 :提出 Compressive Implicit Radar (CoIR)
-
目标: high accuracy sparse radar imaging using a single radar chip
-
Leverages : CNN decoder and compressed sensing
-
贡献:
✅ 设计稀疏线阵: with 5.5x fewer antennas than conventional MIMO arrays
✅ 提出ComDecoder :a fully convolutional implicit neural network architecture
✅ 证明了CoIR的有效性 :in both simulation and real-world experiments,且 不需要 auxiliary sensors
-
-
实验结果
- improved performance over standard mmWave radars and other untrained methods on simulated and real data
- System does not require training data or auxiliary sensors
1 INTRODUCTION
基于光学的Depth imaging及其缺点
- Depth imaging
- crucial for applications like SLAM, autonomous driving, security monitoring
- Typical sensors: cameras, LiDAR
- Cameras: high-resolution near-field depth imaging
- LiDAR: directly outputs dense point cloud with high range/angular resolution
- Limitation : degraded performance in visually degraded environments like fog, smoke
毫米波雷达成像的优点和挑战
- 优点
- penetrate through fog/smoke without performance degradation
- 挑战
- low angular resolution δ ≈ λ / d \delta ≈ \lambda/d δ≈λ/d
- Increasing d d d increases cost, power consumption and readout bandwidth
已有提高角分辨率的工作和缺陷
- 已有思路
- Large physical arrays
- MIMO arrays
- SAR
- Sensor fusion
- Optimization with handcrafted priors
- Deep learning
- 不足
- Slow acquisition
- Increased hardware complexity
- Require large datasets
- Limited generalizability
The proposed CoIR:
-
Key observation:
- INR provides inductive bias towards natural solutions for imaging inverse problems
-
方法
- Leverages implicit neural representations (INRs) + compressed sensing
-
贡献
- Designed sparse linear array with 5.5x fewer antennas
- Proposed convolutional decoder architecture ComDecoder
- Demonstrated improved performance over standard mmWave radars and competitive untrained methods
2 RELATED WORK
2.1 mmWave Imaging Systems
-
提高角度分辨率的方法及其缺点
- Large physical arrays: expensive, large data volumes
- MIMO arrays: requires many radar chips to synthesize large aperture
- SAR techniques:slow imaging, bulky scanners
- Sensor fusion: fails if one modality fails
- Deep learning: requires large labeled datasets, limited generalizability
-
proposed CoIR 的不同:
-
仅使用 single chip sparse MIMO array
-
使用 未经训练 的 神经网络
✅ 无需训练数据
-
2.2 Sparse Radar Imaging
-
稀疏雷达成像技术:
- 1 Sparse aperture array designs
- 2 Sparse reconstruction methods
-
1 Sparse aperture array designs
-
使用欠奈奎斯特采样 减少 天线数
-
优化方法:
✅ Convex relaxations
✅ Prior knowledge of number of reflectors
-
-
2 Sparse reconstruction methods
-
Super-resolution algorithms
✅ MUSIC, ESPRIT
✅ Require incoherent signals, known number of targets
-
Compressed sensing (CS) optimization:
✅ 使用稀疏先验,如 spatial sparsity, TV norm
✅ Challenging to design priors, scene dependent
-
-
proposed CoIR 的不同:
-
Sparse array design
🚩 inspired by prior work but modified due to hardware constraints
-
Uses untrained neural network
🚩 as complex prior instead of handcrafted prior
✅ Neural network prior shows affinity for natural features and noise robustness
-
2.3 Implicit Neural Representations
-
两类INR architectures:
- 1 Convolutional methods
- 2 Coordinate-based MLP methods
-
1 Convolutional methods ,适合:
- Compressed sensing
- Image super-resolution
- Image denoising
- Accelerated MRI
-
2 Coordinate-based MLP methods ,适合:
- Novel view synthesis
- Dynamic illumination
- PDE solutions
- Image deconvolution
-
CoIR中的ComDecoder:
-
属于 Convolutional methods
-
tailored for sparse radar imaging
-
Key properties:
🚩 Convolutional operations capture local spatial information
🚩 Upsampling induces notion of resolution per layer
🚩 Residual blocks smooth optimization and propagate information between layers
🚩 Together these inductive biases improve performance on sparse radar imaging
-
Differences from prior works:
✅ CoIR uses untrained INR as complex prior for sparse radar imaging
✅ Prior works use INR for natural images or other imaging modalities
-
3 RADAR IMAGING BACKGROUND
-
发射信号模型
- y t x ( t ) = e j 2 π ( f 0 t + 1 2 B τ t 2 ) , 0 ≤ t ≤ T y_{tx}(t) = e^{j2π(f_0t + \frac{1}{2}Bτt^2)}, 0 \leq t \leq T ytx(t)=ej2π(f0t+21Bτt2),0≤t≤T
- f 0 f_0 f0: carrier frequency
- B B B: chirp bandwidth
- T T T: pulse duration
-
场景模型 (离散反射体分布)
- x ‾ [ n r , n θ ] ∈ C K × L \overline{x}[n_r, n_\theta] \in \mathbb{C}^{K\times L} x[nr,nθ]∈CK×L
- n r n_r nr: range bin index
- n θ n_\theta nθ: angle bin index
-
回波信号模型
- z [ n , m ] = ∑ n r = 0 K − 1 ∑ n θ = 0 L − 1 x ‾ [ n r , n θ ] e j 2 π ψ θ ( n θ ) m e j 2 π ψ r ( n r ) n + w [ n , m ] z[n,m] = \sum_{n_r=0}^{K-1} \sum_{n_\theta=0}^{L-1} \overline{x}[n_r, n_\theta] e^{j2π\psi_\theta(n_\theta)m} e^{j2π\psi_r(n_r)n} + w[n,m] z[n,m]=∑nr=0K−1∑nθ=0L−1x[nr,nθ]ej2πψθ(nθ)mej2πψr(nr)n+w[n,m]
- ψ θ ( n θ ) = f 0 d c sin ( b θ [ n θ ] ) \psi_\theta(n_\theta) = \frac{f_0 d}{c}\sin(b_\theta[n_\theta]) ψθ(nθ)=cf0dsin(bθ[nθ]): spatial frequency
- ψ r ( n r ) = B N 2 b r [ n r ] c \psi_r(n_r) = \frac{B}{N}\frac{2b_r[n_r]}{c} ψr(nr)=NBc2br[nr]: normalized temporal frequency
- w [ n , m ] w[n,m] w[n,m]: noise
-
Compact matrix form
-
z = F ( x ‾ ) + w z = F(\overline{x}) + w z=F(x)+w
-
F F F: 2D FFT
-
Goal: recover x ‾ \overline{x} x from under-sampled measurements z z z
-
4 PROPOSED METHOD
-
目标 :
- Recover scene reflectivity x ‾ \overline{x} x from under-sampled measurements z z z
-
Measurements :
-
z = M ⊙ F ( x ‾ ) + w z = M\odot F(\overline{x}) + w z=M⊙F(x)+w
-
M M M: binary mask implementing under-sampling
-
w w w: noise
-
-
困难 :
- under-sampling causes grating lobes in sparse array PSF leading to aliasing in image
-
解决方法
-
Optimize weights of untrained deep CNN G ( C ; p ) G(C;p) G(C;p) to solve inverse problem
✅ G G G: untrained CNN,
✅ C C C: fixed noise input,
✅ p p p: CNN parameters
-
Optimization objective:
🚩 p ^ = arg min p ∣ ∣ z − M ⊙ F ( G ( C ; p ) ) ∣ ∣ 2 + λ L ∣ ∣ G ( C ; p ) ∣ ∣ 1 \hat{p} = \argmin_p ||z - M\odot F(G(C;p))||_2 + \lambda_L||G(C;p)||_1 p^=argminp∣∣z−M⊙F(G(C;p))∣∣2+λL∣∣G(C;p)∣∣1
🚩 λ L \lambda_L λL: sparsity regularization strength
-
Key observation:
🚩 INR provides inductive bias towards natural solutions for imaging inverse problems
-
-
优点 :
-
CNN architecture has high impedance to noise
-
Learned solution balances fitting salient features and suppressing artifacts
-
4.1 Sparse Aperture Design
- 目标
- Design a sparse MIMO virtual array that improves imaging accuracy when used with ComDecoder
- 设计准测 (metrics)
- PSF main lobe half-power beamwidth (HPBW)
- Peak sidelobe level (SLL)
- Presence of grating lobes
- 硬件约束
- Max aperture 86λ/2
- Limited to 4 TX and 4 RX due to commercial single radar chip
- 设计方法
- Select 4-element minimum redundancy array for RX to avoid grating lobes
- Grid search over TX positions to minimize SLL
- 比较对象(baselines)
- Full: Ideal full Nyquist sampled array
- Sub-apt: Largest Nyquist sampled MIMO array given constraints
- Sub-samp: Largest sub-Nyquist array given constraints
- 设计结果
- RX: [0, 1, 4, 6] λ/2
- TX: [0, 46, 59, 79] λ/2
- Gives 5.5x fewer antennas than conventional MIMO array
- 优点 :
- Avoids grating lobes
- Minimizes HPBW
- Minimizes SLL
- Satisfies hardware constraints
4.2 Neural Network Architecture
提出 ComDecoder:convolutional decoder architecture
-
ComDecoder :
- Maps latent variables C to image G(C;p)
- 优化:Parameters p optimized to reconstruct image
-
网络结构 :
- Series of upsampling and residual convolution blocks
- Use SiLU activation instead of ReLU
- No upsampling in last layer, uses 1x1 conv instead
-
超参数 :
- 6 layers (including last layer)
- 128 channels per layer
- Fixed input C sampled from uniform distribution
-
优化过程 :
- Update network weights p using backpropagation and Adam
- Takes <50 s per 256x256 image using 2000 iterations
-
优点 :
- SiLU increased expressivity over ReLU
- Upsampling reinforces multi-resolution nature
- Residual blocks enable information flow between layers
- Inductive biases improve performance on sparse radar imaging
5 COMPETING UNTRAINED METHODS
7个baselines: Compared CoIR against several untrained methods
-
1 Delay-and-Sum (DAS)
- Standard beamforming method
-
2 Sparse DAS
- DAS with under-sampled measurements
-
3 Gradient Descent with L1 Regularization (GD+L1 Reg)
- Directly optimizes reflectivity distribution with sparsity prior
-
4 Implicit Neural Representations:
-
4.1 INR-ReLU
✅ MLP-based, uses Fourier feature encoding
-
4.2 SIREN
✅ MLP-based, uses sinusoidal activation functions
-
-
5 Deep Image Prior (DIP)
- U-Net style convolutional autoencoder
-
6 DeepDecoder
- Decoder-only convolutional network
-
7 ConvDecoder
- Variant of DeepDecoder with some modifications
6 SIMULATION RESULTS
在仿真数据上评估所提出的CoIR
-
仿真数据生成:
- Synthesizes radar data cube from 2D reflectivity images
- Uses LiDAR point clouds to generate realistic reflectivity distributions
-
评估标准:
- PSNR, SSIM, MAE between reconstruction and ground truth image
-
实验:
-
1 Vary SNR from 35dB to 11dB
✅ ComDecoder gave superior PSNR over all methods at all SNRs
✅ ComDecoder and DIP gave comparable SSIM
✅ ComDecoder and DIP gave lowest MAE
-
2 Visualize reconstructions at 19dB SNR
✅ ComDecoder gave most accurate recovery of extended reflectors
✅ Other CNN methods also improved over Sparse DAS
✅ SIREN struggled to distinguish clutter and true reflectors
-
3 Additional analyses:
✅ Compared different CNN decoder architectures
✅ Evaluated computational complexity (in supplementary)
-
-
总结:
- ComDecoder 在 simulated data 上 SOTA
7 EXPERIMENTAL RESULTS
在真实采集的Coloradar dataset上评估所有方法
-
Radar system:
-
77 GHz FMCW with 1.282 GHz bandwidth
-
86λ/2 uniform linear array
-
-
Metrics :
- 与 full array DAS reconstruction 进行对比
-
Experiments :
-
1 不同场景下的重建效果
✅ ComDecoder accurately recovered dominant features
✅ DIP also performed well but more artifacts than ComDecoder
✅ SIREN struggled in indoor scene due to noise
-
2 Evaluate 鲁棒性 across multiple outdoor scenes
✅ ComDecoder gave high fidelity reconstructions closest to DAS
✅ SIREN fit strong reflectors but also artifacts
✅ GD+L1 located dominant reflectors but artifacts remained
✅ DIP performed well but more artifacts than ComDecoder
-
- 总结:
- ComDecoder 在 real data 上 SOTA
- 显著好于其他untrained methods
8 DISCUSSION & LIMITATIONS
Limitations
- 1 Assume static scene in forward model
- Cannot handle moving objects
- 2 Single bounce scattering model may not match real-world
- 3 Slow optimization time (tens of seconds)
- Explore different initialization strategies
Future work
- 1 Demonstrated 2D range-angle slices due to linear array
- 2D array needed for full 3D, but increases complexity
- 2 CoIR could benefit other array-based imaging modalities:
- SAR, ultrasound, etc.
9 CONCLUSION
Proposed CoIR
-
1 Designed sparse linear array with 5.5x fewer antennas
-
2 Proposed convolutional decoder architecture ComDecoder
-
3 Demonstrated superior performance on simulated and real mmWave radar data