毫米波雷达成像论文阅读笔记: IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

原始笔记链接:https://mp.weixin.qq.com/s?__biz=Mzg4MjgxMjgyMg==&mid=2247486680&idx=1&sn=edf41d4f95395d7294bc958ea68d3a68&chksm=cf51be21f826373790bc6d79bcea6eb2cb3d09bb1860bba0af0fd5e60c448ca006976503e460#rd
↑ \uparrow 点击上述链接即可阅读全文

IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

毫米波雷达成像论文阅读笔记: IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

在这里插入图片描述

Abstract

  • 背景

    • mmWave radars suffer from low angular resolution due to small apertures and conventional signal processing
    • 稀疏阵列雷达 can increase aperture size while minimizing power consumption and readout bandwidth
  • 方法 :提出 Compressive Implicit Radar (CoIR)

    • 目标: high accuracy sparse radar imaging using a single radar chip

    • Leverages : CNN decoder and compressed sensing

    • 贡献:

      设计稀疏线阵: with 5.5x fewer antennas than conventional MIMO arrays

      提出ComDecoder :a fully convolutional implicit neural network architecture

      证明了CoIR的有效性 :in both simulation and real-world experiments,且 不需要 auxiliary sensors

  • 实验结果

    • improved performance over standard mmWave radars and other untrained methods on simulated and real data
    • System does not require training data or auxiliary sensors
      在这里插入图片描述

1 INTRODUCTION

基于光学的Depth imaging及其缺点

  • Depth imaging
    • crucial for applications like SLAM, autonomous driving, security monitoring
  • Typical sensors: cameras, LiDAR
    • Cameras: high-resolution near-field depth imaging
    • LiDAR: directly outputs dense point cloud with high range/angular resolution
  • Limitation : degraded performance in visually degraded environments like fog, smoke

毫米波雷达成像的优点和挑战

  • 优点
    • penetrate through fog/smoke without performance degradation
  • 挑战
    • low angular resolution δ ≈ λ / d \delta ≈ \lambda/d δλ/d
    • Increasing d d d increases cost, power consumption and readout bandwidth

已有提高角分辨率的工作和缺陷

  • 已有思路
    • Large physical arrays
    • MIMO arrays
    • SAR
    • Sensor fusion
    • Optimization with handcrafted priors
    • Deep learning
  • 不足
    • Slow acquisition
    • Increased hardware complexity
    • Require large datasets
    • Limited generalizability

The proposed CoIR:

  • Key observation:

    • INR provides inductive bias towards natural solutions for imaging inverse problems
  • 方法

    • Leverages implicit neural representations (INRs) + compressed sensing
  • 贡献

    • Designed sparse linear array with 5.5x fewer antennas
    • Proposed convolutional decoder architecture ComDecoder
    • Demonstrated improved performance over standard mmWave radars and competitive untrained methods

2 RELATED WORK

2.1 mmWave Imaging Systems

  • 提高角度分辨率的方法及其缺点

    • Large physical arrays: expensive, large data volumes
    • MIMO arrays: requires many radar chips to synthesize large aperture
    • SAR techniques:slow imaging, bulky scanners
    • Sensor fusion: fails if one modality fails
    • Deep learning: requires large labeled datasets, limited generalizability
  • proposed CoIR 的不同:

    • 仅使用 single chip sparse MIMO array

    • 使用 未经训练 的 神经网络

      ✅ 无需训练数据

2.2 Sparse Radar Imaging

  • 稀疏雷达成像技术:

    • 1 Sparse aperture array designs
    • 2 Sparse reconstruction methods
  • 1 Sparse aperture array designs

    • 使用欠奈奎斯特采样 减少 天线数

    • 优化方法:

      ✅ Convex relaxations

      ✅ Prior knowledge of number of reflectors

  • 2 Sparse reconstruction methods

    • Super-resolution algorithms

      ✅ MUSIC, ESPRIT

      ✅ Require incoherent signals, known number of targets

    • Compressed sensing (CS) optimization:

      ✅ 使用稀疏先验,如 spatial sparsity, TV norm

      ✅ Challenging to design priors, scene dependent

  • proposed CoIR 的不同:

    • Sparse array design

      🚩 inspired by prior work but modified due to hardware constraints

    • Uses untrained neural network

      🚩 as complex prior instead of handcrafted prior

      ✅ Neural network prior shows affinity for natural features and noise robustness

2.3 Implicit Neural Representations

  • 两类INR architectures:

    • 1 Convolutional methods
    • 2 Coordinate-based MLP methods
  • 1 Convolutional methods ,适合:

    • Compressed sensing
    • Image super-resolution
    • Image denoising
    • Accelerated MRI
  • 2 Coordinate-based MLP methods ,适合:

    • Novel view synthesis
    • Dynamic illumination
    • PDE solutions
    • Image deconvolution
  • CoIR中的ComDecoder:

    • 属于 Convolutional methods

    • tailored for sparse radar imaging

    • Key properties:

      🚩 Convolutional operations capture local spatial information

      🚩 Upsampling induces notion of resolution per layer

      🚩 Residual blocks smooth optimization and propagate information between layers

      🚩 Together these inductive biases improve performance on sparse radar imaging

    • Differences from prior works:

      ✅ CoIR uses untrained INR as complex prior for sparse radar imaging

      ✅ Prior works use INR for natural images or other imaging modalities

3 RADAR IMAGING BACKGROUND

  • 发射信号模型

    • y t x ( t ) = e j 2 π ( f 0 t + 1 2 B τ t 2 ) , 0 ≤ t ≤ T y_{tx}(t) = e^{j2π(f_0t + \frac{1}{2}Bτt^2)}, 0 \leq t \leq T ytx(t)=ej2π(f0t+21Bτt2),0tT
    • f 0 f_0 f0: carrier frequency
    • B B B: chirp bandwidth
    • T T T: pulse duration
  • 场景模型 (离散反射体分布)

    • x ‾ [ n r , n θ ] ∈ C K × L \overline{x}[n_r, n_\theta] \in \mathbb{C}^{K\times L} x[nr,nθ]CK×L
    • n r n_r nr: range bin index
    • n θ n_\theta nθ: angle bin index
  • 回波信号模型

    • z [ n , m ] = ∑ n r = 0 K − 1 ∑ n θ = 0 L − 1 x ‾ [ n r , n θ ] e j 2 π ψ θ ( n θ ) m e j 2 π ψ r ( n r ) n + w [ n , m ] z[n,m] = \sum_{n_r=0}^{K-1} \sum_{n_\theta=0}^{L-1} \overline{x}[n_r, n_\theta] e^{j2π\psi_\theta(n_\theta)m} e^{j2π\psi_r(n_r)n} + w[n,m] z[n,m]=nr=0K1nθ=0L1x[nr,nθ]ej2πψθ(nθ)mej2πψr(nr)n+w[n,m]
    • ψ θ ( n θ ) = f 0 d c sin ⁡ ( b θ [ n θ ] ) \psi_\theta(n_\theta) = \frac{f_0 d}{c}\sin(b_\theta[n_\theta]) ψθ(nθ)=cf0dsin(bθ[nθ]): spatial frequency
    • ψ r ( n r ) = B N 2 b r [ n r ] c \psi_r(n_r) = \frac{B}{N}\frac{2b_r[n_r]}{c} ψr(nr)=NBc2br[nr]: normalized temporal frequency
    • w [ n , m ] w[n,m] w[n,m]: noise
  • Compact matrix form

    • z = F ( x ‾ ) + w z = F(\overline{x}) + w z=F(x)+w

    • F F F: 2D FFT

    • Goal: recover x ‾ \overline{x} x from under-sampled measurements z z z

4 PROPOSED METHOD

  • 目标 :

    • Recover scene reflectivity x ‾ \overline{x} x from under-sampled measurements z z z
  • Measurements :

    • z = M ⊙ F ( x ‾ ) + w z = M\odot F(\overline{x}) + w z=MF(x)+w

    • M M M: binary mask implementing under-sampling

    • w w w: noise

  • 困难 :

    • under-sampling causes grating lobes in sparse array PSF leading to aliasing in image
  • 解决方法

    • Optimize weights of untrained deep CNN G ( C ; p ) G(C;p) G(C;p) to solve inverse problem

      G G G: untrained CNN,

      C C C: fixed noise input,

      p p p: CNN parameters

    • Optimization objective:

      🚩 p ^ = arg min ⁡ p ∣ ∣ z − M ⊙ F ( G ( C ; p ) ) ∣ ∣ 2 + λ L ∣ ∣ G ( C ; p ) ∣ ∣ 1 \hat{p} = \argmin_p ||z - M\odot F(G(C;p))||_2 + \lambda_L||G(C;p)||_1 p^=argminp∣∣zMF(G(C;p))2+λL∣∣G(C;p)1

      🚩 λ L \lambda_L λL: sparsity regularization strength

    • Key observation:

      🚩 INR provides inductive bias towards natural solutions for imaging inverse problems

  • 优点 :

    • CNN architecture has high impedance to noise

    • Learned solution balances fitting salient features and suppressing artifacts

4.1 Sparse Aperture Design

  • 目标
    • Design a sparse MIMO virtual array that improves imaging accuracy when used with ComDecoder
  • 设计准测 (metrics)
    • PSF main lobe half-power beamwidth (HPBW)
    • Peak sidelobe level (SLL)
    • Presence of grating lobes
  • 硬件约束
    • Max aperture 86λ/2
    • Limited to 4 TX and 4 RX due to commercial single radar chip
  • 设计方法
    • Select 4-element minimum redundancy array for RX to avoid grating lobes
    • Grid search over TX positions to minimize SLL
  • 比较对象(baselines)
    • Full: Ideal full Nyquist sampled array
    • Sub-apt: Largest Nyquist sampled MIMO array given constraints
    • Sub-samp: Largest sub-Nyquist array given constraints
  • 设计结果
    • RX: [0, 1, 4, 6] λ/2
    • TX: [0, 46, 59, 79] λ/2
    • Gives 5.5x fewer antennas than conventional MIMO array

在这里插入图片描述

  • 优点 :
    • Avoids grating lobes
    • Minimizes HPBW
    • Minimizes SLL
    • Satisfies hardware constraints

4.2 Neural Network Architecture

提出 ComDecoder:convolutional decoder architecture

  • ComDecoder :

    • Maps latent variables C to image G(C;p)
    • 优化:Parameters p optimized to reconstruct image
  • 网络结构 :

    • Series of upsampling and residual convolution blocks
    • Use SiLU activation instead of ReLU
    • No upsampling in last layer, uses 1x1 conv instead
  • 超参数 :

    • 6 layers (including last layer)
    • 128 channels per layer
    • Fixed input C sampled from uniform distribution
  • 优化过程 :

    • Update network weights p using backpropagation and Adam
    • Takes <50 s per 256x256 image using 2000 iterations
  • 优点 :

    • SiLU increased expressivity over ReLU
    • Upsampling reinforces multi-resolution nature
    • Residual blocks enable information flow between layers
    • Inductive biases improve performance on sparse radar imaging

5 COMPETING UNTRAINED METHODS

7个baselines: Compared CoIR against several untrained methods

  • 1 Delay-and-Sum (DAS)

    • Standard beamforming method
  • 2 Sparse DAS

    • DAS with under-sampled measurements
  • 3 Gradient Descent with L1 Regularization (GD+L1 Reg)

    • Directly optimizes reflectivity distribution with sparsity prior
  • 4 Implicit Neural Representations:

    • 4.1 INR-ReLU

      ✅ MLP-based, uses Fourier feature encoding

    • 4.2 SIREN

      ✅ MLP-based, uses sinusoidal activation functions

  • 5 Deep Image Prior (DIP)

    • U-Net style convolutional autoencoder
  • 6 DeepDecoder

    • Decoder-only convolutional network
  • 7 ConvDecoder

    • Variant of DeepDecoder with some modifications

6 SIMULATION RESULTS

在仿真数据上评估所提出的CoIR

  • 仿真数据生成:

    • Synthesizes radar data cube from 2D reflectivity images
    • Uses LiDAR point clouds to generate realistic reflectivity distributions
  • 评估标准:

    • PSNR, SSIM, MAE between reconstruction and ground truth image
  • 实验:

    • 1 Vary SNR from 35dB to 11dB

      ✅ ComDecoder gave superior PSNR over all methods at all SNRs

      ✅ ComDecoder and DIP gave comparable SSIM

      ✅ ComDecoder and DIP gave lowest MAE
      在这里插入图片描述

    • 2 Visualize reconstructions at 19dB SNR

      ✅ ComDecoder gave most accurate recovery of extended reflectors

      ✅ Other CNN methods also improved over Sparse DAS

      ✅ SIREN struggled to distinguish clutter and true reflectors
      在这里插入图片描述

    • 3 Additional analyses:

      ✅ Compared different CNN decoder architectures

      ✅ Evaluated computational complexity (in supplementary)

  • 总结:

    • ComDecoder 在 simulated data 上 SOTA

7 EXPERIMENTAL RESULTS

在真实采集的Coloradar dataset上评估所有方法

  • Radar system:

    • 77 GHz FMCW with 1.282 GHz bandwidth

    • 86λ/2 uniform linear array

  • Metrics :

    • 与 full array DAS reconstruction 进行对比
  • Experiments :

    • 1 不同场景下的重建效果

      ✅ ComDecoder accurately recovered dominant features

      ✅ DIP also performed well but more artifacts than ComDecoder

      ✅ SIREN struggled in indoor scene due to noise
      在这里插入图片描述

    • 2 Evaluate 鲁棒性 across multiple outdoor scenes

      ✅ ComDecoder gave high fidelity reconstructions closest to DAS

      ✅ SIREN fit strong reflectors but also artifacts

      ✅ GD+L1 located dominant reflectors but artifacts remained

      ✅ DIP performed well but more artifacts than ComDecoder

在这里插入图片描述

  • 总结:
    • ComDecoder 在 real data 上 SOTA
    • 显著好于其他untrained methods

8 DISCUSSION & LIMITATIONS

Limitations

  • 1 Assume static scene in forward model
    • Cannot handle moving objects
  • 2 Single bounce scattering model may not match real-world
  • 3 Slow optimization time (tens of seconds)
    • Explore different initialization strategies

Future work

  • 1 Demonstrated 2D range-angle slices due to linear array
    • 2D array needed for full 3D, but increases complexity
  • 2 CoIR could benefit other array-based imaging modalities:
    • SAR, ultrasound, etc.

9 CONCLUSION

Proposed CoIR

  • 1 Designed sparse linear array with 5.5x fewer antennas

  • 2 Proposed convolutional decoder architecture ComDecoder

  • 3 Demonstrated superior performance on simulated and real mmWave radar data

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/45964.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

前端开发,怎么解决浏览器兼容性问题? - 易智编译EaseEditing

解决浏览器兼容性问题是前端开发中常见的挑战之一。不同的浏览器可能对网页元素的渲染和功能支持有所不同&#xff0c;因此需要采取一些策略来确保您的网页在不同浏览器上都能正常运行和呈现。以下是一些解决浏览器兼容性问题的方法和策略&#xff1a; 使用CSS Reset&#xff…

Nuxt3_1_路由+页面+组件+资源+样式 使用及实例

1、 简介 1.1 开发必备 node版本 v16.10.0 我使用的是16.14.0编辑器推荐使用Volar Extension 的VS code插件Terminal 运行nuxt指令 1.2 环境搭建 安装项目&#xff1a; npx nuxilatest init [first_nuxt3]进入项目目录&#xff1a; cd [first_nuxt3]安装依赖&#xff1a;n…

Blazor前后端框架Known-V1.2.13

V1.2.13 Known是基于C#和Blazor开发的前后端分离快速开发框架&#xff0c;开箱即用&#xff0c;跨平台&#xff0c;一处代码&#xff0c;多处运行。 Gitee&#xff1a; https://gitee.com/known/KnownGithub&#xff1a;https://github.com/known/Known 概述 基于C#和Blazo…

探索Perfetto:开源性能追踪工具的未来之光

探索Perfetto&#xff1a;开源性能追踪工具的未来之光 1. 引言 A. 介绍Perfetto的背景和作用 随着移动应用、桌面软件和嵌入式系统的不断发展&#xff0c;软件性能优化变得愈发重要。在这个背景下&#xff0c;Perfetto作为一款开源性能追踪工具&#xff0c;日益引起了开发者…

BC108 矩阵交换

描述 KiKi有一个矩阵&#xff0c;他想知道经过k次行变换或列变换后得到的矩阵。请编程帮他解答。 输入描述 第一行包含两个整数n和m&#xff0c;表示一个矩阵包含n行m列&#xff0c;用空格分隔。 (1≤n≤10,1≤m≤10) 从2到n1行&#xff0c;每行输入m个整数&#xff08;范围-…

java云智慧工地管理平台系统源码

智慧工地将“互联网”的理念和技术引入建筑工地&#xff0c;从施工现场源头抓起&#xff0c;最大程度地收集人员、安全、环境、材料等关键业务数据&#xff0c;依托物联网、互联网&#xff0c;建立云端大数据管理平台&#xff0c;形成“端云大数据”的业务体系和新的管理模式&a…

【CSS动画02--卡片旋转3D】

CSS动画02--卡片旋转3D 介绍代码HTMLCSS css动画02--旋转卡片3D 介绍 当鼠标移动到中间的卡片上会有随着中间的Y轴进行360的旋转&#xff0c;以下是几张图片的介绍&#xff0c;上面是鄙人自己录得一个供大家参考的小视频&#x1f92d; 代码 HTML <!DOCTYPE html>…

BERT、ERNIE、Grover、XLNet、GPT、MASS、UniLM、ELECTRA、RoBERTa、T5、C4

BERT、ERNIE、Grover、XLNet、GPT、MASS、UniLM、ELECTRA、RoBERTa、T5、C4 ELMOBERTERNIE![在这里插入图片描述](https://img-blog.csdnimg.cn/274e31d0f8274c748d05abe2ec65fc73.png)GroverXLNetGPTMASSUniLMELECTRARoBERTaT5C4ELMO BERT

chatGPT-对话爱因斯坦

引言 阿尔伯特爱因斯坦&#xff08; 1879年 3 月 14 日 – 1955 年 4 月 18 日&#xff09;是一位出生于德国的理论物理学家&#xff0c;被广泛认为成为有史以来最伟大、最有影响力的科学家之一。他以发展相对论而闻名&#xff0c;他还对量子力学做出了重要贡献&#xff0c;因…

rfc7234之http缓存

缓存概念 缓存处理请求步骤 缓存如果查询到某个请求已经有缓存&#xff0c;那么需要进一步检查该资源的新鲜度&#xff0c;根据新鲜度和请求中的字段综合评估是否要去服务端拉取新鲜的资源。 注意&#xff1a; 创建响应时候要注意版本匹配&#xff0c;如果服务器响应和客户端…

漏洞指北-VulFocus靶场专栏-中级01

漏洞指北-VulFocus靶场专栏-中级01 中级001 &#x1f338;dcrcms 文件上传 &#xff08;CNVD-2020-27175)&#x1f338;step1&#xff1a;输入账号 密码burp suite 拦截 修改类型为 jpeg 中级002 &#x1f338;thinkphp3.2.x 代码执行&#x1f338;step1&#xff1a;burpsuite …

【ARM】Day6 cotex-A7核UART总线实验

cotex-A7核UART总线实验 1. 键盘输入一个字符‘a’&#xff0c;串口工具显示‘b’ 2. 键盘输入一个字符串"nihao"&#xff0c;串口工具显示“nihao” uart.h #ifndef __UART4_H__ #define __UART4_H__#include "stm32mp1xx_rcc.h" #include "stm3…

el-table实现纯前端查询列表(不走后端接口)

2023.8.16今天我学习了如何使用前端进行数据的查询&#xff0c;有时候后端会直接返回全部的数据&#xff0c;这时候我们就需要用前端进行查找数据。 首先elementUI有自带el-table查询的组件&#xff1a; Element - The worlds most popular Vue UI framework 我们发现在这段代…

Linux网络编程:网络基础

文章目录&#xff1a; 一&#xff1a;协议 二&#xff1a;网络应用设计模式_BS模式和CS模式 三&#xff1a;网络分层模型&#xff08;OSI七层 TCP/IP四层&#xff09; 四&#xff1a;通信过程 五&#xff1a;协议格式 1.数据包封装 2.以太网帧格式和ARP数据报格式 …

【Linux】Centos安装 mariadb 并授权远程登陆

&#x1f468;‍&#x1f393;博主简介 &#x1f3c5;云计算领域优质创作者   &#x1f3c5;华为云开发者社区专家博主   &#x1f3c5;阿里云开发者社区专家博主 &#x1f48a;交流社区&#xff1a;运维交流社区 欢迎大家的加入&#xff01; &#x1f40b; 希望大家多多支…

【云原生】Docker基本原理及镜像管理

目录 一、Docker概述 1.1 IT架构的演进&#xff1a; 1.2 Docker初始 1.3 容器的特点 1.4 Docker容器与虚拟机的区别 1.5 容器在内核中支持2种重要技术 1.6 Docker核心概念 1&#xff09;镜像 2&#xff09;容器 3&#xff09;仓库 二、安装Docker 2.1 Yum安装Docker…

三.net core 自动化发布到docker (创建一个dotnet工程发布)

创建Jenkins-create a job 输入名称&#xff08;建议不要带“”这类的字符&#xff09;&#xff0c;选择自由风格的类型&#xff08;红框标注的&#xff09;&#xff0c;点击确定 用于测试,下面选项基本没有选择-配置代码地址 选择执行shell #!/bin/bash # 获取短版本号 GITHA…

NLPR、SenseTime 和 NTU 加速自动视频纵向编辑

视频人像编辑技术已经在电视、视频和电影制作中得到了应用&#xff0c;并有望在不断发展的网真场景中发挥关键作用。最先进的方法已经可以逼真地将同源音频合成为视频。现在&#xff0c;来自北京模式识别国家实验室&#xff08;NLPR&#xff09;、商汤科技研究和南洋理工大学的…

OpenAI Function calling

开篇 原文出处 最近 OpenAI 在 6 月 13 号发布了新 feature&#xff0c;主要针对模型进行了优化&#xff0c;提供了 function calling 的功能&#xff0c;该 feature 对于很多集成 OpenAI 的应用来说绝对是一个“神器”。 Prompt 的演进 如果初看 OpenAI 官网对function ca…

CloudQuery实战 | 谁说没有一款一体化数据库操作管控云平台了?

文章目录 CloudQuery询盾的地址CloudQuery主页统一入口数据库归纳SQL编辑器权限管控审计中心数据保护数据变更 CloudQuery文档中心了解CloudQuery快速入门安装步骤社区版v2.1.0操作手册1数据查询更新日志 CloudQuery社区和活动 CloudQuery线上实战线上实战主页面展示及数据操作…