Thermal Non-Line-of-Sight Imaging through Rough Surfaces
Revolutionary AI for Seeing Around Corners with Thermal Cameras
Our physics-embedded neural network, NLOSFormer, overcomes limitations of previous methods by integrating real-world physics into its architecture, coupled with a novel data augmentation strategy. This breakthrough delivers high-fidelity, real-time reconstruction of dynamic hidden scenes and accurate relative depth estimation, opening new possibilities for security, search-and-rescue, and autonomous systems.
0
PSNR Improvement (Foam)
0
Dynamic Target Speed
0
Relative Depth Error
Strategic Implications for Enterprise AI
This research significantly advances non-line-of-sight imaging capabilities, presenting immediate opportunities for enhanced operational intelligence in critical sectors.
Robust Performance on Rough Surfaces
NLOSFormer significantly outperforms existing methods across various wall materials, from specular metal to highly diffuse foam, due to its physics-embedded design.
Real-Time Dynamic Imaging
Achieves 4 frames per second reconstruction of moving targets, crucial for practical applications like search-and-rescue and industrial monitoring.
Accurate Relative Depth Estimation
The network's ability to reconstruct the convolution kernel enables inference of target depth variations with an average error of 5%, providing critical spatial awareness.
Enhanced Generalization
A novel data augmentation strategy and the physics-informed architecture mitigate overfitting, allowing robust performance on unseen scenes and complex geometries.
Open-Source Dataset & Code
The introduction of the ThermalNLOS dataset and open-source code accelerates further research and development in thermal NLOS imaging applications.
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Physics-Embedded Neural Network: NLOSFormer
NLOSFormer integrates a kernel estimator branch and an image restorer branch within an end-to-end transformer-based framework. It formulates light transport as a convolution process, estimating the convolution kernel from measurements and using it to guide reconstruction. This physics-guided approach, enforced through a regularization term in the loss function, restricts the solution space to physically feasible reconstructions, leading to superior generalization and accuracy compared to purely data-driven models.
Real-time Imaging of Dynamic Targets
The system achieves real-time thermal NLOS imaging at 4-5 frames per second (fps) for dynamic targets. This is enabled by an efficient data processing pipeline that includes noise reduction and downsampling, streaming data to a GPU for batched inference. Experimental results demonstrate successful reconstruction of human subjects performing motions like squats and arm waving, capturing clear silhouettes with high fidelity.
Data Augmentation for Robustness
To overcome overfitting and enhance generalization, a comprehensive ThermalNLOS dataset was created, featuring a novel geometry-aware data augmentation strategy. This strategy explicitly varies character stroke widths and body shapes, forcing the network to decouple intrinsic object geometry from wall scattering effects. This prevents memorization of intensity blob sizes and significantly improves robustness on unseen scenes and wall materials, as validated by experiments on airplane and guitar models.
Inferring Relative Depth from Kernel
A key innovation is the ability to perform relative depth estimation of moving targets. By analyzing the scale of the reconstructed convolution kernel, which expands linearly with target depth, NLOSFormer can infer depth variations. Experiments with a human target moving from 2m to 4m demonstrate that the predicted depths closely match ground truth with an average relative error of around 5%, showcasing the reliability of the network’s estimated physical model.
Enterprise Process Flow
Raw Thermal Image Input
→
Preprocessing (Noise Removal, Normalization)
→
Kernel Estimator Branch (PCA-reduced kernel)
→
Image Restorer Branch (Kernel-guided reconstruction)
→
Reconstructed Hidden Scene
9.4 dB
PSNR improvement over second-best method on challenging foam surface.
| Material / Scene | NLOSFormer (SSIM/PSNR) | UNet | NLOS-OT | NLOS-I2V | LMS-NLOS |
|---|---|---|---|---|---|
| Metal / Letter T | 0.961/33.12 | 0.952/30.45 | 0.830/27.10 | 0.915/31.80 | 0.890/30.50 |
| White Tabletop / Man | 0.882/28.95 | 0.545/17.85 | 0.535/17.20 | 0.540/16.50 | 0.802/22.50 |
| Card Board / Letter T | 0.845/26.15 | 0.278/9.30 | 0.281/10.50 | 0.250/9.20 | 0.310/12.40 |
| Foam / Man | 0.765/20.85 | 0.210/6.15 | 0.168/5.17 | 0.232/9.51 | 0.220/11.50 |
Case Study: Imaging Through Diverse Rough Surfaces
Experiments using metal, white tabletop, cardboard, and foam walls demonstrate NLOSFormer’s superior robustness. While other methods struggle with increasing surface roughness, our physics-embedded approach consistently reconstructs hidden objects like a T-shaped heater and a standing human figure with high fidelity. The reconstructed kernel maps visually confirm the physical consistency, showing broadening and reduced peak magnitude as surface roughness increases, validating the model’s accuracy.
Advanced ROI Calculator
Estimate the potential return on investment for integrating advanced AI capabilities into your operations.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your AI Implementation Roadmap
A structured approach to integrate cutting-edge AI, ensuring maximum impact and minimal disruption.
Phase 01: Discovery & Strategy
Comprehensive assessment of your current infrastructure, identification of key opportunities, and development of a tailored AI strategy aligned with your business objectives.
Phase 02: Pilot & Proof of Concept
Rapid deployment of a focused AI solution to validate its effectiveness, gather initial data, and demonstrate tangible value within a controlled environment.
Phase 03: Scaled Integration
Phased rollout of the AI solution across relevant departments, ensuring seamless integration with existing systems and robust performance at scale.
Phase 04: Optimization & Future-Proofing
Continuous monitoring, performance tuning, and identification of advanced AI capabilities to maintain competitive advantage and drive sustained innovation.
Ready to Transform Your Enterprise?
Book a complimentary consultation with our AI specialists to explore how these advancements can be tailored to your organization's unique needs.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat