ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training
Unlock unparalleled efficiency in 3D reconstruction: ZipMap delivers linear-time scalability without compromising accuracy.
ZipMap revolutionizes 3D reconstruction for large image collections by introducing a stateful, linear-time feed-forward model. Leveraging Test-Time Training (TTT) layers, it compresses entire image sequences into a compact hidden state, enabling over 700 frames to be reconstructed in under 10 seconds—more than 20 times faster than state-of-the-art quadratic-time systems like VGGT, while matching or exceeding their reconstruction quality. This innovation provides real-time scene state querying and supports efficient sequential streaming, making high-fidelity 3D perception scalable for massive datasets.
Transforming Enterprise 3D Perception
ZipMap addresses the critical bottleneck of quadratic computational cost in existing state-of-the-art 3D reconstruction models. Its linear-time scaling allows enterprises to process massive image and video datasets with unprecedented speed and efficiency, unlocking new possibilities for large-scale environmental mapping, robotics, and augmented reality applications. By integrating real-time scene state querying and streaming capabilities, ZipMap offers a foundational shift towards truly dynamic and scalable 3D vision systems.
20x
Faster 3D Reconstruction
700+
Frames in <10s
100 FPS
Real-time Scene Query
1
Single Forward Pass
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Core Innovation
Architectural Design
Real-time Scene Representation
Linear-Time Scalability with TTT
ZipMap's breakthrough is its use of Test-Time Training (TTT) layers, which compress an entire image collection into a fixed-size set of 'fast weights'. This enables the model to process large image datasets with a computational cost that scales linearly with the number of input images, overcoming the quadratic scaling limitations of prior state-of-the-art global attention mechanisms. This efficient state aggregation ensures global coherence and high scalability for massive datasets.
Stateful Feed-Forward Backbone
The ZipMap architecture combines local window attention for intra-view relationships with global Large-Chunk TTT layers for inter-view information aggregation. Input images are tokenized using a pretrained DINOv2 encoder. The TTT layers dynamically adapt their 'fast weights' via a virtual test-time training objective, storing the scene's global information. This design facilitates bidirectional 3D reconstruction of camera poses, depth maps, and point clouds in a single, rapid forward pass.
Queryable Implicit Scene State
A unique advantage of ZipMap is its ability to compress the entire scene into a compact, queryable hidden state within the TTT layers. This implicit scene representation can be queried in real-time (approximately 100 FPS), independently of the number of input views, to generate pixel-aligned geometry and appearance from novel viewpoints. Furthermore, the model can infer plausible scene structures in unobserved regions, demonstrating an understanding of basic 3D priors.
75 FPS
ZipMap reconstructs over 700 frames in under 10 seconds on a single H100 GPU, achieving 75 FPS – significantly outpacing SOTA methods.
ZipMap's Linear-Time Reconstruction Flow
Input Image Sequence (N Frames)
→
DINOv2 Tokenization (Per Frame)
→
Local Window Attention
→
Global Large-Chunk TTT Layer (Fast Weight Update)
→
Scene State Aggregated
→
Prediction Heads (Pose, Depth, Point Map)
→
3D Reconstruction / Novel View Query
| Method | Complexity | Time (s) |
|---|---|---|
| ZipMap (Ours) | O(N) | 9.999 |
| CUT3R | O(N) | 31.246 |
| TTT3R | O(N) | 31.197 |
| π³ | O(N²) | 151.159 |
| VGGT | O(N²) | 200.364 |
| ZipMap delivers significantly faster reconstruction times, especially for long sequences, outperforming quadratic-time methods by over 20x and linear-time methods by ~3x at 750 frames. Data from Table 7. | ||
| Method | Complexity | AUC@5↑ | AUC@15↑ | AUC@30↑ |
|---|---|---|---|---|
| ZipMap (Ours) | O(N) | 53.34 | 74.97 | 84.30 |
| π³ | O(N²) | 63.10 | 80.31 | 87.40 |
| VGGT | O(N²) | 38.71 | 66.46 | 78.89 |
| CUT3R | O(N) | 46.92 | 70.65 | 81.68 |
| TTT3R | O(N) | 46.37 | 70.33 | 81.51 |
| ZipMap achieves comparable or superior camera pose accuracy on RealEstate10K, matching or exceeding prior state-of-the-art while maintaining linear computational complexity. Data from Table 1. | ||||
Real-time Scene State Querying and Inference
Problem: Traditional 3D reconstruction models generate a static output, making real-time querying for novel views or inferring unseen structures computationally expensive or impossible without re-computation.
Solution: ZipMap's TTT layers create a compact, queryable hidden scene state in a single forward pass. This state can be queried in real-time (~100 FPS) to produce pixel-aligned geometry and appearance at novel viewpoints, effectively acting as an implicit scene representation. Furthermore, it demonstrates an ability to infer common 3D structures (e.g., walls, floors) in unobserved regions, encoding basic scene priors.
Impact: This capability transforms how 3D scenes are interacted with, enabling dynamic applications like real-time navigation, augmented reality overlays, and efficient sequential streaming reconstruction without recalculating the entire scene, leading to unprecedented flexibility and responsiveness.
Calculate Your Potential AI ROI
Estimate the efficiency gains and cost savings your enterprise could realize with advanced AI solutions like ZipMap.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your AI Implementation Roadmap
Deploying advanced AI like ZipMap requires a strategic, phased approach. Our experts guide you through every step, ensuring seamless integration and maximum impact.
Phase 1: Discovery & Strategy
In-depth analysis of your current 3D reconstruction needs and data infrastructure. Define clear objectives and a tailored AI strategy that aligns with your business goals.
Phase 2: Data Preparation & Model Customization
Assist with data ingestion, annotation, and fine-tuning ZipMap for your specific datasets and environmental conditions to maximize accuracy and efficiency.
Phase 3: Integration & Deployment
Seamless integration of ZipMap into your existing workflows and systems. Deployment on your preferred cloud or on-premise infrastructure with full technical support.
Phase 4: Optimization & Scaling
Continuous monitoring, performance optimization, and scaling of your ZipMap solution as your data volumes and operational demands grow. Future-proofing your 3D perception capabilities.
Ready to Revolutionize Your 3D Workflows?
ZipMap offers an unprecedented blend of speed and accuracy for large-scale 3D reconstruction. Connect with our AI specialists to explore how this innovation can drive efficiency and unlock new capabilities for your enterprise.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat