Enterprise AI Analysis
GEODM: GEOMETRY-AWARE DISTRIBUTION MATCHING FOR DATASET DISTILLATION
Dataset distillation aims to synthesize a compact subset of the original data, enabling models trained on it to achieve performance comparable to those trained on the original large dataset. Existing distribution-matching methods are confined to Euclidean spaces, making them only capture linear structures and overlook the intrinsic geometry of real data, e.g., curvature. However, high-dimensional data often lie on low-dimensional manifolds, suggesting that dataset distillation should have the distilled data manifold aligned with the original data manifold. In this work, we propose a geometry-aware distribution-matching framework, called GeoDM, which operates in the Cartesian product of Euclidean, hyperbolic, and spherical manifolds, with flat, hierarchical, and cyclical structures all captured by a unified representation. To adapt to the underlying data geometry, we introduce learnable curvature and weight parameters for three kinds of geometries. At the same time, we design an optimal transport loss to enhance the distribution fidelity. Our theoretical analysis shows that the geometry-aware distribution matching in a product space yields a smaller generalization error bound than the Euclidean counterparts. Extensive experiments conducted on standard benchmarks demonstrate that our algorithm outperforms state-of-the-art data distillation methods and remains effective across various distribution-matching strategies for the single geometries.
Executive Impact
GeoDM offers significant advancements for enterprises, translating into measurable improvements across key operational metrics.
0%
Reduction in Data Storage Footprint
0x
Faster Training Convergence
0%
Improvement in Model Accuracy on Complex Datasets
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
This paper introduces GeoDM, a novel framework for dataset distillation that leverages geometry-aware distribution matching. Unlike traditional methods that operate solely in Euclidean spaces, GeoDM utilizes a Cartesian product of Euclidean, hyperbolic, and spherical manifolds to capture the intricate intrinsic geometry of real-world data.
The core innovation lies in its ability to adapt to underlying data geometry through learnable curvature and weight parameters. This ensures that the distilled data manifold aligns more closely with the original, high-dimensional data manifold, which often exhibits non-Euclidean structures like hierarchies or cycles. An optimal transport loss further enhances distribution fidelity across these diverse geometric components.
GeoDM's technical foundation rests on a product Riemannian space, combining Euclidean, hyperbolic, and spherical manifolds. It employs a Riemannian convolutional neural network capable of handling factor-wise feature maps and geometry-aware operations. Learnable curvature parameters for hyperbolic and spherical components, alongside learnable weights for each geometry, allow dynamic adaptation to data's intrinsic structure.
The framework incorporates a geometry-aware optimal transport (OT) loss to measure and align real and synthetic samples across the product space components. This loss couples different geometries, preserves class-conditional mass, and prevents degenerate solutions. Theoretical analysis demonstrates that this product-space approach yields a strictly tighter generalization error bound compared to single Euclidean latent spaces, validating the benefits of geometry-aware matching.
Extensive experiments on standard benchmarks (MNIST, CIFAR-10, CIFAR-100) confirm GeoDM's superior performance, consistently outperforming state-of-the-art dataset distillation methods. For instance, on MNIST at IPC=1, GeoDM achieves 96.3% accuracy, a 2% improvement over the best baseline.
The method exhibits robustness across various distribution-matching strategies, indicating that the product-space modeling itself, rather than a specific matching objective, drives the performance gains. Ablation studies further highlight the significant improvements gained from incorporating product spaces, curvature adaptation, and optimal transport loss. GeoDM also demonstrates strong generalization across different model architectures and maintains effectiveness on higher-resolution datasets.
74.4%
Accuracy on CIFAR-10 (IPC=10) with GeoDM
Enterprise Process Flow
Curved Feature Network
→
Learnable Curvature & Weights
→
Product Space Embedding (E×H×S)
→
Distribution Matching (NCFM)
→
Geometry-aware Optimal Transport
→
Optimized Distilled Dataset
| Method | Accuracy (%) |
|---|---|
| DM | 63.0 |
| CAFE | 67.5 |
| M3D | 69.9 |
| NCFM | 77.4 |
| GeoDM (Ours) | 78.3 |
Impact of Product Space Modeling on Feature Richness
This study examines how the integration of product spaces enhances the richness of geometric information captured by synthetic images, particularly when facing severe data scarcity at low Images Per Class (IPC) budgets.
- Consistent Gains at Low IPC: GeoDM achieves significant accuracy improvements (e.g., ~2% on MNIST IPC=1) by encoding richer geometric structures, compensating for limited synthetic data.
- Diminished Relative Improvement at High IPC: As the number of synthetic samples increases, the need for additional structural information from product spaces reduces, leading to smaller relative gains.
- Robustness Across DM Variants: The framework's consistent advantage across different distribution matching objectives (DM, DSDM) validates the intrinsic benefit of product-space modeling, beyond specific algorithmic choices.
- Deeper Networks Exploit Non-Euclidean Cues: When using more expressive backbones like ResNet-18, GeoDM maintains a stronger margin over Euclidean baselines, indicating that deeper architectures can better leverage the non-Euclidean geometric information.
The findings underscore that product-space modeling is crucial for dataset distillation, especially under data scarcity, by enabling synthetic images to capture and retain complex geometric properties inherent in real-world data, leading to improved generalization and robustness.
1.01x
Computational Overhead for Curvature & Weights
Calculate Your Potential ROI
Estimate the efficiency gains and cost savings your enterprise could achieve by implementing GeoDM's advanced dataset distillation.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your Implementation Roadmap
A typical phased approach to integrating GeoDM into your existing AI/ML workflows for maximum impact and minimal disruption.
Phase 01: Initial Assessment & Pilot
Evaluate current dataset distillation practices, identify high-impact use cases, and conduct a small-scale pilot of GeoDM on a specific model or dataset to demonstrate value.
Phase 02: Integration & Customization
Integrate GeoDM framework with existing MLOps pipelines. Customize curvature and weight parameters for specific enterprise data geometries. Train initial models with distilled datasets.
Phase 03: Scaled Deployment & Optimization
Roll out GeoDM across more datasets and models. Monitor performance, fine-tune parameters, and continuously optimize for speed, accuracy, and efficiency gains.
Phase 04: Continuous Improvement & Expansion
Establish feedback loops for ongoing refinement. Explore new applications of geometry-aware distillation, expanding to novel data types or complex multimodal scenarios.
Ready to Transform Your AI Workflows?
Discover how geometry-aware dataset distillation can dramatically reduce training times and improve model performance in your enterprise.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat