Enterprise AI Analysis

Leveraging Heterogeneity for Structure Discovery

The core innovation is to transform what often seems like a nuisance – heterogeneous data from multiple demonstrators – into a powerful source of information. If distinct demonstrators consistently exhibit different transition probabilities (e.g., different control patterns for an assembly line robot), this variation must be attributed to their unobserved actions, assuming environment dynamics are shared.

This insight leads to a mixture decomposition where the observable conditional distribution is a sum of latent action-conditioned transition kernels, weighted by demonstrator-specific policies. Assumption 1 outlines the structural premises: Sufficiency/Markov property, Exclusion restriction (demonstrator identity affects next observation *only* through chosen action), and Well-defined latent transitions.

The "sufficiently scattered" condition (Assumption 2 and 3) formalizes this policy diversity, ensuring that demonstrator policies are varied enough to make latent actions geometrically separable and avoid confounding. This prevents collapse where distinct latent actions might otherwise appear identical.

Robust Identifiability in Discrete Systems

For finite observation spaces, the problem is cast as a column-stochastic nonnegative matrix factorization (NMF). The observable conditional distributions become matrices (P*), factorized into latent transitions (T*) and demonstrator policies (Π*).

Theorem 4.1 proves state-wise identifiability up to permutation of latent action labels. It utilizes a minimum-volume criterion on the latent transition kernels, which geometrically means selecting the least spread-out set of kernels that still explain the observed data. This ensures uniqueness among feasible factorizations when demonstrator policies are sufficiently diverse (the "sufficiently scattered" condition).

The "sufficiently scattered" condition on demonstrator policies ensures that different demonstrators assign meaningfully different relative masses to latent actions, preventing them from being confounded.

Extending Identifiability to Complex & Connected Spaces

For continuous observation spaces, the paper extends the identifiability results using a Gram-determinant minimum-volume criterion on embedded latent transition measures (Theorem 4.2). An injective linear map (like a kernel mean embedding) is used to map probability measures to a Hilbert space, allowing for volume minimization.

Crucially, state-wise permutation ambiguities (where the action labels might be permuted differently for each observation 'o') are resolved. Theorem 4.3 demonstrates that if the observation space 'O' is a connected metric space and the transition map varies continuously, then these local permutations must agree globally, reducing the ambiguity to a single global permutation. This global permutation can then be fixed using a small amount of labeled action data (Assumption 5, Corollary 4.1).

This demonstrates how topological and analytical properties of the observation space further enhance the identifiability of latent structures.

Building Practical Algorithms for Real-World Data

The theoretical guarantees inform a practical estimation procedure. The paper proposes an objective function that combines:

• Lfit: A negative log-likelihood term to ensure the model accurately fits the observed data.
• Rvol: A minimum-volume regularizer for latent transitions, encouraging the simplest and most compact latent action representations.
• Rpol: A policy diversity barrier to prevent demonstrator policies from collapsing, ensuring they remain "sufficiently scattered."
• Lanchor: An optional label anchoring term using a small amount of labeled data to resolve the final global permutation ambiguity.

This principled approach suggests a pathway for future algorithms to leverage heterogeneous passive data for representation learning, model learning, and offline reinforcement learning, even without explicit action annotations.