ENTERPRISE AI ANALYSIS

Interactive Previews for Enhanced Control

We propose DiffusionBrowser, a model-agnostic, lightweight decoder framework that allows users to interactively generate previews at any point (timestep or transformer block) during the denoising process. Our model can generate multi-modal preview representations that include RGB and scene intrinsics at more than 4× real-time speed (less than 1 second for a 4-second video) that convey consistent appearance and motion to the final video.

Drawing inspiration from traditional graphics rendering pipelines, we designed DiffusionBrowser to be able to preview auxiliary intrinsic channels such as albedo, depth, and surface normals on top of RGB pixels. We show that these intrinsics emerge early in the generation process and can be decoded using a carefully designed multi-branch, modality-optimized decoder.

Multi-Branch Decoders for Robustness

We mitigate the superposition problem with a multi-branch decoding architecture (MB); see Figure 4 in the paper for an illustration. Instead of a single deterministic head, we introduce K independent decoders, each predicting intrinsic maps. Their ensemble average is trained jointly with individual heads to provide more robust and diverse predictions.

Early Emergence of Scene Intrinsics

Indeed, intrinsic scene representations emerge early in the denoising process. We demonstrate this by training a set of linear probes for the scene intrinsics. Specifically, given a transformer-based diffusion model with N blocks and a denoising schedule that involves Nt steps, we attach linear projection layers, each at a distinct block b and timestep t, to predict target intrinsic maps. The results in Figure 2 show that intrinsics emerge early blockwise and stepwise, supporting our thesis that these semantic features can be useful in early-step preview generation.

Interactive Variation Generation

We propose two variation generation methods to steer between siblings within the same noise level: Stochastic Renoising introduces variation by renoising a clean latent prediction using different random noise, preserving image structure while introducing finer-scale variations. Latent Steering allows for channel-targeted steering by solving an optimization problem to guide the decoded intrinsic map toward a chosen target, generating purposeful variations.

User Study Validation

To evaluate the perceptual quality of our representations, we conducted a user study with 35 participants comparing our method against the x0-pred baseline. Participants were shown two representations alongside a reference video and asked to judge which better predicted video content, exhibited fewer visual artifacts, and more clearly conveyed scene composition. Previews generated by DiffusionBrowser were preferred 74.6%, 72.9%, and 76.9% of the time for content predictability, visual fidelity, and scene clarity, respectively, when compared to the x0-pred baseline.

Benefits of Multi-Modal Previews

Multi-modal previews provide several advantages over latent-space visualizations. First, intrinsic modalities, particularly depth and normals, reveal coarse scene geometry earlier in the denoising process than RGB or latents. Second, base color previews offer simplified appearance information without lighting, making scene layout clearer. Third, our method produces all previews simultaneously from the same features, allowing users to cross-reference modalities at any timestep.

Limitations and Future Work

While our framework enables fast and semantically meaningful previews for video diffusion models, several limitations remain. We deliberately limit our scope to scene intrinsics, and text prompts are not considered; the interaction between intrinsic previews and text-driven conditioning can be explored in future work. Additionally, there are failure cases in steering where steered intrinsics dissipate as denoising progresses. For future work, we aim to explore alternative decoder architectures to improve mode separation and produce clearer, more coherent outputs at higher resolution, as well as expand the intrinsic representations to include additional modalities.