Enterprise AI Analysis

Entropy-Aware Decision for Data Retransmission

A core innovation of this framework is the uncertainty-aware retransmission mechanism. Instead of blindly transmitting full-resolution images, the server intelligently decides when to request additional visual details.

This decision is based on quantifying the VLM's inference uncertainty using the min-entropy of the output tokens. Min-entropy provides a conservative measure of uncertainty: a high min-entropy indicates that the model's prediction is likely unreliable and requires more detailed visual input for refinement.

By aggregating token-level min-entropy values across the generated output sequence, the server obtains a scalar decision statistic. If this average min-entropy exceeds a predefined threshold, the edge device is instructed to retransmit a detail-preserved local image of the identified Region of Interest (RoI).

This approach ensures that critical visual information is only transmitted when necessary, leading to substantial reductions in communication overhead while maintaining or even improving task accuracy.

The paper extensively compares min-entropy with other metrics like Shannon entropy and probability margin, demonstrating its superior effectiveness in distinguishing correct from incorrect predictions and achieving a more favorable trade-off between communication cost and accuracy (Figure 6 & Table II).

Attention-Guided Visual Cropping (ViCrop)

To acquire the detail-preserved local image, the framework integrates Attention-Guided Visual Cropping (ViCrop). This method leverages the VLM's internal attention mechanisms to identify the most semantically important Region of Interest (RoI) within the global image.

Specifically, the server computes a relative attention map by normalizing the raw attention weights from the LLM decoder (focused on image tokens) with a generic attention map. This relative map highlights areas the VLM focuses on when generating an answer to the specific query.

Once the RoI is identified, its bounding box coordinates are transmitted to the edge device. The edge device then crops the original high-resolution image to extract only this semantically critical region, resizes it, and sends it back to the server. This targeted approach avoids transmitting entire high-resolution images, significantly reducing bandwidth usage.

ViCrop, when activated by high inference uncertainty, enhances the VLM's ability to capture fine-grained visual details that might otherwise be discarded during initial downscaling, directly contributing to improved accuracy for complex multimodal tasks.

Performance & Tradeoff Analysis

The proposed framework achieves a superior trade-off between communication cost, computational overhead, and inference accuracy across diverse VLM architectures and datasets.

On benchmarks like TextVQA+OCR and POPE, the framework significantly outperforms high-resolution end-to-end models (e.g., LLaVA-1.5-HD) in communication efficiency while matching or exceeding their accuracy.

For instance, it achieves comparable accuracy to LLaVA-1.5-HD with only 0.28 additional communication cost on TextVQA+OCR, significantly less than the 0.78 required by LLaVA-1.5-HD. Moreover, the selective retransmission based on min-entropy results in a lower expected computational cost compared to unconditional full retransmission or constantly processing higher-resolution images.

The framework also demonstrates generalizability, maintaining its effectiveness across various VLM architectures (InstructBLIP, Qwen2.5-VL) and datasets (A-OKVQA, VQAv2, GQA).

Furthermore, the approach is highly compatibility with existing image compression techniques (e.g., JPEG), allowing for additional communication savings without significant accuracy degradation. This synergy ensures maximum efficiency for enterprise-level VLM deployments.