Enterprise AI Analysis

Advanced Multi-Modal Perception with YOLO11

The system integrates an advanced visual perception module based on a custom-trained YOLO11 model, chosen for its speed and near-optimal performance. This model goes beyond simple object detection to provide detailed semantic classification of victims' status.

Training involved a composite dataset from C2A (Human Detection in Disaster Scenarios) and NTUT 4K (Human Detection & Behavior), with a normalized class taxonomy including:

• High-Priority/Immobile: Identifying 'sit' (sitting, lying, collapsed) states for urgent assessment.
• Low-Priority/Mobile: Detecting 'stand' or 'walk' states.
• General Actions: 'Person', 'push', 'riding', 'watchphone' for broader human presence understanding.
• Visibility & Occlusion: 'Block25', 'Block50', 'Block75' to quantify partial obstruction.

Performance Metrics:

• Precision: 95.86% – High confidence in correct detections.
• Recall: 40.25% – A deliberate trade-off for generalization; critical classes (riding, stand, sit) show high mAP scores (0.978, 0.964, 0.905 respectively), ensuring reliability where it matters most.
• mAP@0.5: 61.34% – Balanced performance, though impacted by lower scores in occlusion classes.

Precision Geolocation & First-Aid Delivery

After detecting and classifying a victim, the system's next critical step is to determine their precise geographical coordinates to enable rapid response. A passive, monocular geolocation method is employed, combining:

• Drone Telemetry: GPS coordinates (latitude, longitude), height, and orientation from the IMU.
• Visual Data: Vertical deviation angle calculated from the victim's position in the camera feed.

This method uses basic trigonometry and spherical cosine law formulas to compute the target's GPS coordinates, achieving a localization error of approximately one meter at 120m altitude, dropping to ~17cm at 20m altitude, which is negligible for intervention teams.

Furthermore, the drone is equipped with an integrated package delivery module. This module, powered by a servo motor, can transport and release first-aid packages (300–700g) containing essential items like medicine, walkie-talkies, mini-GPS, and food rations. This capability transforms the drone from a passive observation tool into an active system, enabling direct intervention in emergency situations.

Robustness through Feedback Loops & Grounding

A key differentiator of this architecture is its inherent robustness, primarily achieved through an internal feedback loop and grounding mechanisms. The Orchestrator Agent (OAg) plays a supervisory role, performing "meta-reasoning" to validate the logical consistency of information from all specialized sub-agents.

If an anomaly or contradiction is detected (e.g., a low-risk assessment for a high-risk situation), the OAg does not propagate the error. Instead, it initiates a self-correcting feedback loop, re-invoking the responsible agent with additional constraints and directives (e.g., "Reevaluate the risk for Entity Person_01 given the 'sit' status for over two minutes. Apply the medical emergency protocol."). This iterative refinement ensures the final output is based on robust analysis, preventing reliance on unprocessed initial findings.

To mitigate the risk of "hallucinations" – incorrect reasoning common in LLMs – the system employs two strategies:

• Grounding in Verifiable Knowledge: The OAg validates every critical piece of information against the structured, factual data stored in the Memory Agent (MAg), which acts as the system's "source of truth."
• Multi-Agent Review: The OAg scrutinizes outputs from specialized sub-agents, ensuring consistency before finalizing decisions.

This robust, iterative process ensures the system reliably closes the perception-reasoning-action loop, generating high-confidence, actionable intelligence and enhancing overall reliability in critical missions.