Enterprise AI Analysis

The Challenge in Pilot Training Assessment

Current pilot flight training evaluation methods primarily rely on flight performance data, neglecting crucial physiological and psychological states during the "observation," "judgment," and "decision-making" phases. This leads to incomplete assessments of a pilot's comprehensive capabilities, especially in complex or emergency scenarios. We address this by linking cognitive alertness with flight ability, leveraging physiological responses to predict performance.

Our approach is grounded in the "OODA loop" theory (Observe, Orient, Decide, Act), a closed-loop model that characterizes the pilot's operational process. By collecting multimodal data across all stages of this loop, we can scientifically and comprehensively assess flight trainees' control capabilities.

Advanced Data Preprocessing for Multimodal Signals

Multimodal physiological data (EEG, ECG, eye-tracking, etc.) and flight control data are inherently noisy and present challenges in integration. To ensure high-quality input for our models, we implemented a robust two-stage preprocessing pipeline: an LSTM-optimized adaptive wavelet fuzzy thresholding denoising algorithm and a Short-Time Fourier Transform (STFT) based feature fusion method.

Our LSTM-optimized adaptive wavelet fuzzy thresholding significantly reduces noise while preserving signal integrity. This method dynamically adjusts filter parameters based on data characteristics, outperforming traditional approaches. For instance, at 10 dB SNR, our method achieved superior signal-to-noise ratios and lower mean square error:

For multimodal data fusion, we employ Short-Time Fourier Transform (STFT). This transforms time-domain signals into 2D time-frequency plots, capturing dynamic spectral characteristics across different modalities. This approach effectively solves the problem of isolated features across different time frames and reduces computational complexity compared to other fusion methods.

The choice of fusion strategy is critical for overall performance:

Robust Model Architecture: Transformer with Efficient Multi-Scale Attention

Our core classification model is built upon an enhanced Transformer network, augmented with an Efficient Multi-Scale Attention (EMA) module. Traditional CNNs struggle with global correlations in deep layers, and RNNs with long-distance dependencies. The Transformer's self-attention mechanism excels at capturing both local and global dependencies, making it ideal for sequence modeling in multimodal data.

The EMA module further refines this by processing information at different scales in parallel through multiple branches, enhancing feature extraction from diverse multimodal data while maintaining computational efficiency. This strategic integration significantly improves the model's ability to focus on critical features and dependencies across time and modalities.

To optimize training and encourage the model to extract complementary information from each modality, we utilize a multimodal orthogonalization loss function (MMO loss). Unlike standard loss functions, MMO loss penalizes redundant information, ensuring that each modality contributes uniquely and effectively to the final classification task. This results in faster convergence and improved fitting ability for complex multimodal temporal data.

Rigorous Experimental Validation and Superior Performance

We validated our algorithm using three distinct datasets: an internally collected Dataset I, the publicly available CogPilot dataset (Dataset II), and the WAUC dataset (Dataset III). A 5-fold cross-validation strategy ensured robust and statistically significant results across different pilot experience levels and flight conditions.

Our proposed EMA-Transformer model consistently outperformed baseline algorithms like CNN, RNN, standard Transformer, and a two-stream LSTM network across all datasets. For Dataset I, our method achieved 89% classification accuracy, a significant improvement over Transformer's 70%. On the challenging CogPilot dataset (Dataset II), we reached 91% accuracy, surpassing previous state-of-the-art results. For Dataset III, our model achieved a remarkable 93% accuracy.

The results demonstrate not only high accuracy but also superior robustness and generalization ability across varied data distributions. This confirms our model's capacity to effectively capture essential features of pilot cognitive load response, making it a practical and reliable tool for pilot training assessment. Our algorithm's ability to deliver consistent, high-accuracy results ensures that pilot training institutions can rely on precise, data-driven evaluations.