Enterprise AI Analysis

MER studies utilize diverse datasets, incorporating features from music (audio, MIDI, metadata) and participants (demographic, survey, physiological signals). Publicly shared datasets like DEAM and AMG1608 primarily feature Western pop music, ranging from 744 to 1,802 excerpts, manually annotated by various participants. Audio software suites like OpenSMILE and MIR Toolbox are used for feature extraction. The distinction between predictive and explanatory modeling frameworks is crucial: large datasets support complex predictive models, while smaller, curated datasets are valuable for explaining musical factors influencing emotion. While visual domain datasets are much larger (e.g., EmoSet with 118,102 images), MER datasets are resource-intensive to annotate directly, leading to efforts in inferring emotions from tags (e.g., MTG-Jamendo, Music4all). Small datasets remain useful for testing new features or as reference standards.

Predictive accuracy in MER has improved significantly over the past decade. Regression models for arousal/valence have seen peaks at 58%/28% (2008), 70%/26% (2010), and 67%/46% (2021). Classification rates increased from 53% to 70% and then 83%. Despite these improvements, comparing studies is challenging due to inconsistencies in metrics, models, and evaluation criteria. The overall accuracy has improved, but valence remains harder to predict than arousal. Recent advancements involve identifying more relevant feature sets, integrating multimodal data, and leveraging neural networks to learn features directly from audio. Other approaches address how emotions are represented across different contexts and languages. The "semantic gap" is often better understood as inherent measurement error from annotations, feature representations, and model limitations.

The meta-analysis highlights a critical need for improvements in MER research practices. Future reports should provide comprehensive information on data, models, and success metrics, including CV descriptions, feature extraction techniques, and actual accuracy measures. The use of Matthews Correlation Coefficient (MCC) for classification and R² for regression is recommended for standardization. Transparency in inter-rater agreement and dataset annotation quality is crucial. Sharing models (code/notebooks) and features is highly encouraged, ideally through platforms like Zenodo or Kaggle, to facilitate direct comparisons and reproducibility. Detailed reporting of stimuli, genres, duration, sampling rates, encoding formats, features (types, extraction software, quantity, transformations, reduction methods), and model details (types, tuning, CV) is essential. A key aspect is the choice of features: domain-knowledge driven, limited features (often in music psychology) versus large feature sets used by machine learning. The study suggests the domain-knowledge approach currently leads to higher model accuracy.