Enterprise AI Analysis

Lightweight Multimodal Adaptation of Vision-Language Models for Species Recognition and Habitat-Context Interpretation in Drone Thermal Imagery

Hao Chenª, Fang Qiuª*, Fangchao Dongª, Defei Yangª, Eve Bohnettᵇ, Li Anᶜ,ᵈ
ª Geospatial Information Science, The University of Texas at Dallas, Richardson, TX 75080, USA
ᵇ Department of Landscape Architecture, University of Florida, Gainesville, FL 32611, USA
ᶜ College of Forestry, Wildlife and Environment, Auburn University, Auburn, AL 36849, USA
ᵈ International Center for Climate and Global Change Research, College of Forestry, Wildlife and Environment, Auburn University, Auburn, AL 36849, USA
Corresponding author: Fang Qiu, Geospatial Information Sciences, University of Texas at Dallas, 800 West Campbell Road, Richardson, TX, 75080, United States, ffqiu@utdallas.edu

This study proposes a lightweight multimodal adaptation framework to bridge the representation gap between RGB-pretrained VLMs and thermal infrared imagery, and demonstrates its practical utility using a real drone-collected dataset. A thermal dataset was developed from drone-collected imagery and was used to fine-tune VLMs through multimodal projector alignment, enabling the transfer of information from RGB-based visual representations to thermal radiometric inputs. Three representative models, including InternVL3-8B-Instruct, Qwen2.5-VL-7B-Instruct, and Qwen3-VL-8B-Instruct, were benchmarked under both closed-set and open-set prompting conditions for species recognition and instance enumeration. Among the tested models, Qwen3-VL-8B-Instruct with open-set prompting achieved the best overall performance, with F1 scores of 0.935 for deer, 0.915 for rhino, and 0.968 for elephant, and within-1 enumeration accuracies of 0.779, 0.982, and 1.000, respectively. In addition, combining thermal imagery with simultaneously collected RGB imagery enabled the model to generate habitat-context information, including land-cover characteristics, key landscape features, and visible human disturbance. Overall, the findings demonstrate that lightweight projector-based adaptation provides an effective and practical route for transferring RGB-pretrained VLMs to thermal drone imagery, expanding their utility from object-level recognition to habitat-context interpretation in ecological monitoring.

Keywords: Vision-language models, drone thermal imagery, multimodal adaptation, wildlife monitoring, habitat-context interpretation