AI in Construction Safety
Research on safety hazard identification and risk warning of smart construction sites combined with large language model
With the acceleration of urbanization and the expansion of engineering construction, the safety management of construction sites faces unprecedented challenges. Traditional hidden danger identification and early warning methods have obvious shortcomings in response speed, recognition accuracy and multi- source information integration. Based on the background of smart construction site construction, this paper proposes a safety hazard identification and risk early warning system that integrates multimodal data and large language model (LLM), constructs a unified processing flow covering text, image and voice data, and designs a multi-channel neural architecture to achieve deep semantic parsing and risk classification. Through field deployment in two typical construction sites, high-rise residential buildings and underground municipal engineering, data collection and model training are completed, and compared with traditional rule methods. The results show that the proposed model is significantly superior to traditional methods in terms of hidden danger identification accuracy, early warning response time and multi-label risk understanding ability, providing a new path for smart building safety management.
Authors: Yi Liu*, Shanghai Communications Polytechnic | Mengyang Pan, Shanghai Urban Construction Vocational College
Executive Impact Summary
The integration of Large Language Models (LLMs) and multimodal data fusion is revolutionizing construction site safety. Our research demonstrates significant advancements in identifying hazards and responding to risks, leading to a safer and more efficient work environment.
0%
Recognition Accuracy
0s
Warning Response Time
0%
Reduced Unhandled Risk Rate
0 Modalities
Data Fusion
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Overview
Methodology
Results & Impact
The construction industry faces growing safety challenges due to increasing project complexity and urbanization. Traditional safety management, relying on manual inspections and limited data sources, is inefficient and reactive. This paper introduces a proactive smart construction site system leveraging Internet of Things (IoT), edge computing, and Artificial Intelligence, with a particular focus on multimodal Large Language Models (LLMs) for superior hazard identification and risk warning.
Our system employs a three-layer architecture: a Perception Layer (sensors, cameras, RFID) for real-time data collection, an Edge Layer for preliminary processing, and a Platform Layer integrating LLMs for deep analysis. A key component is the multimodal LLM, which processes text, image, and voice data through specialized neural architectures (ResNet/ViT for images, BERT/LLaMA for text, ASR for voice). This fusion allows for deep semantic parsing and accurate risk classification, moving beyond static rules to dynamic, context-aware hazard detection. The risk probability is quantified using a multi-factor model (Pr = 1 - product(1-pi)).
Field tests at two diverse construction sites (high-rise residential and underground municipal) confirmed the system's adaptability and robustness. The Multimodal LLM achieved 91% recognition accuracy and a significantly reduced warning response time of 4.8 seconds, outperforming traditional rule-based methods. This leads to an 84% reduction in unhandled risk within the critical first 30 seconds. The findings validate the potential of AI, especially LLMs, to transform construction safety management by providing data-driven, intelligent decision-making and proactive intervention capabilities.
Key Performance Indicator
91% Hazard Recognition Accuracy via Multimodal LLMOur Multimodal LLM achieves a 91% recognition accuracy, significantly surpassing traditional rule-based systems (76%). (Table 5)
Critical Response Time
4.8s Average Warning Response TimeThe system reduces warning response time to just 4.8 seconds, enabling rapid intervention compared to 12.4s for traditional methods. (Table 5)
Smart Construction Site System Architecture Overview
Perception Layer (Data Collection)
→
Edge Layer (Local Processing)
→
Platform Layer (LLM & Analysis)
| Metric | Traditional Rule System | Multimodal LLM |
|---|---|---|
| Recognition Accuracy | 76% | 91% |
| False Positive Rate | 18% | 7% |
| Avg. Warning Response Time | 12.4s | 4.8s |
| Semantic Understanding | Limited, Rule-based | Advanced, Context-aware |
| Multimodal Data Fusion | No | Yes |
Real-world Deployment: High-rise Residential Project (Site A)
Diverse hazard types addressed. The system was deployed in a high-rise residential building project, effectively identifying risks associated with high-altitude work, tower crane operations, concrete pouring, and high voltage electricity. The multimodal data input (video, sensor, voice) proved crucial in capturing varied hazards.
Real-world Deployment: Underground Municipal Engineering (Site B)
Adaptability in complex, closed environments. Deployment in an underground municipal engineering project demonstrated the system's ability to handle unique risks in closed environments, such as toxic gas leakage, abnormal equipment operation, and temporary support failures. This validates the model's generalization capabilities across diverse construction scenarios.
Risk Warning Information Flow
Input Data (Graphics, Text, Voice)
→
Multimodal Model Recognition
→
Risk Level Judgment
→
Warning Information Generation
→
Platform Response and Recording
Quantify Your Potential ROI
Estimate the significant operational savings and reclaimed hours your enterprise could achieve by adopting AI-powered solutions.
Annual Cost Savings
$0
Annual Hours Reclaimed
0
Your AI Implementation Roadmap
A structured approach to integrating advanced AI into your enterprise operations, ensuring measurable results and sustainable impact.
Phase 1: Discovery & Strategy
Comprehensive analysis of current workflows, identification of AI opportunities, and development of a tailored implementation strategy and ROI projections.
Phase 2: Solution Design & Data Preparation
Designing the AI solution architecture, data collection and cleaning, and establishing robust data pipelines for training and deployment.
Phase 3: Development & Integration
Building and training custom LLMs or fine-tuning existing models, followed by seamless integration into your existing enterprise systems and workflows.
Phase 4: Deployment & Optimization
Phased rollout, continuous monitoring, performance tuning, and iterative improvements to maximize efficiency and achieve desired outcomes.
Ready to Transform Your Enterprise with AI?
Partner with us to explore how multimodal LLMs can enhance safety, efficiency, and intelligence across your operations.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat