Autonomous and Adaptive Cyber Incident Detection and Response in Industrial Cyber-Physical Systems using Hierarchical Reinforcement Learning

AI-Powered Cyber Resilience for Critical Infrastructure

This paper presents a novel AI-enabled solution for adaptive cyber incident detection and response in industrial CPS. We propose an autonomous agent capable of optimizing multiple cyber incident indicators and dynamically adjusting detection thresholds based on real-time threat assessments in industrial CPS environments. To address the challenges posed by large and complex state spaces, we adopt a hierarchical reinforcement learning (HRL) framework, which decomposes the adaptive thresholding problem into more tractable sub-tasks. Specifically, we explore and compare both value-based (HDQN) and policy-based (Option-Critic) HRL approaches to highlight the fundamental differences between explicit and implicit hierarchical control. Rather than exhaustively testing every HRL variant—an approach that would be computationally intensive and yield limited additional insight—we focus on evaluating representative architectures that illustrate the key distinctions in learning dynamics and performance.