JARVIS or Ultron? A Survey on the Safety and Security Threats of Computer-Using Agents

Extrinsic threats originate from external entities, such as malicious attackers. These include Adversarial Attacks (Wu et al., 2024a), which manipulate inputs to induce harmful behaviors, like tiny pixel perturbations. Prompt Injection Attacks (Mudryi et al., 2025) embed malicious instructions directly or indirectly (e.g., via webpages) to bypass safety rules. Jailbreak Attacks (Mo et al., 2024) rephrase queries to bypass guardrails and generate unauthorized outputs. Memory Attacks (Wang et al., 2025a) target persistent context to extract sensitive information (Memory Extraction) or poison future reasoning (Memory Injection). Backdoor Attacks (Yang et al., 2024b) insert hidden triggers during training to activate harmful behaviors later. Reasoning Gap Attacks (Chen et al., 2025d) exploit mismatches between multimodal perception and reasoning. System Sabotage Attacks (Luo et al., 2025b) trick agents into destructive operations, like creating fork bombs. Finally, Web Hacking Attacks (Fang et al., 2024b) co-opt CUAs into autonomous hacking tools for SQL injection or data exfiltration.

Defenses against CUA threats are categorized into several types. Environmental Constraints (Yang et al., 2024c) limit agent interactions to prevent harmful actions. Input Validation (Kumar et al., 2024) verifies and sanitizes user inputs. Defensive Prompting (Debenedetti et al., 2024) structures prompts to prevent manipulation. Data Sanitization (Yang et al., 2024b) removes malicious data from training sets. Adversarial Training (Wu et al., 2024a) enhances model robustness against perturbations. Output Monitoring (Fang et al., 2024a) continuously evaluates agent outputs for misalignment. Model Inspection (Wang et al., 2025e), including Anomaly Detection and Weight Analysis, identifies malicious manipulations. Cross Verification (Zeng et al., 2024) uses multiple agents to validate outputs. Continuous Learning and Adaptation (Tian et al., 2023), via Self-Evolution and User Feedback, allows agents to dynamically update models. Transparentize (Sager et al., 2025) enhances interpretability through XAI and Audit Logs. Topology-Guided Strategies (Wang et al., 2025e) improve multi-agent security. Perception Algorithms Synergy (Zheng et al., 2024) combines perception modules for robust UI understanding. Planning-Centric Architecture Refinement (Zhang and Zhang, 2023) improves reasoning and API invocation. Lastly, Pre-defined Regulatory Compliance (Chen et al., 2025e) integrates adherence to standards and ethical guidelines.

CUA safety is evaluated using diverse benchmarks and metrics. Datasets cover Web-based Scenarios like ST-WebAgentBench (Levy et al., 2024), Mobile-based Scenarios such as MobileSafetyBench (Lee et al., 2024a), and General-purpose Scenarios, including Tool-use (ToolEmu, Ruan et al., 2023) and Mixed/Hybrid Environments (OpenAgentSafety, Vijayvargiya et al., 2025). Metrics include Task Completion Rate (TSR) (Yao et al., 2022), Helpfulness (Ruan et al., 2023), Step Success Rate (SSR) (Deng et al., 2023), and Total Correct Prefix (Hua et al., 2024). Safety and robustness are measured by Attack Success Rate (ASR) (Zhan et al., 2024), Completion Under the Policy (CuP) (Levy et al., 2024), Refusal Rate (RR) (Zhang et al., 2024b), and Leakage Rate (LR) (Shao et al., 2024). Measurements use Rule-based checks (Luo et al., 2025a), LLM-as-a-judge (Yuan et al., 2024), and Manual Judge evaluations (Ruan et al., 2023).