AI Agent Governance Framework
Characterizing AI Agents for Alignment and Governance
Authored by Atoosa Kasirzadeh & Iason Gabriel. This paper provides a characterization of AI agents focusing on four dimensions: autonomy, efficacy, goal complexity, and generality. By mapping out key axes of variation and continuity, this framework provides developers, policymakers, and the public with the opportunity to develop governance approaches that better align with collective societal goals.
Executive Impact & Key Insights
The rapid evolution of AI agents demands a nuanced understanding of their capabilities and governance implications. Our analysis highlights critical aspects for enterprise readiness.
AI Agents by 2025 (Salesforce Target)
Core AI Agent Dimensions
AI Agent Definitions Analyzed
Threshold for Impact Visibility
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Understanding AI Autonomy Levels
Autonomy refers to an AI agent's capacity to perform actions without external direction or control. It encompasses the ability to independently determine and execute sequences of actions towards specific goals without requiring step-by-step human guidance. The paper draws inspiration from autonomous vehicle regulation (SAE levels) to propose a scale for AI autonomy, from A.0 (No autonomy) to A.5 (Full autonomy).
Higher levels of autonomy offer significant value by reducing human effort but also demand more rigorous safety verification and monitoring protocols. Full autonomy (A.5) is generally not a desirable goal, as it implies a complete loss of principal control.
Measuring AI Environmental Impact
Efficacy defines an AI agent's ability to interact with and causally impact its environment. This involves both the agent's control over outcomes within that environment and the consequential nature of the environment itself (e.g., impact on human well-being). Efficacy levels range from E.0 (Observation only) to E.5 (Comprehensive control over physical environments).
The framework distinguishes between simulated, mediated (via human intermediaries), and physical environments. A higher efficacy level, especially in physical environments, correlates with increased potential risk and necessitates more comprehensive safety measures and impact assessments.
Navigating AI Goal Structures
Goal complexity describes the intricacy of the objectives an AI agent can pursue, ranging from simple, single-aim tasks to dynamic, multi-objective plans requiring hierarchical decomposition and adaptive strategies. It's measured by factors like plan length, multi-objectivity, and constraints.
More capable AI agents can handle higher levels of goal complexity, from GC.1 (Minimal goal complexity) to GC.5 (Unbounded goal complexity). As complexity increases, so does the challenge of alignment verification, often requiring advanced technical tools and more intensive human oversight.
Assessing AI Task Breadth
Generality refers to an AI agent's ability to operate effectively across different roles, contexts, and cognitive tasks. It progresses from highly specialized agents focused on specific tasks to general-purpose systems capable of shifting between diverse domains. Levels range from G.0 (Null value) to G.5 (Fully general AI system).
Discussions around Artificial General Intelligence (AGI) and general-purpose technologies highlight generality as a critical dimension. Highly general agents may generate systemic effects across multiple sectors, posing unique governance challenges related to risk propagation and economic impact.
A New Lens for AI Governance
This framework provides a structured approach to understand, categorize, and govern diverse AI agents, from narrow task-specific assistants to highly autonomous general-purpose systems, ensuring alignment with collective societal goals.
AlphaGo: Master of Go
AlphaGo is a specialized AI agent designed specifically to play the board game Go in a simulated environment. It demonstrates intermediate autonomy (A.3), able to execute game strategies without human oversight, though human supervision is needed for game initiation and technical issues. AlphaGo has a low level of environmental impact (E.1) as it operates solely in a contained simulated environment, only affecting game outcomes within its rules. Its goal complexity is lower (GC.2), pursuing a single unified goal (maximize win-probability) involving a complex sequence of action. Finally, AlphaGo has low generality (G.1), operating exclusively within the single domain of the Go game.
ChatGPT-3.5: Conversational AI
ChatGPT-3.5 (stand-alone chat completion) is a chat-based language model for dialogue and code generation. It exhibits partial autonomy (A.2), independently generating responses but requiring human input for task initiation and output verification. Its intermediate efficacy (E.2) influences human thinking and decision-making in a mediated environment, without direct physical world impact. ChatGPT-3.5 has moderate goal complexity (GC.3), responding to complicated requests but lacking active reasoning for complex subtasks. It demonstrates a relatively high generality level (G.3), capable of groups of tasks like information retrieval, advice, and translation across different domains.
Claude 3.5 Sonnet: Enhanced Agent with Tool Use
Claude 3.5 Sonnet with tool use is a language model capable of more advanced tasks, leveraging tools like browser control and reasoning protocols. It has intermediate autonomy (A.3), executing extended action sequences without direct human supervision. Its efficacy is higher (E.3), acting via a mediated environment but with capacity to shape the world through computer operations (e.g., API calls). With stronger reasoning, Sonnet 3.5 demonstrates higher goal complexity (GC.4), coordinating multiple objectives and executing longer action sequences. It maintains high generality (G.3), effectively working across multiple domains using language understanding, computer operation, and analytical capabilities.
Waymo: Autonomous Driving
Waymo is a self-driving car system using sensors and computing to navigate roads without human operation. It boasts very significant autonomy (A.4), making real-time decisions in complex real-world environments while maintaining remote monitoring. Waymo has a high degree of efficacy (E.4), directly interacting with the physical world, creating persistent state changes with limited reversibility and real consequences. It exhibits high goal complexity (GC.4), pursuing safe transportation while managing subgoals like route planning and obstacle avoidance, adapting to real-time changes. Waymo has a medium generality level (G.2), mastering the bundle of tasks required for driving and navigation.
Agentic Mapping for Four Kinds of AI Agents
| Autonomy | Efficacy | Goal Complexity | Generality | |
|---|---|---|---|---|
| AlphaGo | A.3 | E.1 | GC.2 | G.1 |
| ChatGPT-3.5 | A.2 | E.2 | GC.3 | G.3 |
| Claude 3.5 Sonnet (with tool use) | A.3 | E.3 | GC.4 | G.3 |
| Waymo | A.4 | E.4 | GC.4 | G.2 |
Calculate Your Potential AI Impact
Estimate the time savings and cost efficiencies your organization could achieve by strategically integrating AI agents.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your AI Agent Governance Roadmap
Implementing robust AI agent governance is a multi-phase journey. Here’s a typical timeline:
01. Framework Development & Understanding
Establish a foundational understanding of AI agent properties (autonomy, efficacy, goal complexity, generality) and their implications. Review existing definitions and taxonomies to build an internal classification system for your AI systems.
02. Agentic Profiling & Risk Assessment
Characterize your current and prospective AI agents using the established framework. Map out their agentic profiles to identify potential risks and opportunities, informing the selection of appropriate governance mechanisms.
03. Governance Integration & Policy Design
Integrate risk-proportionate and domain-specific regulatory measures. Develop internal policies, monitoring protocols, and authentication systems tailored to the unique properties of different AI agents within your enterprise context.
04. Continuous Assessment & Refinement
Implement continuous monitoring and evaluation of AI agent capabilities and their evolving agentic profiles. Establish feedback loops to refine governance strategies, ensuring they remain effective as AI technology advances and deployment contexts change.
Ready to Characterize Your AI Agents?
Leverage our expertise to build a robust framework for understanding and governing your AI agent deployments. Book a consultation today.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat