AI RESEARCH THESIS

Executive Impact Summary

This thesis addresses critical challenges in developing AI agents that are robust and adaptable in ever-changing, open-ended environments. It introduces novel frameworks and methodologies that enhance generalisation capabilities across diverse tasks and interactions.

Abstract

The growing prevalence of artificial intelligence (AI) in various applications underscores the need for agents that can successfully navigate and adapt to an ever-changing, open-ended world. A key challenge is ensuring these AI agents are robust, excelling not only in familiar settings observed during training but also effectively generalising to previously unseen and varied scenarios. In this thesis, we harness methodologies from open-endedness and multi-agent learning to train and evaluate robust AI agents capable of generalising to novel environments, out-of-distribution inputs, and interactions with other co-player agents.

We begin by introducing MiniHack, a sandbox framework for creating diverse environments through procedural content generation. Based on the game of NetHack, MiniHack enables the construction of new tasks for reinforcement learning (RL) agents with a focus on generalisation. We then present Maestro, a novel approach for generating adversarial curricula that progressively enhance the robustness and generality of RL agents in two-player zero-sum games. We further probe robustness in multi-agent domains, utilising quality-diversity methods to systematically identify vulnerabilities in state-of-the-art, pre-trained RL policies within the complex video game football domain, characterised by intertwined cooperative and competitive dynamics. Finally, we extend our exploration of robustness to the domain of large language models (LLMs). Here, our focus is on diagnosing and enhancing the robustness of LLMs against adversarial prompts, employing evolutionary search to generate a diverse range of effective inputs that aim to elicit undesirable outputs from an LLM.

This work collectively paves the way for future advancements in AI robustness, enabling the development of agents that not only adapt to an ever-evolving world but also thrive in the face of unforeseen challenges and interactions.

Broader Impact

The research presented in this thesis advances the field of AI by addressing fundamental challenges in developing robust agents capable of operating in open-ended environments. The insights and methodologies explored here—such as open-ended learning, curriculum learning, and adversarial robustness test ing—hold considerable potential for impact both within and beyond academia.

Within academia, this thesis contributes to the study of reinforcement learning (RL), multi-agent systems, and large language models (LLMs). It introduces new frameworks for evaluating and enhancing the robustness in AI, which rely on more dynamic, adaptable benchmarks than traditional static alternatives. These contributions can help shift the focus of RL research from narrow performance to broader, more generalisable capabilities. Future scholarship can build upon these tools to develop more adaptive agents capable of solving a wider variety of problems, leading to improvements in research methodologies across multiple AI subfields, including curriculum learning, environment design, and simulation-to-reality transfer.

Outside academia, the impact of this work extends to several domains. In industry, robust AI systems trained in open-ended environments can be deployed in areas such as autonomous vehicles and robotics, where the ability to generalise to unseen scenarios is crucial for safety. The advancements in LLM robustness can enhance the reliability of AI-powered tools across domains, from chatbots to healthcare, where AI is increasingly used for decision-making. Moreover, the techniques for adversarial prompt generation and robustness evaluation have already helped improve the safety of flagship LLMs in the industry against malicious attacks. Specifically, the Rainbow Teaming approach introduced in this thesis has been used to evaluate and enhance the safety of Llama 3 models, the most capable open-source LLM released by Meta that has already been integrated with tools such as Facebook, Whatsapp, and Messenger.

Your Robust AI Implementation Roadmap

A phased approach to integrating the principles of robust and open-ended AI into your enterprise, ensuring sustainable growth and adaptability.