Enterprise AI Analysis

Gaming the Arena: AI Model Evaluation and the Viral Capture of Attention

Innovation in artificial intelligence (AI) has always been dependent on technological infrastructures, from code repositories to computing hardware. Yet industry – rather than universities – has become increasingly influential in shaping Al innovation. As generative forms of Al powered by large language models (LLMs) have driven the breakout of Al into the wider world, the Al community has sought to develop new methods for independently evaluating the performance of Al models. How best, in other words, to compare the performance of Al models against other Al models – and how best to account for new models launched on nearly a daily basis? Building on recent work in media studies, STS, and computer science on benchmarking and the practices of Al evaluation, I examine the rise of so-called ‘arenas' in which Al models are evaluated with reference to gladiatorial-style 'battles'. Through a technography of a leading user-driven Al model evaluation platform, LMArena, I consider five themes central to the emerging ‘arena-ization' of Al innovation. Accordingly, I argue that the arena-ization is being powered by a 'viral' desire to capture attention both in, and outside of, the Al community, critical to the scaling and commercialization of Al products. In the discussion, I reflect on the implications of ‘arena gaming', a phenomenon through which model developers hope to capture attention.

Sam Hind, University of Manchester