OwnYourAI ENTERPRISE AI ANALYSIS
The Algorithmic Gaze of Image Quality Assessment: An Audit and Trace Ethnography of the LAION-Aesthetics Predictor
Authors: Jordan Taylor, William Agnew, Maarten Sap, Sarah E. Fox, Haiyi Zhu
Visual generative AI models are trained using a one-size-fits-all measure of aesthetic appeal. However, what is deemed "aesthetic" is inextricably linked to personal taste and cultural values, raising the question of whose taste is represented in visual generative AI models. In this work, we study an aesthetic evaluation model—the LAION-Aesthetics Predictor (LAP)—that is widely used to curate datasets to train visual generative image models, like Stable Diffusion, and evaluate the quality of AI-generated images. To understand what LAP measures, we audited the model across three datasets. First, we examined the impact of aesthetic filtering on the LAION-Aesthetics Dataset (~1.2B images), which was curated from LAION-5B using LAP. We find that the LAP disproportionally filters in images with captions mentioning women, while filtering out images with captions mentioning men or LGBTQ+ people. Then, we used LAP to score ~330k images across two art datasets, finding the model rates realistic images of landscapes, cityscapes, and portraits from western and Japanese artists most highly. In doing so, the algorithmic gaze of this aesthetic evaluation model reinforces the imperial and male gazes found within western art history. In order to understand where these biases may have originated, we performed a trace ethnography of public materials related to the creation of LAP. We find that the development of LAP reflects the biases we found in our audits, such as the aesthetic scores used to train LAP primarily coming from English-speaking photographers and western AI-enthusiasts. In response, we discuss how aesthetic evaluation can perpetuate representational harms and call on AI developers to shift away from prescriptive measures of "aesthetics" toward more pluralistic evaluation.
Executive Impact: Unlocking Value from AI Research
This analysis focuses on the LAION-Aesthetics Predictor (LAP), a critical AI model used for curating datasets and evaluating AI-generated images. Our audit reveals significant biases: LAP disproportionately favors images mentioning women, while filtering out those mentioning men or LGBTQ+ individuals. It also rates realistic landscapes, cityscapes, and portraits by Western and Japanese artists most highly, reinforcing historical male and imperial gazes in art. A trace ethnography indicates these biases stem from LAP's development, which relied on aesthetic scores primarily from English-speaking photographers and Western AI enthusiasts. This calls for a shift from prescriptive, universalist "aesthetics" to more pluralistic evaluation methods in AI development.
0
of top-rated WikiArt images are realistic/representational styles.
0
of LAP training data from AVA consists of photographs.
0
African, Oceanian, Native American, Islamic, Egyptian, or West Asian art rated 6.5+ by LAP.
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Algorithmic Bias Audit
A deep dive into how algorithmic biases manifest in image quality assessment models, specifically examining the LAION-Aesthetics Predictor (LAP) and its impact on dataset curation and AI-generated image evaluation. We reveal how LAP's 'algorithmic gaze' perpetuates historical biases such as the male gaze and imperial gaze, prioritizing certain aesthetics while filtering out others, particularly those related to marginalized communities or non-Western art forms.
97%
of 6+ rated MET art from 5 departments: European Paintings, Photographs, American Wing, Asian Art, Drawings & Prints.
| Identity Group | More Likely to Be Included (PMI > 0) | More Likely to Be Filtered Out (PMI < 0) |
|---|---|---|
| Gender |
|
|
| Religion |
|
|
| Race/Ethnicity |
|
|
The audit revealed LAP's strong preference for realistic images of landscapes, cityscapes, and portraits, particularly those by Western and Japanese artists. This bias, referred to as the 'realist gaze', risks devaluing modern and non-Western artistic cultures, mirroring historical precedents where modern art was mocked. Such aesthetic models may also misalign with artists who enjoy the glitches and surrealism of AI-generated images.
Trace Ethnography & Development
An investigation into the origins of the LAION-Aesthetics Predictor's biases through a trace ethnography of its development. We explore the model's creation process, training data sources (AVA, SAC, LAION-Logos), and the influence of individual taste in its design. This section highlights how the model’s preferences are rooted in data primarily from English-speaking photographers and Western AI enthusiasts, leading to a 'universalist' conception of aesthetics that overlooks diverse cultural values.
LAION-Aesthetics Predictor Development Flow
LAION Founder Creates LAP
→
Trains LAP with AVA, SAC, LAION-Logos
→
Prioritizes Individual Taste
→
Results in Biased Aesthetic Evaluation
| Dataset | Year Released | Images from | Scores from | Consent (Images) | Consent (Ratings) | What it Measures |
|---|---|---|---|---|---|---|
| AVA | 2012 | dpchallenge.com | dpchallenge.com | No | No | Relative rating on a theme |
| SAC | 2022 | T2I generated | AI enthusiasts | Yes | Yes | Absolute rating |
| LAION-Logos | 2022 | LAION-5B | LAION volunteers | No | Yes | Absolute rating |
The "Lena" Image Legacy in Computer Vision
The study draws a parallel between LAP's male gaze and the historical use of the "Lena" image in computer vision. For decades, the Playboy model Lena Forsén's digitized image circulated without her consent to test image compression techniques, reinforcing misogyny. Similarly, LAP's preference for images mentioning women (often without explicit consent) for "high-quality" ratings risks perpetuating representational harms and sexual abuse imagery. This highlights the dangers of de-contextualized aesthetic evaluation in perpetuating harmful biases.
Recommendations for Pluralistic AI
Proposing a shift towards more pluralistic and descriptive aesthetic evaluation in AI. Instead of universalist 'aesthetics,' we advocate for models that articulate specific aesthetic values, such as photorealism, without prescribing them as ideal. This approach encourages transparency, allows artists to choose models aligned with their tastes, and helps mitigate representational harms. We also suggest combining audit and trace ethnography methods to understand both the 'what' and 'why' of algorithmic biases.
The paper argues for a shift from prescriptive, universalist 'aesthetics' towards descriptive, pluralistic evaluations. Instead of a single numeric score, models should articulate specific aesthetic values they prioritize (e.g., photorealism). This approach fosters transparency, allows artists to select models that align with their creative goals, and can help mitigate biases inherent in current AI image generation. The AVA dataset, with its grouping of photography challenges into styles, is cited as an early example of this more descriptive approach.
12yrs
time gap between AVA (2012) and T2I popularization (early 2020s), highlighting outdated training data.
Shifting to Pluralistic Evaluation
Recognize Subjectivity of Aesthetics
→
Move Beyond Universal Scores
→
Adopt Descriptive Metrics (e.g., Photorealism)
→
Enable Pluralistic AI Alignment
Advanced ROI Calculator
Estimate the potential return on investment for adopting a more ethically aligned and pluralistic AI image generation system in your enterprise operations.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your Enterprise AI Implementation Timeline
A phased approach to integrating responsible and pluralistic AI into your image generation workflows.
Phase 1: Bias Audit & Assessment
Conduct a comprehensive audit of existing and proposed AI image generation models and datasets for aesthetic, representational, and cultural biases. Establish a baseline for ethical alignment.
Phase 2: Pluralistic Model Design
Develop or adapt AI models that move beyond universal aesthetic scores towards descriptive evaluations, allowing for diverse artistic and cultural expressions. Focus on transparency in model values.
Phase 3: Diverse Data Curation
Implement strategies for curating training datasets that are culturally rich, ethically sourced, and inclusive of varied art forms and identity groups, minimizing the perpetuation of historical biases.
Phase 4: Stakeholder Engagement & Training
Train internal teams on ethical AI principles and the use of pluralistic evaluation tools. Engage artists and cultural experts to ensure model outputs resonate with diverse communities.
Phase 5: Continuous Monitoring & Iteration
Establish ongoing monitoring systems to detect emerging biases and ensure continuous alignment with evolving ethical standards and cultural sensitivities. Regularly update models and datasets.
Ready to Transform Your Enterprise?
Schedule a consultation with OwnYourAI to discuss how we can help your organization build and deploy ethically aligned, pluralistic AI systems.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat