Enterprise AI Adoption
The Proof Is in the Eating: Lessons Learnt from One Year of Generative AI Adoption in a Science-for-Policy Organisation
This paper presents the key results of a large-scale empirical study on the adoption of Generative AI (GenAI) by the Joint Research Centre (JRC), the European Commission's science-for-policy department. Since spring 2023, the JRC has developed and deployed GPT@JRC, a platform providing safe and compliant access to state-of-the-art Large Language Models for over 10,000 knowledge workers. While the literature highlighting the potential of GenAI to enhance productivity for knowledge-intensive tasks is abundant, there is a scarcity of empirical evidence on impactful use case types and success factors. This study addresses this gap and proposes the JRC GenAI Compass conceptual framework based on the lessons learnt from the JRC's GenAI adoption journey. It includes the concept of AI-IQ, which reflects the complexity of a given GenAI system. This paper thus draws on a case study of enterprise-scale AI implementation in European public institutions to provide approaches to harness GenAI's potential while mitigating the risks.
Many organizations struggle with the uncertainty surrounding Generative AI adoption, lacking clear empirical evidence on impactful use case types and success factors within knowledge-intensive tasks. There's a critical need for practical approaches to harness GenAI's potential while effectively mitigating its inherent risks, especially in public sector innovation.
Executive Impact: Key Adoption Metrics
Tracking the tangible impact of GenAI adoption within the JRC reveals significant engagement and growth, setting a precedent for public sector innovation.
10,000
Knowledge Workers with GenAI Access
12,314
Total Users Registered (Nov 2024)
134
API Projects Activated
78.6%
Level 2 Use Cases: Text/Data Analysis & Interpretation
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
AI-IQ Framework
The AI-IQ framework is a qualitative measure of a GenAI system's capabilities based on its functional architecture. It serves as a relative scale to compare different GenAI systems and evaluate the complexity of solutions needed for specific use cases. It encompasses not only the LLM(s) but also internal components like Retrieval-Augmented Generation (RAG) or agentic loops.
Level 0: Non-conversational - Limited to answering predefined questions, like traditional chatbots. Configuration: Basic semantic search system.
Level 1: Conversational - Engaging in open-ended conversations and generating text on demand. Accuracy limited by outdated knowledge and lack of validation. Configuration: LLM with a custom system prompt and contextual information.
Level 2: Basic RAG - Providing answers based on specific, proprietary knowledge, complementing general LLM knowledge. Configuration: GenAI system with basic Retrieval-Augmented Generation (RAG).
Level 3: Advanced RAG - Accessing, interpreting, and synthesising knowledge from organisation's proprietary sources in a targeted and optimal way, tailored to specific use cases. Configuration: Multiple knowledge bases, tailored RAG modalities, user chooses knowledge tool(s).
Level 4: Basic Agentic - Proactively leveraging specialised data sources and tools, automatically selecting relevant ones from a predefined set. Configuration: Predefined set of tools available; agentic system component interprets requests and uses tools (e.g., API requests, Python code).
Level 5: Full Agentic - Advanced agentic features with autonomy, iteratively utilising tools, adapting to circumstances, and pursuing multiple lines of inquiry. Configuration: Agentic components for breaking down tasks, exploring avenues, identifying optimal solutions, and planning execution steps.
Level 6: Multi-Agentic Systems - Swarm of agents working collaboratively to meet complex objectives, requiring coordination, negotiation, and collective problem-solving. Configuration: Decentralised architecture with multiple autonomous agents interacting and leveraging shared tools/data.
JRC GenAI Compass
The JRC GenAI Compass is a bi-dimensional conceptual framework focusing on the human (AI literacy) and technological (AI-IQ) dimensions of GenAI adoption. It helps organizations gauge their direction in GenAI implementation, facilitating strategic decision-making and balancing system complexity with user expertise. It highlights the importance of user expertise to mitigate risks and maximize value.
Task: Auditability of output quality/accuracy, Fit with core organizational tasks, Ethical/regulatory concerns
People: Required user expertise with GenAI, Digital literacy for quick GenAI expertise acquisition, Level of trust for user acceptance
Structure: Potential value to organization, Potential associated risks (reputational, financial) from poor quality
Technology: GenAI solution AI-IQ score, Organizational resources (financial, skills, time) for implementation and evaluation
Resource Value Pyramid
The JRC's GenAI Resource Value Pyramid maps GenAI use cases across three levels of complexity: Level 1 (general-purpose chatbot-like applications), Level 2 (process automation via API integration), and Level 3 (new, specialized GenAI tools). Each level requires different technical complexity, team AI maturity, and resource investment, yielding varying levels of specificity, output reliability, and fitness for purpose.
Level 1: General-purpose GenAI web application ('chatbot-like') - Lowest complexity, accessed via a custom user interface, allowing safe exploration of basic LLM capabilities. Features like file upload and prompt templates can personalize use. Relies on LLMs processing context window input. Examples: Text enhancement (proofreading, summarising, translating), Programming assistance (writing/debugging code, generating unit tests), Data analysis/interpretation (extracting info from documents), Literature review (summarising scientific literature), Learning (answering questions, explaining concepts), Project/process management (automating tasks, planning assistance), Creative assistance (generating ideas, role-playing, crafting scenarios)
Level 2: Process automation and integration into pre-existing tools via API - Higher complexity, leveraging LLMs via API for process automation or integration with existing tools. Requires technical expertise. Adds value by combining LLM capabilities with project-specific business processes, making processes faster, cheaper, more accurate. Examples: Sending batches of requests to GPT models (summarising news, analyzing consultations), Integrating GPT functionality within an IT system (conversational chatbots for finance, automated text classification), Experimenting (testing API integration, investigating cybersecurity potential)
Level 3: New, specialised GenAI tools that are optimised and evaluated on specific use cases - Highest complexity, involving development of specialized GenAI systems, carefully engineered and rigorously evaluated for complex tasks. Requires high technical expertise and domain-specific knowledge. Aims for high value and impact, but is resource-intensive and requires long-term commitment. Examples: JRC virtual scientific assistant (systematic literature reviews, generating research digests), GenAI system for complex data analysis (e.g., comparing GDP growth across EU regions), Fine-tuned systems for specific domains with high accuracy needs (e.g., legal or medical domains)
12,000
Users granted access by Nov 2024, showing rapid GenAI adoption across EU institutions.
Enterprise Process Flow
Prototype Launch (GPT@JRC)
→
Limited Staff Access (500 users)
→
Pilot Phase Expansion (EU DGs & Institutions)
→
Rapid User Growth (>12,000 users)
→
API Access for Project Teams (134 projects)
→
Ongoing Evaluation & Learning
| Feature | Level 1 (General Purpose) | Level 3 (Specialized) |
|---|---|---|
| Technical Complexity | Low | High |
| User Expertise Required | Low (exploration) | High (domain-specific, AI evaluation skills) |
| Resource Investment | Low | High (long-term, significant engineering) |
| Output Specificity/Reliability | General, potential for hallucinations | High, tailored, rigorously evaluated |
| Examples |
|
|
Level 3 Use Case: Scientific Literature Review Assistant
One high-potential Level 3 use case identified is a JRC virtual scientific assistant specializing in systematic scientific literature reviews. This involves defining objective criteria for selecting and prioritizing scientific publications, and ensuring the LLM handles complex, domain-specific terminology accurately. Such a system aims to drastically reduce the work-intensive and time-consuming tasks currently performed by scientists, requiring careful engineering and rigorous evaluation to maintain high accuracy and relevance.
Calculate Your Enterprise AI Efficiency Gains
Estimate the potential annual savings and reclaimed hours by adopting GenAI in your organization, based on industry-specific efficiency multipliers.
Estimated Annual Cost Savings
$0
Estimated Annual Hours Reclaimed
0
Your Enterprise AI Adoption Roadmap
A structured approach to integrating Generative AI, from initial exploration to scalable, specialized applications, mitigating risks at every step.
Phase 1: Exploration & Prototyping (AI-IQ Level 1)
Initiate safe experimentation with general-purpose GenAI tools (e.g., GPT@JRC). Focus on basic text enhancement, coding assistance, and learning use cases. Establish a community of practice to foster mutual learning and gather initial feedback. This phase is about understanding basic LLM capabilities and user readiness.
Phase 2: API Integration & Process Automation (AI-IQ Level 2)
Leverage GenAI APIs to automate processes and integrate with existing IT systems. Target Level 2 use cases like batch text processing, information extraction, and conversational chatbots for specific domains. Requires technical expertise and a sound understanding of LLM strengths and limitations to combine GenAI with project-specific business processes for efficiency gains.
Phase 3: Specialized System Development & Evaluation (AI-IQ Level 3+)
Develop new, specialized GenAI tools tailored for complex, high-impact organizational tasks. This involves advanced AI techniques (e.g., RAG, agentic loops) and rigorous formal evaluation/benchmarking to ensure accuracy and reliability. Focus on resource-intensive, long-term projects requiring deep domain expertise and AI evaluation capabilities, aiming for high-value scientific or policy applications.
Phase 4: Scaling & Continuous Alignment
Scale successful GenAI solutions across the organization. Implement robust governance frameworks, ensure continuous alignment with ethical principles and regulatory requirements (e.g., AI Act). Foster ongoing research into human-AI collaboration and adapt to the fast-evolving GenAI landscape, ensuring sustained value and mitigating emerging risks.
Ready to Transform Your Enterprise with AI?
Our team of AI experts is ready to help your organization navigate the complexities of Generative AI adoption. From strategy to implementation, let's build a future where AI empowers your workforce and drives innovation securely and effectively.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat