Enterprise AI Deep Dive: Harnessing Generative Models for Requirement Engineering

A comprehensive study by Summra Saleem and her colleagues provides critical, data-driven insights for enterprises considering Generative AI (like ChatGPT and Gemini) for software development lifecycle (SDLC) automation. The research systematically evaluates these large language models (LLMs) against specialized AI solutions across four key Requirement Engineering (RE) tasks: extraction, classification, named entity recognition (NER), and question answering (QA).

The core takeaway for business leaders is clear: while Generative AI is not a silver bullet, it offers a game-changing advantage in specific areas. The study reveals that for complex, nuanced tasks like requirement classification and NER, custom-trained, specialized AI models still hold a significant performance edge. However, for Question Answering on technical documentation, ChatGPT not only competes but achieves state-of-the-art results.

This presents a strategic opportunity: enterprises can achieve immediate ROI by deploying LLMs to create intelligent, conversational interfaces for their knowledge bases, while adopting a more measured, hybrid approach for other RE tasks. This paper underscores the importance of strategic implementation over blanket adoption, highlighting that the greatest value lies in custom solutions that leverage the right AI tool for the right job. At OwnYourAI.com, we specialize in designing these bespoke systems that blend the power of Generative AI with the precision of specialized models.

Prompt Maturity Levels in the Enterprise

The study's three prompt levels map directly to how a business can evolve its use of generative AI.

• Level 1 (Basic): Simple, direct questions. Example: "Is this a functional requirement?" This yields basic results but often lacks context, similar to a new user's first interaction.
• Level 2 (Informed): Includes definitions and context. Example: "Classify the following as a functional or non-functional requirement. A functional requirement specifies what the system should do..." This significantly improves accuracy by providing the AI with guardrails.
• Level 3 (Expert): Provides definitions and concrete examples (few-shot prompting). Example: "...For instance, 'The system shall process payments' is functional." This is the most effective method, especially for Gemini, turning the LLM into a more specialized tool.

Enterprise Insight: The study confirms that higher-quality inputs yield higher-quality outputs. Gemini's strong dependence on expert prompts suggests it has a powerful reasoning engine that thrives on context, while ChatGPT is more robust with less guidance. A successful enterprise strategy requires building a library of expert-level prompts and potentially fine-tuning models to bake this domain knowledge in directly.