Enterprise AI Analysis

Comprehensibility & Acceptability (RQ2, RQ3, RQ4)

Cognitive Load (RQ2): No statistically significant differences were found in intrinsic, extraneous, or germane cognitive load between BPMNGen-generated and expert-created models across varying complexity levels. This is a promising finding, indicating BPMNGen's outputs are equally interpretable and cognitively manageable.

Perceived Usefulness (RQ3): Expert-generated models were perceived as significantly more useful in simpler and moderately complex scenarios (PM1-PM4). However, for the most complex scenario (PM5), the difference in perceived usefulness was not significant, suggesting BPMNGen's usefulness improves relative to expert models as complexity increases.

Perceived Ease of Understandability (RQ3): Expert models were rated significantly easier to understand only for the simplest scenario (PM1); for all other models, no significant difference was observed. This suggests BPMNGen models are generally considered equally understandable.

Comprehension Performance (RQ4): BPMNGen-generated models significantly outperformed expert-created models in moderately complex scenarios (PM3 and PM4), demonstrating clear and structured representation. For the most complex scenario (PM5), expert models led to better comprehension.

The BPMNGen Framework in Action

BPMNGen is an LLM-based conversational framework that automates BPMN 2.0 process model generation from natural language descriptions. Users describe processes in natural language, which are then processed by a Custom Assistant (powered by GPT-4o via API).

The assistant converts user input into a JSON structure containing all necessary information for a BPMN 2.0 process model. This JSON is then converted into an XML document, which defines the BPMN 2.0 model, for frontend visualization and editing. This ensures traceability and allows for iterative refinement.

The framework supports both initial model creation and iterative modification, using structured JSON update messages for efficient, incremental changes rather than full regeneration. A dedicated XML validator ensures compliance with BPMN 2.0 specifications.

Limitations & Future Work

Current Limitations: The evaluation focused on relatively simple process scenarios, and most participants were university students, limiting generalizability to complex industry settings. The evolving nature of LLMs, dependence on specific LLM versions (GPT-3.5 Turbo used), and specific textual input scenarios also pose threats to reproducibility and generalizability. Additionally, the evaluation covered a constrained subset of BPMN 2.0 elements.

Future Opportunities: Future research will focus on expanding BPMNGen's modeling capabilities to include more advanced BPMN 2.0 elements (e.g., data objects, message flows, sub-processes) and choreography diagrams. Integrating Retrieval-Augmented Generation (RAG) will enhance semantic quality by incorporating external contextual information. Exploration of multimodal inputs (sketches, speech), process model modularization, and model-to-text approaches (translating models back to natural language) are also key areas for development.