Evaluating Large Language Models in Exercises of UML Use Case Diagrams Modeling

In recent years, Large Language Models (LLMs) have been extensively used in several Software Engineering tasks, from requirements analysis to coding and software testing. Research has proved that LLMs can effectively generate software models to assist in software documentation. The goal of this study is to assess the capability of LLM agents to generate UML Use Case Diagrams (UCD), starting from software requirements in natural language. We perform the assessment in an educational setting, i.e., we evaluate the capability to solve software modeling exercises tailored for master’s students in SE curricula. Our results, based on the comparison of the results obtained by a human and an LLM solver on 17 UCD modeling exercises, show that LLMs have comparable results in terms of completeness and redundancy of the generated diagrams, with no significant difference if compared to human-proposed solutions.