3.7 KiB
3.7 KiB
collapsed:: true type:: REVIEWS tags:: year:: 2025 venue:: TSE full-title:: Automating UML Class Diagram Generation from Natural Language via Transformer-Based Structured Translation date-start:: 18-09-2025 - 14:13 date-submitted:: external-links:: status:: DONE deadline-submission:: file:: parent:: todoist:: https://app.todoist.com/app/task/tse-2025-06-0540-now-in-your-reviewer-center-6cRhXHmM2jJw8hjg
- ### [[Comments]]
- Summary: This paper presents an approach to generate UML class diagrams expressed in PlantUML starting from software requirements given in natural language. The approach consists of two main steps. A first phase generate a model conforming to a defined schema by means of a Sequence-to-AST translation. A second step generates PlantUML models by relying on a learning phase leveraging both manually curated and automatically extracted datasets. The approach shows promising results, with the Seq2AST model outperforming baseline Seq2Seq approaches and zero-shot LLMs in terms of F1-score, syntactic correctness, and robustness against hallucinations.
- Comments: The problem is timely and the contributions are relevant to the software engineering and model-driven engineering. However, the paper requires a revision to solve presentation issues, unclear methodological details, and insufficient discussion of alternative approaches and evaluation metrics. In particular, my main concerns about the paper are related to the following issues:
- The rationale for introducing a custom AST grammar instead of directly using PlantUML's own grammar is not clear. PlantUML is already a stable and widely accepted language. The authors should clarify what limitations they encountered in PlantUML that justified designing a new grammar.
- The description of the AST representation needs to be improved. It’s unclear what kind of constraints are expressible in the grammar (e.g., are OCL constraints supported? Multiplicities? Invariants?). Also, the semantics referred to appear to be mainly static semantics.
- Several works already explore recommendation or generation of UML elements from context using ML or heuristics. A comparison with these is missing.
- The data augmentation step based on rephrasing is underspecified. How is quality controlled? Are rephrasings semantically equivalent to the originals? This is crucial for training reliable models from small supervised datasets.
- The connection between the two datasets (one derived from supervised student input, the other from ModelSet) is not sufficiently clarified. Are the model components trained separately? How is consistency ensured?
- Figure 1 should be expanded to include all the components and steps discussed in Section III, including the use of ASTs, multi-task training, and inference pipeline.
- Algorithm 1 should be contextualized with respect to the end-to-end process described in Figure 1. Their relationship is unclear.
- The evaluation uses Precision, Recall, F1, and BLEU-4. However, these metrics are insufficient to assess the semantic equivalence between diagrams. Different UML models can express the same requirement in multiple valid ways. Some structural similarity or model comparison metric (e.g., graph edit distance) should be considered.
- To summarize, the paper is about an interesting topic, even though a major revision is required to address the following main issues as discussed above:
- Clarify the motivation and role of the custom AST grammar.
- Better present the data augmentation, dataset preparation, and model training pipelines.
- Improve the figures and align them with the textual description.
- More meaningful metrics for model comparison need to be taken into account for the evaluation.