type:: [[REVIEWS]]
tags:: 
year:: 2025
venue:: [[MODELS]] 
full-title::  LLMA-UML: Extending the UML for Modeling LLM-Agent Systems
date-start:: [[21-05-2025]] - 10:34
date-submitted::
external-links::
status:: [[DONE]] 
deadline-submission:: 
file:: [[@MODELS_2025_paper_3]] 
parent::
todoist:: https://app.todoist.com/app/task/llma-uml-extending-the-uml-for-modeling-llm-agent-systems-6Xh6hgr37HGMWJjg

	- ### [[Highlights]]
	- ### [[Comments]]
		- Summary of the paper: The paper presents a UML profile named LLMA-UML to support the modeling of LLM-based software agents. The profile consists of four main stereotypes (i.e., <<LLMAgent>>, <<Prompt>>, <<Tool>>, and <<Memory>>) and is evaluated through a case study in the software engineering domain.
		- Strenghts:
			- Relevant and interesting topic
			- Overall well written and easy to follow
		- Weaknesses:
			- Evaluation limited to a case study without clearly defined research questions
			- Weak claims regarding the practical usability and adoption of the proposed language
		- Detailed comments: The paper addresses a relevant and timely topic. The manuscript is well-written and generally easy to follow. However, I have some concerns related to the following issues:
			- The manuscript should clarify the objective of the modeling activity. What does it aim to support (e.g., design, analysis, documentation, communication)? Moreover, the case study presented in Section 7 is intended to "illustrate the usefulness" of the approach, but the research questions driving the case study are not stated. What are the authors aiming to investigate or demonstrate?
			- While the UML profile introduces stereotypes like <<LLMAgent>> and <<Prompt>>, the intended stakeholders (e.g., system architects, developers) and the development phases at which the profile should be applied (e.g., early design, deployment) are not clearly identified.
			- The <<Prompt>> stereotype is used to represent both input and output (e.g., user queries and agent responses). However, prompts are typically inputs to LLMs, not outputs. Consider introducing a separate stereotype to represent responses for improved clarity.
			- Figure 2 shows a clarification loop for a scheduling task. Why is such a loop necessary? Could the relevant information not be included in the initial prompt? This example does not effectively showcase the paper’s claims about runtime decisions or emergent behavior.
			- On page 4, the authors recommend using UML Activity Diagrams or Interaction Overview Diagrams to model reasoning steps. It would be helpful to explain why BPMN, which is widely used for service orchestration and workflow modeling, was not considered.
			- To illustrate the benefits of LLMA-UML, the challenges it addresses must be more clearly stated. What specific difficulties does the profile aim to resolve? Given the emergent behavior, reasoning, and planning involved in LLM-based systems, what advantages does modeling bring in practice?
			- The shown case study includes multiple LLM-agents, but inter-agent coordination mechanisms are not addressed. What happens in case of conflicting actions or divergent outputs? Governance models (e.g., consensus, arbitration) should be at least mentioned.
			- The paper states it provides a "theoretical foundation" but does not sufficiently compare LLMA-UML with other modeling frameworks or approaches for multi-agent systems. Consider including a section discussing LLMA-UML with respect to frameworks such as recent LLM-specific architectures (e.g., LangChain, CrewAI). What specific modeling needs does LLMA-UML address with respect to them?
		- Minor comments:
			- Page 6: The paragraph summarizing the stereotypes (<<LLMAgent>>, <<Prompt>>, etc.) repeats earlier content and could be made more concise.
			- Page 7: Replace “chapter” with “section.”
			- Page 10: Reference [6] is incomplete and should include full bibliographic details.
		- Questions for the authors
			- Q1:Wwho are the intended users of this profile (e.g., software architects, system designers, ML engineers), and at which development stages do you envision the profile being most valuable?
			- Q2: You suggest using UML Activity Diagrams or Interaction Overview Diagrams to represent reasoning and control flow. Given that BPMN is commonly adopted for modeling service orchestration and choreographies, why was it not considered a more appropriate option in this context?
			- Q3: What's the expressiveness of the proposed language with respect to recent LLM-specific architectures (e.g., LangChain, CrewAI)? What specific modeling needs does LLMA-UML address?