93 lines
8.5 KiB
Markdown
93 lines
8.5 KiB
Markdown
full-title:: Automated co-evolution of metamodels and code
|
||
external-links:: [ICSE 2023](https://icse2023.hotcrp.com/paper/1237)
|
||
file:: 
|
||
|
||
- ### Paper summary
|
||
- *Please briefly summarize the paper in your own words, explaining to the authors how you understood the paper.*
|
||
- The paper presents an approach to deal with the metamodel/code co-evolution problem. In particular, when a metamodel evolves, the code automatically generated from it will be automatically updated. However, the code previously added by developers to the generated code might be affected and must be adapted to fix potential errors.
|
||
- The proposed approach uses pattern-matching techniques to identify the relationships between the operated metamodel changes and the corresponding code-generation templates. Each match has a corresponding resolution to fix the affected code. The approach has been evaluated on real metamodel evolutions on nine Eclipse projects. The obtained results are encouraging.
|
||
- ### Strengths
|
||
- *Please provide a short bullet-point list of the paper’s key strengths.*
|
||
- - Interesting work about a relevant problem affecting the adoption of model-driven solutions.
|
||
- - Encouraging evaluation.
|
||
- ### Weaknesses
|
||
- *Please provide a short bullet-point list of the paper’s key weaknesses.*
|
||
- - The paper is not always easy to read.
|
||
- - The technical aspects of the proposed approach are not always presented at the right level of details.
|
||
- ### Comments for the authors
|
||
- **Significance:** The paper deals with an important problem in the context of Model Driven Software Engineering. The authors do a good job in motivating the work through a well-crafted example. However, I have some concerns about the paper that are mainly related to the soundness of the approach, as detailed below.
|
||
- **Soundness:**
|
||
- My main concerns are related to the applicability of the approach in general. In particular, first of all, the authors should clearly state earlier in the paper (even in the title) that the approach is technology dependent and, currently, it works with EMF. This is because the matching-based mechanism and the corresponding developed resolutions are dependent on the code generation templates that are applied to produce the implementation of the evolving metamodel.
|
||
- Concerning the evaluation, in RQ1, the authors mention that from the evolved metamodels, 837 errors were observed, among which 771 were direct and 66 indirect. To solve the direct errors, a total of 631 patterns were matched. However, it is not clear to what extent different distributions of the error types given in Table II can potentially change the outcome of the evaluation. In particular, I see that most of the occurred changes are CR2 and CR5, which in my opinion, are among the easiest to address. The former can be solved by potentially removing all the references to the removed type. In contrast, the latter can be fixed by renaming all the occurrences of the initial element in the added code. Of course, these can be considered "default" adaptations, but that might not work for all cases. Such criticalities also affect RQ2. In particular, the precision and recall figures that are given in Tab. VI should be expanded by discussing how they might change with different distributions of occurring metamodel change types.
|
||
- As a side comment, the discussion for the resolution of [CR2] given on page 7 is not clear. It seems that the occurring errors were solved by removing the whole code block, which, indeed, is not always the preferable solution.
|
||
-
|
||
- **Novelty:** The paper presents a novel approach with respect to how the metamodel/code co-evolution problem is currently addressed. However, no baselines have been considered to compare the performance of the proposed approach with existing techniques. For instance, as mentioned by the authors, the problem is very close to the program repair one and to the management of API evolutions. None of the most related approaches mentioned in the paper is considered as a baseline and compared with the proposed approach on common metamodel evolution cases.
|
||
- **Verifiability and Transparency:** The submission includes supplementary material, which is unclear how it is connected with the paper's content. I would have expected to see a HOWTO / README file explaining the content of the zip file, how it was generated and how to use it. The reader is supposed to unzip the package, navigate the content, and figure out what to do with it on her own.
|
||
- **Presentation:** Overall the quality of the writing is good and meets the presentation standards of ICSE.
|
||
- ### Question for authors
|
||
- Q1: Can you elaborate on to what extent the evaluation results can be affected by changing the distribution of the different change types?
|
||
- Q2: Can you clarify why none of the existing works on managing API evolutions has been considered as a baseline and compared with the proposed approach?
|
||
-
|
||
- Notes
|
||
- ((635685cc-d4fc-4f71-99c3-83b04784f2ff))
|
||
- The approach is based on pattern matching mechanisms. THey mach the error with the cause in changed metamodel and the pattern usages of the generated code related to the elements from the metamodels
|
||
- ((63414276-edf8-465c-bd50-3ae99d29e5eb))
|
||
#+BEGIN_IMPORTANT
|
||
This means that to make the approach working it is necessary to know in details the code generation templates!!! Otherwise it is not possible to employ the approach. Somehow the approach is *white-box*
|
||
Related to this aspect, the approach works on EMF
|
||
#+END_IMPORTANT
|
||
- ((635686e4-9376-4751-8d09-e116bf1c169d))
|
||
- This means that some errors are left and their resolution is deferred to the hands of the developers
|
||
- MAIN ASPECT!!!
|
||
- ((635693d3-4ea0-43dd-b210-429bdf9a7171))
|
||
- ((63569538-16f7-465a-9085-d9409a751bb6))
|
||
- These numbers have to be further checked later.
|
||
- ((63569785-8999-4c2e-b13c-44a54cbd3ff8))
|
||
- Please fix the figure, there are overlapping lines
|
||
- ((6341743a-790c-497b-a138-db70d4eb16ae))
|
||
- This seems to be an important part of the approach. It is done between metamodel changes and code usages for the detected errors to prepare the co-evolution.
|
||
-
|
||
- ((63417647-5264-40ba-8dfb-2f24fab49c6b))
|
||
- To be further explored. It is not clear what it means.
|
||
- ((634177f8-fe5e-4fab-b2a6-f69d4c98c9d0))
|
||
- It works because you know details of the code generator!!!! This is not always available / known.
|
||
- In fact this is what is written in the text:
|
||
- ((63417871-f8d2-4b1c-874c-a259a542e28d))
|
||
- ((634177bd-1ec5-4809-94a1-79bfec8142d9))
|
||
- Why *additional code* and not *generated code*?
|
||
- Additional code includes generated code!!!
|
||
- ((6356afa7-ce45-464a-8472-6a888a4ce811))
|
||
- What happens in case more than one pattern can apply?
|
||
- Herein, we distinguish direct errors and indirect errors.
|
||
- **Direct errors** do use a generated code. What does it mean?
|
||
- ((6356b0ac-17c7-4b2b-9dae-a0cc7b1a6116))
|
||
#+BEGIN_CAUTION
|
||
Errors that use generated code element... It does not parse. Similarly for Indirect errors, i.e. errors that do not use generated code elements. The "use" word is not clear here!!!!
|
||
#+END_CAUTION
|
||
- ((6356b0f9-57a4-49a0-bd39-97461aa6192f))
|
||
- So this is an error that does not have nothing to do with the metamodel change?
|
||
- ((6356b1df-6dbd-4a94-ab7e-d35ccb301918))
|
||
- -> MOVED
|
||
- ((6356b5df-4d73-4324-8c05-53955584aec0))
|
||
- It's not clear if this is a human in the loop approach. In this case how have you measured the time?
|
||
- ((6356b631-83a3-4025-87b2-4d7ae428fad4))
|
||
- I hope to see some details about humans involved in the evaluation for doing the manual adaptation.
|
||
- ((6356b68c-fa20-4c45-943d-cf05386d8414))
|
||
- The Evosuite tool needs to be double checked
|
||
- ((6356bef7-8937-48f0-a7fa-b2a0d60ec61f))
|
||
- About tests: this is still a bit obscure at this stage. Mentioned many times but not covered in details.
|
||
- ((6356bfa7-db55-4fc1-83a9-47a049ce4714))
|
||
- Is it a so astonishing result?
|
||
- ((6356c030-abb1-4cf9-aaf8-aae34d80803a))
|
||
- Not clear
|
||
- ((6356c19a-ae8e-431b-95ae-6bfc985a821a))
|
||
- It's not clear if at the end all the errors were covered by the 631 resolutions!!!!
|
||
- ((6356c2dd-d1dc-4103-a911-cd4fe7f787d8))
|
||
- This is interesting to me and it would have been necessary why developers removed code instead of migrating the affected one.
|
||
- ((6356c321-2c39-4811-84f2-36ebd5a957a1))
|
||
- Test generation should be covedered in a more clear manner. It is not clear how it works and consequently there are no ways to gudge it
|
||
- ((6356c5bd-c5a5-419b-ac00-6ba360f33a02))
|
||
- ???
|
||
-
|
||
-
|
||
- |