280 lines
11 KiB
Markdown
280 lines
11 KiB
Markdown
file:: [icse2023-paper685_1666614263567_0.pdf](../assets/icse2023-paper685_1666614263567_0.pdf)
|
||
file-path:: ../assets/icse2023-paper685_1666614263567_0.pdf
|
||
|
||
- evaluations, however, do not investigate how developers perceive the use of RRs and what are the benefits of using RRs
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 635c0dc4-0d1f-4685-a72b-79d47e9f591e
|
||
- s. In this work, we report a controlled experiment carried out with 24 developers where they repaired design models in eight different tasks, with RRs being provided in half of the tasks.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: yellow
|
||
id:: 635c0dda-2f1c-40e8-81c2-3a5a0e186183
|
||
- the provision of RRs leads to developers considering alternatives to fix the inconsistency that were not considered without RRs.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 635c106a-3a7e-4366-9e8a-8ae555b187e3
|
||
- epairing models, however, is an error-prone and laborious task requiring a considerable amount of time and effort from developers
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 635c10b9-ae03-4476-b849-a8d3ac4bc645
|
||
- maintain quality and reduce the time and effort required for the repair, mainly on UML model
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 635c10c6-36a0-45fa-b353-f65287cea138
|
||
- ser studies are also relevant to analyze the implications of applying an RRs strategy from the perspective of its end-users, i.e., developer
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 635c10dd-be0c-4068-8a9b-c3cb50a94662
|
||
- In this paper, we report an experiment to investigate if RRs bring benefits to developers in terms of effectiveness and efficiency when repairing design models
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 635c1672-e5d1-4c47-8a8c-ca1860206644
|
||
- The goal of our study is described by three research questions (RQ): RQ1. Do developers benefit from recommendations when repairing inconsistent design models? RQ2. How do developers perceive the use of repair recommendations when repairing inconsistent design models? RQ3. Do developers have preferred recommendations when repairing inconsistent design models?
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 635c1695-1b9d-4d93-a931-9e5aa79d2e68
|
||
- o answer RQ1, we measure inconsistencies fixed (effectiveness) and the completion time required (efficiency). For RQ2, we asked developers to give us feedback regarding the difficulty and their confidence when repairing the models. To obtain the results of RQ3, we ask the developers to select and rank RRs and create new ones.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 635c16c1-bed9-4507-8ede-d0456ba202ee
|
||
- A RR identifies the operator, the model element, the model element property, and, optionally, a value to change the model element propert
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: purple
|
||
id:: 635c1872-ea83-4b71-b049-f8ce5c14c01f
|
||
- A. Reder and A. Egyed, “Computing repair trees for resolving inconsistencies in design models.” in ASE, M. Goedicke, T. Menzies, and M. Saeki, Eds. ACM, 2012, pp. 220–229. [Online]. Available: http://doi.acm.org/10.1145/2351676.2351707
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 635c18ed-54b0-4bb3-ac37-1f7bb73abe30
|
||
- archezan, R. Kretschmer, W. K. Assunc ̧ ̃ao, A. Reder, and A. Egyed,“Generating repairs for inconsistent models,” Software and Systems Modeling, pp. 1–33, 2022.
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 635c18fb-44dc-4aaf-aa3d-b34f11ca5184
|
||
- R. Kretschmer, D. E. Khelladi, R. E. Lopez-Herrejon, and A. Egyed,“Consistent change propagation within models,” Software and Systems Modeling, pp. 1–17, 2020.
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 635c1903-1df7-42ff-bdde-0be3182f28d0
|
||
- Furthermore, the documentation provided by them was used to implement the approach using their algorithm logic and repair generator functions. Details about how RRs are generated are described in the following.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 635c1947-b18e-42c3-abc2-f255adaa38eb
|
||
- Consistency Rules (CR
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: purple
|
||
id:: 635c1963-b3f4-4a10-9dc9-bddfc819eca5
|
||
- epair Recommendation
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: purple
|
||
id:: 635c1968-3888-4d4b-984f-d7ea09b99e63
|
||
- setCookingMode from CookingMod
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 635c1995-a04b-47fa-af08-ea7f1cdcead5
|
||
- SlowCookingMode to CookingMode, and the operation setCookingMode from SlowCookingMode.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 635c19a2-fa93-4ae0-a076-3462ea5fa1a7
|
||
- These three model elements are the cause of the inconsistency and should be modified to repair the model
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 635c19ca-cac7-40a2-b28e-fb7c22b83427
|
||
- are abstract since they suggest modifying the signature without saying how
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 635c1d73-f97e-49b9-bc27-2df45e6d672f
|
||
- In these cases, a developer is required to define a value for the RRs
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: purple
|
||
id:: 635c1d7e-2f3e-4269-9275-84a320fa61e6
|
||
- randomly ordered
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 635c1d90-eafe-422a-b941-e67a50b8b7bf
|
||
- To the best of our knowledge, however, the use of repair recommendations has never been evaluated from the developers’ perspective
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: yellow
|
||
id:: 635c1e65-6cd9-4980-8840-c8f2228f6196
|
||
- we designed our experiment with the goal to collect and analyze these implications
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: yellow
|
||
id:: 635c1e75-dc61-4a8e-b14d-b7623255328a
|
||
- effectiveness and efficiency
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 635c1f70-a208-45c5-b020-09169345fa7b
|
||
- Fallessi et al. [32], for instance, proposes the R3 scheme (Real, Relevant, and Recen
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 635c1fa5-c090-4d7e-9833-fa5323736fe9
|
||
- mpirical software engineering experts on the use of students and professionals in experimen
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 635c1fb1-3117-49c0-91b5-1cca2bafbef1
|
||
- This scheme is important as the experience of participants may impact the results of the experiment.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 635c1fdd-7307-4ffb-923c-0d760f93839c
|
||
- The main goal of our experiment is to analyze if developers benefit from the provision of RRs and how developers perceive the difference between using RRs compared to not using them. We also aim at investigating if developers have preferred RRs when fixing inconsistent models.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: yellow
|
||
id:: 635c1fff-cda0-492c-9dd6-24c4a0857d9f
|
||
hl-stamp:: 1666981891291
|
||
- The dependent variables are: T: Time spent on the tasks; I: number of inconsistencies per task; IF: number of inconsistencies fixed per task.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 635c20f4-3fae-4e26-bfad-6e9dcbc01120
|
||
- RC: repairs created by participants in WR tasks
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 635c2116-d3d4-4f01-8542-aeb8a05b5816
|
||
- 60
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: yellow
|
||
id:: 635c213f-585f-465f-b19b-bb2f60a862c0
|
||
- For the EFT metric, the null hypothesis H0 states that RR and WR have the same effectiveness.
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 635c21a4-7325-4c00-8f83-2fa820261048
|
||
- effectiveness (EFT)
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 635c21bc-c925-42c9-820b-0252d0cf5f98
|
||
- efficiency (EFY)
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 635c21c0-5cd7-462f-b477-bee8e77ae9f5
|
||
- We analyze these results to understand if when RRs are provided, participants perceive tasks to be less or more difficult while having less or more confidence in performing them.
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: yellow
|
||
id:: 635c21f0-a899-4d57-ba56-50412ee9ff65
|
||
- RQ3. Do developers have preferred recommendations when repairing inconsistent design models?
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: yellow
|
||
id:: 635c55a9-61e4-4da2-8312-6f645abb8be4
|
||
- one ideal RR for each inconsistenc
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 635c55e6-af53-4e24-ad65-fe504e39c393
|
||
- R3 scheme (Real, Relevant, and Recent
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 635c564f-5205-49bf-86b1-2d2ffd16bcd8
|
||
hl-stamp:: 1666995796072
|
||
- The experimental package used was designed following a strategy closely related to the strategy for mitigating threats to validity (see Section III-C)
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: yellow
|
||
id:: 635c578f-3138-4f2f-bae3-4f96bfa3925b
|
||
- ,
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: red
|
||
id:: 635c5930-942d-47cd-81bd-2d6d1a55068b
|
||
- When designing the tasks, we had to consider the diagrams to be used and their inconsistencie
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 635c5977-ab20-49fd-9953-7863e2a51328
|
||
- Rs were generated for all tasks
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: yellow
|
||
id:: 635c5fb6-5e69-464b-acb8-1e4727d80780
|
||
- hree experimenters were in the room observing if participants were following the guidelines explained.
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 635c60be-5aea-4388-9c52-90b140fcba56
|
||
- he order and the type of the tasks changed for each document
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 635c60e8-b74a-40fc-8a06-363801125fa1
|
||
- internal, external, and conclusion
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 635c6100-2174-4480-b9ec-bd76c7be7c1d
|
||
- Construct Validity
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 635c6147-a614-451c-8370-cee335e307d5
|
||
- the models from the dataset were already evaluated and used in other studies
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: yellow
|
||
id:: 635c6265-d1a3-49ad-a14e-cc3a9b56490f
|
||
- task difficulty and completion time
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 635c62b1-e7ea-4d50-8fee-8772afcdd2a6
|
||
- articipants from the pilot studies also reported that most tasks were easy, so we re-balanced these tasks, increasing their difficulty
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 635c62de-c5e6-43ea-bfbc-e5e3da4fbc6a
|
||
- were part of the generated repairs from the same RR task
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 635c631a-3832-4ad0-89db-8463462bbd5d
|
||
- we argue that even inexperienced developers should be able to use RRs
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: purple
|
||
id:: 635c636e-caf6-4e55-a574-d1d37fe673e7
|
||
- RQ1. Do developers benefit from recommendations when repairing inconsistent design models?
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: yellow
|
||
id:: 635c7160-423e-43bd-8b64-9c0e7652798c
|
||
- feel that using RRs is better than not using them.
|
||
ls-type:: annotation
|
||
hl-page:: 9
|
||
hl-color:: yellow
|
||
id:: 635c7497-8669-46ad-8fed-d78614d0893c
|
||
- This improvement is more evident when the repairs required are complex.
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: yellow
|
||
id:: 635c75f8-1804-44b9-b6aa-7093a3d523ae |