13 KiB
13 KiB
type:: REVIEWS tags:: year:: 2024 venue:: JSS full-title:: Software Engineering Education: Results from a Training Intervention based on SonarCloud when Developing Web Apps date-start:: 17-07-2024 - 18:06 date-submitted:: external-links:: status:: DONE deadline-submission:: file:: @JSSOFTWARE-D-24-00573 parent:: todoist:: https://app.todoist.com/app/task/jssoftware-d-24-00573-6Vgxxjvvm4rhxCPc
- ### [[Highlights]]
-
- # Annotazioni
- (16/9/2024, 11:18:00)
- - “In our previous publications [3, 4], we assessed the quality of student-developed (ecommerce) web apps in order to understand whether Computer Science (CS ) undergraduate students are equipped to address three quality characteristics: security, reliability, and maintainability. Our results suggested that the students enrolled in the academic year (a.y.) 2021-22 of a” (“JSSOFTWARE-D-24-00573”, p. 2) #00b036
- - “training intervention” (“JSSOFTWARE-D-24-00573”, p. 2) #f0ff00
- *To be checked what they have done.*
- - “self-determination theory,” (“JSSOFTWARE-D-24-00573”, p. 2) #f0ff00
- *What's that?*
- - “reliability and maintainability” (“JSSOFTWARE-D-24-00573”, p. 2) #f0ff00
- *Why these two? How have you decided such characteristics?*
- - “processed and made available in a dashboard. These” (“JSSOFTWARE-D-24-00573”, p. 3) #00b036
- - “The results suggested that the SATs were useful to help junior programmers detect and fix bugs.” (“JSSOFTWARE-D-24-00573”, p. 4) #f0ff00
- *So, we had already such a study. What's new here?*
- - “Goal-Question-Metric (GQM )” (“JSSOFTWARE-D-24-00573”, p. 5) #00b036
- - “It is worth mentioning that the research presented in this paper, although focused on software engineering education, also has possible implications for researchers and practitioners. This is to say that we will not limit ourselves to presenting results of interest for educators” (“JSSOFTWARE-D-24-00573”, p. 6) #00b036
- - “Also, they were attending courses on computer networks and algorithm design while taking the STW course” (“JSSOFTWARE-D-24-00573”, p. 6) #5fb236
- - “The a.y. 2022-23 students were different from a.y. 2021-22 ones.” (“JSSOFTWARE-D-24-00573”, p. 6) #f0ff00
- *In what respect?*
- - “e-commerce web app” (“JSSOFTWARE-D-24-00573”, p. 6) #5fb236
- - “(i) customers and administrators to log in and log out” (“JSSOFTWARE-D-24-00573”, p. 6) #5fb236
- - “(ii) customers to sign up, navigate the catalogs of items, add items to the cart, buy items, and check their own orders” (“JSSOFTWARE-D-24-00573”, p. 6) #5fb236
- - “(iii) administrators to manage the catalogs of items and customers’ orders” (“JSSOFTWARE-D-24-00573”, p. 6) #5fb236
- - “each web app had to use the MVC pattern, interact with a relational database, and use the technologies presented in the course” (“JSSOFTWARE-D-24-00573”, p. 6) #5fb236
- - “pallet” (“JSSOFTWARE-D-24-00573”, p. 6) #ff4400
- *Pallet?*
- - “76” (“JSSOFTWARE-D-24-00573”, p. 6) #ff4400
- *Where does this number come from?*
- - “five hours” (“JSSOFTWARE-D-24-00573”, p. 7) #5fb236
- - “SonarCloud within GitHub Action workflows and on how to use SonarCloud to detect and resolve security issues.” (“JSSOFTWARE-D-24-00573”, p. 7) #5fb236
- - “and bugs and code smells to quantify, respectively, the reliability and maintainability of software applications.” (“JSSOFTWARE-D-24-00573”, p. 7) #a28ae5
- - “This is because autonomous motivation is fed by the satisfaction of three basic psychological needs: competence, which concerns the experience of effectiveness and mastery; autonomy, which refers to the experience of volition and willingness; and relatedness, which refers to one’s interpersonal connections with others.” (“JSSOFTWARE-D-24-00573”, p. 7) #a28ae5
- - “Moreover, if they resolve bugs and code smells, the underlying reason is that they recognize the usefulness of SonarCloud in developing reliable and maintainable web apps.” (“JSSOFTWARE-D-24-00573”, p. 7) #ffd400
- *I'm not sure about such underlying reason.*
- - “reliability and maintainability of the a.y. 202122 and 2022-23 web apps by using SonarQube” (“JSSOFTWARE-D-24-00573”, p. 8) #5fb236
- - “dependent variables” (“JSSOFTWARE-D-24-00573”, p. 8) #a28ae5
- - “The higher the value of this variable, the lower the reliability of the web app. On the other hand, the number of maintainability issues indicates the number of code smells SonarQube detected in the last commit of a given project. The higher the value of this variable, the lower the maintainability of the web app.” (“JSSOFTWARE-D-24-00573”, p. 8) #5fb236
- - “results of both RQ1 and RQ2,” (“JSSOFTWARE-D-24-00573”, p. 8) #5fb236
- - “post-questionnaire administered to the a.y. 2022-23 STW students.” (“JSSOFTWARE-D-24-00573”, p. 8) #5fb236
- - “2021-22 (45 projects by 120 students without using any SAT) and 2022-23 (65 projects by 151 students using SonarCloud as a SAT).” (“JSSOFTWARE-D-24-00573”, p. 8) #ffd400
- *Teams in 2022-23 were consisting of 2 members (on average) in contrast of more 3 members for the previous year. Was this by accident? or somehow intentional?*
- - “we postulate that this result is partially due to the refactoring operations performed to resolve the code smells detected by SonarCloud.” (“JSSOFTWARE-D-24-00573”, p. 8) #ffd400
- *Or because of smaller groups?*
- - “Yes 5.89” (“JSSOFTWARE-D-24-00573”, p. 9) #ffd400
- - “Yes 10.54” (“JSSOFTWARE-D-24-00573”, p. 9) #ffd400
- *Are these due to some restructuring?*
- - “To verify whether and to what extent the reduction in the number of reliability and maintainability issues is statistically significant between the a.y. 2021-22 and a.y. 2022-23, we tested the following null hypotheses:” (“JSSOFTWARE-D-24-00573”, p. 10) #a28ae5
- - “The former is to study RQ1 while the latter is to study RQ2.” (“JSSOFTWARE-D-24-00573”, p. 10) #5fb236
- - “indicating the presence of a statistically significant difference between the two groups being compared.” (“JSSOFTWARE-D-24-00573”, p. 11) #5fb236
- - “Cliff ’s δ effect size [” (“JSSOFTWARE-D-24-00573”, p. 11) #5fb236
- - “The top three most violated bugs relate to accessibility” (“JSSOFTWARE-D-24-00573”, p. 12) #ffd400
- *How these are related to JSP, Java files as presented in Table 1.*
- - “inadequate use of HTML tags and attributes,” (“JSSOFTWARE-D-24-00573”, p. 12) #5fb236
- - “non-empty statements should change control flow or have at least one side-effect” (“JSSOFTWARE-D-24-00573”, p. 12) #5fb236
- - “happens when a variable declared in a specific scope has the same name as a variable in an outer scope.” (“JSSOFTWARE-D-24-00573”, p. 13) #5fb236
- - “Below, we briefly describe the meaning of each resolution status reported in that table according to SonarSource’s documentation [28].” (“JSSOFTWARE-D-24-00573”, p. 13) #5fb236
- - “This is to say that the analysis with SonarCloud could show a different number of violations from those returned by the analysis with SonarQube.” (“JSSOFTWARE-D-24-00573”, p. 14) #5fb236
- - “A training intervention based on SonarCloud (but not specifically focused on reliability and maintainability) significantly improves the reliability and maintainability of (e-commerce) web apps developed by bachelor students in CS. The magnitude of this improvement is large for both reliability (RQ1) and maintainability (RQ2) issues. Moreover, the improvement, in terms of reliability and maintainability, is statistically significant, regardless of the severity of the issue, although the magnitude of this improvement ranges from small to large.” (“JSSOFTWARE-D-24-00573”, p. 15) #5fb236
- *INTERESTING*
- - “the reason why the students did not use SonarCloud)” (“JSSOFTWARE-D-24-00573”, p. 15) #f0ff00
- *The first and second part of the questionnaire do not make sense for these students, isn't it?*
- - “The third part of the questionnaire concerned the following open question: “Describe your experience (both positive and negative) related to using SonarCloud”.” (“JSSOFTWARE-D-24-00573”, p. 17) #f0ff00
- *Even this part does not make any sense with students of the previous academic year.*
- - “Finding (#1). A training intervention based on a SAT like SonarCloud, even though not specifically focused on the reliability and maintainability quality characteristics, leads CS graduate students to improve the reliability and maintainability of the web apps they develop.” (“JSSOFTWARE-D-24-00573”, p. 17) #5fb236
- *INTERESTING*
- - “as security, reliability, and maintainability.” (“JSSOFTWARE-D-24-00573”, p. 18) #5fb236
- - “This is likely due to autonomous motivations (specifically, integrated extrinsic motivation), which pushed students to resolve reliability and maintainability issues although we neither asked nor forced them to do so.” (“JSSOFTWARE-D-24-00573”, p. 18) #5fb236
- - “Finding (#2). Reliability issues are resolved to a greater extent and perceived as more relevant than maintainability issues.” (“JSSOFTWARE-D-24-00573”, p. 18) #5fb236
- *INTERESTING*
- - “reliability and maintainability” (“JSSOFTWARE-D-24-00573”, p. 21) #5fb236
- - “our training intervention, based on the use of SonarCloud, might also positively affect the reliability and maintainability quality characteristics due to integrated extrinsic motivation. According to the self-determination theory, integrated extrinsic motivation is a kind of autonomous motivation that occurs when people are motivated because the value and regulation of the activity have been integrated within themselves” (“JSSOFTWARE-D-24-00573”, p. 21) #5fb236
- ### [[Comments]]
- Summary: The paper investigates the impact of a training intervention using SonarCloud on the reliability and maintainability of web apps developed by Computer Science (CS) undergraduate students. The study focuses on web apps created in the academic years 2021-22 (group without SonarCloud lectures) and 2022-23 (group with SonarCloud lectures). The key finding is that, although the training was centered on security, the intervention led to improvements in the reliability and maintainability of the web apps.
- Comments: Overall, the paper is well-written and presents interesting insights into software engineering education, particularly in using Static Analysis Tools (SATs) to improve code quality. The comparison between the academic years 2021-22 and 2022-23 provides clear evidence of the intervention's effectiveness. However, some minor clarifications and corrections are required to enhance the paper, as discussed below:
- The focus on reliability and maintainability raises the question of why these particular characteristics were chosen. Just provide a rationale for selecting these two characteristics.
- Earlier in the paper, e.g., in the introduction, clearly articulate the novelty of this work in relation to previous studies and highlight what new knowledge or insights this paper contributes.
- On page 6, the paper mentions differences between the student cohorts of 2021-22 and 2022-23. Provide more information about how the two cohorts differed and whether these differences could have influenced the results.
- On page 7, the authors write: "Moreover, if they resolve bugs and code smells, the underlying reason is that they recognize the usefulness of SonarCloud in developing reliable and maintainable web apps." The authors should consider alternative explanations for the results and whether group size or other factors might have influenced the findings.
- Teams in 2022-23 consisted of 2 members (on average) in contrast to more than three members for the previous year. Was this by accident? or somehow intentional? Discuss if this represents a potential threat to the validity of the study
- Section 4.1: The authors write: "Their mean and median values of KLOC are 13.48 and 9.01, respectively, whereas they are 7.48 and 6.58 for the a.y. 2022-23 projects. Since the minimum functional requirements to be implemented were the same for both the a.y. 2021-22 and 2022-23, we postulate that this result is partially due to the refactoring operations performed to resolve the code smells detected by SonarCloud." The differences in KLOCs can also be due to the size of the groups. Moreover, the difference in the number of files (see Tab 2) could also be due to the restructuring operated by SonarCloud suggestions.
- Section 4.2: The questionnaire seems irrelevant to the 2021-22 cohort, who did not use SonarCloud.
- Minors
- Page 6: "Pallet" -> Should be "palette."
- Page 6: 76 projects -> Clarify where this number of projects come from.
- ### [[REVIEWS/Notes]]
- ### YELLOW CONCERNS
background-color:: yellow
- {{query (and [[ffd400]] [[JSSOFTWARE-D-24-00573]] )}}
collapsed:: true
- ### ❓️Questions
- {{query (and [[question]] [[JSSOFTWARE-D-24-00573]] )[[question]]}}
query-table:: true
query-properties:: [:block]