logseq/pages/hls__INFSOF-D-23-00572_reviewer.md

file:: INFSOF-D-23-00572_reviewer.pdf file-path:: C:\Users\david\Nextcloud\WORK\Referaggi\2023\10-23-IST\INFSOF-D-23-00572_reviewer.pdf

• Section 2 ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6540224f-a91f-436c-a08a-794ff357d3b5
• Our study seeks to analyze the evolution of repository metrics in a population of open-source projects from Github. ls-type:: annotation hl-page:: 4 hl-color:: purple id:: 6544134f-6fe1-43ce-855b-0442f5bfa3c9
• temporal validity is not a concept broadly addressed by the scientific literature on Empirical Software Engineering ls-type:: annotation hl-page:: 5 hl-color:: blue id:: 65441768-f1d0-44a0-b428-1e73422f8b68
• it refers to the extent to which a representative sample collected during a particular time period accurately represents the population at a future point. ls-type:: annotation hl-page:: 5 hl-color:: purple id:: 65441785-9b1e-41c9-b062-7f8cb3433632
• several examples of Empirical Software Engineering studies published in top-ranked journals and conferences still rely on rather old datasets ls-type:: annotation hl-page:: 5 hl-color:: purple id:: 6544181d-9835-4710-9e8e-23d4f8681ad3
• Software repositories offer information on the code, people, and procedures involved in software development, and can provide valuable insights into the growth and evolution of software projects. ls-type:: annotation hl-page:: 5 hl-color:: green id:: 65474c47-22dd-4cd9-ab7b-53e0e3ab3a6d hl-stamp:: 1699171402251
• A longitudinal study consists in capturing repeated observations (or waves) of the same units on the same outcomes over a certain amount of time ls-type:: annotation hl-page:: 7 hl-color:: blue id:: 65474cd2-af99-4513-8c94-15f1f31c30fc
• Bavota et al. [46] studied the evolution of a Java subset of the Apache ecosystem for a period of 14 years, in terms of their size, dependencies among them, and other meta-data ls-type:: annotation hl-page:: 7 hl-color:: green id:: 65474d12-f297-4bbd-bfb9-8c29d5764667
• exponential increase in the size of the projects and the number of dependencies ls-type:: annotation hl-page:: 7 hl-color:: green id:: 65474d18-fe3f-458d-ab14-d70fbba46354
• temporal validity in Software Engineering, ls-type:: annotation hl-page:: 7 hl-color:: green id:: 65474e97-ef0b-4425-a9a2-f6adfbc605ff
• our main contribution is a longitudinal study to evaluate a population of quality open-source projects over six years in order to provide insights on the matter ls-type:: annotation hl-page:: 7 hl-color:: yellow id:: 65474ea3-1ba2-4900-98e8-c0a12b0f77d1
• e the evolution of temporal validity in a population of quality software projects ls-type:: annotation hl-page:: 7 hl-color:: purple id:: 65474f4b-099e-41dc-89ba-757f591b158f hl-stamp:: 1699172176318
• data distribution shift for the analyzed metrics ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 65474f8d-3130-4954-800d-ea34a62d144a hl-stamp:: 1699172248299
• Our aim is to estimate the probability that a dataset remains representative as time passes by ls-type:: annotation hl-page:: 8 hl-color:: purple id:: 65474f96-88c3-470d-bda6-9e12f79ce32f
• 2077 repositories ls-type:: annotation hl-page:: 9 hl-color:: green id:: 6547514e-dc99-45dd-9948-d61c10f1d8be
• statistical hypothesis testing and effect size measures to ensure that our results were valid from a statistical perspective ls-type:: annotation hl-page:: 10 hl-color:: green id:: 654751f4-c608-4efd-b851-6c47a2b6bf00
• α = 0.05 ls-type:: annotation hl-page:: 10 hl-color:: green id:: 654751f9-fe08-4b40-8519-d5ec6bb25df7
• Bonferroni’s correction ls-type:: annotation hl-page:: 10 hl-color:: green id:: 654751fe-d300-4af3-a31b-554c865185db
• The null hypothesis states that the observations’ median of all tested groups are equal. The test is applied to multiple groups simultaneously but cannot identify exactly where and how much the groups are statistically different. When the null hypothesis was rejected, i.e., the median among all the groups is statistically different, we ran a post-hoc pairwise test to identify the pairs of groups of observations that were different. To do this, we used the Dunn’s test[59], in which the null hypothesis states that there is no difference between group ls-type:: annotation hl-page:: 10 hl-color:: green id:: 654757c8-e34f-4db5-8025-c7b11881eb1f hl-stamp:: 1699174522962
• The “beginning of time” was the date of the snapshot retrieval. The event considered for the analysis was: “the snapshot is not representative”, referring to the time when a significant difference is detected between a snapshot and the snapshot of a following wave. ls-type:: annotation hl-page:: 10 hl-color:: green id:: 65475857-337f-4ee8-85f0-9175f9fea23a
• time is a factor that can potentially influence the distributions of repository data ls-type:: annotation hl-page:: 14 hl-color:: blue id:: 654758ec-1a32-4915-8cd2-03a8df7778f5 hl-stamp:: 1699174639975
• The general survival analysis showed it is likely that a population of Java projects will be representative for the first year (82.3%), ls-type:: annotation hl-page:: 15 hl-color:: green id:: 65475930-a48f-487f-8e80-d0db5989685f
• Furthermore, more active projects (Q4) tend to have more contributors, commits, pull-requests, issues, stars and forks than projects with less activity (Q1), and also tend to have less months of development. ls-type:: annotation hl-page:: 19 hl-color:: yellow id:: 65475960-c58d-4db3-a1fb-c0f0c3376911
• eaning that a snapshot can lose validity as time passes by. ls-type:: annotation hl-page:: 19 hl-color:: blue id:: 654759e4-8146-47e2-be07-6cc8ad811878
• In the analysis, we tackled three research questions about whether the population exhibits data shifts, when this occurs, and how much time repositories remain actively maintained ls-type:: annotation hl-page:: 21 hl-color:: green id:: 65475a31-9aa0-4a6a-970b-aca513bb1a55
• statistical differences in the number of contributors, commits, pull-request, closed issues, stars and forks due to their growth over time. ls-type:: annotation hl-page:: 21 hl-color:: blue id:: 65475a37-8a02-4b49-8bef-4410c1318fbd
• we observed that the probability of a snapshot being representative after a year is rather high with a probability of 82.3%, and zero before reaching its second year (579 days). ls-type:: annotation hl-page:: 21 hl-color:: blue id:: 65475a43-1214-48ff-96c4-3f2c92483c62
• However, after five years all variables get values under 25% ls-type:: annotation hl-page:: 21 hl-color:: blue id:: 65475a4e-882e-4bbd-bb76-87bab8e0e85b
• to provide evidence of the significance of temporal validity and the importance of using up-to-date datasets when conducting empirical studies in Software Engineering. ls-type:: annotation hl-page:: 22 hl-color:: blue id:: 65475a6e-995a-4f05-b016-fa313f6c58c5
• We shed light on the role of temporal validity to draw generalizable conclusions from open-source software, and emphasize the importance of capturing the dynamics of project evolution in current software developmen ls-type:: annotation hl-page:: 22 hl-color:: blue id:: 65475a7b-aafc-4de4-8c36-67f4f16b8241
• we provide insights into the characteristics of open-source projects in terms of their update activity. ls-type:: annotation hl-page:: 22 hl-color:: blue id:: 65475a82-74ae-45b0-abdd-ce86fb94defa
• a dataset maintenance process that can help researchers keep their datasets up-to-date and mitigate temporal validity loss ls-type:: annotation hl-page:: 22 hl-color:: blue id:: 65475a88-ebd3-4ba7-80b5-ba27dd4b75a3