logseq/pages/Qualitative Clustering of Software Repositories Based on Software Metrics.md

id:: 63568355-e57b-4a93-b804-5717bca6402b type:: REVIEWS venue:: ICSE year:: 2023 full-title:: Qualitative Clustering of Software Repositories Based on Software Metrics external-links:: #223 - ICSE 2023 (hotcrp.com) file:: status:: DONE

• Paper summary

  • The paper proposes an approach to create clusters of software repositories automatically. The methodology is based on multiple clustering techniques and introduces the adoption of cluster validity and consensus clustering. The approach has been validated by automatically labeling 1659 repositories.

• Strengths

  • Interesting topic
  • Overall well-structured paper

• Weaknesses

  • The novelty of the paper is not evident
  • The strengths of the paper are not supported by a convincing evaluation

• Comments for the authors

  • Significance and Novelty: The paper is about an interesting and relevant SE problem. The availability of automated clustering techniques is paramount to support several software engineering tasks, including identifying and reusing existing open-source repositories. However, my main concerns about the paper are related to its novelty. The authors claim that they aim at i) analysing "open-source software repositories from the perspective of developers' community interest" and ii) overcoming the limitations of existing approaches because "most of them use a single clustering technique". However, the paper does not detail aspects of the methodology that purposely supports i). In particular, the authors make use of different clustering techniques without introducing the management of specific features that are linked to the mentioned concept of "developers' community interest". Concerning ii), the authors do not convince that employing more than one clustering technique is beneficial.
  • Soundness: Following my previous comments, the evaluation is not executed correctly. In particular, what are the research questions you wanted to answer? I expected to see a comparison of the proposed technique with some existing baselines to show that it performs better concerning some metrics. The last paragraph of section IV has to be significantly expanded to discuss the performance of the introduced clustering mechanism. Moreover, the way the proposed clusters are put in relation to the Gartner hype cycle shown in Fig. 3 is unclear to me.
  • Verifiability and Transparency: The submission includes an online appendix available at https://anonymous.4open.science/r/TOM-ClusteringDatasets-1752/README.md. It consists of two CSV files, which contain metrics related to commits and issues of software repositories. The explanation of the metrics is given in an online spreadsheet Clustering metrics.xlsx - Google Sheets. However, the appendix does not give any details on how to replicate the presented experiments including the generation of the different clusters produced by the proposed technique as discussed in the paper.
  • ** The artifacts produced during the experiments are in an online appendix, which is well-structured and organized. The appendix has been very useful to me in understanding how the approach is intended to be used and what are its main goals. These aspects are not properly described in the paper, which lacks an appropriately crafted use case throughout the document.
  • Presentation: Overall, the presentation of the paper is in line with what is expected for ICSE submissions. However, as previously mentioned, the paper needs a severe revision to convincingly present the problem that has been addressed and support the claim that the proposed approach is better than existing clustering techniques.

• Question for authors

  • Q1: Can you elaborate on why the proposed clustering technique performs better than the existing ones?

• Notes

  • ((636133df-d256-405b-8700-fd45a3991aff))
  • ((63605b69-bb28-4f08-981a-f6d37054bee0))
    • This is important. However, the developer pespective might not be necessarily in line with the metrics that are appropriate for the classification.
      • ((636138c2-e2b0-457e-9c23-c2cac8147967))
        • ==Eactely that's the point==
  • ((636138f9-0dfa-417c-be6c-2431211ee381))
    • This seems to be an iterative process!!!!
  • ((6361a23a-7928-4740-9097-e9d888c6a30a))
    • We do not have evidence at this stage that using more than one clustering technique is better!