4.2 KiB
4.2 KiB
collapsed:: true type:: REVIEWS tags:: year:: 2026 venue:: ICSE full-title:: A kNN-Based Recommender System for Test Case Reuse in Agile Development date-start:: 18-09-2025 - 01:59 date-submitted:: external-links:: status:: DONE deadline-submission:: file:: @A kNN-Based Recommender System for Test Case Reuse in Agile Software Development parent:: todoist:: https://app.todoist.com/app/task/3485-a-k-nn-based-recommender-system-for-test-case-reuse-in-agile-development-6cVHmvgxqh6VgCcg
- ### [[Comments]]
- #.tabular
- ### Paper summary
- This paper proposes a KNN-based Recommender System to support the reuse of test cases by leveraging a domain-specific taxonomy and historical user stories. Authors have performed two different experiments. An off-line experiment to identify the configuration that permits to get the achieve the best recommendation performance. The on-line experiment has been performed by using such a configuration with an industrial case study involving two agile projects. Results show promising potential for enhancing test reuse.
- ### Strengths
- + The paper addresses a relevant and practical problem in agile software testing
- + Simple and lightweight solution compared to deep-learning-based alternatives
- ### Weaknesses
- - The central assumption, that similar user stories lead to reusable test cases, is strong and not critically examined
- - The evaluation lacks a proper baseline or comparative study, which limits the assessment of relative performance
- ### Detailed comments for authors
- Novelty: The idea of test case reuse is not new, but this paper innovates by focusing on early-stage reuse in agile environments and using structured user stories with domain taxonomies. In this respect, I found the paper interesting and I liked the simplificity even though effective simplicity of the proposed solution.
- Rigor: The experimental design is solid and includes statistical tests to select the best configuration during the off-line experiments. However, the paper requires revision to address a number of issues as listed below:
- The paper is based on the assumption "similar user stories have similar tests". I found this assumption very strong. Minor variations in implementation can lead to very different tests. It is necessary to clarify whether the retrieved tests were used as inspiration or directly reused. Were they used as-is, adapted, or just served as reference?
- The relevance of the retrieved test cases is not very high: a large portion of accepted tests were only moderately relevant.
- The practical utility of the recommended tests (whether they were reused as-is, adapted, or merely used as inspiration) is not clarified.
- A few important details on the context of the online study (e.g., nature of the company, business domain) are missing, which affects external validity.
- No mention of effort/time metrics, such as time saved or effort required to adapt reused tests, limits the construct validity.
- Relevance: The paper is very relevant to practitioners working in agile contexts. Reuse of test cases could lead to cost and time savings, but the paper does not quantify this aspect as previously pointed out. Moreover, by referring to Fig. 5, the results of the online experiments are not impressive. Nearly half of the accepted tests had low relevance. This needs a more nuanced discussion, especially in the conclusion. Moreover, what does acceptance mean? Were tests reused unchanged? How much effort was required to make them usable?
- Verifiability & transparency: The design and implementation of the RecSys are clearly described. However, the supplementary material includes only the dataset used during the experiments. The developed tools are not included. Thus it is not possible to replicate the performed experients.
- Presentation: Overall, the paper is well-written, logically organized, and easy to follow despite the issues discussed above.
- Questions:
- Q1: To what extent can the recommended test cases be reused as-is versus requiring significant adaptation?
- Q2: How generalizable is the proposed taxonomy-based RecSys to other domains?