diff --git a/pages/ESEM25_paper_260.md b/pages/ESEM25_paper_260.md new file mode 100644 index 00000000..9cd289b7 --- /dev/null +++ b/pages/ESEM25_paper_260.md @@ -0,0 +1,30 @@ +type:: [[REVIEWS]] +tags:: +year:: 2025 +venue:: [[ESEM]] +full-title:: +date-start:: [[26-06-2025]] - 14:49 +date-submitted:: +external-links:: +status:: [[DOING]] +deadline-submission:: +file:: +parent:: +todoist:: +collapsed:: true + + - ### [[Highlights]] + - ### [[Comments]] + - **SUMMARY** + - This paper presents *RAGSum*, a novel retrieval-augmented generation framework for automatic code comment generation. Built on top of a single CodeT5 backbone, RAGSum integrates retrieval and generation through joint fine-tuning to improve the semantic accuracy and contextual relevance of generated comments. The approach is compared against state-of-the-art baselines using standard metrics across benchmark datasets, demonstrating measurable improvements. The work is positioned within the growing interest in unifying retrieval and generation for source code summarization. + - --- + - **COMMENTS** + - The paper tackles an important problem in software engineering, namely the generation of meaningful and accurate comments for source code snippets. While the motivation and the general idea of integrating retrieval and generation using a unified CodeT5 backbone is promising, some sections of the paper would benefit from improved clarity and precision. + - At the beginning of the paper, the sentence describing the propagation of noise due to optimizing retrieval and generation in isolation is rather unclear. A rephrasing could help convey the underlying problem more effectively. Similarly, the expression *"to code"* (p. 2) lacks context and should either be clarified or corrected, as its meaning is ambiguous. + - A sentence on p. 2 describing the DECOM framework is syntactically incorrect and difficult to parse. Consider revising it to clearly explain the role of the multistage deliberation process and how keywords and retrieved comments are used to enhance generation. + - In the methodological discussion, there is a reference to retrieving comments from similar source code snippets. It would be important to clarify what is meant by "similar" and whether syntactic or semantic similarity is considered. Moreover, the paper should address whether code similarity reliably implies comment similarity, as this assumption underpins the retrieval component’s effectiveness. + - When reporting results (e.g., RAGSum's improvements of 4.1%, 5.31%, etc. across different metrics), the paper lacks a qualitative discussion that contextualizes these gains. Are such improvements statistically significant? Do they translate into meaningfully better developer experience or understanding? Including such an analysis would strengthen the empirical claims. + - An important point is raised regarding the potential mismatch between metrics like ROUGE-L and true semantic equivalence. This issue deserves further elaboration, especially since some generated comments may be semantically adequate yet penalized due to surface-level differences with the reference. Addressing this limitation could lead to more informed conclusions and possibly a more nuanced evaluation methodology. + - Overall, the paper makes a valuable contribution to the field but would benefit from improvements in clarity, the resolution of minor ambiguities, and a deeper discussion of the empirical findings. + - --- +- \ No newline at end of file