From 23f7a35203d17000174fb2f2de9432587f4e8d01 Mon Sep 17 00:00:00 2001 From: davidediruscio Date: Thu, 26 Jun 2025 15:00:01 +0200 Subject: [PATCH] [logseq-plugin-git:commit] 2025-06-26T13:00:00.992Z --- pages/ESEM25_paper_260.md | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/pages/ESEM25_paper_260.md b/pages/ESEM25_paper_260.md index 2b5005a9..c58d4c47 100644 --- a/pages/ESEM25_paper_260.md +++ b/pages/ESEM25_paper_260.md @@ -11,20 +11,15 @@ status:: [[DOING]] deadline-submission:: file:: [[@ESEM25_paper_260]] parent:: -todoist:: +todoist:: https://app.todoist.com/app/task/260-when-retriever-meets-generator-a-joint-model-for-code-comment-generation-6c6VQJ4qGr8rx7mc - ### [[Highlights]] - ### [[Comments]] - - **SUMMARY** - - This paper presents *RAGSum*, a novel retrieval-augmented generation framework for automatic code comment generation. Built on top of a single CodeT5 backbone, RAGSum integrates retrieval and generation through joint fine-tuning to improve the semantic accuracy and contextual relevance of generated comments. The approach is compared against state-of-the-art baselines using standard metrics across benchmark datasets, demonstrating measurable improvements. The work is positioned within the growing interest in unifying retrieval and generation for source code summarization. - - --- - - **COMMENTS** - - The paper tackles an important problem in software engineering, namely the generation of meaningful and accurate comments for source code snippets. While the motivation and the general idea of integrating retrieval and generation using a unified CodeT5 backbone is promising, some sections of the paper would benefit from improved clarity and precision. - - At the beginning of the paper, the sentence describing the propagation of noise due to optimizing retrieval and generation in isolation is rather unclear. A rephrasing could help convey the underlying problem more effectively. Similarly, the expression *"to code"* (p. 2) lacks context and should either be clarified or corrected, as its meaning is ambiguous. - - A sentence on p. 2 describing the DECOM framework is syntactically incorrect and difficult to parse. Consider revising it to clearly explain the role of the multistage deliberation process and how keywords and retrieved comments are used to enhance generation. - - In the methodological discussion, there is a reference to retrieving comments from similar source code snippets. It would be important to clarify what is meant by "similar" and whether syntactic or semantic similarity is considered. Moreover, the paper should address whether code similarity reliably implies comment similarity, as this assumption underpins the retrieval component’s effectiveness. - - When reporting results (e.g., RAGSum's improvements of 4.1%, 5.31%, etc. across different metrics), the paper lacks a qualitative discussion that contextualizes these gains. Are such improvements statistically significant? Do they translate into meaningfully better developer experience or understanding? Including such an analysis would strengthen the empirical claims. - - An important point is raised regarding the potential mismatch between metrics like ROUGE-L and true semantic equivalence. This issue deserves further elaboration, especially since some generated comments may be semantically adequate yet penalized due to surface-level differences with the reference. Addressing this limitation could lead to more informed conclusions and possibly a more nuanced evaluation methodology. - - Overall, the paper makes a valuable contribution to the field but would benefit from improvements in clarity, the resolution of minor ambiguities, and a deeper discussion of the empirical findings. + - Summary: Ther paper presents RAGSum, an approach based on retrieval-augmented generation to automatically generate code comments. The approach is compared with different baselines with respect to standard metrics and different benchmark datasets. The performed experiments shows improvements and shows that different components of the approach contributing to acheve the measured performance. + - COMMENTS: The paper tackles an important problem in software engineering, namely the generation of meaningful and accurate comments for source code snippets. While the motivation and the general idea of integrating retrieval and generation using a unified CodeT5 backbone is promising, some sections of the paper would benefit from improved clarity: + - In the methodological discussion, there is a reference to retrieving comments from similar source code snippets. The paper should better support the hypotesis whether code similarity reliably implies comment similarity, as this assumption underpins the retrieval component’s effectiveness. + - When reporting results (e.g., RAGSum's improvements of 4.1%, 5.31%, etc. across different metrics), the paper lacks a qualitative discussion that contextualizes these gains. Are such improvements statistically significant? Do they translate into meaningfully better developer experience or understanding? + - An important point is raised regarding the potential mismatch between metrics like ROUGE-L and true semantic equivalence. This issue deserves further investigation (not in this paper indeed), especially since some generated comments may be semantically adequate yet penalized due to surface-level differences with the reference. + - - --- - \ No newline at end of file