logseq/pages/@ESEM25_paper_260.md

tags:: #zotero title:: @ESEM25_paper_260 item-type:: document original-title:: ESEM25_paper_260 links:: Local library, Web library

• Attachments

  • PDF {{zotero-imported-file FCRZY2MW, "ESEM25_paper_260.pdf"}}

• Notes

  • Annotazioni

    (26/6/2025, 14:47:40)

    • “existing comments and then synthesize a new comment, yet retrieval and generation are typically optimized in isolation, allowing irrelevant neighbors to propagate noise downstrea” (“ESEM25_paper_260”, p. 1) #ffd400 *not very clear *

    • “RAGSum with the aim of both effectiveness and efficiency in recommendations. RAGSum is built on top of fuse retrieval and generation using a single CodeT5 backbone” (“ESEM25_paper_260”, p. 1) #5fb236

  - “Template systems extract salient tokens and stitch them into fixed linguistic patterns [3], [4]; IR systems locate code fragments similar to a query and reuse their comments [5], [6]. Although lightweight, these methods often misalign with the precise semantics of the target snipp” (“ESEM25_paper_260”, p. 1) #5fb236
  * *
  
   
  
  - “Such models learned richer representations, but even the best variants struggled to bridge the modality gap between programming languages and English, leading to generic or inaccurate summaries” (“ESEM25_paper_260”, p. 1) #5fb236
  * *
  
   
  
  - “train retrieval and generation components separately” (“ESEM25_paper_260”, p. 1) #5fb236
  * *
  
   
  
  - “refines retrieved comments to better align with the semantics of the input code query.” (“ESEM25_paper_260”, p. 1) #5fb236
  * *
  
   
  
  - “While EditSum captures essential keywords from the input code snippet during comment generation through its self-editing pipeline, the presence of irrelevant retrieved code can still degrade performance” (“ESEM25_paper_260”, p. 1) #a28ae5
  * *
  
   
  
  - “MR-Sum proposed an extractor that integrates generated and retrieved comments within a unified framework, aiming to align them using an attention mechanism.” (“ESEM25_paper_260”, p. 1) #5fb236
  * *
  
   
  
  - “We argue that though these approaches outperform earlier methods based on separate training paradigms, treating the retriever and generator as distinct tasks may still hinder the overall performance of comment generation.” (“ESEM25_paper_260”, p. 1) #5fb236
  * *
  
   
  
  - “Figure 1 we show the results of using JOINTCOM and CMR-Sum to generate comments for a given input code query. It is evident that compared to the ground-truth comment and the input code, the results generated by both JOINTCOM and CMR-Sum exhibit significant semantic inaccuracies.” (“ESEM25_paper_260”, p. 1) #a28ae5
  * *
  
   
  
  - “fuse retrieval and generation within a single CodeT5” (“ESEM25_paper_260”, p. 1) #5fb236
  * *
  
   
  
  - “RAGSum gains significant improvements with respect to the baselines.” (“ESEM25_paper_260”, p. 2) #5fb236
  * *
  
   
  
  - “These early findings indicate that tightly coupling retrieval and generation can raise the ceiling for comment automation and motivate forthcoming industrial replications and qualitative developer studies.” (“ESEM25_paper_260”, p. 2) #a28ae5
  * *
  
   
  
  - “to code” (“ESEM25_paper_260”, p. 2) #ffd400
  *to code? *
  
   
  
  - “deep learning-based methods for code summarization” (“ESEM25_paper_260”, p. 2) #5fb236
  * *
  
   
  
  - “generation models often struggle with issues such as hallucination and limited access to external knowledge, which can hinder the accuracy and completeness of the generated summaries” (“ESEM25_paper_260”, p. 2) #5fb236
  * *
  
   
  
  - “Another framework for comment generation – DECOM [23] with the multistage deliberation process which use the keywords from source code and the comment of retrieved sample to enhance the performance” (“ESEM25_paper_260”, p. 2) #ffd400
  *It does not parse, please revise. *
  
   
  
  - “Recent research has concentrated on exploring various prompting techniques to better harness the potential of LLMs in this task [24] but the summaries produced by LLMs often differ significantly in expression from reference and tend to include more detailed information than those generated by traditional models [25]” (“ESEM25_paper_260”, p. 2) #5fb236
  * *
  
   
  
  - “Self-Supervised Training of Retriever;” (“ESEM25_paper_260”, p. 2) #2ea8e5
  * *
  
   
  
  - “RetrieverGenerator Joint Fine-tuning” (“ESEM25_paper_260”, p. 2) #2ea8e5
  * *
  
   
  
  - “Self-Refinement Process” (“ESEM25_paper_260”, p. 2) #2ea8e5
  * *
  
   
  
  - “Given a code query qi and its corresponding comment ci, the CodeT5 encoder first produces two representation vectors, which, for simplicity, are also denoted as qi and ci, respectively” (“ESEM25_paper_260”, p. 2) #5fb236
  * *
  
   
  
  - “q+ i” (“ESEM25_paper_260”, p. 2) #5fb236
  * *
  
   
  
  - “this objective helps the model more effectively distinguish between relevant and irrelevant pairs, thereby enhancing its understanding of the semantic relationship between code and comments.” (“ESEM25_paper_260”, p. 3) #5fb236
  * *
  
   
  
  - “RQ1: How effective is the Retriever component of RAGSum in retrieving relevant results compared to the baselines?” (“ESEM25_paper_260”, p. 4) #f19837
  * *
  
   
  
  - “RAGSum, JOINTCOM, and CMR-Sum–by comparing them to the reference comment of the input code.” (“ESEM25_paper_260”, p. 4) #5fb236
  * *
  
   
  
  - “RQ2: How effective is RAGSum compared to the baselines?” (“ESEM25_paper_260”, p. 4) #f19837
  * *
  
   
  
  - “CMR-Sum [14] introduced a joint retriever-generator framework for code summarization, where the retriever and generator are finetuned independently.” (“ESEM25_paper_260”, p. 4) #5fb236
  * *
  
   
  
  - “JOINTCOM [13] also employed a joint retriever-generator paradigm for comment generation,” (“ESEM25_paper_260”, p. 4) #5fb236
  * *
  
   
  
  - “LLama-3.1-8B [16] is a Large Language Model (LLM) developed by Meta AI. Due to resource constraints, we use the 8B-parameter version for inference. In our experiments, the LLM serves as the generator in the RAG framework, with one-shot and few-shot exemplars retrieved using CodeT5 embeddings.” (“ESEM25_paper_260”, p. 4) #5fb236
  * *
  
   
  
  - “RQ3: How does each component of RAGSum contribute to its overall performance?” (“ESEM25_paper_260”, p. 4) #5fb236
  * *
  
   
  
  - “to each input code in the test set, and then calculated the ROUGE-L score between the retrieved comment and the ground truth comment.” (“ESEM25_paper_260”, p. 4) #ffd400
  *Something that needs to be clarified is the process that generates comments by retrieving those of source code, which is similar to that under analysis. Having similar code does it always mean similar descriptive text? *
  
   
  
  - “The retriever of RAGSum is more effective and robust than the baseline methods in fetching relevant information.” (“ESEM25_paper_260”, p. 5) #5fb236
  * *
  
   
  
  - “B. RQ2: How effective is RAGSum compared to the baselines?” (“ESEM25_paper_260”, p. 5) #5fb236
  * *
  
   
  
  - “RAGSum increases 4.1%, 5.31%, 2.63%, 3.13% and 4.68% in terms of C-BLEU, S-BLEU, ROUGE-L, METEOR, and CIDEr, respectively.” (“ESEM25_paper_260”, p. 5) #ffd400
  *are this increase values significant? Can you discuss them from a qualitative point of view? *
  
   
  
  - “Notably, RAGSum leverages relevant knowledge to generate more context-aware comments.” (“ESEM25_paper_260”, p. 5) #5fb236
  * *
  
   
  
  - “C. RQ3: How does each component of RAGSum contribute to its overall performance?” (“ESEM25_paper_260”, p. 5) #5fb236
  * *
  
   
  
  - “may not fully capture semantic equivalence, potentially underestimating the quality” (“ESEM25_paper_260”, p. 6) #ffd400
  *This is a very tricky point, which deserves further investigation. *
  
   
  
  - “In this paper, we proposed RAGSum for automated code comment generation that effectively leverages the existing joint fine-tuning retriever and generator. Empirical evaluation of benchmark datasets showed that RAGSum significantly improved baselines in code summarization. For future work, we plan to explore more dynamic retrieval mechanisms, investigate the scalability of RAGSum to large-scale codebases, and extend our approach to support multilingual codebases and more diverse programming paradigms.” (“ESEM25_paper_260”, p. 6) #5fb236
  * *