logseq/pages/hls__mastropaolo-proposal_1674656747747_0.md

file:: mastropaolo-proposal_1674656747747_0.pdf file-path:: ../assets/mastropaolo-proposal_1674656747747_0.pdf

• software tools that can assist developers with a wide range of activities, from reusing code to writing effective bug reports ls-type:: annotation hl-page:: 7 hl-color:: green id:: 63d5068a-ae7b-451a-ac16-ee3a487e286e
• JSummarizer ls-type:: annotation hl-page:: 7 hl-color:: green id:: 63d5069c-11d3-42b3-b38d-0d379454913e
• data-driven recommenders that can learn how to automate code-related tasks by “looking” at activities performed by real developers in open source project ls-type:: annotation hl-page:: 7 hl-color:: green id:: 63d506b9-ca0f-4af1-a58c-0dce127890cb
• Automating Code-Related Tasks via Pre-trained Models of Code ls-type:: annotation hl-page:: 1 hl-color:: green id:: 63d506f9-c019-4e65-8ea7-db459286745e
• Such a sheer amount of data unlocked the usage of deep learning (DL) models to support code-related tasks, such as automatic bug-fixing [9, 10 , 11 , 12], code summarization [ 13, 14 , 15 ] and code completion[ 16, 17 , 18 , 19, 20 , 21]. ls-type:: annotation hl-page:: 7 hl-color:: green id:: 63d50727-0b4e-4254-8c5f-cde4d6a19465
• Despite the large-scale training, the DL model was able to fix only ∼9% of bugs in Java methods as a real developer would do, posing questions about the possibility of integrating such an approach into recommender systems that can support developers during bug-fixing activities ls-type:: annotation hl-page:: 7 hl-color:: green id:: 63d6ce4b-22e5-44b8-9fc1-5b63233105f8
• Thus, despite the progress in this field, the support that state-of-the-art techniques can offer to developers is still limited, pointing to the need for more researc ls-type:: annotation hl-page:: 8 hl-color:: purple id:: 63d6ce9c-eaab-4902-aeee-634b601ee12d hl-stamp:: 1675021983379
• hese models can then be fine-tuned in a supervised way to learn how to automate specific tasks (e.g., code summarization) ls-type:: annotation hl-page:: 8 hl-color:: purple id:: 63d6cf76-f25a-4cd9-9dc1-566a0986fcaa
• oncrete examples of these applications are CodeBERT [ 24], a pre-trained model for programming language and natural language, and IntelliCode Compose[16], an advanced code completion technique exploiting a multi-layer generative pretrained Transformer for code (GPT-C ls-type:: annotation hl-page:: 8 hl-color:: purple id:: 63d6d0c5-cca0-408a-86d7-afef395322c3
• T5 also allows multi-task learning, meaning that a single model can be trained to learn several tasks at the same time ls-type:: annotation hl-page:: 8 hl-color:: purple id:: 63d6d13a-b8aa-4d1a-a174-05bc483f6c8f
• The idea of using the “pretrain-then-finetune” paradigm to support software engineering(SE) tasks has been firstly proposed by Robbes et al. [ 22], which suggested it as a way to overcome the limited size of training datasets available for specific tasks (e.g., sentiment analysis on software-related corpora such as Stack Overflow discussions ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 63d7c598-cde8-4f6d-8363-7a6307124e9e hl-stamp:: 1675085210794
• Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks ls-type:: annotation hl-page:: 8 hl-color:: green id:: 63d7cca6-842f-4792-80aa-3428b6ec17cc hl-stamp:: 1675087021555
• we did not investigate the role played by pre-training the model and then fine-tuning it. ls-type:: annotation hl-page:: 8 hl-color:: yellow id:: 63d7cce0-29be-4e63-bd6b-6e6364f87a1f hl-stamp:: 1675087074878
• Using Transfer-Learning for Code-Related Tasks ls-type:: annotation hl-page:: 9 hl-color:: green id:: 63d7cd04-93f5-4553-994a-58490e955448 hl-stamp:: 1675087111071
• pretraining phase ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cd11-e1df-4a63-b0da-e4e7a799c6ff hl-stamp:: 1675087159095
• (i) code snippet summarization ls-type:: annotation hl-page:: 9 hl-color:: blue id:: 63d7cdba-9792-42ec-9bc0-5709b66b1c89 hl-stamp:: 1675087292358
• (ii) generation and injection of complete log statements. ls-type:: annotation hl-page:: 9 hl-color:: blue id:: 63d7cdc0-9ef0-4b28-98cd-827fe7b34f01
• ∼10% correct prediction ls-type:: annotation hl-page:: 9 hl-color:: blue id:: 63d7cde1-4383-4bcf-a4a9-8c76a2756241
• pretrained T5 ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cde9-fa31-45d3-9661-120c582348ed hl-stamp:: 1675087346898
• we moved from the automated generation of complete summaries, to code comment comple ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cefc-62c7-46a5-9f9b-04c2291a2aca
• An Empirical Study on Code Comment Completion ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cf4a-4303-432b-b8e0-e495a4e73ba2
• 3 1.1 Expected Contributions and State of the Research In other words, we were not aware of the benefits (if any) brought by pre-training the model, since we did not compare a pre-trained T5 with a non pre-trained T5 (i.e., only fine-tuning). Second, we did not study the impact on performances of mixing tasks (i.e., multi-task) when specializing the model on the set of code-related tasks we aimed at supporting. For such reasons, we extended our ICSE 2021 work by investigating these two specific aspects. We reported the results of this second investigation in the following paper [2]: Using Transfer-Learning for Code-Related Tasks Antonio Mastropaolo , Nathan Cooper, David Nader-Palacio, Simone Scalabrino, Denys Poshyvanyk, Rocco Oliveto, Gabriele Bavota. In IEEE Transactions on Software Engineering (TSE2022), to appear. Our findings showed the boost in performance provided by the pretraining phase, while did not highlight a major role of the multi-task learning. Once established the effectiveness of the “pretrain-then-finetune” paradigm in dealing with code-related tasks, we decided to explicitly focus on two tasks which are characterized by a mix of code and natural language and for which support in the literature is lacking: (i) code snippet summarization (i.e., the ability to describe in natural language a snippet of code), and (ii) generation and injection of complete log statements. As for the former (i.e., code snippet summarization), the results of our ICSE 2021 paper [ 1 ] showed that even the pretrained T5 struggles in generating summaries for entire Java methods(∼10% correct predictions). Thus, as a first step, we studied whether by simplifying the problem it was possible to achieve better results. In particular, we moved from the automated generation of complete summaries, to code comment completion, in which the ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cf6c-e160-4d69-8b5d-ef64f2e819f4
• (∼16% correct predictions) ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cfbb-41c8-42c5-b6e3-88394be725a3
• A first challenge to overcome here is the building of the training datase ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cfd1-3aca-4719-b991-ca8b266613a5
• <method, javadoc> ls-type:: annotation hl-page:: 9 hl-color:: yellow id:: 63d7cfd8-3ec7-4c7e-8c5b-617d7f298916
• As of Today, our dataset counts over 11k pairs and will be soon made available to the research community ls-type:: annotation hl-page:: 9 hl-color:: green id:: 63d7cff1-e728-48b0-ad6f-7cc2a2d0a4eb hl-stamp:: 1675087859361
• LANCE (Log stAtemeNt reCommEnder ls-type:: annotation hl-page:: 10 hl-color:: green id:: 63d7d2a3-4a2a-4680-85e3-170a319a42fb
• Using Deep Learning to Generate Complete Log Statements ls-type:: annotation hl-page:: 10 hl-color:: green id:: 63d7d2bc-e0a4-4c8b-bb12-b7655d3a8e07
• LANCE in order to decide a proper number of statements which may be needed (0 to n) ls-type:: annotation hl-page:: 10 hl-color:: green id:: 63d7d30a-2499-4c9b-97fa-9e6177b83af1
• ata-driven techniques” can support the task of automated variable renaming in the context of Java methods ls-type:: annotation hl-page:: 10 hl-color:: green id:: 63d7d31a-073a-4071-90ec-c44124e40df4
• extractive ls-type:: annotation hl-page:: 12 hl-color:: green id:: 63d7d524-b64f-4660-b5e4-45850686bc81
• abstractive ls-type:: annotation hl-page:: 12 hl-color:: green id:: 63d7d526-e4b8-4e8f-92ff-028494d0f906
• (i) techniques aimed at documenting code snippets ls-type:: annotation hl-page:: 12 hl-color:: green id:: 63d7d59c-004b-4a82-9256-358c87cf4918
• (ii) DL-based approaches ls-type:: annotation hl-page:: 12 hl-color:: green id:: 63d7d59f-f962-4765-895c-cbce7fa2a707
• Code-Related Tasks ls-type:: annotation hl-page:: 15 hl-color:: green id:: 63d7d936-6d72-4d36-adfb-7a6961f16c66
• bi-modal data (i.e., involving both source code and technical natural language) ls-type:: annotation hl-page:: 15 hl-color:: green id:: 63da5551-7fa6-42a9-a66a-bee8cb8cf16e
• e trained model is fine-tuned on smaller and specialized datasets, each one aimed at supporting a specific task ls-type:: annotation hl-page:: 15 hl-color:: green id:: 63da582e-7e3b-4874-816f-878254c27187
• Automatic bug-fixing ls-type:: annotation hl-page:: 15 hl-color:: purple id:: 63da5c2a-9a57-497f-b3f3-67ab86a35147
• he abstraction process includes a mechanism to transform back the abstracted code into its raw format. ls-type:: annotation hl-page:: 15 hl-color:: green id:: 63da5cde-1ecc-475e-8410-064134269db1
• The choice of the four tasks subject of our study (i.e., bug-fixing, mutants injection, asserts generation, and code summarization) is dictated by the will of experimenting with tasks that use, represent, and manipulate code in different ways. In particular, we include in our study tasks aimed at (i) transforming the input code with the goal of changing its behavior (bug-fixing and mutants injection); (ii) “comprehending code” to verify its behavior(asserts generation); and (iii) “comprehending code” to summarize it in natural language(code summarization). ls-type:: annotation hl-page:: 16 hl-color:: green id:: 63da5db0-1788-49d0-981f-4f4a160c0096
• Once the T5 model has been fine-tuned on all these tasks, we run it on the same test sets used in the four referenced works [ 9, 26 , 27 , 14] comparing the achieved results to those reported in the original work. Ta ls-type:: annotation hl-page:: 16 hl-color:: yellow id:: 63da5def-890e-49e0-ad12-5d30f45b5a21
• [:span] ls-type:: annotation hl-page:: 16 hl-color:: green id:: 63da5e2b-3eae-45d1-9aa9-2b6f752eb86a hl-type:: area hl-stamp:: 1675255337534
• substantial ls-type:: annotation hl-page:: 16 hl-color:: yellow id:: 63da5ea9-868e-4f7f-9d5d-83d82ee015e0
• 11% vs 3% of correct predictions ls-type:: annotation hl-page:: 16 hl-color:: green id:: 63da5eae-172a-4698-943f-d42195066d80
• e fine-tuned a single pre-trained T5 model in a multi-task setting on all four tasks, showing that it is able to achieve better results as compared to the four referenced baselines ls-type:: annotation hl-page:: 17 hl-color:: green id:: 63da5ec6-500f-44cd-a7a0-4c9663074e28
• [:span] ls-type:: annotation hl-page:: 17 hl-color:: green id:: 63da605b-9f06-4373-8aab-ffd2ea7e3a9f hl-type:: area hl-stamp:: 1675255897546