logseq/pages/TOSEM-2024-0474.md

type:: [[REVIEWS]]
tags:: 
year:: 2024
venue:: [[TOSEM]] 
full-title:: Why Personalizing Deep Learning-Based Code Completion Tools Matters
date-start:: [[12-10-2024]] - 18:37
date-submitted::
external-links::
status:: [[done]]
deadline-submission:: [[14-10-2024]] 
file:: [[@TOSEM-2024-0474_Proof_hi]] 
parent::
todoist:: https://app.todoist.com/app/task/tosem-2024-0474-reviewer-agreed-6VmWvQppc4wMqw9g

	- ### [[Highlights]]
		-
		- # Annotazioni
			- (14/10/2024, 15:56:19)
			- - “Why Personalizing Deep Learning-Based Code Completion Tools Matters” ([“TOSEM-2024-0474_Proof_hi”, pp. -](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=1&annotation=AHTYAZPP)) #5fb236
			- - “providing unprecedented code generation capabiliti” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=YE7MUC7D)) #5fb236
			- - “learning natural coding patterns observed across many training instances” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=PRQIIUWJ)) #5fb236
			- - “extent to which additional training effort (fine-tuning) aimed at specializing the models towards the code base of a given organization/developer further benefits their code completion capabilities” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=5ML4RUTC)) #5fb236
			- - “training a “generic” DL model on a code completion dataset featuring instances from 2,098 open source projects” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=LRMXUVMS)) #2ea8e5
			- - “further fine-tuned” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=AD9KYNVG)) #2ea8e5
			- - “organization-specific” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=ZL8WEMQ5)) #2ea8e5
			- - “loper-specific dataset” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=593K32PY)) #2ea8e5
			- - “100 different developers” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=UIPVJGS8)) #2ea8e5
			- - “242 DL models” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=CE5FG2WR)) #2ea8e5
			- - “predicting the next token the developer is likely to type” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=PI2WBQCI)) #5fb236
			- - “generation of co” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=QPAZHRMD)) #5fb236
			- - “set includes 159 GB of code” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=IHPX9UP5)) #5fb236
			- - “54M public GitHub” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=TSJVI5ZP)) #5fb236
			- - “generation of meaningful recommendations” ([“TOSEM-2024-0474_Proof_hi”, p. 2](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=3&annotation=3RINQNU9)) #5fb236
			- - “DL model Mb that has been trained on a large and generic code corpus which does not feature an” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=DDU9225Y)) #5fb236
			- - “Also in this case the performance has been assessed on the developer-specific test sets, representing future changes that org’s developers will implement.” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=S84U4XDP)) #5fb236
			- - “While the usefulness of these code recommenders is backed-up by empirical evidence [46], there is still room for improvement when it comes to their performance” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=XRSHZB6K)) #2ea8e5
			- - “One of the open questions when it comes to the adoption of DL-based code completion tools is whether their fine-tuning to the specific organization/developer using them may help in boosting performance.” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=3NK975JY)) #a28ae5
			- - “perform a further training step after the “generic fine-tuning”” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=SMCFLHFA)) #a28ae5
			- - “specializing the DL model to a given code base” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=II8DZ5YY)) #5fb236
			- - “fine-tune one of these models on their code base with the goal of deploying an in-house code r” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=43Q75H72)) #5fb236
			- - “two different levels of personalization” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=8MGHEAHF)) #a28ae5
			- - ““generic” DL-based” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=AAS3NY3N)) #5fb236
			- - “changes performed over time by developers who contributed to projects run by org” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=ZZDBNLRH)) #5fb236
			- - “training, evaluation and test set” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=WEEHJGIB)) #a28ae5
			- - “D-specific training se” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=C44YCSB5)) #5fb236
			- - “Finally, we put together all previously built developer-specific training sets, thus creating an organization-specific training set.” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=B6I6BH2D)) #a28ae5
			- - “Text-To-Text-Transfer-Transformer (T5)” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=GQS3WN8P)) #5fb236
			- - “∼2.3M instances” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=RVDAXBMY)) #5fb236
			- - “verify whether our findings hold for models of different size” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=LNGLL6C3)) #a28ae5
			- *Very good.*
			- - “since we must ensure that the training data used for the baselines does not feature code from the organization (org) we use as a case study” ([“TOSEM-2024-0474_Proof_hi”, p. 3](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=4&annotation=2HRY7H4J)) #a28ae5
			- *Exactly!!!!*
			- - “A very cheap fine-tuning performed on a developer-specific dataset significantly boosts the performance of DL-based code completion tools.” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=IVIYS88A)) #a28ae5
			- - “An organization-specific fine-tuning works better than a developer-specific training, and should be the obvious choice in most of cases” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=V4E6CBII)) #a28ae5
			- - “control on both the pre-training and the fine-tuning datasets, which is the reason why we train the T5 models from scratch.” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=UMXKJLH2)) #5fb236
			- - “the 100 developers responsible for the highest number of contributions (in our context, code additions) across all projects.” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=3THGYXQC)) #5fb236
			- - “100 developer-specific training, evaluation and test sets” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=JIEQV3CU)) #5fb236
			- - “100 developer-specialized DL-based code completion models adopting the T5small architecture.” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=3MJ3YIMN)) #5fb236
			- - “T5large architecture for the top-10 developers” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=LQY7K278)) #5fb236
			- - “To what extent personalizing a DL-based code completion tool to the specific developer using it boosts its performance?” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=XD5Z7M79)) #a28ae5
			- *That's is very interesting!*
			- - “leading to 100 new models based on the T5small architecture (plus 10 for the T5large architecture).” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=EYK5R68P)) #ffd400
			- *Why 100? I was expecting one T5_small and one T5_large model organization specific. Maybe, this will be clarified in Section 2. Anyhow, maybe here the paper needs to be clarified.*
			- - “To what extent personalizing a DL-based code completion tool to a software organization boosts its performance for individual developers of the orga” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=P2HEU6I3)) #a28ae5
			- *This also is very interesting.*
			- - “+5.29%.” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=8WLAMX5Q)) #a28ae5
			- - “7.87%.” ([“TOSEM-2024-0474_Proof_hi”, p. 4](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=5&annotation=D6YLBSGM)) #a28ae5
			- - “To what extent personalizing a DL-based code completion tool can boost its performance?” ([“TOSEM-2024-0474_Proof_hi”, p. 5](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=6&annotation=N3HTIETW)) #a28ae5
			- - “We did not observe an increase in performance comparable to the one provided by the two original specializations” ([“TOSEM-2024-0474_Proof_hi”, p. 5](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=6&annotation=J88MVXYP)) #a28ae5
			- - “(e.g., performing better than GitHub Copilot),” ([“TOSEM-2024-0474_Proof_hi”, p. 5](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=6&annotation=PNDVCQKT)) #a28ae5
			- *This is important by the way. Improve Copilot by considering the particular user or organization.*
			- - “to show the potential of personalization for the code completion task, especially for smaller models (i.e., 60M and 770M parameters), which are easier to deploy on local machines and cheaper to train for an interested com” ([“TOSEM-2024-0474_Proof_hi”, p. 5](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=6&annotation=US5UIPB3)) #a28ae5
			- - “developer-specific” ([“TOSEM-2024-0474_Proof_hi”, p. 5](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=6&annotation=PDJ47Y63)) #2ea8e5
			- - “organization-specific.” ([“TOSEM-2024-0474_Proof_hi”, p. 5](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=6&annotation=REIL5LZX)) #2ea8e5
			- - “contiguous tokens masked” ([“TOSEM-2024-0474_Proof_hi”, p. 5](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=6&annotation=9F68IWRD)) #5fb236
			- - “we exclude those being outliers in terms of number of modified files, i.e., impacting more than Q3 + 1.5 × IQR files, where Q3 is the third quartile and IQR is the interquartile range of the distribution of impacted files across all commits. This process narrowed down the initial set of 1,272,556 commits to 1,114,142 relevant commits.” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=QNHB3PPX)) #ffd400
			- *This step is not clear.*
			- - “Extracting Java methods featuring new code” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=PAARELGN)) #2ea8e5
			- - “We only focus on added lines and ignore the modified ones since our idea is to exploit these methods to generate training instances in which the code written by a specific developer (i.e., the added lines) is masked, and the model is in charge of predicting it.” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=GFQINISK)) #a28ae5
			- - “developer-specific (Section 2.2.1)” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=NKQ23KMB)) #2ea8e5
			- - “organization-specific (Section 2.2.2” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=U26YXSA7)) #2ea8e5
			- - “generic pre-training and fine-tuning datasets (Section 2.3).” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=HD5UYF9V)) #2ea8e5
			- - “Commits mining” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=272UPF72)) #2ea8e5
			- - “686 valid” ([“TOSEM-2024-0474_Proof_hi”, p. 6](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=7&annotation=X5TS8VGB)) #5fb236
			- - “1,148,324 Java methods with at least one new line implemented in a given commit.” ([“TOSEM-2024-0474_Proof_hi”, p. 7](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=8&annotation=MNIKPZ64)) #a28ae5
			- - “training/testing instances from each method extracted in the previous steps.” ([“TOSEM-2024-0474_Proof_hi”, p. 7](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=8&annotation=655TBZRH)) #5fb236
			- - “code completion instances derived from the most recent 500 commits are taken as test set,” ([“TOSEM-2024-0474_Proof_hi”, p. 7](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=8&annotation=MSJU53XK)) #5fb236
			- - “while the rest is split into 90% training and 10% validation” ([“TOSEM-2024-0474_Proof_hi”, p. 7](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=8&annotation=GL54H8NJ)) #5fb236
			- - “early stopping, with checkpoints saved every epoch, a delta of 0.005, and a patience of 5” ([“TOSEM-2024-0474_Proof_hi”, p. 9](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=10&annotation=NQIL37KA)) #5fb236
			- *[[pretrainedmodels]]*
			- - “a ▼ indicates statistically significant decreases in performance.” ([“TOSEM-2024-0474_Proof_hi”, p. 10](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=11&annotation=DJQJD5H9)) #ffd400
			- *Is there any motivation explaining such a decrease in performance?*
			- - “EM predictions with respect to the baseline (p-value <0.05);” ([“TOSEM-2024-0474_Proof_hi”, p. 10](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=11&annotation=UFLJHS8N)) #5fb236
			- - “developer-specific training tends to improve the model’s code completion capabilities. The average improvement is +5.29% in EM predictions” ([“TOSEM-2024-0474_Proof_hi”, p. 12](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=13&annotation=EJS5EC3W)) #ffd400
			- *Even though this is an interesting result, from a software engineering perspective, what it the tradeoff analysis? Does it make sense to perform such training process for only 5.29% of performance increase?*
			- - “he average increase in EM predictions is +7.87% over the baseline, with a mean OR of 7.86.” ([“TOSEM-2024-0474_Proof_hi”, p. 13](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=14&annotation=7GTM9NL3)) #ffd400
			- *See my previous comment. The tradeoff analysis is even more important in this case, considering the effort needed to train organization-specific models.*
			- - “Domain Coverage” ([“TOSEM-2024-0474_Proof_hi”, p. 14](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=15&annotation=76VK9957)) #2ea8e5
			- - “Vocabulary Coverage.” ([“TOSEM-2024-0474_Proof_hi”, p. 14](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=15&annotation=HBS5AMEQ)) #2ea8e5
			- - “Relevance of the Training Data.” ([“TOSEM-2024-0474_Proof_hi”, p. 15](zotero://select/library/items/T5XAY3BM)) ([pdf](zotero://open-pdf/library/items/4HURX3EZ?page=16&annotation=2U2PMTEK)) #2ea8e5
	- ### [[Comments]]
		- SUMMARY: The paper investigates the impact of personalizing deep learning (DL) models on code completion, focusing on two levels: organization-specific and developer-specific. Starting from a generic DL model pre-trained on a diverse code corpus from thousands of repositories, the authors fine-tune models using data from an entire organization and from individual developers. The results indicate that such fine-tuning significantly enhances prediction accuracy at both levels, compared to the generic model.
		- COMMENTS: This is a well-executed empirical study with a clear research question and thorough experimental setup. The authors have made thoughtful decisions regarding dataset construction and the use of two T5 model variants, which are justified within the paper. I found the paper interesting to read and the results valuable for advancing personalized code completion tools. I have very few minor suggestions to improve the already very well-written and structured paper:
			- Considering the multiple dimensions involved in dataset creation and model pre-training, an overview table in Section 2 would be helpful to summarize key aspects. This could include details on metrics, model configurations, and dataset characteristics.
			- The paper addresses the importance of personalizing DL models for code completion, given the potential for unprecedented code generation capabilities when models are tuned to specific coding patterns. Key details, such as training on over 2,000 projects and the incremental performance improvements (+5.29% and +7.87%) for developer-specific and organization-specific models respectively, demonstrate the empirical advantages of fine-tuning. However, a tradeoff analysis would strengthen the practical implications. For example, would these performance gains justify the required resources and training effort?
			-
		-
		-
	- ### [[REVIEWS/Notes]]
		- ((670bd7ce-a597-4c07-9d92-356dfdc43677))
			- **Interquartile Range (IQR)** is a measure of statistical dispersion, representing the range within which the middle 50% of your data lies.
			- To find the IQR:
				- **Q1 (First Quartile)**: The 25th percentile of the distribution, where 25% of commits modified fewer files than this value.
				- **Q3 (Third Quartile)**: The 75th percentile, where 75% of commits modified fewer files.
				- **IQR**: IQR=Q3−Q1\text{IQR} = Q3 - Q1IQR=Q3−Q1, which represents the range within which the central 50% of the number of modified files per commit lies.
			- ### 3. **Identify Outliers Based on the Upper Threshold**
				- Outliers in this context are commits that modified an unusually high number of files. To set a threshold for these:
					- Calculate Q3+1.5×IQRQ3 + 1.5 \times \text{IQR}Q3+1.5×IQR.
					- Any commit that modified more than this threshold number of files is considered an outlier.
	- ### YELLOW CONCERNS
	  background-color:: yellow
		- {{query (and [[ffd400]] [[TOSEM-2024-0474]] )}}
		  collapsed:: true
	- ### ❓️Questions
		- {{query (and [[question]] [[TOSEM-2024-0474]] )[[question]]}}
		  query-table:: true
		  query-properties:: [:block]