type:: [[REVIEWS]]
tags:: 
year:: 2024
venue::  [[TOSEM]]
full-title:: DDASR: Deep Diverse API Sequence Recommendation
date-start:: [[24-01-2024]] - 20:00
date-submitted:: [[08-04-2024]] 
external-links:: [https://mc.manuscriptcentral.com/tosem?URL_MASK=0d841ec919074d6fac2092df0b1b8bb6](https://mc.manuscriptcentral.com/tosem?URL_MASK=0d841ec919074d6fac2092df0b1b8bb6)
status:: [[DONE]] 
deadline:: [[01-03-2024]]
file:: ![TOSEM-2024-0012_Proof_hi.pdf](../assets/TOSEM-2024-0012_Proof_hi_1706122892823_0.pdf)
parent::
todoist:: https://app.todoist.com/showTask?id=7629221514

	- [[@02-23-TOSEM-2024-0012.pdf]]
	- ### [[Highlights]]
		- “developers often find it challenging to familiarize themselves with all APIs in the libraries.” ([“02-23-TOSEM-2024-0012.pdf”, p. 3](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=4&annotation=PI7GEURH)) #5fb236
		- “limited historical data, consequently diminishes the diversity of recommender systems” ([“02-23-TOSEM-2024-0012.pdf”, p. 3](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=4&annotation=3WJJWE3R)) #5fb236
		- “In this paper, we propose DDASR, a framework for recommending API sequences containing both popular and tail APIs.” ([“02-23-TOSEM-2024-0012.pdf”, p. 3](zotero://select/library/items/BCYRLKCX)) #00b036  
		  
		  *this is related to #bias*
		- “DDASR clusters tail APIs with similar functionality and replaces them with cluster centers to produce a pseudo ground truth. Moreover,” ([“02-23-TOSEM-2024-0012.pdf”, p. 3](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=4&annotation=A9IKBBET)) #f0ff00  
		  
		  *this is not clear*
		- “Results demonstrate that DDASR significantly achieves the best diversity without sacrificing accurac” ([“02-23-TOSEM-2024-0012.pdf”, p. 3](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=4&annotation=B2HDQTAG)) #a28ae5  
		  
		  *This is very much connected to [*[**PAPERS/2024-BiasInRSSEwithLLM**]**]**
		- “It is a challenging problem to find appropriate API sequences from the vast array of APIs according to developer requirements” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=ZERMS53B)) #a28ae5
		- “rReview53545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031042 Nan et al.API sequence:① Robot.init ② Robot.mousePress ③ Robot.mouseRelease ④ Robot.mouseWheelCode snippet: public class MouseActionsExample { public static void main(String[] args) { Robot robot = new Robot(); robot.mousePress(InputEvent.BUTTON1_MASK); robot.mouseRelease(InputEvent.BUTTON1_MASK); robot.mouseWheel(3);}}Query: Simulate mouse wheel scrolling to control the scrolling of a webpage.Fig. 1. An example query that needs to be addressed using an API sequence in Java.Moreover, fulfilling user requirements often necessitates more than just a single API. Developers typically need to look up API sequences to solve tasks. For instance, to address a query like ‘simulating mouse wheel scrolling to control the scrolling of a webpage,’ as shown in Fig. 1, a sequence of four APIs, including ‘Robot.init,’ ‘Robot.mousePress,’ ‘Robot.mouseRelease,’ and ‘Robot.mouseWheel,’ needs to be invoked. It is a challenging problem to find appropriate API sequences from the vast array of APIs according to developer requirements [50]. Recently, several approaches have been proposed to recommend APIs for developers. These approaches fall into two major categories: information retrieval-based approaches [24, 48, 62] that search for the most relevant solutions from the historical question repository, and deep learning-based approaches [12, 19, 40] that adopt the sequence-to-sequence (Seq2Seq) model to recommend API sequences generatively. Gu et al. [19] adopt an RNN encoder-decoder and Elnaggar et al. [12] adopt a Transformer encoder-decoder to obtain the results of the API sequence. Martin and Guo [40] apply CodeBERT [14] for the task due to the improved performance of Large Language Models (LLMs). However, existing API recommendation approaches usually recommend popular APIs, neglecting less frequently used ones. For example, deep learning-based approaches [12, 19, 40] remove infrequently appearing words from the vocabulary or treat them as <UNK> tags, making it challenging to recommend infrequently occurring APIs to developers.” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) #00b036
		- “APIs are clustered in a small portion at the head, while tail APIs, despite their low individual occurrence frequency, cover a large scale in quantity” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=H8EN3AHQ)) #5fb236
		- “The long-tail effect” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=6FP23XWZ)) #a28ae5
		- “Hence, learning the representation of tail APIs is essential for API recommendation. However, it is a challenge due to the sparse historical usage data of tail API” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=J3M7LQ2R)) #5fb236
		- “ignoring tail APIs can harm the diversity of a recommendation system” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=Z5WCYUT7)) #5fb236
		- “aggregate diversity” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=MQFTFUDR)) #5fb236
		- “how many different APIs are included in the result se” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=8BQGEA29)) #a28ae5
		- “boost the visibility of those rarely used APIs and increase the aggregate diversity of recommendation results, thus improving the health of the overall software ecosystem.” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=RAXNXSG9)) #a28ae5
		- “balance between the accuracy and diversity” ([“02-23-TOSEM-2024-0012.pdf”, p. 4](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=5&annotation=6GFRJNSZ)) #a28ae5
		- “To alleviate the sparsity, we cluster tail APIs with a similarity matrix and substitute them with cluster centers” ([“02-23-TOSEM-2024-0012.pdf”, p. 5](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=6&annotation=YL2RGPJ6)) #ffd400  
		  
		  *Check if it mentioned as possible threat to validity or bias itself.*
		- “The similarity matrix is built by calculating the similarity through the description and name of tail APIs” ([“02-23-TOSEM-2024-0012.pdf”, p. 5](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=6&annotation=PMGM5TAG)) #ffd400  
		  
		  *Why have you considered this as the most accurate way to do so???
		- “can recommend both non-tail and tail APIs” ([“02-23-TOSEM-2024-0012.pdf”, p. 5](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=6&annotation=GZUM4SBY)) #ffd400  
		  
		  *Check if they refer our investigation about popularity bias of recsys*
		- “Regarding the diversity metric coverage, DDSAR surpasses the baselines by an increase of 45.83% on the diverse Java dataset and 8.03% on the Python dataset.” ([“02-23-TOSEM-2024-0012.pdf”, p. 5](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=6&annotation=LMM6RZ7G)) #a28ae5
		- “We propose DDASR, a novel framework for recommending API sequences, striking a balance between accuracy and diversity.” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=A2Q9SZG4)) #a28ae5
		- “long-tail distribution in API sequence recommendation” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=2W46W42A)) #5fb236
		- “functional similarity” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=3T3N9FNI)) #ffd400
		- “related work in Section 2” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=RS2X9PU5)) #5fb236
		- “technical details of DDASR in Section” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=FZ4KCQJ4)) #5fb236
		- “experimental setup and the evaluation results in Section 4.” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=ITDLHK7T)) #5fb236
		- “We discuss threats to validity in Sectio” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=6T9ELRSX)) #5fb236
		- “MAPO” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=SS7DAKDU)) #5fb236
		- “information retrieval-based approaches struggle to capture the relevance of tail APIs” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=7&annotation=K2GFF3S9)) #a28ae5
		- “When applying LLMs, it is important to consider whether to fine-tune them during the training phase. For instance, models like TransRec [16], UNBERT [63], and LMRecSys [65], benefit from fine-tuning during the training phase. In contrast, models such as ChatGPT [11, 23, 39, 51] are designed to operate without the necessity for task-specific fine-tuning during the training phase.” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=8&annotation=RB5CCCE5)) #5fb236
		- “the diverse recommendation as an end-to-end supervised learning task and constructs ground truth labels to explicitly idealize the optimization target.” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=8&annotation=KI5NUGK4)) #5fb236
		- “Simpson’s Diversity Index and considered the evenness of the number of the items’ classes” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=8&annotation=SZ7MACWU)) #5fb236
		- “development requirements often consists of a series of APIs” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=8&annotation=NCNB5SVD)) #5fb236
		- “directly for recommendations.” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=8&annotation=QV9ZNXBP)) #5fb236
		- “Ziegler et al. [69] presented topic diversification to balance and diversify recommendation results” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=8&annotation=DDL4LGGB)) #5fb236
		- “Pseudo Ground Truth Building” ([“02-23-TOSEM-2024-0012.pdf”, p. 8](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=GJSWDXZU)) #5fb236
		- “Recently, researchers have begun exploring the use of LLMs for sequence recommendation tasks [15]. LLMs can be adapted to user data collection, feature engineering, and feature encoder [38]” ([“02-23-TOSEM-2024-0012.pdf”, p. 5](zotero://select/library/items/BCYRLKCX)) #00b036
		- “functional similarity among tail APIs with their descriptions and names.” ([“02-23-TOSEM-2024-0012.pdf”, p. 8](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=SBCGDDUV)) #5fb236
		- “lustering tail APIs and substituting them with cluster centers.” ([“02-23-TOSEM-2024-0012.pdf”, p. 8](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=HPN9SZYD)) #ffd400  
		  
		  *I think an explanatory example would improve the readability of the paper.*
		- “The Seq2Seq model is used to translate a given natural language query to a ranked list of possible API sequences.” ([“02-23-TOSEM-2024-0012.pdf”, p. 8](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=KAFJS9AN)) #5fb236
		- “ChatGPT [11, 23, 39, 51] are designed to operate without the necessity for task-specific fine-tuning during the training phase.” ([“02-23-TOSEM-2024-0012.pdf”, p. 5](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=U4NAFT9S)) #f0ff00  
		  
		  *Even though training is preferable*
		- “balance accuracy and diversity in the recommended API sequences” ([“02-23-TOSEM-2024-0012.pdf”, p. 8](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=BLAJG6A8)) #a28ae5
		- “When developers input a query, DDASR can recommend an appropriate API sequence solution” ([“02-23-TOSEM-2024-0012.pdf”, p. 8](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=C7QSL8SA)) #a28ae5
		- “Each fragment is a symbol or word representing the class or method name.” ([“02-23-TOSEM-2024-0012.pdf”, p. 8](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=9&annotation=8A5D866X)) #5fb236
		- “regularisation to enhance the diversity of recommendations.” ([“02-23-TOSEM-2024-0012.pdf”, p. 5](zotero://select/library/items/BCYRLKCX)) #00b036
		- “functionalities” ([“02-23-TOSEM-2024-0012.pdf”, p. 9](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=10&annotation=XJUELKZ6)) #a28ae5
		- “This pseudo ground truth includes both non-tail and tail APIs, ensuring comprehensive recommendations. In this context, we use 𝑇 𝐴 and 𝑁 𝐴 to represent tail APIs and non-tail APIs, respectively” ([“02-23-TOSEM-2024-0012.pdf”, p. 9](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=10&annotation=AJ355JCB)) #a28ae5
		- “Inspired by the word embedding technique [5], we recombine these fragments into complete APIs, allowing the model to better learn the calling relationship between APIs and reducing the computation of decoding APIs” ([“02-23-TOSEM-2024-0012.pdf”, p. 9](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=10&annotation=NK26BAHE)) #5fb236
		- “frequency analysis of API occurrences and distinguish between non-tail and tail APIs” ([“02-23-TOSEM-2024-0012.pdf”, p. 9](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=10&annotation=TH7NMHIU)) #a28ae5
		- “APIs with similar functions typically share analogou” ([“02-23-TOSEM-2024-0012.pdf”, p. 9](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=10&annotation=JGEK9352)) #a28ae5
		- “which contains both non-tail and tail APIs, is established by clustering tail APIs and substituting them with cluster centers.” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) #00b036
		- “LTR calculates a loss function and ranks these API sequences.” ([“02-23-TOSEM-2024-0012.pdf”, p. 6](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=10&annotation=MEKH7GU3)) #f0ff00  
		  
		  *Loss function of what?*
		- “we append the official descriptions of their parent APIs” ([“02-23-TOSEM-2024-0012.pdf”, p. 9](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=10&annotation=9L7QMTWR)) #a28ae5
		- “we calculate the functional similarity among tail APIs using both their documentation descriptions and names” ([“02-23-TOSEM-2024-0012.pdf”, p. 10](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=11&annotation=LB79PNHC)) #5fb236
		- “For the tail APIs, we mine their descriptions to facilitate functional similarity calculations. To achieve this, we download both official API documentation and third-party documents. Then we parse the HTML file of each API class to extract their descriptions. Generally, APIs with inheritance relationships tend to have similar functions. Therefore, we also mine descriptions of APIs within these inheritance relationships as a supplementary measure.” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=11&annotation=Q85ASPD2)) #f0ff00  
		  
		  *Everything occur at runtime?*
		- “Pseudo Ground Truth Building” ([“02-23-TOSEM-2024-0012.pdf”, p. 7](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=11&annotation=SIT4AFAT)) #f0ff00  
		  
		  *Why doing that?*
		- “Similarity Calculation.” ([“02-23-TOSEM-2024-0012.pdf”, p. 10](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=11&annotation=4YFYYS7S)) #ffd400  
		  
		  *This is a crucial step of the approach. The authors decided to introduce their similarity calculation, instead of using existing approaches that can calculate the similarity of third-party libraries.*
		- “The LLM encoder-decoder architecture is used in DDASR, with LLMs serving as the encoder to capture developer requirements, and a six-layer Transformer acting as the decoder.” ([“02-23-TOSEM-2024-0012.pdf”, p. 13](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=14&annotation=IG8UGRX3)) #5fb236
		- “to model the rankings of all API sequences generated” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=KLE8JISQ)) #5fb236
		- “increase diversity without sacrificing accuracy.” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=DF27J9JR)) #5fb236
		- “sequence probability distribution” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=SP5A564G)) #5fb236
		- “convex function” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=U28CMAU7)) #ffd400  
		  
		  *I think it is necessary to have an explanatory example to show the different phases of the approach in practice on concrete cases.*
		- “DDASR in comparison” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=2FWUFHSZ)) #5fb236
		- “improve diversity while mainta” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=BJQJNBJJ)) #5fb236
		- “help programmers” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=6MH5IEJ3)) #5fb236
		- “seven million natural language annotations, considered as queries, alongside API sequences” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=SC8CNIXW)) #ffd400  
		  
		  *What are these annotations?*
		- “more than one star.” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=X9GJCFSG)) #ffd400  
		  
		  *It's a not so strict constraint, isn't it?*
		- “To address this imbalance, we primarily rectified the records by adding parentheses where necessary to achieve symmetry.” ([“02-23-TOSEM-2024-0012.pdf”, p. 14](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=15&annotation=WP5KI8MI)) #ffd400  
		  
		  *This can introduce errors, even when the code is syntactically correct. For instance, it is not sure if the missing paranthesis should be added always at a prefixed position (e.g., end of the considered string).*
		- “query-APIseq pairs” ([“02-23-TOSEM-2024-0012.pdf”, p. 15](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=16&annotation=YG8PRPHE)) #ffd400  
		  
		  *It is necessary to better present how query-APIseq pairs are constructed for instance by referring to the example in Fig. 1 or some additional one specifically constructed as running example to be used to show all the building blocks of the proposed approach.*
		- “To adapt BIKER for API sequence recommendation, we make the necessary modifications to its open-source code.” ([“02-23-TOSEM-2024-0012.pdf”, p. 16](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=17&annotation=6ERSGJAY)) #5fb236
		- “DASR” ([“02-23-TOSEM-2024-0012.pdf”, p. 16](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=17&annotation=MW2BECBX)) #ffd400  
		  
		  *Why this? This is defined by the authors, right?*
		- “To restore the veritable accuracy, we also utilize the original ground truth as a reference, and in this case, the accuracy metrics are labeled as BLEU𝑂 , MAP𝑂 , and NDCG𝑂 .” ([“02-23-TOSEM-2024-0012.pdf”, p. 17](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=18&annotation=E53N46L4)) #ffd400  
		  
		  *This concept of veritable accuracy needs to be better explained and motivated.*
		- “(a) Comparison of top-k results under different accuracy metrics” ([“02-23-TOSEM-2024-0012.pdf”, p. 18](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=19&annotation=LUFTZ3YP)) #ffd400  
		  
		  *Which configuration of DDASR has been used to produce the shown accuracy results?*
		- “different architectures” ([“02-23-TOSEM-2024-0012.pdf”, p. 18](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=19&annotation=PBWLY6ER)) #ffd400  
		  
		  *Explain that with architecture you are referring to different LLMs*
		- “We also investigate how DDASR affects accuracy when improving diversity.” ([“02-23-TOSEM-2024-0012.pdf”, p. 19](zotero://select/library/items/BCYRLKCX)) ([pdf](zotero://open-pdf/library/items/I879APBF?page=20&annotation=JDSC9SDJ)) #5fb236
	- ### [[Comments]]
		- SUMMARY: Recommending API sequences is indeed an interesting problem, and the authors decided to go further by investigating popularity bias that can occur when less commonly used API sequences are recommended to developers. The authors overcome limitations related to tail APIs by proposing an approach that leverages cluster-based techniques to enhance the recommendation of tail APIs.
		  
		  COMMENTS: Overall, the paper is well-written and structured; I have some concerns related to presentation issues, which can be fixed by adding clarifications and details that are currently missing; without them some parts of the paper are difficult to understand. 
		  
		  Similarity Calculation: Many approaches exist to calculate the similarity of APIs (e.g., [1,2,3]), but the authors neglected them and introduced a new similarity function instead. It is important to discuss the rationale behind this choice.
		  
		  Pseudo-ground truth: The authors should clarify the rationale behind the construction of pseudo-ground truth. I understand it is related to the usage of clustering techniques. However, it is necessary to improve the motivation by showing concrete examples of what happens if the original ground truth is used instead.
		  
		  Running examples are also needed to explain the data preprocessing phases. For instance, to see concrete cases of query-APIseq pairs, the reader has to wait for the evaluation sections. It is necessary to show examples of input data and corresponding encoding earlier in the paper. The way user queries are matched with query components in the dataset needs additional description. Thus, it is necessary to present better how query-APIseq pairs are constructed, for instance, by referring to the example in Fig. 1 or some additional ones specifically constructed as a running example to show all the building blocks of the proposed approach.
		- The "seven million natural language annotations, considered as queries, alongside API sequences" mentioned in section 4.2 also needs clarification.
		- In the same section, it is necessary to elaborate on the choice of adding missing parentheses. In particular, the authors should discuss the potential risks or drawbacks of the proposed fix, including its impact on code accuracy and potential error introduction.
		- Still, in Sec. 4.2, the authors write, "These are extracted by mining Java projects on GitHub that have garnered more than one star." This is a too-flexible constraint, which risks introducing in the dataset toy projects. This is a relevant threat to validity.
		- The authors introduced DASR for evaluation purposes. It is necessary to add clarification behind such a choice, which is not adequately motivated and suddenly appears in the evaluation section.
		- The veritable accuracy concept mentioned in section 4.5 and used later while discussing the results needs to be better defined and possibly explained with examples.
		- When discussing Fig. 6, please clarify which configuration of DDASR has been used to produce the shown accuracy results
		- In section 4.6.2, it is necessary to clarify that when mentioning the "different architectures" of DDASR, you are referring to the usage of different LLMs.
		  
		  REFERENCES
		  
		  [1] Phuong Thanh Nguyen, Juri Di Rocco, Davide Di Ruscio, Massimiliano Di Penta: CrossRec: Supporting software developers by recommending third-party libraries. J. Syst. Softw. 161 (2020)
		  
		  [2] A. Ouni, R.G. Kula, M. Kessentini, T. Ishio, D.M. German, K. Inoue
		  Search-based software library recommendation using multi-objective optimization Inf. Softw. Technol., 83 (C) (2017), pp. 55-75
		  
		  [3] Ferdian Thung, David Lo, Julia Lawall:
		  Automated library recommendation. WCRE 2013: 182-191
	- ### [[REVIEWS/Notes]]
	- ### ❓️Questions
		- {{query (and [[question]] [[TOSEM-2024-0012]] )[[question]]}}
		  query-table:: true
		  query-properties:: [:block]
-