68 lines
8.8 KiB
Markdown
68 lines
8.8 KiB
Markdown
collapsed:: true
|
||
type:: [[REVIEWS]]
|
||
tags::
|
||
year:: 2026
|
||
venue:: [[ICSE]]
|
||
full-title:: Recommending Relevant Classes for Infrequent API Classes
|
||
date-start:: [[17-09-2025]] - 11:46
|
||
date-submitted::
|
||
external-links::
|
||
status:: [[DONE]]
|
||
deadline-submission::
|
||
file:: [[@Recommending Relevant Classes for Infrequent API Classes]]
|
||
parent::
|
||
todoist:: https://app.todoist.com/app/task/2919-recommending-relevant-classes-for-infrequent-api-classes-6cVHmvX54fQVgpV8
|
||
|
||
- ### [[Highlights]]
|
||
- ### [[Comments]]
|
||
- #.tabular
|
||
- ### Paper summary
|
||
- The paper proposes APIrel, an approach to mitigate bias when recommending API classes, even in the absence of frequent usage patterns in code. APIrel combines mined patterns and document features to train a classifier to predict relationships between infrequently used or new APIs. The paper evaluates APIrel by considering 5 libraries and shows promising results.
|
||
- ### Strengths
|
||
- + Relevant long-tail problem in API recommendation, where most APIs are rarely used and thus difficult to relate or recommend
|
||
- + Interesting attempt to move beyond client-based mining, by relying on API documentations
|
||
- ### Weaknesses
|
||
- Presentation is confusing and poorly structured, with forward references, missing definitions, and mixing of problem and solution
|
||
- The tool is not accessible, making reproducibility and verification impossible.
|
||
- Evaluation is limited and lacks user studies (e.g., developers' feedback on relevance predictions).
|
||
- ### Detailed comments for authors
|
||
- Novelty: The idea of leveraging document-based features rather than usage-based mining for inferring relationships among APIs is interesting and reasonably novel. However, novelty is somewhat undermined by vague distinctions from prior work. The “two lines of research” (document vs. client-based) are repeatedly mentioned but never formally defined with illustrative examples.
|
||
- Rigor: The technical rigor is compromised by the lack of concrete examples to clarify design choices, assumptions, and implementation steps. Several parts are abstract and theoretical without operational grounding (e.g., the use of “patterns,” “mined rules,” and model training steps).
|
||
- Relevance: The addressed problem is highly relevant for developers who need better tooling to understand or recommend infrequent APIs. However, the current presentation makes it difficult to assess the actual impact or applicability of the method.
|
||
- Verifiability & transparency: Broken link to the repository makes it impossible to validate or test the tool. Moreover, it is unclear how the tool is to be used, what the inputs are, or how results are visualized or interpreted. The data collection process (e.g., how patterns are mined or APIs extracted) is not reproducible due to insufficient procedural detail. The fact that SearchCode system is no longer online does not help in this regard.
|
||
- Presentation: This is the main problem of this paper. The writing is often unclear and the reader is supposed to go trhough the text many times to get the meaning of the presented concepts. The motivation is not well separated from the solution, especially in Section 2. I suggest adding explanatory examples, without them many parts of the paper hare hard to follow. Moreover, firgure and tables are not always correctly references and there are too many forward references. For instance section 7 is mentioned in the introduction.
|
||
- Detailed comments:
|
||
- page 1: "As there are many API classes, it is challenging to identify relevant API classes for a given API class." - Sentence is poorly written and redundant. Please rephrase to clearly state what the challenge is: e.g., is the difficulty in discovering dependencies, usage patterns, or semantic relationships?
|
||
- page 1: "APIrel uses mined patterns as seeds and compares API documents to build labeled data." - What exactly are the “mined patterns”? How are they extracted, from what sources, and at what stage of the process?
|
||
- page 1: "Given the documents of two API classes, our trained model can predict whether they are relevant." - Please clarify what type of documents are used here (e.g., Javadoc, README, class summaries). This needs to be made explicit.
|
||
- page 1: "As introduced in Section 7," - This is an overly early forward reference. Consider avoiding such jumps in the introduction. Core ideas should be self-contained in the early sections.
|
||
- page 1: "For instance, API documents rarely mention relevant API classes, especially when such classes are infrequent." - Statements like this should be supported by concrete examples to make the issue tangible to the reader.
|
||
- page 1: "For instance, researchers use Apriori algorithms to mine the relevant libraries." - This usage of Apriori is unclear. Specify what is mined, how support is defined, and what the items represent in this context.
|
||
- page 1: "APIrel. It is the first approach that infers relevant classes from documents without predefined templates." - What kinds of documents are used? This should have been clarified much earlier. The lack of early context makes it difficult to evaluate the novelty.
|
||
- page 2: "The document of HSSFWorkbook." - Ambiguous phrase. What is meant by "the document"? Please use consistent terminology throughout (e.g., API class description, Javadoc, etc.).
|
||
- page 2: "From the clients" - Please define what is meant by "clients" in this context. Client projects? Users? Code examples?
|
||
- page 2: "Table 3" - Typo. Likely meant to reference “Table 1.” Please correct.
|
||
- page 2: "SearchCode" - This resource appears to be offline. If it’s critical to the example or dataset, either replace it with an active resource or explain how the necessary data was retained.
|
||
- page 3: "Definition 1. The relevant API classes of an API class are classes that can be called together to implement functionalities." - The phrase “called together” is vague. Please formalize this definition and clarify what is meant by "calling a class."
|
||
- page 3: "To infer hidden patterns, we reduce the inference of infrequent relevant API classes into a classification problem..." - This paragraph needs a worked-out example to clarify the input/output of the classification function. The formal description is too abstract alone.
|
||
- page 4: "Line 1 builds a dictionary from mined patterns..." - This would be more understandable if tied to a running example. Refer to specific APIs from the motivating use case to make the logic concrete.
|
||
- page 4: "Line 6 checks whether the c1 and c2 classes appear in our dictionary..." - Please elaborate on this logic with an example. Without referring back to a real code snippet or API class, it remains abstract.
|
||
- page 6: "API documents in Table 3" - Table 3 appears to repeat library names listed elsewhere and does not clearly present documents. Please clarify what is meant by "API documents" here.
|
||
- page 6: "The repository of SearchCode has millions of projects..." - It's unclear how SearchCode contributes to the methodology. Is it used for mining patterns, extracting features, or labeling data?
|
||
- page 6: "4.2 No Baseline Statement" - This section reads more like an appendix to Related Work. Consider moving it and restructuring Section 2 to present motivation and prior work in a more integrated, example-driven fashion.
|
||
- page 6: "As introduced in Section 7," - Section 7 is referenced repeatedly and contains important information. Consider moving this section earlier in the paper (e.g., after the introduction).
|
||
- page 6: "The first line of approaches analyzes documents with templates..." - The distinction between the two families of approaches (document-based and client-based) is referenced frequently. Please clarify this with illustrative examples.
|
||
- page 7: "The accuracy" - Accuracy of what exactly? Prediction of relevant classes? Clarify what the ground truth is and whether human validation was involved.
|
||
- page 10: "We evaluated APIrel on five popular libraries..." - Given the subjective nature of what counts as “relevant,” evaluation should involve developers to verify the usefulness of predictions.
|
||
- Questions
|
||
- Q1: What exactly are the “documents” used as input by APIrel? Are they structured Javadoc descriptions, source code comments, online documentation, or something else?
|
||
- Q2: How is “relevance” between two API classes defined and operationalized during model training and evaluation? What constitutes a positive or negative label? Can such labels automatically assessed without any human involvment?
|
||
- ### [[REVIEWS/Notes]]
|
||
- ### YELLOW CONCERNS
|
||
background-color:: yellow
|
||
- {{query (and [[ffd400]] [[ICSE2026-paper2919]] )}}
|
||
collapsed:: true
|
||
- ### ❓️Questions
|
||
- {{query (and [[question]] [[ICSE2026-paper2919]] )[[question]]}}
|
||
query-table:: true
|
||
query-properties:: [:block] |