logseq/pages/API_recommendation-ASEJ.md

type:: REVIEWS tags:: year:: 2023 venue:: ASE JOURNAL full-title:: Prompt Enhance API Recommendation: Visualize the User’s Real Intention behind this Query date-start:: 22-11-2023 - 11:09 date-submitted:: external-links:: status:: DONE deadline-submission:: 13-12-2023 file::

- [[Highlights]]
	- #+BEGIN_IMPORTANT
	  ((65806a9b-9b36-4a71-9699-2506a39906a6))
	  ((65806c18-109b-432b-bd7d-afad04d26996))
	  ((65806c26-cd73-4805-b857-0f62fbc05abf))
	  #+END_IMPORTANT
	-
	- ((65806b91-a7b2-41f3-9e70-86002423ecbd))
		- Which one?
	- ((65806c53-5faa-487f-bde7-5225eb1280a1))
		- Maybe concludes the paper?
	- ((6581693c-af27-4541-825f-e4667460fe08))
		- What does it mean?
	- ![image.png](../assets/image_1702978870292_0.png){:height 576, :width 545}
	- ![image.png](../assets/image_1702982495859_0.png){:height 425, :width 546}
		- ((65817399-f4c1-4531-a55f-02e68edf843a))
			- A better description is needed. It's not clear how the process works, the difficulties encountered while creating the training dataset etc. What't the technology that has been employed for sypporting the search? I guess SO posts are indexed to support the query phase?
	- ((658173f8-cd24-4041-997a-f94ca7e1e01e))
		- Why not use SO dumps?
	- ((6581744a-456c-4e2b-82e1-e46e47b0d57b))
		- It does not parse.
	- ((6581762a-391d-4643-84cf-62edb7b07070))
		- What's the prompt template is not clear yet.
	- ((65817824-f565-499d-bae7-d87c05d031cd))
		- This is really error-prone.
	- ((65817c5d-f829-4546-a84f-78847212edfa))
		- What does it mean? Experimental results are supposed to answer research questions ...
	- ((65817ea6-ba70-4448-b573-09018836c965))
		- This is obscure ...
	- ((65817f84-e452-490a-9750-fae8dafc9c54))
		- What does "how far" mean?
	- ((6581964a-a0f2-4fae-91ba-0cb4a8d976e7))
		- The concept of prompt templates is not clear. It should be introduced in the paper before its adoption by means of some illustrative examples.
	- ((6581a132-73c7-4223-ac39-9f77a9eaa1af))
		- Not clear to me.
	- ((6581a161-6e54-4eb5-98f6-a4c2e5fd6e6c))
		- Where?
	- ((6581aadc-49c8-4b2e-8873-083ea4fc2e09))
		- The comparison is asectic, without any qualitative discussion of the obtained results.
	- ((6581ab7a-5e13-413b-bb42-12d26fd69094))
		- No evidence of this.
	- ((6581b100-102b-408f-9870-89f27c4bd18b))
		- I'm not sure this has been put in this way since the beginning of the paper.
	- ((6581b13b-e893-4367-b1c7-0427d95dffe8))
		- It does not parse.
	- ((6581b160-b575-46dd-a0a2-a03d70156c96))
		- By the way, it is not clear if the approach outperforms existing baselines like FOCUS etc. specifically designed for API recommendation!
	- ((6581b1f6-e1d0-4ee0-afcf-15202ac9ea5f))
		- This is not evident
- [[Comments]]
	- The paper introduces PTAPI, a novel API recommendation approach that leverages StackOverflow posts to enhance user queries, refining requests and capturing user intents more effectively. While the paper addresses an interesting and relevant problem, it requires significant improvements for acceptance. Key concerns include a lack of clarity, detailed explanations, and supporting evidence to reinforce contributions and findings. Several issues have been identified:
		- **Comparison with Baselines:** The paper focuses on API recommendation but fails to compare PTAPI with existing baselines in the field, such as MAPO, GAPI, or FOCUS. A comparison with these approaches is essential to establish the relevance and effectiveness of PTAPI.
		- **Prompt Template and Prompt Learning:** The proposal to introduce a prompt template for API recommendation lacks clarity. A comprehensive explanation of the approach, including prompt template creation, is necessary for reader understanding. The concept of prompt learning is introduced abruptly; its relevance should be clarified from the outset. It remains unclear if the combination of the prompt template with the user description is a one-time process or can undergo several iterations.
		- **Language Model Construction:** The process of constructing a language model for similarity calculation requires a more in-depth explanation. Details on technology used, challenges faced in dataset creation, and the method supporting the search, especially concerning Stack Overflow posts, are crucial for clarity.
		- **Data Extraction and Regular Expressions:** The choice to extract data from Stack Overflow HTML pages rather than using dumps needs justification. Additionally, the use of regular expressions to extract hyperlinks and API names may be error-prone and should be addressed cautiously.
		- **PTAPI framework description:** Figure 3 requires a detailed description. Currently, it is presented as a sketch, lacking necessary details for readers to fully comprehend its content.
		- **Evaluation Concerns:**
			- RQ1 needs rephrasing for clarity, aligning with the standard that experiments answer research questions. The meaning and interpretation of "how far" in the context of the experiment remain unclear.
			- The reference to "top 10 most similar problems as templates" requires further explanation.
			- The term "parameter" lacks context and clarification regarding its specific mention in the paper.
			- Comparison results with BIKER and BRAID lack qualitative discussion, necessitating additional insights into the obtained results.
			- Comparison with existing baselines like FOCUS is not addressed, which is crucial for understanding the novelty and effectiveness of PTAPI.
			- The claimed effectiveness of PTAPI at both the method and class levels needs clearer supporting evidence in the experimental evaluation.
	-
- [[REVIEWS/Notes]]