logseq/pages/TOSEM-2023-0432.md

type:: REVIEWS tags:: year:: 2024 venue:: TOSEM full-title:: APIDocBooster: An Extract-Then-Abstract Framework Leveraging Large Language Models for Augmenting API Documentation date-start:: 04-01-2024 - 21:33 date-submitted:: external-links:: status:: DONE date-submitted:: 27-01-2024 deadline-submission:: 17-01-2024 file:: parent:: todoist:: https://app.todoist.com/showTask?id=7513237246

- ### [[Highlights]]
	- #+BEGIN_IMPORTANT
	  ((65aa8533-9ee5-477f-8f7a-096c2b45b9a7))
	  ((65aa9399-d50a-43aa-ac24-319efdebdb11))
	  ((65ae6939-d56b-459a-a8db-7459ee83abb7))
	  ((65b53353-51f4-4e98-a213-a533458e4d31))
	  #+END_IMPORTANT
	- ((65aa93cf-854a-4d0c-9d3d-6150dcf45874))
		- So it seems, first it extracts summaries without any lenght restrictions, and then produces concise summaries.
	- ((65aa89e6-79df-4e82-b742-60e9074e3e4f))
		- **Update summarization** is to generate summaries to API documentation
		- Extractive summarization selects **insights sentences** about the API of interest from externar sources
			- Insights sentences provide API-relevant insights not covered in the API documentation
			- However extractive summarization tends to generate redundant sentences, thus readability is affected
	- ((65aa9289-84d9-4c4f-83b9-a143b338e35a))
	- ((65aa92eb-e333-4add-a6c6-92712f4ee4bb))
		- GPT COST
	- ((65aa93f9-c387-4b64-8513-7bcccf3aa3da))
		- The lenght limitation aspect is not clear [[question]]
	- ((65aa9490-1b18-4f22-88ab-757a93a34762))
		- How do you ansure that the input for this phase does not reach the 8k limit? [[question]]
	- ((65ae68c0-c2e0-438d-aa13-a2e3af5020e3))
		- What can you say about such metrics that have been shown to be not optimal in many situations? [[question]]
	- ((65ae696d-687c-45d7-8895-406903895f7a))
		- ((65ae6a02-61ce-4073-80b9-a8aab3e06777))
			- What is the property of tanh? [[question]]
	- ((65b533fb-538b-4ae9-8219-3c5b9eeed815))
		- How can you include version information of the API under analysis? [[question]]
			- That's fine:
				- ((65b544e3-9d23-44cb-b219-be2d5522a526))
			- but it does not answer my question about version of the target API.
	- ((65b56e8b-ee70-46cc-96a8-f86e4f380c17))
		- This seems overlaping with what is presented in Sec. ((65b56eb1-cd47-44c1-b734-64d180583793))
		- ((65b56eb1-cd47-44c1-b734-64d180583793))
			- I would move sec 2.3 in Section 4. Mixing details about part of the performed experiments before the presentation of the approach is a bit confusing.
	- I found the second figure shown in README file of [autumn-city/APIDocBooster: Revising my paper and submitting to ICSE (github.com)](https://github.com/autumn-city/APIDocBooster) very informative. I suggest to put it also in the paper while presenting the data collection and labeling process.
	-

• • Comments

    • The paper introduces APIDocBooster, an automated approach designed to augment API documentation by extracting additional sentences from external sources, such as StackOverflow and YouTube. Semantically distinct from the original API documentation, the extracted material is used to augment the documentation with extractive summaries. The paper addresses an interesting and relevant topic, exhibiting well-written content, and overall, it is well-structured. However, I have some suggestions to enhance clarity and readability:
      • Consider moving the presentation of the Empirical study from Section 2.3 to Section 4. This would help avoid confusion by separating technical details from investigative aspects.
      • The second figure in the README file (autumn-city/APIDocBooster) provides valuable information. Consider incorporating it into the paper during the data collection and labeling process discussion.
      • On page 4, line 11, improve the sentence "Extract Phase allows input without length limitations" to clarify the aspect of length limitation, which is currently unclear.
      • In the threats to validity section, discuss the usage of ROUGE metrics and their known performance issues in certain scenarios (http://arxiv.org/abs/2312.15475).
      • Address the critical limitation of the approach related to the management of API versions. Specifically, highlight that there is no guarantee that retrieved sentences align with the specific version of the target API. This lack of version control may result in imprecise and potentially outdated information in the augmented documentation, e.g., the target API has a newer version than the shown documentation.

  • REVIEWS/Notes

  • ❓️Questions

    • {{query (and question TOSEM-2023-0432 )question}} query-table:: true query-properties:: [:block]