83 lines
7.4 KiB
Markdown
83 lines
7.4 KiB
Markdown
type:: [[REVIEWS]]
|
|
tags::
|
|
year:: 2024
|
|
venue:: [[SANER]]
|
|
full-title:: Cross-project software defect prediction based on generative adversarial network-enhanced graph neural networ
|
|
date-start:: [[15-11-2023]] - 22:36
|
|
date-submitted:: [[09-12-2023]]
|
|
external-links::
|
|
status:: [[DONE]]
|
|
deadline-submission:: [[09-12-2023]]
|
|
file:: 
|
|
|
|
- [[Highlights]]
|
|
- #+BEGIN_IMPORTANT
|
|
((656b52cf-2992-4e98-b033-1f627fa51465))
|
|
((656b7bef-29f8-4a65-9f84-6d7e234800c5))
|
|
- Interesting to see what are the features used to characterize projects
|
|
((656c6076-eef6-4ae9-b98c-f35dc9a90bb8))
|
|
((656c60d7-3b50-470f-8d41-4f36d1c26f31))
|
|
#+END_IMPORTANT
|
|
- ((6573334a-93e2-4bfd-9cbc-97b49a8726c6))
|
|
- The division in subgraph structures is done for what?
|
|
- [[question]] What is it the community division algorithm?
|
|
- ((65734553-2a35-45b3-845e-059344acebf2))
|
|
- Why and how subgraphs are obtained?
|
|
- ((656ce737-2cd4-42ce-8110-b45d6059673f))
|
|
- ==THIS IS THE KEY==
|
|
- ((656ce7ac-b2e3-4a83-a467-f892a7949d4c))
|
|
- Why respectively, when you are talking about node feature vectors only?
|
|
- An explanatory example is missing to understand the feasibility, strengths and limitations of the proposed approach
|
|
- Fig 4 is not effective in this direction. It is not even clear what graphs encode in that figure.
|
|
-
|
|
- ((656ced59-8613-4e84-bd37-2fa299048aea))
|
|
- Which chapter?
|
|
- ((656cfa50-1fa2-4cce-98f9-04aabaa6ff05))
|
|
- Details about the execution environment/setting is also required.
|
|
- ((656cfafa-d344-4e36-9ce0-454feaa5e614))
|
|
- How did you select them?
|
|
- In general it is necessary to describe properly the experiment settings and justify them with respect to research questions that authors wanted to answer by means of the performed experiments.
|
|
- ((656cfb59-6774-46c3-bbd9-376481ba7855))
|
|
- For all the projects the feature value is always 20. Does it mean that the results do not change if such value is modified? Having a distribution of projects also with respect to such a dimension can be interesting to investigate.
|
|
- ((656cfce6-86d0-4ff2-84bb-46d17611e38e))
|
|
- what is it?
|
|
- ((656cfd2e-f621-441d-86de-ba520b181053))
|
|
- What's the criteria for such a selection?
|
|
- ((656cfdd0-4622-4406-80ff-601b74818c49))
|
|
- It is not clear how the approach works for projects very different in terms of application domains. Another qualitative discussion to be done is how the system can predict fault when the source and target projects are developed e.g. with different styles, patterns, technologies, third-party libraries, etc.
|
|
- ((656cfe55-696b-43a1-8698-de48258081c0))
|
|
- #+BEGIN_WARNING
|
|
Check if all of them are from the same ecosystem.
|
|
#+END_WARNING
|
|
-
|
|
- [[Comments]]
|
|
- Notes
|
|
- The paper is not always easy to read. It is not self-contained and it is necessary to go trough it several times. The usage of the employed algorithms and technologies in general is not justified.
|
|
- Many times the authors refer to the manuscript as a chapter.
|
|
- Starting from the abstract, the authors use terms and concepts that are not introduced making the sentences hard to understand.
|
|
- Overall, I suggest seriously revise the paper with the aim of clarifying the presentation of both the problems and of the essentials of the proposed solution.
|
|
- The related work needs also major revision, since it presents two techniques (i.e., GNN and GAN) that can be employed for defect prediction, without instead discussing and presenting the limitations of existing approaches for cross-project defect predictions. Authors can consider existing systematic studies and literature reviews that have been published on this topic.
|
|
- The writing is not accurate, the paper requires major editing ti put it in the right shape even from the style point of view.
|
|
- The evaluation section does not introduce any research questions that the authors wanted to answer by means of the performed experiments. The comparison that has been done by considering the different metrics is not target the important aspect of the ideal evaluation i.e., if the approach can really predict defects. Assessing the accuracy of the proposed approach should involve humans that can check if defects that are predicted by the methods really occur.
|
|
- Review
|
|
- The paper is about an interesting topic. However, I have many concerns on the paper that are related to the following issues:
|
|
- The paper's writing lacks precision, necessitating significant editing for stylistic improvements and overall coherence. The paper lacks self-containment, necessitating repeated readings. The justification for employed algorithms and technologies needs to be revised. Refraining from referring to the manuscript as a chapter and introducing terms without prior explanation would enhance overall readability.
|
|
- Authors mention early in the paper that projects are encoded as graphs and subsequently divided into subgraphs without clarifying how such encoding is done, why the division in subgraphs is performed, and what the goal is.
|
|
- A substantial revision is recommended to enhance the clarity of problem statements and the essential aspects of the proposed solution. Furthermore, the related work section needs a major revision. It introduces GNN and GAN techniques for defect prediction without discussing the limitations of existing cross-project defect prediction approaches. Authors are advised to consult existing systematic studies and literature reviews on this topic.
|
|
- Explanatory examples are needed to convey better how the method works. Fig. 4 is not effective in this direction. It does not even clarify what the shown graphs encode in that figure.
|
|
- The evaluation section of the paper needs to explicitly outline the research questions the authors aimed to address through their conducted experiments. The comparative analysis, which incorporates various metrics, needs to precisely target a crucial aspect of an ideal evaluation, i.e., the ability of the approach to predict defects genuinely. Assessing the accuracy of the proposed methodology should extend beyond quantitative metrics and involve human evaluators who can validate whether the predicted defects actually manifest in practice.
|
|
- The selection of the six projects needs to be justified, and in general, the paper has to describe the experiment settings and explain them concerning the research questions that the authors wanted to answer through the experiments that were performed.
|
|
- In Table I, the feature value is always 20 for all the projects. Does it mean the results do not change if such a value is modified? Having a distribution of projects also with respect to such a dimension can be interesting to investigate.
|
|
- How the approach works for different projects in terms of application domains needs to be clarified. Another qualitative discussion is how the system can predict fault when the source and target projects are developed, e.g. with different styles, patterns, technologies, third-party libraries, etc.
|
|
-
|
|
-
|
|
- [[REVIEWS/Notes]]
|
|
- #+BEGIN_IMPORTANT
|
|
((656c6852-42eb-4df8-a10a-60ae28fe8cf9))
|
|
This is related to what Alfonso was saying about the adoption of ML techniques to deal with the problem of graph isomorphisms
|
|
((656ce3cc-d092-4aab-b8ba-b2131f0ec8bf))
|
|
Maybe to have a look at for the work aiming at distinguishing data generated by ChatGPT?
|
|
#+END_IMPORTANT
|
|
-
|
|
-
|
|
- {{embed [[SANER2024-REVIEW-INSTRUCTIONS]]}} |