142 lines
21 KiB
Markdown
142 lines
21 KiB
Markdown
type:: [[REVIEWS]]
|
||
tags::
|
||
year:: 2024
|
||
venue:: [[TOSEM]]
|
||
full-title:: SOGPTSpotter: Detecting ChatGPT-Generated Answers on Stack Overflow
|
||
date-start:: [[06-12-2024]] - 20:59
|
||
date-submitted:: [[06-12-2024]]
|
||
external-links::
|
||
status:: [[DONE]]
|
||
deadline-submission:: [[26-09-2024]]
|
||
file:: [[@SOGPTSpotter: Detecting ChatGPT-Generated Answers on Stack Overflow]]
|
||
parent::
|
||
todoist:: https://app.todoist.com/app/task/tosem-2024-0498-reviewer-agreed-6W4xfPjp6MwQg5cg
|
||
|
||
- ### [[Highlights]]
|
||
- **# Annotazioni **
|
||
|
||
(6/12/2024, 21:19:11)
|
||
- “human” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=FSIY92BV)) #2ea8e5
|
||
- “exchange of knowledge and providing solutions to programming challenges” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=X3R2FH45)) #5fb236
|
||
- “SOGPTSpotter” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=GSBRS96N)) #a28ae5
|
||
|
||
()
|
||
- “Recently, there has been a significant increase in the number of answers generated by ChatGPT, which can lead to incorrect and unreliable information being posted on the site.” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=8C3J2TAG)) #5fb236
|
||
- “detecting whether a post is ChatGPT-generated remains a challenging task” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=IQXGQCDE)) #a28ae5
|
||
- “answers” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=LYBD7TAR)) #2ea8e5
|
||
- “ChatGPT answers” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=43D6737H)) #2ea8e5
|
||
- “We also conducted a real-world case study on Stack Overflow” ([Ma et al., p. 1](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=2&annotation=ZH4ID7QI)) #5fb236
|
||
- “Therefore, maintaining the quality of these answers is essential to ensure the reliability of the platform and its continued ser” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=RBBB35M7)) #5fb236
|
||
- “Reinforcement Learning from Human Feedback” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=AD36DRXQ)) #5fb236
|
||
- “software enginee” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=H3AW6MGU)) #ffd400
|
||
|
||
*Please replace arxiv references with the corresponding published version (if any).*
|
||
- “dramatic reduction in traffic to Stack Overflow” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=SHFZX5B2)) #5fb236
|
||
- “Another major impact is ChatGPT-generated answers to Stack Overflow posts.” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=BSNMUB2W)) #a28ae5
|
||
- “However, the site soon banned all answers produced by the model due to concerns about the accuracy of the information provided.” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=5SBKFKXQ)) #a28ae5
|
||
- “"substantially harmful"” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=G9AFVC3I)) #a28ae5
|
||
- “The main issue lies in the fact that while the responses generated by ChatGPT often appear to be plausible, they are frequently incorrect.” ([Ma et al., p. 2](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=3&annotation=KRY2W9IC)) #e56eee
|
||
- “non-existent command, git delete commit,” ([Ma et al., p. 3](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=4&annotation=57QXLB5C)) #a28ae5
|
||
- “t. This information is incorrect and could lead to confusion for users who follow these steps” ([Ma et al., p. 3](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=4&annotation=9PCIC4UV)) #e56eee
|
||
- “ack the necessary expertise to validate the accuracy of these responses before posting them.” ([Ma et al., p. 3](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=4&annotation=F3IAJZGQ)) #a28ae5
|
||
- “detecting ChatGPT content is challenging in many domains, as the content is designed to mimic human language and structure.” ([Ma et al., p. 3](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=4&annotation=925ZRABZ)) #a28ae5
|
||
- “high-quality” ([Ma et al., p. 3](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=4&annotation=RXZB6Q3N)) #ffd400
|
||
|
||
*what do you mean?*
|
||
- “detects ChatGPT-generated content on Stack Overflow;” ([Ma et al., p. 4](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=5&annotation=396MGFH7)) #a28ae5
|
||
- “Generative Pretrained Transforme” ([Ma et al., p. 5](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=5&annotation=XDYELL46)) #5fb236
|
||
- “It generates responses by predicting the likelihood of a word given the previous words used in the tex” ([Ma et al., p. 5](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=5&annotation=9RYEELZR)) #5fb236
|
||
- “Siamese Neural Network” ([Ma et al., p. 6](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=6&annotation=I8NAXVKR)) #5fb236
|
||
- “reference answer” ([Ma et al., p. 6](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=6&annotation=UZ5B3RTU)) #ffd400
|
||
|
||
*THis is not clear. THe reference answer should be by humans, isn't it?*
|
||
- “comprehensive benchmark that enables the accurate detection of ChatGPT-generated content in Stack Overflow” ([Ma et al., p. 6](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=6&annotation=HDJUBCMQ)) #5fb236
|
||
- “A high-quality answer generated by ChatGPT, designed to serve as a benchmark with distinct non-human features for comparison.” ([Ma et al., p. 6](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=6&annotation=ES4GTYCY)) #ffd400
|
||
|
||
*THis is a bit obscure. It is not clear what are the characteristics of the answers available in this set. Especially, how to ensure to make it different from the ChatGPT answer. It is necessary to explain why it is necessary to have this, instead of having simply pairs of human and chatGPT answers.*
|
||
- “o” ([Ma et al., p. 6](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=6&annotation=TRVBUT4P)) #5fb236
|
||
- “he question title and question body are regarded together as the complete question. The dataset preparation process involved several key steps: selecting and filtering data from Stack Overflow, generating reference answers, and” ([Ma et al., p. 7](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=7&annotation=URP8F2Z9)) #ffd400
|
||
|
||
*It's not clear why we have two sets of questions generated by ChatGPT. This needs to be rechecked.....*
|
||
- “16,847 high-quality post questions and their corresponding high-quality answers based on the specified criteria.” ([Ma et al., p. 7](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=7&annotation=697Q73Y7)) #5fb236
|
||
- “et, we generated a reference answer designed to encapsulate the typical features of non-human answers, as identified in related empirical research [9, 32, 37, 49, 59, 69]. The reference answers serve as a” ([Ma et al., p. 8](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=8&annotation=WUSNCATH)) #ffd400
|
||
|
||
*Maybe the goal is training the model with couples of human and ChatGPT generated answers? If yes this needs to be clarified earlier! TO BE CHECKED!*
|
||
- “6000 questions” ([Ma et al., p. 8](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=8&annotation=PJS4ME96)) #5fb236
|
||
- “diversity of tags” ([Ma et al., p. 8](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=8&annotation=PVKM6KFH)) #5fb236
|
||
- “6,000 high-quality question” ([Ma et al., p. 8](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=8&annotation=55N9L3NX)) #ffd400
|
||
|
||
*what was the quested quality?*
|
||
- “xceed 1000 tokens in length.” ([Ma et al., p. 9](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=9&annotation=MVD77AV4)) #ffd400
|
||
|
||
*Is this respected? Sometimes LLMs do not follow these kids of requests.*
|
||
- “Chain-of-Thought ChatGPT Prompt” ([Ma et al., p. 9](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=9&annotation=Q4CDDQKB)) #ffd400
|
||
|
||
*I don't see a substantial difference with the "Reference Prompt"*
|
||
- “Persona Prompt: For the last segment, we used a persona approach [19], framing ChatGPT as a professional and experienced developer. This prompt aimed to produce answers that demonstrate deep expertise and” ([Ma et al., p. 10](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=10&annotation=MRUZALMZ)) #ffd400
|
||
|
||
*I don't see how such a requirement can be actually satisfied.*
|
||
- “a reference answer, a human answer, and a ChatGPT answer.” ([Ma et al., p. 10](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=10&annotation=PKYHZMLT)) #5fb236
|
||
- “similarity between the reference answer and the other answers.” ([Ma et al., p. 10](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=10&annotation=MIJ4723R)) #5fb236
|
||
- “key similarities and differences” ([Ma et al., p. 10](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=10&annotation=QE4FDT86)) #5fb236
|
||
- “By using BigBird, we ensure that our model can capture the comprehensive context and nuances of lengthy texts.” ([Ma et al., p. 11](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=11&annotation=KKL2ANWU)) #5fb236
|
||
- “11GB NVidia GeForce RTX 2080 Ti graphics card” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=IIQR2NM8)) #5fb236
|
||
- “various baseline models for comparison” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=98XFVA6A)) #5fb236
|
||
- “accuracy” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=9FWQ46WW)) #5fb236
|
||
- “precision” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=C8EL7L9V)) #5fb236
|
||
- “recall” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=CWCVEBDS)) #5fb236
|
||
- “F1 score” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=G9J953C4)) #5fb236
|
||
- “RQ1:” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=GFE7KLC7)) #5fb236
|
||
- “RQ2:” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=B3PQJIVF)) #5fb236
|
||
- “RQ3:” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=GFPPLZLM)) #a28ae5
|
||
|
||
*INTERESTING*
|
||
- “RQ4” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=XHZUMLSC)) #ffd400
|
||
|
||
*This overlaps with RQ1, isn't it?*
|
||
- “RQ5” ([Ma et al., p. 12](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=12&annotation=DSFY3SMT)) #5fb236
|
||
- “The splitting process was carried out randomly” ([Ma et al., p. 13](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=13&annotation=3RXH3PA6)) #a28ae5
|
||
- “: The validation set is used to tune the hyperparameters and make decisions regarding early stopping during the training process.” ([Ma et al., p. 13](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=13&annotation=CZP8U435)) #5fb236
|
||
- “the predictability of” ([Ma et al., p. 13](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=13&annotation=I3S73TLV)) #ffd400
|
||
- “This high recall ensures that most ChatGPT-generated content is identified, and high precision ensures that the identified content is indeed ChatGPTgenerated.” ([Ma et al., p. 14](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=14&annotation=TT8GW58B)) #5fb236
|
||
- “ChatGPT-generated content without misclassifying human-generated content.” ([Ma et al., p. 14](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=14&annotation=F9TK7B2R)) #5fb236
|
||
- “length of the input text affect the performance of SOGPTSpotter?” ([Ma et al., p. 16](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=16&annotation=GRN57F5E)) #ffd400
|
||
|
||
*lenght of the text used for training or during the inference phase?*
|
||
- “improved F1-scores with longer input texts.” ([Ma et al., p. 16](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=16&annotation=LXD4QCV6)) #5fb236
|
||
- “which were applied to the answers to make them more challenging to detect.” ([Ma et al., p. 18](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=18&annotation=IS6CNM63)) #5fb236
|
||
- “For each type of adversarial attack, we ran SOGPTSpotter and the baseline models on our modified test dataset and measured their detection accuracy using F1-score. This allowed us to quantify the impact of each type of adversarial attack on the performance of our tool and the baseline models, providing valuable insights into their robustness against such attacks. The results are shown in Table 4.” ([Ma et al., p. 18](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=18&annotation=WMWHX376)) #ffd400
|
||
|
||
*Details on how attacks have been performed and measured are needed. For instance, are those performed on the same dataset used for the previous research questions? How questions have been selected?*
|
||
- “ignificantly influences the performance of SOGPTSpotter, with longer texts providing better F1-scores.” ([Ma et al., p. 18](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=18&annotation=VZLTN3DJ)) #5fb236
|
||
- “substitution” ([Ma et al., p. 18](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=18&annotation=BRZAGHWI)) #a28ae5
|
||
- “Deep-Word-Bug” ([Ma et al., p. 18](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=18&annotation=ZJTX9XCS)) #5fb236
|
||
- “Answer to RQ5: The field study results demonstrate the practical effectiveness of our SOGPTSpotter tool in a real-world setting on Stack Overflow. Our model successfully identified ChatGPT-generated content with a high acceptance rate of 94% for post edits submitted for community review. Despite some challenges with shorter answers and posts containing long code snippets, the overall high acceptance rate indicates that our tool can significantly aid in moderating AI-generated content on Q&A platforms.” ([Ma et al., p. 22](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=22&annotation=G2K7T6H4)) #ffd400
|
||
|
||
*This research question is not convincing. The users trusted that the 47 posts were generated but we cannot be sure that they were so, isn't it? It is necessary to provide a better argumentation for support such research question.*
|
||
- “the selection of post answers from Stack Overflow as human answers.” ([Ma et al., p. 22](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=22&annotation=KPUH5UBD)) #5fb236
|
||
- “While our SOGPTSpotter performed well on both the experimental datasets and real-world case studies, improvements in LLMs might lead to more sophisticated AI-generated content that is harder to detect.” ([Ma et al., p. 22](zotero://select/library/items/JZNGIUNE)) ([pdf](zotero://open-pdf/library/items/KIZ5VMMI?page=22&annotation=FGRPTJLL)) #ffd400
|
||
|
||
*Yes, this require monitoring efforts as future work.*
|
||
-
|
||
- ### [[Comments]]
|
||
- Summary: The paper presents SOGPTSpotter, an automated tool for detecting Stack Overflow answers that are generated by ChatGPT. The approach is essentially based on BigBird-Based Siamese Network and according to the performed experiments it outperforms existing approaches. In particular, four research questions have been presented to compare the approach with 6 alternative approaches and to investigate the performance of SOGPTSpotter even in case of adversarial attacks and when applied on different application domains.
|
||
- Comments: The paper addresses an interesting and relevant topic and is overall well-written and well-structured. Below are some comments and suggestions for improvement, aiming to clarify certain concepts and strengthen the presentation:
|
||
- Clarify the distinction between human-generated reference answers and ChatGPT answers. It is not evident why a separate set of reference answers is needed instead of directly comparing human and ChatGPT answers. The characteristics of the reference answers must also be explained in detail, including how they are differentiated from ChatGPT-generated answers. I understand that two different prompts are used for generating reference and ChatGPT-generated answers. However, it is necessary to show that the two sets of generated answers are different and with respect to what.
|
||
- The reasoning behind having two sets of questions generated by ChatGPT needs further elaboration. Is the goal to simulate variability in ChatGPT outputs? If so, this should be clearly stated earlier in the paper.
|
||
- The criteria for determining "high-quality" questions and answers (e.g., 6,000 high-quality pairs) should be explicitly described. What were the quality checks, and how was compliance with those criteria ensured?
|
||
- The claim that answers do not exceed 1,000 tokens should be verified. Large language models sometimes fail to adhere strictly to such constraints, and clarification is needed on how this issue was managed.
|
||
- Explain whether the length of the input text impacts training or inference performance, as this distinction affects the interpretation of the results.
|
||
- The methodology for creating and measuring adversarial attacks needs more details. Are these attacks applied to the same dataset used for the other research questions? How were the adversarial examples generated, and how were the evaluation metrics computed?
|
||
- The conclusion drawn from the field study (RQ5) is not entirely convincing. The users trusted that the 47 posts were generated but we cannot be sure that they were so, isn't it? It is necessary to provide a better argumentation for support such research question.
|
||
- As the paper mentions, future improvements in LLMs might lead to more sophisticated AI-generated content. Highlight the need for ongoing monitoring and adaptability of the approach as an important avenue for future work.
|
||
- Replace arXiv references with their corresponding published versions, if available.
|
||
- ### [[REVIEWS/Notes]]
|
||
- ### YELLOW CONCERNS
|
||
background-color:: yellow
|
||
- {{query (and [[ffd400]] [[TOSEM-2024-0498]] )}}
|
||
collapsed:: true
|
||
- ### ❓️Questions
|
||
- {{query (and [[question]] [[TOSEM-2024-0498]] )[[question]]}}
|
||
query-table:: true
|
||
query-properties:: [:block] |