62 lines
6.3 KiB
Markdown
62 lines
6.3 KiB
Markdown
tags:: [[Mentoring]], [[people/Umar]], [[#zotero]]
|
||
title:: @Prompt-Injection based Adversarial Attacks in Large Language Models
|
||
item-type:: [[document]]
|
||
original-title:: Prompt-Injection based Adversarial Attacks in Large Language Models
|
||
language:: en
|
||
authors:: [[Muhammad Umar Zeshan]]
|
||
library-catalog:: Zotero
|
||
links:: [Local library](zotero://select/library/items/G3NC2PN7), [Web library](https://www.zotero.org/users/1039502/items/G3NC2PN7)
|
||
|
||
-
|
||
- ### Attachments
|
||
- [Zeshan - Prompt-Injection based Adversarial Attacks in Larg.pdf](zotero://select/library/items/N65IM7XB) {{zotero-imported-file N65IM7XB, "Zeshan - Prompt-Injection based Adversarial Attacks in Larg.pdf"}}
|
||
- ### Notes
|
||
- # Annotazioni
|
||
- (19/1/2024, 16:08:19)
|
||
- “trendy” (Zeshan, p. 1) #ff6666
|
||
- * *
|
||
- “prompt attacks in Large Language Models” (Zeshan, p. 1) #a28ae5
|
||
- * *
|
||
- “LLMs” (Zeshan, p. 1) #ff6666
|
||
- * *
|
||
- “better” (Zeshan, p. 1) #ffd400
|
||
- *Nothing has been said so far. Better than that? *
|
||
- “ethical issues, bias, and responsible use become crucial factors to take into account” (Zeshan, p. 1) #ffd400
|
||
- *These are critical points. Are you mentioning them because you want to plan to deal with them? If not, why are you mentioning them? *
|
||
- “insufficient safety assessments and guardrails are accompanying this drive toward AI integration” (Zeshan, p. 1) #5fb236
|
||
- * *
|
||
- “A prompt should ideally generate an answer that is accurate, sufficient in both form and content, and of the appropriate length” (Zeshan, p. 2) #5fb236
|
||
- * *
|
||
- “several human feedback-incorporating fine-tuning procedures have been developed to guarantee that LLM outputs are both safe and consistent with human values.” (Zeshan, p. 2) #5fb236
|
||
- * *
|
||
- “around the security.” (Zeshan, p. 2) #ffd400
|
||
- *Some more details about security issues one can have with LLMs are needed. *
|
||
- “According to Zou et al.’s research from 2023, adversarial sequence creation can be automated, producing an infinite number of these attacks. Furthermore, they demonstrate how safety precautions can be circumvented by appending a single adversarial sequence to several damaging prompts” (Zeshan, p. 2) #ffd400
|
||
- *Some illustrative examples are needed here! *
|
||
- “Adversarial attacks detection in RSSE” (Zeshan, p. 2) #ffd400
|
||
- *This section should be on presenting Rec Systems in Software Engineering. Thus the title should be changed and the content should be expanded with the aim of presenting a quick overview on RSSE. *
|
||
- “Adversarial attempts produce perturbations to trick and confuse systems by breaking them down, impairing their ability to provide recommendations. For instance, an adversarial attack on recommender systems may support or disparage a product, depending on the intent, which would have a detrimental effect on the final recommendations. Similarly, malicious users may expose recommender systems to hazardous artifacts by altering training data that is accessible through OSS platforms. Software system disruptions may arise if a recommender is trained to deliver harmful outcomes based on Adversaries. For instance, a recent study reveals that there have been attempts to force Android apps to open ports covertly, enabling unwanted access. Security concerns in machine learning systems and all-purpose recommender systems are investigated via research on adversarial machine learning (AML)” (Zeshan, p. 3) #ffd400
|
||
- *I would show some technical details that will be used later in the text. *
|
||
- “Adversarial Attacks in Promptbased Learning” (Zeshan, p. 4) #ffd400
|
||
- *All the different approaches that are overviewed in this section need to be expanded with concrete examples. It is essential to expand because, as far as I understood, this will underpin and motivate the planned work. *
|
||
- “trendy” (Zeshan, p. 4) #ff6666
|
||
- *I don't like it; it is not for scientific documents. *
|
||
- “backdoor attacks” (Zeshan, p. 4) #5fb236
|
||
- * *
|
||
- “Our Methodology” (Zeshan, p. 5) #ffd400
|
||
- *For what?
|
||
- It is necessary to conclude the previous section by listing the challenges you plan to address! By clearly listing the challenges at the end of the previous section, in this section you can discuss how you plant to deal with them. *
|
||
- “Limitations in previous methods” (Zeshan, p. 6) #ffd400
|
||
- *This must be moved to the previous section before listing the challenges that you plan to address. *
|
||
- “The main drawback of previous research is that it is case study oriented; for example, it did not define the attacker’s objective in rapid injection assaults in terms of the type of the attack, attacker wants to carry.” (Zeshan, p. 6) #ffd400
|
||
- *The document is plenty of sentences like this one, we are missing concreteness, you have to provide explanatory examples to help reader understand what's the problem by referring to real cases. *
|
||
- “Rather than simply translating a text into English, they demonstrated how an assailant may lead an LLM astray and have them compose a sonnet about pandas. The main drawback of this kind of caseby-case research is how difficult it is to come up with novel prompt injection attacks or carry out a thorough analysis and comparison of other prompt injection assaults. Even the latest studies are focusing on some particular type of prompt injections, which also allow for enhancing the attack by combining all the possible attack scenarios.” (Zeshan, p. 6) #ffd400
|
||
- *Suddenly, we lost the reference to RSSE! *
|
||
- “Research Questions” (Zeshan, p. 6) #ffd400
|
||
- *Are you still interested in RSSE? The research questions you defined are not related to RSSE. *
|
||
- “ll kinds of adversarial attacks defined in literature are discussed which are relevant in this new trendy topic of prompt learning attacks.” (Zeshan, p. 6) #ffd400
|
||
- *Thus, what's the plan? Are you planning to work on a survey work? What do you expect to do further than expanding what you have listed in 3.2? *
|
||
- “threat model is proposed, in which a combined attack model is applied which concatenates all the possible types of attacks in prompts by the attacker.” (Zeshan, p. 6) #ffd400
|
||
- *I don't see it in Section 3.2 *
|
||
- “Prompt Injection Attacks” (Zeshan, p. 8) #ffd400
|
||
- *Is this supposed to be a new type of attack? But it is not new, isn't it? * |