375 lines
15 KiB
Markdown
375 lines
15 KiB
Markdown
file:: [SANER2024_paper_11_1700084266286_0.pdf](../assets/SANER2024_paper_11_1700084266286_0.pdf)
|
||
file-path:: ../assets/SANER2024_paper_11_1700084266286_0.pdf
|
||
|
||
- Neural code models have demonstrated their efficacy across a range of code comprehension tasks, including vulnerability detection, code classification, automatic code summarization, completion, clone detection, etc
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575d712-2716-4f9f-9a5d-49d2ef081c36
|
||
- obustness of models in the realm of code comprehension and its associated applications
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575d71b-d7bb-4213-a4c3-009d26e90016
|
||
- NLP-like techniques to craft adversarial code instances, primarily by perturbing variable and token names
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575d729-bfff-4353-9616-bb53ae86ae34
|
||
- Can we perturb the structural aspects of code while preserving its semantics, thereby generating more disruptive adversarial examples that elude current structural-unaware approaches?
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575d84d-e659-4d00-b0db-0a0415360a94
|
||
- active-learning guided adversarial attack framework
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575d862-87ed-444e-9a9e-230a6ef47609
|
||
- semantic-preserving translations
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575d869-e6b5-4205-b76a-9593eb95f98e
|
||
- adaptive adversarial discriminator and token selector,
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575d871-dac4-4b1b-92d6-8e4eb3b3dcb4
|
||
- We substantiate ALANCA’s efficacy through comprehensive evaluations across four distinct code comprehension tasks, demonstrating its ability to effectively confound a range of neural models, including pre-trained models and LLMs used in software engineering
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575d887-11b4-42cf-bbb0-35f2edb9a2c9
|
||
- nsuring software security, program quality, and development efficiency.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575d8a0-7c5b-4d84-aea0-4764d165dfed
|
||
- remarkable performance across various code-related tasks, particularly in code comprehension
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575da8e-f7c8-433b-b7c4-aac62ffe8b36
|
||
- vulnerability detectio
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575da92-34c0-4f51-8b08-d78053512b64
|
||
- code classificatio
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575da95-4400-4864-93a8-bba2fe7cadc2
|
||
- ode summarization
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575da9a-4ecc-4c6d-9492-084173be4657
|
||
- code completio
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575da9d-6e43-45af-8f0a-8cfefdbf821d
|
||
- clone detection
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575daa2-bcbe-4528-8bd8-f3c52bb1f1f6
|
||
- ulnerability in code models remains largely unresolved.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575dab7-69bf-420b-a3f9-9536e2371a88
|
||
- obustness in neural code models and prevent the inadvertent introduction of faulty code or vulnerabilities into real-world systems
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6575dad1-3ad2-46ab-b464-72398906e939
|
||
- minor adjustments to ordinary inputs, imperceptible to humans, can lead to neural models producing misleading results
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6575daf9-cbc0-40bf-b84a-d2907592593c
|
||
- Many code model applications come as packaged software or offer restricted APIs, limiting the frequency of queries to the target model. Unfortunately, these constraints are frequently overlooked or inadequately considered in existing research
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: blue
|
||
id:: 6575db4a-5e63-40fe-82b0-50de4c47d428
|
||
- we introduce ALANCA, an active-learning guided adversarial attack framework designed to execute effective black-box attacks on a wide array of code comprehension models and tasks.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: blue
|
||
id:: 6575db74-e580-4e8f-9fe1-4a1f38cfa462
|
||
- statistical information to initially generate a pool of candidate adversarial examples
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 6575dbab-beb3-4e7e-9b84-5f2e4152b07d
|
||
- We tested ALANCA on eight different models, including both pre-trained models and LLMs in SE.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 6575dbe0-de9d-45c5-939b-02ecb027a85d
|
||
- he results of our evaluations clearly demonstrate that ALANCA effectively confuses and neutralizes various neural models with high efficiency
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: blue
|
||
id:: 6575dbec-8e88-445e-ab66-ce8604d1c1fe
|
||
- extensive evaluations on four distinct code comprehension tasks: Code Summarization, Method Name Prediction, Code Classification, and Code Clone Detection
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: blue
|
||
id:: 6575dbf6-6ffc-4f69-80bc-1fa57819b56c
|
||
- neural code models
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 6575dc09-2074-40ed-8c6f-fe4f97726dcb
|
||
- Our evaluation involves eight target models and four code comprehension tasks, and ALANCA consistently achieves a high attack success rate.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: blue
|
||
id:: 6575dc4b-81a6-4254-94ad-b31f151e32f4
|
||
- F = f : X → Y
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 6575dc68-46db-4872-a8cf-529ef342957b
|
||
- The objective of an adversarial attack is to transform an input example to mislead the classifier.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: blue
|
||
id:: 6575dce2-7118-4127-a285-a98b23de4a12
|
||
- only the input features and output results of the target model are accessible, and the number of queries is limited.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 6575dd69-1b8a-433d-964f-b05d19e77247
|
||
- This approach combines a precise, formal representation of functionality with a natural and human-understandable representation of language, presenting a potential solution for these issues
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 6575ddf1-a615-4a6b-867d-a05f0af45348
|
||
- er obtaining the labels for the sampled examples, the sampling model itself is updated using the new labeled set{(x, l)|x ∈ DU n−kn }.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 6575de8e-370e-4b33-8da8-5330d3f3f993
|
||
- The active learning system continues to iterate until the model performance meets the desired requirements or until the number of queries is exhausted
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: blue
|
||
id:: 6575deba-14d9-470f-9783-0efde297f499
|
||
- estricted access to the target model
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 6575df1e-2a6f-49b1-a86f-ae8039221635
|
||
- raining of ALANCA with only a limited number of labels obtained from the target model
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 6575df2e-44b7-41de-8b08-76ffe336f49e
|
||
- Guided Code Transforme
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 6575df6b-b7ab-43da-90c8-bbbfea5547b5
|
||
- maintain the semantic meaning of the code while making it challenging for the target model to correctly understand and classify
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575df80-6efd-42fe-bc1d-1f2b5bf4dee0
|
||
- roduces a varied collection of candidate adversarial examples, denoted as T , each meticulously crafted to explore various attack strategies
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: yellow
|
||
id:: 6575dfaf-7d00-4762-a5f9-28690486f3ca
|
||
hl-stamp:: 1702223792925
|
||
- Adversarial Example Discriminato
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575dfdd-7b51-4fc2-9bd2-f302fea9ed92
|
||
- ALANCA employs a discriminator module to score and rank candidate examples in the set T .
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575dff2-371b-4d79-bfed-816d09a01336
|
||
- he discriminator module is trained to gauge the quality and effectiveness of the generated adversarial examples.
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e00a-22d9-426a-9812-bb329511027b
|
||
- oken Selecto
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e013-51e7-48d6-af93-ec9ef754542f
|
||
- selecting examples to query from the target model and updating the components (Lines 19 to 23) is carefully crafted to capture essential information necessary for conducting effective adversarial attacks
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e04d-780f-4dfb-bcec-78205ac55e02
|
||
- perturbation remains within a specified threshold.
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e2ae-f09c-436a-8f9a-c375b0a82b69
|
||
- To generate code transformations of high quality, it is essential to consider the similarity or distance (||x′ − x||)
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: yellow
|
||
id:: 6575e2bc-e630-4510-9694-4164a29b379c
|
||
- Functional similarity refers to the equivalence in behavior between two pieces of code.
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: yellow
|
||
id:: 6575e352-ebdd-424c-b1d0-60207026e293
|
||
- Refactoring involves making small modifications to the source code that preserve the program’s behavior.
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e40d-dfbd-4158-bee3-72c26d7389ee
|
||
- dead code elimination → dead code insertio
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e41e-cffc-4be9-b05a-5598cc0217fe
|
||
- Our objective is to ensure a diverse range of refactoring techniques while also constraining the extent of textual changes introduced.
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e42b-a58e-4072-bd1b-2470548a2013
|
||
- ALANCA utilizes a series of parameters that are calculated based on statistics to determine where the modifications or additions occur within the code
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 6575e444-e0bc-4af4-b276-7c67855da55a
|
||
- semantic-preserving transformations
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: purple
|
||
id:: 6575e455-4e5d-4c59-9a3d-84982919016b
|
||
- ttacking effectiveness
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: yellow
|
||
id:: 6575e468-5282-4bf9-8854-35c8d0b68c45
|
||
- CODEBERT was built from the pre-trained language model BERT with 125M parameters, while CODE2VEC has an encoder-only model with encoded paths of ASTs as input.
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 6575e4d3-8550-466f-b4fa-837f369724b5
|
||
- Consequently, certain vulnerabilities or adversarial examples may be more effective or evident in one model compared to another.
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 6575e4e0-d7c4-432c-a1ee-27023df30a10
|
||
- , it is crucial to employ adaptive strategies that take into account the specific target models and tasks involved
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: blue
|
||
id:: 6575e519-b70c-49b6-9cd1-d769c7063c5f
|
||
- This module is designed to identify vulnerable spots in code snippets and classify them according to their vulnerable type
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: yellow
|
||
id:: 6575e52d-42cc-42f7-84ad-7ab967568727
|
||
- arget model’s prediction with a Siamese network structure
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: yellow
|
||
id:: 6575e568-fb57-49df-9f69-31f5e2c1464d
|
||
- We employ CodeBERT [11] as the underlying framework for the token selector model.
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 6575e62f-00ea-4fd8-9fb3-db84efe0c08d
|
||
- ➂
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: yellow
|
||
id:: 6575e67a-6067-4c0f-962b-7dc424247900
|
||
- We excluded code snippets that lacked function parameters, variable declarations, and methods containing fewer than five lines before conducting our experiments.
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 6575e6d6-7328-4985-b00d-37263ae6ef5e
|
||
- three main types of code neural models:
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: blue
|
||
id:: 6575e6ff-54da-4b73-acd8-8e4662d73efd
|
||
- To highlight the significance of each path, CODE2VEC employs an attention mechanism to assign appropriate weights to the AST paths.
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e795-b41f-47ab-a59b-74f6f2906afb
|
||
- CODE2SEQ [8] also utilizes AST paths to represent code snippets. It employs an encoder-decoder model along with bi-LSTM units to embed the nodes of the AST paths and subsequently make sequential predictions
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e7a5-fb7e-4527-9538-efb3e79c06f1
|
||
- 2.3 million functions with paired documentation from six programming languages was utilized.
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e7bb-52e2-4d2d-9c04-b9462107cba2
|
||
- ransformer-based neural model that follows a similar encoder-decoder architecture as BAR
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e7cb-2e76-4e25-8852-e948bc0178e5
|
||
- To evaluate the effectiveness of the attacks, we conducted a thorough analysis of the impact of adversarial examples on the prediction outputs of the target models
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e7f5-a731-4045-bf37-4aa36663e67e
|
||
- ccuracy and F1 score for classification tasks
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e801-b834-461e-bedc-6239c039cfa3
|
||
- we were able to observe varying degrees of impact on their prediction capabilities
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: yellow
|
||
id:: 6575e81b-3f49-469c-beb7-c81bda6adf41
|
||
hl-stamp:: 1702225949640
|
||
- predicted labels are changed after the transformation process
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: yellow
|
||
id:: 6575e84d-d1b5-48ab-84c2-33f8f2519cd7
|
||
- I(P (xi)̸ = P (x′ ij )) is an indicator function that evaluates to 1 if the predictions for xi and x′ ij differ, and0 otherwise.
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e8a4-335c-4f12-86ac-dcc1d7d7b900
|
||
- ow many predictions were altered compared to the original predictions
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: blue
|
||
id:: 6575e8ac-37f6-445a-bc40-13a6ed7bf3fc
|
||
- ratio of transformed examples for which the prediction differs from the original prediction:
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e8fc-cec8-4c0d-b6d0-d3be113e21e5
|
||
- if the model changes its prediction on correct examples
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 6575e908-72f4-4944-9caa-90385a543ec8
|
||
- Ro =
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: yellow
|
||
id:: 6575e970-ec66-47ad-ab2f-0049c6d0c1ca
|
||
- active-learning guided adversarial attack framework for neural code models
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: yellow
|
||
id:: 6575ee7e-212a-42a3-8a26-9a142754745c |