logseq/pages/hls__SANER2024_paper_11_1700084266286_0.md

file:: SANER2024_paper_11_1700084266286_0.pdf file-path:: ../assets/SANER2024_paper_11_1700084266286_0.pdf

• Neural code models have demonstrated their efficacy across a range of code comprehension tasks, including vulnerability detection, code classification, automatic code summarization, completion, clone detection, etc ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575d712-2716-4f9f-9a5d-49d2ef081c36
• obustness of models in the realm of code comprehension and its associated applications ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575d71b-d7bb-4213-a4c3-009d26e90016
• NLP-like techniques to craft adversarial code instances, primarily by perturbing variable and token names ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575d729-bfff-4353-9616-bb53ae86ae34
• Can we perturb the structural aspects of code while preserving its semantics, thereby generating more disruptive adversarial examples that elude current structural-unaware approaches? ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575d84d-e659-4d00-b0db-0a0415360a94
• active-learning guided adversarial attack framework ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575d862-87ed-444e-9a9e-230a6ef47609
• semantic-preserving translations ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575d869-e6b5-4205-b76a-9593eb95f98e
• adaptive adversarial discriminator and token selector, ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575d871-dac4-4b1b-92d6-8e4eb3b3dcb4
• We substantiate ALANCA’s efficacy through comprehensive evaluations across four distinct code comprehension tasks, demonstrating its ability to effectively confound a range of neural models, including pre-trained models and LLMs used in software engineering ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575d887-11b4-42cf-bbb0-35f2edb9a2c9
• nsuring software security, program quality, and development efficiency. ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575d8a0-7c5b-4d84-aea0-4764d165dfed
• remarkable performance across various code-related tasks, particularly in code comprehension ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575da8e-f7c8-433b-b7c4-aac62ffe8b36
• vulnerability detectio ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575da92-34c0-4f51-8b08-d78053512b64
• code classificatio ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575da95-4400-4864-93a8-bba2fe7cadc2
• ode summarization ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575da9a-4ecc-4c6d-9492-084173be4657
• code completio ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575da9d-6e43-45af-8f0a-8cfefdbf821d
• clone detection ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575daa2-bcbe-4528-8bd8-f3c52bb1f1f6
• ulnerability in code models remains largely unresolved. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575dab7-69bf-420b-a3f9-9536e2371a88
• obustness in neural code models and prevent the inadvertent introduction of faulty code or vulnerabilities into real-world systems ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6575dad1-3ad2-46ab-b464-72398906e939
• minor adjustments to ordinary inputs, imperceptible to humans, can lead to neural models producing misleading results ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6575daf9-cbc0-40bf-b84a-d2907592593c
• Many code model applications come as packaged software or offer restricted APIs, limiting the frequency of queries to the target model. Unfortunately, these constraints are frequently overlooked or inadequately considered in existing research ls-type:: annotation hl-page:: 1 hl-color:: blue id:: 6575db4a-5e63-40fe-82b0-50de4c47d428
• we introduce ALANCA, an active-learning guided adversarial attack framework designed to execute effective black-box attacks on a wide array of code comprehension models and tasks. ls-type:: annotation hl-page:: 1 hl-color:: blue id:: 6575db74-e580-4e8f-9fe1-4a1f38cfa462
• statistical information to initially generate a pool of candidate adversarial examples ls-type:: annotation hl-page:: 2 hl-color:: green id:: 6575dbab-beb3-4e7e-9b84-5f2e4152b07d
• We tested ALANCA on eight different models, including both pre-trained models and LLMs in SE. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 6575dbe0-de9d-45c5-939b-02ecb027a85d
• he results of our evaluations clearly demonstrate that ALANCA effectively confuses and neutralizes various neural models with high efficiency ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 6575dbec-8e88-445e-ab66-ce8604d1c1fe
• extensive evaluations on four distinct code comprehension tasks: Code Summarization, Method Name Prediction, Code Classification, and Code Clone Detection ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 6575dbf6-6ffc-4f69-80bc-1fa57819b56c
• neural code models ls-type:: annotation hl-page:: 2 hl-color:: green id:: 6575dc09-2074-40ed-8c6f-fe4f97726dcb
• Our evaluation involves eight target models and four code comprehension tasks, and ALANCA consistently achieves a high attack success rate. ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 6575dc4b-81a6-4254-94ad-b31f151e32f4
• F = f : X → Y ls-type:: annotation hl-page:: 2 hl-color:: green id:: 6575dc68-46db-4872-a8cf-529ef342957b
• The objective of an adversarial attack is to transform an input example to mislead the classifier. ls-type:: annotation hl-page:: 2 hl-color:: blue id:: 6575dce2-7118-4127-a285-a98b23de4a12
• only the input features and output results of the target model are accessible, and the number of queries is limited. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 6575dd69-1b8a-433d-964f-b05d19e77247
• This approach combines a precise, formal representation of functionality with a natural and human-understandable representation of language, presenting a potential solution for these issues ls-type:: annotation hl-page:: 2 hl-color:: green id:: 6575ddf1-a615-4a6b-867d-a05f0af45348
• er obtaining the labels for the sampled examples, the sampling model itself is updated using the new labeled set{(x, l)|x ∈ DU n−kn }. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6575de8e-370e-4b33-8da8-5330d3f3f993
• The active learning system continues to iterate until the model performance meets the desired requirements or until the number of queries is exhausted ls-type:: annotation hl-page:: 3 hl-color:: blue id:: 6575deba-14d9-470f-9783-0efde297f499
• estricted access to the target model ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6575df1e-2a6f-49b1-a86f-ae8039221635
• raining of ALANCA with only a limited number of labels obtained from the target model ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6575df2e-44b7-41de-8b08-76ffe336f49e
• Guided Code Transforme ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6575df6b-b7ab-43da-90c8-bbbfea5547b5
• maintain the semantic meaning of the code while making it challenging for the target model to correctly understand and classify ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575df80-6efd-42fe-bc1d-1f2b5bf4dee0
• roduces a varied collection of candidate adversarial examples, denoted as T , each meticulously crafted to explore various attack strategies ls-type:: annotation hl-page:: 4 hl-color:: yellow id:: 6575dfaf-7d00-4762-a5f9-28690486f3ca hl-stamp:: 1702223792925
• Adversarial Example Discriminato ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575dfdd-7b51-4fc2-9bd2-f302fea9ed92
• ALANCA employs a discriminator module to score and rank candidate examples in the set T . ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575dff2-371b-4d79-bfed-816d09a01336
• he discriminator module is trained to gauge the quality and effectiveness of the generated adversarial examples. ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e00a-22d9-426a-9812-bb329511027b
• oken Selecto ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e013-51e7-48d6-af93-ec9ef754542f
• selecting examples to query from the target model and updating the components (Lines 19 to 23) is carefully crafted to capture essential information necessary for conducting effective adversarial attacks ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e04d-780f-4dfb-bcec-78205ac55e02
• perturbation remains within a specified threshold. ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e2ae-f09c-436a-8f9a-c375b0a82b69
• To generate code transformations of high quality, it is essential to consider the similarity or distance (||x′ − x||) ls-type:: annotation hl-page:: 4 hl-color:: yellow id:: 6575e2bc-e630-4510-9694-4164a29b379c
• Functional similarity refers to the equivalence in behavior between two pieces of code. ls-type:: annotation hl-page:: 4 hl-color:: yellow id:: 6575e352-ebdd-424c-b1d0-60207026e293
• Refactoring involves making small modifications to the source code that preserve the program’s behavior. ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e40d-dfbd-4158-bee3-72c26d7389ee
• dead code elimination → dead code insertio ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e41e-cffc-4be9-b05a-5598cc0217fe
• Our objective is to ensure a diverse range of refactoring techniques while also constraining the extent of textual changes introduced. ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e42b-a58e-4072-bd1b-2470548a2013
• ALANCA utilizes a series of parameters that are calculated based on statistics to determine where the modifications or additions occur within the code ls-type:: annotation hl-page:: 4 hl-color:: green id:: 6575e444-e0bc-4af4-b276-7c67855da55a
• semantic-preserving transformations ls-type:: annotation hl-page:: 4 hl-color:: purple id:: 6575e455-4e5d-4c59-9a3d-84982919016b
• ttacking effectiveness ls-type:: annotation hl-page:: 4 hl-color:: yellow id:: 6575e468-5282-4bf9-8854-35c8d0b68c45
• CODEBERT was built from the pre-trained language model BERT with 125M parameters, while CODE2VEC has an encoder-only model with encoded paths of ASTs as input. ls-type:: annotation hl-page:: 5 hl-color:: green id:: 6575e4d3-8550-466f-b4fa-837f369724b5
• Consequently, certain vulnerabilities or adversarial examples may be more effective or evident in one model compared to another. ls-type:: annotation hl-page:: 5 hl-color:: green id:: 6575e4e0-d7c4-432c-a1ee-27023df30a10
• , it is crucial to employ adaptive strategies that take into account the specific target models and tasks involved ls-type:: annotation hl-page:: 5 hl-color:: blue id:: 6575e519-b70c-49b6-9cd1-d769c7063c5f
• This module is designed to identify vulnerable spots in code snippets and classify them according to their vulnerable type ls-type:: annotation hl-page:: 5 hl-color:: yellow id:: 6575e52d-42cc-42f7-84ad-7ab967568727
• arget model’s prediction with a Siamese network structure ls-type:: annotation hl-page:: 5 hl-color:: yellow id:: 6575e568-fb57-49df-9f69-31f5e2c1464d
• We employ CodeBERT [11] as the underlying framework for the token selector model. ls-type:: annotation hl-page:: 6 hl-color:: green id:: 6575e62f-00ea-4fd8-9fb3-db84efe0c08d
• ➂ ls-type:: annotation hl-page:: 6 hl-color:: yellow id:: 6575e67a-6067-4c0f-962b-7dc424247900
• We excluded code snippets that lacked function parameters, variable declarations, and methods containing fewer than five lines before conducting our experiments. ls-type:: annotation hl-page:: 6 hl-color:: green id:: 6575e6d6-7328-4985-b00d-37263ae6ef5e
• three main types of code neural models: ls-type:: annotation hl-page:: 6 hl-color:: blue id:: 6575e6ff-54da-4b73-acd8-8e4662d73efd
• To highlight the significance of each path, CODE2VEC employs an attention mechanism to assign appropriate weights to the AST paths. ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e795-b41f-47ab-a59b-74f6f2906afb
• CODE2SEQ [8] also utilizes AST paths to represent code snippets. It employs an encoder-decoder model along with bi-LSTM units to embed the nodes of the AST paths and subsequently make sequential predictions ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e7a5-fb7e-4527-9538-efb3e79c06f1
• 2.3 million functions with paired documentation from six programming languages was utilized. ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e7bb-52e2-4d2d-9c04-b9462107cba2
• ransformer-based neural model that follows a similar encoder-decoder architecture as BAR ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e7cb-2e76-4e25-8852-e948bc0178e5
• To evaluate the effectiveness of the attacks, we conducted a thorough analysis of the impact of adversarial examples on the prediction outputs of the target models ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e7f5-a731-4045-bf37-4aa36663e67e
• ccuracy and F1 score for classification tasks ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e801-b834-461e-bedc-6239c039cfa3
• we were able to observe varying degrees of impact on their prediction capabilities ls-type:: annotation hl-page:: 7 hl-color:: yellow id:: 6575e81b-3f49-469c-beb7-c81bda6adf41 hl-stamp:: 1702225949640
• predicted labels are changed after the transformation process ls-type:: annotation hl-page:: 7 hl-color:: yellow id:: 6575e84d-d1b5-48ab-84c2-33f8f2519cd7
• I(P (xi)̸ = P (x′ ij )) is an indicator function that evaluates to 1 if the predictions for xi and x′ ij differ, and0 otherwise. ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e8a4-335c-4f12-86ac-dcc1d7d7b900
• ow many predictions were altered compared to the original predictions ls-type:: annotation hl-page:: 7 hl-color:: blue id:: 6575e8ac-37f6-445a-bc40-13a6ed7bf3fc
• ratio of transformed examples for which the prediction differs from the original prediction: ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e8fc-cec8-4c0d-b6d0-d3be113e21e5
• if the model changes its prediction on correct examples ls-type:: annotation hl-page:: 7 hl-color:: green id:: 6575e908-72f4-4944-9caa-90385a543ec8
• Ro = ls-type:: annotation hl-page:: 7 hl-color:: yellow id:: 6575e970-ec66-47ad-ab2f-0049c6d0c1ca
• active-learning guided adversarial attack framework for neural code models ls-type:: annotation hl-page:: 10 hl-color:: yellow id:: 6575ee7e-212a-42a3-8a26-9a142754745c