logseq/pages/SANER2024_paper_194.md

type:: REVIEWS tags:: year:: 2024 venue:: SANER full-title:: Are Adversarial Examples Suitable To Be Test Suites for Testing Deep Neural Networks date-start:: 15-11-2023 - 22:40 date-submitted:: 10-12-2023 external-links:: status:: DONE deadline-submission:: 09-12-2023 file::

- [[Highlights]]
	- #+BEGIN_IMPORTANT
	  ((6575f9a5-9503-43bc-8289-588ccbb928cd))
	  ((6575f9cb-a047-4cd7-9696-78deac13e7f2))
	  ((6575fc29-2ab6-4ff1-9214-c7eeaff40e66))
	  #+END_IMPORTANT
	- ((6575facc-59b7-43b4-8e16-96651ad23617))  #card #DL
	  id:: 65c8d476-6c2a-4f2b-afd8-49a829a0929e
	- ((6575fb8c-c5b4-430f-ade9-bc145a087250)) #IMPORTANT
	- ((6575fe4a-23d3-4ef9-a0e0-b2fb8e820abb))
		- What is y? is it the class of x?
	- ((6575ff46-9ff0-48a3-9507-d3ceb182b021))
		- This should be training loss, isn't it?
	- ((65761ab0-87d6-41a4-a44f-0dffbbc27076))
		- This is supposed to be a question, right?
	- ((65761d6a-3ff9-44c8-91a8-8c0072becbf0))
		- What do you mean with this?
- [[Comments]]
	- This paper delves into adversarial attacks and their applicability in testing deep neural networks. While it demonstrates that adversarial examples can expose vulnerabilities in DL models, it asserts that they fall short of serving as comprehensive test suites when viewed through the lens of test adequacy.
	- The topic presented holds significant potential, addressing the critical area of DL testing, which is gaining prominence. Regrettably, the paper's readability is inconsistent and does not approach the intended problem from a software engineering perspective. The writing style lacks homogeneity, with the initial sections being more accessible, while the inner parts exhibit less precision, a distinct style, and various grammatical and editorial errors.
	- The paper remains theoretical mainly, needing more concrete evidence for the drawn conclusions, and it does not provide replication packages. Despite the potential significance of the topic, there needs to be a software engineering perspective in the discussion to discern how the presented results contribute to evidence in the field of software engineering.
	- As the authors rightly highlight, DL testing is increasingly crucial today, and a clear understanding of the role and accuracy of adversarial examples is vital. However, for this paper to be more impactful in the software engineering realm, a clearer connection to software engineering perspectives and a more consistent presentation style supported by concrete evidence would be beneficial.
- [[REVIEWS/Notes]]
	- This can be of interest for [[people/umar]]
		- [[DL Testing]]
			- ((6575f97a-99a2-4368-8506-d1760dd988c5))
			- ((6575fa27-6d2d-44a0-b41d-4557c3701eff))
			- ((6575fa6a-4509-497d-984e-40d247cc0835))
	- The paper provides foundations demonstrating that although adversarial examples can detect the vulnerability of DL models, they cannot be considered as test suites from the perspective of test adequacy.
	- To this end the paper discusses adversarial attacks from the perspective of connections between **gradients' geometrical properties** and **local minima of the loss function**. This demonstrates that optimized AEs fall into several limited limited local minima. To explain such a concept, the paper make use of feature visualization technologies and optimized gradient-based attack algorithms to show that AEs share similar feature representations.
	- The given results suggest that **multi-objective search techniques** can produce more **diverse test suites** and cover **more decision logic**.
	-

• {{embed SANER2024-REVIEW-INSTRUCTIONS}}