logseq/pages/hls__SATToSE_2023_paper_5_1683217643397_0.md

file:: SATToSE_2023_paper_5_1683217643397_0.pdf file-path:: ../assets/SATToSE_2023_paper_5_1683217643397_0.pdf

• Software Engineering for Artificial Intelligence. ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6453df5c-c0c0-4e35-92e7-43ff9ced7b58 hl-stamp:: 1683218295711
• e additional social and environmental factors come into play ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6453df66-63a3-4572-bd75-50197199fed7 hl-stamp:: 1683218293105
• role of input testing as a early indicator of the real-world performance of deep learning models in the context of image recognition ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6453df73-2a0a-413e-9624-48425243e7e0
• Nonetheless, when input testing is applied, the performance of the model drastically drops (reaching ≈30%), possibly highlighting the need for revisiting image recognition MODELS. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6453df9e-231f-4e52-9f7d-909f1ff998c9 hl-stamp:: 1683218337426
• researchers and practitioners have been focusing on object recognition and image classification, developing a large amount of AI systems with good performance ls-type:: annotation hl-page:: 1 hl-color:: green id:: 6453dfb7-09d8-4648-a74c-7db4128ec340 hl-stamp:: 1683218361425
• Their results indicated poor performance due to socio-environmental factors that impacted the in-vivo experimentation. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6453dfe7-0700-4df0-a1de-e4dc4d7528e8 hl-stamp:: 1683218410790
• ecological validity ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6453dff9-debe-4939-b741-9676db58bca8
• we aim to understand how generalizable the experimental results previously presented would be in a real-case scenario ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6453e003-ae4a-4e13-8ff5-24d8a585e808
• altering the inputs of the model to simulate an in-vivo experimentation [9]. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6453e019-586e-4f99-b895-ad46389550bc
• discover that the performance of the same model drastically drops when input testing is applied, hence suggesting that (1) the currently available MODELS would not properly WORK in practice and (2) input testing may provide insights to machine learning engineers on the generalizability of the model in practice, hence possibly informing their design actions. ls-type:: annotation hl-page:: 2 hl-color:: yellow id:: 6453e03f-e76a-470f-8a2b-f9980b90a56f hl-stamp:: 1683218498040
• gainst datasets using validation strategies such as percentage split or cross-fold validation. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6453e17c-dd74-416b-906c-bc8b414a4c46 hl-stamp:: 1683218817227
• o the best of our knowledge, there is no study that attempted to provide indications of the ecological validity of the models. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6453e187-9402-4147-8336-728a65cf1822
• although accuracy is one of the most analyzed metrics for understanding the effectiveness of an AI model, it is not an appropriate measure for unbalanced datasets since it does not distinguish between the numbers of correctly classified examples of different classes, leading to erroneous conclusions [ 21 ]. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6453e1b4-c5a4-4b55-b68c-a6ce99b491aa hl-stamp:: 1683218871061
• perspective ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6453e1dc-a155-4614-8b3b-6b389404d45d
• researchers ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6453e1e0-5ee3-4997-b134-0bdb796e1f95
• practitioners ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6453e1e2-501e-4e35-98bc-7ab0e1679946
• application of input testing methods impact the performance of an engineered Convolution Neural Network ls-type:: annotation hl-page:: 3 hl-color:: green id:: 6453e266-bd2e-4f72-90f5-51cfca81f772
• Convolutional Neural Network (CNN) ls-type:: annotation hl-page:: 3 hl-color:: yellow id:: 6453e278-5ca7-4a88-8045-0faefbd45f0d hl-stamp:: 1683219065836
• Once we had established a baseline, ls-type:: annotation hl-page:: 3 hl-color:: yellow id:: 6453e326-7ee7-44c9-a95d-7223e809e6d3
• potential behavior of the CNN in a real-world context. ls-type:: annotation hl-page:: 3 hl-color:: purple id:: 6453e386-6eca-4e1a-889a-dff25c89743f hl-stamp:: 1683219336471
• identify potential issues in the training set data. ls-type:: annotation hl-page:: 3 hl-color:: purple id:: 6453e3c0-8868-490b-befa-edc66fbd9a23
• different noises on the test set data to simulate a real-world scenario, e.g, rain or fog. ls-type:: annotation hl-page:: 3 hl-color:: yellow id:: 6453e3cd-e8c4-40dd-a28c-26578f1e411d hl-stamp:: 1683219408594
• Our replication study corroborates previous findings in the field of image recognition through AI. The performance of the CNN model is over 90% in terms of accuracy, F-Measure, Recall, and Precision. ls-type:: annotation hl-page:: 5 hl-color:: yellow id:: 6453e437-a1c2-478a-883b-d245eaed4d69 hl-stamp:: 1683219520161
• Our preliminary results indicated that the application of input testing methods lets the performance of the CNN decrease up to 60% with respect to what reported in literature. The overall performance ranged, indeed, between 19% to 33% in terms of precision, recall, F-Measure, and accuracy. ls-type:: annotation hl-page:: 5 hl-color:: yellow id:: 6453e45d-6ae0-4504-aca8-27c7d3c2667e