4.7 KiB
- Mdnotes File Name: khomhSoftwareEngineeringMachineLearning2018
Green Annotations (18/12/2020, 00:14:17)
"we still experience failures and shortcomings in the resulting soft ware systems. The main reason is the shift in the development paradigm in duced by ML and AI." (Khomh et al 2018:81)
"with ML techniques, these rules are inferred from training data (from which the requirements are gener ated inductively)." (Khomh et al 2018:81)
"This paradigm shift makes reasoning about the be havior of software systems with ML components difficult, resulting in software systems that are intrinsi cally challenging to test and verify." (Khomh et al 2018:81)
"the learned behavior of an MLbased system might be incorrect, even if the learning algorithm is imple mented correctly, a situation in which traditional testing techniques are ineffective." (Khomh et al 2018:81)
"critical problem is how to effectively develop, test, and evolve such systems, given that they don't have (complete) specifications or even source code corresponding to some of their critical behaviors." (Khomh et al 2018:81)
TESTING (ML) SYSTEMS THAT LACK SPECIFICATIONS OR EVEN SOURCE CODE (note on p.81)
"AI technology's strength comes from the ability to abstract up from different factors of varia tion between environments, to obtain models that can general ize and transfer to situations that weren't encountered before" (Khomh et al 2018:82)
"AI tech nologies' main challenge is" (Khomh et al 2018:82)
"he need for sufficient, labeled data to cover all important factors (features) of a given problem." (Khomh et al 2018:82)
"AI, in fact, needs more training data than humans do!" (Khomh et al 2018:82)
"appli cations of AI still risk being limited to domains in which labeled data is cheap." (Khomh et al 2018:82)
"instead of touting a "100 percent selfdriving car," auto motive companies should advertise their products as "AIassisted cars," with a clear list of the ways in which AI is assisting." (Khomh et al 2018:82)
"If a traditional computer science algorithm can solve a problem, we should just use that." (Khomh et al 2018:82)
"how can we perform adequate quality assurance (QA) of AI models, given that the number of environments in which the mod els will be deployed is unlimited and that the human operator will re quire a detailed explanation of any failures?" (Khomh et al 2018:82)
"use AI tech nology to reduce the search space of the environments to be tested" (Khomh et al 2018:82)
"AI impacts the hu mans' recommendations, those rec ommendations are also a human filter for AI failures." (Khomh et al 2018:83)
"Creating an efficient syntax for automatic differentiation that can deliver ease of implementation, per formance, usability, and flexibility is important but difficult." (Khomh et al 2018:83)
CHALLENGES (note on p.83)
"esting and debugging these implementations are also salient challenges." (Khomh et al 2018:83)
"How should software develop ment teams integrate the AI model lifecycle (training, testing, deploying, evolving, and so on) into their software process?" (Khomh et al 2018:84)
"What new roles, artifacts, and activities come into play, and how do they tie into existing agile or DevOps processes?" (Khomh et al 2018:84)
"testing" (Khomh et al 2018:84)
"intersections" (Khomh et al 2018:84)
"critical challenges of as suring the quality of AI and software systems in general." (Khomh et al 2018:84)