logseq/pages/hls__Yohannis e Kolovos - 2022 - Towards model-based bias mitigation in machine lea.md

file:: [Yohannis e Kolovos - 2022 - Towards model-based bias mitigation in machine lea.pdf](file://C:\Users\david\Zotero/storage/YKTMX6QH/Yohannis e Kolovos - 2022 - Towards model-based bias mitigation in machine lea.pdf) file-path:: file://C:/Users/david/Zotero/storage/YKTMX6QH/Yohannis e Kolovos - 2022 - Towards model-based bias mitigation in machine lea.pdf

• users have to implement their code in general/statistical programming languages, which can be demanding for users with little programming and fairness in machine learning experience. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6443aff4-d368-41c5-8e3f-0fab050a9696
• These are some instances that show how biases in machine learning can promote unfairness. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6443b029-7c3c-4976-bdb2-636d041bfb2f
• bias metrics and debiasing algorithms ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6443b074-9c01-41ef-b382-1a57aabf1efc
• Data scientists usually work using their intuitions to narrow down the number of combinations of algorithms, parameters, and other factors to find the best models for given goals, datasets, and domains ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 6443b0c6-c048-4241-a9e7-a72164d9abf4
• ith lots of experimentation and trial and error [12 ], they have to go through all the narrowed combinations and test the produced models to identify which models are the best. Moreover, regardless of the availability of machine learning libraries, data scientists have to craft the search process from scratch in general/statistical programming languages (e.g., Python, R). ls-type:: annotation hl-page:: 1 hl-color:: green id:: 644a3766-d009-4482-a078-0d804d5c6544
• They also do not have to code the implementation of the search process in general/statistical programming languages since it can be automatically generated and then fine-tuned later on. ls-type:: annotation hl-page:: 1 hl-color:: green id:: 644a37aa-573a-4390-b59a-afcbcd227cc9
• model and automate bias measurement and mitigation in machine learning. ls-type:: annotation hl-page:: 1 hl-color:: purple id:: 644a3855-1187-418f-bf42-23cb6705a978
• bias measurement and mitigation ls-type:: annotation hl-page:: 2 hl-color:: yellow id:: 644a3871-b847-40e7-ab54-d14701ae1758 hl-stamp:: 1682586829236
• bias mitigation model ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3877-8dbe-4b60-a0f4-57aa593c99c5
• experiment with different kinds of bias metrics ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a39d3-3b4b-4023-b1e9-38cfeccf8d49
• bias mitigation algorithms, datasets, classifiers, and their parameters to find the best combinations that reduce biases but with acceptable accuracy. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a39da-3532-42a1-b98f-26f120dea15c
• “the absence of any prejudice or favouritism toward an individual or group based on their inherent or acquired characteristics” ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3a41-c8f5-494d-ad66-3cdf2a45bb94
• The absence of fairness can be caused by a bias which is a systematic error or distortion from the actual state of affairs due to flaws in data collection and processing, study design, analysis, and interpretation [ 38 ]. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3ada-9495-4c00-8f51-927606cd2ff5
• Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A Survey on Bias and Fairness in Machine Learning. ACM Comput. Surv. 54, 6, Article 115 (jul 2021), 35 pages. https://doi.org/10.1145/3457607 ls-type:: annotation hl-page:: 11 hl-color:: green id:: 644a3ae6-cfaa-4b6b-aa8e-965b590cad14
• For example, in 2006, it was found that an algorithm used for recidivism prediction produces a much higher false-positive rate for black people than white people ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3b17-7aea-4f0b-be03-dc0ff4d16772
• There are other examples of biases in machine learning, but these three examples already demonstrate that, in practice, machine learning is not always fair, and the unfairness can bring disadvantages to specific groups. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3b42-adb7-47a4-880d-e295bf7b7b85
• Euclidean Distance, Manhattan Distance, and Mahalanobis Distance are metrics to measure individual fairness ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3b6a-35ac-4991-8cc8-ef1568904f34
• Statistical Parity Difference ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3bca-a832-48ca-91ae-859bef40d6a1
• The decision tree to automatically select the most appropriate bias metrics for particular situations. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 644a3bf7-d6c6-496d-af65-4a854829aa7d
• [:span] ls-type:: annotation hl-page:: 3 hl-color:: green id:: 644a3c42-6144-4d05-9317-9f85ea65bca7 hl-type:: area hl-stamp:: 1682586688904
• SAT is suspected to contain structural discrimination ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3c64-e752-4389-9cdd-a3d1948891ab
• Statistical Parity Difference ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3c76-900d-44cb-95a8-f9380b1fe2d6
• Disparate Impact ls-type:: annotation hl-page:: 2 hl-color:: green id:: 644a3c7d-17a7-4b38-b524-b3eb3f5aa6f4
• For example, SAT is suspected to contain structural discrimination [ 26, 35 ]. Statistical Parity Difference [ 18 , 26 , 35 ] and Disparate Impact [ 20, 26, 35 ] are the common metrics to measure equal fairness ls-type:: annotation hl-page:: 2 hl-color:: yellow id:: 644a3c81-dd2f-4e74-a637-d0cb20e328bb hl-stamp:: 1682586825238
• Debiasing algorithms have been developed to reduce biases in machine learning, and they can be categorised based on the stages where they are applied in machine learning pipelines: pre-processing, in-processing, and post-processing ls-type:: annotation hl-page:: 3 hl-color:: green id:: 644a3df9-68ba-4e15-b2c8-c98f7422d25a
• The work in [ 26 , 35 ] provides guidance for choosing the most appropriate debiasing algorithms for specific situations. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 644a3e02-ae48-46a5-9755-0a242f3941eb
• [:span] ls-type:: annotation hl-page:: 4 hl-color:: green id:: 644a3e1b-78e8-4449-a717-d97ba70e719f hl-type:: area hl-stamp:: 1682587161734
• particularly in the pre-processing stage, since biases that are reduced in that stage are intrinsic in datasets ls-type:: annotation hl-page:: 3 hl-color:: green id:: 644a3e40-5617-48f2-a0e7-309689551deb
• Learning Fair Representation (LFR ls-type:: annotation hl-page:: 4 hl-color:: green id:: 644a3e55-8d49-4324-bcda-faebf39ba3e9
• Bias mitigation can also be applied in in-processing by interfering with training algorithms. T ls-type:: annotation hl-page:: 4 hl-color:: green id:: 644a3e77-292e-413c-9d38-524ff88516fc
• predicted attribute, ls-type:: annotation hl-page:: 5 hl-color:: green id:: 644a3ec7-5884-4911-b8a0-638a6265c4fb
• favourable class ls-type:: annotation hl-page:: 5 hl-color:: green id:: 644a3ecd-e055-4e5e-bbcf-4d8c17793358
• sensitive attributes ls-type:: annotation hl-page:: 5 hl-color:: green id:: 644a3ed1-a755-46e8-804a-52bc6e173ffb
• privileged and unprivileged classes ls-type:: annotation hl-page:: 5 hl-color:: green id:: 644a3ed5-ae88-4c50-94ed-76a275fde574
• They also select the bias metrics and mitigation algorithms that will be applied to datasets, models, and predictions ls-type:: annotation hl-page:: 5 hl-color:: green id:: 644a3ef2-5156-4b76-911c-3a22d4beadee
• Measure Original Biases Prior to Prediction (T02). In this task, users measure the original biases of the datasets in different metrics, such as accuracy, mean difference, etc. The results are later used as benchmarks when compared with the measurement results after prediction (T04) or bias mitigation (T06). ls-type:: annotation hl-page:: 5 hl-color:: yellow id:: 644a3f06-4ed0-4d0b-ac12-d922c6be39e9
• Measure Original Biases Post Prediction (T04). In this task, users measure the original accuracies and biases after performing the prediction in Task T03. The results are used as benchmarks later when comparing them against biases after bias mitigation. This action is skipped if prediction or Task T03 is not performed. ls-type:: annotation hl-page:: 5 hl-color:: yellow id:: 644a3f9a-2318-4e82-9f20-dca4539de90b
• Therefore, they can provide a solid foundation for a well-bounded DSL, such as FairML. ls-type:: annotation hl-page:: 9 hl-color:: green id:: 644a401e-db5d-41e8-9b21-73ef015968f6
• we can avoid it by leveraging 3rd party engines such as P2J, which can transform textual Python programs into Jupyter notebooks ls-type:: annotation hl-page:: 9 hl-color:: green id:: 644a4036-a9ae-4500-b2f7-673e6debd0e2
• Julius A Adebayo et al. 2016. FairML: ToolBox for diagnosing bias in predictive modeling. Ph. D. Dissertation. Massachusetts Institute of Technology. ls-type:: annotation hl-page:: 10 hl-color:: green id:: 644a4059-8cd7-4e2d-b769-30ff7ff33612
• . They support data exploration, transformation, visualisation, and different algorithms for machine learning and data mining. Moreover, all of them are extensible, which means users can add new modules or scripts. However, as far as we are aware, there are no built-in modules for measuring and mitigating bias. ls-type:: annotation hl-page:: 10 hl-color:: green id:: 644a4088-1bc7-4422-a45c-922a10764bfb
• Arbiter [ 52], a domain specific-language designed for ethical machine learning. ls-type:: annotation hl-page:: 10 hl-color:: green id:: 644a4094-3643-4f83-bbbc-c0e0f2dbd69d
• transparency, fairness, accountability, and reproducibility ls-type:: annotation hl-page:: 10 hl-color:: green id:: 644a40a0-218e-4db0-b416-464e9daf9813
• Julian Zucker and Myraeka d’Leeuwen. 2020. Arbiter: A Domain-Specific Language for Ethical Machine Learning. Association for Computing Machinery, New York, NY, USA, 421–425. https://doi.org/10.1145/3375627.3375858 ls-type:: annotation hl-page:: 11 hl-color:: green id:: 644a40bc-1b7d-4652-bb0c-e5ef852be51b