230 lines
10 KiB
Markdown
230 lines
10 KiB
Markdown
-
|
||
file:: [Yohannis e Kolovos - 2022 - Towards model-based bias mitigation in machine lea.pdf](file://C:\Users\david\Zotero\/storage/YKTMX6QH/Yohannis e Kolovos - 2022 - Towards model-based bias mitigation in machine lea.pdf)
|
||
file-path:: file://C:/Users/david/Zotero/storage/YKTMX6QH/Yohannis e Kolovos - 2022 - Towards model-based bias mitigation in machine lea.pdf
|
||
|
||
- users have to implement their code in general/statistical programming languages, which can be demanding for users with little programming and fairness in machine learning experience.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6443aff4-d368-41c5-8e3f-0fab050a9696
|
||
- These are some instances that show how biases in machine learning can promote unfairness.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6443b029-7c3c-4976-bdb2-636d041bfb2f
|
||
- bias metrics and debiasing algorithms
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6443b074-9c01-41ef-b382-1a57aabf1efc
|
||
- Data scientists usually work using their intuitions to narrow down the number of combinations of algorithms, parameters, and other factors to find the best models for given goals, datasets, and domains
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 6443b0c6-c048-4241-a9e7-a72164d9abf4
|
||
- ith lots of experimentation and trial and error [12 ], they have to go through all the narrowed combinations and test the produced models to identify which models are the best. Moreover, regardless of the availability of machine learning libraries, data scientists have to craft the search process from scratch in general/statistical programming languages (e.g., Python, R).
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 644a3766-d009-4482-a078-0d804d5c6544
|
||
- They also do not have to code the implementation of the search process in general/statistical programming languages since it can be automatically generated and then fine-tuned later on.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 644a37aa-573a-4390-b59a-afcbcd227cc9
|
||
- model and automate bias measurement and mitigation in machine learning.
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: purple
|
||
id:: 644a3855-1187-418f-bf42-23cb6705a978
|
||
- bias measurement and mitigation
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: yellow
|
||
id:: 644a3871-b847-40e7-ab54-d14701ae1758
|
||
hl-stamp:: 1682586829236
|
||
- bias mitigation model
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3877-8dbe-4b60-a0f4-57aa593c99c5
|
||
- experiment with different kinds of bias metrics
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a39d3-3b4b-4023-b1e9-38cfeccf8d49
|
||
- bias mitigation algorithms, datasets, classifiers, and their parameters to find the best combinations that reduce biases but with acceptable accuracy.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a39da-3532-42a1-b98f-26f120dea15c
|
||
- “the absence of any prejudice or favouritism toward an individual or group based on their inherent or acquired characteristics”
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3a41-c8f5-494d-ad66-3cdf2a45bb94
|
||
- The absence of fairness can be caused by a bias which is a systematic error or distortion from the actual state of affairs due to flaws in data collection and processing, study design, analysis, and interpretation [ 38 ].
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3ada-9495-4c00-8f51-927606cd2ff5
|
||
- Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A Survey on Bias and Fairness in Machine Learning. ACM Comput. Surv. 54, 6, Article 115 (jul 2021), 35 pages. https://doi.org/10.1145/3457607
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 644a3ae6-cfaa-4b6b-aa8e-965b590cad14
|
||
- For example, in 2006, it was found that an algorithm used for recidivism prediction produces a much higher false-positive rate for black people than white people
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3b17-7aea-4f0b-be03-dc0ff4d16772
|
||
- There are other examples of biases in machine learning, but these three examples already demonstrate that, in practice, machine learning is not always fair, and the unfairness can bring disadvantages to specific groups.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3b42-adb7-47a4-880d-e295bf7b7b85
|
||
- Euclidean Distance, Manhattan Distance, and Mahalanobis Distance are metrics to measure individual fairness
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3b6a-35ac-4991-8cc8-ef1568904f34
|
||
- Statistical Parity Difference
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3bca-a832-48ca-91ae-859bef40d6a1
|
||
- The decision tree to automatically select the most appropriate bias metrics for particular situations.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 644a3bf7-d6c6-496d-af65-4a854829aa7d
|
||
- [:span]
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 644a3c42-6144-4d05-9317-9f85ea65bca7
|
||
hl-type:: area
|
||
hl-stamp:: 1682586688904
|
||
- SAT is suspected to contain structural discrimination
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3c64-e752-4389-9cdd-a3d1948891ab
|
||
- Statistical Parity Difference
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3c76-900d-44cb-95a8-f9380b1fe2d6
|
||
- Disparate Impact
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 644a3c7d-17a7-4b38-b524-b3eb3f5aa6f4
|
||
- For example, SAT is suspected to contain structural discrimination [ 26, 35 ]. Statistical Parity Difference [ 18 , 26 , 35 ] and Disparate Impact [ 20, 26, 35 ] are the common metrics to measure equal fairness
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: yellow
|
||
id:: 644a3c81-dd2f-4e74-a637-d0cb20e328bb
|
||
hl-stamp:: 1682586825238
|
||
- Debiasing algorithms have been developed to reduce biases in machine learning, and they can be categorised based on the stages where they are applied in machine learning pipelines: pre-processing, in-processing, and post-processing
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 644a3df9-68ba-4e15-b2c8-c98f7422d25a
|
||
- The work in [ 26 , 35 ] provides guidance for choosing the most appropriate debiasing algorithms for specific situations.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 644a3e02-ae48-46a5-9755-0a242f3941eb
|
||
- [:span]
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 644a3e1b-78e8-4449-a717-d97ba70e719f
|
||
hl-type:: area
|
||
hl-stamp:: 1682587161734
|
||
- particularly in the pre-processing stage, since biases that are reduced in that stage are intrinsic in datasets
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 644a3e40-5617-48f2-a0e7-309689551deb
|
||
- Learning Fair Representation (LFR
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 644a3e55-8d49-4324-bcda-faebf39ba3e9
|
||
- Bias mitigation can also be applied in in-processing by interfering with training algorithms. T
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 644a3e77-292e-413c-9d38-524ff88516fc
|
||
- predicted attribute,
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 644a3ec7-5884-4911-b8a0-638a6265c4fb
|
||
- favourable class
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 644a3ecd-e055-4e5e-bbcf-4d8c17793358
|
||
- sensitive attributes
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 644a3ed1-a755-46e8-804a-52bc6e173ffb
|
||
- privileged and unprivileged classes
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 644a3ed5-ae88-4c50-94ed-76a275fde574
|
||
- They also select the bias metrics and mitigation algorithms that will be applied to datasets, models, and predictions
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: green
|
||
id:: 644a3ef2-5156-4b76-911c-3a22d4beadee
|
||
- Measure Original Biases Prior to Prediction (T02). In this task, users measure the original biases of the datasets in different metrics, such as accuracy, mean difference, etc. The results are later used as benchmarks when compared with the measurement results after prediction (T04) or bias mitigation (T06).
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: yellow
|
||
id:: 644a3f06-4ed0-4d0b-ac12-d922c6be39e9
|
||
- Measure Original Biases Post Prediction (T04). In this task, users measure the original accuracies and biases after performing the prediction in Task T03. The results are used as benchmarks later when comparing them against biases after bias mitigation. This action is skipped if prediction or Task T03 is not performed.
|
||
ls-type:: annotation
|
||
hl-page:: 5
|
||
hl-color:: yellow
|
||
id:: 644a3f9a-2318-4e82-9f20-dca4539de90b
|
||
- Therefore, they can provide a solid foundation for a well-bounded DSL, such as FairML.
|
||
ls-type:: annotation
|
||
hl-page:: 9
|
||
hl-color:: green
|
||
id:: 644a401e-db5d-41e8-9b21-73ef015968f6
|
||
- we can avoid it by leveraging 3rd party engines such as P2J, which can transform textual Python programs into Jupyter notebooks
|
||
ls-type:: annotation
|
||
hl-page:: 9
|
||
hl-color:: green
|
||
id:: 644a4036-a9ae-4500-b2f7-673e6debd0e2
|
||
- Julius A Adebayo et al. 2016. FairML: ToolBox for diagnosing bias in predictive modeling. Ph. D. Dissertation. Massachusetts Institute of Technology.
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: green
|
||
id:: 644a4059-8cd7-4e2d-b769-30ff7ff33612
|
||
- . They support data exploration, transformation, visualisation, and different algorithms for machine learning and data mining. Moreover, all of them are extensible, which means users can add new modules or scripts. However, as far as we are aware, there are no built-in modules for measuring and mitigating bias.
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: green
|
||
id:: 644a4088-1bc7-4422-a45c-922a10764bfb
|
||
- Arbiter [ 52], a domain specific-language designed for ethical machine learning.
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: green
|
||
id:: 644a4094-3643-4f83-bbbc-c0e0f2dbd69d
|
||
- transparency, fairness, accountability, and reproducibility
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: green
|
||
id:: 644a40a0-218e-4db0-b416-464e9daf9813
|
||
- Julian Zucker and Myraeka d’Leeuwen. 2020. Arbiter: A Domain-Specific Language for Ethical Machine Learning. Association for Computing Machinery, New York, NY, USA, 421–425. https://doi.org/10.1145/3375627.3375858
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 644a40bc-1b7d-4652-bb0c-e5ef852be51b |