Auto saved by Logseq
This commit is contained in:
@@ -0,0 +1,56 @@
|
||||
-
|
||||
full-title:: How fair are we? From conceptualization to automated assessment of [[fairness]] definitions
|
||||
type:: [[conferencepaper]]
|
||||
external-links:: [Fairness_Metamodel - Online LaTeX Editor Overleaf](https://www.overleaf.com/project/640efd76bcfb98bbf9da63b4)
|
||||
todoist:: https://todoist.com/showTask?id=6808996721
|
||||
year:: 2023
|
||||
status:: [[REJECTED]]
|
||||
date-submitted:: [[05-05-2023]]
|
||||
venue:: [[ASE]]
|
||||
|
||||
tags:: #bias #fairness #machinelearning
|
||||
|
||||
- ## READINGS
|
||||
- [[@Towards model-based bias mitigation in machine learning]]
|
||||
- [[@A Survey on Bias and Fairness in Machine Learning]]
|
||||
- [[question]] Is there any work on fairness assessment?
|
||||
- [[question]] Are removing or mitigating fairness?
|
||||
- | L | M | M | G | V | S | D|
|
||||
| | | | 27 *Intro* | 28 | 29 *Sec 2*, Sec 3| 30|
|
||||
|1 | 2 Sec 4 Sec 5| 3 Sec 6,7 | 4| 5| | |
|
||||
- ## TASKs
|
||||
- DONE Rivedere sezione [[2]]
|
||||
date-submitted:: [[02-05-2023]]
|
||||
- DONE Vedere Sezione 3
|
||||
date-submitted:: [[02-05-2023]]
|
||||
- DONE Vedere [[Sezione 4]]
|
||||
date-submitted:: [[02-05-2023]]
|
||||
- DONE Vedere Sezione 5
|
||||
date-submitted:: [[04-05-2023]]
|
||||
- DONE Vedre Sezione 6
|
||||
date-submitted:: [[05-05-2023]]
|
||||
- DONE Vedere Sezione 7
|
||||
date-submitted:: [[05-05-2023]]
|
||||
- ## QUESTIONS TO ANSWER
|
||||
- {{query (and [[question]] [[PAPERS/FAIRNESS]])[[question]]}}
|
||||
- ## WRITING NOTES
|
||||
- ## Abstract
|
||||
background-color:: green
|
||||
- **Abstract da rivedere**
|
||||
- Fairness is a crucial property of software systems that affect people's lives, especially when they rely on machine learning techniques. Thus, there needs to be a clear and universal definition of what constitutes fair software in different domains and contexts. Therefore, software engineering (SE) researchers have proposed various methods and tools to assess and improve the fairness of software systems automatically. However, most of these approaches are based on predefined and fixed constituting elements of fairness definitions, such as metrics, criteria, and domains, that may not suit different users' and stakeholders' needs and preferences. To address this limitation, we present a novel approach, called \tool, that allows users to customize and define their own fairness concepts using a dedicated model-based modeling environment. Our system guides the user through the steps of specifying the fairness metrics, their composition, and the application domain of interest. Using a semi-automated strategy, we conduct a structured study to collect and analyze relevant literature on fairness assessment in SE. We compare \tool with the existing approaches and evaluate how they support the features identified by our study. Our results show that \textit{i)} most of the current approaches do not support user-defined fairness concepts and \textit{ii)} our approach can cover an additional application domain that is not supported by currently available tools, \ie bias in recommender systems for software engineering.
|
||||
- Proposta
|
||||
- Fairness is defined as “the absence of any prejudice or favouritism toward an individual or group based on their inherent or acquired characteristics”. Thus, fairness is a crucial concept, which is widely adopted in ethics and social domains. However, with the proliferation of machine learning in software systems, software engineering (SE) researches have been putting a lot of efforts to assess and improve fairness of software systems automatically. However, most of the proposed fariness assessment techniques are based on predefined fairness definitions, metrics, and criteria, which may not suit different users' and stakeholders' needs and preferences. To address this limitation, we present a novel approach, called \tool, that allows users to customize and define their own fairness concepts using a dedicated model-based modeling environment. Our system guides the user through the steps of specifying the fairness metrics, their composition, and the application domain of interest. Using a semi-automated strategy, we presents the results of a structured study to collect and analyze relevant literature on fairness assessment in SE. We compare \tool with the selected approaches and evaluate how they support the distinguishing features identified by our study. Our results show that \textit{i)} most of the current approaches do not support user-defined fairness concepts and \textit{ii)} our approach can cover an additional application domain that is not supported by currently available tools, \ie bias in recommender systems for software engineering.
|
||||
- Versione iniziale
|
||||
- Given a specific domain, the fairness of a software system has been defined as the absence of any bias that can affect the outcomes. In this respect, the software engineering (SE) community has been attracted by the fairness assessment task by proposing methodologies that detect possible biases automatically. While fairness is widely adopted in the ethics and social domains, such a definition is still vague in SE, thus affecting the fairness assessment process negatively. Although several automatic toolkits cover the whole process, none of them allows users to personalize fairness-related concepts, \ie metrics, their composition, and the application domain. To fill the gap, we propose a novel approach, called \tool, that aims at improving the existing literature by using a dedicated metamodel to drive the user in the definition of the fairness concept. First, we conduct a structured study to collect relevant approaches in the domain by adopting a semi-automated strategy. Afterward, we compare \tool with the analyzed strategies with aim of assessing how the existing literature supports the elicited features. Our findings show that \textit{i)} there is a lack of methodologies that allow tailored definitions of fairness and \textit{ii)} the proposed approach is able to cover an additional application domain alongside the traditional ones, \ie bias in recommender systems for software engineering.
|
||||
- ==Finale==
|
||||
- Fairness is a key concept in ethics and social domains, but also a challenging property to engineer in software systems. With the increasing use of machine learning in software systems, software engineering (SE) researchers have been developing techniques to assess and improve the fairness of software systems automatically. However, most of these techniques rely on predefined fairness definitions, metrics, and criteria, which may not capture the diverse needs and preferences of different users and stakeholders. To address this limitation, we propose a novel approach, called \tool, that enables users to customize and define their own fairness concepts using a dedicated modeling environment. Our approach guides the user through the steps of specifying the fairness metrics, their composition, and the application domain of interest. We also present a process we followed to collect and analyze relevant literature on fairness assessment in SE. We compare \tool with the selected approaches and evaluate how they support the distinguishing features identified by our study. Our results show that \textit{i)} most of the current approaches do not support user-defined fairness concepts and \textit{ii)} our approach can cover an additional application domain that is not supported by currently available tools, namely bias in recommender systems for software engineering.
|
||||
- ## Introduction
|
||||
background-color:: green
|
||||
- Over the last few years, machine learning (ML) has become very popular and increasingly pervasive as it underpins several systems we interact with to perform our daily personal activities (e.g., by interacting with intelligent speakers, or while getting relevent recommendations during the navigation of e-commerce systems). Interestingly, ML is being used to create data-driven decision making systems for which algorithmic solutions are complex to develop. However, the volume of training data, the relations among the pieces of collected information, and the continuous data acquisition and learning activities conducted to improve the accuracy of ML-intensive systems, make it difficult for developers to properly verify that such systems are free from bias. To mitigate such issues, the scientific community has recently started to evaluate ML-intensive systems in terms of quality attributes, including explainability, privacy, and fairness \cite{villamizar_requirements_2021,muccini_software_2021,giray_software_2021}.
|
||||
-
|
||||
- Fairness (and relative unbias) can be defined as \textit{"the absence of any prejudice or favouritism toward an individual or group based on their inherent or acquired characteristics"} \cite{mehrabi_survey_2021}. Although the concept of \textit{fairness} has been introduced primarily in the context of machine learning (ML) systems (mainly for legal reasons \cite{caton_fairness_2020,mehrabi_survey_2021}), the concept of \textit{bias} originates in the ethical domain (referring to the general concept of \textit{discrimination}) and has been adapted to the AI and ML domain \cite{olteanu2019social}. In general, an ML system is said to be \textit{biased} (or \textit{unfair}) if the output of the ML model is directly correlated to the value of a set of so-called \textit{sensitive variables}, like the race or gender of a person. Starting from a definition of bias for a particular domain, several fairness analyses can be computed by selecting a set of metrics compliant with a particular scope. The relevance of software fairness in ML-intensive systems has been also made popular by infamous incidents happened to the recruitment instrument employed by Amazon (Reuters, 2018) and the criminal recidivism predictions made by the commercial risk assessment software COMPAS (AI Incident Base, 2016).
|
||||
- ## Section 2
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
Reference in New Issue
Block a user