[logseq-plugin-git:commit] 2026-01-25T09:30:30.294Z

pages/@Automated Testing of Social Bias and Non-Discrimination Requirements in Generative AI.md

@@ -0,0 +1,81 @@
+tags:: [[#zotero]]
+title:: @Automated Testing of Social Bias and Non-Discrimination Requirements in Generative AI
+item-type:: [[journalArticle]]
+original-title:: Automated Testing of Social Bias and Non-Discrimination Requirements in Generative AI
+language:: en
+authors:: [[Sergio Morales García]]
+library-catalog:: Zotero
+links:: [Local library](zotero://select/library/items/QPEMC7ER), [Web library](https://www.zotero.org/users/1039502/items/QPEMC7ER)
+
+- [[Abstract]]
+	- The rapid proliferation of generative artificial intelligence (AI) systems has enhanced the creative and analytical capabilities of software systems to unprecedented levels. However, both their popularity with the general public and their increasing integration into software systems across social, industrial, and decision-making contexts raise pressing concerns about bias and compliance with international non-discrimination regulations. Despite growing awareness and research on these issues, bias assessment in generative AI remains fragmented, highly context-dependent, and often tied to specific AI models or datasets. Current bias assessment practices lack scalability, transparency, and reproducibility, limiting their applicability across diverse AI development environments and regulatory frameworks.
+- ### Attachments
+	- [PhD_Thesis___Sergio_Morales](zotero://select/library/items/7B9ZFN9J) {{zotero-imported-file 7B9ZFN9J, "PhD_Thesis___Sergio_Morales.pdf"}}
+	- [PDF](zotero://select/library/items/N8N4G2HJ) {{zotero-imported-file N8N4G2HJ, "García - Automated Testing of Social Bias and Non-Discrimination Requirements in Generative AI.pdf"}}
+- ### Notes
+	- # Review: Automated Testing of Social Bias
+	  
+	  ### **Thesis Summary**
+	  
+	  The thesis proposes a model-driven framework for the systematic assessment of social bias and non-discrimination requirements in Generative AI systems. The core contribution is a three-layered architecture that uses **Domain-Specific Languages (DSLs)** and meta-models to bridge the gap between abstract ethical guidelines (like the **EU AI Act**) and concrete technical testing (García). The framework is implemented through two primary tools: **LangBiTe** for text-to-text models and **ImageBiTe** for text-to-image models, both of which automate the generation, execution, and evaluation of bias tests (García)(García).
+	  
+	  ### **Organization of the Document**
+	  
+	  The dissertation is organized into seven chapters:
+	  
+	  - 
+	  **Chapter 1-2**: Motivation and Background on Generative AI bias and legal/ethical frameworks (García).
+	  
+	  - 
+	  **Chapter 3**: State of the Art in bias detection and MDE for AI (García).
+	  
+	  - 
+	  **Chapter 4**: The **Conceptual Framework**, detailing the shared meta-models for ethical requirements and test scenarios (García).
+	  
+	  - 
+	  **Chapter 5**: **LangBiTe**, focusing on text-to-text LLMs, including case studies on "augmented" prompt datasets and a leaderboard observatory (García).
+	  
+	  - 
+	  **Chapter 6**: **ImageBiTe**, focusing on text-to-image representational harms (stereotyping, denigration) and distribution-based evaluations (García).
+	  
+	  - 
+	  **Chapter 7**: Conclusions, a summary of scientific publications in top-tier venues, and a roadmap for future work (García).
+	  
+	  ### **Strengths of the Approach**
+	  
+	  - 
+	  **Traceability and Formalization**: By using MDE, the approach ensures a clear line of sight from non-discrimination requirements to specific test results, making the process repeatable and platform-agnostic (García).
+	  
+	  - 
+	  **Scalability via Augmentation**: The "LLM-assisted" generation of prompt templates (augmentation) allows developers to rapidly scale their testing libraries without exhaustive manual labor (García).
+	  
+	  - 
+	  **Stakeholder Inclusivity**: The DSL-based interface allows non-technical stakeholders (e.g., legal experts or ethicists) to participate in defining what constitutes "bias" for a specific project context (García).
+	  
+	  - 
+	  **Cross-Modal Versatility**: The framework successfully addresses both textual (boolean/quantitative oracles) and visual (distributional/qualitative oracles) representational harms (García).
+	  
+	  ### **Limitations and Threats to Validity**
+	  
+	  - 
+	  **Subjectivity of "LLM-as-Judge"**: The reliance on a **Community identifier LLM** or a **Qualitative evaluator LLM** introduces its own potential biases. As the author notes, "community identification is a subjective task" where even humans frequently disagree (García).
+	  
+	  - 
+	  **The Human-in-the-Loop Bottleneck**: While augmentation is fast, the thesis admits that human validation is "extremely recommended" to ensure the synthetic prompts and oracles align with a specific moral framework, which may limit full automation (García, p. 77)(García).
+	  
+	  - 
+	  **Context-Specific Sensitivity**: The findings in the case studies show that models might pass "generic" tests but fail significantly in specific contexts like **healthcare** or **law**, suggesting that a universal "unbiased" model does not yet exist (García).
+	  
+	  ### **Suggestions for Improvement and Future Work**
+	  
+	  - 
+	  **Mixture-of-Agents (MoA)**: To reduce the subjectivity of a single LLM-as-judge, the author suggests a 3-layer architecture of multiple agents to average out evaluations and increase confidence (García).
+	  
+	  - 
+	  **Continuous Monitoring**: Integrating the framework into a **CI/CD pipeline** would allow for the continuous monitoring of models as they evolve or are fine-tuned (García, p. 124).
+	  
+	  - 
+	  **Expansion of Modalities**: Future work could extend this model-driven approach to **audio** or **video** generation, which are increasingly relevant in the generative AI landscape.
+	  
+	  - 
+	  **Personalization of Evaluators**: Training the "Community identifier" models specifically on the stakeholders' moral and cultural criteria to ensure better alignment between the automated and manual analyses (García).

pages/PhD_Thesis_Garcia.md

@@ -1,9 +1,8 @@
 type:: [[REVIEWS/PhDThesis]] 
 tags:: 
 year:: 2026
-
 venue:: 
-full-title::
+full-title:: Automated Testing of Social Bias and  Non-Discrimination Requirements in  Generative AI
 date-start:: [[25-01-2026]] - 10:27
 date-submitted::
 external-links::
@@ -11,22 +10,8 @@ status:: [[DOING]]
 deadline-submission:: 
 file::
 parent::
-todoist::
+todoist:: [[@Automated Testing of Social Bias and Non-Discrimination Requirements in Generative AI]] 
 collapsed:: true
 
 	- ### [[Comments]]
-		- #.tabular
-			- ### Paper summary
-				-
-			- ### Strengths
-				- +
-			- ### Weaknesses
-				- -
-			- ### Detailed comments for authors
-				- Novelty:
-				- Rigor:
-				- Relevance:
-				- Verifiability & transparency:
-				- Presentation:
-				- Detailed comments:
 -