12 KiB
12 KiB
tags:: #todoist-task, Talks, WORK, CONFERENCES date:: 31-08-2025 - 11:46 progress:: {{renderer :todomaster}}
- ### **Invited talk Tasks**
- DONE Raffinare l'introduzione del concetto di agenti AI
id:: 68ac6e54-e572-42f4-8116-625b4d85d757
collapsed:: true
- Creare / trovare una figura che rappresenti i concetti principali graficamente
- DONE Rivedere le slides di presentazione dei 5 tipi di agenti
- DONE Fare le slide conclusive
id:: 68b4c69b-fa0b-41c1-9633-b87c6cf6ab33
:LOGBOOK:
CLOCK: [2025-09-01 Mon 00:04:03]--[2025-09-01 Mon 00:04:04] => 00:00:01
:END:
- DONE Ridurre un pochino il numero di slides
id:: 68b4c69b-4d77-4192-9cff-d9f87cb200f4
- DONE Sistemare le immagini nel summary
id:: 68b4c69b-7fdf-4663-89af-ac6df65d88af
- Punti da sottolineare:
- Challenges related to the adoption of single-agent systems
- ### **Notes from the venue**
- **Program**: [LLMA4SE 2025 - 1st International Summer School on LLM-based Agents for Software Engineering](https://i3lab.unex.es/summer-school/#agenda)
- **Venue**: [Universidad de Extremadura](https://www.unex.es/) - [Caseres](https://es.wikipedia.org/wiki/C%C3%A1ceres)
- #### **READINGS**
collapsed:: true
- {{query (and [[Invited talk at LLMA4SE Summer School website]] [[Reading]]) (not [[GOALS-TODOIST]]) (not [[TODOIST-LOGSEQED]]))}}
query-table:: true
query-properties:: [:block]
- #### **First day of the event [[01-09-2025]]**
collapsed:: true
- ##### I talked with different people after my talk
- I talked with a student working with Javier from **Universitat Politècnica de Catalunya** on the EU project [HIVEMIND | Human-centred collaboratIVE MultI-ageNt framework for accelerating software Development and maintenance - HIVEMIND](https://hivemind-project.eu/) [[PROJECTS/MOSAICO]]
- **Jesús M. González Barahona** offered to collaborate concerning the "community" aspects of agents.
- I talked with **Jie Zhang** <jie.zhang@kcl.ac.uk>
- Here is the AIware arXiv track I mentioned, the deadline is this Friday: [https://2025.aiwareconf.org/track/aiware-2025-arxiv-track](https://2025.aiwareconf.org/track/aiware-2025-arxiv-track)
- There is also a benchmark and dataset track with the same deadline.
- The efficiency benchmark I mentioned is: [https://arxiv.org/abs/2402.02037](https://arxiv.org/abs/2402.02037)
- The test case generation benchmark (under submission): [https://arxiv.org/abs/2508.00408](https://arxiv.org/abs/2508.00408)
- [[Benchmarks]] seem to be very relevant and is attracting the community. We need to focus on that. There is one slide from the talk of Jesus we can have a look at ((68b57fc1-5214-405c-b4f7-58296590c06d))
- ##### **Generative AI models running in your own structure**
id:: 68bac2bb-1db8-41cb-81ed-7ef7f76e1491
- [tdd-workshop / SelfHostable-AI-Models · GitLab](https://gitlab.com/tdd-workshop/selfhostable-ai-models)
- 
- ((68b572ad-6444-494c-8929-43d2665321e6)) for instance if we are fine-tuning we can share the results of the fine-tuning
- ((68b572d4-f669-47ca-b942-f3a52b9e0683)) parts of the open-source aspects are applied to models
- ((68b57311-0b75-4a2a-86a9-b7236e7d2611))
- Important for different reasons, e.g., trust, data leakage,. You really control the information provided by the model.
- ((68b5737f-d5d0-41fd-bdc5-e574964d08ed)) [[STAR]]
id:: 68b57381-292d-4e53-8b79-b0378d3ddcca
- Papers on ((68b57a82-c115-4a56-a85b-4241b54bda29))
- What is fine-tuning ((68b57e64-b29f-42ea-9620-be4bbfa7edc2))
- Have a look at OpenRouter to see the attributes that are given to each of the managed models [[PROJECTS/MOSAICO/WP2]]
id:: 68b598e8-992a-433f-b685-5318c5ba0f4d
:LOGBOOK:
CLOCK: [2025-09-02 Tue 11:28:12]--[2025-09-02 Tue 11:28:13] => 00:00:01
:END:
- ##### **Workshop 1. Test-driven development with the help of generative AI**
- 
- 
- 
- 
- 
- 
- http://www.promptingguide.ai
- {:height 354, :width 519}
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- ```
export OPENROUTER_API_KEY="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"
```
- [Giovanni Rosa / Currante · GitLab](https://gitlab.com/grosa1/currante)
- [tdd-workshop / TDD-for-Code-Generation-Lab-Sept-25 · GitLab](https://gitlab.com/tdd-workshop/tdd-for-code-generation-lab-sept-25)
- #### **Second day of the event [[02-09-2025]]**
- ##### **MAS for Code Generation - Jie M.Zhang, King's College London**
- *History of agents*
- 1980s- software agents
- 1990s - MAS
- 2010 - Agents in deep reinforcement learning
- 2020- LLM-based agents, AI agents
- *There are a lot of definition about agents.*
- There is one that has been mentioned, from *Wooldrige, MIchael "Intelligent agents: The key concepts"*
- Hugging face in its "Agent course" gives the following definition: An agent is a system that leverages an AI model to interact with its environment....
- Tutorial fro ICML 2025: Jailbreking LLMs and Agentic Systems
- Mistral AI has also another definition:
- >AI agents are autonomous systems powered by large language models (LLMs) that, given high-level instructions, can plan, use tools, carry out processing steps, and take actions to achieve specific goals. These agents leverage advanced natural language processing capabilities to understand and execute complex tasks efficiently and can even collaborate with each other to achieve more sophisticated outcomes.
- [Agents Introduction | Mistral AI](https://docs.mistral.ai/agents/agents_introduction/#:~:text=%E2%80%8B,to%20achieve%20more%20sophisticated%20outcomes.)
- *When to use multiple agents?*
- A number of questions need to be considered including
- How big and complex is the task?
- Budget?
- Capabilities of a single agent?
- Do the agents have different expertise that I need?
- Advantages of multi-agents
- Objectivity: provide more reliable and objective feedback towards sub-task performance
- Clear instructions: task-switches can lead to significant performance degradation
- Scalability: Easier to scale systems by adding more agents
- Fault tolerance: if one agent fails, others can continue
- How to use agents for code genration?
- look at the "Tutorial fro ICML 2025: Jailbreking LLMs and Agentic Systems"
- [A survey of self-evolving agents](https://arxiv.org/abs/2507.21046) [[PROJECTS/MOSAICO]] [[TEACHING/SE4AS]] [[Reading]]
id:: 68b6a403-1ce2-4fe5-9668-54e32ac0e222
:LOGBOOK:
CLOCK: [2025-09-02 Tue 10:20:45]--[2025-09-02 Tue 10:20:46] => 00:00:01
:END:
- *Challenges and Opportunities*
- TODO [2503.13657 - Why Do Multi-Agent LLM Systems Fail?](https://arxiv.org/abs/2503.13657) [[Reading]]
- Non-determinism
- [En Empirical Study of the non-determinism of chatgpt and code generation](https://arxiv.org/abs/2308.02828) [[Reading]]
- Efficiency
- SWE-Effi: Re-Evaluating SWE Agent Solutions for their Efficiency
- [SWE Effi](https://centre-for-software-excellence.github.io/SWE-Effi/about/introducing-SWE-effi)
- [EffiBench: Benchmarking the Efficiency of Automatically Generated Code](https://arxiv.org/abs/2402.02037)
id:: 68b6a86a-0914-4535-b10a-d81519358f6a
- Hallucination
- [Hallucination Detection in LLM](https://arxiv.org/abs/2502.15844) [[Reading]]
-
-
- *Books*:
- Agent-based software development - Michael Luck, D'Inverno et
- [Agent-based Software Development - Michael M. Luck, Ronald Ashri, Mark D'Inverno - Google Books](https://books.google.es/books/about/Agent_based_Software_Development.html?id=AXMhngEACAAJ&redir_esc=y)
-
- ##### **AI and Software Development**
- Expectation from Industry about AI
- For CTO the main reason is 20% productivity gain
- For Engineer managers/team leads: faster delivery, satisfaction (+10 hours/week saved). They are worried about quality and maintainability
- For senior/staff developers: Skeptical, fear of being behind or deskilled (19% slower for experienced devs)
- For junor developers: embrace it quickly, unclear how it affects learning
- Il ruolo di AI secondo Microsoft
- 
- Risks from an industry perspective
- Intellectual property
- Security
- Reliability
- Reliance (e.g., what happens when Copilot is down)
- Guardrails
- Trust
- Evaluation
- How to compare different tools/agents?
- [Vibe Busters - Your AI-Generated Codebase is Haunted.](https://vibebusters.com/)
- When it works best
- Scripting
- Small projects
- Prototyping
- Code review
- When it doesn't
- Very large codebases
- Custom undocumented frameworks or tooling
- Debugging internals
- [MIT report: 95% of generative AI pilots at companies are failing | Fortune](https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/) [[Reading]]
- Why should we study
- Not knowing things in themselvers limits your ability to use them ot to realize that what you are reading is wrong [[TEACHING/SE4AS]]
- TESTING EFFECT
- >In psychology, the testing effect is ==the phenomenon where taking a practice test or retrieving information from memory enhances long-term memory retention and learning more effectively than passively restudying the same material==. This finding shows that the act of "testing" is not just an assessment tool but also a powerful learning tool, improving one's ability to recall information later.
-
- ##### **From workflow-based to fully-agentic applications: smolagents and LangGraph - Antonio Garcia-Dominguez**
collapsed:: true
- [agarciadom/llma4se-2025: Materials for LLMA4SE talk + workshop](https://github.com/agarciadom/llma4se-2025) [[TEACHING/SE4AS]]
- *From LMs to agents*
- Agents = LM + tools + prompt
- tool is a manually written code that an LM can invoke to retrieve information, or permorm an action on our behalf
- Interesting patterns of integrating different agents
-
- ##### **Workshop 2 - Development of agentic applications with human-in-the-loop via LangGraph**
source:: [agarciadom/llma4se-2025: Materials for LLMA4SE talk + workshop](https://github.com/agarciadom/llma4se-2025/tree/main)
collapsed:: true
- default langsmith API KEY
- ```
LANGSMITH_API_KEY="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"
```
- The Service langsmith API KEY
- ```
LANGSMITH_API_KEY="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"
```
- Context schema is to configure the agent
-