12 KiB
12 KiB
tags:: #todoist-task ADMIN/MISSIONI progress:: {{renderer :todomaster}} external-link:: Program - FSE 2025
- ### Tasks
- DONE Preparazione slides per [[PAPERS/FSE2025-NIER]]
id:: 68435168-0fca-49d3-b735-8363d04c92ff
:LOGBOOK:
CLOCK: [2025-06-21 Sat 11:11:22]--[2025-06-21 Sat 11:11:23] => 00:00:01
:END:
- ### Notes
- #### Slides preparation
collapsed:: true
- "Teamwork makes the dream work: LLMs-Based Agents for GitHub README.MD Summarization", has been accepted for presentation in the Ideas, Visions, and Reflections track at FSE 2025
- Instructions to upload / send the slides ✉: [Web Link](https://outlook.office365.com/owa/?ItemID=AAMkADM1NGNiNjk0LTY0ZGUtNDgzOC04MDM5LWNhODNkYWNjNjU4YwBGAAAAAACT6qp78kRgRKuUMBdWEga%2FBwCOhWlC8F7PRKzlljZYZYQmAAAAAAEMAACOhWlC8F7PRKzlljZYZYQmAAgNERPBAAA%3D&exvsurl=1&viewmodel=ReadMessageItem)
- #### Day 1
collapsed:: true
- **FSE Keynote: Mark Harman, Peter O’Hearn, and Shubho Sengupta****[Harden and Catch for Just-in-Time Assured LLM-Based Software Testing: An Industrial Perspective and Open Research Challenge](https://conf.researchr.org/program/fse-2025/program-fse-2025/?date=Mon%2023%20Jun%202025%2BTue%2024%20Jun%202025%2BWed%2025%20Jun%202025%2BThu%2026%20Jun%202025%2BFri%2027%20Jun%202025&past=Show%20upcoming%20events%20only#)**
collapsed:: true
- Abstract:
collapsed:: true
- Despite decades of research and practice in automated software testing, several fundamental concepts remain ill-defined and under explored, despite their enormous potential real world impact. We show that these concepts raise exciting new challenges in the context of Large Language Models for software code and test generation. More specifically, we normally define and investigate the properties of hardening and catching tests. A hardening test is one that seeks to protect a code change against future regressions, while a catching test is one that catches such a regression, or a fault in new functionality introduced by a change. Hardening tests can be generated at any time, and may become catching tests when a future revision is caught. We also define and motivate the Catching “Just in Time” (JiTTest) Challenge, in which tests are generated “just in time” to catch new faults before they land into production, showing that it can also be repurposed to catch latent faults in legacy code. We set these challenges in the context of the work we have been doing on automated test generation at Meta, reviewing how we came to where we are now, and why we believe these open challenges represent such exciting opportunities for researchers, due to the enormous potential real world impact.
- **Bio:** Mark Harman is a Research Scientist at Meta London and a professor at University College London. He joined Meta following acquisition of his startup Majicke. He has published over 300 papers, with over 45,000 citations, and an H index of 105, making him the most highly cited scientist in the field of both Software Testing and of Program Analysis. His work has been deployed throughout Meta’s platforms for the past eight years, directly impacting over 3 billion people who rely on its product’s for social networking, community building and communication. His work has also directly impacted more than 200 million small companies that use Instagram, Facebook and WhatsApp to reach their customers and indirectly impacted many others that have deployed technology based on it, such as Microsoft, Google and Amazon. For his scientific work, Harman received the IEEE Harlan Mills Award and the ACM Outstanding Research Award in 2019. In 2020, he was elected a fellow of the Royal Academy of Engineering.
- **Bio:** Peter O’Hearn is a researcher at Meta AI and a Professor at University College London. He has made significant contributions to programming languages, logic, and software verification. Peter developed Separation Logic and Incorrectness Logic, theories which have been used in various reasoning tools, including Infer, a program analyzer that has detected hundreds of thousands of bugs at Facebook and other companies. Prior to joining Meta, Peter co-founded Monoidics, a verification startup that developed Infer and was acquired by Facebook in 2013. Peter has received numerous awards for his work, including the Godel Prize and being elected a Fellow of the Royal Society and Fellow of the Royal Academy of Engineering.
- **Bio:** Shubho Sengupta was a member of Meta’s FAIR organization for eight years. He has been working on GPUs for 20 years, developing the data parallel programming primitives that power most of compute workloads on GPUs today and authored the first standard library for GPUs (CUDPP). He has been working on AI for 10 years, first as a member of Baidu’s SVAIL and then as a member of FAIR. His AI journey started with speech to text and text to speech models with widely cited papers as DeepSpeech and DeepVoice. As part of these projects, he popularized the use of HPC techniques in AI and made synchronous training work at scale. Since then he has worked on large scale Reinforcement Learning (OpenGo), privacy and Machine Learning (CrypTen, Private ID) and more recently in generating software code from LLMs with assurances (Assured LLMSE and TestGen-LLM).
- A probabilistic typrewriter meets a world full of rules
- Should our tiiks be autonomous?
- Collection of LLMs that take care of the different SE tasks including testing, fixing bugs, etc.
- Autonomy is one direction and not the creation of the first line of codes
- Rethinking of the nowdays used tools including IDEs even though rarely does a field gets a chance to reinvent itself completely
- At what SE tasks are LLMs still not good?
- LLMs can replace junionor programmers?
- **How much oracle information do you need to weed out false positive test executions?**
- How can we distinguish regression test fails from failure of new functionalities?
- If a test doesn't even build a revision, can it still give useful signals?
- Finding a bug for an LLM is still difficult
- Ok, with LLMs we are generating code faster, but less buggy?
- Automated test generation is more important than ever!!!
- Testing vs Verification
- Grounds for optimism today! (oracle problem)
-
- #### Day 2
- **[Risk Assessment Framework for Code LLMs via Leveraging Internal States](https://conf.researchr.org/program/fse-2025/program-fse-2025/?date=Mon%2023%20Jun%202025%2BTue%2024%20Jun%202025%2BWed%2025%20Jun%202025%2BThu%2026%20Jun%202025%2BFri%2027%20Jun%202025&past=Show%20upcoming%20events%20only#)**
collapsed:: true
- Industry Papers
- [Yuheng Huang](https://conf.researchr.org/profile/fse-2025/yuhenghuang1) The University of Tokyo, [Lei Ma](https://conf.researchr.org/profile/fse-2025/leima) The University of Tokyo & University of Alberta, [Keizaburo Nishikino](https://conf.researchr.org/profile/fse-2025/keizaburonishikino) Fujitsu Limited, [Takumi Akazaki](https://conf.researchr.org/profile/fse-2025/takumiakazaki) Fujitsu Limited
- **Goal: Identify error patterns of LLMs!!**
- 
- 
-
- 
- **[An Empirical Study of Issues in Large Language Model Training Systems](https://conf.researchr.org/program/fse-2025/program-fse-2025/?date=Mon%2023%20Jun%202025%2BTue%2024%20Jun%202025%2BWed%2025%20Jun%202025%2BThu%2026%20Jun%202025%2BFri%2027%20Jun%202025&past=Show%20upcoming%20events%20only#)**
collapsed:: true
- Industry Papers
- [Yanjie Gao](https://conf.researchr.org/profile/fse-2025/yanjiegao) Microsoft Research, [Ruiming Lu](https://conf.researchr.org/profile/fse-2025/ruiminglu) Shanghai Jiao Tong University, [Haoxiang Lin](https://conf.researchr.org/profile/fse-2025/haoxianglin) Microsoft Research, [Yueguo Chen](https://conf.researchr.org/profile/fse-2025/yueguochen) Renmin University of China
- 
-
- **[Hallucination Detection in Large Language Models with Metamorphic Relations](https://conf.researchr.org/program/fse-2025/program-fse-2025/?date=Mon%2023%20Jun%202025%2BTue%2024%20Jun%202025%2BWed%2025%20Jun%202025%2BThu%2026%20Jun%202025%2BFri%2027%20Jun%202025&past=Show%20upcoming%20events%20only#)**
collapsed:: true
- Research Papers
- [Borui Yang](https://conf.researchr.org/profile/fse-2025/boruiyang) Beijing University of Posts ad Telecommunications, [Md Afif Al Mamun](https://conf.researchr.org/profile/fse-2025/mdafifalmamun) University of Calgary, [Jie M. Zhang](https://conf.researchr.org/profile/fse-2025/jiemzhang) King's College London, [Gias Uddin](https://conf.researchr.org/profile/fse-2025/giasuddin2) York University, Canada
- [DOI](https://doi.org/10.1145/3715735)
-
- **[Migrating Code At Scale With LLMs At Google](https://conf.researchr.org/program/fse-2025/program-fse-2025/?date=Mon%2023%20Jun%202025%2BTue%2024%20Jun%202025%2BWed%2025%20Jun%202025%2BThu%2026%20Jun%202025%2BFri%2027%20Jun%202025&past=Show%20upcoming%20events%20only#)**
collapsed:: true
- Industry Papers
- [Celal Ziftci](https://conf.researchr.org/profile/fse-2025/celalziftci1) Google, [Stoyan Nikolov](https://conf.researchr.org/profile/fse-2025/stoyannikolov) Google, Inc., [Anna Sjovall](https://conf.researchr.org/profile/fse-2025/annasjovall) Google, Inc., [Bo Kim](https://conf.researchr.org/profile/fse-2025/bokim) Google, [Daniele Codecasa](https://conf.researchr.org/profile/fse-2025/danielecodecasa) Google, Inc., [Max Kim](https://conf.researchr.org/profile/fse-2025/maxkim) Google
- TODO Da vedere questo paper [[@Migrating Code At Scale With LLMs At Google]]
- Da vedere questo paper per capire i software engineering tasks che hanno considerato per fare trainining della versione interna di Gemini
- **[Integrating Large Language Models and Reinforcement Learning for Non-Linear Reasoning](https://conf.researchr.org/program/fse-2025/program-fse-2025/?date=Mon%2023%20Jun%202025%2BTue%2024%20Jun%202025%2BWed%2025%20Jun%202025%2BThu%2026%20Jun%202025%2BFri%2027%20Jun%202025&past=Show%20upcoming%20events%20only#)**
collapsed:: true
- Research Papers
- [Yoav Alon](https://conf.researchr.org/profile/fse-2025/yoavalon1) University of Bristol, [Cristina David](https://conf.researchr.org/profile/fse-2025/cristinadavid) University of Bristol
- [DOI](https://doi.org/10.1145/3715761)
- This is also interesting to see
- **[Smaller but Better: Self-Paced Knowledge Distillation for Lightweight yet Effective LCMs](https://conf.researchr.org/program/fse-2025/program-fse-2025/?date=Mon%2023%20Jun%202025%2BTue%2024%20Jun%202025%2BWed%2025%20Jun%202025%2BThu%2026%20Jun%202025%2BFri%2027%20Jun%202025&past=Show%20upcoming%20events%20only#)**
- Research Papers
- [Yujia Chen](https://conf.researchr.org/profile/fse-2025/yujiachen) Harbin Institute of Technology, Shenzhen, [Yang Ye](https://conf.researchr.org/profile/fse-2025/yangye) Huawei Cloud Computing Technologies Co., Ltd., [Zhongqi Li](https://conf.researchr.org/profile/fse-2025/zhongqili) Huawei Cloud Computing Technologies Co., Ltd., [Yuchi Ma](https://conf.researchr.org/profile/fse-2025/yuchima) Huawei Cloud Computing Technologies, [Cuiyun Gao](https://conf.researchr.org/profile/fse-2025/cuiyungao) Harbin Institute of Technology, Shenzhen
- [DOI](https://doi.org/10.1145/3729405)
- Anche questo e' da vedere come riferimento a differenti code related tasks
-
-
-