logseq/pages/@White-box LLM-supported Low-code Engineering%3A A Vision and First Insights.md

tags:: #zotero date:: 2018 title:: @White-box LLM-supported Low-code Engineering: A Vision and First Insights item-type:: journalArticle original-title:: White-box LLM-supported Low-code Engineering: A Vision and First Insights language:: en authors:: Thomas Buchmann, René Peinl, Felix Schwägerl library-catalog:: Zotero links:: Local library, Web library

• Abstract
  • Low-code development platforms promise to empower citizen developers to define core domain models and rules for business applications. However, as domain rules grow complex, LCD platforms may fail to do so effectively. Generative AI, driven by large language models (LLMs), offers source code generation from natural language but suffers from its blackbox nature. Therefore, rather than having LLMs generate entire applications from single prompts, we advocate for a white-box approach allowing citizen developers to specify domain models, attaching constraints and operations as natural language annotations. These are fed incrementally into an LLM contextualized with the generated application stub. This results in precise, explainable, and deterministic generation of static application components, while offering citizen developers an appropriate level of abstraction. We report on a case study in manufacturing execution systems, where the implementation of the approach provides first insights.

• Attachments

  • File PDF {{zotero-imported-file IA6G7ECR, "Buchmann et al. - 2018 - White-box LLM-supported Low-code Engineering A Vision and First Insights.pdf"}}

• Notes

  • Annotazioni

    (27/7/2024, 12:18:08)

    • “grow complex” (Buchmann et al., 2018, p. 1) #ff6666

    • “offers source code generation from natural language but suffers from its blackbox nature.” (Buchmann et al., 2018, p. 1) #5fb236

    • “white-box approach” (Buchmann et al., 2018, p. 1) #ffd400 What do you mean?

    • “We report on a case study in manufacturing execution systems, where the implementation of the approach provides first insights.” (Buchmann et al., 2018, p. 1) #5fb236

    • “prompt” (Buchmann et al., 2018, p. 1) #ffd400 THere are too many papers in CORR. I would replace them with papers appeared in peer-reviewed venues!

    • “All further constraints and operations are phrased as natural-language annotations and remain attached to the domain model until the application stub has been generated.” (Buchmann et al., 2018, p. 1) #ffd400 It's a chain-of-though approach. What's the novelty?

    • “well-known weakness in doing maths can be overcome by teaching them how to use a calculator” (Buchmann et al., 2018, p. 2) #5fb236

    • “tool-use” of LLMs” (Buchmann et al., 2018, p. 2) #5fb236

    • “,” (Buchmann et al., 2018, p. 2) #ff6666

    • “do few-shot learning by presenting them some examples to be able to generate the desired output.” (Buchmann et al., 2018, p. 2) #5fb236

    • “One of the first to evaluate the use of LLMs in model driven development is [3].” (Buchmann et al., 2018, p. 2) #a28ae5

    • “applicability of LLMs to Business Process Management” (Buchmann et al., 2018, p. 2) #a28ae5

    • “identification of processes based on documents” (Buchmann et al., 2018, p. 2) #5fb236

    • “Vidgof et al.” (Buchmann et al., 2018, p. 2) #ff6666 Missing ref.

    • ““which tasks can be achieved with already existing models” and that data sets, benchmarks and new LLMs specialized on process management are required.” (Buchmann et al., 2018, p. 2) #5fb236

    • “[4] use” (Buchmann et al., 2018, p. 2) #ffd400 Avoid using references as subject of sentences. Write instead Forell et. al [4] use...

    • “mining declarative process models from textual descriptions,” (Buchmann et al., 2018, p. 2) #ff6666 Repeatead !!!

    • “Design of an initial domain model” (Buchmann et al., 2018, p. 2) #5fb236

    • “Partial code generation” (Buchmann et al., 2018, p. 2) #5fb236

    • “sage of a LLM contextualized with the existing source code in order to fill in the generated stubs one after another” (Buchmann et al., 2018, p. 2) #5fb236

    • “LLM GitHub Copilot 3.” (Buchmann et al., 2018, p. 4) #5fb236

    • “Creating the resulting source code requires manual copyand-paste in our experimentation, but will be automated in our envisioned approach.” (Buchmann et al., 2018, p. 4) #5fb236

    • “seven of the nine operation bodies were provided correctly by the LLM, while we had to re-phrase two of them.” (Buchmann et al., 2018, p. 4) #5fb236

    • “conversational nature of LLMs.” (Buchmann et al., 2018, p. 4) #5fb236

    • “We expect, however, that the here envisioned approach is applicable, also to input field validation and the like.” (Buchmann et al., 2018, p. 4) #ffd400 *too high level, too preliminary. I understand this is a workshop submission, but I think the propometed ideas are very high level. Aspects like generalizability,ways to mitigate hallucinations, etc are not mentioned at all. I see the proposed approach as a kind of chain of thoughs, and as a such I don't see clearly explained the research contribution of the envisioned work. Moreover, I'm missing to see the envisioned approach contextualized in some modeling/software engineering process.

    The white-box of the proposed approach is limited, because it relies on LLMs that by nature are black box and as users we can "touch" the inputs and analyze the outputs without any control of the inner behaviour of the model.*

    • “deterministic generation of applications” (Buchmann et al., 2018, p. 5) #ffd400 I'm not convinced about the deterministic characteristcs of the generation phase!