17 KiB
17 KiB
type:: REVIEWS tags:: year:: 2024 venue:: MODELS full-title:: Extensions and Scalability Experiments of a Generic Model-Driven Architecture for Variability Model Reasoning date-start:: 01-05-2024 - 12:55 date-submitted:: 13-05-2024 external-links:: status:: DONE deadline-submission:: file:: @Extensions and Scalability Experiments of a Generic Model-Driven Architecture for Variability Model Reasoning parent:: todoist:: https://app.todoist.com/app/task/3777-extensions-and-scalability-experiments-of-a-generic-model-driven-architectu-6V2Wc3wcwpX6xgF6
- ### [[Highlights]]
collapsed:: true
- # Annotazioni
- (13/5/2024, 11:06:30)
- - “xtensions and Scalability Experiments of a Generic Model-Driven Architecture for Variability Model Reasoning” ([Author, 2024, p. 1](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=1&annotation=UIV4ZPMG)) #ffd400
- *What are the scientific challenges you addressed with this paper? It seems you performed engineering work to add the support for textual specification of variability models and for additional solvers. I appreciated the development effort even thought the research challenges are not clearly discussed.*
- - “ABSTRACT” ([Author, 2024, p. 1](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=1&annotation=ESMDHMVV)) #f0ff00
- *The abstract needs to be revised. Its not effecting in describing what's the goal of the paper.*
- - “situation,” ([Author, 2024, p. 1](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=1&annotation=SWINQBE8)) #f0ff00
- *What situation? What's the problem?*
- - “this initial proposal in three key ways: (1) we add the ability to reason on textual” ([Author, 2024, p. 1](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=1&annotation=TV9SP7KP)) #f0ff00
- *What's the problem and what are the limitations of exiting technologies? That's not clear.*
- - “there is no standard language to express variability models.” ([Author, 2024, p. 1](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=1&annotation=WTM34CAN)) #f0ff00
- *What about feature models and corresponding tools like featureide*
- - “These approaches do not fulfill the need for Domain-Specific VMLs (VML) beyond EFMs which is well attested in the literature. In particular, the use of DSVMLs in SPLE projects has both been argued for [17] and their use in combination with or as outright replacements for traditional FMs was explored in [67]. Unfortunately, tooling for a DSVML implies constructing, as is the case with the approaches cited above, an ad-hoc model transformation and analysis pipeline that will tightly couple the DSVML to the solver.” ([Author, 2024, p. 2](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=2&annotation=PNCE8WAE)) #f0ff00
- *What's the problem?*
- - “implements the MDA” ([Author, 2024, p. 2](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=2&annotation=47JGGF7H)) #f0ff00
- *What does it mean?*
- - “make the MDA demonstrably more versatile both in terms of the input VMLs and the output solver input languages.” ([Author, 2024, p. 2](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=2&annotation=DSVJE8B4)) #f0ff00
- *More versatile with respect to what?*
- - “A Proposal for a Generic MDA for Variability Model Reasoning” ([Author, 2024, p. 2](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=2&annotation=DQIFRMAF)) #ffd400
- *It's hard to follow without a concrete and explanatory example. The paper is not self-contained. There are many references to underpinning technologies and background concepts that make the paper difficult to follow.*
- - “The PLEIADES MDA uses CLIF as a pivot language between various VMLs and various solver input languages handling 𝑁 VMLs and 𝑀 solver input languages with only 𝑁 + 𝑀 transformations rather than 𝑁 × 𝑀 if variability models were directly transformed into formal knowledge bases as done in almost all previous tools reasoning on variability models.” ([Author, 2024, p. 2](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=2&annotation=9EWH9M82)) #f0ff00
- *Too long*
- - “hus superior or equal to that of the languages accepted as input by the four main solver paradigms used to reason on variability models in previous work” ([Author, 2024, p. 2](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=2&annotation=5JQ8YVZG)) #ffd400
- *How can you claim this in general?*
- - “The gray elements with dotted outlines correspond to the extensions necessary to cover the MDA’s limitations,” ([Author, 2024, p. 3](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=3&annotation=L5CWE8AM)) #ffd400
- *We are at sec 2.1 and I'm still missing a clear discussion of the "MDA limitations" mentioned by the authors.*
- - “MDA” ([Author, 2024, p. 3](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=3&annotation=FS3873J8)) #ffd400
- *MDA is precise technology consisting of different and precise OMG standard. In this paper, MDA is instead used to refer to some previous work presented by the authors elsewhere and that is extended by this work. The use of the term MDA is thus confusing in the paper.*
- - “The MDA is designed as a REST Web Service [56] where the model in a given VML is accompanied by two declarative specifications. The first encodes the semantics of the VML in the form of a mapping from the VML abstract syntactic elements to CLIF sentence elements.” ([Author, 2024, p. 3](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=3&annotation=K2WUCLV4)) #ffd400
- *The paper is full of sentences like this that refers terms and concepts that are not properly introduced in the paper. The reader is supposed to know them.*
- - “Textual VMLs” ([Author, 2024, p. 4](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=4&annotation=4AKYMUSZ)) #5fb236
- - “larger set of solver families” ([Author, 2024, p. 4](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=4&annotation=F9EYAWE4)) #ffd400
- *larger with respect to what?*
- - “calability benchmarks” ([Author, 2024, p. 4](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=4&annotation=N9SVHH34)) #5fb236
- - “measure the “cost”” ([Author, 2024, p. 5](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=5&annotation=QSVYF6DB)) #ffd400
- *What is the used metric?*
- - “transforms the textual model directly” ([Author, 2024, p. 5](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=5&annotation=RTZUJ3SX)) #5fb236
- - “The Universal Variability Language.” ([Author, 2024, p. 5](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=5&annotation=DX5MZEKK)) #a28ae5
- - “Common Variability Language (CVL)” ([Author, 2024, p. 5](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=5&annotation=GGKQHWV9)) #5fb236
- - “Variability Exchange Language (VEL)” ([Author, 2024, p. 5](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=5&annotation=5PT2PXW9)) #5fb236
- - “Implementing the Extension.” ([Author, 2024, p. 5](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=5&annotation=9VRUEVXV)) #a28ae5
- - “yellow” ([Author, 2024, p. 6](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=6&annotation=SDVF8XF9)) #ffd400
- *Something is wrong. I don't see elements in yellow.*
- - “Adding solvers from the SMT family” ([Author, 2024, p. 6](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=6&annotation=XM4CNB3E)) #ffd400
- *What are left then?*
- - “the other two solver families?” ([Author, 2024, p. 6](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=6&annotation=ANQF35M8)) #ffd400
- - “the of the” ([Author, 2024, p. 6](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=6&annotation=FMS6LIL8)) #ff6666
- - “blue” ([Author, 2024, p. 7](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=7&annotation=K7JUGBFS)) #ffd400
- *I don't see components in blue.*
- - “Solver Bridge, the only change to the MDA is that an additional subclass must be added corresponding the new solver, though this is mainly tied to the implementation.” ([Author, 2024, p. 7](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=7&annotation=PJ7C7JMJ)) #ffd400
- - “scaling as the sizes of the variability models grow.” ([Author, 2024, p. 7](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=7&annotation=J5RCUHQP)) #5fb236
- - “hundreds of features and constraints” ([Author, 2024, p. 7](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=7&annotation=JLHATBAY)) #5fb236
- - “Following these works, we therefore informally define “a reasonable amount of time” as reporting results in less than 2 seconds, using the lower bound given by said research.” ([Author, 2024, p. 7](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=7&annotation=XQEVGAVG)) #ffd400
- *Given an input specification in my opinion this very much depends on the solver and not on the components you devised, isn't?*
- - “Nevertheless, for the cases examined, this overhead is minimal and is unlikely to be the cause of future scalability issues.” ([Author, 2024, p. 8](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=8&annotation=D5DIYELK)) #ffd400
- *I don't see motivations for potential overheads.*
- - “Both the curves and the data suggest, nevertheless, a clear exponential trend as the model size grows, which corresponds to the expectations one would have of combinatorial problems as is the case of determining satisfiability of variability models” ([Author, 2024, p. 8](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=8&annotation=L9BLK9U2)) #ffd400
- *This means that eventually you can go beyond the 2 seconds if the model size increases.*
- - “This is quite unexpected, as we originally expected Z3 to outperform all others, as it is a mature and well-supported tool” ([Author, 2024, p. 9](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=9&annotation=5S9IEY5W)) #ffd400
- *BY the way, this is out of your control, right?*
- - “variability model verification and configuration automation for models written in a variety of languages and leveraging solvers from different paradigms.” ([Author, 2024, p. 9](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=9&annotation=L35LGTDW)) #a28ae5
- - “Concerning the PLEIADES extensions that we present in this paper, the closest related approach is FlamaPy’s [30]. It supports multiple VMLs, including UVL. However, it supports a single solving paradigm, SAT, whose KRL is known to be far less expressive than those of the CSP, CLP and SMT paradigms incorporated in our PLEIADES extension. These three paradigms can all be viewed as extensions of SAT to concisely represent non-Boolean, first-order and soft constraints. Adding new solver paradigms would require rethinking FlamaPy’s MDA with its pivot language at the abstract syntax level, in the direction of PLEIADES with a semantic level pivot language.” ([Author, 2024, p. 10](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=10&annotation=GBCDQF3C)) #ffd400
- *I suggest to move this part earler in the paper to motivate and support the paper!*
- - “The first extension, detailed in Section 3.1, concerns the capacity of the MDA to handle purely textual VMLs, whose absence was its main limitation” ([Author, 2024, p. 10](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=10&annotation=QG3JAHZH)) #a28ae5
- - “extended MDA to make it capable of reasoning on models written in UVL” ([Author, 2024, p. 10](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=10&annotation=A4BDSU4H)) #a28ae5
- - “SMT solvers[23], complementing the CSP and CLP paradigms already incorporated in the original MDA.” ([Author, 2024, p. 10](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=10&annotation=299964QN)) #a28ae5
- - “The second major contribution” ([Author, 2024, p. 10](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=10&annotation=QK4LRRHI)) #ffd400
- *WHat is the first major contribution? It is not listed, maybe it should be the extensions of PLEIADES previously mentioned?*
- - “The extended MDA has one major limitation that is worth discussing: while integrating new textual languages or solvers will not, in principle, require any architectural changes, they do imply a non-trivial implementation effort. This is due to the simple fact that a custom CLIF code generator must be fit to a (new or existing) parser for said language, and, for the solver, a code generator from a generic CSP representation has to be designed.” ([Author, 2024, p. 10](zotero://select/library/items/5DGDNYBE)) ([pdf](zotero://open-pdf/library/items/JMZHRYHA?page=10&annotation=DRLBJJY5)) #ffd400
- *This goes in contraddition with the "lowcost" for extending the architecture mentioned in Sec. 3*
- ### [[Comments]]
- #.tabular
- Summary of the paper
- The paper presents extensions to the PLEIADES tool to support the textual specification of variability models and integrate additional solvers. To this end, the authors discuss the software components that have been devised as extensions of the existing PLEIADES architecture. The resulting approach has been applied to consume UVL models available from a publicly available dataset and apply three solvers, i.e., Z3, SWI Prolog, and Minizinc.
- Summary of main points for/against the paper
- + Relevant topic
- + Engineering effort to extend an existing framework
- - Limited scientific challenges
- - Not always precise and easy to read
- Detailed evaluation
- The paper presents engineering and development efforts to add support to the PLEIADES framework for the textual specification of variability models and for additional solvers. Thus, the paper needs to be revised to clarify and highlight the novel scientific contributions beyond engineering achievements. In other words, the extensions introduced are practical and relevant; however, the paper must communicate the underlying scientific challenges these extensions are meant to address.
- The paper often refers to advanced concepts and previous works without sufficient explanation or background, which can confuse readers not already familiar with the subject. I suggest presenting an explanatory example to motivate the paper and show the limitations of existing technologies, which are addressed by the proposed approach. For instance, the limitations that are mentioned in the related work section in the paragraph "Concerning the PLEIADES ... pivot language" can be moved earlier in the paper and expanded to clearly show the limitations of the baseline.
- The scalability benchmarks presented are not convincing. From what I understand, PLEIADES functions primarily as a tool that facilitates the use of various solvers. As such, the bottleneck could potentially be the execution time of these supported solvers (thus, PLEIADES is only marginally involved in generating the problem to be solved), which might exceed the considered 2 seconds depending on the complexity of the specified constraints and problems involved.
- As a minor comment, the acronym "MDA" refers to a precise technology consisting of OMG standards. In this paper, MDA is instead used to refer to the PLEIADES architecture. I suggest revising the usage of the MDA acronym throughout the paper.
- Questions to authors (rebuttal phase)
- XXX