logseq/pages/PAPERS___Query_model_repository.md

type:: journalpaper external-links:: todoist:: https://todoist.com/showTask?id=5331451779 external-links:: Software: Practice and Experience Rebuttal Letter - Google Docs date:: 13-11-2022 - 22:55 status:: ACCEPTED year:: 2024 venue:: Wiley Journal full-title:: Advanced discovery mechanisms in model repositories priority::
title:: PAPERS/Query_model_repository progress:: {{renderer :todomaster}}

- ## TODOs
	- DONE **Apologie for the broken PDF. Something wrong happened during the submission and the compilation of the uploaded latex files (THIS ONLY IN THE COVER LETTER)**
		- ((6468d660-3711-41a7-b2df-64da06d14e54))
		- ((6468d6d1-6cdc-4f87-a5a1-c69a580e1585))
	- DONE **Add [[links]] to the publicly available installation of the system and to the GitHub repository with the [[source]] [[code]]**
		- ((6468d663-3cb7-49fd-9227-3e565bfb1d3f))
		- ((64688a2e-43c1-402f-b072-3993237e6acc))
		- ((6468d6de-a529-490a-ae48-a0f66a4f54a5))
		- ((6468d6e0-f8fb-4969-8969-2a7c4b9995b1))
	- DONE **Add the rationale / motivations underpinning the list of features shown in Table 3 and Table 4 (clarify the method of answering RQ1, RQ2. Otherwise, one cannot tell whether your lists are complete, or indeed correct)**
		- We could say that those features are those that we collected [[by]] reading the [[PAPERS]] we selected. Those that are not covered are mentioned in the future [[WORK]] of some of the analysed [[PAPERS]]
			- ((6468d761-2db1-4b76-9601-ff97284fc6b1))
			- ((6468d6d4-1378-4228-96c8-2a19757d9481))
		- We can end the section containing Table 3 and Table 4 [[by]] saying what are the features that we decided to implement in MDEForge-Search and why such a selection
			- ((6468d767-2e1d-474a-a238-512a41756c16))
			- ((6468d6e1-cb2f-4792-9eae-bc3e0ffef4d8))
	- DONE **Add clearly what's new with respect to MDEForge (make clear the effort that has been put and avoid giving the impression that this is just a [[WORK]] that integrated existing components)**
		- ((6468d770-33c6-459c-b004-d8c5a3e8a5a7))
		- ((6468d66c-d159-4aec-b38c-cad8a68e5e44))
	- DONE **Why ElasticSearch? Why have we decided to use it? Simplify it's description**
		- ((64688a2e-2cbc-4e1a-8c57-b8e1b47f9762))
		- ((6468d6db-00f0-499b-bb53-9132f9076907))
	- DONE **Improve the [[presentation]] of the query [[language]]**
		- Remove Fig. 8
		- add a EBNF instead of the metamodel of the language
		- add explanatory examples
			- ((64688a2e-428c-42d2-9b58-f5a434a04f10))
	- DONE **Motivate the transformation chain case study**
		- I would say that it has been done in the context of the Lowcomote project
			- ((64688a2e-7fa0-41d0-923b-031533448d4c))
	- DONE **Improve the [[description]] of the DROID case study**
		- Say something more about how it works
		- Say that it was not available in MDEForge
			- ((64688be7-aa32-4e9f-99b2-60abc0e87ea4))
	- DONE **Clarify what we mean with reusability and distinguish it from discovery**
		- ((6468d6d7-15d8-45ea-a088-57b645cd2b24))
	- DONE Aggiornare la cover letter
- ## [[Comments]]
	- Reviewing: 1
		- [[Comments]] to the [[Author]]
		  [[#green]]==This [[Paper]] reports on MDEForge-Search, a system which provides advanced discovery features in model repositories. After a brief [[motivation]], a literature study is presented from which a set of relevant repository systems are extracted. Their supported features are grouped [[by]] [[category]], and innovative features are added that have not been addressed so far [[by]] existing systems. Subsequently, the architecture of MDEForge-Search is presented. Advanced discovery mechanisms are explained in general terms, and a case study of using MDEForge-Search is described.==
		- [[#green]]==Model repositories have gained increasing attention and are crucial for the success of model-driven software engineering. Thus, the [[Paper]] addresses an important topic - and proposes a system that seems to support a broad set of features, including advanced search. However, several issues need to be addressed that are described below.==
		- Unfortunately, in the proof presented to the reviewer [[#red]]==several figures were broken (missing inscription inside boxes)==; see fig. 1, [[2]], 4, 7, and 8. It seems that the text is contained in PDF, but it has a white color. This problem should be fixed before a potential resubmission. [[#red]]==The descriptions of the figures in the main text did not suffice to reconstruct the missing information.==
		  id:: 6468d660-3711-41a7-b2df-64da06d14e54
		- Furthermore, [[#red]]==MDEForge-Search does not seem to be publicly available==. No reference to the software Resources is given in the [[Paper]]. An internet search returned a [[link]] to MDEForge, a predecessor project that has been dismissed (as stated on the web page). Thus, it seems that only the [[authors]] of the [[Paper]] have been able so far to use MDEForge-Search.
		  id:: 6468d663-3cb7-49fd-9227-3e565bfb1d3f
		- A systematic and comprehensive literature study was performed to compare related [[WORK]]. The result is summarized in Tables 3 and 4, in which the features supported [[by]] existing systems are listed. Some of these features are not supported [[by]] any of these systems.
			- These should be motivated specifically, since [[#red]]==they result from the requirements the [[authors]] consider crucial and are not a direct result of the survey==.
			  id:: 6468d761-2db1-4b76-9601-ff97284fc6b1
			- Furthermore, it should be discussed and stated explicitly [[#red]]==which features are supported [[by]] MDEForge-Search (all of them?)==.
			  id:: 6468d767-2e1d-474a-a238-512a41756c16
			- Finally, [[#red]]==MDEForge-Search should be compared to MDEForge, its predecessor.==
			  id:: 6468d770-33c6-459c-b004-d8c5a3e8a5a7
		- The architecture of MDEForge-Search reuses existing software. This is fine since it does not make sense to "reinvent the wheel". However, it should be stated more clearly what goes beyond the reused components: [[#red]]==What has been added to glue these components together?==
		  id:: 6468d66c-d159-4aec-b38c-cad8a68e5e44
		- Concerning the functionality of MDEForge-Search, the advanced discovery mechanisms in Section 5 seem to constitute its added value. Partly, the [[presentation]] and the [[motivation]] of these mechanisms should be improved:
			- [[#red]]==In Section 5.1, the paragraphs on the scoring function could be deleted because it is provided [[by]] Elasticsearch and does not constitute an original contribution [[by]] the [[authors]].== [[#red]]==Instead, it should be motivated why Elasticsearch was selected (what are the benefits of using it?).==
			  id:: 64688a2e-2cbc-4e1a-8c57-b8e1b47f9762
			- The [[presentation]] of the [[#red]]==query [[language]] in Section 5.[[2]].1 should be improved==. On the one hand, it contains information that is not crucial for understanding (the processing chain in fig. 8, which is standard in Xtext). On the other hand, the [[language]] is described poorly. An EBNF would be more helpful than the metamodel, and more examples should be given to illustrate its capabilities.
			  id:: 64688a2e-428c-42d2-9b58-f5a434a04f10
			- [[#blue]]==The platform reuse== mechanisms allow to define and execute tools on [[MODELS]] inside the repository. But [[MODELS]] are heterogeneous artifacts, potentially living in different ecosystems. [[#red]]==How difficult is it to support such a feature?== And if the intention is to provide playgrounds, [[#red]]==is it desirable to performed experiments directly on the repository?==
			- The [[WORK]] on the [[#red]]==transformation chain service could be motivated better.== When is it needed, which problem does it solve? I can imagine applications in build systems for [[MODELS]]. Is this the intent behind this service?
			  id:: 64688a2e-7fa0-41d0-923b-031533448d4c
		- Concerning the case study, the [[#red]]==motivation behind and the functionality of the Droid framework should be described==. For readers not being familiar with this kind of recommender system, its benefit is not obvious. Furthermore, it should be clarified whether the integration with [[#red]]==Droid is a novel feature of MDEForge-Search==, or whether it has already been available in MDEForge (as fig. 16 may suggest).
		  id:: 64688be7-aa32-4e9f-99b2-60abc0e87ea4
	- Reviewing: [[2]]
		- SUMMARY:
		  [[#green]]==To advance model artifact reuse, the [[Paper]] reports about the [[author]]'s [[WORK]] on discovery of [[modeling]] artifacts. It gives an overview and identifies challenges of existing methods of [[modeling]] artifact discovery (systematic literature review), an presents the design of a query-based process for cloud-based discovery: MDEForge-Search on MDEForge. The discovery process should be easy, covering features from 3.5, and fit for reusing artifacts.==
		- OVERALL EVALUATION:
		  Potential impact:
			- The [[Paper]]
				- [[#green]]==Identifies the currently most pressing issues regarding model discovery and reuse.==
				- [[#green]]==Solves many of these in a generalizable and extensible way.==
				- [[#green]]==May enrich the workflow of users/developers/researchers in the various [[modeling]] communities.==
				- [[#green]]==Once the approach is fully implemented and databases curated, the [[Paper]] might have significant impact in MDE.==
		- Soundness and novelty:
			- [[#green]]==Good [[description]] of the systematic literature [[Comments]] process.==
			- [[#green]]==Identification of open challenges justifies the novelty of the contribution.==
		- Replicability:
			- The relevant details of their systematic literature review are already part of the paper.
			- However, [[#red]]==I cannot find [[links]] to test out their prototype, or sources, see below.==
			  id:: 64688a2e-43c1-402f-b072-3993237e6acc
		- Quality of [[presentation]]:
			- The [[Paper]] is generally well written and presents a thorough analysis of the current state-of-the-art and open challenges.
			- [[#green]]==Especially Section 3 is of very high quality and captures a wide variety of current issues.==
			-
		- DETAILED REMARKS
			- Many [[#red]]==figures are broken, i.e. lots of their text is completely unreadable.== As a result, some sections are hard to follow.
			  id:: 6468d6d1-6cdc-4f87-a5a1-c69a580e1585
			- [[#red]]==Are the features listed in 3.3 and 3.4 derived from the 15 [[PAPERS]], you studied? Please clarify the method of answering RQ1, RQ2==. Otherwise, one cannot tell whether your lists are complete, or indeed correct. Nevertheless, your extensive lists seem reasonable.
			  id:: 6468d6d4-1378-4228-96c8-2a19757d9481
			- Although the [[Paper]] focuses on discovery, I would like to see some elaboration on what entails "reusability" of artifacts for you. From the introduction on, you use both terms "discovery" and "reusability", as if they are well-defined. [[#red]]==Do you see "reusability" as: cloud-based remote model operation executions are possible (compare p. 16, 4.[[2]].[[2]], 5.3)? Would it be desirable, to offer all model operations that are possible locally also remotely? Why did you choose the ones you offer?==
			  id:: 6468d6d7-15d8-45ea-a088-57b645cd2b24
			- The explanation of [[#red]]==ElasticScore's score(q,d) function seems irrelevant on the chosen abstraction level. This also holds for many other parts in sections 4-6, they get a bit  lost in unncessary details.==
			  id:: 6468d6db-00f0-499b-bb53-9132f9076907
			- In your [[Abstract]], you write [[#red]]=="the applicability of [your] approach was assessed". Does this mean, you tested a working prototype, which integrates all features you wrote about in sections 4, 5, 6, 7? How did you assess it? Did I miss a download link/web access to your tool?==
			  id:: 6468d6de-a529-490a-ae48-a0f66a4f54a5
			- In general, you write many parts as if it were descriptions of a working tool. [[#red]]==Please clarify which parts are an approach/conceptual and which parts exist as working software. If working a working prototype or parts of it exist, please link== them.
			  id:: 6468d6e0-f8fb-4969-8969-2a7c4b9995b1
			- [[#red]]==Which of the features of your list do you not cover with your approach? Or do you cover all of them?==
			  id:: 6468d6e1-cb2f-4792-9eae-bc3e0ffef4d8
			- Giving the user a screenshot of the model in the discovery phase would be very helpful. Similarly, the playground would be more useful, if the [[MODELS]] are shown/can be interacted with directly.
			- Referenc[[#red]]==e on [[page]] 21, line 14 is broken.==
			- [[#red]]==Please do not abbreviate CRUD (create, read, update, delete), RS (recommender system) at first usage==
			- [[#red]]==The marking of line number 4 (p. 18, l. 23), at the bottom of your listing, although intended, is confusing.==