11 KiB
-
Design methods for the new database era: a systematic literature review authors: Noa Roy-Hubara, Arnon Sturm year: 2020 doi: 10.1007/s10270-019-00739-8 zotero: (Open) URL: http://link.springer.com/10.1007/s10270-019-00739-8 abstract: Over the last decade, a range of new database solutions and technologies have emerged, in line with the new types of applications and requirements that they facilitate. Consequently, various new methods for designing these new databases have evolved, in order to keep pace with progress in the field. In this paper, we systematically review these methods, with a view to better understanding their suitability for designing new database solutions. The study shows that while research in the field has expanded continuously, a range of factors still require further attention. The study identified important criteria in database design and analyzed existing studies accordingly. This analysis will assist in defining and recommending key areas for future research, guiding the evolution of design methods, their usability and adaptability in real-world scenarios. The study found that current database design methods do not address non-functional requirements; tend to refer to a pre-selected database; and are lacking in their evaluation.
-
Database technologies
-
Relational databases
Relational databases have dominated the field of database solutions for a long time and still remain a leading data- base solution.
They support the atomicity, consistency, isolation, and durability (ACID) notions (thus they reinforce the concept of a transaction)
Collectively, these features make relational databases best suited for managing structured data.
-
Object-oriented databases
They overcome the gap between relational models and object-oriented programming languages. Thus the application and the database speak the same language.
They support ACID properties, as well as OO properties as encapsulation, polymorphism, and inheritance.
-
NoSQL databases
Not Only SQL is used to describe four types of databases: key-value databases, column stores, document stores, and graph databases.
Most NoSQL databases do not support ACID and are instead based on the basically available, soft state, and eventual consistency (BASE) principles
In NoSQL systems, data structures are defined when reading the data as opposed to when writing it.
There is no acceptable standard query language.
NoSQL databases are more flexible and less rigid; for this reason, they do not support some concepts, such as joins and nested queries. Thus, when designing NoSQL databases, it is crucial to take into effect the needed queries. Any design that does not take the queries into account runs the risk of being ineffective in answering the desired queries.
For example, in a column-based database, if data are not stored together in a column family it cannot be queried together. It would be up to the programmer to create such a link in the application code, if this is at all possible. However, if the queries are first-class citizens, so to speak, in the design process, then the created column families will be able to answer them in a simplified manner.
-
Key-value databases
Data are organized as an associative array of entries, consisting of key-value pairs. Data processing is very fast.
-
Column Stores
Data are stored in columns rather than in rows. Most column stores are linked to analytical frameworks such as MapReduce, which enable fast analytics.
-
Document-oriented databases
They contain data that are de-normalized, semi-structured, and stored hierarchically in te form of key-value pairs (such as JSON and BSON).
Documents do not require a uniform structure. They usually support secondary indexes, which assist in full-text search and retrieval. The relationship between documents is expressed either by embedding a document in another document, of by referencing it in another document.
-
Graph databases
Data are represented as a set of vertices, linked together by edges. Most graph database providers implement property graph, in which both node and edges can be endowed with properties in the form of key-value pairs, and a name. Moreover, edges are binary and directed.
-
NewSQL
New group of databases, which share functionalities of traditional relational databases while offering some of the benefits of NoSQL technologies.
-
They use data model and support ACID
-
But they are also scalable and distributed, allowing for a higher level of performance over relational databases when processing a heavy workload.
With many new technologies, the need to choose the right type of database for a specific task is becoming a key challenge.
Designing a database capable of meeting these needs is key in exploiting new technologies to the fullest extent possible.
Important
: while flexible and schema-less, there is still a need for data models in the NoSQL world to better understand the storage of data.
-
Database design methods
Most of the existing approaches focus one specific technology instead of creating an integrative approach for several types of databases simultaneously.
!
Only three papers suggested methods for creating a hybrid database solution
-
Bjeladinovic, S.: A fresh approach for hybrid SQL/NoSQL database design based on data structuredness. Enterp. Inf. Syst. 12, 1–19 (2018)
The method was used to design a hybrid database that could support both relational and NoSQL databases. However, it did not provide a means for choosing the proper NoSQL type.
-
Herrero, V., Abelló, A., Romero, O.: NOSQL design for analytical workloads: variability matters. In: International Conference on Conceptual Modeling, pp. 50–64. Springer, Cham (2016)
The goal of the study was to design relational and co-relational (i.e., NoSQL databases excluding the graph database) databases for analytical workloads. Based on this method, the relevant type of database is chosen (relational or co-relational); as in the previous work, the study did not include any criteria for selecting from the different co-relational databases.
-
Zečević, I., Bjeljac, P., Perišić, B., Stankovski, S., Venus, D., Ostojić, G.: Model driven development of hybrid databases using lightweight metamodel extensions. Enterp. Inf. Syst. 12, 1–18 (2018)
They used a model-driven engineering approach for designing hybrid databases. Following this method, one model was used in each design step to create a hybrid solution. In this study, similar to the previous ones, no criteria for choosing the required NoSQL database were provided, and the method was only applied to a document database.
With regard to methods specificity, we identified a significant trade-off: if the method was tailored to a specific database, then it was not usable with other databases (we should remember that many database models exist today). If a method is fitted to work with all types of databases, then (unavoidably) it becomes more complex and more difficult to learn and use. This probably explains why most of the studies chose to focus on one specific NoSQL database type — given that this is more inclusive, with many more implementation options than a method tailored to one specific provider.
Within the surveyed papers, the** most addressed types ** were the graph and document databases, which are considered as the best suited databases for representing more complex data. Key-value databases, which are the simplest, garnered less attention.
Martin Fowler refers to schema-less design as an “implicit schema,” which is not good because “it is scattered amongst all the code that accesses the data, making it hard to find, thus slowing down any further development on top of the data structure.”
Steven Lott further stresses that abandoning the formal schema approach may lead to an informal, chaotic schema, and may even devolve into anarchy.
There is the need for a single place where the data structure is defined and can be used as an interface for those who are interested in processing the data.
-
-
Concerning non functional requirements (and thus DB selection)
Non-functional requirements were rarely used in the methods.
The choice of the right type of database for specific tasks is a crucial step in the design phase.
Design methods should provide full and accurate guidelines.
These could be presented in an easy-to-use format (such as designing relational databases with ERD), or by tools that take all considerations (data, functional, non-functional) into account.
Tools might be more formal and would lean on concepts such as machine learning in order to recommend the best design. If the former is the case, then the guidelines should require as little user interpretation as possible, i.e., different users with the same set of requirements would end up with the same design.
The Quest for a Database Selection and Design Method--@roy-hubaraQuestDatabaseSelection
-
Concerning the evaluation
This is crucial:
!
Mainly case studies and illustrative example are performed in the surveyed papers.
-
Concerning integrative approaches
Most methods are not integrative, i.e., they assume one type of database per application.
All methods assume that the database type is chosen a priori and is a sole fit for all the application requirements and goals.
However, they mostly discussed the combination of relational and NoSQL databases, which would require the user to decide upon the needed NoSQL technology (i.e., key-value, document, columnar, or graph database). This may be an issue, given that not all users have enough familiarity with all the technologies to be able to choose wisely.
-
What's next
- Use known concepts as much as possible. Results indicate that many of the studies used a known conceptual model as an anchor, or as a first step toward a new method. The use of a known model had the advantage of a known starting point, which could ease the adoption of the method.
- Data requirements are not enough. Other types of requirements have a crucial impact on database design. This could be the type of queries (e.g., key-value database only supports simple get and put operations; if complex queries and many updates are needed, another type of database is probably preferable), or their form (e.g., the design of column families relies heavily upon the needed queries).
- Choosing a database is just as important. With over 300 existing database solutions today, the carefully selection of the appropriate databases for the application at hand is crucial. Choosing a wrong database may prove to be an expensive mistake in the long term.
- Evaluations are hard. A case study approach might be adequate, but it must be complex (as in the real world), extensive, and comprehensible.
- Schema evolution. New design methods must take this into consideration, in order to make the evolution process possible and easy.
