365 lines
13 KiB
Markdown
365 lines
13 KiB
Markdown
file:: [managingDataIntensiveEcosystems-chapter_1669986389575_0.pdf](../assets/managingDataIntensiveEcosystems-chapter_1669986389575_0.pdf)
|
||
file-path:: ../assets/managingDataIntensiveEcosystems-chapter_1669986389575_0.pdf
|
||
|
||
- e elaborate on the recent research effort devoted to the mining, analyzing, and evolving data-intensive software ecosystems
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6389f972-b49a-44e1-acef-4887efe6f7d9
|
||
- automated support
|
||
ls-type:: annotation
|
||
hl-page:: 1
|
||
hl-color:: green
|
||
id:: 6389f979-4478-452d-8bec-d59292e278c1
|
||
- Data-intensive software ecosystems comprise one or several databases and a collection of applications connected with the former
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0ccc-385c-4366-bdf9-f63f17c29661
|
||
hl-stamp:: 1670057166296
|
||
- it is the database’s responsibility to handle the query with its best performance
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0cf5-2a37-412e-8281-6a320d9ce449
|
||
- the increasing use of NoSQL databases poses new challenges for developers and researcher
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0d0c-a7e6-48de-b01d-edd1f457fb8c
|
||
- hybrid
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0d1b-c0d0-4b7a-af5b-d9dd34238158
|
||
- hybrid polystores
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0d1e-6115-463a-bdd5-ce2458135ffd
|
||
- We present recent research initiatives aiming to address those challenges
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0d2b-5c16-4390-a29f-62eb0c47ada5
|
||
- discuss mining techniques to determine how data is stored and managed in a data-intensive ecosystem
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0d35-d1fd-4e4d-9484-17c19ce17509
|
||
- static analysis and visualization techniques that exploit the mined information on the storage and manipulation of the ecosystem data
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0d4e-10df-4dfb-a1d9-c83cd776fcc4
|
||
- findings of empirical studies related to data-intensive software ecosystems
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b0d5d-e6f6-4fd3-b0c2-dbfed75a8308
|
||
- 2.1 Introduction
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: yellow
|
||
id:: 638b1330-a66f-4fbd-8308-3e24b3505f97
|
||
hl-stamp:: 1670058804184
|
||
- A subsystem may rely on one or more databases and their data will likely travel through the entire ecosystem. For maintaining and evolving this interconnected system network, it is fundamental to understand how data is handled all over it.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 638b15bc-87f0-42f4-ae17-7d364cb53bf6
|
||
hl-stamp:: 1670059455134
|
||
- approaches to mining how data is stored and managed in a data-intensive ecosystem. Such knowledge can serve various purposes, e.g., reverse engineering, re-documentation, visualization, or quality assurance approaches
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 638b15de-480f-489b-85a7-71155777af54
|
||
- We present two techniques to study the interaction points in the applications where they communicate with databases.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: yellow
|
||
id:: 638b15ed-79e5-4ff4-994c-9062c9857199
|
||
hl-stamp:: 1670059503856
|
||
- object-oriented languages usually follow the DAO (Data Access Object) design pattern to isolate the application/business layer from the persistence laye
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 638b1684-bca4-4a47-8cbf-5097fa38b4b0
|
||
hl-stamp:: 1670059655030
|
||
- DAO class implements all the functionality required for fetching, updating, and removing its domain objects
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 638b16b3-d511-4e2b-be5f-c41320b8a352
|
||
- many systems use combinations of multiple libraries
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 638b16e6-d8ed-4308-952f-b02393b68341
|
||
- hese libraries can partly or completely hide the actual SQL queries executed by the programs, generating queries at runtime before sending them to the database server
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 638b1736-ad72-417d-b1db-9ad4f809100b
|
||
- Statement.execute(...)
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 638b2d33-bf7e-402a-ba39-0bf7f4be6618
|
||
- identify the source code locations where database queries are executed and extract the set of actual SQL queries for each location
|
||
ls-type:: annotation
|
||
hl-page:: 4
|
||
hl-color:: green
|
||
id:: 638b2d88-428c-43ec-a7ce-0132bf1b1e7b
|
||
- the extraction process balances precision and the computation overhead of in-depth static analyses
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b2ddc-89ef-4e37-a330-b7c7a140e42d
|
||
- hey analyzed the evolution of the same three systems as they have been developed for more than seven years.
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b2dfa-e45b-4298-9463-455ab228e899
|
||
- schema-less storage allows faster data structure change
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b9415-f580-413a-9136-eb79b085e269
|
||
hl-stamp:: 1670091801940
|
||
- multiple
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b9442-98db-427f-8081-61be8efb6379
|
||
- co-existing implicit schemas
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b9446-8220-44c7-845e-39cb3df65e80
|
||
- support schema evolution in the schema-less NoSQL environmen
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b947b-7c4e-41ae-9825-4fe8a7f3755c
|
||
- schema extractio
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b9487-a71a-4c69-ae1b-bdfb29990f6e
|
||
- schema genera-
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b948a-d603-407e-96a1-2151ee52fc2d
|
||
- optimization
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b948c-af44-49aa-9c3b-f721a86e17a8
|
||
- allenges of NoSQL systems. A popular one is to support schema evolution in the schema-less NoSQL environment [12]. For example, researchers study automatic schema extraction [13], schema generation [14], optimization [15], and s
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b9490-bc1b-4652-ab79-50524392a68d
|
||
- ema suggestions [16]. Behind the scenes, such approaches mainly rely on a static analysis of the sou
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 638b9495-bbf1-4b0d-930f-39c9d06b0fc2
|
||
- andle the excessively dynamic features of the languag
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 638b94d5-5d8e-4865-b49a-6067089ab001
|
||
- h type inferenc
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 638b94da-ded3-4edd-af78-29438e30cc67
|
||
- data flo
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 638b94dc-6b1f-4193-8727-33c1ba2eb19c
|
||
- graphs
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 638b94df-25a6-453c-a6e3-ae10a235b365
|
||
- call
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 638b94e2-d06e-41c0-987f-329c7ab673eb
|
||
- r e q u i r e ( " mongoose " ) ;
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 638bda2b-cba2-444d-98b1-e16e31547976
|
||
- Schema is mapped to a MongoDB collection and defines the structure of the documents within that collection
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 638bda69-4d7e-4f8b-b4c1-807b48288907
|
||
- CodeQL,3 a code analysis engine developed by GitHu
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: green
|
||
id:: 638dea05-b734-42cf-ba67-af7596d43005
|
||
- We presented two static analysis approaches to study how applications communicate with their databases.
|
||
ls-type:: annotation
|
||
hl-page:: 9
|
||
hl-color:: green
|
||
id:: 638deb12-a99e-444b-bd76-6aeaad13b8a3
|
||
- Brink et al. present a quality assessment approach for SQL statements embedded in PL/SQL, COBOL, and Visual Basic code [
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: green
|
||
id:: 638deb40-0b44-435b-b4a8-0bbf70907f8c
|
||
- we show static analysis approaches in Section 3.2
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 638debc6-2a34-4970-9460-d8a86f1689be
|
||
- dependencies between the database and different components of an ecosystem
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 638debd4-f758-4129-82c6-4905b8dd4727
|
||
- because the application code and the database depend on each other, they evolve in parallel [8], resulting in increased complexity of the database communication code.
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 638dec2a-f439-463a-b203-d9877383dd22
|
||
- SQLInspect
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 638dec44-23ff-4e7a-a9fd-5ae1049bdf3e
|
||
- A lightweight analyzer can pinpoint a mistake already in the IDE before the developer commits it
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 638dec55-f0e7-461c-80d9-3bcd782aa073
|
||
- estimate the effort required in the past for adapting the applications to database schema changes
|
||
ls-type:: annotation
|
||
hl-page:: 13
|
||
hl-color:: green
|
||
id:: 638ded2c-cf12-4c80-8695-5f46a662aa3f
|
||
- hey rely on the previous analysis approach w
|
||
ls-type:: annotation
|
||
hl-page:: 13
|
||
hl-color:: green
|
||
id:: 638ded56-fbf9-4c36-9b42-a7ff0d7dd641
|
||
- information visualization field,
|
||
ls-type:: annotation
|
||
hl-page:: 13
|
||
hl-color:: green
|
||
id:: 638dee0b-a1bc-4832-8919-305b72fc7be2
|
||
- visualization is the preferred way of getting acquainted with large data
|
||
ls-type:: annotation
|
||
hl-page:: 13
|
||
hl-color:: green
|
||
id:: 638dee14-e0fe-4830-8c50-bde4f328bb66
|
||
- As we could see in the previous example, the city metaphor for visualizing software systems in 3D has been widely explored and has led to various implementations and approaches
|
||
ls-type:: annotation
|
||
hl-page:: 16
|
||
hl-color:: green
|
||
id:: 638df06f-0f30-4dcf-a5d9-7d7dc8a91631
|
||
- he resulting view seamlessly integrates data sources into a software city and enables a comprehensive understanding of a system’s source code and data.
|
||
ls-type:: annotation
|
||
hl-page:: 16
|
||
hl-color:: green
|
||
id:: 638e28f3-eba8-4572-887b-30397eab9037
|
||
- static analysis approach to identify SQL code smells in the database communication layer of data-intensive application
|
||
ls-type:: annotation
|
||
hl-page:: 17
|
||
hl-color:: green
|
||
id:: 638e2990-90d3-4a2a-89ed-c3a5459f611d
|
||
- a visualization approach of database behavio
|
||
ls-type:: annotation
|
||
hl-page:: 18
|
||
hl-color:: green
|
||
id:: 638e2a05-1563-4813-b432-011e4ce0af27
|
||
- 4 Empirical Studies
|
||
ls-type:: annotation
|
||
hl-page:: 19
|
||
hl-color:: green
|
||
id:: 638e2a2c-065e-468f-87ad-a0625628cc60
|
||
- Libraries.io
|
||
ls-type:: annotation
|
||
hl-page:: 19
|
||
hl-color:: green
|
||
id:: 638e2bce-6955-4294-817a-ffa094cf5ac2
|
||
- 40,609 projec
|
||
ls-type:: annotation
|
||
hl-page:: 19
|
||
hl-color:: green
|
||
id:: 638e2be7-2942-4fdf-9ecc-57eabdafa4f9
|
||
- However, the popularity of SQL technologies has recently decreased concerning NoSQL datastores
|
||
ls-type:: annotation
|
||
hl-page:: 19
|
||
hl-color:: green
|
||
id:: 638e2bf8-24be-4141-b33c-cb22f44e2f32
|
||
- more than 56% of systems relying on a key-value database also use another technology, typically relational or document-oriented.
|
||
ls-type:: annotation
|
||
hl-page:: 19
|
||
hl-color:: green
|
||
id:: 638e2c1a-13a4-4716-b1c8-695a7fb5d1b9
|
||
- revalence, Impact and Evolution of SQL Bad Smells
|
||
ls-type:: annotation
|
||
hl-page:: 21
|
||
hl-color:: green
|
||
id:: 638e2c48-923f-447a-846f-f588420318c0
|
||
- they further examined the 72 projects for which they could execute the tests and collect coverage reports.
|
||
ls-type:: annotation
|
||
hl-page:: 22
|
||
hl-color:: green
|
||
id:: 638e2d52-582f-4984-9dcb-cf3b8a10ee4c
|
||
- Non DB methods test coverage rate
|
||
ls-type:: annotation
|
||
hl-page:: 22
|
||
hl-color:: green
|
||
id:: 638e2d62-a0be-4741-b382-c2951241651e
|
||
- DB access methods test coverage rate
|
||
ls-type:: annotation
|
||
hl-page:: 22
|
||
hl-color:: green
|
||
id:: 638e2d67-1393-456a-b7cb-01e57f8fdc71
|
||
- The results indicate that the database manipulation code was poorly tested: 46% of the projects did not test half of their DB methods, and 33% did not test DB communication.
|
||
ls-type:: annotation
|
||
hl-page:: 22
|
||
hl-color:: green
|
||
id:: 638e2dce-2fe3-4753-a5a3-560b5b2d0a82
|
||
- This chapter summarized the recent research efforts devoted to mining, analyzing and evolving data-intensive software ecosystems.
|
||
ls-type:: annotation
|
||
hl-page:: 26
|
||
hl-color:: green
|
||
id:: 638e2e04-e975-4500-9586-d3d57e619fdf
|
||
- (1) both the databases and the programs are essential ecosystem artifacts
|
||
ls-type:: annotation
|
||
hl-page:: 26
|
||
hl-color:: green
|
||
id:: 638e2e18-c878-4bc6-82f0-be9f1b040b0b
|
||
- (2) mining, analyzing and visualizing what the programs are doing on the data may considerably help in understanding the system in general, and the databases in particular,
|
||
ls-type:: annotation
|
||
hl-page:: 26
|
||
hl-color:: green
|
||
id:: 638e2e21-7b89-4ec0-99c7-98df121af432
|
||
- (3) database interactions may suffer from quality problems and technical debt and should be better tested
|
||
ls-type:: annotation
|
||
hl-page:: 26
|
||
hl-color:: green
|
||
id:: 638e2e29-dfa7-459c-8a5c-98829e51c895
|
||
- (4) software evolution methods should devote more attention to the program-database co-evolution problem
|
||
ls-type:: annotation
|
||
hl-page:: 26
|
||
hl-color:: green
|
||
id:: 638e2e2f-89f1-4fa9-935a-0655e8616e76
|
||
- 4.6 Reflections
|
||
ls-type:: annotation
|
||
hl-page:: 23
|
||
hl-color:: red
|
||
id:: 638e2f92-7895-4037-8c73-41e7df23f499
|
||
hl-stamp:: 1670262677877 |