tags:: [[#zotero]] date:: 2026 title:: @Determining Application Test Results Using Adaptive JSON item-type:: [[journalArticle]] original-title:: Determining Application Test Results Using Adaptive JSON language:: en library-catalog:: Zotero links:: [Local library](zotero://select/library/items/PIWLIAUZ), [Web library](https://www.zotero.org/users/1039502/items/PIWLIAUZ) - [[Abstract]] - JSON has become the most popular data exchange format for APIs, especially with the rise of both REST and GraphQL APIs. However, conventional techniques for writing and updating API tests remain laborious and time-consuming. Recent API testing solutions, such as Snapshot Testing and Traffic Mirroring Testing, attempt to address these challenges by leveraging a new data-driven testing method where each JSON API response is compared directly against a baseline. Unlike conventional testing methods, where the expected results and comparison logic are stored together in the test code, this new approach separates them. However, issues such as dynamic data or frequently changing yet irrelevant fields can still cause tests to fail unexpectedly, leading to high maintenance costs. This paper introduces the Adaptive JSON Comparison process, a novel methodology for creating flexible, stable, and maintainable tests for validating JSON API responses. It presents Skeleton JSON, a structural abstraction of the API response, and Adaptive JSON, a hybrid format that selectively combines JSON nodes with Skeleton JSON nodes. The transformation is controlled by a declarative data structure, Adaptation Set, which allows engineers to define precisely which parts of an API response to validate by value and which by structure, while systematically ignoring irrelevant or dynamic sections. Furthermore, a flexible JSON Comparison Filter enables the application of customized comparison logic to specific JSON nodes. We implemented and applied the Adaptive JSON Comparison method to both Snapshot Testing and Traffic Mirroring Testing scenarios. Our evaluation of a suite of APIs at a global e-commerce company demonstrates a marked reduction in spurious test failures and a significant decrease in the manual effort required to maintain test suites. By addressing key challenges in JSON-Driven Testing, our work streamlines quality assurance and makes automated testing more practical and scalable. - ### Attachments - [PDF](zotero://select/library/items/8AEAME62) {{zotero-imported-file 8AEAME62, "2026 - Determining Application Test Results Using Adaptive JSON.pdf"}} - ### Notes - I'm reviewing a research paper and I took the following notes: # Annotations (14/09/2025, 17:59:08) - “writing and updating API tests” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #a28ae5 - “Recent API testing solutions, such as Snapshot Testing and Traffic Mirroring Testing, attempt to address these challenges by leveraging a new data-driven testing method where each JSON API response is compared directly against a baseline.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #5fb236 - “Unlike conventional testing methods, where the expected results and comparison logic are stored together in the test code, this new approach separates them.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #a28ae5 - “This paper introduces the Adaptive JSON Comparison process, a novel methodology for creating flexible, stable, and maintainable tests for validating JSON API responses.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #a28ae5 - “Skeleton JSON, a structural abstraction of the API response, and Adaptive JSON, a hybrid format that selectively combines JSON nodes with Skeleton JSON nodes.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #5fb236 - “Adaptation Set, which allows engineers to define precisely which parts of an API response to validate by value and which by structure, while systematically ignoring irrelevant or dynamic sections” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #a28ae5 - “In recent years, Representational State Transfer (REST) has become the most popular API style.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #5fb236 - “JavaScript Object Notation (JSON) has become the most popular data exchange format for APIs.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #5fb236 - “However, conventional techniques for writing and updating API tests, where the expected API responses and the comparison logic are stored together in the test code, remain laborious and timeconsuming.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #a28ae5 - “a new data-driven testing (DDT) method, herein referred to as JSON-based data-driven testing (JSON-DDT), which leverages JSON as data sources, has emerged.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #a28ae5 - “g[” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #ff6666 *missing space* - “g[” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #ff6666 *Missing space. Many occurrences of this problem.* - “[14],” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #ffd400 *References contains many items from the gray literature even for supporting critical aspects of the paper. Instead of referring medium.com or postman.com posts, I suggest refer instead peer-reviewed papers.* - “By separating the test environment settings, the test input parameters, and the expected API responses from the comparison logic, and by making the comparison logic a standard sharable library, a lot of manual steps to write test code are avoided in this new method, therefore makes the tests easy to write and friendly for humans to understand.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #5fb236 - “However, JSON-DDT has its own limitations and challenges, since the flexibility of using customizable test code to define and compare the expected results is largely reduced.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 1) #ffd400 *The mentioned limitation is not evident and properly stressed. Authors introduce that the paper is going to address challenges that are not clear at this very early point of the work.* - “is resilient to insignificant changes” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ffd400 *What does it mean?* - “JSON Schema” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ffd400 - “distinct advantages for the specific context of automated API testing” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ffd400 *Such advantages need to be clarified. For instance, what's new with respect to 10.1016/j.knosys.2016.03.020 and 10.1109/MODELS50736.2021.00033 ?* - “Skeleton JSON is designed with simplicity and test automation as its primary goals. It is lightweight,” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ffd400 *Ok saying that at Section 2, but I expect to see some validation later in the paper supporting such mentioned strengths of Skeleton JSON.* - “An Adaptive JSON object can be: (1) a direct subset of the original JSON; (2) a complete skeleton derived from all or part of the original JSON; or (3), most powerfully, a hybrid containing a selection of both original and skeletonized nodes.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ffd400 *Quite vague.* - “,” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ff6666 - “The core principle of Adaptive JSON is selective abstraction. Engineers can choose to preserve the exact values for stable fields while converting volatile or irrelevant fields to their Skeleton JSON type representation.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ffd400 *What are the requirement that have driven the definition and development of "Adaptive Json"* - “• The price field, which may fluctuate or be irrelevant to a specific test case, is abstracted to its type, "number".” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #5fb236 - “This targeted approach allows engineers to create robust validation rules that are resilient to insignificant data changes, directly addressing a primary cause of brittleness in automated testing.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #a28ae5 - “This design separates the testing logic (the consumer) from the core comparison engine (the provider)” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #5fb236 - “The Adaptive JSON Comparison Provider is the core component, responsible for executing the comparison. It contains two primary internal modules: • Automated Generation: The Adaptive JSON Conversion Module, which takes a raw JSON object and an Adaptation” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 2) #ff6666 *This can be removed because it's not updated with respect to the content of Fig. 3. The next paragraph seems to be updated.* - “To implement the concepts of Adaptive JSON, we designed a modular system based on a consumer-provider architecture, as illustrated in the teaser Figure 3. This design separates the testing logic” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 3) #ffd400 *This is a repetition in the text.* - “**Adaptive JSON Comparison Provider**” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 3) #ffd400 *The usage of ** ** seems to be related to MD syntax for making the text bold.* - “In this paper, we addressed the critical challenge of brittleness in modern data-driven API testing. We identified that naive comparisons of JSON responses in techniques like Snapshot Testing and Traffic Mirroring lead to high maintenance costs due to frequent, spurious failures from dynamic or irrelevant data variations.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 7) #ffd400 *This requires to be substantiated with some concrete and motivational example.* - “Our core contributions include the concepts of Skeleton JSON and the hybrid Adaptive JSON format, which are controlled by a declarative Adaptation Set. This approach allows engineers to precisely define which parts of an API response to validate by value and which by structure. As demonstrated in our case studies, applying this method to Snapshot Testing and Traffic Mirroring effectively filters out irrelevant data noise, significantly reducing false positives and making these powerful testing techniques more practical and robust in real-world environments.” (“Determining Application Test Results Using Adaptive JSON”, 2026, p. 8) #ffd400 *The strenghts of the Skeleton Json and Adaptive JSON is not presented with respect to existing approaches that are able to retrieve JSON schema out JSON documents. The proposed approach is not compared with any API testing techniques. THe paper is not always well written. There are repeated text. THe authors can use additional papges to properly present the strenghts and uthe limtiations of the proposed approach.* COnsider that those that are tagget with #5fb236 are just highlights, those that are tagged with #e56eee and #a28ae5 are imporant sentences. Please pay attention instead to the notes that are tagged with #ffd400. Those that are tagged with #ff6666 are typos or errors. Could you please draft a review by organizing it as follows: SUMMARY: Just a few sentence to summarize the work STRENGHTS: WEAKNESSES: COMMENTS: Organize the notes with respect to the following criteria: - `Novelty` - `Rigor` - `Relevance (of the contribution)` - `Verifiability and Transparency` - `Presentation` And then add a Detailed Comments section to report the notes that contain issues or typos. Can you also formulate three explicit questions by considering the comments above?