logseq/pages/hls__TOSEM-2023-0169_Proof_hi_1692256438403_0.md

file:: TOSEM-2023-0169_Proof_hi_1692256438403_0.pdf file-path:: ../assets/TOSEM-2023-0169_Proof_hi_1692256438403_0.pdf

• ur study assesses the current use and evolution of these templates in large-scale projects, and their impact on issue tracking metrics, including resolution time, number of reopens, and comments. ls-type:: annotation hl-page:: 2 hl-color:: purple id:: 64de89fe-6607-46f8-95bf-921f86f8f931
• Projects with a template exhibited markedly reduced resolution time (374.56 days to 103.46 days) and reduced comment count (4.90 to 4.34) compared to those without. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64e46bbc-a9a4-483c-9a4c-0afa49507f27
• hus, our research underscores issue templates’ positive impact in large-scale projects, offering recommendations for improved effectiveness ls-type:: annotation hl-page:: 2 hl-color:: yellow id:: 64e46bfb-ea9a-4e13-9690-dfd2350889b3
• Use of issue forms further significantly decreased resolution time, the number of reopenings, and discussion extent. ls-type:: annotation hl-page:: 2 hl-color:: purple id:: 64e46c1d-0e12-4b8e-ad42-6b214b750044
• In GitHub, issue tracking is done with GitHub Issues, a built-in feature of GitHub. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64e46c47-9b10-43ec-b388-24f3922c3966
• Earlier and more traditional issue tracking systems, such as Bugzilla and Jira, can be categorized as structured issue tracking systems. ls-type:: annotation hl-page:: 2 hl-color:: green id:: 64e46d05-aa01-45b4-9cf0-5b1fd4c52358
• Although the unstructured notion of GitHub Issue provides a flexible and faster way to create issues, in the long run, it hinders maintaining the issue tracking system ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46dce-0d3c-4614-98f3-2cef8c811ae7
• The lack of a strict structure in issue tracking may increase the number of incomplete issue reports ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46de3-50cd-4540-85f2-a152b0e6052e
• For example, a bug type of issue should include multiple types of crucial information, such as Steps to Reproduce or Environment information. ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46df6-f44f-4355-a584-5e99d63686f5
• “Issues are often filed missing crucial information like reproduction steps or tested version.” ls-type:: annotation hl-page:: 3 hl-color:: purple id:: 64e46e15-13d9-4d43-b227-3c6a1ec8cdda
• maintainers proposed adding a template for bug reports. “We’d like issues to gain custom fields, along with a mechanism (such as a mandatory issue template, perhaps powered by a newissue.md in root as a likely-simple solution) for ensuring they are filled out in every issue.” ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46e34-dc86-4809-822e-2be6f0d5351c
• The main aim of this study is to understand the impact of GitHub issue templates and gain insights into their usage. ls-type:: annotation hl-page:: 3 hl-color:: purple id:: 64e46e45-6c3d-426d-9529-0972789280eb
• RQ1 What is the current status of issue templates on large-scale projects? ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46e75-f0cf-4d3e-86b2-0628d56aeffd
• RQ2 How does issue template usage evolve over time? ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46e7b-41f9-4bd6-8c98-bbaf713e1fb7
• RQ3 Do contributors conform to the templates? ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46e85-7b9d-406a-8222-cca9cf1254f0
• RQ4 How do issue templates affect issue tracking metrics (time to resolution, number of reopening events, number of comments)? ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46f06-7778-406c-810d-c4dd95fae92f
• RQ5 How is the perception of issue templates among project maintainers? ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46f0c-d60a-4e3b-ab26-40a0a5b4fa5b
• Section 5, we present our results ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46f6a-f8f3-44cc-9a51-b58f093fdda8
• discuss them in Section ls-type:: annotation hl-page:: 3 hl-color:: green id:: 64e46f77-991b-4306-b286-a3b1b4421594
• An issue template is a pre-defined set of fields with descriptions that can be used to create an issue ls-type:: annotation hl-page:: 3 hl-color:: blue id:: 64e46fae-2bd1-44ab-bec5-46862321f495
• Chen et al. [ 3 ] discussed that issue titles and descriptions should be high quality. They developed a method to generate titles from the issue descriptions automatically ls-type:: annotation hl-page:: 6 hl-color:: green id:: 64e4711d-e348-437c-ba23-6ef01ac90078
• Template Usage in Software Developmen ls-type:: annotation hl-page:: 6 hl-color:: green id:: 64e47186-5fe6-4506-ac45-881f15453101
• ull request templates positively impacts the maintainability of an open-source project, such as less time for code review, fewer duplicated pull requests, and invalid comments ls-type:: annotation hl-page:: 6 hl-color:: green id:: 64e471b2-9bbf-49ce-8544-94310a030ceb
• Similar to our findings, contributors and maintainers rate the usefulness of templates positively but also report challenges in their use and suggest potential improvements for better user interaction and automation ls-type:: annotation hl-page:: 7 hl-color:: blue id:: 64e471e9-f332-444d-8098-a83bba46ae7d hl-stamp:: 1692693025749
• However, their study does not differentiate between different template types or analyze the historical change. In this study, we focus on the issue templates only, and we try to understand the historical evolution and the effect of different template types ls-type:: annotation hl-page:: 7 hl-color:: purple id:: 64e47214-11bc-4687-b771-f7a82a9548e3 hl-stamp:: 1692693041042
• Their analysis concluded that the difference is negligible between the active and inactive projects in terms of the use of issue templates. ls-type:: annotation hl-page:: 7 hl-color:: green id:: 64e472ae-6b19-41b1-9ce1-f3b2596bf22e
• 6] Ozren Dabic, Emad Aghajani, and Gabriele Bavota. 2021. Sampling Projects in GitHub for MSR Studies. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). IEEE, 560–564. https://doi.org/10.1109/MSR52588.2021.00074 ls-type:: annotation hl-page:: 23 hl-color:: green id:: 64e47313-bf26-4a91-9a12-57fe5ca01260
• We cloned the remaining 100 repositories to get the content and the change history of issue templates ls-type:: annotation hl-page:: 8 hl-color:: green id:: 64e47391-c8af-44bb-9a3c-5ea53cb9c5c7
• Santiago Dueñas, Valerio Cosentino, Jesus M. Gonzalez-Barahona, Alvaro del Castillo San Felix, Daniel Izquierdo-Cortazar, Luis Cañas-Díaz, and Alberto Pérez García-Plaza. 2021. GrimoireLab: A toolset for software development analytics. PeerJ Computer Science 7 (7 2021), ls-type:: annotation hl-page:: 23 hl-color:: green id:: 64e473c8-b750-448b-8f29-0ca477f9bd15
• One of them parses the content of the templates written in Markdown language, while the other does the same for the YAML language ls-type:: annotation hl-page:: 8 hl-color:: green id:: 64e4779c-c458-45bf-98cb-457082277b16
• categorize templates by their use cases ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 64e477db-c7bb-4e02-ac50-a44e0cedca2c
• identify the components ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 64e477ea-6d18-4952-94d8-1dcd883df18c
• analyze the content of the template chooser men ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 64e47805-c82a-4d59-9abd-477b8cd18866
• classify its use cases ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 64e47815-84fb-49c1-ab5b-b7577a96f781
• four perspectives ls-type:: annotation hl-page:: 8 hl-color:: green id:: 64e47adb-72df-4339-8683-d2b8984c7d9e
• 4.2.1 Template Category. ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 64e47ebd-c290-4860-a423-652efcf4296d
• 4.2.2 Template Content and Components. ls-type:: annotation hl-page:: 8 hl-color:: blue id:: 64e47eef-017e-4a66-a174-ffb53f8b0a34
• To do this, we parsed the template content and extracted the headings. Then, we classified them based on keywords. ls-type:: annotation hl-page:: 8 hl-color:: green id:: 64e47f09-153a-419a-886a-baad5d577fc9
• 350 templates are independently and manually labeled by two authors of the stud ls-type:: annotation hl-page:: 8 hl-color:: yellow id:: 64e47f49-fb2c-4189-8b76-35b2c30c9d80
• we came out with four categories: Bug, Feature Request, Task, and Question ls-type:: annotation hl-page:: 8 hl-color:: green id:: 64e47f7e-65bf-4517-8365-4a92832b2cd4
• Template Chooser Menu. ls-type:: annotation hl-page:: 9 hl-color:: green id:: 64e49b4f-236c-4c70-adbc-81f799322d60
• We measure a conformance score for each issue, and a score can be calculated only if the project had issue templates when the issue was opened ls-type:: annotation hl-page:: 10 hl-color:: green id:: 64e49c1d-027e-498d-b2a2-df759422b979
• omponent.title in issueComponent.title or issueComponent.title in component.title) and issueComponent has a valid body ls-type:: annotation hl-page:: 10 hl-color:: yellow id:: 64e49c59-ee48-496b-adbb-593e10a3febd
• It compares the expected title headings in a template with the actual titles in an issue ls-type:: annotation hl-page:: 10 hl-color:: yellow id:: 64e49c92-a10e-498a-ac93-787e3afc37bd
• A component is correctly filled if its body is not empty and is not equal to the following strings: No response, N/A ls-type:: annotation hl-page:: 10 hl-color:: yellow id:: 64e49cbc-f2a1-41b9-a153-6a30cc4dbf1d
• creation and the resolution. ls-type:: annotation hl-page:: 10 hl-color:: green id:: 64e49d2d-d99c-4f1f-a89d-e5a6150155c5
• Measured in minutes ls-type:: annotation hl-page:: 10 hl-color:: yellow id:: 64e49d3a-b50d-450c-ae0f-004af4143eaf
• We hypothesize that issues created from a template are resolved faster, are reopened fewer times, and have more occasional comments than issues created without a template ls-type:: annotation hl-page:: 11 hl-color:: purple id:: 64e49d6a-b4a2-4ed9-8fe9-d595f8ff920d
• RQ5: Perception of Project Maintainers ls-type:: annotation hl-page:: 11 hl-color:: green id:: 64e49e72-4619-423e-a23f-4e18904e1fef
• 97 projects directly use the issue templates, 1 project8 uses organization-level templates, 1 project9 redirects users to an external issue creation form, and when this form is filled in, the user is redirected back to GitHub with a draft of the issue ls-type:: annotation hl-page:: 11 hl-color:: yellow id:: 64e49f5c-ec04-4c11-a287-a52cc91fefdd hl-stamp:: 1692705397510
• As Figure 7 and 8 suggest, more projects increasingly prefer issue forms that are more strict but still configurable. ls-type:: annotation hl-page:: 14 hl-color:: green id:: 64e4a019-6403-4ee0-ad1f-5f2c03e98e71
• Headings start with # character for the templates written in Markdown language. In issue forms, which are written in YAML language, the structure is more strict, and the heading of each component is located in the label section of template attributes ls-type:: annotation hl-page:: 8 hl-color:: yellow id:: 64e4a066-0252-487a-b9ff-fbc0db0ea3a7
• 1,661,788 issues. 209,654 issues have zero conformance to templates, while 394,285 issues have non-zero conformance. ls-type:: annotation hl-page:: 15 hl-color:: yellow id:: 64e4a418-2c50-4f2c-a39a-79ea7819abeb
• The conformance score is basically the ratio of correctly filled template components over the total number of components available for a template categor ls-type:: annotation hl-page:: 10 hl-color:: yellow id:: 64e4a535-08f0-4702-98db-b7294a90dfe4
• Issues created when the project has an issue template have less time to resolution and fewer comments. The difference is statistically significant. However, the effect of the conformance on three metrics is insignificant. Also, when YAML-based templates are used, the time to resolution is significantly lower, and the number of comments and reopening events is less ls-type:: annotation hl-page:: 16 hl-color:: green id:: 64e4a5cd-a3d6-4a45-bd19-f06aa3736925
• issue templates are widely used ls-type:: annotation hl-page:: 19 hl-color:: blue id:: 64e4a6c0-7254-49c4-9b0b-636fa2998f84
• Extracting the template components from Markdown files: We developed a text parser that extracts the template components by checking the headings. However, not all templates use headings to define titles, we observed some templates prefer writing titles in different formats, such as writing headings with bold or using “·”, “◦” at the beginning of headings, but our parsing algorithm cannot handle such situation ls-type:: annotation hl-page:: 21 hl-color:: yellow id:: 64e4a768-c90e-4675-8465-93a2e36b92e8
• To understand the overview of issue templates, we sampled 100 projects from GitHub. ls-type:: annotation hl-page:: 22 hl-color:: yellow id:: 64e4a7a6-d866-4e41-a257-84c993087e1f
• standardizing their issue reporting process ls-type:: annotation hl-page:: 22 hl-color:: green id:: 64e4acc0-9a4f-46e3-97e1-6a19ae50fe5b