302 lines
13 KiB
Markdown
302 lines
13 KiB
Markdown
-
|
||
file:: [TOSEM-2023-0169_Proof_hi_1692256438403_0.pdf](../assets/TOSEM-2023-0169_Proof_hi_1692256438403_0.pdf)
|
||
file-path:: ../assets/TOSEM-2023-0169_Proof_hi_1692256438403_0.pdf
|
||
|
||
- ur study assesses the current use and evolution of these templates in large-scale projects, and their impact on issue tracking metrics, including resolution time, number of reopens, and comments.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: purple
|
||
id:: 64de89fe-6607-46f8-95bf-921f86f8f931
|
||
- Projects with a template exhibited markedly reduced resolution time (374.56 days to 103.46 days) and reduced comment count (4.90 to 4.34) compared to those without.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 64e46bbc-a9a4-483c-9a4c-0afa49507f27
|
||
- hus, our research underscores issue templates’ positive impact in large-scale projects, offering recommendations for improved effectiveness
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: yellow
|
||
id:: 64e46bfb-ea9a-4e13-9690-dfd2350889b3
|
||
- Use of issue forms further significantly decreased resolution time, the number of reopenings, and discussion extent.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: purple
|
||
id:: 64e46c1d-0e12-4b8e-ad42-6b214b750044
|
||
- In GitHub, issue tracking is done with GitHub Issues, a built-in feature of GitHub.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 64e46c47-9b10-43ec-b388-24f3922c3966
|
||
- Earlier and more traditional issue tracking systems, such as Bugzilla and Jira, can be categorized as structured issue tracking systems.
|
||
ls-type:: annotation
|
||
hl-page:: 2
|
||
hl-color:: green
|
||
id:: 64e46d05-aa01-45b4-9cf0-5b1fd4c52358
|
||
- Although the unstructured notion of GitHub Issue provides a flexible and faster way to create issues, in the long run, it hinders maintaining the issue tracking system
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46dce-0d3c-4614-98f3-2cef8c811ae7
|
||
- The lack of a strict structure in issue tracking may increase the number of incomplete issue reports
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46de3-50cd-4540-85f2-a152b0e6052e
|
||
- For example, a bug type of issue should include multiple types of crucial information, such as Steps to Reproduce or Environment information.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46df6-f44f-4355-a584-5e99d63686f5
|
||
- “Issues are often filed missing crucial information like reproduction steps or tested version.”
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 64e46e15-13d9-4d43-b227-3c6a1ec8cdda
|
||
- maintainers proposed adding a template for bug reports. “We’d like issues to gain custom fields, along with a mechanism (such as a mandatory issue template, perhaps powered by a newissue.md in root as a likely-simple solution) for ensuring they are filled out in every issue.”
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46e34-dc86-4809-822e-2be6f0d5351c
|
||
- The main aim of this study is to understand the impact of GitHub issue templates and gain insights into their usage.
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: purple
|
||
id:: 64e46e45-6c3d-426d-9529-0972789280eb
|
||
- RQ1 What is the current status of issue templates on large-scale projects?
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46e75-f0cf-4d3e-86b2-0628d56aeffd
|
||
- RQ2 How does issue template usage evolve over time?
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46e7b-41f9-4bd6-8c98-bbaf713e1fb7
|
||
- RQ3 Do contributors conform to the templates?
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46e85-7b9d-406a-8222-cca9cf1254f0
|
||
- RQ4 How do issue templates affect issue tracking metrics (time to resolution, number of reopening events, number of comments)?
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46f06-7778-406c-810d-c4dd95fae92f
|
||
- RQ5 How is the perception of issue templates among project maintainers?
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46f0c-d60a-4e3b-ab26-40a0a5b4fa5b
|
||
- Section 5, we present our results
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46f6a-f8f3-44cc-9a51-b58f093fdda8
|
||
- discuss them in Section
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: green
|
||
id:: 64e46f77-991b-4306-b286-a3b1b4421594
|
||
- An issue template is a pre-defined set of fields with descriptions that can be used to create an issue
|
||
ls-type:: annotation
|
||
hl-page:: 3
|
||
hl-color:: blue
|
||
id:: 64e46fae-2bd1-44ab-bec5-46862321f495
|
||
- Chen et al. [ 3 ] discussed that issue titles and descriptions should be high quality. They developed a method to generate titles from the issue descriptions automatically
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 64e4711d-e348-437c-ba23-6ef01ac90078
|
||
- Template Usage in Software Developmen
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 64e47186-5fe6-4506-ac45-881f15453101
|
||
- ull request templates positively impacts the maintainability of an open-source project, such as less time for code review, fewer duplicated pull requests, and invalid comments
|
||
ls-type:: annotation
|
||
hl-page:: 6
|
||
hl-color:: green
|
||
id:: 64e471b2-9bbf-49ce-8544-94310a030ceb
|
||
- Similar to our findings, contributors and maintainers rate the usefulness of templates positively but also report challenges in their use and suggest potential improvements for better user interaction and automation
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: blue
|
||
id:: 64e471e9-f332-444d-8098-a83bba46ae7d
|
||
hl-stamp:: 1692693025749
|
||
- However, their study does not differentiate between different template types or analyze the historical change. In this study, we focus on the issue templates only, and we try to understand the historical evolution and the effect of different template types
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: purple
|
||
id:: 64e47214-11bc-4687-b771-f7a82a9548e3
|
||
hl-stamp:: 1692693041042
|
||
- Their analysis concluded that the difference is negligible between the active and inactive projects in terms of the use of issue templates.
|
||
ls-type:: annotation
|
||
hl-page:: 7
|
||
hl-color:: green
|
||
id:: 64e472ae-6b19-41b1-9ce1-f3b2596bf22e
|
||
- 6] Ozren Dabic, Emad Aghajani, and Gabriele Bavota. 2021. Sampling Projects in GitHub for MSR Studies. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). IEEE, 560–564. https://doi.org/10.1109/MSR52588.2021.00074
|
||
ls-type:: annotation
|
||
hl-page:: 23
|
||
hl-color:: green
|
||
id:: 64e47313-bf26-4a91-9a12-57fe5ca01260
|
||
- We cloned the remaining 100 repositories to get the content and the change history of issue templates
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: green
|
||
id:: 64e47391-c8af-44bb-9a3c-5ea53cb9c5c7
|
||
- Santiago Dueñas, Valerio Cosentino, Jesus M. Gonzalez-Barahona, Alvaro del Castillo San Felix, Daniel Izquierdo-Cortazar, Luis Cañas-Díaz, and Alberto Pérez García-Plaza. 2021. GrimoireLab: A toolset for software development analytics. PeerJ Computer Science 7 (7 2021),
|
||
ls-type:: annotation
|
||
hl-page:: 23
|
||
hl-color:: green
|
||
id:: 64e473c8-b750-448b-8f29-0ca477f9bd15
|
||
- One of them parses the content of the templates written in Markdown language, while the other does the same for the YAML language
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: green
|
||
id:: 64e4779c-c458-45bf-98cb-457082277b16
|
||
- categorize templates by their use cases
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: blue
|
||
id:: 64e477db-c7bb-4e02-ac50-a44e0cedca2c
|
||
- identify the components
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: blue
|
||
id:: 64e477ea-6d18-4952-94d8-1dcd883df18c
|
||
- analyze the content of the template chooser men
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: blue
|
||
id:: 64e47805-c82a-4d59-9abd-477b8cd18866
|
||
- classify its use cases
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: blue
|
||
id:: 64e47815-84fb-49c1-ab5b-b7577a96f781
|
||
- four perspectives
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: green
|
||
id:: 64e47adb-72df-4339-8683-d2b8984c7d9e
|
||
- 4.2.1 Template Category.
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: blue
|
||
id:: 64e47ebd-c290-4860-a423-652efcf4296d
|
||
- 4.2.2 Template Content and Components.
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: blue
|
||
id:: 64e47eef-017e-4a66-a174-ffb53f8b0a34
|
||
- To do this, we parsed the template content and extracted the headings. Then, we classified them based on keywords.
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: green
|
||
id:: 64e47f09-153a-419a-886a-baad5d577fc9
|
||
- 350 templates are independently and manually labeled by two authors of the stud
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: yellow
|
||
id:: 64e47f49-fb2c-4189-8b76-35b2c30c9d80
|
||
- we came out with four categories: Bug, Feature Request, Task, and Question
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: green
|
||
id:: 64e47f7e-65bf-4517-8365-4a92832b2cd4
|
||
- Template Chooser Menu.
|
||
ls-type:: annotation
|
||
hl-page:: 9
|
||
hl-color:: green
|
||
id:: 64e49b4f-236c-4c70-adbc-81f799322d60
|
||
- We measure a conformance score for each issue, and a score can be calculated only if the project had issue templates when the issue was opened
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: green
|
||
id:: 64e49c1d-027e-498d-b2a2-df759422b979
|
||
- omponent.title in issueComponent.title or issueComponent.title in component.title) and issueComponent has a valid body
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: yellow
|
||
id:: 64e49c59-ee48-496b-adbb-593e10a3febd
|
||
- It compares the expected title headings in a template with the actual titles in an issue
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: yellow
|
||
id:: 64e49c92-a10e-498a-ac93-787e3afc37bd
|
||
- A component is correctly filled if its body is not empty and is not equal to the following strings: No response, N/A
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: yellow
|
||
id:: 64e49cbc-f2a1-41b9-a153-6a30cc4dbf1d
|
||
- creation and the resolution.
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: green
|
||
id:: 64e49d2d-d99c-4f1f-a89d-e5a6150155c5
|
||
- Measured in minutes
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: yellow
|
||
id:: 64e49d3a-b50d-450c-ae0f-004af4143eaf
|
||
- We hypothesize that issues created from a template are resolved faster, are reopened fewer times, and have more occasional comments than issues created without a template
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: purple
|
||
id:: 64e49d6a-b4a2-4ed9-8fe9-d595f8ff920d
|
||
- RQ5: Perception of Project Maintainers
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: green
|
||
id:: 64e49e72-4619-423e-a23f-4e18904e1fef
|
||
- 97 projects directly use the issue templates, 1 project8 uses organization-level templates, 1 project9 redirects users to an external issue creation form, and when this form is filled in, the user is redirected back to GitHub with a draft of the issue
|
||
ls-type:: annotation
|
||
hl-page:: 11
|
||
hl-color:: yellow
|
||
id:: 64e49f5c-ec04-4c11-a287-a52cc91fefdd
|
||
hl-stamp:: 1692705397510
|
||
- As Figure 7 and 8 suggest, more projects increasingly prefer issue forms that are more strict but still configurable.
|
||
ls-type:: annotation
|
||
hl-page:: 14
|
||
hl-color:: green
|
||
id:: 64e4a019-6403-4ee0-ad1f-5f2c03e98e71
|
||
- Headings start with # character for the templates written in Markdown language. In issue forms, which are written in YAML language, the structure is more strict, and the heading of each component is located in the label section of template attributes
|
||
ls-type:: annotation
|
||
hl-page:: 8
|
||
hl-color:: yellow
|
||
id:: 64e4a066-0252-487a-b9ff-fbc0db0ea3a7
|
||
- 1,661,788 issues. 209,654 issues have zero conformance to templates, while 394,285 issues have non-zero conformance.
|
||
ls-type:: annotation
|
||
hl-page:: 15
|
||
hl-color:: yellow
|
||
id:: 64e4a418-2c50-4f2c-a39a-79ea7819abeb
|
||
- The conformance score is basically the ratio of correctly filled template components over the total number of components available for a template categor
|
||
ls-type:: annotation
|
||
hl-page:: 10
|
||
hl-color:: yellow
|
||
id:: 64e4a535-08f0-4702-98db-b7294a90dfe4
|
||
- Issues created when the project has an issue template have less time to resolution and fewer comments. The difference is statistically significant. However, the effect of the conformance on three metrics is insignificant. Also, when YAML-based templates are used, the time to resolution is significantly lower, and the number of comments and reopening events is less
|
||
ls-type:: annotation
|
||
hl-page:: 16
|
||
hl-color:: green
|
||
id:: 64e4a5cd-a3d6-4a45-bd19-f06aa3736925
|
||
- issue templates are widely used
|
||
ls-type:: annotation
|
||
hl-page:: 19
|
||
hl-color:: blue
|
||
id:: 64e4a6c0-7254-49c4-9b0b-636fa2998f84
|
||
- Extracting the template components from Markdown files: We developed a text parser that extracts the template components by checking the headings. However, not all templates use headings to define titles, we observed some templates prefer writing titles in different formats, such as writing headings with bold or using “·”, “◦” at the beginning of headings, but our parsing algorithm cannot handle such situation
|
||
ls-type:: annotation
|
||
hl-page:: 21
|
||
hl-color:: yellow
|
||
id:: 64e4a768-c90e-4675-8465-93a2e36b92e8
|
||
- To understand the overview of issue templates, we sampled 100 projects from GitHub.
|
||
ls-type:: annotation
|
||
hl-page:: 22
|
||
hl-color:: yellow
|
||
id:: 64e4a7a6-d866-4e41-a257-84c993087e1f
|
||
- standardizing their issue reporting process
|
||
ls-type:: annotation
|
||
hl-page:: 22
|
||
hl-color:: green
|
||
id:: 64e4acc0-9a4f-46e3-97e1-6a19ae50fe5b |