logseq/pages/hls__ase2022-paper108_1661352547766_0.md

file-path:: ../assets/ase2022-paper108_1661352547766_0.pdf file:: ase2022-paper108_1661352547766_0.pdf title:: hls__ase2022-paper108_1661352547766_0

• n-gram-base ls-type:: annotation hl-page:: 1 id:: 63063a7c-1f1f-4833-9a37-20f185180632
• allows for precisely and efficiently measuring the similarity of code ls-type:: annotation hl-page:: 1 id:: 63063aa0-134b-43ac-a304-0de166e31190
• language-agnosti ls-type:: annotation hl-page:: 1 id:: 63063aae-e6f7-492b-ac1b-7c685e8771eb
• handle incomplete or partially incorrect code ls-type:: annotation hl-page:: 1 id:: 63063ab3-3683-42b4-ab50-23dd74f3704e
• efficient ls-type:: annotation hl-page:: 1 id:: 63063ab5-108f-4eaf-acd0-1b7334e6016e
• automated program repai ls-type:: annotation hl-page:: 1 id:: 63063cad-79f4-45da-8134-64ac39b638da
• predicting code from natural language descriptions of the desired functionality ls-type:: annotation hl-page:: 1 id:: 63063cb3-dc58-49e2-b55c-1fa1234f333c
• bilingual evaluation understudy ls-type:: annotation hl-page:: 1 id:: 63063cd3-3c09-45f8-bf15-444de176fc9c
• e find at least 21 papers published since 2015 that use BLEU as a metric to evaluate code prediction ls-type:: annotation hl-page:: 1 id:: 63063d0f-c404-455d-91eb-9f7481e1db50
• The basic idea of BLEU is to compare a prediction against one or more reference solutions by computing the overlap between n-grams, i.e., contiguous sequences of code tokens ls-type:: annotation hl-page:: 1 id:: 63063d27-0931-46bb-89ba-6436d127cdbf
• trivially shared n-grams, i.e., n-grams that occur across code written in the same language without implying any deeper relationship or semantic similarity. ls-type:: annotation hl-page:: 1 id:: 63063df8-a6dc-4bee-96b2-2512719da00b
• This paper presents CrystalBLEU, a metric to precisely and efficiently evaluate code similarity despite trivially shared n-grams. ls-type:: annotation hl-page:: 1 id:: 63063e31-2f9f-4a2b-9308-d98c5c9b71d3
• The approach is an extension of BLEU that removes trivially shared n-grams before computing the n-gram overlap between two pieces of code ls-type:: annotation hl-page:: 1 id:: 63063e49-ec06-4fb1-8a5f-567100bf353e
• To identify trivially shared n-grams, CrystalBLEU analyzes a corpus of code and identifies n-grams that occur frequently across many examples ls-type:: annotation hl-page:: 1 id:: 630642a3-0ec8-4822-8ae9-1c7c7f9ab432
• distinguishability ls-type:: annotation hl-page:: 1 id:: 630642b3-af01-4920-8ea7-a8e4fe046264
• distinguishability measures how much more similar code examples known to be semantically equivalent are compared to code examples that are not equivalent to each other. ls-type:: annotation hl-page:: 2 id:: 630642cc-fe7c-48e7-8633-ef1149f8f512
• Concode ls-type:: annotation hl-page:: 2 id:: 630642eb-1e39-4764-83c2-f36c4d42c4a1
• BigCloneBench ls-type:: annotation hl-page:: 2 id:: 630642ee-9b86-4ec4-866e-d0143aa8d660
• it is more effective at distinguishing semantically similar code from code merely written in the same language. ls-type:: annotation hl-page:: 2 id:: 63064327-6edb-4a1d-9e22-12f553aa941f
• most common n-grams in programming languages appear relatively more often than the most common n-grams in natural languages, and that they appear in many different programs regardless of their semantic similarity. ls-type:: annotation hl-page:: 2 id:: 63064c06-55b1-45a7-bda0-b7c861303d12
• distinguishability ls-type:: annotation hl-page:: 2 id:: 63064c0a-1933-4f6f-ae9b-04d65cafd1dd
• CrystalBLEU, which ignores trivially shared n-grams when comparing two pieces of code ls-type:: annotation hl-page:: 2 id:: 63064c19-dc64-4ece-b112-ddfbe89fc638
• modified precision ls-type:: annotation hl-page:: 2 id:: 63064e2b-f532-437f-a1a6-6466f735d592
• odified precision ls-type:: annotation hl-page:: 3 id:: 6309dae9-1d7b-43c2-8f7b-2a5312d1b9c1
• trivially shared n-grams, ls-type:: annotation hl-page:: 4 id:: 6309dafd-1db8-44d6-9e31-aef476b63017
• 325 ls-type:: annotation hl-page:: 5 id:: 6309dcb8-0ad1-4050-bc82-47e116305357
• We now present the CrystalBLEU metric, which in contrast to BLEU, increases distinguishability by accounting for trivially shared ngrams ls-type:: annotation hl-page:: 5 id:: 6309e00a-1ea8-4ea1-aa59-94490799d9ec
• we identify trivially shared n-grams. ls-type:: annotation hl-page:: 5 id:: 6309e00f-9a1a-46dd-954c-0a7eb0e906bf
• BLEU score that accounts for the trivially shared n-grams ls-type:: annotation hl-page:: 5 id:: 6309e012-ff0e-45bb-acee-870461906a76
• How well does CrystalBLEU distinguish between similar and dissimilar code? ls-type:: annotation hl-page:: 6 id:: 6309e040-49ed-4a07-ab50-74b7174ff025
• avoid misleading results provided by BLEU? ls-type:: annotation hl-page:: 6 id:: 6309e043-4d6e-41d4-abb2-6ff76202290b
• ShareCode: Semantically equivalent, human-written code. ls-type:: annotation hl-page:: 6 id:: 6309e049-b50a-4077-8946-7ef78d2d86f7
• ShareCode ls-type:: annotation hl-page:: 7 id:: 6309e069-4d1b-4350-b0fe-648bcd92100d
• BigCloneBench ls-type:: annotation hl-page:: 7 id:: 6309e07b-2a59-4dd1-930a-779695afda54
• Concode: Code generation task. ls-type:: annotation hl-page:: 7 id:: 6309e085-79eb-4784-8423-63ad6e96d128
• CrystalBLEU: Precisely and Efficiently Measuring the Similarity of Code ls-type:: annotation hl-page:: 1 id:: 6309e970-d42e-4be1-9735-9c239019a46a