15 lines
1.4 KiB
Markdown
15 lines
1.4 KiB
Markdown
-
|
|
- The Bilingual Evaluation Understudy Score, or BLEU for short, is a metric for evaluating a generated sentence to a reference sentence.
|
|
- A perfect match results in a score of 1.0, whereas a perfect mismatch results in a score of 0.0.
|
|
- The score was developed to evaluate the prediction made by automatic machine translation systems. Even though it is not perfect, it offers 5 compelling benefits:
|
|
- quick and expensive to calculate
|
|
- easy to understand
|
|
- language independent
|
|
- it correlates highly with human evaluation
|
|
- it has been widely adopted
|
|
- The BLEU score was proposed by Kishore Papineni, et. al. in their 2002 paper [BLEU: a Method for Automatic Evaluation of Machine Translation](http://www.aclweb.org/anthology/P02-1040.pdf)“.
|
|
- The approach works by counting matching n-grams in the candidate translation to n-grams in the reference text, where 1-gram or unigram would be each token and a bigram comparison would be each word pair. The comparison is made regardless of word order.
|
|
- ``The primary programming task for a BLEU implementor is to compare n-grams of the candidate with the n-grams of the reference translation and count the number of matches. These matches are position-independent. The more the matches, the better the candidate translation is.``
|
|
- ((63063d27-0931-46bb-89ba-6436d127cdbf))
|
|
- ((63063df8-a6dc-4bee-96b2-2512719da00b)) #card
|
|
id:: 65c8d455-5ace-4f50-a571-e08b0196a72d |