BLEU and ROUGE measure surface overlap, not meaning
BLEU scores n-gram precision against one or more references with a brevity penalty; ROUGE scores reference n-gram recall. Both are fast and reproducible, and both are blind to paraphrase: a correct restatement scores as badly as a wrong one. They are diagnostics for surface form, not judges of quality.
BLEU precision
Clip each candidate n-gram count by its reference count, then take the geometric mean of precisions across n.
Brevity penalty
Short candidates are penalized so a model cannot win by emitting only high-precision fragments.
ROUGE recall
Fraction of reference n-grams the candidate covers — favors long, inclusive outputs.
MATHEMATICAL FORMULATION · BLEU
BLEU = BP · exp(Σ w_n · log p_n), BP = min(1, e^(1 − ref/cand))
Precisions [0.8, 0.6, 0.4, 0.2] with equal weights give a geometric mean of about 0.44; a candidate shorter than the reference is then penalized by the brevity factor.
REAL-WORLD PRODUCTION ENGINEERING
BLEU remains standard for machine translation but correlates weakly with human judgment on open-ended generation.
ROUGE is standard for summarization; extraction-based summaries score higher than abstractive ones because they copy n-grams.