Evaluating Generative Models

A fluent response is not evidence that a model improved. Fluency is what these models produce most reliably, so it is present in the good answers and in the bad ones. To find out whether a change helped, we need a question narrow enough for a measurement to answer. The question has to be chosen first, because each measurement answers only one.

A language model can be evaluated by how much probability it assigns to text that it did not train on. That asks whether its continuation distribution fits held-out language. It does not directly ask whether an answer follows instructions, cites evidence correctly, or solves the user’s problem.

For a factual question with a known answer, correctness may be checked against a reference. Exact string matching can be useful for rigid outputs but rejects valid paraphrases. Open-ended responses need a rubric that states what counts as correct, complete, and useful.

The paraphrase problem is easy to meet. Take the previous lesson’s question about when the library opens on Saturday, with a reference answer of ten. A response of ten in the morning is right, and it does not match the reference character for character. Exact matching suits outputs that have one correct form, such as a date in a fixed format or a label from a short list.

The first row is the one this page calculates, and it is the one most easily mistaken for the others. A model can become better at predicting ordinary text without becoming more accurate, more obedient to instructions or more careful with sources. Part 2 measures predictive fit. Part 3 returns to the rest.

A comparison is a claim that one thing changed. If the model changed and the prompt changed with it, the difference in score belongs to both, and there is no way to divide it between them afterward. That is the reason for recording every setting, including the ones that seem incidental.

Use comparable test inputs and record the prompt, sampling settings, model version, and scoring procedure. A model that receives source passages is being tested under different conditions from one answering from its parameters alone.

Keep final test examples separate from training and model selection. Repeatedly tuning against the test set turns it into another selection set. Check for near duplicates and shared source material as well as exact overlap.

The usual arrangement gives each part of the data one job.

A test set loses its value gradually, not all at once. Each time a change is kept because it scored better on the test examples, the choice has been fitted a little to those particular examples. After enough rounds a high test score partly reports how well the choices suit that set, and the set can no longer say how the model will do on examples nobody has looked at.

Overlap does the same damage from the other side. A test question that also appears in the training data, or appears there with a word changed, or is drawn from the same source document, can be answered from memory. It then measures recall of the training data where it was meant to measure ability on new inputs.

Start with what the model hands us. At each position of a held-out text it produced a probability for every token in its vocabulary, and we know which token actually came next. The probability it gave that token is the whole of the raw material. A probability near one is a good prediction, and a probability near zero is a bad one.

For every held-out position, record the probability of the observed target. Take its negative natural logarithm and average over scored positions. Perplexity is the exponential of that mean.

The negative logarithm turns those probabilities into a cost. It is zero for a probability of one and grows without limit as the probability falls toward zero, so a confident miss costs far more than a hesitant one. Averaging gives a cost per token, and the exponential converts that average back to a scale that reads like a number of choices.

Mean negative log likelihood = −(1 / N) Σₜ ln pₜ(yₜ)
Perplexity = exp(Mean negative log likelihood)

For three equally probable vocabulary tokens:
p(target) = 1 / 3
Mean loss = −ln(1 / 3) = ln(3)
Perplexity = exp(ln(3)) = 3

Lower values mean the model assigned more probability to the observed targets on this evaluation. Perplexity can be understood as an effective average uncertainty, not a literal count of choices considered at every position.

Raising the wrong token’s logit reduces the target probability and increases both metrics. A confidently wrong model can therefore score worse than a less certain one.

The live example makes these numbers by hand. It scores three positions over a vocabulary of three tokens, starts every logit at zero, and adds a boost to one logit in each row. With the boost on the target the model is confidently right. With the boost on another token it is confidently wrong. At a boost of 1 on the target, which is where the example starts, the calculation runs as follows.

p(target) = e¹ / (e¹ + 2) ≈ 0.576
Mean loss = −ln(0.576) ≈ 0.551
Perplexity = exp(0.551) ≈ 1.736

Read the perplexity column against the vocabulary of three. A value of 3 is what spreading probability evenly over the three tokens earns. Confidence in the right token brings it down toward 1, which is the floor, reached only by giving the target all of the probability. Confidence in a wrong token sends it past 3, to 4.7, then 9.4, then 56.6.

That last figure is the reason for the warning about counts. There are three tokens, so no model is choosing among 56 of them. When every position has the same target probability, perplexity is one divided by that probability, and 56.6 is one divided by the 0.018 the target was left with. Perplexity reports how surprised the model was by what actually came next. A model can be more surprised than an even guess would have been, and the uniform row at 3 beats every row in which the confidence went to a wrong token.

Perplexity is measured per token under a particular tokenizer and scoring protocol. Comparing values across different tokenizations or differently masked datasets can be misleading because the prediction units changed.

The dependence on units is arithmetic, and the three-token example shows it. Suppose a text is scored as three tokens at a probability of one third each. Suppose a second tokenizer cuts the same text into six pieces, and the text as a whole receives the same total probability. The total cost is unchanged. The cost per token is halved, because there are twice as many tokens to share it.

Total negative log likelihood = 3 × ln(3) ≈ 3.296

Scored as 3 tokens:  exp(3.296 / 3) = 3
Scored as 6 tokens:  exp(3.296 / 6) ≈ 1.732

Nothing about the model’s grasp of the text differs between those two lines. Only the size of the pieces differs, so a comparison of 3 against 1.732 would be a comparison of tokenizers. The same thing happens when two evaluations score different positions, since a mean over the easy positions and a mean over all of them are different means.

A lower perplexity on one corpus does not establish a better assistant or image generator. Report the data and protocol beside the score, and use task-specific tests for the behaviors that matter.

import numpy as np
from oop_ml.numpy.modern import TokenCrossEntropy

result = TokenCrossEntropy().measure(
    np.zeros((3, 3)), np.array([0, 1, 2]),
)
print(result.value)
print(result.perplexity)

The snippet scores the uniform case. It prints a mean loss of about 1.0986, which is ln 3, and a perplexity of 3.

Everything in Part 2 scored text that someone else wrote. The model was asked how probable it found a continuation, and it never produced one. Once a model generates, there is a new thing to judge, which is the output itself, and the quality of an output is not a probability.

For RAG, ask whether the necessary evidence was retrieved, whether the answer is correct, and whether each cited source supports its claim. These checks distinguish a retrieval miss from an answer-generation error.

Take the previous lesson’s question again, when the library opens on Saturday, and suppose the answer comes with a citation. The three checks are separate, and an answer can pass one and fail another. If the Saturday passage was never retrieved, the first check fails and the model had nothing to work from. If it was retrieved and the answer says nine, the second check fails. If the answer says ten and cites the weekday passage, the answer is correct and the third check fails, because the cited source says nine.

Human evaluators need concrete criteria and examples of how to apply them. Model-based judges also need validation: their preferences can depend on wording, answer order, and response length. Inspect disagreements and error categories instead of treating one judge’s score as ground truth.

A judge is a measuring instrument and can be tested like one. Show it the same two answers in both orders. If its preference follows the position and not the answer, then order is part of what it measures. Rewording the instructions, or padding one answer with empty sentences, tests the other two dependencies the same way. Where the judge and a careful human reader disagree, the disagreements show what the judge is really responding to.

For random generation, evaluate multiple samples and report variation when it matters. A cherry-picked response shows that a behavior is possible, not how reliably it occurs.

The remedy is to count. Draw several samples from the same prompt with the same settings, record how many show the behavior, and report that count beside any example. One good sample among many failures and one good sample among many successes look the same when only the sample is shown.

For an image generator, individual realism and coverage of the data distribution are separate concerns. A generator that produces one convincing picture repeatedly has little diversity. Inspect random samples and look for memorization, missing categories, and repeated structures.

The lesson on adversarial networks met this as mode collapse. A generator that has learned to draw one convincing digit passes every inspection of a single picture and fails as a model of handwritten digits. Realism is a property of one picture. Coverage is a property of the whole set of pictures a generator produces, and it can only be judged on a set.

Fréchet Inception Distance, or FID, compares summary statistics of real and generated image features from a particular pretrained network. It is a distribution-level comparison, not a correctness score for one picture. Sample count, preprocessing, reference data, and feature extraction need to be consistent.

Unpacking that helps. A pretrained image network turns each picture into a list of numbers called features. Over many pictures those features have a mean and a spread. The distance compares the mean and spread of the real pictures’ features with the mean and spread of the generated pictures’ features, and a smaller distance says the two sets are distributed more alike. No single picture has a distance, so the measure cannot say which pictures are wrong.

It shares perplexity’s weakness about units. Change the number of samples, the resizing, the reference pictures or the network that extracts the features, and the value changes although the generator has not. Two distances can be compared only when all of those were held the same.

For a conditional generator, also evaluate whether the output matches the requested condition. Neither a low image-distribution distance nor attractive examples alone establishes that the model obeys a text prompt.

The page calculates language-model loss and perplexity only. It does not present FID results or benchmark claims for any trained generator.