---
title: "Evals: how you know your AI actually works"
url: "https://barbairians.com/blog/evals-how-you-know-your-ai-works"
description: "The demo always works. The question is whether it works on your data, at 4 pm on a Friday and after the next model update. A test set answers that; an impression does not."
---

# Evals: how you know your AI actually works

September 24, 2026·3 min read·BarbAIrianS

The demo always works. The vendor picked the examples, the presenter knows what to ask, and the room nods. Then the system meets real email, crooked invoice scans and the customer who writes everything in one sentence with no commas — and nobody knows whether it works, because nobody decided what “works” means.

Evaluation is the boring answer to that question. And the only one that survives a change of model.

## An impression is not a measurement

The most common way companies judge AI goes like this: someone tests it for an hour, types a few questions and says “pretty good”. The trouble is that an hour of human testing is a dozen or so examples, chosen by someone who knows what to expect. The rare cases — the ones that cost money — almost never appear.

And an impression cannot be repeated. When you change the model or the prompt three months later, you do not know whether things got better, worse, or merely different.

## A test set is a company asset

A test set is a collection of real examples with the expected result. Nothing more — and nothing more valuable in the whole AI project.

-   **Real data, not invented.** Pull cases from history: ordinary ones, hard ones and the ones that once went wrong.
-   **The expected result written by someone who knows the process.** Not by a model. If two people on the team disagree on the right answer, you have found ambiguity in the process, not in the AI.
-   **It grows.** Every mistake in production becomes a new test case. After a year the set knows your business better than any documentation.

## How to score the results

Not every task can be judged the same way.

**Closed answers** — a category, an amount, a date, yes or no. A comparison is enough: it matches or it does not. Count the hit rate separately for each category, because an average hides the places where the system falls apart.

**Open answers** — a summary, a draft email, an answer to a question. Here you need criteria: does it contain the key facts, does it invent nothing, is the tone right. A second model with clear instructions can do part of this scoring, but a person always checks a sample — otherwise you are grading a model with a model and do not know which one is wrong.

**Cost and time** — are results too. A model that is marginally better and several times slower can be the worse choice for a process that has to run during a call with a customer.

## Evals as insurance against your vendor

This is where sovereignty comes back in. A hosted model’s vendor can update it without asking. Sometimes it gets better. Sometimes your process, tuned to the old version, starts going wrong in ways nobody notices for a week.

With a test set you notice within the hour. You also have a number to take back to the vendor, a reason to pin a version, or a basis for comparing an alternative — including an open-weight model running on your own hardware. Without the set, all you have is a feeling that “it has been worse lately”.

## The minimum to start

Fifty real cases, an expected result for each, a simple script that runs them through the system and counts the hits. Run it on every model and prompt change. That is a day’s work — and the difference between a company that knows its AI works and one that hopes it does.

-   evaluation
-   quality
-   testing
-   models

## Keep reading

-   [AI factories in Europe: who will own the tokens](https://barbairians.com/blog/ai-factories-in-europe-who-owns-the-tokens)
-   [The model contract: what not to sign with an AI vendor](https://barbairians.com/blog/the-model-contract-what-not-to-sign-with-an-ai-vendor)