What a real eval suite looks like, and how to ask for it

A founder says the team tests constantly. The direct answer is simple: a real eval suite is a set of repeatable product tests, built from real customer-like work, with a pass line that can block release. If it cannot stop a deploy, it is probably evidence theater.

This matters because evals are often the least-checked part of AI diligence. An empirical study of open-source AI agent testing analyzed 39 agent frameworks and 439 agentic applications. It found that agent-specific testing tools such as DeepEval appeared in around 1% of tests. That does not mean every startup is careless. It means testing claims deserve inspection.

If you are using a broader technical due diligence checklist for AI startups, make evals one of the first artifacts you ask for. They tell you whether the team has a real measuring stick for the product’s behavior.

What an eval actually is

An eval is a driving test on the startup’s own streets. A benchmark is the manufacturer’s test track. The track matters, but it does not tell you whether the car handles the pothole outside your office or the confusing intersection where everyone hesitates.

For an AI product, an eval is a repeatable check against examples the product should handle. The example might be a customer support email, a messy invoice, a legal clause, or a voice transcript. The eval feeds that input into the product and checks the output against an expected result or a scoring rule.

The important word is repeatable. A founder can run a good demo once. A real eval can run again tomorrow after the team changes the prompt, changes the model, edits the retrieval step, or ships a new feature. If the product used to pass and now fails, the suite catches the slip.

For a non-technical investor, you do not need to judge the code first. Ask for the folder, the data file, and the last run. Good teams can show the machine that grades the product.

Benchmark, demo, and eval are different things

A benchmark compares general model ability, a demo proves a workflow can look good once, and an eval checks whether this product still works on its own job. Mixing these up is how weak AI products borrow credibility from strong underlying models.

A benchmark is usually public. It might test math, coding, reasoning, or tool use. Benchmarks help teams choose a model, but they rarely match the buyer’s workflow. A claims product can use a model with great public scores and still mishandle the insurer’s odd document format.

A demo is a performance. Demos are useful because they show what the product is trying to become. But a demo often uses a known input, a friendly path, and a founder who knows where not to click.

An eval is closer to the inspection pit at a garage. The car goes over the same pit each time, and the mechanic checks specific parts. In AI, the suite might check whether the answer cites the right source, extracts the correct dollar amount, routes the task to human review, or avoids inventing a policy.

So when a founder says the model scored well on a benchmark, ask what product eval that benchmark maps to. When they show a demo, ask which eval cases cover the same workflow. If there is no bridge from benchmark or demo to eval, you are looking at a story, not release evidence.

The four parts a real suite has

A real eval suite has four parts: a golden set from real inputs, a pass threshold, regression cases from failures, and automatic release wiring. Missing one does not always kill a seed-stage deal, but it tells you exactly where the testing story is thin.

The golden set is the core sample. It is a saved collection of examples the product must handle. “Golden” does not mean perfect. It means the team treats the expected answer as the reference. Good golden sets come from actual customer inputs, sales calls, support tickets, uploaded files, or hand-built examples that closely mimic them. Bad golden sets are clean prompts someone invented for the demo.

The pass threshold is the release line. It says how many cases must pass before the team can ship. The line may be strict for simple extraction, looser for a messy writing task, or split by risk category. The exact standard depends on the job. The key is that the line exists before the release, not after the founder sees the score.

Regression cases are old bruises kept in the test drawer. When the AI fails in production, the team saves that input, the bad output, and the corrected expectation. From then on, the suite checks that the same failure does not return. This is how an AI product develops memory outside the model itself.

Automatic wiring means the suite runs when the product changes. Engineers often call this CI, short for continuous integration. In plain English, it is the automatic check that runs when code or prompts change. The strongest version blocks a deploy when important evals fail. A weaker but still useful version posts the result where the team cannot miss it.

There are tools that can help you locate these artifacts. eval-audit is a free CLI that statically reports eval coverage for a repo with no API keys.

What good answers sound like

Good answers mention artifacts, failures, and release rules. They should make you feel that the team could hand the suite to another engineer and get the same result.

Ask, “Can you show me the eval suite for the main workflow?” A strong answer sounds like: “Yes. The golden set is in this directory. It comes from anonymized onboarding files and support tickets. The high-risk category has a stricter pass line, and this GitHub check blocks release if the main suite fails.”

Ask, “What failed recently?” A strong answer sounds like: “Last week the model started missing cancellation clauses in scanned contracts. The team added regression cases, changed the extraction prompt, and kept the worst image-quality cases in human review.”

Ask, “What happens when the model provider changes?” A strong answer sounds like: “The same suite runs against the new model before switching. The last comparison showed lower cost but worse refusal behavior, so the compliance workflow stayed on the old model.”

Those answers are specific because real testing leaves tracks. File names, run history, thresholds, failure notes, and owner names are normal. You are not asking for perfection. You are asking whether the company can tell the difference between a product that got better and a product that merely looked better in a room.

This is also one of the technical AI startup red flags investors miss: a founder who cannot name a failed eval may not be measuring failure in a repeatable way.

What evasive answers sound like

Evasive answers replace evidence with confidence. They often start with true statements that do not answer the question. The trap is letting a founder talk about effort when you asked for a release control.

“We test constantly” is not enough. It might mean manual clicking before demos. It might mean engineers try prompts in chat. It might mean unit tests cover normal software code, while the AI behavior itself is not checked. Ask where the eval results live.

“Our customers would tell us if it failed” is not enough. Customers are a slow alarm bell. By the time they complain, the bad output may have already damaged trust, created review work, or entered a customer workflow.

“The benchmark scores are strong” is not enough. Benchmarks can support model choice, but they do not prove the startup’s workflow works. The question is whether the product passes its own driving test.

“The suite is too messy to show” is not enough either. Early evals can be messy. That is fine. A folder of rough JSON files, screenshots, and reviewer notes is still better than a clean story with no artifact. Messy evidence can be improved. Invisible evidence cannot be inspected.

The limitation investors should not ignore

An eval suite existing says nothing by itself about whether the cases are any good. It proves that a measuring machine exists. It does not prove the ruler is marked correctly.

The cases may be too easy. They may come from the happy path, not from the ugly customer inputs where the product earns its margin. They may be stale because the product has moved into a new workflow. They may reward fluent answers while missing factual errors.

This is why diligence should inspect both the machinery and the sample. Ask where the cases came from. Ask which cases are hardest. Ask what percentage came from production failures. Ask who reviewed expected answers. Ask whether any customer segment is underrepresented.

For wrapper risk, this matters even more. If the startup depends on one model provider, the suite is the instrument panel for switching, comparing, and surviving provider changes. That is the pressure behind The Wrapper Test: what remains true when the model underneath improves, changes, or moves closer to the customer.

The practical ask is small. Before the next call ends, ask for the main eval folder, the last run, one failed case, the pass threshold, and the deploy rule. If those five artifacts do not exist, write that down before the demo starts glowing again.