AI evaluation LLM capabilities AI agents business strategy

When AI checks its own homework: a surprising look at how models evaluate answers

Corteus Team · Jun 29, 2026 · 7 min read

A new study reveals that large language models often struggle to reliably evaluate answers, performing worse than their generation capabilities in 3 out of 4 benchmarks.

If you’ve used ChatGPT or Claude, you know it types an answer back. You ask it a question, and it generates text. Maybe it drafts an email, summarizes a document, or even writes a bit of code.

But what if you asked that same AI to check an answer? Not just its own, but an answer someone else wrote. Perhaps a colleague’s report, or a customer service response.

Many of us quietly assume that judging something is easier than creating it. Think about it. It’s often simpler to spot a mistake in a spreadsheet than to build the entire financial model from scratch. This seems like common sense, right?

Here’s the thing: for the powerful AI models we use today, like the ones behind your favorite chatbots, that common sense assumption might be totally wrong. And that’s a big deal if you’re planning to use AI for quality control or self-correction.

A new paper, published recently by Sambaran Bandyopadhyay on arXiv 2606.28050v1, shines a light on this. It suggests that large language models (LLMs) – those sophisticated AI brains powering tools like OpenAI’s GPT-4 or Anthropic’s Claude 3 Opus – are often worse at judging answers than they are at generating them. This confused me for years.

Why does this matter for your business?

Imagine you’re building an AI system to automate customer support. The AI drafts a response, and then you want it to “self-correct” by having another part of the AI check its own draft for accuracy before sending. Or perhaps your AI reviews legal documents for specific clauses, and you expect it to reliably evaluate if a clause is present and correct.

If the AI is actually less capable at judging than at creating, your automated quality control might be a lot less reliable than you think. You could be shipping errors, missing critical details, or simply getting a false sense of security. It means that the common practice of using an “LLM-as-a-Judge” (where an AI model acts as an evaluator) might be built on shaky ground.

How do we know AI struggles with judgment?

The researchers set up a controlled experiment. They focused on what’s called “in-context Question Answering” (in-context QA). This means they gave the AI a specific passage of text – say, a paragraph from a company policy document – and then asked it a question that could only be answered from that passage. This removes all the AI’s vast general knowledge, forcing it to act like a student with an open textbook.

Then, they tested two distinct tasks:

Generation: The AI read the passage and wrote an answer to the question.
Evaluation: The AI read the passage, the question, and a candidate answer (either one it generated itself or one provided). Then it decided if that candidate answer was correct based only on the passage.

The results were quite revealing. Across four well-known benchmarks for question-answering – SQuAD 2.0, DROP, HotpotQA, and MuSiQue – the AI’s ability to generate answers often outperformed its ability to judge them. For SQuAD 2.0, DROP, and HotpotQA, the models generated accurate answers more often than they could correctly identify accurate answers.

Accuracy (%) Generation Evaluation Generation Evaluation Generation Evaluation Generation Evaluation The bar chart shows that for SQuAD 2.0, DROP, and HotpotQA, generation accuracy (blue) was higher than evaluation accuracy (purple), with MuSiQue being the exception.

For example, on the SQuAD 2.0 benchmark, the generation model scored 75% accuracy, but the evaluation model only managed 60%. On DROP, it was 80% for generation versus 70% for evaluation. HotpotQA showed a similar trend: 78% for generation, 65% for evaluation. This is a pretty significant gap of 10-15 percentage points.

Interestingly, the complex MuSiQue benchmark, which requires combining information from multiple parts of the text (often called “multi-hop” questions), was the sole exception. In that specific scenario, evaluation was actually a bit better, with 65% accuracy compared to 55% for generation. That’s a key nuance.

Why does this happen? The AI isn’t really “looking”

The researchers dug deeper, using something called “attention analysis.” This is like watching where the AI’s “eyes” are focusing as it processes text. What they found was pretty stark.

When the AI was generating an answer, it paid close attention to the source passage. It read through the text carefully, picking out details. But when it was evaluating an answer, it spent 3 to 5 times less time looking at that same source passage.

Even weirder, it barely looked at the candidate answer it was supposed to be judging. It was like a food critic who only glances at the menu, barely tastes the dish, and then gives a review. Not very thorough.

This suggests that the AI isn’t truly performing a deep, critical review when it’s asked to evaluate. It’s more like it’s making a quick, superficial judgment. And that’s why its accuracy drops.

To confirm this wasn’t just a quirk of how the models were originally trained, the researchers used a technique called LoRA fine-tuning (Low-Rank Adaptation). This allows small, targeted adjustments to an AI model for a specific task. They tried to improve evaluation through fine-tuning, and it actually degraded the model’s generation capabilities. Likewise, fine-tuning for better generation made the model over-accept answers during evaluation. It’s not a simple fix; there’s a fundamental asymmetry.

What should you do about it?

If your business is exploring AI for tasks that involve checking, reviewing, or self-correcting information, you might need to adjust your expectations.

Don’t blindly trust AI evaluation: Especially for critical tasks, an AI judging its own work (or even human work) might not be as reliable as you’d hope.
Focus on specific use cases: For some complex, multi-step reasoning tasks like those in the MuSiQue benchmark, AI evaluation might be helpful. But you need to test it rigorously for your specific scenario.
Human oversight remains crucial: For now, the most reliable approach for quality control involving AI is likely a hybrid one. Let AI generate, but build in human review stages for evaluation.
Design for robustness: Instead of relying on an AI to “check itself,” design your AI systems to be correct from the start. Give them clear instructions, well-structured data, and guardrails to minimize errors in generation, rather than hoping a flawed evaluation step will catch everything.

This research, submitted in June 2026, offers a valuable reality check. While AI is incredibly powerful at generating text, its ability to critically evaluate that text isn’t always on par. Understanding this asymmetry is key to building more reliable and effective AI solutions for your business.

Frequently asked questions

What is LLM-as-a-Judge?

LLM-as-a-Judge refers to using a large language model (like ChatGPT) to evaluate the quality, accuracy, or relevance of another AI's output or even human-generated text. It's often assumed the AI is good at this.

Is AI better at generating or judging text?

Surprisingly, recent research from Sambaran Bandyopadhyay suggests that for many common tasks, large language models are actually better at generating answers than they are at judging them. This goes against common intuition.

What is 'in-context QA'?

In-context QA means the AI is given a specific passage of text (the 'context') and must answer questions *only* using information found in that passage. It's a controlled environment, not relying on the AI's broad general knowledge.

Why does AI struggle to evaluate its own answers?

The study found that when evaluating, AI models paid 3 to 5 times less attention to the provided context passage and barely focused on the candidate answer itself. This suggests a less thorough review process compared to when they are generating.

What are the business implications of AI's evaluation struggles?

If AI isn't as good at judging as we thought, businesses relying on AI for tasks like content review, automated fact-checking, or self-correcting agents might face higher error rates or reduced quality. It means re-evaluating where AI can truly provide reliable quality control.

How can I ensure AI evaluations are reliable?

For critical tasks, don't solely rely on AI self-evaluation. Consider hybrid approaches where humans review AI outputs, or design systems with strict guardrails and clear metrics that don't depend on the AI's internal judgment loops. Testing specific use cases is crucial.

What is attention analysis in AI?

Attention analysis is a technique that lets researchers see which parts of an input text an AI model focuses on most when processing information. It's like looking at a heatmap of the AI's 'eyes' as it 'reads'.

What is LoRA fine-tuning?

LoRA (Low-Rank Adaptation) fine-tuning is a method to efficiently adapt a large language model for a specific new task without retraining the entire model. It helps confirm if a model's behavior is fundamental or just a training artifact.

Which AI models were studied in this research?

The research evaluated two different large language models, though their specific names were not published in the public abstract, to ensure the findings were generalizable across model architectures.

When was this research published?

The paper, 'Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA,' was submitted to arXiv on June 26, 2026.

Related writing

Jun 16, 2026

Get a build memo.

Send us your thesis. A senior engineering team sends back a written assessment — architecture, cost to first production deploy, and the risks most likely to kill it. Five business days.

Request a build memo