AI evaluation life sciences benchmarks

LifeSciBench: how OpenAI measures AI on real science

Corteus Team · Jun 17, 2026 · 6 min read

A plain-English look at LifeSciBench, OpenAI's new test that checks how well AI handles real life science research, and what its low scores tell us.

If you use ChatGPT, you already do a small version of this. You ask it something, read the answer, and decide in your head whether it was any good. You are grading it. The question this post is about is what happens when you make that grading serious, careful, and run by real experts. That is what OpenAI just did for science.

On June 17, 2026, OpenAI introduced something called LifeSciBench. The plain-English version: it is a big, hard test that checks how well AI can do real life science research, and the results are honestly a little humbling. OpenAI’s announcement describes it as a way to measure and improve how well AI supports real-world life science research.

Before going further, three words worth pinning down, because the whole story rests on them.

A benchmark is a shared test. It hands the same set of tasks to many AI models and grades them the same way, so the scores can be compared side by side. Think of a standardized exam: everyone sits the same paper, so the results mean something.

Life sciences is the study of living things. Biology, genetics, drug research, the science behind medicine. The people doing it are scientists in labs and pharmaceutical companies.

Model evaluation is just checking how good an AI is at a job. Not by reading the brochure, but by giving it real tasks and grading what comes back.

So LifeSciBench is a benchmark for model evaluation in the life sciences. A careful test of how well AI does real science.

What makes this test different from a quiz

Most AI tests you may have heard about are closer to a trivia round. They ask questions with one right answer, often multiple choice, and count how many the model gets. That measures recall. It does not measure whether the AI can actually do the work.

LifeSciBench goes the other way. Its 750 tasks were written to mirror what scientists do on an ordinary day. Reading evidence and weighing it. Designing an experiment. Checking whether a result holds up. Explaining findings clearly. The answers are written out in full, not picked from a list of options.

Picture the difference between a multiple-choice driving test and actually being put behind the wheel in traffic. One checks if you memorized the rules. The other checks if you can drive. LifeSciBench is the road test.

The tasks span seven kinds of research work, including handling evidence, designing and optimizing experiments, scientific reasoning, validation, and communicating results. The point is breadth: not one narrow skill, but the spread of things a real researcher juggles.

Who built it, and why that matters

A test is only as good as the people who wrote it. Here the numbers do real work.

OpenAI says 173 scientists wrote the tasks, each holding a Ph.D. and carrying biotechnology or pharmaceutical experience. Then a second group, 453 reviewers, checked the tasks, and 97% of them held doctorates as well.

In plain terms: the people who built this test do this science for a living, and a separate set of experts double-checked their work. That is what stops a test like this from being a list of questions some engineer guessed at. It is the difference between a cooking exam written by chefs and one written by someone who has only read recipes.

How the grading actually works

Here is the clever part, and it is simpler than it sounds.

Every task comes with its own checklist of what a good answer should contain. The technical name is a rubric, but a checklist is the right mental picture. Across the whole test there are 19,020 of these criteria, which works out to about 25 per task. So an answer is not judged by a vague gut feeling. It is checked against a specific list of things it should have said.

A task counts as passed only if the answer scores 70% or higher against its checklist. A strict bar, on purpose.

This is the most useful idea in the whole story for a non-scientist. When you grade an AI by a clear checklist instead of a shrug, you find out exactly what it missed and where. Vague grading hides problems. Specific grading exposes them.

What the scores actually were

Now the humbling part. These are passing rates, the share of tasks each model fully passed:

GPT-Rosalind: 36.1%
GPT-5.5: 25.7%
Gemini 3.1 Pro: 23.6%
GPT-5.4: 20.7%
Grok 4.3: 13.0%

The best model in the field passed about one task in three. The rest did worse.

If that sounds like bad news, read it again. A good hard test is supposed to produce scores like this. If every model had scored 95%, the test would be too easy to teach anyone anything. The low numbers are the signal. They mark out, in plain figures, how far real-world science work still is from solved.

The weak spot the test exposed

The scores also pointed at one clear gap. The tasks did not all come as plain text. Many included supporting material like figures, data files, and documents, the kind of thing a scientist actually reads.

When the work involved those files, the AI struggled. The best model, GPT-Rosalind, passed 45.1% of text-only tasks but dropped to 28.1% once a task came with these supporting files. Reading words is one thing. Reading a chart or a data file and reasoning over it is clearly harder for the model, and the test caught that precisely.

There was a second reason the tasks were tough. About 79% of them needed several steps of reasoning rather than a single leap. Real problems rarely have a one-line answer, and stringing several careful steps together is where things tend to go wrong.

Why this should matter to you, even far from a lab

You may never touch a genomics task. The lesson still travels.

The way to judge any AI tool is the way LifeSciBench judges these models. Give it the real work you need done, not a tidy demo. Grade the output against a clear checklist of what a right answer looks like. Pay attention to the cases where it fails, because that is where it will fail you too.

Most people judge ChatGPT by whether one answer felt impressive. LifeSciBench is a reminder that feeling impressed and being reliable are not the same thing. A model can sound confident and still pass only a third of the work that counts. The honest move, whether you are buying an AI tool or building one, is to test it hard and trust the checklist over the charm.

Sources: openai.com

Frequently asked questions

What is a benchmark, in plain terms?

A benchmark is a standard test built to measure something the same way every time. For AI, it is a fixed set of questions or tasks plus a fixed way of grading them. Because every model takes the same test, you can line up the scores and compare them fairly, the way a standardized exam compares students.

What does 'life sciences' mean here?

Life sciences is the study of living things and how they work. It covers biology, genetics, drug research, and the science behind medicine. The people who do this work are the scientists in labs and pharmaceutical companies who study cells, test new treatments, and figure out why a result happened.

What is model evaluation?

Model evaluation is the act of checking how good an AI model is at a job. Instead of trusting a marketing claim, you give the model a set of tasks, see what it produces, and grade the answers against a known standard. LifeSciBench is one careful way to do that for science work.

What is LifeSciBench?

LifeSciBench is a test built by OpenAI to measure how well AI handles real life science research tasks. It contains 750 tasks written by working scientists and grades each answer against detailed checklists. It was described in OpenAI's announcement on June 17, 2026.

Who wrote the tasks?

According to OpenAI, 173 scientists with biotechnology or pharmaceutical experience wrote the tasks, and each held a Ph.D. A separate group of 453 reviewers checked the work, and 97% of them held doctorates. So the test was built and checked by people who do this work for a living.

How does the grading work?

Each task comes with its own checklist of things a good answer should include, called a rubric. Across the whole test there are 19,020 of these criteria, about 25 per task. An answer is graded against its checklist, and it passes only if it scores 70% or higher.

What were the scores?

The strongest model, GPT-Rosalind, passed 36.1% of tasks. Others scored lower: GPT-5.5 at 25.7%, Gemini 3.1 Pro at 23.6%, GPT-5.4 at 20.7%, and Grok 4.3 at 13.0%. Even the best result means most tasks were not passed.

Why are the scores so low?

Because the test is meant to be hard. The tasks reflect real research, which often involves messy evidence and many steps of reasoning. About 79% of tasks needed several reasoning steps. Low scores are a feature here: a test that AI aces tells you nothing about where it still struggles.

What is the weak spot the test found?

AI did much worse when a task came with supporting files like figures or data, instead of just text. The best model dropped from 45.1% on text-only tasks to 28.1% on tasks with these files. Reading and reasoning over real research material, not just words, is still a clear gap.

Why should a non-scientist care about this?

Because it shows how to judge any AI tool honestly. The lesson is to test a model on the real work you need done and grade it against a clear standard, rather than trusting a demo. That habit applies whether the job is science, contracts, or customer support.

Related writing

Jun 17, 2026

Get a build memo.

Send us your thesis. A senior engineering team sends back a written assessment — architecture, cost to first production deploy, and the risks most likely to kill it. Five business days.

Request a build memo