Skip to content

Measuring how AI agents handle everyday wet-lab science

OpenAI

Developed with Ligo, the benchmark tests AI models on 100 routine wet-lab tasks using raw experimental evidence such as gels, blots, and microscopy.

Key points

  • Tasks span protein methods, imaging and cytometry, quantification assays, nucleic acid methods, and molecular cloning.
  • GPT-6 Astra achieved a 66.7% mean reward at the highest reasoning-effort setting, compared to 43.8% for GPT-5.6 Sol.
  • The evaluation tested seven OpenAI models across 37 reasoning-effort configurations and 18,500 total attempts.
  • Failure modes remain in resolving adjacent colonies, distinguishing control bands on PCR gels, and handling focus-dependent contrast changes.

Why it matters

Biologists and AI developers now have an evaluation suite to measure whether multimodal models can reliably interpret raw bench data and catch experimental errors.

Sources