Evaluate clinical trial AI against the people who do the work
Text is not what is scarce. What is scarce is evaluation in a domain where being wrong has consequences, with what a qualified person actually decided on the same item beside the model's answer.
What we offer
- Free: The AI scoreboard: seven models against practitioners on the same published decisions, with both public corrections.
- Free: The CRLN-AIE-001 protocol and its results, published under CC BY 4.0 with a DOI for every version.
- Paid work: Model evaluation: your model on rubric-scored clinical trial decisions, compared with a published human baseline, with the method and its limits stated.
- Paid work: Expert-made data: ratings of model answers, written and keyed scenarios, and red-team tasks, created under contract by practicing clinical research professionals who chose to opt in to paid work. Opting in is off by default.
Pricing is shared on a short call, never in writing first.
Proof you can check
- A result. On 114 published clinical trial conduct decisions, two of seven AI models were statistically indistinguishable from practitioners, four scored modestly higher and one scored well higher.
- A dated benchmark. Models from three vendors and Practice Lab learners on the same site decisions, with stated confidence and stability under reshuffled options. It says first what the set cannot show.
- Corrections in public. Version 0.2 of our evaluation protocol publicly reversed the conclusion of version 0.1, and version 0.2.1 retired a held-out set whose answers could be read. Every version stays citable beside the note that corrects it.
How it works
- Tell us the task your model does and the decisions that matter to you.
- We agree the items, the rubric, the human baseline and who does any expert work, in writing.
- You receive scored results against the baseline, with the method and its limits stated.
What we never do
- Use learner data to train or tune your model, in any form.
- License row-level learner records, at any price, under any agreement.
- Share learners' free-text reasoning in their own words.
- Share anything from which a person could be identified. Nothing about a group of fewer than five people leaves the platform.
- Take work on patient recruitment, prescreening, matching or referral.