Compare AI models on your own questions
Ask one question of several models side by side, then score an assistant's checks across models to pick the one that gets them right for the least money.
On this page
Models change every few months, and their prices differ by a hundred times. Compare models lets you decide with your own questions rather than someone else's benchmark.
Models side by side#
Choose up to three models. Search by lab or by name. Give them shared instructions and source files if you like, then send a question.
| Claude Haiku 5.5 | GPT-6 Luna | Gemini 3.8 Flash | |
|---|---|---|---|
| Answers | 1 | 1 | 1 |
| Your picks | 1 | 0 | 0 |
| Cost per answer | $0.0011 | $0.0009 | $0.0004 |
| Time | 2.1 s | 1.6 s | 1.2 s |
| Cited sources | 1 of 1 | 1 of 1 | 0 of 1 |
Each answer arrives in its own column with the time it took, what it cost, and the sources it used. Press Pick answer on the one you prefer and the scorecard keeps count.
Tag a model with @1, @2, or @3 in your question to ask only that one. Save as check turns any question into a behavior check with a rule in your words.
Scoring an assistant#
Models side by side tells you what you prefer. To find out which model gets the facts right, press Score an assistant, choose one, and go through four steps.
- Questions. Use its behavior checks, or write some for it. See Behavior checks.
- Models. A ready-made set (the cheapest, a mid-priced one, and the most capable) or your own choice, up to six. Ask each question once, twice, or three times to see how steady a model is.
- Run. The cost is shown before you start. The run stops if your allowance runs out.
- Results. A table shows how many questions each model got right every time, where it was inconsistent, the cost per thousand answers, and the time per answer. One model is marked as the pick. It is the one with the most right answers, and then the lowest cost.
| Model | Right every time | Mixed | Per 1,000 answers | Per answer |
|---|---|---|---|---|
| GPT-6 Luna · recommended | 9 of 10 | 1 | $1.10 | 1.6 s |
| Claude Haiku 5.5 | 9 of 10 | 1 | $1.40 | 2.1 s |
| Gemini 3.8 Flash | 7 of 10 | 2 | $0.60 | 1.2 s |
Press the Use button and the assistant switches to that model. Every answer is kept, so you can read exactly where a model went wrong, and you can download the whole run as a spreadsheet.
Assistants side by side#
The second tab opens whole assistants next to each other in a grid of up to six tiles. Each tile is the real chat or expert room for that assistant, so you can put the same question to two assistants built on different sources, or watch a chat and its expert room at once. Tiles stay open between visits on that browser.
| 1 · Support guide · chat | 2 · Sales desk · chat |
|---|---|
| Can I return a cut board? | Can I return a cut board? |
| No. Cut boards cannot be returned… | Returns are easy, and we offer… |
From comparison to assistant#
Under How the models compare, Build an assistant with it turns the comparison into a private assistant. Name it, choose the model it answers with, and press Continue to builder. Its instructions, sources, and saved checks come along. You decide when to publish it.
Updated Nov 3, 2026