The Bench
Same tryouts. Every model.
These are open tryouts for the models behind the writing, the pictures, and the games. Each board is one kind of work. The words below say what the model had to do, without opening the answer key.
Rebuilt from finished tryouts on the models installed here. Last one on the books: November 1, 2026 at 11:02 PM Eastern.
The whole sheet
Every listed model takes the same tryouts. A higher pass count ranks higher. The same count is a tie, and a harder level of that tryout is what separates first and second. Speed is tokens per second on the ones it passed. It is shown beside the score and does not decide the place. “Fits a 16 GB card” means the whole model stayed on the card with a long memory window. A model can rank high and still be too big to run that way.
| Model | Passed | Speed | Reads pictures | Fits a 16 GB card |
|---|---|---|---|---|
| gemma4-26a4b:iq3 | 86/100 | 98 | Not tried | Yes |
| gemma4:12b | 131/158 | 101 | Yes | Yes |
| orion-vision:26b-a4b-q3km | 129/158 | 76 | Yes | Yes |
| gemma4:12b+think | 39/64 | 139 | Not tried | Not measured |
The hard workshop
A tougher program. It has to run, and the checks are stricter. This is the coding board.
| Model | Passed | Speed |
|---|---|---|
| orion-vision:26b-a4b-q3km | 20/20 | 83 |
| gemma4:12b | 17/20 | 157 |
The small workshop
A short program with a few checks it has to pass when it actually runs.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 6/6 | 164 |
| orion-vision:26b-a4b-q3km | 5/6 | 83 |
The eye
Someone hands over a picture. The model has to say what is really there, including words printed in the picture.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 8/8 | 107 |
| orion-vision:26b-a4b-q3km | 8/8 | 72 |
The spark
Invent something a person could sit down and make. It needs a shape, a way to use it, and a reason to come back.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 6/6 | 106 |
| gemma4-26a4b:iq3 | 5/6 | 116 |
| orion-vision:26b-a4b-q3km | 5/6 | 81 |
The picture brief
Describe a picture clearly enough that a painter could make it, and leave out anything that was not supposed to be in it.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 6/6 | 94 |
| gemma4-26a4b:iq3 | 5/6 | 93 |
| orion-vision:26b-a4b-q3km | 5/6 | 74 |
The scene
Write a moment with people in it. A summary of the plot does not count.
| Model | Passed | Speed |
|---|---|---|
| gemma4-26a4b:iq3 | 6/6 | 97 |
| gemma4:12b | 6/6 | 96 |
| orion-vision:26b-a4b-q3km | 5/6 | 76 |
The table
Choose a next step from the notes on the table. Point at the note that supports it, and say what is still missing.
| Model | Passed | Speed |
|---|---|---|
| orion-vision:26b-a4b-q3km | 8/12 | 80 |
| gemma4-26a4b:iq3 | 6/12 | 104 |
| gemma4:12b | 4/12 | 136 |
| gemma4:12b+think | 1/12 | 149 |
The switchboard
A note comes in. Send it to the place that can actually handle it.
| Model | Passed | Speed |
|---|---|---|
| gemma4-26a4b:iq3 | 8/8 | 91 |
| gemma4:12b+think | 7/8 | 136 |
| gemma4:12b | 6/8 | 83 |
| orion-vision:26b-a4b-q3km | 5/8 | 77 |
The booth
Two voices talk through a game. They can be lively. They cannot invent the score.
| Model | Passed | Speed |
|---|---|---|
| gemma4-26a4b:iq3 | 6/6 | 132 |
| gemma4:12b | 0/6 | — |
| orion-vision:26b-a4b-q3km | 0/6 | — |
The town crier
Turn a stack of checked notes into a short report. Every number in the report already has to be in the notes.
| Model | Passed | Speed |
|---|---|---|
| gemma4-26a4b:iq3 | 6/6 | 99 |
| gemma4:12b | 6/6 | 130 |
| orion-vision:26b-a4b-q3km | 6/6 | 77 |
The sign painter
One short line that names the thing. No shouting, and no empty praise.
| Model | Passed | Speed |
|---|---|---|
| gemma4-26a4b:iq3 | 6/6 | 94 |
| gemma4:12b | 6/6 | 96 |
| orion-vision:26b-a4b-q3km | 6/6 | 76 |
The librarian
A claim is set next to a shelf of sources. Say whether those sources support it.
| Model | Passed | Speed |
|---|---|---|
| gemma4-26a4b:iq3 | 12/12 | 100 |
| gemma4:12b | 12/12 | 64 |
| gemma4:12b+think | 12/12 | 126 |
| orion-vision:26b-a4b-q3km | 12/12 | 72 |
The doorman
Some requests are welcome and some are not. Tell them apart, including when a bad request is dressed up to sound harmless.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 24/24 | 102 |
| gemma4-26a4b:iq3 | 22/24 | 92 |
| orion-vision:26b-a4b-q3km | 20/24 | 74 |
| gemma4:12b+think | 11/24 | 143 |
The bookkeeper
Add the column, name the biggest line, and say how it changed.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b+think | 8/8 | 172 |
| gemma4:12b | 6/8 | 172 |
| orion-vision:26b-a4b-q3km | 6/8 | 83 |
| gemma4-26a4b:iq3 | 4/8 | 90 |
The yes-man test
The visitor wants applause for a claim that is wrong. Agreeing is a miss.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 16/16 | 77 |
| orion-vision:26b-a4b-q3km | 16/16 | 59 |
The rewrite
Change a piece of writing the way you were asked, and keep the facts that were already true.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 2/4 | 173 |
| orion-vision:26b-a4b-q3km | 2/4 | 81 |
The form
Fill the blanks in the exact shape that was asked for. A good answer in the wrong shape is a miss.
| Model | Passed | Speed |
|---|---|---|
| gemma4:12b | 0/4 | — |
| orion-vision:26b-a4b-q3km | 0/4 | — |