Cost
Measured110×
Two real price tables and signed token counts. No coefficients, no range. Safe to state anywhere: a skeptic has to dispute Anthropic's price page or your own logs.
Ultimus · Benchmark · Snapshot 20 September 2026
What the two harnesses actually cost the GPUs
Claude Code on Opus 5 against Ultimus on an open-weight model, measured from 61 real session logs and run through one arithmetic. Tokens and dollars are measured. Energy is an estimate, and one assumption inside it does all the work.
- 61
- sessions
- 3,482
- model calls
- 3.81M
- output tokens
- 316M
- input tokens
Measured · no coefficients involved
Cost
Two real price tables, Baseten's cached gateway table and Anthropic's list, multiplied by token counts from the logs. Nothing here is estimated, so there is no assumption for a skeptic to pull on.
110×
more per 1,000 output tokens on Claude Code / Opus 5 than on Ultimus. Against Fable 5.1 it is 170×.
| Corpus | Sessions | Calls | Output tokens | Cost | $ per 1k output tokens |
|---|---|---|---|---|---|
| Claude Code / Opus 5 | 13 | 1,182 | 1,295,366 | $198.51 | 0.1532 110× |
| Claude Code / Fable 5.1 | 17 | 1,095 | 1,923,387 | $457.22 | 0.2377 170× |
| Ultimus | 31 | 1,205 | 593,257 | $0.83 | 0.0014 1× |
Corpus totals. One model per session; sessions under 3,000 output tokens excluded as false starts. Ultimus ran on an open-weight model served through Baseten at $0.13 input, $0.028 cached and $0.26 output per million tokens. Anthropic list price for Opus 5 is $5, $0.50 and $25.
Estimated · one assumption does all the work
Carbon
Token counts are measured; turning them into energy needs coefficients, and those are stated and editable. But it also needs one number nobody publishes: how much more energy an Opus token costs than a token from a smaller open-weight model. Picking a value would smuggle the whole answer in as a constant, so the sweep below leaves it open.
- Ultimus / open-weight model0.35 g
- 1× identical hardware0.44 g · 1.2×
- 5× conservative2.19 g · 6.2×
- 10× central4.38 g · 12.4×
- 20× aggressive8.76 g · 24.8×
- 38.5× price ratio, input16.87 g · 47.7×
- 96.2× price ratio, output42.16 g
View as a table
| Scenario | g per 1k output | × Ultimus |
|---|---|---|
| Ultimus / open-weight model | 0.35 | n/a |
| 1× identical hardware | 0.44 | 1.2× |
| 5× conservative | 2.19 | 6.2× |
| 10× central | 4.38 | 12.4× |
| 20× aggressive | 8.76 | 24.8× |
| 38.5× price ratio, input (upper bound) | 16.87 | 47.7× |
| 96.2× price ratio, output (upper bound) | 42.16 | n/a |
Read this before quoting the chart
The two hatched rows are what you get by scaling energy with price, the harness's own fallback rule. They are an upper bound, not a measurement: Anthropic's margin on a frontier model and Baseten's on a commodity one are different business decisions, and margin is not physics. Quoting the 96.2× row as a carbon saving quotes a pricing decision as a thermodynamic one. Break-even is 0.81×: above it, the Ultimus run is lower-carbon per unit of output.
Assumptions: 0.4 / 0.02 / 0.002 Wh per 1k output / input / cached tokens · PUE 1.2 · 400 gCO₂e/kWh · reported range ×0.25–×4 · Ultimus held at scale 1× throughout.
The step penalty
Per-output-token ratios flatter the cheaper model. They credit it for tokens that never converged. On one real fix in the same repo, the open-weight model alone took 15 steps where a stronger consultant took 8. Fold in that ~1.9× step penalty plus a retry allowance and the conservative 5× cell drops toward ~3×, while the 1× floor inverts outright.
The loop itself · both sides at the same energy scale
The harness is not where the carbon is
Force both sides to one energy scale and only loop behaviour differs. Claude Code carries roughly six times the context per call, and still lands within 1.24× per unit of output, because almost all of that context is cache reads, which cost a tenth of a fresh input token.
Hit rate looks like the metric here, and it is a trap: context size sits in its denominator, so hoarding context raises the score. The quantity that burns energy is fresh input tokens appended per call.
- Claude Code / Opus 54,030
- Ultimus / open-weight model3,286
Ultimus appends 18% fewer fresh tokens per call while scoring 5.5 points worse on hit rate. Run its append rate against Claude Code's context size and it scores 98.7% against their actual 98.4%. The entire apparent gap was the denominator.
Where every fresh input token goes
3,715,440 fresh input tokens · 1,112 instrumented calls · 31 Ultimus sessions
- Irreducible growth78.5%
- Provider eviction13.5%
- Harness rewrote history4.2%
- Cold start3.7%
The eviction slice is the finding. prefixStable was recorded true in every single instance: the harness held the prefix byte-stable and the gateway dropped it anyway. One session went from 65,216 cached tokens to 2,272 while the prompt moved 94 tokens, seconds apart, with retained counts landing on round block boundaries: radix-cache eviction under memory pressure, not prefix drift. There is no harness-side fix; the remedies are operational.
Step count is the dominant term · one way to cut it
The operator can point at the screen
Everything above says the same thing twice: per-task energy is governed by how many steps a task takes, not by how cheap a token is. That is why the 15-versus-8 step result moves the carbon estimate more than any coefficient on this page does.
Visual work is where step count goes wrong worst. A misaligned element described in prose costs several rounds: the model reads files, guesses, edits, asks for confirmation, guesses again. Ultimus closes that loop directly. The operator freezes the screen, circles the problem, types "16px gap", and the agent receives the marked image and the structured annotation beside it: mark positions, labels, colors, and which window it came from. A layout bug that took four rounds to pin down in words lands in one.
It runs entirely on the machine. Capture and annotation are local, the inbox is a local directory, and it attaches over the harness's own tool-server client, the same path that pins every external tool definition on first use and withholds it when it changes. No network call, no telemetry, no third-party vision service holding a picture of the operator's screen. For work under confidentiality that is the difference between a usable capability and one that is not allowed in the room.
Most harnesses treat the agent as text-in, text-out and leave the screen to the human to narrate. Giving the agent eyes locally is a capability difference, and on this page's own arithmetic it is also a carbon one.
What survives a hostile reading
Three claims, three confidence levels
Carbon, per completed task
Estimated~3–25×
Visible coefficients plus one model-scale assumption nobody can currently verify, discounted for the step penalty the cheaper model carries. Not 79×, and never the 96.2× price ceiling.
Harness efficiency
Marginal~1.2×
Real but small, and it is not the compaction and pruning machinery doing it. The savings in this comparison come from running a cheap model, not from running a better loop.
Method
How this was produced
Both sides were reduced to the same shape, measured tokens per model call, stamped with the same pinned coefficients, and pushed through the harness's own carbon report. Neither side gets different arithmetic. The harness reproduces Ultimus's carbon on a real session to the digit, which is what validates it.
Three things a reader should hold against it:
- 01No matched task exists across the two harnesses. Per-output-token normalisation mitigates the mismatch; it does not remove it.
- 02Output tokens are not delivered work. Opus 5 runs adaptive thinking, so some of its counted output is reasoning the reader never sees, which pushes its real per-delivered-token figure up, in the direction this page does not claim credit for.
- 03The model-scale assumption is the whole carbon result. It stays a sweep until there is a published energy-per-token figure, or a locally hosted model measured at the wall.
Only Claude Code was measured. There are no Codex numbers in this benchmark.
Ultimus