Recovered from a shell timeout
The first training call hit the 120-second OpenCode shell timeout after 2,700 logged steps. Fable submitted a saved checkpoint that scored 13/25 on public validation.
Run postmortem · Published 2026-11-09
Claude Fable 5.2xhigh reasoning
Claude Fable 5.2 at xhigh scored 17/100 in 52m 48s. Its trained model reached 24/25 on public validation; estimated API-equivalent cost was $5.11.
17/ 100
hidden exact
hidden exact
One-hour track · xhigh reasoning · provisional
ARI score scaleexact-match accuracy
17%
0255075100
StatusProvisional
Valid seeds1 / 3
Score rank#22
Estimated API-equivalent cost$5.11
Points / $3.33
Average time52:28
01 · Verdict
A working model that struggled on the hidden exam
Fable 5.2 produced a valid submission and answered 17 of 100 hidden questions exactly. The same package scored 24/25 on public validation. The grader evaluated the original autonomous submission, and a separate diagnostic confirmed that its trained weights loaded and produced valid byte probabilities. This is one provisional run on the standard one-hour track.
One valid autonomous run produced a provisional 17% hidden exact-match score.
#22
Score position17% on the common 0–100 scale. Tied valid scores share a rank.
1 of 3 required seeds · Hidden exact-match accuracy, not public calibration accuracy.
Anthropic’s AI R&D limits. Anthropic says it limited gains during training in some capabilities that could accelerate frontier AI development, including improving large language model architectures and optimizing accelerator kernels. It says ordinary machine learning research was unaffected. Because ARI Bench asks the agent to design and train a model, those limits could have affected this result. This single run can’t establish whether they did. Anthropic’s disclosure
02 · Approach
What Claude Fable 5.2 built
Fable built a 6.44-million-parameter byte transformer with eight layers, eight attention heads, 256-dimensional embeddings and a 1,024-byte context. The training generator produced synthetic questions in exam format. At inference, the submitted model predicted each byte directly from the prefix.
01
Synthetic questions for training
The generator made examples for copying, arithmetic, sequences, mappings, records, schedules and other short tasks. It borrowed vocabulary from the supplied text and assigned extra weight to answer bytes during training.
02
All answers came from the trained model
The submitted inference code used the transformer for every nonempty prefix. It did not contain dedicated routines for sorting, arithmetic or looking up records. The model had to learn those operations from examples and transfer them to new wording and inputs.
03
A verified final package
The finalized artifact was 11,918,705 bytes compressed, within the 16,000,000-byte cap. It matched the stored candidate byte for byte. In the isolated grader image, the trained weights loaded on CUDA and produced finite, normalized, nonuniform byte distributions on synthetic prefixes.
03 · Trajectory
How the run unfolded
Completed three more training jobs
Fable set explicit timeouts for its later training calls. The three jobs ran for about 15, 16 and 7.3 minutes and logged 21,000, 22,300 and 10,100 steps respectively. Public validation reached 23/25 twice, then 24/25.
Selected the 24/25 public candidate
Fable finalized its fourth valid submission and exited normally after 52 minutes 48 seconds. All 19 broker requests received responses. The run used OpenCode 1.19.5, xhigh reasoning and a 128,000-token output allowance.
17 of 100 hidden answers matched
The independent grader accepted the unchanged artifact. It missed the first byte on 69 of its 83 incorrect answers. The model scored 0/26 across local rule, numeric sorting and dialogue questions, and 1/22 on answers at least ten bytes long.
04 · Assessment
Good, bad, and unresolved
Recovered and finished with a working model
Fable recovered a checkpoint, completed three subsequent jobs and improved public validation from 13/25 to 24/25. Its final package loaded successfully and remained under the size cap.
Public validation overstated hidden performance
The final transformer had to generalize from the patterns in its synthetic generator. The number of first-byte misses indicates that many predictions were already wrong before exact formatting could account for the failure. The result supports a generalization gap; it does not isolate the effect of any one training choice.
One xhigh run
The score reflects this run's autonomous strategy. It does not establish a stable ranking or show what a different strategy would have scored. An earlier attempt stopped when its WSL environment shut down, before a candidate was submitted. That attempt had no hidden grade and is excluded from the published result.
05 · Evidence
The retained run record
Every archived run selected for this exact model-and-effort row is shown below. The narrative uses aggregate run evidence and final artifact structure; it does not expose hidden questions, answers, or raw private transcripts.
Swipe horizontally to see all run columns.
| Run | Score | Time | Cost at run | Artifact | Candidates | Grade |
|---|---|---|---|---|---|---|
| run-0145 | 17% | 52:28 | $5.11 | 11.92 MB | 4 | Valid |
260.9KInput tokens
33.2KVisible output tokens
1.02MCache-read tokens
24 completed model responses; 37,280 reasoning tokens are included in the API-equivalent estimate. Response allowance: 128,000 tokens.
01
One run, one outcome
Autonomous runs are stochastic. Two more valid seeds are required before this can be treated as an official estimate.
02
Candidate behavior
The agent made 4 candidate submission actions across the published run set. Selected artifacts averaged 11.92 MB compressed.
03
Time use
The runs averaged 52:28 of wall time. 1 run explicitly finalized before the one-hour limit.
Retained totals: 25 agent messages · 4 candidate actions · 1 published run.
06 · Economics
What the result cost
$5.11 per displayed run at current configured rates.
The charts use an estimated API-equivalent cost of US$5.1088806, displayed as $5.11. The estimate covers 24 completed model responses: 260,941 uncached input tokens, 33,176 visible output tokens, 37,280 reasoning tokens and 1,015,563 cache-read tokens, with no cache writes in the retained OpenCode token counters. It uses the Anthropic endpoint rates retained through OpenRouter on November 9, 2026: $8 per million input tokens, $40 per million output tokens including reasoning and $0.20 per million cache-read tokens. The OpenRouter key usage counter rose $5.66 over the measured window. The key counter does not itemize this run, and the source of the difference from the token estimate was not verified. Usage from the interrupted attempt is excluded from the token estimate.
At current configured API-equivalent rates, the displayed run cost is $5.11 and value is 3.33 score points per dollar.
#30
Value positionValue rank #30 of 46 valid priced entries. Higher score points per dollar indicate better value.
Current configured API-equivalent cost / run · pricing snapshot 2026-11-09.
07 · Context
Nearby results
Adjacent rows provide score context without treating small one-seed differences as settled model rankings.
Compare Claude Fable 5.2 with the complete ARI Bench field, including score, current cost, value per dollar, effort, and seed status.
View the leaderboard