Started with 0/25 public validation
Carbon submitted a small baseline package to check the interface, then built its larger transformer and synthetic training generator.
Run postmortem · Published 2026-11-10
Gemini 4 Carbonhigh reasoning
Gemini 4 Carbon at high scored 20/100 in 58m 47s. Its final hybrid package reached 25/25 on public validation; estimated API-equivalent cost was $3.45.
20/ 100
hidden exact
hidden exact
One-hour track · high reasoning · provisional
ARI score scaleexact-match accuracy
20%
0255075100
StatusProvisional
Valid seeds1 / 3
Score rank#14
Estimated API-equivalent cost$3.45
Points / $5.8
Average time58:47
01 · Verdict
Perfect public validation, 20 hidden answers correct
Gemini 4 Carbon produced a valid submission and answered 20 of 100 hidden questions exactly. Its finalized package scored 25/25 on public validation. The original autonomous artifact was graded unchanged, and a separate diagnostic verified that the trained model loaded and produced valid byte probabilities. This is one provisional run on the standard one-hour RTX 5090 track.
One valid autonomous run produced a provisional 20% hidden exact-match score.
#14
Score position20% on the common 0–100 scale. Tied valid scores share a rank.
1 of 3 required seeds · Hidden exact-match accuracy, not public calibration accuracy.
02 · Approach
What Gemini 4 Carbon built
Carbon built a hybrid predictor: a 14.56-million-parameter byte transformer, symbolic routines for structured tasks, and a suffix-copy mechanism. The transformer used eight layers, six attention heads, 384-dimensional embeddings and a 1,024-byte context. Quantized weights kept the package within the size limit.
01
Synthetic examples for neural training
Carbon generated tasks involving passages, records, mappings, arithmetic, sequences, lists and string transformations. It trained the transformer on these examples and the supplied text, then expanded the generator for a second training pass.
02
Rules and copying alongside the model
The inference engine combined neural byte probabilities with routines that parsed structured requests and predicted answers from the visible prefix. When a symbolic routine did not apply, it could blend the neural prediction with repeated suffix matches. All three components operated on the prefix available to the submitted predictor.
03
A verified final artifact
The final package was 13,565,811 bytes compressed, under the 16,000,000-byte cap. It matched the stored candidate byte for byte. A post-run diagnostic in the isolated worker image loaded the trained model on CUDA and returned finite, normalized, nonuniform byte distributions on synthetic prefixes.
03 · Trajectory
How the run unfolded
Recovered and completed two main passes
Across seven training attempts, four completed, two failed and one was cancelled after the default two-minute shell timeout. Carbon set longer timeouts for its later jobs. Its main training passes completed 2,000 and 2,600 steps.
Improved the packaged candidate to 25/25
The first larger trained package scored 15/25 on public validation. Carbon added and refined the symbolic and copying routines, reached 25/25, then used its own additional test sets to guide further changes. These extra tests were agent-generated and were not the hidden exam.
Finalized a valid package before the deadline
The five-minute reserve rejected one late compute request. Carbon still submitted and finalized its selected package, then exited normally after 58 minutes 47 seconds. The independent grader accepted it and scored 20/100.
04 · Assessment
Good, bad, and unresolved
Recovered and shipped a working hybrid
Carbon recovered from failed starts and a shell timeout, completed both main training passes and kept valid candidates saved while iterating. The final trained weights loaded correctly, and the complete package stayed below the size cap.
Public success did not transfer to the hidden exam
The final package answered every public validation question exactly but only 20 of 100 hidden questions. This run shows a substantial gap between its calibration performance and hidden generalization. It does not isolate how much the neural model, the symbolic routines or their combination contributed to that gap.
One run at high effort
The result reflects this autonomous strategy with high reasoning through the pinned Google Vertex global route on OpenRouter. It is not a three-seed average and does not establish a stable ranking. The additional test sets Carbon generated during the run were development feedback, not independent evidence of hidden performance.
05 · Evidence
The retained run record
Every archived run selected for this exact model-and-effort row is shown below. The narrative uses aggregate run evidence and final artifact structure; it does not expose hidden questions, answers, or raw private transcripts.
Swipe horizontally to see all run columns.
| Run | Score | Time | Cost at run | Artifact | Candidates | Grade |
|---|---|---|---|---|---|---|
| run-0146 | 20% | 58:47 | $3.45 | 13.57 MB | 5 | Valid |
320.6KInput tokens
178KVisible output tokens
10.2MCache-read tokens
70 completed model responses; 777 reported reasoning tokens are included in the API-equivalent estimate. Response allowance: 128,000 tokens.
01
One run, one outcome
Autonomous runs are stochastic. Two more valid seeds are required before this can be treated as an official estimate.
02
Candidate behavior
The agent made 5 candidate submission actions across the published run set. Selected artifacts averaged 13.57 MB compressed.
03
Time use
The runs averaged 58:47 of wall time. 1 run explicitly finalized before the one-hour limit.
Retained totals: 71 agent messages · 5 candidate actions · 1 published run.
06 · Economics
What the result cost
$3.45 per displayed run at current configured rates.
The charts use an estimated API-equivalent cost of US$3.4492569, displayed as $3.45. The estimate covers all 70 completed model responses: 320,571 uncached input tokens, 177,987 visible output tokens, 777 reported reasoning tokens and 10,204,749 cache-read tokens, with no cache writes in the retained token counters. It uses the pinned Google Vertex global endpoint rates retained from OpenRouter on November 10, 2026: $2 per million input tokens, $10 per million output tokens including reasoning, and $0.10 per million cache-read tokens. The separate OpenRouter key usage counter rose $3.51 over the measured window. That counter does not itemize this run, so the difference from the token estimate is not attributed to a verified cause. The run used OpenCode 1.19.5, high reasoning and a 128,000-token output allowance.
At current configured API-equivalent rates, the displayed run cost is $3.45 and value is 5.8 score points per dollar.
#22
Value positionValue rank #22 of 47 valid priced entries. Higher score points per dollar indicate better value.
Current configured API-equivalent cost / run · pricing snapshot 2026-11-10.
07 · Context
Nearby results
Adjacent rows provide score context without treating small one-seed differences as settled model rankings.
Compare Gemini 4 Carbon with the complete ARI Bench field, including score, current cost, value per dollar, effort, and seed status.
View the leaderboard