Run analysis 002 · August 12, 2026

Grok 4.6 was fast.The context was expensive.

A 26% hidden score in under 24 minutes made Grok 4.6 the strongest single ARI Bench run published at the time. Its $9.80 cost came from one very specific place: 18 million cached-context tokens.

26/ 100
hidden exact

One-hour track · xhigh reasoning · one seed · provisional

run startfirst candidatefinalized early
StatusValid
Cost$9.7988
Time used23:30
Final artifact10.26 MB
Candidate actions10
Run0084

Grok 4.6 delivered a genuinely strong run, then stopped while it still had more than half of the hour available.

It produced a valid, self-contained artifact and answered 26 of 100 private questions exactly. The run was operationally clean: the agent stayed in one pinned session, the broker recorded no protocol errors, nine candidate packages were promoted successfully, and the selected candidate was finalized before the deadline.

But the result is still provisional. It is one autonomous run, not the three seeded runs required for an official estimate. The right interpretation is “best observed Grok 4.6 run so far,” not “repeatable Grok 4.6 average.”

The expensive part was not fresh prompting or output. $9.00 came from cached context reads.

OpenCode ran a long, highly active agent session. As the transcript accumulated code, tool results, validation output, and decisions, later model turns revisited the growing history. Grok’s automatic prompt cache reduced the price of that repeated prefix—but cached tokens are discounted, not free.

$9.7988

Archived API-equivalent total. The public price reconstruction below reproduces it exactly.

$9.000018.0M cache-read tokens · 91.8%
$0.4058202.9K fresh input · 4.1%
$0.393065.5K output · 4.0%

Reconstruction uses the OpenRouter catalog rates available on August 12, 2026: $2/M fresh input, $6/M output, and $0.50/M cached input. OpenRouter documents Grok cache reads at one-quarter of normal input price and automatic prompt caching. Sources: OpenRouter model catalog and prompt-caching documentation.

Caching helped substantially. Repricing those 18 million repeated tokens as ordinary $2/M input would put the same usage near $36.80. The cache saved roughly $27; it simply could not make an enormous repeated context free.

Grok did not rely on a language model alone. It built a hybrid completion system: explicit task-solving logic handled recognizable structures, while statistical copying and a small trained byte model covered uncertain cases.

01

Deterministic solver layer

A broad inference program handled structured records, local mappings, arithmetic, ordering, sequences, text transforms, time and date operations, and extraction-style completions.

02

Copy and local statistics

Prefix matching, short-context frequency estimates, and retrieval-like behavior gave the system a way to reuse information already visible in context.

03

Neural fallback

It trained a compact byte-level transformer, first as a quick checkpoint and then on a larger mixture that included synthetic task-shaped data.

04

Package-first iteration

It promoted a valid checkpoint around minute nine, then repeatedly packaged and tested updates instead of waiting until the end to discover an artifact failure.

The run was unusually compressed. Grok moved from inspection to a timeout-safe candidate quickly, completed its larger training pass with most of the hour still available, and then spent the final stretch widening and retesting its inference logic.

Reframed the task

Inspected the starter and visible calibration interface, then chose a hybrid solver rather than treating ordinary next-byte training as sufficient.

Established a valid checkpoint

The first promoted artifact was only 3.21 MB and already saturated the small public calibration set.

Trained the larger fallback

A compact neural model trained on a mixture of ordinary and synthetic task-shaped text. The packaged artifact grew to roughly 10.26 MB.

Iterated aggressively

Grok repeatedly expanded the solver, ran local checks, and promoted fresh packages. All nine submitted candidates remained valid.

Finalized early

The model deliberately ended the run with about 36½ minutes unused. The hidden grader then produced the official 26% score.

The good

Fast insight and disciplined packaging

Grok identified the need for in-context problem solving, built a broad hybrid system, created an early checkpoint, and never lost the official artifact path.

The bad

Public saturation caused blind iteration

Every promoted package scored 25/25 on public calibration, while the hidden result was 26/100. Once the visible signal saturated, many later edits could not be ranked meaningfully.

The unknown

Would another 36 minutes help?

Maybe—but the run had no unsaturated public metric to guide it. More time could have diversified tests, or merely increased complexity and context cost.

The session’s implementation volume was high: 137 recorded messages, 181 tool uses—including 95 edits—and 56 compute commands. That intensity helped produce a sophisticated artifact quickly. It also explains the growing transcript and the dominant cache-read bill.

Against the three-run Grok 4.5 result already on ARI Bench, Grok 4.6 was both stronger and cheaper in this observed run. The comparison is directional, not definitive: Grok 4.6 has one xhigh run, while Grok 4.5’s displayed number averages three high-effort seeds.

Grok 4.5 · three-run average16%

$15.25 per run · 1.05 score points per dollar

Grok 4.6 · one provisional run26%

$9.80 · 2.65 score points per dollar

See Grok 4.6 in context—including score, cost, value per dollar, seed count, and results from other frontier models.

View the leaderboard