Gemini built a hybrid byte predictor that combines a trained transformer, a prefix n-gram copier and a symbolic solver. It generated synthetic examples, trained several models and repeatedly tested packaged candidates.
01Symbolic rules with a neural fallback
The inference code handles local mappings, records, arithmetic, sequences and passages with explicit rules. Otherwise it uses the trained byte transformer, lightly blended with a prefix n-gram copying distribution.
02Seven successful training jobs
Eight training attempts included seven successful jobs totaling 36,700 steps, including the 200-step starter warmup. The substantive models used six layers, width 384, six attention heads, RoPE, RMSNorm and SwiGLU. Training mixed the public corpus with generated examples and weighted answer bytes more heavily. The selected weights came from the successful 8,000-step model_v6 pass, with a 512-byte neural context.
03A verified package under the size cap
Fourteen candidate submissions included thirteen valid packages and one oversized package. Gemini explicitly finalized its selected candidate. The 10,098,911-byte gzip artifact fits the 16 MB cap. The graded code, configuration and weights match the saved selected candidate; large weight matrices use int8 quantization with row scales.