About the benchmark

Build Minecraft from scratch for the web, including crafting, ore progression, and mobs. Each model worked in its own project directory.

The Builds page links directly to the published games. Screenshots show the actual attempts.

Model names and effort settings come from the completed Pi build sessions. GPT-6 Sol used max; GPT-6.1 Sol, Sonnet 5.5, and Opus 5.5 used xhigh. Gemini 4 Argon used high. Its completed restart replaces the earlier medium-effort build.

Effort is the setting recorded by Pi, rather than a measure of how much reasoning a provider performed. All five builds ran through GitHub Copilot.

Build time measures wall-clock time from the original Minecraft build prompt to the assistant’s completion response in the same Pi session. It includes coding, tool execution, and testing, and excludes later Tailnet setup, publishing, and unrelated follow-ups.

Opus 5.5 has no recorded build-completion response. Its card shows time to the first playable URL response instead; testing was still in progress. This is an earlier milestone, not a completed-build time.

These are elapsed session timings, not model inference time. Token and cost totals cover the recorded session and may include follow-up work, so their scope can differ from the build timing.

Token totals include input, output, cache reads, and cache writes recorded by Pi. Each USD total sums the token costs recorded by Pi for its completed build session, including cached input and output. Separate aborted and setup sessions are excluded.

These are model-rate estimates, not verified Copilot charges or invoices. They cover the build session, rather than the work to publish this benchmark site.

Open the games on the same device and browser. Try movement, mining, placing blocks, crafting, ore progression, and mobs.

This is an attempt catalog with no assigned scores. Prompt wording and follow-up requests varied between sessions, so the results are not a controlled test with equal time or token budgets.