You could swap in a top-tier model and get about the same as a mid-tier one
On August 1, Supabase open-sourced a benchmark called Evals that runs Claude Code, Codex, and OpenCode against real backend tasks - building database tables, fixing broken Edge Functions, patching RLS policies. The result is a little counterintuitive: the strongest models score 100% out of the box, while mid-tier models, once given the platform's "skills" files, climb from the 70s and 80s to 100%.
The model name ranks lower than whether the agent actually understands your project.
The numbers, in plain terms
- Opus 5 and Kimi K3 hit 100% on the Build stage with no skills loaded at all.
- Sonnet 5 78% bare, 100% after loading Supabase skills.
- GPT-5.6 Sol 89% up to 100%.
- GPT-5.4 mini 78% up to 89%.
Here's the translation: run Sonnet 5 or GPT-5.6 Sol, then feed the platform's skills file - think of it as a current operations manual for the AI - and the model catches up to the top tier. Small spend, big lift.
Three failure modes, not sugar-coated
- Agents prefer hand-writing SQL migrations instead of using declarative schema tooling - a trap for later.
- They hand-roll auth instead of calling the purpose-built @supabase/server package.
- Docs habits split hard: Codex-class agents read about 8 doc pages per scenario, while Claude Code checks docs in under 40% of runs, leaning on training memory that breaks the moment the API changes.
Why this matters to you
When you hire an AI to write your backend, don't just ask "which model." Ask "can it read my project's context and the latest docs." A small team on a mid-tier model with skills loaded is steadier and cheaper than a top-tier model running blind.
Zoom out: the top coding models are already neck and neck - on one tracker Mythos 5, Fable 5, and GPT-5.6 Sol sit within 1.7 points of each other. Paying more for those last points rarely pays off.
The benchmark reruns daily, so scores move as models ship updates. Don't treat one screenshot as gospel; judge a tool by steady performance in your own scenarios.
Wrap-up
The deciding factor in AI-written code is shifting from "how big the model" to "how accurate the context." For related reading, the hands-on comparison of Claude Code, Codex and other coding agents, plus how AI coding agents are actually evaluated.