An open benchmark comparing how Claude Code behaves on the same website tasks across five platforms: Shopify, WordPress, Astro, Next.js, and plain HTML.
The methodology is published before the data, deliberately. Criticism of a method is only worth anything before the results exist. Afterwards it is very hard for anyone — including the person who ran it — to tell a genuine methodological objection from a dislike of the outcome.
So this page currently contains a method, a task list, a schema, and no findings.
<!-- BENCHMARK-RESULTS:START -->
Results
Nothing has been measured yet. There are no results on this page, and no figure below should be cited, because there are none.
The methodology, the task list and the data schema are published first, on purpose. They are the part worth arguing with, and criticism of a method is only useful before the data exists — afterwards it is very hard to tell a genuine methodological objection from a dislike of the result.
The dataset is research/benchmark/results.csv in the public repository. It currently contains a header row and nothing else.
When trials run, this section is regenerated by scripts/benchmark-analyse.py from the raw rows. It is not written by hand, so the page cannot say something the dataset does not.
<!-- BENCHMARK-RESULTS:END -->
The question
When Claude Code is asked to do the same website task on five different platforms, what differs — and how much of the difference is the platform rather than the task?
Not "which platform is best". That question has no answer, and a benchmark claiming to have found one would mostly be measuring its own choice of tasks.
What is held constant
- The same six tasks, written before any trial, with acceptance criteria fixed
at the same time.
- One model and one Claude Code version, both recorded per trial rather than
described in prose.
- A fresh working directory per trial, with no context carried between them.
- **No
CLAUDE.md** unless a task specifies one. Supplying a good one is exactly
the intervention this benchmark should not silently make on its own behalf.
Only the platform varies, along with what the platform forces to differ: the starter project, the build command, the deploy target.
The tasks
Six, chosen because each has an objective pass condition rather than an aesthetic one.
| Task | What it is really testing | |
|---|---|---|
| T-01 | Ship a page that is actually indexable | Whether the content reaches the initial HTML, not just the browser |
| T-02 | A form that cannot silently fail | Server-side validation, and an error a person can see |
| T-03 | Make an existing page keyboard-operable | Focus management, without breaking mouse use |
| T-04 | Fix a layout shift | Whether the fix survives a second viewport |
| T-05 | Add structured data that is not a lie | Whether it invents AggregateRating for a page with no reviews |
| T-06 | Deploy it | Whether the pipeline has been seen to fail |
The last criterion on T-06 is the point of that task. A pipeline nobody has watched go red is not a pipeline, and "I set up CI" is not the same claim as "I demonstrated CI catching a broken commit."
T-05 is the one worth watching. It gives the model a page with nothing to review and sees whether it fills the gap with plausible-looking fields anyway.
Three runs per cell, minimum
Agentic output varies between identical prompts. One run per platform per task would produce a table that looks authoritative and is mostly noise.
Where three runs disagree, the disagreement is reported as a range. It is never collapsed into a mean with the spread thrown away — the spread is frequently the most useful thing in the row.
Six tasks × five platforms × three runs is 90 trials. That is the smallest honest version of this.
What it cannot measure
- Whether the code is good. Lint counts and Lighthouse scores are proxies,
and weak ones. A passing build says nothing about whether the abstraction was right.
- Long-horizon maintenance. Every task here is short. The failure that costs
most in real projects — a decision that turns out wrong six months later — is invisible at this timescale.
- Accessibility. Automated checks catch a minority of WCAG failures. A clean
axe run is not an accessible site and will not be described as one here.
- The counterfactual. There is no human control arm. This measures how the
tool behaves across platforms, not whether using it beat not using it.
The conflict of interest
This is published by a store that sells Claude Code workflows. That is a real conflict, and saying so does not remove it.
Two things partially offset it. The raw dataset is published, so anyone can recompute the summary and get a different answer if the summary is wrong. And the harness is in the public repository, so the collection can be repeated by someone with no stake in the outcome.
If a result here favours the products sold on this site, treat it as the least reliable result in the set.
Reproducing it
Everything is in the public repository under research/benchmark/: the methodology, the task list with acceptance criteria, the schema field by field, and the dataset. The harness is scripts/benchmark-harness.py.
The harness records trials. It does not run Claude Code and it does not score anything — a harness that both produced and graded the work would be marking its own homework.
scripts/benchmark-harness.py --self-test
scripts/benchmark-harness.py init --platform astro --task T-01 --run 1 \
--model <model-id> --tool-version "$(claude --version)"The dataset
The raw data is one CSV, in the public repository, and it is the file every figure on this page is generated from:
research/benchmark/results.csv · browse the directory
It currently contains a header row and no data. That is not an oversight; see the top of this page.
Licensed CC BY 4.0. Use it, recompute it, and disagree with the summary — if your recomputation gives a different answer, the summary is wrong and I would like to know.
Citing this
Joyner, J. (2026). *Claude Code Website Development Benchmark 2026.* Site Builder Stack. https://sitebuilderstack.com/pages/claude-code-website-development-benchmark-2026
Cite the date tested, the model id and the Claude Code version alongside it. All three are printed with any results, all three will be stale within months, and a benchmark quoted without them is folklore.
Update history
- September 2026 — methodology, tasks, schema and harness published. No
trials run.