Original data · checked 2026-09-08
How AI-ready are the web's biggest WordPress sites?
We took the Tranco top 25,000 domains, kept the 2,080 that provably run WordPress, and checked each one live for llms.txt, AI-crawler rules in robots.txt, and schema.org markup.
The short version
- 29% (597) publish an llms.txt file.
- 81% (1,694) say nothing about AI crawlers in robots.txt, leaving the decision to each crawler's defaults.
- 17% (345) block at least one AI crawler. Only 2% (41) explicitly welcome them. See the robots.txt AI crawler guide.
- 84% carry schema.org structured data.
- 0 expose a WebMCP manifest. The standard is still a Chrome origin trial.
- Average score: 38.3/100. Only 25 sites earned an A.
Grade distribution
| Grade | Sites | Share |
|---|---|---|
| A | 25 | 1% |
| B | 438 | 21% |
| C | 1,113 | 54% |
| D | 504 | 24% |
robots.txt and AI crawlers
We classify each site's robots.txt in one of three ways: it blocks at least one named AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot or anthropic-ai), it explicitly allows them, or it is silent.
| Stance | Sites | Share |
|---|---|---|
| Silent | 1,694 | 81% |
| Blocks some | 345 | 17% |
| Explicitly allows | 41 | 2% |
Most sites have not made a decision either way. Of the 2,080 sites, 26 both publish llms.txt and explicitly welcome AI crawlers: the clearest signal a site can send today.
Most common schema.org types
| Type | Sites |
|---|---|
| WebSite | 1,582 (76%) |
| WebPage | 1,134 (55%) |
| Organization | 1,133 (54%) |
| BreadcrumbList | 1,078 (52%) |
| ImageObject | 588 (28%) |
| CollectionPage | 368 (18%) |
| Person | 201 (10%) |
| Article | 124 (6%) |
Structured data is the one area where WordPress is already strong, largely because SEO plugins emit it by default. Very few sites mark up what they actually sell or publish beyond the basics (Article on 6%).
Method
Candidates: the Tranco top 25,000 domains. A site counted as WordPress only with a live same-origin signature (wp-json REST root, a generator meta tag, or same-host wp-content/wp-includes assets). Each was then scored with the same checks as the free scanner, described on the methodology page. Checked 2026-09-08. Results are a snapshot; sites change.
Full data, free to reuse with attribution (CC BY 4.0): cold-start-2026-full.json. Browse every site: the full report.
What to do about it
- Decide your AI-crawler policy and say it in robots.txt. Silence is the most common answer and the least useful. Copy-paste rules here.
- Publish an llms.txt. It takes ten minutes and 29% of big sites already have.
- Check your own site, free and instant, below.
How does your site compare?
Scan it against the same checks used for these 2,080 sites.
Scan your site