Can AI crawlers read WordPress sites?
Server-rendered
AI crawlers can read this platform by default.
WordPress serves complete HTML by default, which puts it near the top for AI readability — the usual problems are structured data and bot blocking, not rendering.
What is actually happening
PHP renders the full page server-side, so the content is in the initial response. What tends to break AI visibility instead is an over-eager security plugin or CDN rule that blocks unfamiliar user agents, catching AI crawlers along with scrapers.
13 of the major AI crawlers — including GPTBot, OAI-SearchBot, ChatGPT-User — fetch your HTML and parse it without running a JavaScript engine. Anything your page adds after hydration is not part of what they read.
What to do on WordPress
- 01Check your security plugin (Wordfence, Sucuri) and CDN bot rules for blanket user-agent blocking — allow OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User and PerplexityBot explicitly.
- 02Use a schema plugin, or your SEO plugin's schema module, to emit Organization and Article JSON-LD.
- 03Serve llms.txt as a real file rather than a rewritten route, so it returns text/markdown rather than the theme.
- 04Turn off any "block AI crawlers" toggle unless you have decided you want that — several plugins ship it on by default now.
What trips people up
- Page builders that render content client-side (some Elementor widgets, most carousels) hide that content from crawlers even on a server-rendered site.
- The default WordPress robots.txt is virtual. Adding a physical file overrides it entirely, including the wp-admin disallow.
- Cloudflare's AI crawler blocking is a one-click setting that silently removes you from AI answers.
Where the files go on WordPress
Upload llms.txt and robots.txt to the web root over FTP or your host's file manager. A plugin-generated virtual robots.txt is overridden by a physical file.
Stop guessing where your site sits
The baseline above is the platform. The free scan measures your actual site: how many words a crawler reads, whether your pages come back as empty shells, and which crawlers your robots.txt lets in.
Scan my WordPress site