Turn WordPress Content into AI Answer Citations

The platform pitfall: plugin bloat around good HTML

PHP renders the whole post on the server, so the article text is in the raw HTML. Readability is usually fine on WordPress. What goes wrong is everything layered on top.

Schema conflicts. An SEO plugin such as Yoast SEO or Rank Math emits a full graph: WebPage, Article, Organization, WebSite, breadcrumbs. Add a dedicated schema plugin, or a theme with its own markup, and the same post carries two Article entities with different authors, dates or headlines.

Slow first byte. An uncached page that runs dozens of plugin hooks can take seconds to start responding, and crawlers fetch on a timeout. Crawlable’s own fetcher gives up after 12 seconds. A page cache, from your host or a caching plugin, takes that time out of the request.

No /llms.txt. WordPress does not create one, so a request for /llms.txt falls through to the theme’s 404 page. Crawlable counts an HTML page at that address as no file.

Security plugins and CDN bot rules are the other quiet failure: a blanket rule against unfamiliar user agents blocks AI search crawlers along with the scrapers it was meant for.

STEP 1 OF 3

Serve a plain-text /llms.txt from the web root

Upload llms.txt to the folder that holds wp-config.php. WordPress’s rewrite rules only take over when no real file exists at the requested path, so the web server serves your file directly and WordPress never sees the request.

# Example Journal

> Example Journal publishes practical guides to home espresso: grinders, machines, beans and technique, tested in our own kitchen.

## Guides

- [Choosing a burr grinder](https://example.com/burr-grinder-guide/): What grind consistency changes in the cup, and which grinders deliver it.
- [Dialling in espresso](https://example.com/dial-in-espresso/): A step-by-step method for dose, yield and time.

## Reviews

- [Espresso machines under $500](https://example.com/espresso-machines-under-500/): Six machines tested side by side.

## About

- [About the team](https://example.com/about/): Who writes and tests for Example Journal.

# .htaccess in the WordPress root, above "# BEGIN WordPress".
# Serves llms.txt and llms-full.txt as UTF-8 plain text.
<FilesMatch "^llms(-full)?\.txt$">
  ForceType text/plain
  AddDefaultCharset UTF-8
</FilesMatch>

# In your server { } block. Serves the files from disk as UTF-8 plain text.
location ~ ^/llms(-full)?\.txt$ {
    charset utf-8;
    try_files $uri =404;
}

• ✓The Apache or Nginx block only makes the content type and character set explicit. The file is served without it.
• ✓Keep the shape Crawlable validates: one # heading, a > summary, ## sections, absolute links with a description. A well-formed file earns 15 of the 20 llms.txt points; /llms-full.txt beside it adds 5.
• ✓Check it with curl -sI https://example.com/llms.txt. You want a 200 with text/plain or text/markdown, not a redirect and not text/html.

STEP 2 OF 3

Update robots.txt directly, without a plugin

WordPress generates a virtual robots.txt. A physical robots.txt in the web root replaces it entirely, so the file below carries WordPress’s own defaults in the * group alongside the AI crawler groups.

# robots.txt: upload to your WordPress root, next to wp-config.php.

# Search and answer engines: allowed, so posts can be retrieved and cited.
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
Allow: /
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

# Bulk training crawlers: a licensing decision. Change Disallow to Allow to permit them.
# Google-Extended also controls grounding in Gemini Apps and Vertex AI.
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: CCBot
User-agent: Bytespider
User-agent: meta-externalagent
Disallow: /

# Everyone else, including Googlebot and Bingbot: WordPress's own defaults.
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: https://example.com/wp-sitemap.xml

• ✓This file earns the full 20 crawler-governance points: retrieval bots named and allowed, training crawlers in a group of their own.
• ✓Point the Sitemap line at the sitemap you actually serve: /wp-sitemap.xml from core, or /sitemap_index.xml if Yoast or Rank Math generates it.
• ✓Once the physical file exists, robots settings in SEO plugins stop applying. Edit the file instead.
• ✓If your host does not let you write to the root, the robots_txt filter in a must-use plugin can prepend the same groups to the virtual file.

STEP 3 OF 3

Prune duplicate schema into one Article graph

First, count what the raw HTML contains: curl -s https://example.com/your-post/ | grep -o '"@type": *"[A-Za-z]*"' | sort | uniq -c. Two Article or two WebPage entries mean two plugins are writing schema.

The simplest fix is to keep one plugin’s schema and switch it off everywhere else. If Yoast is your SEO plugin, keep its graph and disable schema in the others. If you would rather own the markup, the must-use plugin below turns Yoast’s graph off and writes a single Organization, WebSite and Article graph linked by @id.

<?php
/**
 * Plugin Name: Unified article schema
 * Description: One JSON-LD graph per post. Switch off schema output in every other plugin and the theme first.
 * Save as wp-content/mu-plugins/unified-schema.php (must-use plugins load automatically).
 */

// Yoast SEO: stop its graph so this one is the only one on the page.
add_filter( 'wpseo_json_ld_output', '__return_false' );

add_action( 'wp_head', function () {
	if ( ! is_singular( 'post' ) ) {
		return;
	}

	$post      = get_queried_object();
	$site      = home_url( '/' );
	$permalink = get_permalink( $post );

	$graph = array(
		'@context' => 'https://schema.org',
		'@graph'   => array(
			array(
				'@type' => 'Organization',
				'@id'   => $site . '#organization',
				'name'  => get_bloginfo( 'name' ),
				'url'   => $site,
			),
			array(
				'@type'     => 'WebSite',
				'@id'       => $site . '#website',
				'url'       => $site,
				'name'      => get_bloginfo( 'name' ),
				'publisher' => array( '@id' => $site . '#organization' ),
			),
			array(
				// Use 'TechArticle' for documentation and tutorials.
				'@type'            => 'Article',
				'@id'              => $permalink . '#article',
				'headline'         => get_the_title( $post ),
				'description'      => get_the_excerpt( $post ),
				'datePublished'    => get_the_date( 'c', $post ),
				'dateModified'     => get_the_modified_date( 'c', $post ),
				'author'           => array(
					'@type' => 'Person',
					'name'  => get_the_author_meta( 'display_name', $post->post_author ),
					'url'   => get_author_posts_url( $post->post_author ),
				),
				'publisher'        => array( '@id' => $site . '#organization' ),
				'isPartOf'         => array( '@id' => $site . '#website' ),
				'mainEntityOfPage' => $permalink,
			),
		),
	);

	// JSON_HEX_TAG escapes < and >, so a title containing </script> cannot close the tag.
	echo '<script type="application/ld+json">' . wp_json_encode( $graph, JSON_HEX_TAG | JSON_UNESCAPED_SLASHES ) . "</script>\n";
} );

• ✓The filter only silences Yoast. In Rank Math, switch off the Schema module under Rank Math → Dashboard → Modules; for any other plugin or theme, use its own schema setting.
• ✓An Article counts as complete for Crawlable’s schema pillar when it has an author. That earns the full 25 points.
• ✓Crawlable does not deduct for a duplicate; it checks that one complete entity exists. The count above is how you find duplicates.

Frequently asked questions

Do I need a plugin to add llms.txt to WordPress?

No. Upload a file named llms.txt to the folder that holds wp-config.php. The web server serves real files before WordPress handles the request, so it appears at /llms.txt with no plugin.

Will a physical robots.txt break my SEO plugin?

It replaces the virtual robots.txt that WordPress and SEO plugins generate, so their robots settings stop applying. Copy anything you still need, usually the Sitemap line, into the file.

Which schema plugin should I keep?

One. Keep whichever emits the most complete entities for your content, such as an Article with its author and dates, and switch schema output off in every other plugin and in the theme.

Can AI crawlers read WordPress sites?

Usually, yes. WordPress renders posts to HTML on the server, so the text is in the response. The exceptions are page-builder widgets that load their content with JavaScript, and security or CDN rules that block AI user agents.

More WordPress gotchas

• ◆Page builders that render content client-side (some Elementor widgets, most carousels) hide that content from crawlers even on a server-rendered site.
• ◆The default WordPress robots.txt is virtual. Adding a physical file overrides it entirely, including the wp-admin disallow.
• ◆Cloudflare's AI crawler blocking is a one-click setting that silently removes you from AI answers.

Further reading

• How to Make React Apps Crawlable for AI Search: SSR, Prerendering and Dynamic Rendering

  Google can render React; ChatGPT, Claude and Perplexity crawlers can't. Code for Vite SSR, Next.js and Astro prerendering, dynamic rendering, and a curl check.

• What GPTBot Actually Sees When Your Site Relies on Client-Side JavaScript

  AI crawlers like GPTBot and OAI-SearchBot fetch raw HTML without executing client-side JS. Here is what they actually extract from SPAs.

• Why AI Search Engines Can’t Read Client-Side Rendered (CSR) Sites

  Most AI crawlers fetch raw HTML and never run your JavaScript. Here's what they actually see on a client-rendered site, how to test it, and how to fix it.

• What is Generative Engine Optimization (GEO)? The Complete 2026 Guide

  How ChatGPT Search, Perplexity and Claude find, read and cite pages, why that differs from Googlebot, and what the research says actually improves AI visibility.

All guides on the blog