What is Claude-User?
Claude-User · Anthropic · User-action fetcher
Claude-User is Anthropic's crawler. It fetches a page when a person asks the assistant about it. Fetches a page on behalf of a Claude user following a link.
What it does
Claude-User fetches a page because a user asked about that specific link. Fetches a page on behalf of a Claude user following a link.
It does not execute JavaScript. Whatever your server returns in the initial HTML response is the entire page as far as Claude-User is concerned — if your content is rendered client-side, this crawler reads an empty container.
Should you block Claude-User?
Only if you want out of Anthropic's AI answers. Claude-User fetches a page when a person asks the assistant about it, so blocking it means your pages stop being available as a source there.
A user who shares your link with Claude gets told the page could not be fetched. If your business involves people sharing your documentation, pricing or research with an assistant, this is a direct loss.
To allow it
User-agent: Claude-User
Allow: /To block it
User-agent: Claude-User
Disallow: /Put the rule in your robots.txt at the site root. A group that names Claude-User explicitly takes precedence over your User-agent: * group, so naming it is how you make your intent survive later edits to the wildcard rules.
Verifying a real Claude-User request
Verify against Anthropic's published IP ranges. Like other user-action fetchers, it produces sporadic single requests rather than a crawl pattern, so traffic-shape heuristics tend to misread it.
The mistake people make
Bot-protection services classifying it as an unknown automated client and serving a challenge page. The challenge is what gets read, and the answer describes your CAPTCHA rather than your product.
Anthropic's other crawlers
- ClaudeBotCollects content for Anthropic model training.
- Claude-SearchBotIndexes pages to support Claude search results.
Frequently asked questions
- What is Claude-User?
- Claude-User is Anthropic's crawler. It fetches a page when a person asks the assistant about it. Fetches a page on behalf of a Claude user following a link.
- Should I block Claude-User?
- Only if you want out of Anthropic's AI answers. Claude-User fetches a page when a person asks the assistant about it, so blocking it means your pages stop being available as a source there.
- How do I block Claude-User in robots.txt?
- Add a group to the robots.txt at your site root: "User-agent: Claude-User" followed by "Disallow: /". Use "Allow: /" instead to allow it explicitly.
Further reading
- The Complete Guide to robots.txt for AI Crawlers: Which Bots to Allow
GPTBot vs OAI-SearchBot, ClaudeBot vs Claude-User, Googlebot vs Google-Extended: which AI crawlers to allow, which to block, and copy-ready robots.txt templates.
- How to Create and Optimize an llms.txt File for Your Website
The llms.txt format, where the file goes, how to keep it token-efficient, working examples for Next.js and static sites, and an honest account of who uses it.
- What is Generative Engine Optimization (GEO)? The Complete 2026 Guide
How ChatGPT Search, Perplexity and Claude find, read and cite pages, why that differs from Googlebot, and what the research says actually improves AI visibility.
Test your site against 15 AI crawlers now
The free scan parses your robots.txt, evaluates every AI crawler against it, and tells you how much of a page Claude-User would actually come away with.
Run the free scan