What is GPTBot?
GPTBot · OpenAI · Training crawler
GPTBot is OpenAI's crawler. It collects content that may be used to train AI models. Collects content for OpenAI model training. Blocking it does not remove you from ChatGPT answers.
What it does
GPTBot collects page content to be used in model training. Collects content for OpenAI model training. Blocking it does not remove you from ChatGPT answers.
It does not execute JavaScript. Whatever your server returns in the initial HTML response is the entire page as far as GPTBot is concerned — if your content is rendered client-side, this crawler reads an empty container.
Should you block GPTBot?
It is a licensing decision, not a visibility one. GPTBot collects content that may be used to train AI models; blocking it does not remove you from Google Search or from AI answer engines.
Your content stops being collected for future OpenAI model training. Nothing changes about how you appear in ChatGPT today, because ChatGPT answers about current events are retrieved live, not recalled from training. This is the crawler people block when they mean "do not train on my work", and blocking it for that reason is coherent.
To allow it
User-agent: GPTBot
Allow: /To block it
User-agent: GPTBot
Disallow: /Put the rule in your robots.txt at the site root. A group that names GPTBot explicitly takes precedence over your User-agent: * group, so naming it is how you make your intent survive later edits to the wildcard rules.
Verifying a real GPTBot request
OpenAI publishes the IP ranges its crawlers operate from as JSON files on its own domain. Check a request's source IP against that list before trusting the user agent string — the string itself is trivially spoofable and commonly is, by scrapers hoping your firewall has an allow rule for it.
The mistake people make
Blocking GPTBot and assuming that removes you from ChatGPT. It does not. If that was your goal you blocked the wrong bot, and if it was not your goal you may have given up training-data inclusion for nothing.
OpenAI's other crawlers
- OAI-SearchBotBuilds the index behind ChatGPT search results. Blocking it removes you from ChatGPT search.
- ChatGPT-UserFetches a page when a ChatGPT user follows or asks about a specific link.
Frequently asked questions
- What is GPTBot?
- GPTBot is OpenAI's crawler. It collects content that may be used to train AI models. Collects content for OpenAI model training. Blocking it does not remove you from ChatGPT answers.
- Should I block GPTBot?
- It is a licensing decision, not a visibility one. GPTBot collects content that may be used to train AI models; blocking it does not remove you from Google Search or from AI answer engines.
- How do I block GPTBot in robots.txt?
- Add a group to the robots.txt at your site root: "User-agent: GPTBot" followed by "Disallow: /". Use "Allow: /" instead to allow it explicitly.
Further reading
- The Complete Guide to robots.txt for AI Crawlers: Which Bots to Allow
GPTBot vs OAI-SearchBot, ClaudeBot vs Claude-User, Googlebot vs Google-Extended: which AI crawlers to allow, which to block, and copy-ready robots.txt templates.
- How to Create and Optimize an llms.txt File for Your Website
The llms.txt format, where the file goes, how to keep it token-efficient, working examples for Next.js and static sites, and an honest account of who uses it.
- What is Generative Engine Optimization (GEO)? The Complete 2026 Guide
How ChatGPT Search, Perplexity and Claude find, read and cite pages, why that differs from Googlebot, and what the research says actually improves AI visibility.
Test your site against 15 AI crawlers now
The free scan parses your robots.txt, evaluates every AI crawler against it, and tells you how much of a page GPTBot would actually come away with.
Run the free scan