Skip to content
Crawlable

OAI-SearchBot

OpenAI · Search index crawler

Operator
OpenAI
Runs JavaScript
No
Blocking costs visibility
Yes

What it does

OAI-SearchBot builds the index that answers are retrieved from. Builds the index behind ChatGPT search results. Blocking it removes you from ChatGPT search.

It does not execute JavaScript. Whatever your server returns in the initial HTML response is the entire page as far as OAI-SearchBot is concerned — if your content is rendered client-side, this crawler reads an empty container.

Should you block OAI-SearchBot?

Blocking it removes you from the index that answers are drawn from. Your pages stop being available as a source, and you will simply not be cited.

You are removed from the index that ChatGPT search draws on. When a user asks a question your page answers, your page is not a candidate. There is no partial version of this: an unindexed page is not ranked low, it is absent.

To allow it

User-agent: OAI-SearchBot
Allow: /

To block it

User-agent: OAI-SearchBot
Disallow: /

Put the rule in your robots.txt at the site root. A group that names OAI-SearchBot explicitly takes precedence over your User-agent: * group, so naming it is how you make your intent survive later edits to the wildcard rules.

Verifying a real OAI-SearchBot request

OpenAI publishes a separate IP range file for the search crawler, distinct from the GPTBot list. Verify against that one specifically — the ranges are not interchangeable.

The mistake people make

Copying a "block AI scrapers" robots.txt snippet from a blog post. Most of those snippets were written when GPTBot was the only OpenAI crawler and now sweep up the search crawler alongside it, quietly removing the site from ChatGPT search.

OpenAI's other crawlers

Can OAI-SearchBot reach your site?

The free scan parses your robots.txt, evaluates every AI crawler against it, and tells you how much of a page OAI-SearchBot would actually come away with.

Run the free scan