Google-Extended
Google · Training crawler
What it does
Google-Extended collects page content to be used in model training. Controls use of your content for Gemini training. Does not affect Google Search ranking.
It runs on rendering infrastructure, so client-side content is generally visible to it. That makes it the exception rather than the rule among AI crawlers.
Should you block Google-Extended?
Blocking it keeps your content out of the training corpus. It does not remove you from any answer surface, because this crawler is not what answers are built from.
Your content stops being used for Gemini model training and for grounding in Google's generative products. Your Google Search ranking is entirely unaffected — this token does not touch Search at all.
To allow it
User-agent: Google-Extended
Allow: /To block it
User-agent: Google-Extended
Disallow: /Put the rule in your robots.txt at the site root. A group that names Google-Extended explicitly takes precedence over your User-agent: * group, so naming it is how you make your intent survive later edits to the wildcard rules.
Verifying a real Google-Extended request
Google-Extended is not a crawler with its own user agent making requests. It is a robots.txt token that controls how content already fetched by Google's standard crawlers may be used. There is no request to verify, which makes it unique on this list.
The mistake people make
Expecting to see Google-Extended in your server logs. You will not, because nothing requests pages under that name. People spend real time trying to confirm it is working by looking for traffic that does not exist.
Can Google-Extended reach your site?
The free scan parses your robots.txt, evaluates every AI crawler against it, and tells you how much of a page Google-Extended would actually come away with.
Run the free scan