cohere-ai
Cohere · Training crawler
What it does
cohere-ai collects page content to be used in model training. Collects content for Cohere model training.
It does not execute JavaScript. Whatever your server returns in the initial HTML response is the entire page as far as cohere-ai is concerned — if your content is rendered client-side, this crawler reads an empty container.
Should you block cohere-ai?
Blocking it keeps your content out of the training corpus. It does not remove you from any answer surface, because this crawler is not what answers are built from.
Your content is excluded from Cohere's training data collection. Cohere sells models to enterprises rather than running a consumer answer engine, so there is no citation surface at stake.
To allow it
User-agent: cohere-ai
Allow: /To block it
User-agent: cohere-ai
Disallow: /Put the rule in your robots.txt at the site root. A group that names cohere-ai explicitly takes precedence over your User-agent: * group, so naming it is how you make your intent survive later edits to the wildcard rules.
Verifying a real cohere-ai request
Cohere's crawler is comparatively low-volume. If you are seeing high request rates under this name, treat the user agent as unverified and check the source addresses.
The mistake people make
Leaving it out of a robots.txt that otherwise names every AI crawler. If your position on training data is a policy, it should be stated consistently, including for the vendors with no consumer product.
Can cohere-ai reach your site?
The free scan parses your robots.txt, evaluates every AI crawler against it, and tells you how much of a page cohere-ai would actually come away with.
Run the free scan