cohere-ai — Cohere's Training Crawler
cohere-ai collects training data for Cohere's language models. Learn how to block it in robots.txt.
QUICK FACTS
cohere-ai What is cohere-ai?
cohere-ai is the web crawler operated by Cohere, a Canadian AI company building enterprise-focused language models. The crawler collects web content for training Cohere's Command and Embed model families.
How to Block cohere-ai
Add the following to your robots.txt file (located at the root of your website):
User-agent: cohere-ai Disallow: /
What Happens When You Block cohere-ai
Your content will not be used for Cohere model training.
Should You Block cohere-ai?
cohere-ai is a training crawler — it collects data to build AI models. If you want to prevent your content from being used in future AI training by Cohere, block it. This is a one-way decision: blocking today only affects future crawls, not data already collected.
cohere-ai vs Other Cohere Crawlers
Cohere currently operates cohere-ai as a standalone crawler. Unlike companies like OpenAI and Anthropic that split functionality across multiple user-agents, Cohere uses a single identifier for its AI crawling operations.
GENERATE YOUR ROBOTS.TXT
Use our visual generator to create a robots.txt file that blocks cohere-ai and any other crawlers you want to opt out of.