Claude-User — Anthropic's Live Fetch Agent

What is Claude-User?

Claude-User activates when a Claude user asks the assistant to read or browse a web page. It is a user-triggered fetcher, not a training crawler. The content fetched is used to answer the user's question in that conversation and is not stored for model training. Blocking Claude-User reduces your visibility when users ask Claude to look up information.

How to Block Claude-User

Add the following to your robots.txt file (located at the root of your website):

User-agent: Claude-User
Disallow: /

What Happens When You Block Claude-User

Claude will not fetch your pages when users ask it to browse a URL. Training data collection (ClaudeBot) and search indexing (Claude-SearchBot) are not affected.

Should You Block Claude-User?

Claude-User is a user-triggered fetcher — it only activates when a real human asks the AI to read your page. It does not autonomously crawl your site. Blocking it prevents the AI from citing your content when users explicitly request it, which may reduce your referral visibility from Anthropic's products.

Claude-User vs Other Anthropic Crawlers

Anthropic operates multiple crawlers, each serving a different purpose:

Each crawler operates independently. Blocking Claude-User does not block ClaudeBot or Claude-SearchBot or anthropic-ai — you must add a separate rule for each.