PerplexityBot — Perplexity AI's Search Crawler
PerplexityBot indexes pages for Perplexity AI search. Learn how to block or allow it in robots.txt and what changes when you do.
QUICK FACTS
PerplexityBot What is PerplexityBot?
PerplexityBot crawls and indexes web pages so they can appear as cited sources in Perplexity AI's answer engine. Perplexity positions itself as an AI-powered search engine that cites sources directly. Blocking PerplexityBot means your content will not appear in Perplexity search results.
How to Block PerplexityBot
Add the following to your robots.txt file (located at the root of your website):
User-agent: PerplexityBot Disallow: /
What Happens When You Block PerplexityBot
Your pages will not appear in Perplexity AI search results. Perplexity-User (live fetch) may still attempt to access your pages.
Should You Block PerplexityBot?
PerplexityBot is a search indexer — it indexes your content so it can appear in Perplexity AI's AI-powered search results. Blocking it removes your visibility in that product but does not affect traditional search engines (Google, Bing). Consider whether AI search traffic is valuable to you before blocking.
PerplexityBot vs Other Perplexity AI Crawlers
Perplexity AI currently operates PerplexityBot as a standalone crawler. Unlike companies like OpenAI and Anthropic that split functionality across multiple user-agents, Perplexity AI uses a single identifier for its AI crawling operations.
GENERATE YOUR ROBOTS.TXT
Use our visual generator to create a robots.txt file that blocks PerplexityBot and any other crawlers you want to opt out of.