AI crawler · Perplexity

PerplexityBot

Perplexity's crawler for surfacing and linking websites in its search results.

Operator
Perplexity
Feeds
Perplexity search results
Type
Live (answers users now)
robots.txt token
PerplexityBot

Full user-agent string, as Perplexity documents it (version numbers can change):

Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

What PerplexityBot does

Perplexity says PerplexityBot surfaces and links websites in Perplexity search results, and is not used to crawl content for AI foundation models.

Does it follow robots.txt? Yes, according to Perplexity's documentation. Perplexity says robots.txt changes can take up to 24 hours to apply.

How many popular sites block PerplexityBot

13.5% of the Tranco top 500 sites whose robots.txt answered block PerplexityBot for the whole site (43 of 319), as of . The number comes from our weekly AI crawler access study, which counts a site as blocking only when its robots.txt disallows the site root for this token.

robots.txt: allow or block PerplexityBot

Put the group in the robots.txt at the root of every host it should cover (example.com and www.example.com each need their own).

Allow

User-agent: PerplexityBot
Allow: /

Block

User-agent: PerplexityBot
Disallow: /

Setting rules for several AI crawlers at once? The free AI robots.txt generator writes the groups for all of them.

A crawler follows only the most specific group that names it. Once you add a User-agent: PerplexityBot group, it ignores your User-agent: * rules, so repeat any Disallow lines (admin areas, carts) you still want applied to it.

What blocking does and does not do

  • Does: It asks Perplexity not to crawl your site for its search results, which reduces the chance you are linked as a source.
  • Does not: It does not cover Perplexity-User, which Perplexity says generally ignores robots.txt for user-requested fetches.

Anyone can put "PerplexityBot" in a user-agent. To confirm a visit is genuine, compare its IP with Perplexity's published list: https://www.perplexity.ai/perplexitybot.json.

Check whether your site blocks PerplexityBot

Enter your page and our free crawler check opens with it filled in. It reads your robots.txt for PerplexityBot and the other nine AI crawler tokens, then requests the page as PerplexityBot to catch a server or CDN refusal. No signup.

Frequently asked questions

How do I block PerplexityBot in robots.txt?

Add a group that names it: "User-agent: PerplexityBot" on one line and "Disallow: /" on the next, in the robots.txt at the root of each host you want covered.

Does blocking PerplexityBot remove my site from AI answers?

It asks Perplexity not to crawl your site for its search results, which reduces the chance you are linked as a source. It does not cover Perplexity-User, which Perplexity says generally ignores robots.txt for user-requested fetches.

Is PerplexityBot a live crawler or a training crawler?

Live: it fetches pages to answer users now, which is the path to being cited. Does it follow robots.txt? Yes, according to Perplexity's documentation. Perplexity says robots.txt changes can take up to 24 hours to apply.

How can I check whether my site blocks PerplexityBot?

Run the free SEO Sets AI crawler check. It reads your robots.txt rules for this and nine other AI crawler tokens, and for real user agents it also requests your page with that name to see whether your server or CDN refuses it.

Official documentation

Perplexity: Perplexity Crawlers. Facts on this page were read from it on November 17, 2026. Vendors change their crawlers; the vendor's page wins if it disagrees with ours.

Other Perplexity crawlers: Perplexity-User · All AI crawlers

See what to fix on your own site

The free Signal plan connects your Google Search Console and shows your top 3 pages with the most search upside, with no time limit and no credit card. AI citation tracking starts on Surge at $49 a month.

SEO Sets Support

Press Enter to send · Shift+Enter for new line