AI crawler · OpenAI
OAI-SearchBot
The crawler that decides whether your pages can appear as sources in ChatGPT search answers.
- Operator
- OpenAI
- Feeds
- ChatGPT search
- Type
- Live (answers users now)
- robots.txt token
- OAI-SearchBot
Full user-agent string, as OpenAI documents it (version numbers can change):
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
What OAI-SearchBot does
OpenAI uses OAI-SearchBot to find and surface websites in ChatGPT's search features. It is not used to train OpenAI's models.
Does it follow robots.txt? Yes. OpenAI documents that OAI-SearchBot follows robots.txt, and says search-related robots.txt changes can take about 24 hours to take effect.
How many popular sites block OAI-SearchBot
10.3% of the Tranco top 500 sites whose robots.txt answered block OAI-SearchBot for the whole site (33 of 319), as of . The number comes from our weekly AI crawler access study, which counts a site as blocking only when its robots.txt disallows the site root for this token.
robots.txt: allow or block OAI-SearchBot
Put the group in the robots.txt at the root of every host it should cover
(example.com and www.example.com each need their own).
Allow
User-agent: OAI-SearchBot Allow: /
Block
User-agent: OAI-SearchBot Disallow: /
Setting rules for several AI crawlers at once? The free AI robots.txt generator writes the groups for all of them.
User-agent: OAI-SearchBot group, it ignores your User-agent: * rules, so repeat
any Disallow lines (admin areas, carts) you still want applied to it.
What blocking does and does not do
- Does: Per OpenAI, a site that blocks OAI-SearchBot will not be shown in ChatGPT search answers.
- Does not: It does not stop GPTBot (training) or ChatGPT-User (fetches a user asks for). OpenAI notes a blocked site can still appear as a plain navigational link.
- If you want to be cited in ChatGPT search but keep your content out of training, allow OAI-SearchBot and block GPTBot. OpenAI treats the two settings independently.
Anyone can put "OAI-SearchBot" in a user-agent. To confirm a visit is genuine, compare its IP with OpenAI's published list: https://openai.com/searchbot.json.
Check whether your site blocks OAI-SearchBot
Enter your page and our free crawler check opens with it filled in. It reads your robots.txt for OAI-SearchBot and the other nine AI crawler tokens, then requests the page as OAI-SearchBot to catch a server or CDN refusal. No signup.
Frequently asked questions
How do I block OAI-SearchBot in robots.txt?
Add a group that names it: "User-agent: OAI-SearchBot" on one line and "Disallow: /" on the next, in the robots.txt at the root of each host you want covered.
Does blocking OAI-SearchBot remove my site from AI answers?
Per OpenAI, a site that blocks OAI-SearchBot will not be shown in ChatGPT search answers. It does not stop GPTBot (training) or ChatGPT-User (fetches a user asks for). OpenAI notes a blocked site can still appear as a plain navigational link.
Is OAI-SearchBot a live crawler or a training crawler?
Live: it fetches pages to answer users now, which is the path to being cited. Does it follow robots.txt? Yes. OpenAI documents that OAI-SearchBot follows robots.txt, and says search-related robots.txt changes can take about 24 hours to take effect.
How can I check whether my site blocks OAI-SearchBot?
Run the free SEO Sets AI crawler check. It reads your robots.txt rules for this and nine other AI crawler tokens, and for real user agents it also requests your page with that name to see whether your server or CDN refuses it.
Official documentation
OpenAI: Overview of OpenAI crawlers. Facts on this page were read from it on November 17, 2026. Vendors change their crawlers; the vendor's page wins if it disagrees with ours.
Other OpenAI crawlers: ChatGPT-User, GPTBot · All AI crawlers
Related
- AI crawler checkFree, no signup: is any AI crawler blocked on your site?
- AI robots.txt generatorFree: allow or block each AI crawler and copy the rules.
- AI crawler directoryEvery AI crawler our check reads, with robots.txt rules.
- Claim your business in ChatGPTA first-hand, step-by-step guide.
- Compare SEO SetsDated, sourced comparisons with Otterly and Peec.
See what to fix on your own site
The free Signal plan connects your Google Search Console and shows your top 3 pages with the most search upside, with no time limit and no credit card. AI citation tracking starts on Surge at $49 a month.