What is the difference between OAI-SearchBot, GPTBot and ChatGPT-User?
They do three different jobs: one indexes pages for search, one gathers pages for training, and one fetches a page for a person who asked.
- What it is for
- OAI-SearchBot: ChatGPT's search answers
- GPTBot: Training OpenAI's models
- ChatGPT-User: A person's request in ChatGPT or a custom GPT
- Crawls on its own
- OAI-SearchBot: Yes
- GPTBot: Yes
- ChatGPT-User: No, only when a person asks
- If you block it
- OAI-SearchBot: Your site is not shown in ChatGPT search answers
- GPTBot: Your content is not used for training
- ChatGPT-User: A person asking ChatGPT to read your page may get nothing
- Follows robots.txt
- OAI-SearchBot: Yes
- GPTBot: Yes
- ChatGPT-User: OpenAI says the rules may not apply, because a person started the request
- Published IP list
- OAI-SearchBot: openai.com/searchbot.json
- GPTBot: openai.com/gptbot.json
- ChatGPT-User: openai.com/chatgpt-user.json
| Item | OAI-SearchBot | GPTBot | ChatGPT-User |
|---|---|---|---|
| What it is for | ChatGPT's search answers | Training OpenAI's models | A person's request in ChatGPT or a custom GPT |
| Crawls on its own | Yes | Yes | No, only when a person asks |
| If you block it | Your site is not shown in ChatGPT search answers | Your content is not used for training | A person asking ChatGPT to read your page may get nothing |
| Follows robots.txt | Yes | Yes | OpenAI says the rules may not apply, because a person started the request |
| Published IP list | openai.com/searchbot.json | openai.com/gptbot.json | openai.com/chatgpt-user.json |
All of this comes from OpenAI's own crawler page, read on 18 November 2026. OpenAI says each setting is independent of the others.
Which one decides whether ChatGPT can recommend my site?
OAI-SearchBot does, and it is the only one of the three that matters for showing up in ChatGPT search.
OpenAI's wording is plain: sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. It recommends allowing OAI-SearchBot in robots.txt and allowing requests from its published IP ranges. ChatGPT-User is not used to decide whether content appears in search, and GPTBot has nothing to do with it either.
Can I block GPTBot and still show up in ChatGPT?
Yes. Blocking GPTBot only says your content should not be used for training.
OpenAI gives this exact example: a site can allow OAI-SearchBot to appear in search results while disallowing GPTBot. This robots.txt does that:
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /One detail worth knowing: if you allow both, OpenAI says it may use the results of a single crawl for both purposes, to avoid crawling your site twice.
What are the user-agent strings?
Each crawler names itself in the user-agent, so you can find it in your server logs by searching for its name.
- OAI-SearchBot: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
- GPTBot: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
- ChatGPT-User: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot
OpenAI notes that the version numbers may change, so match on the name (for example "OAI-SearchBot"), not the whole string. When either crawler fetches your robots.txt file it may add a "robots.txt" marker to the string, which makes those requests easy to tell apart in logs that do not record the path.
A user-agent can be faked. To be sure a request is really from OpenAI, check its IP address against the published list for that crawler.
Does ChatGPT-User obey robots.txt?
Not reliably, and OpenAI says so.
ChatGPT-User is not a crawler in the usual sense. It visits a page when a person asks ChatGPT or a custom GPT a question that needs it. OpenAI's page says that because these actions are started by a user, robots.txt rules may not apply. If you need to keep a page away from it, robots.txt is not enough: block it at your server or firewall using the published IP list.
For most companies the opposite problem is more common. A firewall or bot-protection setting quietly blocks ChatGPT-User, and a customer who pastes your link into ChatGPT gets an answer that says the page could not be read.
What is OAI-AdsBot?
It is a fourth agent that checks the landing pages of ads submitted to ChatGPT.
OpenAI says OAI-AdsBot visits only pages submitted as ads, to check them against its policies and to work out when the ad is relevant, and that what it collects is not used for training. If you do not advertise on ChatGPT you will not see it.
How long does a robots.txt change take to work?
About a day for search.
OpenAI says it can take around 24 hours from a robots.txt update for its search systems to adjust. That covers whether you are allowed to appear. Changes to what your pages say take longer to show up in answers, because the pages have to be crawled again.
How do I check that my site lets them in?
Look in three places: robots.txt, your firewall, and your logs.
- Open yoursite.com/robots.txt and look for a group that names OAI-SearchBot, and for a "User-agent: *" group with "Disallow: /". Either can shut the crawler out.
- Check your CDN or bot protection. Many block AI crawlers with one switch, and that switch overrides whatever robots.txt says.
- Search your server logs for "OAI-SearchBot". Requests that got a 200 response mean it is reading your pages. A 403 means something is blocking it.
The free Kortex audit checks AI crawler access for you, along with what ChatGPT, Gemini and Claude answer when your customers ask about what you sell. For the steps after the crawlers are in, see how to get recommended by ChatGPT.
Where do the other AI companies' crawlers fit?
Anthropic and Perplexity split their bots the same way, and Google uses a robots.txt name with no crawler behind it.
- ClaudeBot, Claude-SearchBot and Claude-User
- PerplexityBot and Perplexity-User
- Google-Extended
- One file that covers all of them: robots.txt for AI crawlers