Arrow 
Allow OAI-SearchBot if you want your public pages considered for ChatGPT Search. Decide separately whether to allow GPTBot for potential model training. ChatGPT-User handles certain user-initiated visits; allowing it does not replace the search crawler.
Choose by purpose, then verify delivery
A request containing “GPT” in its name is not enough to tell you why it arrived. Keep a record of the crawler, the policy you intend and the response it actually receives. This comparison is specific to OpenAI; other providers have their own controls.
What is the difference between the three agents?
| Agent | Documented role | Decision for a public source page |
|---|---|---|
| OAI-SearchBot | Discovery for ChatGPT search features | Allow if search visibility is the objective. |
| GPTBot | Crawls content that may be used for foundation-model training | Make an independent training-policy decision. |
| ChatGPT-User | Certain visits initiated by a ChatGPT or Custom GPT user | Assess user access separately from automatic search crawling. |
OpenAI’s crawler documentation distinguishes these uses. Its search opt-out can still leave navigational links, and robots.txt rules may not apply to user-initiated ChatGPT-User actions. None of these identities is an authentication mechanism.
Should you allow OAI-SearchBot and block GPTBot?
That is a coherent policy when you want search discovery while declining potential training collection through GPTBot. It is not a requirement to allow both. Write the business decision first so a later CDN change does not accidentally reverse it.
For an illustrative software publisher, public integration documentation can remain available to search while account pages stay authenticated. The training preference applies independently. A crawler allow rule does not make an account page public, and a robots disallow rule does not secure it.
Use the search-without-training configuration guide for a complete example with repeated path restrictions.
How do you know a request is really OpenAI?
Check the actual connecting IP against the current provider-published ranges for that specific agent. A user-agent string is easy to copy. If a reverse proxy sits in front of the origin, use its trusted client-IP handling rather than accepting an arbitrary forwarded header.
Compare the search, GPTBot and ChatGPT-User lists separately. Retain the check date, matched range and request evidence. A local curl request with a copied name tests name-based behavior only; it cannot impersonate the provider’s network.
Why can an allowed crawler still fail?
A robots permission and a successful fetch are different checks. A WAF, rate limit, redirect, login requirement or challenge page can interrupt delivery. Inspect a public article and robots.txt with GET requests, including the response body. A successful status containing a challenge is not the article.
If Cloudflare is involved, follow the 403 and challenge diagnostic and match the request to the responsible security rule.
What proves that the change worked?
First establish the narrow outcome: a verified request reached the expected public content after the policy change. Then observe indexing and citations separately under recorded questions, markets and product surfaces. An allowed crawl does not guarantee inclusion or a citation.
Keep the response and source URL using the ChatGPT citation-tracking worksheet. Do not treat a crawler log entry as a search impression.
Sources and editorial scope
Reviewed October 21, 2026 against the linked provider documentation. Procedures and examples are Arrow AI implementation guidance, not measured customer results. Policies and dashboard controls can change. Access does not guarantee indexing or citation.
The Arrow read
Make public sources reachable and verifiable.
Start with the crawler decision guide or run a free GEO audit to identify the next page worth improving.