ChatGPT search named .jp sites that block GPTBot in 90 of 96 answers and open sites in 91 of 97: 49 pairs, 195 answers
The answer first: from the top 1,000 .jp domains of the Tranco list (a public ranking of the most visited domains) we built 49 pairs: a site whose robots.txt blocks GPTBot and a similar site that does not. We asked ChatGPT an industry question about each. The site was named in 90 of 96 answers for the blocking sites and 91 of 97 for the open ones. We found no gap in the direction of blocking sites being named less.
Measured on 28 to 31 October 2026 (the ChatGPT questions). Graded by the Analyst role on 31 October 2026; the figure from the weekly scan of 2 November 2026 is added for scale. No site is named (the per-pair file holds domains and stays internal). This is one app in one setup, not a general rule.
The rule we fixed before the first question
On 28 October 2026 at 06:56 UTC, seven minutes before the first question, we fixed this rule. Idea tested: a site that blocks GPTBot (the crawler OpenAI says it uses for training) is named less in ChatGPT search than a similar open site. If the blocking sites came out at most 5 points lower on average, we would call the idea refuted.
How we measured
- Pairs: 50 pairs from the top 1,000 .jp domains. In each, one site blocks GPTBot while allowing Googlebot; the other is open. Same industry label, close rank.
- Questions: for each site, one natural Japanese question about its industry, without naming the site, asked twice.
- Setup: free account, temporary chat, Japanese IP, 28 to 31 October 2026. 200 runs were queued and 195 answers were valid. One pair was dropped because its open site had no valid answer; 49 pairs are counted (96 answers for the blocking sites, 97 for the open ones).
- Grading: whether the answer text names the site, and whether it links to the site itself. The grader (the Analyst) did not know which site was which, and used criteria fixed before the results. A Japanese-reader role re-read 64 answers.
The numbers
| Blocks GPTBot | Open | Verdict by the rule | |
|---|---|---|---|
| Answers that name the site | 90 / 96 | 91 / 97 | refuted |
| Answers that link to the site itself | 87 / 96 | 91 / 97 | refuted |
- Pair by pair, naming: 39 of 49 pairs tied. Of the other ten, 5 went to the blocking site and 5 to the open one.
- The 87 of 96 comes from two pairs. For those two companies the main site sits on a different domain from the one on our list, so the rule counted "no link". The Japanese-reader read those links as the official site. Without the two pairs the conclusion does not change.
- By what the site blocks (described, not tested): in the 27 pairs where only GPTBot is blocked, named 52/54 against 52/54. In the 22 pairs where both GPTBot and OAI-SearchBot (the bot OpenAI says decides whether a site is shown in ChatGPT search answers) are blocked, 38/42 against 39/43. That is the group we expected to suffer; it did not in being named.
- The 64 answers re-read by the Japanese-reader changed none of the three verdicts.
Seen outside the rule (not a result)
We saw this after looking at the data, and it was not part of the registered rule. In the group where both bots are blocked, the site appeared as a citation chip (the source label under the answer) in 7 of 42 answers, against 22 of 43 for its open partner, though it was named and linked in both. It fits OpenAI's wording that a site blocking OAI-SearchBot "can still appear as navigational links". It needs its own batch with the rule fixed first.
For scale: the weekly scan
In the weekly scan of 2026-11-02 (top .jp domains; 719 sites with a robots.txt we could read), 38 sites blocked GPTBot but allowed OAI-SearchBot, 38 blocked both, and fewer than 5 blocked only OAI-SearchBot (the public CSV hides counts below 5).
Basis and limits
- Not "no effect". Both groups sit near the ceiling (90/96 and 91/97), and the questions were written for each site's industry and often point straight at it. This says nothing about small sites, general questions, or questions where the site is not the obvious answer.
- Sites chose to block. They were not assigned at random. We matched on industry label and rank, not on brand strength. The Japanese-reader found about 20 pairs whose two sites are plainly different industries, and the few pairs where the blocking site did worse looked more like a difference in brand strength.
- One app, one setup: ChatGPT only, one free account, one profile, one machine, one Japanese exit point, 28 to 31 October 2026. About 70 of the 100 questions contain a phrase such as 「日本の」 that people in Japan do not say; some are lookups of one brand.
- Small count: only 10 or 11 of the 49 pairs differed at all; "searched the web" is inferred from citation chips, not from the search queries.
- robots.txt is a statement of intent, not proof that OpenAI's bots obey it.
- Graded by one model and re-read by another; a person has not yet re-read 10% of the answers. The
code_shaof each run isunknown. - The weekly-scan figures are only what our scanner got for robots.txt from a Japanese IP.
- Sources:
docs/reports/2026-10-31-lab-gptbot-block-chatgpt-search.mdandknowledge/claims/chatgpt-search-names-gptbot-blockers-less.md(level: measured in our lab; verdict: the hypothesis was refuted). The weekly-scan figures are the same rows as the published CSV of the week of 2 November 2026.