Lists the 4 AI crawlers affecting the public score and stamps, 6 AEO+ visa-page crawlers and other AI crawlers one by one — whether robots.txt blocks each and on which line — and produces a fixed robots.txt.
First check your website for free, then use this tool from your passport page.
These are measured numbers from IDAEO's own websites, not yours. They show what each company's crawlers actually brought back during this period after reading our sites.
Period: 2026-09-28 to 2026-10-04 (Japan time, 7 days). Source: Cloudflare traffic analytics, retrieved 2026-10-05. Measured sites: idaeo.ai, washinmura.jp
In these 7 days: about 917,548 reads by verified AI-company crawlers, plus about 616,412 reads by verified search-engine crawlers (Googlebot, bingbot, Applebot); 35 clicks back to our sites from AI chat interfaces; 18,239 clicks back from regular Google/Bing search.
Clicks from AI chat are few; they are totals observed in this period only, not a long-term ratio. Clicks from regular search are more numerous but also describe this period only.
| Platform | Crawlers (purpose) | AI-company crawler reads (verified) | Search-engine crawler reads (verified) | Clicks from AI chat | Clicks from regular search |
|---|---|---|---|---|---|
| OpenAI (ChatGPT) | ChatGPT-User, GPTBot, OAI-SearchBot (real-time reads for users, training, search) | 20,455 | — | 14 | — |
| Anthropic (Claude) | Claude-User, ClaudeBot (real-time reads for users, training) | 469,249 | — | 1 | — |
| Googlebot, GoogleOther (search, other use) | 5,839 | 217,109 | 9 | 15,345 | |
| Microsoft (Bing/Copilot) | bingbot (search) | — | 275,410 | 4 | 2,894 |
| Perplexity | PerplexityBot (search) | Not yet verified by Cloudflare; no number shown | — | 7 | — |
| Meta AI | meta-externalagent (training) | 233,806 | — | 0 | — |
| Apple | Applebot (search) | — | 123,893 | Cannot be measured (no identifiable referring address) | — |
| Amazon | Amazonbot (training) | 187,558 | — | Cannot be measured (no identifiable referring address) | — |
| ByteDance | Bytespider (training) | 612 | — | Cannot be measured (no identifiable referring address) | — |
| DuckDuckGo | DuckAssistBot (real-time reads for users) | 29 | — | Cannot be measured (no identifiable referring address) | — |
Reads by training crawlers do not turn directly into clicks. Allowing them gives AI the chance to read the information your site publishes. Clicks are only one result we can measure.
That is why our approach is always to allow: let every AI crawler read your site.
“Verified” means Cloudflare classified the visitor as a verified bot. Reads that use the same name but are not verified (including probes from our own check tool) are not counted. Numbers are Cloudflare visit estimates. Clicks count only visits that carry a referring address; clicks without one (for example, links opened inside apps) are not counted, so the real number may be higher.
“Clicks from regular search” covers Google and Bing result pages, which may include clicks from AI summaries shown there; these cannot be separated.