Bot view reads one page of your site twice, once as an AI crawler (OpenAI's search crawler, OAI-SearchBot) and once as a browser, and shows the two side by side.
| Row | What it means |
|---|---|
| Status | The answer the server gave each read, such as 200 or 403. |
| Words | Words of readable text on the page. |
| Tokens (estimate) | Characters divided by 4, a rough guide to how much text an AI model gets. |
A warning shows only when it is true: the crawler was refused while the browser was let in, the crawler got much less text (under 70% of the browser's words), or the crawler ended on a different page.
How it is worked out
Nothing is read until you press Compare. Neither read runs JavaScript, as most AI crawlers do not, so text added by scripts after loading is missing on both sides. Only pages on your project's own website can be read.
Where to find it
Pages, choose a page, then the Bot view card.
What to do next
If the crawler was refused, check your robots.txt and firewall rules in AI crawlers. If it got much less text, make sure the main content is in the page itself, not loaded later by scripts.