KlientBrain AI VisibilityDocs
Help Sharing

Connect Cloudflare

Send AI crawler visits from your Cloudflare site to AI crawlers as they happen, with a small Worker.

If your site is on Cloudflare, a small Worker can send each visit from an AI crawler to your project as it happens. There is no log file to download or upload. The Worker changes nothing for your visitors: every page is returned exactly as your site sent it.

Set it up

  1. Open Settings, then Connected Apps, then Connect Cloudflare. Only a workspace Admin can see this tab.
  2. Choose Generate API key and copy it. It is shown once.
  3. In the Cloudflare dashboard open Workers & Pages, choose Create application, then Start with Hello World, name it, and choose Deploy.
  4. Choose Edit code, delete what is there, and paste the script you get from Copy the Worker script. Choose Deploy.
  5. In the Worker's Settings, open Variables and secrets, add a Secret named KB_API_KEY, and paste your key.
  6. In Settings, then Domains & Routes, choose Add, then Route. Enter your site, for example www.example.com/*, and pick your zone.

Your site's DNS record must have the Cloudflare proxy on (the orange cloud). Workers only see traffic that goes through Cloudflare.

Check it works

Last event received at the top of the card shows the time, the crawler and the page of the newest visit. Choose Check now to look again. The AI crawlers page includes the visits within a minute.

If nothing arrives:

  • The route does not match your site's address. www.example.com/* does not match example.com/*.
  • The website saved on your project is not this site. Visits are matched to a project by its website, and the card's Details lists the websites it knows.
  • The key was revoked or pasted wrongly. In the Worker's logs, a 401 means the key.

What is sent

For each visit from an AI crawler, the time, your site's address, the page path without anything after a question mark, the status code and the crawler's name. No IP addresses, cookies or referrers are sent, and visits from people are never sent. We keep counts per day, crawler, page and status, as for an uploaded log.

Good to know

  • Cloudflare counts every request through a Worker. Check their current plan limits for a busy site.
  • To stop, delete the route in Cloudflare and revoke the key on this card.

Need help?

Email support@klientbrain.com and tell us what you were doing.