All your connected log sources live in one place: Data sources. It lists everything that sends AI crawler visits to this project, with its domain, status and when data last arrived.
Where to find it
Open AI crawlers and choose Data sources at the top of the page.
The three figures
- Connected: how many sources are sending data to this project.
- Last data received: the newest data from any source.
- Domains: the domains your sources cover. The Cloudflare Worker also covers every subdomain of your website.
The table
One line per source, with Domain, Provider, Status, Last data received and Added.
| Status | What it means |
|---|---|
| Integration active | The Cloudflare Worker is sending visits. |
| Awaiting activation | The key is made, but no data has arrived for this project yet. |
| Uploaded | Log files you uploaded. |
| Disconnected | The key was revoked. Data received before stays. |
When a line has many uploads, it counts only the newest ones and says so, for example Latest 200 batches, and Added shows "-".
Check now
Check now reads the list again, so a source you just set up shows as soon as its first data arrives. It does not contact your site or your provider.
Add a source
Choose New data source to see the ways to connect one: the Cloudflare Worker or a Log file upload. See Connect Cloudflare for the Worker. Only a workspace Admin sees this button.
Disconnect a source
Choose Disconnect on its line, then Disconnect again to confirm. This revokes the key the source sends with, which stops data collection immediately. Data already received stays.
- The Cloudflare Worker key is shared by the whole workspace, so disconnecting it stops the Worker for every project. To also remove the Worker, delete it in your Cloudflare dashboard under Workers & Pages.
- Anything else using a revoked key stops working too.
- Uploaded files have no Disconnect and cannot be removed one at a time. Delete all log file data on AI crawlers permanently removes all AI crawler data for the project, Cloudflare Worker data included, not just uploaded files. It cannot be undone.
Only a workspace Admin can disconnect a source. Team members who can upload logs can see the list. Client logins do not see this page.