Blog

AI crawlers and AI referrals are different measurements

AnalyticFast Team · 2026-10-31

A practical guide by the AnalyticFast Team. Examples are illustrative; this article reports no original research or customer performance results.

An AI bot fetching a page, an assistant mentioning your product and a person clicking an assistant’s link are three different events. Only the last starts an ordinary human website journey. Keeping them separate makes both your SEO decisions and revenue reporting more defensible.

Distinguish the questions

Was a page fetched by an automated agent?

Inspect edge or server request logs and documented bot signals. Many fetchers do not run your browser tracker.

Did someone click through from an assistant?

Inspect observed referral or campaign context in a real visit, then follow the visitor’s recorded journey.

Did an assistant mention or recommend the product?

A crawler request or referral count cannot establish the number of mentions, answers or impressions.

Did an AI-referred visitor buy?

Join the observed human visit to a verified payment through the supported checkout handoff.

Read bot controls by purpose

OpenAI distinguishes the search crawler OAI-SearchBot, the training crawler GPTBot and user-triggered ChatGPT-User requests. Its official crawler documentation describes their separate roles and controls. Anthropic likewise documents distinct ClaudeBot, Claude-SearchBot and Claude-User agents in its crawler guide. Do not assume a single robots rule has the same effect for every agent or guarantees inclusion in an answer.

What AnalyticFast’s browser tracking can show

The source classifier recognizes supported assistant referrer domains and explicit source tags, including ChatGPT, Claude and Perplexity. That means a human referral can be grouped as AI when its context is available. Known crawler and automation user-agent patterns are excluded from normal ingestion. These filters are heuristics, not proof that every retained visitor is human or that every bot is detected.

A JavaScript analytics script is not a complete crawl log. Bots that only fetch HTML may never request it. A normal visit can also lose its referrer or arrive through a copied link and appear direct. Do not relabel all direct traffic as AI traffic to fill that gap. A crawl spike is not evidence of more human buyers.

A practical measurement checklist

1. Record which measurement you are examining: bot requests, human referrals, mentions or paid outcomes.
2. For crawler analysis, use server/edge evidence and the provider’s documented signals. A user-agent string can be spoofed.
3. For human referrals, inspect the landing page and source context without adding personal data to URLs.
4. For revenue, verify the payment metadata handoff and compare currency-separated net amounts, including refunds.
5. Label sample size, observation period and missing coverage before deciding which content to change.

Improve the page before claiming an AI SEO result

Publish useful answers with stable URLs, readable HTML, accurate canonical metadata and references where claims need support. Check that your chosen robots rules and network controls allow the uses you intend. Those are sensible publishing practices, not a guarantee of crawling, citation, rankings or revenue. Evaluate actual human referral journeys over a consistent period before calling a change successful.

This is an implementation guide based on documented agent roles and the current source classifier. It is not a dataset of AI search behavior, a citation study or a live crawl experiment.

Add your website to AnalyticFast, or start with the installation guides and payment integrations.