Bot Classification Research Note

Bot Classification Research Note

Finding

A user-agent string is a claim, not verified identity. Google recommends forward-confirmed reverse DNS or published IP-range matching for Google requests. Cloudflare's current verified-bot model also recognizes signed requests, published ranges with stable user agents, and reverse DNS.

Google Analytics automatically excludes traffic from known bots and spiders, but its documentation qualifies this as exclusion to the extent possible. Site owners cannot use the remaining Analytics count as proof that every session was human.

RFC 9309 defines how crawlers identify a product token and read robots.txt. It also says the protocol is not access authorization. A compliant product token helps policy application, but it does not prevent spoofing or establish a person behind a request.

Reporting taxonomy

The defensible reporting categories are verified automation, declared but unverified automation, likely automation, likely human activity, and unknown. User-triggered agents may be automated at the network layer while representing a person's immediate action.

Business actions provide stronger audience evidence than a page request. A confirmed purchase, verified subscription, completed interview request, or authenticated product action is still not proof that every preceding event was human, but it is closer to the outcome the publisher cares about.

Publication rule

Avoid perfect-detection claims. Preserve uncertainty, document the classification method, and separate traffic cleanliness from human demand.