User-Agent

Agent & Bot Identity IANA permanent request 2 spellings

User-Agent is a self-declared string. It has always been a self-declared string. Everything built on top of it — bot detection, crawler policy, analytics segmentation, the entire robots.txt enforcement model — rests on the assumption that clients tell the truth about themselves, which is an assumption and not a mechanism.

That was tolerable when the consequence of a lie was a skewed analytics number. It is less tolerable now that the question is whether an AI agent may read your content, and the answer is being derived from a string the agent chose. Work like Web Bot Auth and the Signature-Agent header exists to replace assertion with proof, and it is early.

Seventy-one providers declare it in a contract, usually to require it, occasionally to require a specific format.

The registry

Listed in the IANA HTTP Field Name Registry as a permanent entry. Defined in RFC 9110, Section 10.1.5: HTTP Semantics.

In the catalog

Declared by 71 providers across 8,804 published specification files in the API Evangelist catalog, where it appears as a request header — sent by the client.

It is spelled 2 different ways across those contracts — User-Agent, user-agent. HTTP field names are case-insensitive (RFC 9110, §5.1), so every one of these is the same header. They are not the same string, which is why generated clients disagree about it.

Using it

Send an honest one with contact information if you are the client — a hostname or an email in the string is what gets you a conversation instead of a block when your traffic looks odd. If you are the server, do not build access control on it. Use it for diagnostics and shape analysis, and use something cryptographic for decisions.