Web Bot Auth — how to make your site agent-ready

What it is

Web Bot Auth is an IETF draft where bots sign their HTTP requests using HTTP Message Signatures (RFC 9421) and publish their public keys at a well-known URL. Receiving sites verify the signature against those keys to distinguish authentic, well-behaved bots from impersonators.

Why it matters

Today, bot identification is based on the User-Agent string — trivially forged.
Cryptographic attribution lets a site offer favourable treatment (higher rate limits, access to gated content, skip CAPTCHA) only to verified bots.
For sites running bots, publishing keys makes them identifiable to receivers.

Remediation Prompt

I want to improve my site's agent readiness. Please implement the following fix for Web Bot Auth across our codebase.

Instructions:
Please fix the Web Bot Auth issue on my site so it is agent-ready.

How we test it

Step	Method	URL	Notes
A	GET	`/.well-known/http-message-signatures-directory`	JSON

Body cap 64 KB. Parse as JSON with schema check.

Pass Warn Fail Matrix

Condition	Status	Score
File exists, parses as valid JWKS with ≥1 key	pass	1.0
File exists, parses but has no keys	warn	0.4
File exists but invalid JSON	fail	0.0
404	not_applicable (most sites don't run bots)

Important: not_applicable for a 404 here. Absence is not failure for this particular standard — a site only needs this if it makes requests to other sites on behalf of users. We note absence in evidence but exclude the weight from the denominator.

Sub Tests

id	Weight	Pass when
`directory-present`	0.5	200 with parseable JSON
`has-keys`	0.5	JWKS has ≥1 valid entry

Remediation Prompt

I want to make my site's outgoing bot requests identifiable via Web Bot Auth.

1. Generate an Ed25519 or ECDSA P-256 keypair for the bot.
2. Publish the public key at /.well-known/http-message-signatures-directory as JWKS:
   {
     "keys": [
       { "kty":"OKP","crv":"Ed25519","kid":"my-bot-2026-04","x":"<base64url>","alg":"EdDSA" }
     ]
   }
3. Configure the bot to sign every outgoing HTTP request using HTTP Message Signatures (RFC 9421) with the following covered components:
     @method, @authority, @target-uri, @created, @expires
   Include a `Signature-Agent: "<my-bot-url>"` header.
4. Rotate keys at least yearly. Keep old keys in the directory for 30 days after rotation.

If I am not running my own bot and only serving content, I can skip this check — it does not apply.

Implementation Examples

Static JSON served from `/.well-known/http-message-signatures-directory`

{
  "keys": [{
    "kty": "OKP",
    "crv": "Ed25519",
    "kid": "example-bot-2026-04",
    "x": "11qYAYKxCrfVS_7TyWQHOg7hcvPapiMlrwIaaPcHURo",
    "alg": "EdDSA"
  }]
}

Node.js request signing (conceptual)

Use a library that implements RFC 9421 (e.g. httpbis-digest-headers + a message-signatures helper). Sign on each outbound fetch.

Common Mistakes

Serving the directory with Content-Type: application/octet-stream; should be application/json.
Including the private key by mistake (!!) — audit before publishing.
Forgetting kid — receivers need it to pick the right key.
Rotating keys without a grace period — breaks in-flight requests.