Markdown Content Negotiation

Pass Warn Fail Matrix

We additionally try to detect token savings: compute tokens(html) vs tokens(markdown) using a cheap tokeniser approximation (chars/4). If markdown is ≥40% smaller, tag the evidence with goodReduction: true.

Sub Tests

Remediation Prompt

I want my site to respond with Markdown when an AI agent sends Accept: text/markdown.

Implement this in one of two ways (either is fine, both is better):

1) Content negotiation on the original URL.
   When a request includes Accept: text/markdown, return:
     - Status 200
     - Content-Type: text/markdown; charset=utf-8
     - A hand-curated or cleanly-generated markdown version of the page (no nav chrome, no cookie banner, no ads)
     - A Vary: Accept header

2) URL-suffix fallback.
   Serve the same markdown at /index.md relative to the HTML page's URL
   (e.g. /docs/foo/ → /docs/foo/index.md or /docs/foo.md).

For tooling:
- If I'm on Next.js, generate markdown from the same MDX source the page is built from. Strip shell components (navbar, footer).
- If I'm on a CMS, export the post body as markdown and serve it directly.
- Make sure the markdown path is in my sitemap alternative or at least in /llms.txt.

Do not include HTML in the markdown version. Do not include navigation menus. Do include the page title as H1, the author/date as short metadata, and the full body.

Implementation Examples

Next.js dynamic (route handler)

// src/app/[...slug]/route.ts — runs BEFORE the page if mime is markdown
export async function GET(req: Request, ctx: { params: { slug?: string[] } }) {
  const accept = req.headers.get('accept') ?? '';
  if (!accept.includes('text/markdown')) return; // let page render HTML
  const md = await buildMarkdownFor(ctx.params.slug);
  return new Response(md, { headers: { 'content-type': 'text/markdown; charset=utf-8', vary: 'Accept' } });
}

Cloudflare Workers (Rules)

• Request Header Transform: on ends_with(http.request.uri.path, "/index.md"), set Accept: text/markdown
• URL Rewrite: strip /index.md and proxy to same-path
• Response Header Transform: add Vary: Accept

See the Cloudflare docs article for the exact recipe.

Static site

Pre-generate *.md alongside every *.html at build time. Serve with the right content-type.

Common Mistakes

• Content-Type is text/plain — fails the check; must be text/markdown.
• Keeping the navigation sidebar inside the markdown → defeats the token-saving purpose.
• Infinite loop: agent requests /index.md, server serves HTML with a hidden "prefer markdown" note that links to /index.md/index.md. (Strip the directive from the markdown version.)
• Missing Vary: Accept — caches may serve the wrong representation to the next client.

References

• Markdown for Agents (Cloudflare blog)
• RFC 9110 §12.5.1 — content negotiation
• State of AI agent content negotiation — Checkly, 2026

Test Fixtures

• pass-accept.json — homepage returns markdown on Accept
• pass-index-md.json — /index.md route returns markdown
• warn-vary-html.json — Vary: Accept but body is HTML
• fail-html-only.json — always HTML