> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getdedica.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Build your own middleware: protocol v1

> Implement the Dedica middleware protocol v1 in any stack: the well-known file, heartbeats, the policy, agent detection, serving and the /ask and /mcp proxy.

If your site isn't on Cloudflare, implement the Dedica middleware protocol v1 in your own stack (a Spring Boot filter, an Express middleware, an nginx module, a reverse proxy). Dedica treats every middleware the same. The [Cloudflare Worker](https://docs.getdedica.com/middleware/cloudflare-worker.md) is the reference implementation.

> [!WARNING]
>
> **Fail open.** Give every call to Dedica a timeout. On any error, timeout, unknown policy schema or missing answer, send the request to your origin exactly as if the middleware weren't there.

## Configuration

| Value | Meaning |
| - | - |
| `site_id` | Your site's public id (a UUID). Not a secret. |
| `token` | Shown once when you create it. Keep it in your platform's secret store. |
| API base | `https://api.getdedica.com`. Every path below is under it. |

Send these headers on **every** call to Dedica:

| Header | Value |
| - | - |
| `X-Dedica-Middleware-Token` | Your token. |
| `X-Dedica-Transmitter` | `<name>/<version>`, for example `spring-boot/0.3.0`. Dedica shows it in the dashboard. |

## 1. Serve the well-known file

`GET https://<your domain>/.well-known/dedica.json` must return `200` with `Cache-Control: no-store` and:

```json theme={null}
{"site_id": "<site_id>", "middleware": "spring-boot", "version": "0.1.0"}
```

## 2. Send heartbeats

Call `POST /api/v1/middleware/heartbeat/` with `{"middleware": "...", "version": "..."}` at least every 5 minutes, or fetch the policy that often (a policy fetch counts as a heartbeat). A `401` means a wrong or replaced token.

## 3. Verification

When you run the middleware check in Dedica, it fetches your well-known file and requires the right `site_id` and a heartbeat from the last 15 minutes.

## 4. Fetch the policy

`GET /api/v1/middleware/policy/` returns what to do for your site:

```jsonc theme={null}
{
  "schemaVersion": 1,           // anything else: ignore the policy and pass everything through
  "ttlSeconds": 30,             // reuse it this long, then refetch
  "version": "9f2c…",           // changes whenever anything else does
  "siteId": "…",
  "mode": "report",             // "report": pass everything through. "autopilot": serve
  "detection": {
    "traditionalBots": ["googlebot", …],   // search engines, link previews, monitors
    "traditionalBotsExact": [],
    "agentUa": ["gptbot", "claudebot", …], // AI agents and crawlers
    "agentUaExact": [],
    "signatureAgentDomains": ["chatgpt.com"],
    "botLikePattern": "bot\\b|…",
    "humanPageVeto": {
      "exact": [["<ua>", "human|agent|question"]],
      "contains": [["applebot-extended", "agent"], ["applebot", "human"], …],
      "botLikePattern": "bot\\b|crawl|spider|slurp"
    }
  },
  "twins": {"enabled": true, "negotiate": true, "suffix": ".md", "includePaths": [], "excludePaths": [], "questionParams": []},
  "artifacts": [{"path": "/llms.txt", "action": "proxy", "contentType": "text/plain; charset=utf-8", "version": "…"},
                {"path": "/openapi.json", "action": "fallback", "contentType": "application/json", "version": "…"}],
  "serve": {"baseUrl": "https://api.getdedica.com/api/v1/middleware/serve/", "version": "…"},  // null unless autopilot
  "query": null,                // /ask and /mcp settings (section 6); null: pass both through
  "queryAvailability": {}
}
```

Back off on failures (the reference worker waits 15 seconds, 60 seconds for an unknown schema and 5 minutes for 401 or 403). If Dedica is down, you may keep using a policy with `serve` for up to 15 minutes after the last successful fetch.

### Agent detection

Match on the lowercased `User-Agent`, with every needle list sorted longest first:

* A search engine, link preview or monitor (`traditionalBots`) is never an agent.
* A browser navigation (`Sec-Fetch-Mode: navigate` and `Sec-Fetch-Dest: document`) is a person.
* A known AI agent (`agentUa`), a `Signature-Agent` header in `signatureAgentDomains`, or a `botLikePattern` match without `Sec-Fetch-Mode` is an agent.

## 5. Serve (autopilot only, GET and HEAD only)

Check these in order. The first match wins, and anything that answers nothing goes to your origin.

1. `X-Dedica-Origin-Fetch: 1` (Dedica's own crawler): always your origin.
2. The path is in `artifacts`: `proxy` serves it from Dedica for every visitor, `block` returns 403 and `passthrough` goes to your origin. `fallback` asks your origin first and returns its answer when it has the file itself (any 3xx, or a 2xx that isn't `text/html`); otherwise serve it from Dedica, and if Dedica has nothing either, return your origin's answer. Pass through any action you don't know.
3. `humanPageVeto` says `human` (or an unknown bot-like user agent): your origin. With `?mode=agent`, only `human` keeps the normal page.
4. The path ends with `twins.suffix` (`/pricing.md`) and matches `includePaths` and not `excludePaths`: serve it from Dedica as a shared, cacheable answer.
5. `twins.negotiate` is on, the client isn't a search engine, and it is a detected agent, sends `Accept: text/markdown` or asks for `?mode=agent`: serve the page's agent version with `Cache-Control: private, no-store`. Forward the query string.
6. Your origin answered `404` or `410` to an agent with a body that isn't markdown: keep the status and replace the body with a short markdown note pointing to `/llms.txt` and the home page.

On a cold start, wait for the policy on paths Dedica may manage (`/llms.txt`, `*/llms.txt`, `*.md`, `/openapi.json`, `/mcp/server-card`, `/.well-known/*`), on `/ask` and `/mcp` (any method, so an agent's first call isn't sent to your origin), and on `?mode=agent`. Every other request goes to your origin while the policy loads.

### Calling the serve API

Call `GET <serve.baseUrl><path without leading slash>?<query>` with:

* the token header;
* the visitor's `User-Agent` and `Accept`;
* `X-Dedica-Version: v1:<artifact version or serve.version>`;
* `X-Dedica-Category: public` (shared answers) or `eligible` (private agent versions).

Forward `If-None-Match` and `If-Modified-Since` only for shared answers. Never let the path leave `serve.baseUrl`: reject `.` and `..` segments.

| Dedica answers | Do this |
| - | - |
| `200`, `304` (shared only), `428` | Return it to the visitor with `X-Dedica-Served: 1` and `Vary: accept, accept-encoding, user-agent, sec-fetch-mode, sec-fetch-dest`. Keep `ETag`, `Last-Modified` and `Cache-Control` only for shared answers. Cap the body (the reference worker: 8 MB, 10 seconds). |
| `404` | Dedica has nothing for this path. Your origin answers. |
| Anything else, or a timeout | Your origin answers. Back off a few seconds on 401, 403 and 5xx. |

## 6. Proxy /ask and /mcp

While `query` is `null`, pass both through. Once the policy enables a route, it looks like this:

```jsonc theme={null}
"query": {
  "enabled": { "mcp": true, "ask": true },   // a route that is off passes through to your origin
  "askPath": "/ask", "mcpPath": "/mcp",
  "baseUrl": "https://api.getdedica.com/api/v1/middleware/query",
  "requestsPerMinute": 30
}
```

Proxy the enabled routes to `query.baseUrl`, which must be Dedica's API origin, and stream the response:

| Request on your domain | Proxy to |
| - | - |
| `POST /mcp` | `<baseUrl>/mcp` |
| `GET /ask?q=<question>` | `<baseUrl>/ask` |
| `POST /ask` (NLWeb v0.55) | `<baseUrl>/ask/nlweb` |

Forward only `accept`, `content-type`, `origin`, `user-agent`, `sec-fetch-*`, `mcp-*` and `last-event-id`, plus the token header. Failures are explicit JSON errors (JSON-RPC for `/mcp`, NLWeb `failure` for `/ask`), never your origin, because the visitor asked for Dedica's endpoint. Limit each visitor to `query.requestsPerMinute` requests a minute.

## 7. Report traffic

Report every page load, from people, AI agents and bots alike, so Dedica can show how much of your traffic is agents. Don't report static files (scripts, styles, images, fonts, media), the sub-requests a browser page makes (`Sec-Fetch-Dest` other than `document`, `iframe` or `frame`), prefetches (`Sec-Purpose` or `Purpose: prefetch`), requests with `X-Dedica-Origin-Fetch: 1` or the well-known file.

Call `POST /api/v1/middleware/events/` with `Content-Type: application/x-ndjson`: a batch of up to 1000 events, one JSON object per line, and optionally an `X-Dedica-Batch-Id` header.

> [!WARNING]
>
> Never send an IP address, cookies, your token or the full query string. Identify a visitor with a daily pseudonym: the hex SHA-256 of `<ip>|<user-agent>|<YYYY-MM-DD>|<salt>`, with a secret salt Dedica doesn't have. From the query string keep only `utm_source`, `utm_medium`, `utm_campaign`, `ref`, `source` and the policy's `twins.questionParams`, and from the referer only its origin and path.

```jsonc theme={null}
{
  "id": "<uuid>", "ts": "2026-11-05T10:00:00.000Z",
  "method": "GET", "host": "shop.example", "path": "/pricing.md", "query": "?utm_source=chatgpt.com",
  "status": 200, "latency_ms": 42,
  "outcome": "twin",
  "served_by_dedica": true,
  "category": "public",       // public, eligible or null
  "client": {"kind": "ai_agent", "matched": "gptbot", "signature_agent": null},   // kind: ai_agent, bot or unknown; null without a policy
  "user_agent": "...",        // up to 512 characters
  "referer": "https://chatgpt.com/c/123",   // origin and path, if any
  "visitor": "<64 hex characters>",        // the daily pseudonym
  "country": "DE",            // if your platform knows it, else null
  "headers": {"accept": "...", "accept-language": "...", "sec-fetch-mode": "...", "sec-fetch-dest": "...", "sec-fetch-site": "..."},
  "mode": "autopilot", "policy_version": "..."   // null without a policy
}
```

`path` and `outcome` are required. `outcome` says what the middleware did:

| `outcome` | Meaning |
| - | - |
| `proxied` | A proxied path such as `/mcp` or `/ask`. |
| `artifact` | A file Dedica manages, served from Dedica. |
| `artifact_origin` | A `fallback` file your origin has itself. |
| `blocked` | A file the policy blocks. |
| `twin` | An explicit markdown URL such as `/pricing.md`. |
| `negotiated` | The agent version of a normal page. |
| `human_page` | A search engine, preview or unknown crawler kept the normal page. |
| `agent_404` | Your origin's 404 or 410, with a markdown body for an agent. |
| `origin` | Passed through to your origin. |

Batch events after the response, in the background, so reporting never slows down or changes a response. The reference worker sends a batch at 1000 events or 5 seconds after the first one, with a 5-second timeout.

Dedica answers `202` with `{"accepted": n, "rejected": n, "errors": [...]}` (the first 10 rejected lines and why). A bad line never fails the rest of the batch; more than 1000 events is a `400`. If Dedica answers `429` or `503`, or can't be reached, keep the batch and send it with the next one (the reference worker keeps up to 5000 events). Drop it on any other answer.

> [!NOTE]
>
> Dedica accepts traffic reports but doesn't show them in the dashboard yet.

## Not in the protocol yet

Guardrails (blocking agents, route rules, rate limits) and bot identity verification. Once the middleware is connected, its token can't be changed.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.