Validate adapter agents with mock upstream fixtures

Documentation Index

Fetch the complete documentation index at: https://docs.adcontextprotocol.org/llms.txt

Use this file to discover all available pages before exploring further.

​

The four-step recipe

# 1. Boot a mock upstream for your specialism
npx @adcp/client@latest mock-server sales-social --port 4250 &

# 2. Run your AdCP agent, configured to use http://localhost:4250 as its upstream
./your-agent.sh        # Python / Go / Rust / TS — language-agnostic

# 3. Grade your agent against the matching storyboard
npx @adcp/client@latest storyboard run http://localhost:3001/mcp sales_social \
  --auth $TOKEN --json > grader.json

# 4. Assert your agent actually called the upstream
curl -s http://localhost:4250/_debug/traffic
# {"traffic": {"POST /oauth/token": 1, "POST /event/track": 6, ...}}

​

Why traffic counters

• Handler short-circuits before the upstream call when the input doesn’t match a non-spec branch (e.g. empty members[] on sync_audiences → returns shape-valid empty response, never POSTs).
• OAuth client is wired but never invoked from any handler; tree-shaking is defeated with a void fetchUpstreamToken; literal.
• Synthetic placeholder data is injected to satisfy the upstream’s required-field schema, masking real-data shape mismatches.

​

What’s available

• The upstream’s domain endpoints — shaped to match the real platform’s public contract.
• GET /_lookup/<resource>?<adcp_field>=<value> — runtime resolution from AdCP-side identifiers to upstream tenant IDs. Harness convention.
• GET /_debug/traffic — hit counters keyed by <METHOD> <route-template>, no auth, harness-only. Harness convention. Read after the storyboard run; assert each headline route is hit at least once.

​

CI integration

jobs:
  validate-adapter:
    runs-on: ubuntu-latest
    services:
      mock-upstream:
        image: node:20
        ports: ['4250:4250']
        # Use a bootstrap script that runs `npx @adcp/client@latest mock-server <specialism>`.
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/start-agent.sh &              # Your agent in your language
      - run: ./scripts/wait-for-port.sh 3001
      - run: |
          npx @adcp/client@latest storyboard run http://localhost:3001/mcp <storyboard_id> \
            --auth $TOKEN --json > grader.json
      - run: |
          # Assert each expected upstream route was hit at least once
          curl -s http://localhost:4250/_debug/traffic | \
            jq -e '.traffic["POST /oauth/token"] >= 1 and .traffic["POST /event/track"] >= 1'

​

Iteration loop

• Storyboard passing but traffic gate fails on N endpoints — classic façade. The handlers for those routes either short-circuited or never got exercised by the storyboard’s input.
• Storyboard partial with cascade skips — an early step (get_products, get_signals) returned a shape-valid response missing fields that the runner extracts state from. Downstream steps skip with unresolved context variables. Fix the early step’s response shape and most cascade skips clear.
• Storyboard failing on a single step + traffic gate clean — usually a one-line shape bug (wrong field name, missing required field, status mismatch). The per-step details names the field via JSON pointer.
• Traffic gate empty (0 hits everywhere) + agent appears to start — agent boot threw a recoverable error after listening on port. Check the agent’s stderr.

​

Limitations

​

Storyboard limitations

• Storyboards under-cover payload variety. A storyboard step may pass shape with an empty input where a real adapter never gets exercised on the variant that matters. Tracked at adcontextprotocol/adcp#3785.

​

Runner / tooling caveats

• Storyboard cascade skips silently when an early step’s response is shape-valid but missing fields the runner extracts state from. The error you see is on the downstream step, not the early one — investigate “skipped” steps before “failed”. Tracked at adcontextprotocol/adcp#3796 (runner-side).
• Mock seed data may not match storyboard fixture inputs. If you see 404s on _lookup/<resource>, the storyboard’s payloads may reference IDs the mock doesn’t seed. Either widen the mock’s seed or seed scenario state at runtime via the compliance test controller.

​

Traffic gate limitations

• Necessary, not sufficient. A handler can call upstream with synthetic placeholder data and still satisfy the hit-count assertion. For agents in regulated channels (audience uploads, conversion tracking, signed requests), additional integration tests against the real upstream’s payload validation are still required before production.
• Idempotency replay is not exercised by traffic counters. A façade that ignores idempotency_key and doubles upstream writes will pass the hit-count gate. Use the storyboard’s idempotency-replay scenarios + a separate counter check (same idempotency_key → same hit count) if your platform has at-most-once semantics.
• Outbound webhook delivery is not exercised by upstream traffic counters. Traffic counters live on the upstream the agent calls into; agent → buyer webhook signing/delivery is graded by the storyboard runner’s --webhook-receiver flag, separately. Both gates apply for adapters that emit webhooks.

​

What’s next

• Validate your agent — the broader storyboard-runner-driven validation checklist (fuzz, multi-instance, request-signing, webhook conformance).
• Compliance test controller — seed scenario state at runtime when storyboards need fixtures the mock doesn’t provide.
• Build an agent — the language-agnostic guide to building an AdCP agent.
• Compliance catalog — the full taxonomy of universal / protocol / specialism storyboards.