Storyboard troubleshooting

Common failure patterns when running AdCP compliance storyboards — missing fixtures, signature challenges, envelope drift, context echo, capability mismatches, and state-machine error codes.

When a compliance storyboard fails against your agent, the runner reports a step name and error text. This page maps the most common error patterns to their root causes and fixes, so you can resolve each class of failure without spelunking through SDK source or runner internals.

Each section shows the error you’ll see, what it means, and what to change in your agent.

Unknown fixture errors

× (unknown step): PRODUCT_NOT_FOUND: Package 0: Product not found: test-product

The storyboard’s sample_request references a hardcoded ID (test-product, test-pricing, campaign_hero_video, gov_acme_q2_2027, etc.). The runner expects the agent to have that ID in its catalog before the mutating step runs.

Fix: Implement comply_test_controller and honor the seed scenarios declared in the storyboard’s fixtures: block. When prerequisites.controller_seeding: true is set, the runner auto-injects a fixtures phase that calls seed_product, seed_pricing_option, seed_creative, seed_plan, or seed_media_buy in foreign-key order before the main phases execute.

See Compliance test controller — Scenarios for the full seed contract. Agents that return UNKNOWN_SCENARIO on a seed call grade the storyboard not_applicable — they are not penalized for missing sandbox surface, but they cannot pass storyboards that depend on pre-seeded state.

Signature challenge missing on 401

× (unknown step): expected error="request_signature_required", got error="(none)"

The storyboard sent an unsigned request to an operation declared in get_adcp_capabilities.request_signing.required_for. Your agent rejected with 401 but did not include a WWW-Authenticate: Signature ... challenge header, so the runner could not resolve the error code from the transport binding.

Fix: Emit the RFC 9421 challenge header on every 401 caused by missing or invalid signatures. The runner resolves the error code via the transport binding order — if the WWW-Authenticate header is absent, the error classification falls back to “(none)” even when the JSON body carries a useful message.

The reference SDK constructs these errors via RequestSignatureError from @adcp/sdk/signing with .code: RequestSignatureErrorCode. The full taxonomy (request_signature_required, request_signature_header_malformed, request_signature_tag_invalid, request_signature_window_invalid, request_signature_key_unknown, etc.) is enumerated in that module. Your agent SHOULD surface the same code on the challenge so SDK-speaking callers can recover automatically.

See Signed Requests (Transport Layer) for the challenge-header format and the transport-error binding order.

Response envelope drift

× (unknown step): Response contains errors array

The vector used check: error_code but your response surfaces the error on a shape the runner’s client-detection order didn’t expect. In practice, this means your agent returned errors[] when the transport layer already carried adcp_error, or vice versa — the storyboard asserted a single error code and the runner resolved it from a different layer than you emitted on.

Fix: Pick one error surface per response and stick to it per the envelope vs. payload two-layer model. On MCP: adcp_error for structured content; errors[] for task-payload errors. On A2A: the same layers apply — transport error in the envelope, application error in the task artifact’s DataPart.

The runner’s check: error_code is shape-agnostic — it resolves from either layer — but if your agent emits both simultaneously they can disagree, and the runner grades the resolved code against the vector’s expectation. Choosing one surface and being consistent avoids the divergence.

Context echo failures

× (unknown step): expected field "context.correlation_id" = "xyz", got (missing)

Your agent returned a response that does not include the context: object from the request. Every storyboard step that sends context: { correlation_id: ... } asserts that context.correlation_id echoes unchanged in the response.

Fix: Preserve the full context: object verbatim on every response, including errors. The echo contract is normative — buyers use correlation_id to stitch multi-agent flows, and the runner grades every context-carrying step on it. See Context and sessions — Normative echo contract.

Captures use the same contract in reverse: storyboards that pass "$context.<name>" through context_outputs: rely on the capture populating after the producer step’s validations pass. A downstream step reading $context.foo when the producer failed or omitted context: grades as unresolved_substitution.

Capability-vector mismatch (runner declared, agent doesn’t support)

× (unknown step): capability X asserted but not declared in get_adcp_capabilities

The storyboard dispatched a step that requires a capability your agent does not advertise in its get_adcp_capabilities response. The runner should auto-skip these steps; if you’re seeing them graded as failures instead, either the capability is declared at the wrong key or the runner is missing the auto-skip path.

Fix: Double-check your get_adcp_capabilities.tools list and any required-for fields (request_signing.required_for, idempotency.supported_tools, etc.). For vectors that apply only to specialized agents, the storyboard author can use skipVectors to flag the opt-out explicitly; as an implementer, the fix is almost always on the capability declaration rather than the vector.

Required-for composition

× (unknown step): missing auth — step requires authenticated or signed

The runner encountered a mutating step that expected either authenticated credentials or a signed request, and the transport carried neither. Typically this means the test kit didn’t declare auth.api_key AND the agent doesn’t advertise request-signing support — leaving the runner with no way to authenticate the call.

Fix: Either (a) declare auth.api_key in the test kit so the runner uses Bearer auth, or (b) advertise request-signing via get_adcp_capabilities.request_signing so the runner signs the request instead. The runner’s requireAuthenticatedOrSigned gate accepts either path — it fails only when both are absent.

Bearer-only agent: no auth mechanism contributed (assert_mechanism)

{
  "check": "assertion",
  "passed": false,
  "description": "Probe validations failed.",
  "expected": ["auth_mechanism_verified"],
  "actual": []
}

The mechanism_required phase found no contribution to auth_mechanism_verified from either prior phase. This is the most common failure for Bearer-only (API-key-only) agents, and it is not a real auth problem — it is a test-kit configuration gap.

What happened: The api_key_path phase has skip_if: "!test_kit.auth.api_key" — it is skipped entirely unless the test kit declares an API key. The oauth_discovery phase 404s on /.well-known/oauth-protected-resource/... (expected for Bearer-only agents; those failures are silently ignored by the runner). With both phases contributing nothing, assert_mechanism sees actual: [].

The --auth TOKEN distinction: The --auth TOKEN flag you pass to the runner is the runner’s own session credential — it authorizes the runner’s own requests to your agent. It is entirely separate from test_kit.auth.api_key, which is the specific Bearer value the api_key_path phase sends during its positive-key and invalid-key probes. These are not the same token and are not interchangeable.

Fix: Every AdCP test kit declares its probe API key under auth.api_key using the demo-<kit>-v1 naming convention. The default test kit (acme-outdoor) uses demo-acme-outdoor-v1. Configure your agent to accept the kit’s probe key as a valid compliance-testing credential alongside your production key. The demo-<kit>- prefix is the AdCP conformance handle — accept any Bearer token matching the prefix for the kits you run against (the suffix can rotate across spec versions, the prefix stays stable):

serve({
  authenticate: verifyApiKey({
    keys: {
      [PRODUCTION_TOKEN]: { principal: 'my-principal' },
      // Accept the default compliance kit's probe key (and any future suffix rotation)
      'demo-acme-outdoor-v1': { principal: 'compliance-runner' },
    }
  })
})

Agents that only accept their own production token skip api_key_path (no test-kit key matches), fail oauth_discovery (no PRM), and land at actual: [] in assert_mechanism. Adding the demo key to your allowed-keys set is sufficient — you do not need a PRM endpoint or an OAuth issuer.

Do not serve a fake /.well-known/oauth-protected-resource/... pointing at a non-existent issuer to “pass” the OAuth phase. That triggers the advertised-but-unserved failure mode the storyboard was designed to catch.

See Known spec ambiguities — PRM required for non-OAuth agents for background on why the carve-out exists and how the api_key_path / oauth_discovery phase semantics were designed.

INVALID_STATE vs INVALID_TRANSITION

Two codes that are easy to confuse:

• INVALID_STATE — the canonical AdCP media-buy error code for “the resource is in a state that doesn’t allow this action.” Used on create_media_buy/update_media_buy/pause/resume/cancel against a media buy that cannot transition as requested. See media-buy/specification.mdx and media-buy/media-buys/index.mdx for authoritative usage.
• INVALID_TRANSITION — specific to the comply_test_controller sandbox primitive. Emitted when a runner requests a state-machine transition the seller rejects (e.g., forcing approved → archived without going through active). See Compliance test controller — Scenarios.

A storyboard vector that asserts INVALID_STATE on a production task but your agent returns INVALID_TRANSITION is an error-code vocabulary mismatch — INVALID_TRANSITION is not in the canonical enum at static/schemas/source/enums/error-code.json and should not appear outside the compliance test controller.

When none of the above matches

If you hit a failure that doesn’t map to anything here, check the known spec ambiguities page — some storyboards are blocked on resolved-but-unreleased spec gaps, and the workaround is tracked there.

Still stuck? File an issue at adcontextprotocol/adcp with the full runner output and the storyboard name. Maintainers can usually narrow the pattern from the error signature.