Repository navigation
[Conformance] ACS-Core over the wire: the reference Guardian accepts unsigned, replayed and stale envelopes and publishes no chain head #184
Description
[Conformance] ACS-Core over the wire: the reference Guardian accepts unsigned, replayed and stale envelopes, publishes no chain head, and answers system/ping, four minimum hooks and wrapped MCP with method-not-dispatched
Filed against #92. Every row below is a single reproduction against the reference Guardian, and every request and response pair is held as a capture file.
What you ran
| what | value |
|---|---|
| repository | GenAI-Security-Project/agent-control-standard at bfdb898be4a9bcaedd90529b480ae5b62e94f29a, main, the merge of #105 on 2026-09-10 |
| tree under test | reference-implementations/agt/ |
| worktree | clean throughout, with no uncommitted change on any run |
| host | Linux 7.0.0-31-generic x86_64 |
| Bun | 1.3.13, where CI pins 1.3.11. The shim behaviour #111 reports under 1.4.2 did not occur here |
| other runtimes | Node v24.19.0, Python 3.14.3 for the adapter, jq 1.8, trash-cli 7.2.0 installed the way the workflow does |
| AGT SDK | agent-control-specification@0.3.1-beta.0 from the lockfile; OPA from the bundled linux-x64 binary; no OPA path override needed |
| Guardian | ACS_GUARDIAN_PORT=18787 bun run guardian, posture proceed, the default, with policy/manifest.yaml, mapping.yaml and policy/lib unmodified |
| port | 8787 was already held by an unrelated local HTTP service on this workstation, so the Guardian moved to 18787. OBS-1 records what the shim did before the port was moved |
| full pin list | the capture 00-environment.txt |
Two harnesses ran, the control first and the adapter second.
-
The tree's own, as a positive control.
bun testover the whole suite, then the README's "Drive one hook by hand" payloads, rm -rf / and ls -la, through the shipped hook at hosts/claude-code/acs-hook.ts. -
An independent host adapter, acs_vector_adapter.py, which imports nothing from the tree and runs on the standard library alone. It builds envelopes from the published request-envelope and tool-call-request schemas. It signs them per Specification section 10: JCS of the envelope with the signature removed, HMAC-SHA256 under an HKDF session key, with the IKM recorded in the results file. It performs the handshake once per session, posts to the Guardian, and records every request and response pair.
| the adapter drove | detail |
|---|---|
| the ACS-Core negative-conformance corpus | 28 members at probityai/agent-evidence-vectors/vectors-acs-core, corpus digest 7eeedf34, read at ACS 9d4a9da. 16 members carry a wire request; 12 are framework-, approver- or deployment-role and carry none |
| fourteen named probes | one per ACS-Core obligation the corpus cannot express as a single request |
| run 1, vectors as written | kept because its uniform deny evaluation_failed is itself a finding, D10 |
| run 2, tools mapped | the corpus's abstract records.* tools rewritten to Bash and ls so policy content cannot mask a wire-level verdict. This is the run scored below |
What you expected
The conformance page, under "ACS-Core (mandatory baseline)", says a v0.1.0-conformant deployment MUST implement the handshake, the envelope, the six-hook minimum taxonomy, all five dispositions, SessionContext with the Guardian publishing the chain head on responses, replay protection, the baseline HMAC-SHA256 signature, decision honoring, system/ping, and wrapped MCP. The tree's README claims exactly one profile, acs-core, calls the claim "qualified", and carries a status table that already concedes several rows. This report measures each row on the wire. It also adds what the table does not list.
Per row, the sentence that set the expectation:
Section 10: "A signature over the section 10 canonical input is REQUIRED in ACS-Core ... A verifier MUST recompute this canonical form and MUST reject a signature that does not cover it (
SIGNATURE_INVALID, 17.1)."
Section 10.3: "Guardians MUST reject requests whosetimestampis more than the negotiated skew window ... returningTIMESTAMP_OUT_OF_WINDOW(-32006), MUST reject duplicaterequest_idvalues within the session withREPLAY_DETECTED(-32005), and SHOULD reject duplicatenoncevalues."
Section 8.6: "For every step where the Guardian writes a ContextEntry ... the Guardian MUST include the resultingchain_hashin its response, and thatchain_hashMUST be covered by the response signature."
Section 13: "Guardians MUST always returndecision: "allow"forsystem/pingregardless of policy ...system/pingMUST NOT return an ACS-specific error."
Section 4: "Version mismatch terminates withUNSUPPORTED_VERSION(-32001)."conformance.md: "ClientHello includesprofiles_supported; ServerHello includesprofiles_accepted." Section 10.3: "skew_window_msin ServerHello."
conformance.mdhook taxonomy: "At minimum:sessionStart,userMessageoragentTrigger,toolCallRequest,toolCallResult,agentResponse,sessionEnd."
conformance.md: "Wrapped MCP:protocols/MCP/*."
README "Verify": "bun test: the whole suite; exactly one test skips."
What happened
Positive control
| step | result | capture |
|---|---|---|
bun install |
82 packages, exit 0 | 00-bun-install.log |
bun test (fresh clone, no trash) |
1084 pass, 1 skip, 26 fail across 1111 tests; every failure is bun: command not found: trash from trashDir() in packages/conformance/test/*.test.ts |
01-bun-test.log |
bun add -g trash-cli@7.2.0, then bun test packages/conformance |
125 pass, 7 fail, all "a beforeEach/afterEach hook timed out" (5000 ms default); trash takes ~1.6 s per directory on this host and the after-hooks trash several |
02b-bun-test-conformance-with-trash.log |
same, --timeout 30000 |
132 pass, 0 fail | 02c-bun-test-conformance-timeout30s.log |
| Guardian banner | the README's four lines exactly, on 18787 | 03e-guardian-stdout-port18787.log |
shim, rm -rf / |
{"hookSpecificOutput":{...,"permissionDecision":"deny","permissionDecisionReason":"This command was blocked because it matches a destructive-shell-command pattern. Policy: destructive_shell_command_blocked, from AGT's stock bundle (agt_stock). Matched at offset 0."}}, exit 0, 4 envelopes, 1 chain entry |
03f-positive-control-hook-deny.log, 03h-positive-control-envelopes.jsonl |
shim, ls -la |
{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"allow"}}, exit 0 |
03g-positive-control-hook-allow.log |
| adapter P0a/P0b (same two commands through the independent envelope builder) | deny with reason_codes: ["destructive_shell_command_blocked"]; allow |
05-adapter-run-toolmap-Bash/wire/*-P0a-deny.json, *-P0b-allow.json |
So the tree does what its README says it does, byte for byte, and the adapter reproduces it without the tree's code. Everything below is measured against that baseline. Across the 350 envelopes the Guardian logged for the whole session, its stderr carried zero schema-failure lines: every response it sent was schema-valid.
Probes (Guardian obligations, one reproduction each)
Wire files are captured one per probe under the run directory; each holds the exact request body and the exact response.
| # | probe | spec | expected | observed | verdict |
|---|---|---|---|---|---|
| P1 | request with no signature |
10, 17.1 | -32004 SIGNATURE_INVALID |
decision: allow |
FAIL |
| P2 | HMAC with every byte inverted | 10 | -32004 |
decision: allow |
FAIL |
| P3 | same request_id twice in one session |
10.3 | -32005 REPLAY_DETECTED on the second |
decision: allow, twice |
FAIL |
| P4 | timestamp 3600 s in the past |
10.3 | -32006 TIMESTAMP_OUT_OF_WINDOW |
decision: allow |
FAIL |
| P5 | timestamp 3600 s in the future |
10.3 | -32006 |
decision: allow |
FAIL |
| P6 | chain_hash and signature on a content-bearing response |
8.6 | both present | neither present (the chain entry is written to .acs/session-context.jsonl) |
FAIL |
| P7 | client metadata.session_state.chain_hash = f x 64 |
8.6 (MAY) | chain_mismatch deny or -32007 |
decision: allow, no reason code |
observation, MAY |
| P8 | system/ping |
13 | decision: allow |
-32011 method not dispatched by this Guardian: system/ping |
FAIL |
| P9 | ClientHello with acs_version / supported_versions = 9.9.9 |
4 | -32001 UNSUPPORTED_VERSION |
ServerHello negotiated_version: 0.1.0 |
FAIL |
| P10 | ServerHello fields | conformance.md, 10.3 | profiles_accepted, skew_window_ms |
keys: methods_evaluated, negotiated_version, on_decision_failure, selected_transport, timeout_config |
FAIL |
| P11 | steps/sessionStart |
conformance.md taxonomy | a decision | -32011 |
FAIL |
| P12 | protocols/MCP/tools/call |
conformance.md wrapped MCP | a decision | -32011 |
FAIL |
| P13 | same nonce twice |
10.3 (SHOULD) | -32005 |
allow | observation, SHOULD |
| P14 | body > 1 MiB | tree's own cap | -32010 |
-32010 |
PASS (project choice, not spec) |
Corpus, vectors-acs-core with tools mapped to Bash
| status | n | members |
|---|---|---|
| PASS | 5 | v07180a3 R-007 accept, v3e946bc R-002 accept, va6d6e95 R-003 accept (-60 s, inside the window), vf3679e6 R-013 accept, vff6b4d4 R-011 accept |
| FAIL | 9 | vb87b86b and vd67cd20 R-001 (invalid / subset-coverage signature, allowed), vef655ce R-002 (replay, allowed), vb10df32 R-003 (-3600 s, allowed), v25f01d8 R-006 (no chain_hash, no signature on the response), v100f749 R-011, v661266c and vfcab242 R-012/R-013, vc333473 R-007 (intent, approval and approver identity ride beside the request in the corpus and have no wire field on steps/toolCallRequest; the Guardian never receives them and allows) |
| NOT_DRIVEN | 10 | framework/agent/deployment-role members with no wire request or whose property is the Observed Agent's behaviour (R-004, R-005 x2, R-008, R-009, R-010, R-013 x2 budget, R-014, R-015); each was still sent where it had a request and the Guardian's answer is on file |
| NOT_APPLICABLE | 2 | R-016 x2: acs-audit is not claimed |
| UNMEASURABLE | 2 | v82b6d11 (revocation, unmeasurable by the corpus's own declaration); v7b0b32f (signature-covers-chain_hash cannot be tested when neither field is on the wire) |
Run as written, with the corpus tools records.lookup and records.read and the argument id, every one of the 16 wire members came back deny with the reason code evaluation_failed and a message saying that mapping.yaml reads that tool's policy target from an argument the call did not send. See D10.
Defect table
Severity is against the ACS-Core MUST list. "One-line fix" names the function and the change; each is small because the surrounding machinery already exists (the chain is computed, the session store is keyed by session, validateEnvelope is total, errorResponse takes a code).
| id | what | where | reproduction | one-line fix | sev |
|---|---|---|---|---|---|
| D1 | No signature verification. An unsigned envelope and one whose HMAC is byte-inverted both receive a policy decision. The schema keeps params.signature optional and nothing in the Guardian reads it. |
packages/guardian/src/validate-envelope.ts (signature not required), server.ts dispatch (no verify step); handshake.ts negotiates no key |
P1 wire/*-P1-unsigned.json; P2 wire/*-P2-badsig.json; corpus vb87b86b, vd67cd20 |
In dispatch, after validateEnvelope and before the method switch: verifySignature(envelope, sessionKey(session_id)) over jcs(envelopeWithoutSignature), else errorResponse(rpcId, -32004, "SIGNATURE_INVALID"); key from HKDF(deployment IKM env var, session_id). |
MUST |
| D2 | No replay protection. The same request_id is accepted twice in one session; the same nonce too. |
server.ts evaluateStep -> appendContextEntry; session-context-store.ts already holds every request_id per session |
P3 wire/*-P3-replay.json; P13; corpus vef655ce |
In appendContextEntry (or just before it), if (session.entries.some(e => e.request_id === id)) throw new ReplayError() mapped to -32005. |
MUST |
| D3 | No timestamp window. +/-3600 s is accepted; the ServerHello never declares skew_window_ms. |
server.ts dispatch; handshake.ts buildServerHello |
P4, P5; corpus vb10df32 |
if (Math.abs(Date.now() - Date.parse(params.timestamp)) > SKEW_MS) return errorResponse(rpcId, -32006, ..., {skew_window_ms: SKEW_MS}); add skew_window_ms: 300000 to the ServerHello. |
MUST |
| D4 | Chain head not published. The chain entry is computed and written to disk, then dropped; the response carries no chain_hash and no signature, so 8.6 and the response half of 10 are both unmet. |
server.ts evaluateStep (ignores appendContextEntry's return), acs-result.ts finalResult |
P6 wire/*-P6-chain.json; the Guardian's own session-context.jsonl has the hash the response omits |
const entry = appendContextEntry(...) and finalResult(envelope.params, decision, entry.hash) setting chain_hash; then sign the result envelope with the same session key as D1. |
MUST |
| D5 | system/ping is refused with an ACS-specific error, the one thing section 13 says it MUST NOT do. #129 carries this defect on its own. |
server.ts dispatch, falls through to METHOD_NOT_DISPATCHED_CODE |
P8 wire/*-P8-ping.json |
Before the isToolCallRequest branch: if (envelope.method === "system/ping") return successResponse(envelope.id, {type:"final", acs_version, request_id, decision:"allow", payload:{status:"ok", echo: payload.echo, server_timestamp: new Date().toISOString()}}), and skip the chain entry. |
MUST |
| D6 | Handshake is not negotiated. A ClientHello declaring only 9.9.9 gets negotiated_version: 0.1.0; profiles_supported is never read and profiles_accepted never sent. The README states the ClientHello is not read; the consequence is that the version and profile rules in section 4 and conformance.md are unenforced. |
handshake.ts buildServerHello (takes no ClientHello) |
P9 wire/*-P9-version.json; P10 |
buildServerHello(clientHello, env): intersect supported_versions with ["0.1.0"], else -32001 with data.supported_versions; echo profiles_accepted: clientHello.profiles_supported intersected with ["acs-core"]. |
MUST |
| D7 | Four of the six minimum hooks are not dispatched. steps/sessionStart (and by the same code path userMessage, agentResponse, sessionEnd) return -32011. mapping.yaml already maps them to AGT points. |
server.ts dispatch (two predicates, then -32011) |
P11 wire/*-P11-sessionStart.json |
Add a third branch for the four mapped-not-evaluated methods that appends the chain entry and returns allow (the ServerHello already tells the client they are allow-by-default; answering them keeps the chain complete). |
MUST |
| D8 | Wrapped MCP not dispatched. | same | P12 | Route protocols/MCP/tools/call through the toolCallRequest assembler with tool.name from the MCP payload. |
MUST |
| D9 | Test suite needs two things the README "Verify" step does not say. (a) Without trash on PATH, 26 tests fail; README "Install" lists trash only for scripts/. (b) With trash-cli 7.2.0 installed, 7 tests in packages/conformance/test/{upstream-watch,surfaces}.test.ts still fail because their afterEach trashes several directories at ~1.6 s each against Bun's 5000 ms hook default; bun test --timeout 30000 is green. The workflow comment says the undeclared dependency "is tracked as an issue". |
README.md Install/Verify; packages/conformance/test/*.test.ts trashDir; package.json scripts.test |
01-bun-test.log, 02b-...log, 02c-...log |
README: bun add -g trash-cli@7.2.0 under Install and bun test --timeout 30000 under Verify; or "test": "bun test --timeout 30000" in package.json. (#110 reports the Windows half of the same class.) |
docs |
| D10 | An unregistered tool is denied by a mapping message, not a policy. Any steps/toolCallRequest whose tool is not in policy_target_argument.by_tool falls to default: command; a call with no command argument is denied evaluation_failed with a message about mapping.yaml. Fail-closed is the right posture, but the reason code says the Guardian broke, when what happened is that the deployment governs no such tool. Every corpus member as written hit this. |
packages/guardian/src/map-verdict.ts resolvePolicyTargetArgument; server.ts evaluateStep catch |
wire/*-v3e946bcfde26bbe2-step1.json, run as written |
In the catch, or before assembly: if the tool is neither in by_tool nor in the manifest's tools:, deny with reason_codes: ["tool_unregistered"] and a reason naming the tool, leaving evaluation_failed for real throws. |
low |
Observations that are not defects against the text:
OBS-1, a foreign endpoint at the Guardian URL. The README default port 8787 was held by an unrelated local HTTP service, so the Guardian exited EADDRINUSE. That is correct. The shim, pointed at 8787, returned permissionDecision allow and exit 0. Its audit entry classified the failure kind as transport, in the capture 03d-audit-foreign-endpoint-on-8787.jsonl.
Section 6.4 is satisfied, since the proceed is audited. One line is worth adding to the host README: a parsed-but-non-envelope body is classified as transport, so the audit log cannot separate a Guardian that is down from something else answering on that port. 8787 is a common default for local proxies.
OBS-2. P7 and P13 are MAY and SHOULD and are listed for completeness only.
OBS-3, the corpus's own gap. Nine corpus members carry Intent, an approval, or an approver response beside the request. steps/toolCallRequest has no field for any of them, so a Guardian that receives only the wire cannot be measured on R-007/R-011/R-012/R-013 by a host adapter; those rows in the FAIL bucket are a statement about the wire format as much as about this Guardian. That is recorded here so the count is not read as nine independent Guardian defects.
Where behaviour and the tree's own table agree, and where they do not
The README's ACS-Core status table already concedes replay, signature, ping, wrapped MCP, the four hooks and the chain head. This run confirms each concession on the wire and adds three the table does not carry: the handshake accepts any version (D6), system/ping is answered with an ACS-specific error where the specification says it MUST NOT be (D5), and the test suite's own verify step is not reproducible from the README (D9). D1 to D8 are one tree; D2, D3, D5 and D7 are each an afternoon because the state they need already exists.
Current Priority Scope
Feeds the runnable Guardian reference implementation (D1 to D8, D10) and feeds conformance evidence (the adapter, the probe set and the corpus run are reusable against any Guardian). D9 is documentation.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsAccepted
Activity
jrcrittenden commentedon Oct 6, 2026
A partial implementation is now available in #229.
It addresses the concrete D1-D10 wire probes in this report, including authenticated request and response envelopes, stale/replay rejection, published chain heads,
system/ping, handshake negotiation fields, the six-hook minimum, wrapped MCPtools/call, and explicittool_unregistereddecisions. The PR also adds adversarial and integration coverage and updates the adapter documentation.This is not a claim that the full issue is resolved. Known follow-up work includes persisting replay state across Guardian restarts, enforcing negotiated session capabilities on subsequent calls, and preserving chain continuity when the bounded session-context store evicts a session.
The change has passed local repository guards, the strict documentation build, AGT typechecking, and the focused changed-area tests. It has not yet been externally reviewed by a maintainer or independent reviewer, so the implementation and its conformance claims should be treated as proposed pending review. This issue should remain open while #229 is reviewed and the remaining gaps are assessed.