fix: remove hardcoded capability and evidence gates (#90)

* fix: remove hardcoded capability and evidence gates

* fix: reduce duplicated admission plumbing

* fix: centralize capability set assembly

Author: Brad-Edwards

docs/cyberbattlesim-researcher-command.md

@@ -19,6 +19,14 @@ raes-adapters inspect --backend cyberbattlesim-chain
 pinned native source is available and byte-verified, along with the qualified
 source commit and supported RAES profiles.
 
+Native `validate`, `run --mode smoke`, and `run --mode study` are currently
+fail-closed for the selected task. Its attacker-action-log and
+availability-series requirements cannot be verified at artifact-field and
+data-quality granularity by the pinned RAES contract. These commands exit `3`
+before runtime planning, native import, output reservation, or simulator
+effects. Issue #86 owns the CyberBattleSim capture/manifest remediation; the
+authored task is not weakened in the interim.
+
 The native simulator remains separately installed because upstream publishes no
 selected index or release artifact. Build or acquire the wheel from commit
 `854d6966607fb68645651f55b0f97221bd293e0d`, verify the complete identity

docs/decisions/capability-and-evidence-claim-guardrails.md

@@ -0,0 +1,122 @@
+# Capability and evidence-claim integrity guardrails
+
+GitHub issue #84 is the authority for this repository-wide correction. This
+note fixes the claim boundaries every adapter, shared helper, command, and
+conformance lane must respect. It defines no new RAES contract, capability,
+evidence type, status vocabulary, or implementation plan.
+
+## Keep four claims distinct
+
+An authored task requirement, a backend capability declaration, a captured
+run artifact, and a post-run satisfaction claim are different facts with
+different owners:
+
+- `ExperimentTaskModel` owns what evidence the author requires. A requirement
+  is demand, not proof that an adapter can capture it.
+- The published RAES `BackendManifest` and capability models own what the
+  selected production target advertises. A capability is admissible only when
+  its production component and capture path are executable; a constraint,
+  source-ledger row, injected driver, or planned feature is not a substitute.
+- `ExperimentEvidenceRecordModel`, `ExperimentDerivedMeasureModel`, and
+  `ExperimentArtifactRefModel` describe what one run actually emitted. A file
+  containing an evaluator summary is not an action log, availability series,
+  host-compromise series, or reward-component record merely because all of
+  them concern the same episode.
+- `validate_experiment_run_against_task()` owns the task/run satisfaction join.
+  Conformance, qualification, source admission, cleanup, or a successful run
+  cannot bypass that join or manufacture its inputs.
+
+The pinned RAES contract validates a semantic `satisfies_refs` entry by
+reference identity. `ExperimentEvidenceSatisfactionReferenceModel` cannot
+express required-field coverage or the availability, redaction, withholding,
+or loss status of those fields. Therefore a generic authored evidence concept
+must not be placed in `satisfies_refs` until the applicable published RAES
+contract can express and validate the complete artifact-and-field witness.
+Current researcher tasks that require such a concept must fail closed even if
+that makes an adapter or example temporarily non-runnable. An exact authored
+artifact identity remains admissible only where the existing RAES validator
+can verify its identity and any authored digest/path constraints directly.
+
+## Canonical incumbents
+
+| Concern | Canonical owner and required use |
+| --- | --- |
+| Manifest and capability shape | RAES `BackendManifest`, `BackendCapabilitySet`, component capability models, `backend_manifest_payload()`, capability-admission helpers, and `RuntimeTarget` presence/signature validation. Do not add an adapter capability schema or infer support from component existence alone. |
+| Runtime admission | `RuntimeManager.plan()`, public target components, `ApplyResult`, snapshot-transition validation, participant admission/history validation, evaluator result validation, and cleanup receipt validation. Every affirmative production claim must survive its applicable public execution path. |
+| Experiment evidence | RAES capture-spec, evidence-record, derived-measure, artifact, run, and task models plus `_experiment_evidence.py` and `_researcher_support.py` for mechanics only. Backend-local code owns source projection; shared helpers must not assign semantic evidence identities. |
+| Task/run joins | `validate_experiment_run_against_task()` and `validate_experiment_study_against_tasks_and_runs()`. Do not copy their join logic or predeclare a positive result in `_BackendAdapter`, a manifest constraint, or a pack file. |
+| Manifest conformance | `run_conformance_probe()`, canonical RAES report projection/writer, and executable adapter-local probes. `affirmative_capability_pointers()` is inventory only; a static pointer-to-reference table and one broad pass flag are not proof that each leaf was exercised. |
+| Source truth | Backend qualification records, source admission, scenario/source ledgers, and loss disclosures. These establish provenance and bounded source facts; they do not satisfy per-run capture requirements. |
+| Failures and disclosure | RAES `Diagnostic`/`DiagnosticModel`, `diagnostic_model()`, `ApplyResult`, stable command exit codes, and `base.redaction`. Missing, unavailable, withheld, redacted, lossy, unsupported, and failed must remain distinct non-success dispositions. |
+| Persistence | `atomic_write_json_artifact()`, the RAES conformance report writer, exclusive confined output roots, and inventory-last sealing. Runtime state remains behind RAES runtime/control-plane ownership; no evidence registry, cache, or adapter store is introduced. |
+| Verification | Existing repository tests, clean-installed distribution probes, the single nox graph, and the `PR Gate`. Negative tests must exercise every registered adapter and shared command path, not a hand-maintained subset that silently omits the next adapter. |
+
+Backend manifests may contain declarative values, but those values are not
+self-authenticating. Shared constructors such as standard evaluator,
+orchestrator, or cleanup capability builders may factor shape only after the
+caller supplies truth established by the production path. They must not grant
+support merely because several gym-style adapters are expected to share it.
+Likewise, reporting every PrimAITE capability as an open conformance gap does
+not make its affirmative production manifest truthful; an inadmissible adapter
+is allowed to break.
+
+## Validation and security path
+
+| Layer the design passes | Required treatment |
+| --- | --- |
+| Authentication and authorization | The current researcher command is local and adds no auth surface. Participant authority still comes from exact manifest/selection/configuration joins and public participant admission. Any later network surface must use the RAES strict-default control-plane security, verified identity, role/target authorization, request limits, denial audit, and redacted exception handling; adapters do not add endpoints. |
+| Secrets and environment bindings | No claim path reads credentials, a secret store, `.env`, or ambient configuration. Do not add token options or environment-selected capability/evidence overrides. Native credentials, action details, observations, rewards, argv, environment maps, and source paths never enter portable evidence or diagnostics. |
+| Static input and config shape | Reuse closed `argparse` choices, per-backend required/foreign argument checks, pack digest validation, confined child resolution, duplicate-key-rejecting JSON loading, RAES SDL parsing, closed contract models, participant joins, target config normalizers, and selected-source admission. No arbitrary import, driver, profile, schema, or capture map is caller-selectable. |
+| Runtime validators | Preserve manifest/component checks, capability admission, plan/resource/dependency validation, `ApplyResult` shape, snapshot transitions, participant action/result/history joins, evaluator/proposition checks, and cleanup verification. A native transition with an unverifiable projection is failure, not partial evidence satisfaction. |
+| OS and process exposure | Keep relative confined outputs, exclusive mode-0700 creation, atomic publication, no shell interpolation or runtime download, clean-install isolation, cleared `PYTHONPATH`, `PYTHONSAFEPATH=1`, and discarded native stdout/stderr. Do not put evidence payloads, credentials, or native paths in argv or filenames. |
+| Error envelopes and observability | Reuse bounded `_CommandFailure` messages, RAES diagnostics, canonical report projection, and default-deny redaction. Logs and terminal output may carry safe identities, pointers, counts, and dispositions only. Never serialize exception text, rejected values, native output, object representations, or tracebacks. |
+| Artifact publication | Validate RAES models and task/run joins before sealing success; write the final inventory last. A checksum proves byte identity, not semantic completeness or safety. Failure, cleanup failure, or an unsatisfied requirement cannot be published with a successful disposition. |
+
+## Extension seam
+
+The extension seam is the existing backend strategy boundary, parameterized by
+the live `BackendManifest`, the authored `ExperimentTaskModel`, and the actual
+validated evidence records/artifact bytes from that run. A future published
+RAES satisfaction contract may be consumed there without changing authored
+task semantics or adding a repository schema. Until that owner can validate
+field-level witnesses and negative data-quality states, the seam returns no
+semantic satisfaction claim and lets the canonical task/run validator reject
+the run.
+
+A future adapter registers with the existing command/target strategy and is
+automatically included by repository-wide claim-integrity tests. It must not
+require editing a global evidence allowlist, standard capability grant, copied
+schema, or backend-name conditional.
+
+## Gotchas and anti-patterns
+
+- Do not retain `evidence_satisfies_refs`, a backend-name evidence allowlist,
+  unconditional `supports_* = True`, or a test-only/injected-driver bypass.
+- Do not promote a source-ledger reference, capability pointer, conformance
+  evidence id, capture-spec declaration, content checksum, or evidence-record
+  existence into per-run satisfaction.
+- Do not let an evaluator summary satisfy an action/observation/time-series
+  requirement when those records and required fields were not emitted.
+- Do not treat missing, unavailable, redacted, withheld, lossy, unknown,
+  unsupported, partial, or unverified as aliases for satisfied.
+- Do not define a local field-witness DTO, evidence status enum, validator,
+  exception hierarchy, manifest extension, profile, registry, or persistence
+  service to work around a missing RAES contract.
+- Do not make positive tests depend only on current adapters. Mutation and
+  negative cases must catch a new manifest leaf, a new adapter registration,
+  an omitted artifact, missing required content, a negative data-quality state,
+  and a static reference reintroduced through any shared path.
+
+## Non-goals and boundaries
+
+Issue #84 does not implement missing action logs, availability or compromise
+series, reward-component capture, source qualification, deterministic replay,
+scientific equivalence, or new researcher backends. It does not rewrite
+authored task semantics merely to keep current examples runnable.
+
+It also does not add or change a RAES schema, capability vocabulary, evidence
+type, status model, validator, profile, diagnostic envelope, controller, store,
+HTTP surface, console script, distribution, lockfile, or workflow. If a
+published RAES contract cannot express and verify the required claim, the
+repository records the gap by failing closed rather than creating local
+authority.

docs/decisions/cyberbattlesim-conformance-guardrails.md

@@ -74,14 +74,12 @@ off as a copied fixture family. The published realization harness is used only
 for realization-envelope questions it actually models; it is not a generic hook
 for unrelated source-protocol assertions.
 
-Every affirmative manifest capability must join to passing executable evidence.
-Derive the capability addresses from `backend_manifest_payload()` at runtime and
-join them to stable adapter-probe evidence references. The join may be a small
-module-local test/probe inventory, but it is not portable authority: it carries
-only manifest JSON pointers and evidence references, contains no copied
-capability values or expected manifest payload, and fails closed when a newly
-declared affirmative capability has no passing evidence. Negative declarations
-and limitations must also be exercised so absence is not mistaken for support.
+Every affirmative manifest capability is derived from
+`backend_manifest_payload()` at runtime and retained as unresolved inventory.
+There is no module-local pointer-to-reference join: a broad conformance pass,
+source validation, or adapter probe cannot certify each leaf. Negative
+declarations and limitations must also be exercised so absence is not mistaken
+for support.
 Where RAES already provides `evidence_refs`, `limitation_refs`, claim
 `limitations`, or `explicit_non_claims`, reuse those fields rather than creating
 adapter equivalents.
@@ -98,7 +96,7 @@ constructed `RuntimeTarget` with an injected deterministic driver to run the
 published profile/fixtures and bounded local probes. It must cover manifest and
 source/profile identity, all four surfaces on success and failure, reset and
 stream dispositions, action/observation/evaluator separation, terminal
-semantics, cleanup, capability-to-evidence closure, and portable serialization.
+semantics, cleanup, unresolved capability inventory, and portable serialization.
 It must not import the native simulator or depend on network, user-home state,
 or an editable checkout.
 

docs/decisions/cyberbattlesim-qualification-guardrails.md

@@ -106,7 +106,8 @@ native-readiness plan must extend this same isolated source-native run instead
 of stopping at the upstream smoke. For issue #28, that means running the
 adapter conformance path with the real `CyberBattleSimDriver`, serializing the
 result through `backend_conformance_report_payload()`, collecting
-`cyberbattlesim_source_protocol_diagnostics()` and manifest capability evidence,
+`cyberbattlesim_source_protocol_diagnostics()` and unresolved manifest
+capability inventory,
 and checking the emitted RAES payloads remain bounded and free of native action
 ids, observations, reward vectors, object representations, paths, environments,
 tracebacks, and hidden state. This is adapter-readiness evidence in this repo;

docs/decisions/cyborg-conformance-guardrails.md

@@ -86,22 +86,21 @@ construction.
 | Packaging and CI | The single `pyproject.toml`/`uv.lock`, `_verification_envs()`, `_tests()`, `_distributions()`, `probe_installed_identity.py`, the existing CI workflow and `PR Gate`, strict docs/policy gates, and canonical `nox -s verify`. Extend these paths; do not create a second lock, workflow, or unenforced verification graph. |
 
 The CyberBattleSim conformance module is the closest repository precedent for
-canonical report projection, manifest-derived evidence closure, RAES
-diagnostics, weakness references, and hostile-value tests. Reuse its composition
-pattern, not its source protocol, action model, seed semantics, cleanup claims,
-capability inventory, or backend-specific helper module.
+canonical report projection, RAES diagnostics, weakness references, and
+hostile-value tests. Reuse its composition pattern, not its source protocol,
+action model, seed semantics, cleanup claims, capability inventory, or
+backend-specific helper module.
 
-## Local probes, capability evidence, and leakage
+## Local probes, capability inventory, and leakage
 
 Derive affirmative capability JSON pointers from the live
-`backend_manifest_payload()` and join them to stable references from probes that
-actually passed. A small module-local pointer-to-evidence requirement map is a
-closure check, not capability authority: it contains no copied capability
-values, fails when a new affirmative surface has no evidence, and does not turn
-constraints or component names into capabilities. Exercise negative and
-unsupported declarations as well, especially replay, autonomous execution,
-bounded concurrency, execution control, accounts, ACLs, generated artifacts,
-persistent volumes, and cleanup when undeclared.
+`backend_manifest_payload()` as unresolved inventory only. A broad published
+conformance disposition, source-ledger validation, or adapter-local probe does
+not prove every capability leaf, so there is no pointer-to-evidence requirement
+map or positive join. Exercise negative and unsupported declarations as well,
+especially replay, autonomous execution, bounded concurrency, execution
+control, accounts, ACLs, generated artifacts, persistent volumes, and cleanup
+when undeclared.
 
 Weaknesses are machine-resolvable references derived from qualification
 `admission.limitations`, `known_defects`, and the selected loss disclosures.
@@ -225,11 +224,12 @@ does not rewrite, append, or locally reclassify that case.
 The executable adapter-local probes separately construct a supported switch/VM
 topology with a hostile injected native handle, validate the source selection
 and every declared runtime surface, verify cleanup, and inspect actual portable
-projections. Capability evidence requires all three independent references:
-the bounded published conformance disposition, validated source-ledger evidence,
-and passing adapter-runtime diagnostics. A future RAES release that publishes a
-constructive list-domain or equivalent governed witness seam can replace this
-unsupported case without a local schema or profile.
+projections. The bounded published conformance disposition, validated
+source-ledger evidence, and passing adapter-runtime diagnostics remain three
+independent facts; their conjunction is not per-leaf capability evidence. A
+future RAES release that publishes a constructive list-domain or equivalent
+governed witness seam can replace this unsupported case without a local schema
+or profile.
 
 The checked-in full tier remains hermetic at ordered seeds `(3, 153)` and keeps
 `native_conformance=false`; the existing qualified-source reproducer owns native
@@ -247,8 +247,8 @@ registry, or environment binding.
 A new qualified selection may add source resources, evidence bindings, seeds,
 or a native harness without changing the RAES profile corpus, report type,
 diagnostic model, control-plane persistence, or cross-simulator base API. A new
-affirmative manifest claim automatically creates an evidence-closure gap until
-a passing probe reference is joined to its derived pointer.
+affirmative manifest claim automatically remains an unresolved inventory gap
+until a published contract can verify the owning production path.
 
 ## Gotchas and anti-patterns