Skip to content

Latest commit

 

History

History
1932 lines (1658 loc) · 115 KB

spec_artifact_storage_service.md

File metadata and controls

1932 lines (1658 loc) · 115 KB

Workstream Immutable Artifact Storage Specification

Status And Scope

This is the canonical target contract for WS-ART-001 under ADR 0013. Runtime schemas change only when their owning implementation chunk merges.

The specification covers provider-neutral immutable byte storage, independent verification, scoped original-guide access, submission/checker binding, Operator observability, and durable recovery. It does not implement review decisions, contribution, compensation, reputation, semantic search, public publication, physical deletion, or Flow Node.

Architecture

FastAPI and Celery composition roots
-> ExternalServiceAdapterFactory[ArtifactStoreBootstrap]
-> PostgreSQL exact namespace claim
-> initialized ArtifactStore
   -> LocalStorageAdapter
   -> S3CompatibleArtifactStore
      -> MinIO integration
      -> AWS S3 v0.1 production
-> ArtifactService
-> PostgreSQL records, bindings, receipts, audit, jobs, and recovery

Initial provider selection changes only composition-root configuration. Product services never import a concrete storage adapter. Replicas persist provider profile and storage namespace, so changing a populated deployment requires an explicit verified migration rather than a hot endpoint switch.

Only the internal ArtifactStorageOrchestrator receives the writable ArtifactStore port. Guide, task, submission, checker, and review owners call the closed capability ports defined below. Composition roots may construct the orchestrator but cannot expose it as a FastAPI dependency or Celery task parameter. Static architecture tests reject raw-port imports, orchestrator injection, and provider mutation calls outside the artifact-storage module.

Ownership

Concern Owner
authorization and exact resource permission Authorization Service
content identity, product provenance, binding, lifecycle, audit Workstream
verification result and recovery coordination Workstream/PostgreSQL
private immutable bytes and protocol observations configured object store
review packet manifests WS-REV using ART materialization
reviewer evidence artifact aggregates/uploads Future REV-owned intent; not approved for v0.1

Provider credentials grant transport access only. They are not actor identity, product role, product authorization, or proof of stored content.

Canonical Records

SubmissionBundlePreparation

One contributor operation accepts exactly one outer ZIP. The ZIP remains in the existing bounded ArtifactScratchManager through safe tree inspection, canonical archive/semantic-manifest identity, unchanged-work comparison, and mandatory platform/locked-guide checks. It is not candidate storage, carries no provider reference, cannot cross a process boundary, and is discarded on failure or process loss.

Only a passing preparation enters generic durable admission. The retired ArtifactUploadSession/ArtifactUploadItem staging path is absent from the current runtime and schema. Submission-bundle preparation enters generic durable admission directly after its bounded checks pass.

ArtifactStorageAdmissionLedger

A PostgreSQL-owned generic admission service prevents every ingest producer from creating unbounded durable storage. It is installed before guide, contributor, or checker-output cutover and is the only path to provider put or observe_put_result. Configured limits cover cumulative unique bytes at every applicable task, producer, project, and deployment scope. After Workstream prepares the complete source and knows its canonical SHA-256 and exact size, one transaction reserves every applicable scope charge before provider I/O.

The producer scope is the authenticated actor for caller-supplied bytes and the fixed service principal for system-generated bytes. Guide ingestion charges its project, producer, and deployment. Contributor uploads additionally charge the task. Checker outputs charge the originating task/project, checker service principal, and deployment; they never consume or blame the contributor's actor quota.

Each scope charge is unique by scope plus canonical content identity and has a CAS-protected state: provisional, completed, or released. provisional and completed count against quota. Exact replay returns the same charge, and a competing same-content operation cannot create a second charge. Provider acknowledgement or a fresh complete-object observation changes provisional to completed. A timeout or acknowledgement loss remains provisional until deterministic key recovery resolves it. Fresh authoritative absence changes it to released; an exact replay must atomically reacquire capacity before another provider call. A confirmed object, including quarantined or integrity-mismatched content that v0.1 cannot delete, remains completed. Concurrent transitions use row locks, uniqueness, and CAS so stale execution cannot release or finalize a charge.

Each charge records id, scope type/id, canonical SHA-256, exact byte count, producer type/reference, state, CAS version, reservation/completion/release timestamps, and the operation identity that created it. Limits are explicit configuration for task, producer, project, and deployment scopes; a missing, malformed, or nonpositive required limit fails startup rather than creating an unbounded scope. Product callers never supply scope collections. They submit one of the closed producer requests defined below; artifact orchestration loads the authoritative project, task, submission-bundle operation, checker-run, actor/service, and deployment facts, derives the complete AdmissionScopeSet internally, and commits all reservations or none. A missing canonical relationship fails before provider I/O rather than silently omitting a quota scope.

Operators inspect capacity through the bounded read-only admission-usage API, with exact scope filters and pagination. Metrics alert before each configured limit is exhausted. Capacity expansion changes reviewed configuration and follows the operations runbook; v0.1 never deletes or edits ledger charges to manufacture capacity.

ArtifactPutAttempt

One durable attempt owns the gap between Transaction A and Transaction B. Fields include producer request type/reference, producer admission or system output reference, content commitment, deterministic object identity, admission-charge references, operation/request digests, status, next-run time, executor UUID, database-clock lease expiry, execution generation, bounded terminal result, receipt/replica references, CAS version, and timestamps.

Statuses are prepared, put_in_flight, acknowledgement_unknown, object_confirmed, absent_replay_required, integrity_mismatch, provider_unavailable, and conflict. Transaction A creates prepared before provider I/O. Claiming the attempt sets put_in_flight with a fresh executor, lease, and generation. Provider acknowledgement completes Transaction B. Timeout, cancellation, process loss, or an expired in-flight lease leaves a durable resolution target; it never relies on an ArtifactReplica or ArtifactVerificationJob already existing.

A bounded PostgreSQL scanner publishes prepared, acknowledgement_unknown, and expired put_in_flight attempts. The resolver uses read-only observe_put_result, then opens and hashes any observed object. Matching bytes complete the original charges and Transaction B facts exactly once. Authoritative absence releases charges and moves the producer attempt to absent_replay_required. Mismatched bytes remain charged, quarantine the key, and create an incident. Terminal writes require matching executor and generation. No background resolver performs another provider write.

Submission-bundle preparation reserves bounded scratch through the existing scratch ledger, not a durable open-session slot. Failed or lost preparations release scratch without altering completed durable-byte charges. Provider I/O never starts until every pre-submit gate passes and every applicable durable scope has capacity.

ArtifactContent

An immutable provider-neutral fact identified by canonical SHA-256 and exact byte count. It also records detected media type and creation time. A display name is presentation metadata and is not content identity.

ArtifactReplica

One content copy in one adapter/provider. Fields include content ID, adapter name, immutable provider profile, immutable storage-namespace ID, opaque provider object reference, verification state, availability state, integrity state, latest verification receipt, and observation timestamps.

Replica states are independent closed fields:

verification_state = pending | verified | missing | integrity_mismatch
availability_state = unknown | available | unavailable
integrity_state    = unknown | valid | invalid

Transaction B sets the ArtifactPutAttempt to object_confirmed and creates the replica as pending/unknown/unknown. A matching complete read makes the admission bindable and replica verified/available/valid. A provider-unavailable or conflict job result does not fabricate a replica observation.

A confirmed absent object sets the replica to missing/unavailable/unknown. Before any binding exists, and only while the original producer attempt remains eligible, an exact replay by the original authorized producer may move that same replica to pending/unknown/unknown, set the attempt back to object_confirmed, append a new operation receipt, and create a new verification job. After a binding exists, the missing replica is terminal and cannot return to pending in v0.1. A digest/size mismatch sets the replica to integrity_mismatch/available/invalid, fails the pre-binding attempt when applicable, and quarantines access; it never returns to pending.

Provider references are internal. They never appear in contributor, reviewer, project-owner, or public API responses.

ArtifactStorageNamespace

One immutable deployment-level row fences every provider operation before any replica exists. Its singleton key is fixed. It records backend, adapter name, provider profile, a canonical non-secret namespace descriptor, and namespace_fingerprint = sha256(canonical_json(descriptor)). The descriptor contains the normalized provider identity needed to distinguish LocalStorage, MinIO, and AWS S3, including region, bucket or private-root identity, normalized endpoint identity where applicable, private prefix, and addressing style; it contains no credential or resolved secret.

Every descriptor also carries the reserved adapter, backend, and provider_profile keys. S3-compatible profiles then require exactly these additional keys; missing, unknown, or extra keys fail closed:

Provider profile Exact additional descriptor keys
minio-v1 addressing_style, bucket, endpoint_identity, private_prefix, region
aws-s3-v1 addressing_style, bucket, private_prefix, region

endpoint_identity is the non-secret SHA-256 identity of the normalized MinIO endpoint. It is required for minio-v1 and forbidden for native aws-s3-v1, which has no configured endpoint.

For LocalStorage, the durable root must already exist with owner-private permissions before application startup. Its descriptor stores only a hash over the normalized path plus filesystem device/inode identity. Replacing or remounting that root at the same configured path therefore changes the fingerprint. The typed factory constructs a non-mutating bootstrap that pins the root descriptor. PostgreSQL admits that exact namespace identity, then the bootstrap rechecks that the configured path still names the pinned descriptor before it creates layout files through that descriptor. Replacement before initialization fails without mutating either root; replacement after the final check cannot redirect descriptor-relative writes into the replacement root.

Startup inspects and pins only the configured namespace identity before the transaction. It and every provider operation transactionally insert-or-validate the singleton before layout mutation or artifact-byte I/O. A concurrent first writer claims the namespace; a loser with a different fingerprint fails before provider access. Replicas and put attempts reference the namespace row and fingerprint. A populated deployment cannot change it without a separately reviewed maintenance migration.

ArtifactProviderProbeResult

One append-only result from exactly one AWS proof executor. It records probe type, expected and STS-observed caller ARNs, release ID, storage namespace ID and fingerprint, AWS account/region, canonical bucket-policy digest, common challenge nonce, proof-suite version/result, bounded evidence digest, PostgreSQL start/completion times, and expiry. It contains no credential or raw provider response. Probe types are readiness, runtime_immutability, and negative_access.

ArtifactProviderActivation

One immutable production-release proof permits AWS S3 provider operations. It references one passing result of every probe type and records their shared release, storage namespace, AWS account/region, exact role ARNs, policy digest, nonce, proof version, PostgreSQL activation time, expiry, and superseded activation ID. It contains no credential. LocalStorage and MinIO never create probe or activation records.

Only the credential-free activation coordinator writes this record after every fixed live proof passes with identical bindings and an overlapping valid time window. Production startup and every AWS provider operation require one unexpired successful record matching the exact release, namespace, roles, policy, and proof version. Any mismatch or expiry fails before provider I/O.

ArtifactBinding

An immutable relation from content to project, resource type/id, logical role, actor/service attribution, scope version, timestamp, and optional superseded binding. Upload staging is not a binding. Replacement appends a new binding and preserves history.

ArtifactOperationReceipt

Append-only Workstream evidence for one immutable put acknowledgement. It links the exact producer admission/attempt and replica and records operation, idempotency key, request_digest, opaque provider_object_ref, replay observation, bounded outcome/details, attempt number, correlation ID, and created_at. Adapter and namespace identity resolve through the linked replica. A response digest, provider receipt, credentials, signed headers, endpoints, raw provider responses, and product secrets are forbidden.

ArtifactVerificationJob

A durable complete-object observation request. Fields include content/replica, expected digest/size, status, attempt count, publication timestamps, next run, automatic attempt limit, retry-exhausted timestamp, executor UUID, execution lease expiry, execution generation, terminal error, immutable receipt reference, and CAS version.

Statuses are:

pending
running
verified
missing
integrity_mismatch
provider_unavailable
conflict

Only pending, expired running, and non-exhausted provider_unavailable jobs may be re-published automatically. verified, missing, integrity_mismatch, and conflict are terminal and cannot be retried through the Operator route. Only an exhausted provider_unavailable job is eligible for an authorized recovery attempt that creates a new job. Every other status fails closed without creating an attempt, job, or audit success record.

ArtifactRecoveryAttempt

A reason-bound Operator authorization/audit envelope to retry one verification. Fields include requester actor/profile, authorization decision evidence, reason, recovery class, target type/id, source_verification_job_id, retry_verification_job_id, optional parent_recovery_attempt_id, client idempotency key, canonical request digest, status, terminal result, audit IDs, and timestamps. It has no Celery executor, lease, or generation; the retry ArtifactVerificationJob is the sole execution owner.

Statuses are requested, succeeded, and failed. One eligible exhausted provider_unavailable source has exactly zero or one recovery attempt across its lifetime, enforced by a unique source-job constraint. The retry job's terminal transaction updates the envelope and terminal audit. verified maps to succeeded; provider_unavailable, missing, integrity_mismatch, and conflict map to failed with the exact terminal code. The source job never changes. A subsequently exhausted retry job may be the source of a new chained attempt. The earlier source can never be reused, including after the first attempt succeeds or fails and regardless of a different idempotency key.

ArtifactStore Port

The asynchronous port is:

put(source: CommittedArtifactSource)
observe_put_result(commitment: ArtifactCommitment)
open(provider_object_ref, byte_range?)
head(provider_object_ref)

CommittedArtifactSource contains the server-computed canonical digest, exact size, media type, and bounded second-pass stream as one sealed value. Only the Workstream preparation service constructs it; callers cannot pair an arbitrary stream with an arbitrary digest. put returns an opaque provider reference and bounded protocol observation. observe_put_result is read-only: it resolves the deterministic object identity and returns a bounded observation without replaying a write. open returns a bounded asynchronous byte stream. head returns existence, exact provider-reported size when available, media type when available, and bounded metadata needed for protocol handling.

The port has no verify, retain, release, delete, list, signed URL, search, product binding, SQLAlchemy, audit, authorization, or provider-receipt lookup. Workstream verifies using open and owns receipts in PostgreSQL.

Product Capability Ports

Product modules receive only these narrow asynchronous capabilities:

GuideArtifactIngestPort.ingest(GuideArtifactIngestRequest)
SubmissionBundlePreparationCommand.prepare(SubmissionBundlePreparationRequest)
SubmissionAdmissionConsumptionPort.consume(SubmissionAdmissionConsumptionRequest)
GuideDocumentManifestPort.load(GuideDocumentManifestRequest)
GuideDocumentGrant.open(opaque_document_handle)
CheckerOutputBindingPort.bind_checker_output(CheckerOutputBindingRequest)
PostSubmissionMaterializationPort.materialize(PostSubmissionEvaluationRequest, consumer)
CheckerArtifactOutputPort.store(CheckerOutputArtifactRequest)
CheckerArtifactOutputPort.recover(CheckerOutputSelector)
ArtifactOperatorReadPort.list_bindings/list_replicas/list_receipts/
get_verification_job/get_recovery_attempt/list_audit_events/admission_usage(...)
ArtifactOperatorRecoveryPort.retry_verification(ArtifactRecoveryRequest)

Every protected durable mutation in this port family obtains fresh, action-specific authority at its protected transaction boundary. Opaque PreparedAuthorizationHandle values remain process-local implementation custody; checker-output store/recovery requests do not carry them. The hidden HTTP submission-bundle preparation request carries immutable ActorIdentityFacts, request and correlation IDs, contributor selectors, packet metadata, and an asynchronous byte source rather than a prepared handle; ART obtains separate prepared handles internally immediately before materialization and durable put intent. The typed methods fix their expected actions, and handles never enter route schemas, outbox/Celery payloads, provider interfaces, or serialized contracts.

CHECKERS owns the post-submit materialization and output-custody consumer ports in app.modules.checkers.api.materialization and app.modules.checkers.api.output_custody. ART imports those public contracts and supplies their concrete custody implementations through composition. CHECKERS execution does not import ART APIs or private ART modules to consume them.

GuideArtifactIngestRequest contains prepared authority, exact project, guide, guide-source snapshot and item IDs, logical role, and authorized byte source. The service resolves the server-bound media type from the locked snapshot item. Callers supply no media type, digest, size, content ID, or provider reference. The guide manifest resolves committed original-document metadata for one exact project/guide/snapshot/setup generation. Its opaque GuideDocumentGrant opens only assigned document handles, rechecks fixed-reader authority and current source custody, and releases verified bytes through bounded scratch. The final buffer must match the original's SHA-256 and size immediately before it is yielded. No guide-source binding or extraction materialization request exists in the current execution path. SubmissionBundlePreparationRequest is exported only from app.modules.artifacts.api. It contains authenticated actor/identity-link facts, request and correlation IDs, contributor task/assignment/predecessor and idempotency selectors, packet metadata, and one outer ZIP byte source. It exposes neither raw authorization context nor a prepared handle. There is no upload-session compatibility port. PreparedBundleMaterializationRequest is internal and process-local; it wraps opaque prepared authority, the exact task/assignment context, the current PreparedArtifact generation, the effective plan, inspection, manifest, change gate, packet, and exact policy/checker selectors. It is not a product capability port and may not cross a route, background execution, provider, or module public boundary. PostSubmissionEvaluationRequest selects one immutable Submission, its exact binding/content, locked context and evaluation generation. ART resolves the consumed admission and admitted verified replica through its own records plus the TASK-owned SubmittedBundlePort. It verifies exact bytes and the canonical manifest before an async consumer receives a scoped read-only file view. The typed result carries material custody plus the existing closed evaluation value; it is not a durable checker result. ARCH-04D2 requires exact live service authority and the current CHECKERS execution lease before protected reads, then revalidates the same receipt after I/O. CheckerOutputArtifactRequest contains a CHECKERS-owned reservation selector and generated byte source. store obtains fresh authority for each phase, resolves the exact evaluation/run/checker-worker-lease/output-slot facts, applies the owner-declared media type and byte ceiling, and uses generic ART put and independent verification. recover(selector) uses the same current owner and authority facts to recover the stable operation without a byte source; it never puts or regenerates bytes. The current production composition supplies an zero-slot reservation port and deny-only output authority, so neither method is a live evaluator capability. The Submission binding is instead the terminal result of the typed SubmissionAdmissionConsumptionPort: it consumes one ready admission against the exact TASK-supplied Submission identity and locked lineage. The remaining checker-output binding request contains the same exact selector plus the put attempt and independent verification receipt identities. A flush-only ART participant re-resolves current reservation and authority, verifies complete receipt/job/attempt/replica/content ancestry, and creates or replays one immutable run/slot binding in the caller's transaction. Each typed method fixes its own authority; no request selects the action. ArtifactRecoveryRequest contains the authenticated Operator context, exact source verification-job ID, reason, client idempotency key, expected source-job CAS version, and canonical authorization resource facts. Reason-bound Operator recovery remains on its existing bounded authorization path and is intentionally outside the prepared durable-mutation interface cut. The read port's admission-usage request contains only bounded scope filters and pagination. It returns current reserved/completed byte usage and configured limits; it cannot release charges or mutate capacity.

The binding lookup resource vocabulary is exactly project, project_guide, guide_source_snapshot, guide_source_snapshot_item, task, submission, or checker_run. Review packet lookup remains deferred until WS-XINT-002-07A. Review/response evidence binding is future/planned and unavailable; 07B is reserved pending separate REV-owned intent. The audit resource vocabulary is exactly artifact_binding, artifact_content, artifact_replica, artifact_receipt, artifact_verification_job, or artifact_recovery_attempt. Adding a product or artifact resource requires an explicit contract change; callers cannot submit an unregistered string.

None accepts an adapter, provider object reference, storage namespace, caller- assembled quota scopes, a resource type outside the closed binding/audit vocabularies, or caller-selected server content ID. The implementation resolves canonical relationships, authorization resources, producer identity, and admission scopes inside artifact orchestration. The internal orchestrator is not itself a product capability. Operator routes receive the read or recovery capability they require, never the raw store or broad orchestrator.

FastAPI and Celery composition roots construct the selected non-mutating ArtifactStoreBootstrap through ExternalServiceAdapterFactory[ArtifactStoreBootstrap], claim its exact namespace in PostgreSQL, and only then initialize the byte-only ArtifactStore. They construct one internal orchestrator and expose only the required narrow port to each route/service/task. Architecture tests parse imports, constructor annotations, dependency providers, and Celery task parameters; checking only textual put call sites is insufficient.

Range Semantics

A byte range has zero-based offset and optional nonnegative length with an exclusive end. Offset equal to object size returns an empty stream. Offset past the end is invalid. A partial read never establishes complete-object integrity.

Stable Adapter Errors

The port maps provider behavior to stable typed errors:

artifact_storage_unavailable
artifact_object_missing
artifact_precondition_failed
artifact_input_mismatch
artifact_integrity_failure
artifact_range_invalid
artifact_operation_conflict
artifact_limit_exceeded

No error exposes a bucket, provider key, endpoint, credential, signed header, or raw provider body.

S3-Compatible Adapter

The concrete production adapter is named S3CompatibleArtifactStore. AWS S3 is its only v0.1 production provider. MinIO proves the same protocol contract in local integration tests and CI. Cloudflare R2 is deferred and has no active runtime profile or credential contract.

The v0.1 AWS profile supports global general-purpose buckets only. Workstream does not emit the account-regional namespace request header, so account-regional bucket names ending in -an fail the canonical bucket gate rather than being accepted with incomplete request semantics.

Configuration includes:

  • provider profile aws_s3 or minio;
  • explicit AWS or configured MinIO region;
  • omitted endpoint URL for native AWS S3 and required endpoint for MinIO;
  • dedicated private Workstream artifact bucket and private prefix; the bucket contains no other application's objects;
  • addressing style;
  • credential mode, local/CI MinIO static credentials held only as redacted private settings, and no resolved credential persistence;
  • connect/read/write/pool timeouts;
  • total verification deadline and maximum buffered bytes.

Backend is exactly disabled, local, or s3_compatible. Credential modes are aws_workload_identity and local_static. AWS S3 production uses aws_workload_identity; MinIO supports local_static only in local/CI. Production static credentials are forbidden.

An AWS S3 deployment selects exactly one workload-identity method: roles-anywhere, assume-role-with-web-identity, container-role, or iam-role. Workstream validates the configuration required by the selected method, constrains the pinned botocore credential resolver to that provider before any provider is loaded, and verifies the resolved method. Explicit credentials, environment access keys, shared credential/config files, arbitrary credential processes, login/SSO, obsolete EC2/Boto credential sources, and every unselected workload provider are rejected. Roles Anywhere invokes only the pinned AWS signing helper with fixed arguments and private certificate/key files; its bounded output remains in memory and its temporary credentials refresh automatically. It does not use an ambient credential_process profile or a metadata server. Web-identity STS, ECS container metadata, and EC2 instance metadata use explicit no-proxy sessions. Web identity also rejects ambient endpoint and CA overrides. Tests poison every nonselected source and prove that it is never read, executed, or contacted. This is an allowlisted workload-identity contract, not authority to use the ambient SDK default chain.

The current async S3 SDK pair is pinned to aiobotocore==3.8.0 and botocore==1.43.3. The bounded uplift preserves the real MinIO protocol suite in the OmniCore dependency environment. An SDK upgrade is a reviewed contract change because credential-provider precedence and refresh windows are operational behavior.

Runtime Eligibility After Chunk 02B1

minio is the only active s3_compatible profile in this chunk. It requires an explicit HTTP(S) endpoint, region, dedicated bucket, private prefix, local_static credentials, and a local/development/test environment. Startup still claims the exact minio-v1 namespace in PostgreSQL before any object operation. Local and CI run the real digest-pinned MinIO server and provision only the dedicated test bucket; tests do not emulate S3 calls.

local/CI MinIO names a non-production provider-eligibility class, not a literal endpoint-host rule. Repository-managed Compose and CI publish MinIO on host loopback only. A separately containerized Workstream process may use an operator-controlled private MinIO endpoint in a local, development, or test environment. Publicly exposed MinIO and every production-environment MinIO configuration are outside v0.1; neither can satisfy native AWS live proof.

aws_s3 accepts only a native endpoint-less AWS configuration, explicit region, exactly one allowlisted workload-identity method, an immutable release digest, exact runtime/readiness/negative role ARNs, exact runtime/readiness managed-policy ARNs and the expected canonical bucket-policy digest. Startup claims the exact namespace, then loads a matching fresh activation before credential-environment inspection or provider I/O. The adapter resolves its isolated credential session lazily only after the guard succeeds. Every later AWS SDK call reloads the same activation ID selected for that logical store operation. MinIO conformance is not AWS activation evidence.

Cloudflare R2 has no v0.1 runtime mode, credential issuer, sidecar, secret contract, configuration profile, or deployment proof. Any future provider initiative must perform current discovery and pass the same ArtifactStore conformance suite before any runtime configuration is introduced.

Production rejects local, plaintext HTTP, loopback/local endpoints, invalid credential-mode/profile combinations, and unknown provider profiles. Validation errors, resolved credentials, and exception object graphs must not retain secrets.

Object Identity

Production writes require a sealed CommittedArtifactSource before provider I/O. The object key uses its server-computed canonical digest:

<private-prefix>/sha256/<hex[0:2]>/<hex[2:]>

The key contains no actor, customer, project, task, filename, media type, or secret. Workstream stores it as an opaque provider reference.

Conditional Write

The adapter uses conditional no-overwrite semantics. A precondition failure is an exact replay candidate, not success. The adapter heads and then opens the existing object; Workstream independently verifies it before reuse.

The dedicated AWS bucket policy also denies s3:PutObject when the If-None-Match header is absent; S3 requires the present value to be * for this operation. Adapter conformance is not the immutability boundary by itself. The production harness attempts an unconditional overwrite with the runtime role, requires an explicit denial, and proves the original object bytes remain unchanged.

v0.1 uses conditional single-request PutObject only and enforces expected_size <= 512 MiB before I/O. Multipart upload, CopyObject publication, and provider-specific finalization algorithms are deferred. A future multipart initiative must prove one atomic no-overwrite and recovery algorithm against AWS S3 before changing this limit.

The runtime principal is restricted to the dedicated bucket and completed- object prefix and only the actions required for conditional put, head, get, and trustworthy missing-object classification. It has no delete, copy, object-list operation in Workstream, bucket administration, lifecycle mutation, public- access mutation, or cross-prefix object permission. AWS uses a least-privilege role/policy.

Production activation uses these exact resource templates:

BUCKET_ARN = arn:${AWS_PARTITION}:s3:::${BUCKET}
OBJECT_ARN = arn:${AWS_PARTITION}:s3:::${BUCKET}/${PRIVATE_PREFIX}/sha256/*
RUNTIME_ROLE_ARN = configured Workstream runtime IAM role ARN
RUNTIME_POLICY_ARN = the one configured customer-managed runtime policy ARN

The principal/action matrix is closed:

Principal Exact allowed IAM actions Exact resource
Workstream runtime role s3:PutObject, s3:GetObject OBJECT_ARN only
Workstream runtime role s3:ListBucket BUCKET_ARN only
deployment readiness role s3:GetBucketPolicy, s3:GetBucketPolicyStatus, s3:GetBucketAcl, s3:GetBucketPublicAccessBlock, s3:GetBucketOwnershipControls, s3:GetBucketVersioning, s3:GetLifecycleConfiguration, s3:GetEncryptionConfiguration BUCKET_ARN only
deployment readiness role iam:GetRole, iam:ListRolePolicies, iam:ListAttachedRolePolicies the exact configured runtime, readiness, and negative role ARNs only
deployment readiness role iam:GetPolicy, iam:GetPolicyVersion the exact configured runtime and readiness managed-policy ARNs only
deployment readiness role access-analyzer:ValidatePolicy, access-analyzer:CheckAccessNotGranted, access-analyzer:CheckNoPublicAccess * because these policy-check APIs do not support a narrower resource in this contract
deployment negative-test role no S3, IAM, or Access Analyzer action none
infrastructure bootstrap principal no Workstream-defined allow statement and no proof credential; environment-owned provisioning only outside the Workstream runtime/probe policy

HeadObject uses the runtime role's s3:GetObject permission. AWS returns 404 for a missing key only when the caller also has s3:ListBucket; without it, a missing key is masked as 403. The bucket-level permission therefore exists only so HeadObject can produce authoritative absence for this dedicated bucket. ArtifactStore has no list method and Workstream never calls ListObjects or ListObjectsV2. A 403 is always provider_unavailable, never missing. The IAM permission technically permits the runtime principal to enumerate this dedicated bucket if that principal is compromised; AWS offers no narrower permission that also enables missing-key 404 classification. The bucket and opaque hash-only key namespace are therefore mandatory containment boundaries.

sts:GetCallerIdentity is called by each proof executor to bind observed identity and requires no identity-policy grant. The runtime policy contains exactly the three listed S3 actions on their exact resource classes. It contains no wildcard S3 action and no DeleteObject, DeleteObjectVersion, CopyObject, ACL, tagging, multipart, bucket-administration, lifecycle, or public-access action. The readiness role contains no S3 object action. v0.1 uses bucket-default SSE-S3 (AES256), which the readiness probe verifies, and grants no KMS action.

The bucket-policy deny matrix is closed:

Sid Effect Exact actions Exact resources Exact principal Exact condition
DenyInsecureTransport Deny s3:* BUCKET_ARN, OBJECT_ARN * Bool: {"aws:SecureTransport": "false"}
DenyNonRuntimeObjectData Deny s3:* OBJECT_ARN * ArnNotEquals: {"aws:PrincipalArn": RUNTIME_ROLE_ARN}
DenyUnconditionalPut Deny s3:PutObject OBJECT_ARN * Null: {"s3:if-none-match": "true"}

No additional statement may weaken, bypass, or create an exception to these denies. For DenyUnconditionalPut, SigV4 and the S3 API require the present If-None-Match value to be * for this operation.

Identity policies therefore cannot grant an unexpected same-account, same-organization, or external role object access. The deployment-only readiness proof rejects any extra runtime/readiness allow action, resource, attached policy, inline policy, or bucket-policy exception. It validates the canonical policy documents, all four Block Public Access flags, bucket-owner-enforced object ownership, ACL state, SSE-S3 encryption, lifecycle safety, and the three deny statements. Access Analyzer custom checks prove no public access and no access outside the exact actions/resources above. Live anonymous, negative-test-role, and readiness-role GetObject, PutObject, and DeleteObject calls against a known object must all deny. Bootstrap-principal denial is proved through policy evaluation only; bootstrap credentials are never supplied.

The same deployment proof reads the provider lifecycle configuration and fails activation when any enabled rule can delete/expire a completed object under the configured private prefix. AWS inspection covers Expiration and NoncurrentVersionExpiration for every rule whose filter can intersect the prefix. Storage-class transitions and abort-incomplete-multipart rules are not completed-object deletion. These checks use the separate read-only deployment identity in the live proof harness; the runtime principal is not granted bucket administration merely to inspect policy or lifecycle configuration. Provider-readiness inspection is not part of ArtifactStore and is not called by a product service.

AWS proof uses separate executors and append-only ArtifactProviderProbeResult records. Each process uses only its own identity. Roles Anywhere additionally attempts both other roles and requires explicit access denial; obtaining credentials for another role fails the proof:

Probe Execution identity and boundary
readiness independent deployment job with the short-lived readiness role; policy/ACL/public-access/lifecycle/IAM inspection only
runtime immutability command executed inside the deployed Workstream runtime workload identity; conditional write, unconditional-overwrite denial, missing-key 404, head/get, and byte-preservation proof only
negative access independent deployment job with the short-lived negative-test role; known-object read/write/delete denials only

The readiness role cannot assume the runtime or negative-test role. The runtime workload cannot assume either deployment role. Readiness reads each exact role trust policy and requires its configured canonical digest and three disjoint workload subjects. Supported closed trust shapes are:

  • OIDC: one exact Allow with sts:AssumeRoleWithWebIdentity, exact issuer sub and audience sts.amazonaws.com under StringEquals.
  • EC2: runtime only, one condition-free sts:AssumeRole Allow for ec2.amazonaws.com, with provisioner-owned instance-profile attachment.
  • Roles Anywhere: one exact service trust for rolesanywhere.amazonaws.com, the required AssumeRole/TagSession/SetSourceIdentity actions, exact source account and trust-anchor ARN, and exact issuer/role-specific subject CNs. One shared profile permits exactly the three configured roles. Readiness verifies its enabled state and canonical security settings, and the enabled trust anchor's CA material. Each certificate must acquire its own credentials, receive explicit access denial for both other roles, then reacquire its own credentials. Other helper failures cannot stand in for denial.

For Roles Anywhere, both probe and activation records also retain the exact anchor/profile ARNs and canonical state digests. The four fields are all present for that method or all absent for the other supported methods. Group matching and every activation lookup include them. An updated CA or profile requires matching fresh proof; retained records are not rewritten. Wildcard or broader trust, role chaining and extra trust conditions/statements fail closed. The bootstrap principal is never supplied to any probe. Each probe asks STS for its observed caller ARN and commits one immutable result bound to probe type, expected/observed caller ARN, release ID, namespace fingerprint, bucket-policy digest, common challenge nonce, proof version/result, bounded evidence digest, PostgreSQL start/completion times, and expiry. A probe whose observed ARN differs from its expected ARN fails without an activation.

A credential-free activation coordinator has database authority but no AWS credentials. It creates one immutable ArtifactProviderActivation only after loading the three committed probe results and proving identical release, namespace, policy digest, nonce, proof version, time window, and unexpired pass results. Policy/IAM evaluation of the bootstrap principal is a fourth required readiness artifact; bootstrap credentials remain absent.

The runtime result must complete before the readiness and negative results start. The coordinator and activation-table trigger both enforce runtime.completed_at <= readiness.started_at and runtime.completed_at <= negative.started_at. This makes every known-object denial depend on the runtime proof having already retained the common-nonce challenge, rather than accepting the 403 that S3 can return for a missing key when a principal lacks ListBucket.

The proof command accepts no pass value or caller-authored observation, and no product/API/configuration input can write proof or activation rows. The trusted deployment proof process, its database writer credential and database administrators are the v0.1 trust root for those rows. v0.1 does not claim resistance to a malicious holder of that credential; a distinct writer role or external attestation would require a separately reviewed trust-boundary change.

Each five-minute group retains one immutable 49-byte challenge under the exact production content-addressed prefix. These objects create no product binding or receipt, but deliberately exercise the production object policy and no-expiry lifecycle. At that cadence they add 105,120 objects and 5,150,880 payload bytes per 365-day year; hosted capacity monitoring includes them, and orphan cleanup does not delete them.

Production startup and every AWS provider I/O load an exact matching unexpired activation before calling AWS. v0.1 caps probe/activation validity at 15 minutes and schedules all three probes plus coordination every 5 minutes. The complete runtime, readiness, negative-access and coordination sequence has a fixed 120-second v0.1 allowance. Native AWS configuration requires max(operation_total_deadline, complete_read_deadline) + persistence_margin + clock_safety_margin + proof_sequence_allowance + refresh_interval <= activation_validity. The coordinator and activation-table trigger require the authoritative database clock at activation insertion minus runtime.started_at to remain within proof_sequence_allowance; the live deployment also measures that interval, successive runtime start times and commit latency. At maximum budgets, 300-second proof starts and a 120-second sequence leave no scheduling or commit-latency slack. Deployments must start earlier or configure smaller operation deadlines for margin; the arithmetic is not a continuous-availability guarantee. A late or missed group fails closed. Expiry or any release, namespace, role, policy-digest, or proof-version drift fails closed with artifact_provider_live_proof_required before provider I/O. MinIO protocol conformance never activates production AWS.

This bounded proof model treats authorized AWS infrastructure administrators as trusted during the maximum 15-minute validity window. v0.1 does not claim protection from a malicious cloud administrator and does not require S3 Object Lock; changing that threat boundary requires a separate human-approved architecture decision.

Before each AWS call, PostgreSQL time must show activation_remaining_ttl >= operation_total_deadline + persistence_margin + clock_safety_margin. The terminal transaction reloads the same activation and requires it still to match and remain unexpired before committing any success, receipt, replica, verification result, or audit success. If a put may have succeeded but activation expires before terminal commit, the durable put attempt remains acknowledgement-unknown and can only be resolved after fresh activation; no stale proof authorizes a terminal fact.

Integrity Rules

Workstream computes SHA-256 and byte count while ingesting and while performing the independent verification read. ETag, Content-MD5, provider checksum headers, and user metadata are not canonical integrity facts.

The shared adapter uses only behavior proved by LocalStorage/MinIO conformance and the AWS live readiness profile.

LocalStorage Adapter

LocalStorage implements the same port and conformance vectors. It uses a pre-provisioned private configured root, opaque content-derived paths, exclusive no-overwrite publication, bounded off-event-loop file I/O, no-follow path handling, private permissions, full read verification, and sanitized failures.

Startup accepts only an empty provider-temporary directory or the exact sealed two-link state left after immutable publication and before temporary unlink. A single-link writable or sealed provider temporary blocks startup with an operator-cleanup error; initialization does not delete it because another live process may still own that file.

Local development recovery requires a maintenance window in which every API and Celery process using that local root is stopped. The Operator may remove or move a file from tmp/ to an owner-private quarantine directory outside the artifact root only after verifying that its name matches the canonical .put.<32 lowercase hex>.tmp shape, it is a regular file owned by the service account, it has exactly one link, and its mode is 0600 or 0400. Workstream must then restart and revalidate the complete layout. Any different name, owner, type, link count, or mode is an integrity incident and must not be repaired by this procedure.

The initial v0.1 provider contract has no retain/release methods or provider-owned receipt metadata. No compatibility adapter or dual format exists.

Ingest Choreography

  1. Authorization Service permits the exact upload action and resource.
  2. Workstream writes the complete untrusted source to bounded private scratch, computes digest/size, and rejects a mismatched client commitment before any provider call.
  3. Transaction A atomically claims or validates the deployment storage namespace, derives and reserves every applicable durable-storage scope charge as provisional, links the authoritative producer source reference, and commits the server-computed digest, size, media type, operation identity, canonical request digest, exact limits, one ArtifactPutAttempt, and CAS. Any quota, relationship, or namespace failure rolls back the complete transaction.
  4. Workstream passes the sealed CommittedArtifactSource from the artifact orchestration service to the adapter outside the transaction.
  5. The claimed attempt is set to put_in_flight; the adapter conditionally stores or resolves an exact replay candidate.
  6. Transaction B validates the reservation/CAS, completes every provisional charge, records content, replica, and operation receipt, sets the ArtifactPutAttempt to object_confirmed, and sets the replica to pending/unknown/unknown.
  7. The transaction creates an outbox/publication obligation for one verification job.
  8. Celery independently opens and hashes the complete object.
  9. A fenced transaction records the immutable verification receipt and marks the replica bindable only when digest and size match.

No provider call occurs inside a PostgreSQL transaction. No product binding is created before independent verification.

Chunk 02C1 implements step 3 only. Chunk 02C2 implemented hidden steps 4 through 9 with caller-only committed-source writes, read-only put resolution, durable verification jobs, typed receipts, and PostgreSQL fencing. WS-XINT-002-03 now connects those paths to transaction-bound fixed-service authorization and one bounded pending-work Beat scan. Operator routes and product cutover remain independently gated.

Acknowledgement loss leaves the durable put attempt and charges provisional. The attempt scanner publishes resolution after an ambiguous outcome or expired execution lease. Resolution performs a fresh read-only observation and full hash. A confirmed object completes the same charges and Transaction B facts. Fresh authoritative absence releases them and moves the ArtifactPutAttempt to absent_replay_required. Replay must atomically reacquire every applicable charge before another provider call. Integrity-mismatched or quarantined existing bytes remain completed and charged.

Prepared Byte Sources

Contributor uploads, authorized caller-supplied guide bytes, and generated checker logs/outputs do not have a trusted server commitment before production. They all use a bounded two-pass PreparedArtifact boundary: first write to a private process-owned ephemeral scratch file while hashing/counting and enforcing the same 512 MiB per-object limit; close the file; compare a commitment only for producer contracts that explicitly permit one; then seal the computed commitment and second-pass stream together as CommittedArtifactSource.

Guide ingest never permits a caller digest or size commitment. The raw upload must match the snapshot item's server-bound declared PDF/DOCX/PPTX or text/markdown media type; format inspection validates the actual file instead of trusting a header alone. Markdown admission requires byte-preserved UTF-8 without NUL and does not parse, render or rewrite the original. Preparation computes the content digest and byte count. Durable admission locks the exact project/guide/snapshot/item, requires a draft guide, and revalidates artifact.guide_source.ingest. A committed document_stored receipt, or exact confirmed observation after acknowledgement loss, establishes that original's readiness. Once every assigned original is committed, the existing setup generation can be queued. Shared submission verification is a separate workflow.

Current guide inputs are document uploads plus ordinary task-example text stored with guide metadata in PostgreSQL. URL fetching and durable URL source selectors are unavailable. Any future external-source adapter requires its own approved scope and retrieval controls; it is not a fallback for current guide input. Retained binding/extraction evidence is read-only and cannot drive new setup.

The scratch manager uses one private dedicated root and a cross-process locked reservation ledger. Because the first pass does not know actual size, every preparation atomically reserves the full 512 MiB per-object maximum before file creation. Configuration fixes aggregate reserved bytes, file count, preparation concurrency, reservation TTL, and a minimum-free-space floor below the volume quota. Exhaustion or ENOSPC fails with a stable infrastructure error and no provider call. One total preparation-plus-upload deadline is strictly shorter than the reservation TTL by a cleanup margin; v0.1 has no scratch heartbeat or lease renewal. Scratch files use private permissions, no-follow path handling, random opaque names, and are never authoritative or referenced by product records. The owning job releases its reservation after in-flight I/O closes. The named periodic Celery Beat cleanup task removes a reservation/file only while holding the ledger lock and only after its database-independent wall-clock expiry; a conforming live operation must already have cancelled before that point. Startup performs the same idempotent cleanup. Chunk 02A2 implements the cleanup mechanics without activating them. Chunk 02A3 wires API-startup cleanup and one named Celery Beat task as the periodic owner. Tests cover concurrent processes, full-max quota boundaries, low disk, slow active work versus cleanup, deadline cancellation, crash, stale cleanup, and symlink/non-regular entries.

The preparation settings use the standard WORKSTREAM_ environment prefix:

Environment variable Default Contract
WORKSTREAM_ARTIFACT_SCRATCH_ROOT unset Dedicated private scratch root; required before runtime activation and must not equal, contain, or sit inside WORKSTREAM_ARTIFACT_LOCAL_ROOT.
WORKSTREAM_ARTIFACT_SCRATCH_AGGREGATE_RESERVED_BYTES 2147483648 Maximum bytes reserved across the shared root. Each active preparation reserves the full 512 MiB hard maximum.
WORKSTREAM_ARTIFACT_SCRATCH_MAXIMUM_FILES 8 Maximum live scratch reservations/files.
WORKSTREAM_ARTIFACT_SCRATCH_MAXIMUM_CONCURRENCY 4 Maximum concurrent preparations; cannot exceed the file limit.
WORKSTREAM_ARTIFACT_SCRATCH_MAXIMUM_WORKSPACE_ENTRIES 2000 Maximum entries removable from one private processing workspace; it must be at least the accepted submission-ZIP entry limit.
WORKSTREAM_ARTIFACT_SCRATCH_MINIMUM_FREE_BYTES 536870912 Free-space floor retained in addition to the next full reservation.
WORKSTREAM_ARTIFACT_SCRATCH_RESERVATION_TTL_SECONDS 3900 Database-independent wall-clock expiry for abandoned reservations.
WORKSTREAM_ARTIFACT_PREPARATION_TOTAL_DEADLINE_SECONDS 3400 Total first-pass and provider-consumption deadline, aligned with the maximum selected POST execution budget.
WORKSTREAM_ARTIFACT_SCRATCH_CLEANUP_MARGIN_SECONDS 300 Required margin between the total deadline and reservation TTL.
WORKSTREAM_ARTIFACT_SCRATCH_CLEANUP_INTERVAL_SECONDS 300 Celery Beat cadence for the named stale-scratch cleanup task; accepted range is 1 through 86400 seconds.
WORKSTREAM_ARTIFACT_PENDING_WORK_SCAN_INTERVAL_SECONDS 60 Celery Beat cadence for one authority-bound, database-cutoff pending-work page.
WORKSTREAM_ARTIFACT_PENDING_WORK_SCAN_PAGE_SIZE 100 Hard combined put-attempt and verification-job publication page bound.
WORKSTREAM_GUIDE_SETUP_CONTINUATION_SCAN_PAGE_SIZE 100 Independent bound for publishing terminal verified guide-setup continuations.
WORKSTREAM_ARTIFACT_EXECUTION_LEASE_SECONDS 900 PostgreSQL-clock executor lease with no heartbeat.
WORKSTREAM_ARTIFACT_COMPLETE_READ_DEADLINE_SECONDS 600 Total deadline covering provider-open acquisition and the complete stream.
WORKSTREAM_ARTIFACT_TERMINAL_PERSISTENCE_MARGIN_SECONDS 120 Lease time reserved for the terminal fenced transaction.
WORKSTREAM_ARTIFACT_PROVIDER_OBSERVATION_MAXIMUM_ATTEMPTS 5 Closed automatic observation budget before provider_unavailable is exhausted.
WORKSTREAM_ARTIFACT_STREAM_BUFFER_BYTES 1048576 Bounded streaming buffer, limited to at most 1 MiB.
WORKSTREAM_ARTIFACT_OPERATION_LOCK_TIMEOUT_SECONDS 1800 Maximum wait for a private cross-process artifact-store operation lock before failing closed.

Contributor outer-ZIP inspection uses additional startup-fixed bounds. These limits govern scratch inspection only; they do not activate submission routes or durable admission. Raising the 512 MiB ceiling requires a separately approved operational change.

Environment variable Default Maximum Contract
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_ENTRIES 2000 100000 Central-directory entries plus deterministic implicit directories.
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_PATH_BYTES 1024 4096 UTF-8 bytes in one normalized POSIX path.
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_PATH_DEPTH 32 256 Segments in one normalized path.
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_CENTRAL_DIRECTORY_BYTES 8388608 67108864 Bounded central-directory allocation guard.
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_ENTRY_BYTES 134217728 536870912 Actual expanded bytes for one regular file.
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_EXPANDED_BYTES 536870912 536870912 Actual expanded bytes across the complete outer archive.
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_COMPRESSION_RATIO 100 10000 Maximum expanded-to-compressed ratio for one file.
WORKSTREAM_ARTIFACT_SUBMISSION_ZIP_MAXIMUM_INSPECTION_SECONDS 300 1800 Complete synchronous inspector deadline inside the preparation deadline.

The delivered post-submit materializer uses the same ledger to reserve a submission workspace's complete expanded byte count and entry count before projection. Workspace reservations participate in aggregate byte/free-space checks but do not become artifact storage. Cleanup uses the workspace's own bounded entry reservation rather than the live prepared-file limit, so every archive accepted at the configured entry ceiling is removable after any terminal outcome.

After fixed-service authorization, the canonical outer-ZIP inspector projects the already-inspected manifest into one callback-scoped sealed tree using descriptor-relative creation. Regular files use fixed 0400 or normalized executable 0500; directories use 0500. Checker adapters receive no scratch path, shell, subprocess, or execution primitive. The tree is destroyed first; only then do bounded, path-redacted platform/default results return. ARCH-04B neither runs project-policy primitives nor persists final checker evidence.

Pre-submission checker catalogue configuration is separate from ZIP inspection bounds:

Environment variable Default Format Contract
WORKSTREAM_ARTIFACT_PRE_SUBMISSION_CHECKER_DISABLED_IDS empty closed catalogue IDs Startup-fixed comma-separated stable IDs. Unknown or duplicate IDs fail startup. Any disabled mandatory definition makes preparation infrastructure-unavailable; disabled advisory definitions remain visible in the effective plan and do not block remaining execution.

Enabled artifact storage also requires an explicit durable-byte policy. None of these limits has a default, and startup fails unless all four are positive:

Environment variable Default Contract
WORKSTREAM_ARTIFACT_ADMISSION_TASK_MAXIMUM_BYTES unset Maximum cumulative unique provisional/completed bytes charged to one task. Guide bytes have no task scope.
WORKSTREAM_ARTIFACT_ADMISSION_PRODUCER_MAXIMUM_BYTES unset Maximum cumulative unique provisional/completed bytes charged to one canonical human actor or fixed service identity.
WORKSTREAM_ARTIFACT_ADMISSION_PROJECT_MAXIMUM_BYTES unset Maximum cumulative unique provisional/completed bytes charged to one project.
WORKSTREAM_ARTIFACT_ADMISSION_DEPLOYMENT_MAXIMUM_BYTES unset Maximum cumulative unique provisional/completed bytes charged to the deployment namespace.

The first committed scope row pins its configured limit. A later process whose configuration disagrees fails admission instead of silently changing the persisted capacity contract.

The S3-compatible provider settings use the same prefix:

Environment variable Default Contract
WORKSTREAM_ARTIFACT_STORE_BACKEND disabled Set to s3_compatible to select the S3 protocol adapter.
WORKSTREAM_ARTIFACT_S3_PROVIDER_PROFILE unset Closed value: minio for local/CI or aws_s3 for activation-gated native AWS.
WORKSTREAM_ARTIFACT_S3_REGION unset Required canonical S3 region.
WORKSTREAM_ARTIFACT_S3_ENDPOINT_URL unset Required HTTP(S) endpoint for MinIO; forbidden for native AWS.
WORKSTREAM_ARTIFACT_S3_BUCKET unset Required private bucket name.
WORKSTREAM_ARTIFACT_S3_PRIVATE_PREFIX workstream/artifacts Canonical private key prefix; object references remain server-derived SHA-256 paths.
WORKSTREAM_ARTIFACT_S3_ADDRESSING_STYLE virtual Closed value: path or virtual; local compose uses path.
WORKSTREAM_ARTIFACT_S3_CREDENTIAL_MODE unset MinIO requires local_static; native AWS requires aws_workload_identity.
WORKSTREAM_ARTIFACT_S3_AWS_WORKLOAD_IDENTITY_METHOD unset Native AWS selection: roles-anywhere, assume-role-with-web-identity, container-role, or iam-role; never used by MinIO.
WORKSTREAM_RELEASE_ID unset Native AWS-only immutable deployed image/release digest in sha256:<64 lowercase hex> form.
WORKSTREAM_ARTIFACT_S3_ROLES_ANYWHERE_TRUST_ANCHOR_ARN / TRUST_ANCHOR_SHA256 unset Exact anchor and canonical CA/state digest, required only for Roles Anywhere. Both names use the full WORKSTREAM_ARTIFACT_S3_ROLES_ANYWHERE_ prefix.
WORKSTREAM_ARTIFACT_S3_ROLES_ANYWHERE_PROFILE_ARN / PROFILE_SHA256 unset Exact shared profile and canonical security-state digest, under the same prefix.
WORKSTREAM_ARTIFACT_S3_ROLES_ANYWHERE_ROLE_ARN unset This process's exact role; application uses runtime, each proof job uses its own role.
WORKSTREAM_ARTIFACT_S3_ROLES_ANYWHERE_CERTIFICATE_FILE / PRIVATE_KEY_FILE unset Private mounted identity files, under the same prefix; never the CA signing key.
WORKSTREAM_ARTIFACT_S3_RUNTIME_ROLE_ARN unset Exact runtime IAM role ARN used for product object calls and runtime proof.
WORKSTREAM_ARTIFACT_S3_RUNTIME_TRUST_POLICY_SHA256 unset Canonical digest of the reviewed runtime role trust policy.
WORKSTREAM_ARTIFACT_S3_RUNTIME_POLICY_ARN unset Exact sole customer-managed runtime policy ARN.
WORKSTREAM_ARTIFACT_S3_READINESS_ROLE_ARN unset Exact independent readiness-job role ARN.
WORKSTREAM_ARTIFACT_S3_READINESS_TRUST_POLICY_SHA256 unset Canonical digest of the reviewed readiness role trust policy.
WORKSTREAM_ARTIFACT_S3_READINESS_POLICY_ARN unset Exact sole customer-managed readiness policy ARN.
WORKSTREAM_ARTIFACT_S3_NEGATIVE_ROLE_ARN unset Exact independent negative-test role ARN; it has no attached or inline policy.
WORKSTREAM_ARTIFACT_S3_NEGATIVE_TRUST_POLICY_SHA256 unset Canonical digest of the reviewed negative-test role trust policy.
WORKSTREAM_ARTIFACT_S3_BOOTSTRAP_PRINCIPAL_ARN unset Exact environment provisioning principal evaluated without supplying its credentials.
WORKSTREAM_ARTIFACT_S3_BUCKET_POLICY_SHA256 unset Expected canonical bucket-policy digest in sha256:<64 lowercase hex> form.
WORKSTREAM_ARTIFACT_S3_ACCESS_KEY_ID unset MinIO local/CI secret only; forbidden for native AWS.
WORKSTREAM_ARTIFACT_S3_SECRET_ACCESS_KEY unset MinIO local/CI secret only; forbidden for native AWS.
WORKSTREAM_ARTIFACT_S3_SESSION_TOKEN unset Optional MinIO local/CI secret only; forbidden for native AWS.
WORKSTREAM_ARTIFACT_S3_CONNECT_TIMEOUT_SECONDS 5 Provider connection timeout, at most 60 seconds.
WORKSTREAM_ARTIFACT_S3_READ_TIMEOUT_SECONDS 60 Provider read timeout, at most 1800 seconds.
WORKSTREAM_ARTIFACT_S3_WRITE_TIMEOUT_SECONDS 1800 Provider write timeout, at most 3600 seconds.
WORKSTREAM_ARTIFACT_S3_POOL_TIMEOUT_SECONDS 5 Client-pool acquisition timeout, at most 60 seconds.
WORKSTREAM_ARTIFACT_S3_OPERATION_TOTAL_TIMEOUT_SECONDS 1800 Total provider-operation timeout, at most 3600 seconds.
WORKSTREAM_ARTIFACT_S3_MAX_POOL_CONNECTIONS 16 Bounded client connection pool, from 1 through 256.
WORKSTREAM_ARTIFACT_MAXIMUM_BYTES 536870912 Hard object maximum; configuration cannot exceed 512 MiB.

MinIO is restricted to local, development, and test environments. Native AWS becomes runtime-eligible only while an exact immutable three-probe activation remains fresh. Native AWS configuration rejects static credentials, configured endpoints, unselected or ambient credential sources, arbitrary container credential URLs, and custom instance-metadata endpoints. The selected method accepts only its exact closed AWS_* environment allowlist; every other AWS_* or BOTOCORE_* SDK behavior control fails before SDK session construction. S3 object operations and credential metadata operations explicitly ignore ambient HTTP proxy variables. IAM-role validation requires IMDSv2 and the isolated fetcher disables IMDSv1. Even valid native AWS settings let API, Celery task-process and Beat construct the inert typed bootstrap and validate the persisted namespace claim before requiring an activation. Absent activation still fails before credential-source probing, SDK session construction, resolver loading, or provider I/O. The direct scratch-cleanup task uses only ArtifactScratchManager; it neither constructs a provider nor claims provider activation.

Every process sharing a scratch root must use the identical complete setting set. The root marker binds a canonical fingerprint of those limits and startup fails closed on mismatch; changing limits therefore requires an empty, deliberately reprovisioned scratch root rather than an in-place mixed rollout.

The same canonical ArtifactScratchManager owns pre- and post-submit projection workspaces. Pre-submit uses the current prepared contributor ZIP; post-submit uses only the original selected by the immutable Submission's consumed admission. Both share one inspected, sealed-tree projection implementation. Async post-submit consumers receive only entry facts and bounded file reads, with no paths, provider handles or credentials. The view is revoked before workspace and preparation cleanup finishes, including failure and cancellation.

ARCH-04B implements this hidden input path. It verifies complete original digest and size, then rebuilds the semantic manifest, reserves expansion against the existing aggregate scratch quota, and rechecks stored material selection after I/O. Database transactions and row locks do not span provider/consumer execution. ARCH-04D2 supplies exact fixed-service input authority before and after I/O; ARCH-04C owns durable attempts/results and their final publication. Byte drift rejects materialization; this read does not fabricate an incident or mutate Submission lifecycle state.

After ambiguous provider completion, the durable put-attempt resolver first calls read-only observe_put_result with the persisted commitment; it does not regenerate bytes or replay a mutation. If authoritative absence is confirmed and the owning durable source can be regenerated, the original caller may prepare it again and compare the complete digest/size. Identical bytes may replay the original operation after admission capacity is reacquired. Changed guide bytes require a new guide version with its own document declaration, internal snapshot and setup. Changed checker-output bytes conflict with the existing run/slot operation; a distinct logical run or owner-issued slot is required. Neither reuses the old operation, snapshot, or binding identity. A generator that cannot reproduce exact bytes fails its old infrastructure attempt instead of fabricating replay. Bytes are never placed in PostgreSQL, Redis, Celery payloads, logs, or audit.

Durable Publication And Execution

Post-commit Celery publication is an optimization. A periodic PostgreSQL scanner is the durable guarantee. It finds pending jobs whose publication deadline passed and running jobs whose execution lease expired, then publishes them duplicate-safely within a configured SLA.

Celery acquisition uses the PostgreSQL clock and atomically writes a fresh executor UUID, lease expiry, and incremented execution generation. A configured end-to-end verification deadline covers the complete stream, not only socket inactivity. It is derived from maximum object size and minimum supported throughput and must be shorter than the lease by a result-persistence margin. The execution cancels at that deadline even when bytes continue arriving. v0.1 uses no heartbeat.

Terminal writes require the same executor UUID and generation. A zero-row update means the execution is stale; it writes no terminal state or audit event. Put-attempt resolution and verification both resolve a current fixed service actor/link and exact action/resource when execution starts, then revalidate that authority inside the same transaction as terminal CAS, receipt, replica/attempt, recovery, and audit writes. Suspension, revocation, resource drift, stale executor, or generation mismatch writes no terminal fact. Race tests cover each authority input for both service-execution mutation classes. 02C2 exposes a typed two-phase authority seam so hidden behavior can be tested, but only the deny implementation is available in production. The merged AUTH-09E and AUTH-PREP foundations supply runtime admission and the generic prepared-authority protocol without an ART consumer or activation. The exact AUTH artifact-internal activation chunk later supplies evaluator integration and the scanner schedule; ART does not change action availability or inspect the static service-action matrix.

Verification Result Matrix

Observation Job result Replica effect Retry meaning
complete bytes match digest and size verified bindable, available, valid none
object absent after bounded consistency/retry policy missing missing/unavailable/unknown terminal after binding; before binding exact same-item replay resets to pending/unknown/unknown and creates a new job
size or digest mismatch integrity_mismatch integrity_mismatch/available/invalid, quarantined unrecoverable in v0.1; security incident and no overwrite
timeout/throttle/retryable provider failure provider_unavailable unchanged/not bindable bounded automatic retry, then Operator
request/identity/CAS contradiction conflict unchanged/not bindable security/data incident; no generic repair
stale executor/generation no terminal write no effect current execution owns outcome

Storage failures never become contributor checker findings, review decisions, contribution records, compensation-award exposure, or reputation events.

Operator Recovery

Only submission-bundle and checker-output verification jobs can be retried. Task-import sources use the existing automatic put-observation and verification scanner lifecycle; they have no task selector and do not enter this task-bound Operator retry contract. Guide uploads finish at committed original-document custody; retained guide verification rows cannot be scanned, claimed, executed, or recovered, including replay of a retained recovery chain. Eligibility follows the originating put producer, so sharing content with a guide does not disable submission checking.

The recovery class is closed:

provider_observation

provider_observation creates a fresh verification job. It cannot write a verified result without reading the complete object. Contradictory PostgreSQL facts yield terminal conflict and an incident; v0.1 exposes no generic record- repair operation. Recovery does not mutate, overwrite, retain, release, or delete provider bytes.

The retry command accepts only a terminal provider_unavailable job whose automatic attempt budget is exhausted. It rejects pending, running, verified, missing, integrity_mismatch, conflict, and a non-exhausted provider-unavailable job. Rejection tests prove that no recovery attempt, new job, or initiation-success audit is created for those states.

Each verification job may be the source of at most one recovery attempt for its entire lifetime. Exact request replay returns that attempt. A different key, post-terminal ancestor reuse, or concurrent second request against the same source conflicts without creating an attempt, job, or audit effect. Further recovery must target the immediate retry job, and only when that job independently exhausts provider_unavailable.

Operator authorizes recovery and provides a bounded non-empty reason. Celery executes under the fixed workstream.artifact.verifier service principal. The requester is never presented to the provider as execution authority.

Recovery API Contract

These internal Operator APIs are introduced before product cutovers:

GET  /api/v1/operator/artifacts/bindings?resource_type={type}&resource_id={id}
GET  /api/v1/operator/artifacts/contents/{content_id}/replicas
GET  /api/v1/operator/artifacts/replicas/{replica_id}/receipts
GET  /api/v1/operator/artifacts/verification-jobs/{job_id}
POST /api/v1/operator/artifacts/verification-jobs/{job_id}/retry
GET  /api/v1/operator/artifacts/recovery-attempts/{attempt_id}
GET  /api/v1/operator/artifacts/audit-events

Every route requires the exact Authorization Service action/resource decision defined by the owning chunk. A retry request supplies a reason, client idempotency key, required project/task IDs, optional submission ID, and expected source-job CAS version. A submission ID remains optional because submission-bundle verification precedes Submission creation. The transaction creates the recovery envelope and retry verification job and returns 202 with attempt, source-job, and retry-job IDs. Idempotency scope is requester, source job, recovery class, and client key; a canonical request digest makes exact replay return the original IDs and altered replay conflict without side effects. The recovery-attempt GET returns its own status, immutable source-job status, current retry-job status, terminal code, chain identity, and audit IDs so no database inspection is required.

The binding lookup is the discovery entry point from a known project, guide, task, submission, checker run, or future review resource. It requires exact resource scope, supports an optional logical-role filter, and returns bounded binding/content summaries plus replica and current verification-job IDs. Audit listing supports exact project/resource/content/job/attempt filters and returns those stable Workstream IDs.

No route returns provider object references, bucket/key, endpoint, credentials, signed URLs, or raw provider responses. Pagination is bounded and stable.

Exact Submission-Bundle Admission

One contributor operation receives one outer ZIP into bounded private scratch. Workstream computes its exact SHA-256/byte count, walks the complete safe outer archive tree, and generates a deterministic SubmissionBundleManifest from normalized file/directory paths, entry type, file SHA-256/byte count, and the normalized executable flag for each regular file. Nested archives remain opaque in v0.1. Exact archive or semantic-manifest equality with the immediate prior immutable Submission rejects before checker/provider I/O. The v0.1 structural safety boundary accepts only stored and raw-DEFLATE member compression; other ZIP compression methods fail closed before admission.

For a regular Unix-created ZIP entry with valid mode metadata, executable is true when any execute bit is present. It defaults false for non-Unix or invalid mode metadata and is null for directories. Ownership, group, read/write bits, setuid/setgid/sticky bits, timestamps, compression, and comments are excluded. Symlinks/special entries remain rejected. Materialization never restores raw archive modes: fixed startup-validated modes produce sealed read-only files, read-and-execute files, and read-and-traverse directories. The flag never authorizes execution by itself. Pre-submit and post-submit materialization use the same projection and ArtifactScratchManager retains secure cleanup custody. The exact numeric modes are one startup configuration shared by every process using the scratch root. Cleanup under an exclusive generation/lease may restore only the manager-owned directory write bit after no-follow ownership/type validation, then remove the tree; archive-supplied modes are never applied.

Mandatory platform and locked Project Guide checks form one ordered EffectivePreSubmissionExecutionPlan assembled from the central versioned PreSubmissionCheckerCatalogue. Custody and identity phases consume the typed, verified results produced by the earlier archive/manifest capabilities; materialization and policy phases may consume the same sealed read-only scratch tree plus those typed facts. They do not reopen or independently reinterpret the uploaded ZIP:

artifact custody/safety result phase
-> artifact and semantic-manifest identity result phase
-> sealed materialization phase
-> Workstream default policy phase
-> locked Project Guide policy phase
-> one bounded result/evidence envelope

The catalogue is the only name/version/dispatch registry. Each entry declares its classification, phase/order/dependencies, typed inputs, resource budget, stable result, policy provenance, and explicit enabled|disabled state. Startup validation rejects missing dependencies, duplicate IDs/versions, unknown policy primitives, invalid ordering, and any configuration that would treat a disabled mandatory entry as passing. A disabled mandatory security, integrity, or accountability entry fails preparation closed before durable or provider effects. A disabled advisory entry is recorded and skipped. Project policy and task parameters cannot toggle catalogue availability.

ART's already implemented ZIP safety, manifest, executable, and change gates are registered capabilities in this plan; 04B does not reimplement them. The locked project policy consumes the server-derived manifest and sealed workspace through the same execution API. It may require task.toml or any other project-specific file, but no such filename is a Workstream universal default.

Infrastructure exhaustion or mandatory catalogue unavailability returns:

HTTP 503
code = pre_submission_infrastructure_unavailable

This is infrastructure state, not a checker result, review decision, or contributor outcome. It creates no durable artifact, admission, Submission, compensation, contribution, or reputation effect. Scratch is cleaned and process loss requires reupload without manager/operator approval.

The hidden 04A2 structural inspector uses the closed internal failure tokens submission_archive_malformed, submission_archive_limit_exceeded, submission_archive_unsafe_entry, submission_archive_encrypted, submission_archive_collision, submission_archive_integrity_failure, and submission_archive_timeout. They contain no submitted paths or parser details. The public authorized ZIP preparation route maps these typed rejections to its existing non-retryable HTTP 422 invalid_request envelope and returns the exact closed token as detail. Unexpected parser, programming, and infrastructure failures are not translated to contributor-invalid archive responses.

The hidden 04A3 semantic change gate adds the closed internal failure tokens submission_archive_unchanged, submission_manifest_unchanged, and submission_canonical_predecessor_unavailable; stale revalidation uses submission_predecessor_stale. The first two distinguish exact archive equality from equal semantic content with different ZIP packaging. The third fails closed when a non-first Submission predecessor lacks ART-owned canonical archive and manifest identities; legacy caller-owned package or manifest fields are never a fallback. These failures occur before checker or provider I/O and create no Submission, checker, review, admission, outbox, contribution, compensation, reputation, task-state, or assignment-state effect. Their eventual HTTP/API mapping remains deferred to the submission-admission route chunk.

Only a passing result is handed immediately to generic durable admission in the same process. Workstream writes the outer ZIP once, independently reads it back, and publishes a ready bindable admission only after exact verification. Existing put attempt, observation, receipt, verification, scanner, and recovery records own durable ambiguity. Submission creation locks and consumes the exact admission; post-submit materialization resolves its immutable binding and recomputes integrity before checker execution.

PreSubmitEvidenceSet

The sealed-tree callback returns one canonical ordered result containing both platform/default and locked project-policy entries. It performs no database write. After scratch cleanup, a separate caller-transaction service locks and reloads the active actor and identity link, task, active assignment, immediate predecessor, locked guide lineage, effective submission-artifact policy and compiled pre-submit policy. Drift fails closed before evidence persistence.

For a checker-remediation replacement, TASK first supplies the exact current authorized source selected from its own locked predecessor and the retained CHECKERS result. ART checks that source before reading ZIP bytes and at each later context relock; a missing, foreign or superseded cause denies intake.

PreSubmitEvidenceSet stores the exact actor/task/assignment/predecessor, prepared generation, archive digest/size, semantic-manifest identity, locked guide and policy hashes, effective-plan identity and catalogue identity. ARCH-04F3 also carries the server-selected checker_remediation_manifest_id for replacement. It is absent for an initial Submission and binds the existing evidence operation identity and passing capability; ART does not copy the checker result body into a second findings store.

Ordered PreSubmitEvidenceResult rows store the closed definition, policy trace, phase, classification, status and bounded codes directly; required provenance is never hidden in generic metadata. A result-manifest digest binds the complete canonical envelope.

The operation identity hashes every input custody and locked-policy fact. A database uniqueness constraint permits one set for that identity. Exact replay returns the existing durable evidence set but never returns a new pass capability; the caller must re-prepare the exact bundle before a later submission attempt. Changed facts or changed deterministic results conflict. The v0.1 baseline installs the normalized set and result tables plus composite identity-link, assignment, task/project, locked guide/policy and predecessor lineage constraints. Set and result rows reject update, delete and truncate; result membership closes with the creating transaction, and populated evidence prevents destructive downgrade. Replay revalidates every ordered result member before trusting the set. Blocking results create evidence only—never an ArtifactContent, provider write, admission, Submission or lifecycle effect.

A passing evidence set additionally yields a process-local single-use capability bound to the prepared generation, immediate predecessor and effective-plan hash. ART-04C must consume that capability immediately; the durable evidence-set ID cannot substitute for it. The only failure audit projection is pre_submission_check_failed, containing bounded IDs, catalogue identity, stable codes, counts and categories without submitted paths, filenames, scratch/provider references, credentials, raw output or free-form messages.

SubmissionBundleDurableIntent

Before provider I/O, the final preparation transaction persists one immutable submission-producer intent linked by foreign keys to the exact passing PreSubmitEvidenceSet and generic ArtifactPutAttempt. Those typed joins preserve the actor and identity link, project, task, assignment, predecessor identity/version, locked guide/snapshot/policy lineage, semantic-manifest identity/digest, archive digest/size/media type, effective-plan digest, storage scheme, and operation identity required for database-only verified-admission publication. The replacement's exact remediation cause is part of the final preparation AUTH resource and durable lineage through evidence to the verified admission.

It is not a bindable admission, Submission, review, or contribution fact.

The intent, transaction-local authorization evidence, provisional capacity reservation, and put attempt commit atomically before the prepared ZIP is handed to ArtifactStore. One-to-one uniqueness on the evidence set and put attempt fences concurrent continuation. Recovery and 04C2 reload the typed row; they never parse the opaque put-attempt request digest and never depend on scratch, a prepared handle, or process-local capability state.

If the process dies before this intent commits, the old evidence row cannot mint or remint authority. Recovery requires a complete fresh upload and checker execution, which produces a new prepared generation, evidence identity, and single-use pass capability. If the process dies after intent commit, the existing generic put-attempt observation and recovery machinery owns the technical obligation without another submission-specific recovery aggregate. The v0.1 baseline installs the immutable join and extends only the existing generic put-attempt and receipt producer shapes needed for submission bundles.

SubmissionBundleAdmission

One immutable verified preparation with closed status:

status                         = ready | consumed | stale
durable_intent_id              = UUID
put_attempt_id                 = UUID
actor_profile_id               = UUID
identity_link_id               = UUID
project_id                     = UUID
task_id                        = UUID
assignment_id                  = UUID
predecessor_submission_id      = UUID | null
checker_remediation_manifest_id = UUID | null (required for replacement)
locked_policy_context_hash    = SHA-256 copied from immutable pre-submit evidence
artifact_content_id            = UUID
verified_replica_id            = UUID
verification_receipt_id        = UUID
put_operation_receipt_id       = UUID | null
put_observation_receipt_id     = UUID | null
semantic_manifest_id/hash      = immutable canonical manifest identity
pre_submit_evidence_set_id      = UUID
ready_at                       = timestamp
consumed_at                    = timestamp | null
consumed_by_submission_id      = UUID | null
consumed_by_submission_version = positive integer | null
stale_at                       = timestamp | null
stale_reason                   = bounded token | null

The immutable pre-submit evidence set retains the canonical inspected semantic manifest body as JSONB, including file hashes/sizes and directory entries, once. Database validation and recovery therefore use the same numeric representation. Inspection, recovery and the database guard require unique Unicode-case-folded paths and exact-spelling directory ancestors. PostgreSQL 16 uses the frozen Unicode 15 mapping that matches the backend, rather than locale-dependent lowercase. The admission references its identity and hash. Consumption validates that body and returns detached file metadata with archive commitments only for a consumed result; stale results contain no material. Exact consumed replay uses the scoped read_consumed operation and freshly validates the retained binding AUTH receipt. Consumption performs no provider read. Consumption also requires the canonical hash of the exact Submission summary and contributor attestation to match retained pre-submit evidence, before either new binding or consumed replay. A deferred database guard compares the final bound Submission text with that evidence; the existing lineage guard prevents clearing or replacing a bound admission. Older retained evidence without metadata stays unchanged and cannot supply current consumption. Existing post-submit material selection keeps its exact immutable ART lineage and inspected-byte checks; it does not use this new consumption projection. B5 reconciles ART and CHECKERS request limits before durable admission; B6 reuses that projection when committing the Submission and its initial request. Delivery activation remains separate.

It records preparation actor/profile and identity-link provenance, project, task, assignment, immediate predecessor, exact locked task/guide/policy context, ArtifactContent, SubmissionBundleManifest, immutable pre-submit evidence-set identity, timestamps, terminal reason, and optional consuming Submission. Status-field constraints require terminal facts only in their matching state. A matching hash is first persisted on the immutable evidence set and the admission trigger requires the copied value, so a direct insert cannot invent a different policy-context identity. A unique Submission admission reference plus locked/CAS transition permits at most one consumer.

ART-04C2 publishes ready only inside the generic verifier's successful terminal transaction through a narrow submission publisher. It reloads and matches the durable intent, passing evidence, put attempt, content, verified replica, and successful full-read verification receipt. Direct provider acknowledgement and observed-confirmed recovery use distinct nullable receipt references; exactly one must match. Guide and checker-output verification do not create submission admissions. The public preparation POST returns bounded operation/current-admission state and does not wait for verification.

ready -> consumed occurs only in the transaction creating Submission and binding. ready -> stale occurs only when consumption proves task closure, predecessor advancement, or locked-context replacement. Authority loss alone does not stale the admission. Both terminal states are irreversible. Client abandonment may leave ready unbound indefinitely; it has no Submission, review, contribution, compensation, or reputation effect.

All states count against existing completed-byte task/actor/project/deployment capacity. Existing Operator admission-usage projections expose bounded unbound ready and stale counts/bytes. No admission expiry, release, deletion, provider cleanup, retention process, or new recovery aggregate exists. Exact preparation replay returns the original admission without another provider write, charge, evidence set, or admission.

Authorization At Durable Boundaries

The preparation route authorizes artifact.submission_bundle.prepare before accepting bytes. Immediately before capacity reservation and put-attempt creation, the owning transaction consumes AUTH's transaction-local prepared capability. AUTH and typed TASK/PROJECT capabilities reload/lock their own actor, identity-link, project authority, assignment, task, predecessor, locked context, action availability, and canonical resource facts. ART does not access AUTH-owned persistence. Authorization evidence, capacity, and durable put intent commit atomically; provider I/O starts only afterward.

Revocation after durable put intent does not cancel fixed-service verification or recovery. It does prevent later binding until Submission creation receives a new human decision. That transaction consumes both fresh human submission.create and fixed-service ActionId artifact.submission.binding.create capabilities, mapped to PermissionId artifact.binding.create, locks and matches admission/task/assignment/predecessor/context, creates the immutable Submission and binding, and marks admission consumed.

For replacement, both final AUTH resources and TASK consumption must match the admission's exact checker-remediation cause. The new Submission and generation-one evaluation dispatch commit through the same canonical writer. Direct SQL cannot substitute that cause or the required AUTH receipts. A retained uncaused revision dispatch cannot be invented into this lineage: migration 0034_checker_remediation_replacement refuses upgrade before DDL if its exact cause and receipts cannot be proven, preserving the 0032 schema and rows. Any denial, cancellation, stale execution, or persistence failure rolls back all effects. After authority succeeds, proven task closure, predecessor advancement, or locked-context replacement may commit only ready -> stale plus bounded evidence, never a Submission/binding. Authorization/concealment precedes detailed admission status errors.

Stable outcomes for an already authorized matching resource include:

409 submission_bundle_admission_already_consumed
409 submission_bundle_admission_context_changed
409 submission_bundle_admission_stale
403 <canonical AUTH denial; no admission state disclosure>

After cutover, the public request is:

{
  "summary": "...",
  "contributor_attestation": "...",
  "submission_bundle_admission_id": "..."
}

Clients do not provide package URI, provider reference, canonical digest manifest, semantic-manifest hash, or server content IDs.

Product Submission-Bundle APIs

POST   /api/v1/tasks/{task_id}/submission-bundle-preparations
POST   /api/v1/tasks/{task_id}/submissions

The preparation POST performs scratch intake, inspection, checks, and durable handoff as one process-local orchestration. It returns only a Workstream operation/admission identity and bounded redacted status. An exact idempotent POST retry may observe the same durable operation after durable intent; v0.1 exposes no separate preparation-status GET route. No retry resolves a scratch path, every retry obtains fresh preparation authority, and APIs never return provider internals.

Task-Import Source Custody

PILOT-02A exposes project-bound JSON source custody through the existing ART admission, put, independent verification and automatic recovery owners:

POST /api/v1/projects/{project_id}/task-import-sources
PUT  /api/v1/projects/{project_id}/task-import-sources/{source_id}/content
GET  /api/v1/projects/{project_id}/task-import-sources/{source_id}
GET  /api/v1/projects/{project_id}/task-import-sources/{source_id}/content

Declaration requires a UUID Idempotency-Key, canonical lowercase SHA-256 commitment and a byte count from 1 through 8 MiB. ART creates one immutable, UUIDv7 source record per project/key, retaining the declaring actor, identity link and exact declaration AUTH decision. Media type is application/json. Same-key replay requires fresh current authority; changed hash or byte count conflicts. The key selects the original record and never determines its ID.

Upload receives exact UTF-8 bytes with application/json and validates the published TASK JSON contract through bounded ART scratch before durable admission. It rejects malformed JSON, duplicate object members, invalid or duplicate external IDs, invalid rows and more than 500 tasks. The full bytes must match the declaration. Invalid input creates no put attempt, capacity charge or stored source content. Final admission locks the exact source and consumes fresh artifact.task_import_source.upload authority in the same transaction as the single source-linked put intent and its project, original producer and deployment charges.

The closed task_import_source producer role references that declaration with an exact project/hash/count/media composite foreign key. Repeated upload uses the same attempt and existing replay/recovery rules, including lost provider acknowledgement. Verification belongs to the existing fixed verifier; the source does not acquire a separate artifact binding or lifecycle. Provider and capacity failures are infrastructure outcomes, not task-validation decisions.

Status returns the declaration or its retained attempt/verification state with the original byte commitment. verified denotes retained full-read evidence; download additionally checks exact source/attempt/receipt/job/content/replica ancestry and independently reads and verifies every byte into bounded scratch before successful response headers. Downloads expose source facts and exact JSON bytes, never provider coordinates. A corrupt or unavailable provider read fails without delivering content or committing a successful read decision.

All four routes require an active human covered Project Manager through existing project.task.manage. Source selector denials are concealed as 404. Declaration, upload and read use distinct closed actions and exact typed resource commitments; revoked authority cannot replay a prior success.

A declaration is the durable parent of every admitted source attempt. Failed or abandoned declarations and admitted sources remain retained immutable ART facts. They do not claim that a batch or any Tasks were created. There is no source deletion API or retention scheduler, and automatic recovery cannot create Tasks. Atomic DRAFT import, project/external-ID conflicts and explicit screen/release are implemented by the TASK batch owner, with separate CLI conversion and batch commands. They are not ART operations; source storage alone never invokes them. When a client converts CSV, the retained source is the received JSON, not the original CSV.

Guide And Checker Binding

Guide-source ingestion stores immutable original documents through ArtifactStore. PostgreSQL records the source manifest, versions, exact content identity, upload receipts and run custody. It does not store extracted guide bodies. Current uploads accept PDF, DOCX, PPTX and UTF-8 Markdown (.md). Task examples are required ordinary PostgreSQL text inputs alongside the guide metadata; they do not create Workstream Tasks. Embedded Markdown bodies, separate extraction jobs and the three-agent setup APIs are removed. Submission ZIP verification remains a separate workflow.

Upload admission inspects the bounded declared file format and computes size and SHA-256. A committed document_stored receipt establishes guide readiness; it does not mark a shared replica verified for submissions. Once all exact snapshot items are committed, Workstream queues the same setup generation for one unified Celery compilation. Incomplete or uncertain storage never triggers inference. Recovery of a committed upload rechecks current project, guide, snapshot and generation before publishing that continuation.

The runtime receives an immutable manifest with opaque document handles. Trusted bounded read_document and search_document tools inspect only assigned originals through the fixed guide-reader authority and isolated parser. ART locks and checks the draft guide, current snapshot, setup generation, fenced attempt, exact ingest/put/receipt, content, replica and storage namespace. It stages and verifies the original through ArtifactScratchManager before yielding bytes. Forged, foreign, stale, expired or closed grants deny without reading an object. Models receive bounded verified text, never S3 credentials, bucket listings, object keys or URLs. Document code is not executed.

The OpenAI Agents SDK adapter uses those neutral tools without allocating provider files or workspaces. Runtime, model/provider, instructions, tool bounds and timeouts are independently configured and captured in immutable attempt evidence. The default model request timeout is 300 seconds; the whole-run timeout is separate. PROJECTS records parser-bound source-access receipts. Those receipts prove exact source lineage; they do not alone prove that a model phase observed content.

Every draft, review and correction must pass strict schema, canonical, context and source validation. Each phase must initially read every assigned original, for ready and blocked results alike. Only offset-zero reads returning text or proving a genuinely empty original satisfy initial observation. Search and offset-only reads remain audited and budgeted but cannot substitute for initial observation. The phase evidence is cleared before each invocation. Validated findings and proposals cite exact source versions; ready proposals account for every assigned document. Known invalid completed inference output receives at most two value-free corrections across draft and review within the original budgets. Exhaustion is invalid-terminal; uncertain provider execution is never replayed as correction or by redelivering the job.

Historical provider resources retain their immutable allocation and identity receipts. Their separate bounded cleanup reconciler deletes only recorded IDs, without inference, source access or discovery by listing. Unknown historical allocation outcomes rely on recorded provider expiry. Current neutral inference allocates none of these resources. Scratch retains aggregate quotas and crash cleanup and is never durable artifact storage.

The setup result contains sufficiency findings and distinct pre-submission and post-submission policy proposals. A blocked guide ends at findings; a ready guide stops at draft output. Project Manager decisions and fresh manual reruns belong to POL-05; post-submit projection belongs to POL-06. No setup finding accepts a submission or enables an unregistered evaluator. Retained prior records remain immutable but cannot supply current guide execution evidence.

The exact post-submit request and material facts persisted from ART's authorized selection retain binding/content IDs, outer-ZIP digest/byte count, semantic-manifest hash, locked policy/checker identities and checker implementation identity. The normal service path names the same verified admission and exact binding as pre-submit evidence. Migration 0009 (ARCH-04D1) enforces the exact nine-field tuple through the ART-owned art_submission_material_matches SQL function at terminal CHECKERS commit. Completed results require material; infrastructure failures with material must also match. The comparison uses immutable admission ancestry, not the replica's current health. Upgrade excludes writers across preflight and installation, preserves valid retained evidence and refuses invalid lineage without data repair or deletion. ARCH-04D2 supplies exact materialization, execute and finalize authority. For a registered capability that produces logs or output bytes, those outputs must become verified artifact bindings through its reserved slots. The current structural catalogue produces no such artifacts.

ARCH-04B2 supplies the hidden checker-output custody boundary. The typed store and byte-free recover(selector) paths enforce one owner-issued slot ceiling, reuse generic admission, put observation and verification, and return only exact verified identities. The flush-only binding participant preserves the full verified ancestry and never privately reads CHECKERS rows. Store and recovery resolve owner facts only through the typed CHECKERS port; the separate Operator resource-to-project lookup remains an explicit read concern and is not an output-admission path. ARCH-04C supplies hidden CHECKERS reservation, currentness and durable evaluation with an exact empty output set. The current structural catalogue declares zero output slots; controlled nonempty test reservations prove ART mechanics only. A future output-producing registered capability must compose bindings only for slots actually reserved by its owner. ARCH-04D2 supplies exact materialization, execution and finalization authority, with database-bound execute/finalize receipts. ARCH-04E2-B also retains the actual input-materialization decision through the ART result and terminal CHECKERS custody, and verifies it before publishing a routing source. Its hidden TASK operation commits the complete authorized outcome. Output write/bind authority remains unavailable; B8 supplies completion delivery. Selected request and completion handlers are registered in production composition.

Current ARCH-04C behavior records known post-authorization material failures as terminal, unroutable material_unavailable, after scratch cleanup and fresh finalization authority. It records no verified material custody, member results or completion event, and exact replay performs no provider I/O. Authority or selection denial and late lineage drift remain nonterminal. This hidden executor neither changes TASK state nor provides automatic or public retry. ARCH-04F4 adds a separate hidden, reason-bound Operator command that uses the coordinator's next generation only for an exact current terminal infrastructure_failed run; the prior result and ART evidence remain immutable. The existing request handler delivers its authorized successor without production registration. Unfinished expired-lease recovery retains the existing attempt and generation.

The target outage policy remains distinct from that current implementation. The hidden Operator retry preserves evaluation_pending while platform failure is retried; no automatic or public checker retry service is active. A provider object confirmed missing after its content was bound is a terminal artifact incident in v0.1: the immutable binding remains, evaluation stays blocked, and no contributor, Project Manager, or Operator route may replace or rebind the bytes. A future separately approved backup/replica-repair initiative may restore exact content from an independently verified copy. Integrity mismatch on an existing content-addressed key is likewise unrecoverable in v0.1: the replica remains quarantined, evaluation stays blocked, and a security incident is required. Recovery never overwrites the poisoned key.

Pre-submit and post-submit outage continuation are distinct contracts. Before durable artifact intent, the former cleans scratch, preserves no reusable attempt, returns the stable 503 above, and requires reupload after scratch or process loss. After durable handoff, existing ART put/verification recovery owns ambiguity. Post-submit continuation must respect retained terminal evidence and current-generation custody. No path fabricates accept, needs_revision, or reject.

Authorization Dependencies

Artifact mechanics may be implemented independently, but no production route, adapter dispatch, background provider read, binding, retry, or recovery becomes active until the Authorization Service owns the exact action/resource decision and the internal service-principal contract.

Provider credentials cannot satisfy this dependency. Development shortcuts are forbidden in production.

Migration And Removal

The initial v0.1 implementation is a clean cut:

  • committed-source preparation and the provider-neutral ArtifactStore are the only writable storage path;
  • the Flow Node backend is removed from configuration without an alias;
  • obsolete caller-owned URI/hash and storage-scheme fields are removed in their owning guide/submission cutovers;
  • the misleading submission /finalize repair route is replaced by POST /api/v1/operator/submissions/{submission_id}/pre-review-gate-repair; normal submission creation still enters evaluation automatically;
  • no dual write, nullable shadow field, fake verified backfill, fallback adapter, compatibility constructor, or second factory remains;
  • the v0.1 baseline contains only the current artifact schema and no fabricated backfill; pre-v0.1 development databases/storage namespaces are reprovisioned and authoritative bytes are reingested through the current port;
  • the v0.1 baseline installs the durable admission ledger and prepared put-attempt tables;
  • the v0.1 baseline includes polymorphic contract-v2 operation receipts, typed put-observation receipts, verification jobs and receipts, and execution-mode/observation fencing;
  • the baseline's guide binding, format-classification, custody-incident and extraction tables remain retained evidence only. The original-document cutover rejects new binding/extraction writes and preserves existing rows;
  • current guide custody binds exact project/guide/snapshot/item, committed original digest/size, and run generation. Task-example hash/count commit to immutable guide metadata. Current inference reads assigned originals through scoped grants, without creating guide bindings or extraction usages;
  • snapshot declarations use sanitized source_label, fixed document/upload metadata and the guide_source_snapshot.task_examples manifest. Caller-owned content identity and private fetch/storage locators are excluded.

The baseline proves fresh installation, fail-closed refusal of old/nonempty databases, and no artifact bytes in PostgreSQL. It is not downgradable.

The checker-output custody migration adds exact Submission version and narrow checker-request digest fields to checker-output attempts plus verified put/receipt ancestry on checker bindings. A successful binding atomically seals the terminal attempt/job and replica identity under the existing job -> replica -> attempt -> content lock order. The database rejects later ancestry changes, including writers using an older transaction snapshot; replica health and availability remain mutable. Rolling back publication also rolls back the seals. It refuses any retained checker-output attempt or binding whose missing evaluation/slot digest or verified ancestry cannot be proven; it neither invents those facts nor deletes retained data.

Verification Strategy

Remaining v0.1 dependency order

The pre-submit checker materializer is a mandatory part of preparation, so its fixed-service AUTH activation must merge after hidden ART-04B1-04B3 and before contributor preparation is activated. ARCH-04B delivers hidden exact post-submit input with exact ARCH-04D2 fixed-service authority. ARCH-04B2 hidden checker-output custody is delivered; ARCH-04C now owns durable CHECKERS execution/results, and ARCH-04D2 owns their exact live activation. ARCH-04E separately owns TASK routing. Historical ART-06A/06B labels do not open duplicate implementation lanes. This ordering prevents a live contributor route whose mandatory checker read is unavailable.

ART-07A1 supplies metadata-only packet membership types and a type-only port. It implements no resolver or byte access. REV-03B now stores normalized packets referencing live guide ingests with committed-upload custody; retained extraction bindings remain read-only. REV-04A immutable Review source storage, REV-04B shared FinalAcceptance storage, CON-03C contribution/award storage and CON-07 hidden submitter participation with complete frozen awards are delivered. REV-04C composes hidden FinalAcceptance/TASK/CON effects. Exact source authority, database complete-set enforcement, currentness proof, audit/outbox and activation remain prerequisites for runtime recognition; reviewer participation and fulfillment remain separate. ART later supplies an exact, authorized reviewer-packet byte capability, while REV owns queueing, leases, decisions, and the reviewer note/findings. The approved v0.1 review flow does not upload a reviewer revision artifact. CON owns ContributionRecord lifecycle and records the accepted immutable Submission and ART identity without provider I/O. Client delivery/export is outside this v0.1 initiative and requires a separately planned egress capability.

  • one conformance suite runs against LocalStorage and real MinIO;
  • offline validation proves the closed AWS S3 configuration and activation contract; actual provider eligibility still requires live deployment proof;
  • integration tests use real S3-compatible API calls, not monkeypatching;
  • concurrent puts cannot overwrite existing bytes;
  • acknowledgement loss, exact replay, oversized-object refusal, truncation, changed bytes, missing object, range reads, timeout, throttle, and provider differences are covered;
  • workload-identity failure, periodic re-publication, duplicate Celery delivery, expired execution lease, a continuously progressing stream that exceeds the total deadline, stale finalization, and Operator retry are covered;
  • least-privilege runtime credentials, exact bucket-scoped s3:ListBucket for missing-key classification, absence of an application object-list surface, denied delete/admin actions, AWS public-access controls, lifecycle safety, and anonymous-read denial are covered by deployment proof;
  • cross-resource authorization and secret non-retention are covered;
  • changed behavior has discriminating regression tests at its real boundary; coverage percentages are diagnostic only, not global or per-subsystem gates;
  • backend CI executes and reconciles the full suite exactly once across its semantic lanes, preserving real PostgreSQL/S3 proof and failure propagation;
  • independently executable services/examples retain their behavioral and integration checks. CI integrity review confirms required execution without bypasses; no percentage score substitutes for these proofs;
  • final proof uses real HTTP APIs and visible jobs/recovery, not direct database inspection.

Application route tests prove one exact application image. They do not claim rolling-fleet atomicity. Public activation requires homogeneous compatible instances or an external deployment barrier.

Deferred Flow Node Adapter

FN-ART-002 is a separate inactive initiative. It may later implement the same ArtifactStore port and conformance vectors. It cannot change product records or services, cannot introduce a runtime fallback, and requires an explicit maintenance cutover after v0.1 is proven.

Deferred R2 Adapter

Cloudflare R2 is outside the v0.1 dependency graph. Any later provider initiative must perform current provider discovery, implement the same ArtifactStore contract, pass its own conformance and security proof, and use an explicit no-fallback maintenance cutover.

Provider Contract References

Operator diagnosis, admission-pressure alerts, and configuration-driven quota rollback are documented in the artifact storage operations runbook.

Evaluation content capacity before durable admission

ARCH-04E1B-B5 checks representability after private ZIP inspection and before pre-submit attempt reservation or durable upload intent. The CHECKERS-owned content value binds exact historical policy facts, task criteria, prepared packet text and every inspected file. Required evidence is projected only from matching verified evidence/{key} files; no EvidenceItem or evaluator result is fabricated. Missing task criteria remain empty input for the existing post-submit checker.

The locked catalogue remains unchanged: at most 1,024 files, 1,000-character paths, the existing field limits and 1 MiB total request. Shared content reserves 1,024 bytes for the finite record-identity envelope. ART's larger ZIP container limits do not imply every inspected archive can be admitted for evaluation. Rejection releases scratch and creates no attempt, put intent or ready admission. Both review-policy modes use these same content rules. ARCH-04E1B-B6 reuses this projection with real identities and the full request digest. ART consumption now retains an immutable admission/binding/decision receipt, required by a deferred database guard on each new consumed admission. Existing retained rows are not backfilled or deleted; absent receipt custody cannot use current consumed replay. The caller transaction commits this receipt with the Submission and its initial evaluation request. ARCH-04E1B-B7 consumes that request through the selected handler and existing exact-byte materializer. Selected request/completion handlers are registered in production composition, and ARCH-02I-B exposes public ZIP preparation plus admission-backed initial/checker-remediation Submission creation. The bounded local public API and Celery service drill starts from public guide inputs with recorded inference.