test(e2e,docs)!: the suites, the CI leg and the public docs speak records

[yousefh409]

What

The top of the OSS stack, re-pointed at the automation RECORD.

Suites. The automations-e2e fixture harness composes the engine the way the
umbrella now does — the four-verb automations seam into createApps, the
runner map, the one internal create op — and its 19 suites assert records rather
than apps-with-triggers. Three suites are new, one per flow that had no cover:
agent-on (code authoring, boot reconcile, kill-switch survival),
vendo-automate (chat authoring, every when shape, cross-owner refusal), and
missing-agent (duplicate name throws at boot; an unregistered name is a FAILED
run row and no fallback brain ever runs).

fixtures/integration moves off /apps/import-with-triggers onto one shared
createAutomation harness helper, and gains the leg nothing covered: Vendo
Cloud's signed heartbeat knock on POST /api/vendo/tick — 202 {fired:1},
idempotent on a retry with a fresh delivery id, 401 for an unsigned stranger and
for a wrong key. It signs with the base64url-decoded secret, so it agrees with
the door's own verifySignature rather than with a restatement of the scheme.
That leg needs the deployment to HOLD a tick secret — the key every credential is
verified against — which the test now asserts as an explicit precondition rather
than assuming.

CI. The automations leg gets ANTHROPIC_API_KEY, on pushes to main only:

ANTHROPIC_API_KEY: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && matrix.name == 'automations' && secrets.ANTHROPIC_API_KEY || '' }}

pull_request and merge_group both evaluate to '', so nothing a contributor
or a fork can trigger reaches the key, and live-agentic's own describe.skipIf
makes the empty string a clean no-op everywhere else.

Docs. capabilities/automations is rewritten around the record and the two
authoring doors; backend/automate is a new page for .on() (every shape, the
declaration-time refusal, what a redeploy does), slotted into "In your backend"
after run. production/vendo-cloud, production/deploying,
reference/http-routes and one stale sentence in reference/cli stop saying
automations run in Cloud — they run in your process, and Cloud only knocks. The
dead deploy/scheduler-and-webhooks link in packages/automations/README.md now
points at the live page.

Demo-bank. The seeded automations and their run history are records, in
Maple's own consumer voice. Two real defects came out of running it:

• a seeded run row carried no owner, so the engine's own reader rejected every
  one of them (invalid run row …: Required) and the console's Automations tab
  would have shown an empty history. Every owner-scoped read filters on that
  field.
• away-drill's dev server used MAPLE_DIST_DIR=.next/away-drill, NESTED inside
  the distDir next build wipes. It is now the sibling .next-away-drill, the
  rule fixtures/context-e2e already follows for this same app, with the
  matching .gitignore entry.

Why the docs changed more than the rename

Three claims in them were wrong in a way that costs a reader real time:

• VENDO_TICK_SECRET was described as the BYO-cron credential. Both credentials
  the door accepts are verified against it (tickAuthorized), so with it unset
  even Cloud's signed heartbeat is refused. capabilities/automations,
  production/vendo-cloud and production/deploying now say so.
• GET /automations was documented as carrying pendingGrants/grantSetId.
  list returns plain redacted AutomationRecord[].
• .on()'s validation error was paraphrased. It is now the message
  validateCron actually throws, and the budget default is the real 50.

Server-side snippets show the RunContext every verb takes last, because
vendo.automations is the engine itself — there is no ctx-injecting facade.

Gate

Rebased onto automations/s2c-vendo @ af8f2d0, then: pnpm build 21/21,
dependency-guard OK (30 packages), portability-gate all legs green, and
fixtures/{automations-e2e,redteam,integration} + examples/demo-bank typecheck
clean.

• fixtures/integration — 47 pass, 1 skipped, 1 fail (BUG C below, proven
  pre-existing at the base). The Cloud-heartbeat leg passes.
• automations-e2e — 66 pass, 1 skipped, 3 fail (BUGs A and B below). Identical
  to the pre-rebase run, so the rebase changed nothing.
• redteam-e2e — 6 files, 23 pass, 1 skipped, 0 fail.
• demo-bank — 25 files, 119 pass, 0 fail. away-drill boots and passes in 24s
  now that its dist dir is a sibling; it had been timing out at 240s.

Three PRODUCT bugs these suites caught live in code this change does not touch.
The assertions are correct and should not be weakened to go green:

1. runs.list({ owner }) / ({ agent }) throw Unknown vendo_runs ref key: subject. runs-surface.ts:49-50 filters on refs vendo_runs does not
   declare (routing.ts:583 has only automation_id, status). Both are
   documented public filters and both are mapped from ?owner / ?agent on
   GET /runs, so this is a 500 on a public route.
2. A vendo.json manifest schedule can never be armed. manifest-triggers.ts
   states that "fn: steps capture an EMPTY consent surface", but
   consentSurface passes every step tool through verbatim, so capture rejects
   the app's own function with unknown tool in automation: fn:chase and the
   fold-in creates nothing.
3. fixtures/integration/tests/machine-skin — a guarded app read that HOLDS a
   live standing grant returns pending-approval instead of ok. Reproduced at
   origin/automations/s2c-vendo with this lane's files reverted, so it is not
   this change; my only edit there is the rename importAutomation → importApp.

Base

Stacked on automations/s2c-vendo (@ af8f2d0), so this diff is only the S3 lane
— e2e suites, the CI leg, the public docs, demo-bank. Do not merge ahead of the
lanes below it.

A note on the docs conflict ahead

main moved during this build (a507b92 → af2d337) and retired
E-SCHED-001. I reverted this lane's edits to that page so main's tombstone
applies cleanly. One correction still needs folding into it by whoever owns it:
the tick door verifies BOTH credentials against VENDO_TICK_SECRET, so a
Cloud-only deployment needs the secret too. That correction does ship, in the
three automations pages this lane owns.

---

Summary by cubic

Repoints automations from app-with-triggers to first‑class automation records across tests, docs, and demo, and gates the live‑agentic CI leg. Old: automations were triggers on apps and ran in Cloud. New: automations are records that run in your process; Cloud only wakes via POST /api/vendo/tick. Side effects: per‑record HMAC for webhooks, enable/capture and run ledger bind to record IDs, and docs reflect the new model.

• Adds ANTHROPIC_API_KEY to the automations CI leg only on pushes to main (skipped for PRs/merge groups).
• Public docs: new /backend/automate for .on(), rewrites automations and Cloud pages around records, fixes GET /automations shape, and updates links.
• E2E: automations-e2e, integration, and redteam harnesses and suites now create, enable, and run records; three new suites cover code authoring (agent.on), chat authoring (vendo_automate), and missing/duplicate runners; external webhooks verify per record with redacted keys.
• Demo bank: seeds automation records and run history; fixes missing owner on seeded runs and moves away-drill dist dir to a sibling to avoid build wipes.

Rollout and migration

• Set VENDO_TICK_SECRET in all deployments. Cloud’s signed heartbeat and any bearer credential are verified against it; without it, ticks are refused.
• Update any consumers of GET /automations: it returns redacted AutomationRecord[] (no pendingGrants or grantSetId).
• If you verify external webhooks, switch to per‑record HMAC keys minted at create; list/get redact them.
• Ensure agent runners are uniquely registered at boot; duplicate names throw at registration, missing names produce failed runs with no fallback runner.

Known product issues not changed by this PR (tests document them):

• runs.list({ owner|agent }) 500s on unknown ref keys.
• vendo.json manifest schedules cannot be armed due to consent-surface mismatch.
• Guarded app read with a standing grant returns pending-approval instead of ok.

^{Written for commit ee87fdd. Summary will update on new commits.}

[greptile-apps]

Greptile Summary

This PR moves fixtures, demo data, CI coverage, and documentation to standalone automation records. End-to-end checks found three failures that prevent expected automation workflows: public run filtering by owner or agent returns validation errors, manifest schedules that invoke app-local functions fail after being armed, and machine callbacks request approval again despite a matching active standing grant. These paths should be corrected before merge.

Confidence Score: 2/5

Not safe to merge until the verified automation execution, run filtering, and standing-grant regressions are fixed.

Each reported failure was reproduced through a composed runtime path with persisted state and captured output, covering public HTTP requests, manifest fold-in and ticking, and guarded machine callbacks.

Files Needing Attention: packages/vendo/src/wire/automations.ts, packages/automations/src/consent.ts, packages/vendo/src/wire/box.ts, and the corresponding store routing and guard matching implementations.

T-Rex Logs

What T-Rex did

• Created a minimal public runs-filter runtime reproduction script and captured initial runtime output and HEAD source contract to support the P1 finding.
• Executed the full-stack manifest function schedule fixture and captured evidence that the test source was copied from HEAD.
• Ran the machine-skin test with an active standing grant and logged the dependency layout blocker and the HEAD outcome with the standing grant parked.
• Validated contract behavior by reviewing the runs-filter repro script, the after-log with exit codes, and the HEAD source contract log.
• Reviewed the machine-skin standing-grant flow logs to verify the standing grant path and callback context behavior.

_{Ran code and verified through T-Rex}

Comments Outside Diff (6)

1. packages/vendo/src/wire/automations.ts, line 53-54 (link)

   Public run filters reference undeclared indexes

   GET /runs?owner=… and GET /runs?agent=… are accepted by the public route, but each request fails with HTTP 400 because runs.list translates them to subject and agent reference filters that vendo_runs does not declare. The unfiltered endpoint succeeds, so clients cannot use either documented filter. Add persisted routing projections and declared refs for these fields, or reject/remove the unsupported query parameters before calling the runs surface.

   Artifacts

   
   

   Minimal public runs-filter runtime reproduction script

   • This Vitest source creates a real store and composed handler, then invokes baseline, owner-filtered, and agent-filtered public GET requests; it is the exact executable reproduction.

   Public runs filter runtime output at HEAD

   • This executed Vitest log records the command, working directory, and observed HTTP bodies: unfiltered GET succeeds while owner and agent filters each return 400 unsupported-ref validation errors, proving the public filter failure.

   HEAD source contract for public run filters and store refs

   • This command capture prints the relevant HEAD route, runs-surface, and store-routing lines, showing that owner maps to subject and agent maps to agent while vendo_runs declares neither ref.

   

   _{Ran code and verified through T-Rex}

2. packages/automations/src/consent.ts, line 256-257 (link)

   Manifest function schedules are validated as host tools

   A manifest schedule with an app-local function step such as fn:chase creates and arms an automation record, but its first tick ends in an error: unknown tool in automation: fn:chase. Consent capture adds the fn: step to the host-tool surface and later resolves it through the host descriptor map, where app-local functions do not exist. Exclude fn: references from host-tool consent validation and dispatch them through the machine/app-function boundary so manifest schedules can complete.

   Artifacts

   
   

   Executed full-stack manifest function schedule fixture source

   • Exact HEAD source of the existing fixture executed to create a machine app, serve a vendo.json fn schedule, fold it in, arm it, and tick it; it defines the failing runtime path.

   Capture proving the executed test source was copied from HEAD

   • Command output records the exact git-show command that exported the executed fixture source from HEAD ee87fdd; it establishes the source artifact’s provenance.

   Full-stack manifest fn schedule reproduction output

   • Observed Vitest output from the real apps, manifest, automations, guard, store, and box fixture flow; it shows unknown tool in automation fn:chase and the resulting error run, confirming the runtime failure.

   Affected consent source lines at HEAD

   • Captured HEAD source lines show declared step tools entering the consent surface and being rejected when absent from the host descriptor map; it pinpoints the defect.

   

   _{Ran code and verified through T-Rex}

3. packages/vendo/src/wire/box.ts, line 307-315 (link)

   Machine callbacks discard matching standing grants

   A machine-backed app callback with a valid active standing grant for host_invoices_list returns pending-approval instead of ok. The box route marks every app-token callback as presence: "away"; away grant matching then requires an automation ID even for the app-bound source: "automation" grant used by this flow. Preserve app-bound machine authority in this callback context, while retaining the automation-ID requirement for actual automation-triggered away runs.

   Artifacts

   
   

   Machine-skin test source with active standing grant

   • Captured the exact fixture source executed to seed a matching app-bound standing grant and issue the guarded box callback; it establishes the expected ok response.

   Parent revision execution attempt blocked by pnpm dependency layout

   • Attempted the same fixture at the parent revision in a detached worktree, but pnpm aborted before Vitest because it required a non-interactive modules-directory purge; no parent behavior was executed.

   HEAD machine-skin test output with standing grant parked

   • Ran the narrow machine-skin fixture at HEAD and observed the guarded read return pending-approval where the fixture expected ok, validating the failure.

   Box callback context marks request away

   • Captured the executed callback route source showing that the app-token callback uses away presence and the app ID, explaining why away-only grant matching rejects the app-bound grant.

   

   _{Ran code and verified through T-Rex}

4. General comment

   Public run owner and agent filters always fail because their translated store refs are undeclared

   • Bug
     • GET /runs?owner=user_a returns HTTP 400 with Unknown vendo_runs ref key: subject; GET /runs?agent=support returns HTTP 400 with Unknown vendo_runs ref key: agent. The unfiltered equivalent returns HTTP 200 with the persisted run.
   • Cause
     • packages/vendo/src/wire/automations.ts:53-54 accepts owner and agent; packages/automations/src/runs-surface.ts:49-50 translates them to subject and agent refs; but packages/store/src/routing.ts:583 declares vendo_runs refs only as automation_id and status.
   • Fix
     • Make the persisted vendo_runs routing projection and indexes support the documented filters (for example, persist owner subject and agent projections and declare subject/agent in the route refs), or remove/reject the unsupported public query parameters before invoking the runs surface. Add composed wire tests for both filters.

   _{Ran code and verified through T-Rex}

5. General comment

   Manifest fn: schedule cannot execute because it is treated as a host tool

   • Bug
     • A vendo.json schedule such as { "cron": "* * * * *", "fn": "chase" } folds into an armed automation record, but its scheduled run ends in error because fn:chase is not in the host tool descriptor map. The existing full-stack fixture demonstrates the failure after the real automation tick.
   • Cause
     • consentSurface adds every declared steps tool, including the app-local fn: reference, to the consent surface (packages/automations/src/consent.ts:256-257). captureGrants subsequently resolves each item through byName and throws for fn:chase (packages/automations/src/consent.ts:304-307). fn: steps should not be resolved as host registry tools in this path.
   • Fix
     • Exclude fn: step references from the host-tool consent surface (and ensure the runtime dispatch path resolves them through the machine/app function boundary rather than the host tool registry). Add a full-stack regression test asserting a manifest fn: schedule both creates an armed record and completes its tick with status: "ok".

   _{Ran code and verified through T-Rex}

6. General comment

   Machine-backed app read ignores its active app-bound standing grant

   • Bug
     • The machine-skin end-to-end flow stores a valid standing grant for host_invoices_list (subject=user_ada, matching descriptor hash, tool scope, appId equal to the active app, source="automation") and then invokes the guarded host read through the box callback. The response is pending-approval, not ok, so the machine cannot complete a previously authorized read without a new approval.
   • Cause
     • packages/vendo/src/wire/box.ts:307-315 constructs all app-token box callbacks with presence: "away". Guard grant matching then applies the away-only condition in packages/guard/src/guard.ts:390-400: it requires grant.automationId === ctx.trigger?.automationId. The seeded app-bound automation grant has no automation ID and the callback context has no trigger, so the valid app grant is rejected before the default read policy parks the call.
   • Fix
     • Preserve the intended app-bound machine authority when constructing the box callback context or adjust presenceMatches to accept an app-bound, source:"automation" standing grant for an app-token callback when grant.appId === ctx.appId. Keep the existing automation-ID requirement for actual automation-triggered away runs; add an end-to-end regression covering an app-token guarded read with a matching grant and an ungranted/destructive callback that still parks.

   _{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "test(e2e,docs)!: the suites, the CI leg ..." | Re-trigger Greptile}