Skip to content

Commit b73ed48

Browse files
committed
docs(qa): re-point the checklist clauses the 17.6.0 release-verification run proved stale
The 17.6.0 run (#21330, subject 617f25f, Console 31971ff1e28f) failed five clauses whose expectation the platform deliberately contradicts, and found three expected-fail notes that 17.6.0 has since made pass. Each revision cites the commit that changed the behaviour and the run's measurement: - access-security.audit-log-browser rev 3: delete rows are written but not served to non-system readers since 30c530e (#21194). - api-backend.filter-comparand-conformance rev 3: POST /query refuses in the request schema (VALIDATION_FAILED), GET and the engine INVALID_FILTER (#20116). - api-backend.date-range-preset-matrix rev 2: the preset message is an ordering-position refusal; an equality is refused by the temporal door. - records-forms.import-transform-matrix rev 2: refusals carry no tracker number since f115b1f (#21188). - studio-authoring.view-authoring-live rev 2: the object door serves a container's expanded <object>.<key> items, by design (#7163). - automation.packaged-flow-subflow-disable-refusal rev 2: the remedy sequence holds (36d043b, 0d9349f); a caller's enable is guarded by design (679f95e). - automation.packaged-flow-clone-contract rev 2: durability fixed (cb4c31d); reachability still fails, now filed as #21332. - access-security.packaged-flow-write-door-parity rev 2: the /automation doors keep the packaged-base lock since 4b45afa (#20817). Claude-Session: https://claude.ai/code/session_018zT8d8NpiQ1ExhuNd5TxY6 Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1caa603 commit b73ed48

5 files changed

Lines changed: 58 additions & 40 deletions

File tree

‎docs/qa/platform-checklist/areas/access-security.json‎

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1199,7 +1199,7 @@
11991199
"title": "The audit-log browser surfaces attributable events over sys_audit_log with correct actor/object, filters, and a before/after payload drawer",
12001200
"since": "v16",
12011201
"status": "active",
1202-
"revision": 2,
1202+
"revision": 3,
12031203
"priority": "P1",
12041204
"surface": "mixed",
12051205
"personas": [
@@ -1220,21 +1220,21 @@
12201220
"open /system/audit-log; wait for the table to settle; screenshot",
12211221
"locate all three events in the page and read their action / object / actor cells",
12221222
"narrow to writes and confirm the narrowing is server-side; capture the re-issued /api/v1/data/sys_audit_log request with its $filter. ⚠️ There is no Action dropdown on this console: use the prebuilt filtered views (Recent / Writes / Auth / Config) or the generic Filter Records → Where → Add filter builder to constrain action",
1223-
"click the delete row → the side drawer opens; screenshot the Before (old_value) / After (new_value) JSON panels",
1223+
"click an UPDATE row (e.g. PATCH a showcase_contact first) → the side drawer opens; screenshot the Before (old_value) / After (new_value) JSON panels. Since 17.6.0 no delete row is served to the page (#21194), so the drawer is checked on an update row",
12241224
"cross-check the API twin directly: GET /api/v1/data/sys_audit_log?$filter=... for each of the three actions and compare actor/object/action to the page",
12251225
"attempt to forge the trail: POST and PATCH /api/v1/data/sys_audit_log — both must be refused (get+list only)"
12261226
],
12271227
"acceptance": [
12281228
{
1229-
"clause": "all three ops produce audit rows with the correct action, actor and target: login→action 'login' attributed to the member; delete→action 'delete' with object_name showcase_task + record_id; settings write→action 'config_change'",
1229+
"clause": "all three ops produce audit rows with the correct action, actor and target: login→action 'login' attributed to the member; settings write→action 'config_change' — both SERVED on the data API; delete→action 'delete' with object_name showcase_task + record_id is WRITTEN but, since 17.6.0, NOT served to a non-system reader (administrators included): the ledger serves only rows about records the reader can read, and a deleted record is readable by nobody (30c530e5, #21194)",
12301230
"oracle": "api",
1231-
"verify": "GET /api/v1/data/sys_audit_log returns the three rows; action/actor(user_id)/object_name/record_id match what each op did (fields per packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts)",
1232-
"evidence": "the three audit rows"
1231+
"verify": "GET /api/v1/data/sys_audit_log returns the login and config_change rows with action/actor(user_id)/object_name/record_id matching what each op did (fields per packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts), and returns NO row for $filter action='delete' even to the admin who deleted; the delete row's existence and attribution are proven from the store itself (the audit table in the telemetry database, read under system context or with the server stopped) — a delete row SERVED on the data API is the FAIL, and so is a delete row missing from the store",
1232+
"evidence": "the two served rows + the delete row's stored copy + the empty delete-filtered read"
12331233
},
12341234
{
12351235
"clause": "the browser renders those same rows: after a screenshot confirms the table painted, the DOM rows carry the same action/actor/object the API returned — the page shows server truth, not a recomputation",
12361236
"oracle": "dom",
1237-
"verify": "post-screenshot, the three rows' Action/Object/Actor cells equal the API values (read the DOM only after render is confirmed — hydration-race)",
1237+
"verify": "post-screenshot, the served rows' Action/Object/Actor cells equal the API values (read the DOM only after render is confirmed — hydration-race)",
12381238
"evidence": "screenshot + the row DOM read"
12391239
},
12401240
{
@@ -1250,9 +1250,9 @@
12501250
"evidence": "the drawer screenshot annotated 'before/after panels, not a diff'"
12511251
},
12521252
{
1253-
"clause": "the API twin reconciles with the page: GET /api/v1/data/sys_audit_log returns the same three events with matching actor/object — a page row without a backing API row (or vice versa) is a FAIL",
1253+
"clause": "the API twin reconciles with the page: GET /api/v1/data/sys_audit_log returns the same served events (login, config_change) with matching actor/object, and both omit the delete — a page row without a backing API row (or vice versa) is a FAIL",
12541254
"oracle": "api",
1255-
"verify": "field-by-field compare of the page's three rows against the /api/v1/data/sys_audit_log bodies",
1255+
"verify": "field-by-field compare of the page's rows against the /api/v1/data/sys_audit_log bodies",
12561256
"evidence": "API list vs page rows"
12571257
},
12581258
{
@@ -1283,7 +1283,8 @@
12831283
"change": "new — audit-log browser over sys_audit_log: attributable events, server-side filter, before/after payload drawer, API cross-check, append-only guard",
12841284
"ref": "claude/platform-test-checklist-ocwugl"
12851285
},
1286-
{ "revision": 2, "date": "2026-08-18", "change": "re-pointed clause 2 and step 5 at controls that exist. The item named an 'Action filter' dropdown; this console offers prebuilt filtered views (Recent / Writes / Auth / Config) plus a generic Filter Records / Where / Add filter builder. Server-side narrowing — the property the clause is actually for — is provable through the Writes view; naming a control that does not exist invites the absence-inference trap (#9453 CF-9)", "ref": "#9386" }
1286+
{ "revision": 2, "date": "2026-08-18", "change": "re-pointed clause 2 and step 5 at controls that exist. The item named an 'Action filter' dropdown; this console offers prebuilt filtered views (Recent / Writes / Auth / Config) plus a generic Filter Records / Where / Add filter builder. Server-side narrowing — the property the clause is actually for — is provable through the Writes view; naming a control that does not exist invites the absence-inference trap (#9453 CF-9)", "ref": "#9386" },
1287+
{ "revision": 3, "date": "2026-10-02", "change": "re-pointed clause 1 (and the clauses that counted its three rows) at 17.6.0's ledger read rule. 30c530e5 (#21194) serves a non-system reader, administrators included, only audit rows about records it can read, so the delete row is written but never served on the data API; the 17.6.0 release-verification run measured the admin's delete-filtered read at 0 rows while the stored row carried correct attribution. The clause now asserts the login and config_change rows served, the delete row stored and NOT served; step 6's drawer check moves to an update row, since no delete row reaches the page", "ref": "#21330" }
12871288
]
12881289
},
12891290
{
@@ -2273,10 +2274,10 @@
22732274
},
22742275
{
22752276
"id": "access-security.packaged-flow-write-door-parity",
2276-
"title": "Write-door parity on a packaged flow: PUT/DELETE /automation/:name must refuse the same packaged artifact that PUT /meta/flow/:name refuses (ADR-0126 §2 locked base) — EXPECTED FAIL today, the /automation door sails through on manage_metadata alone",
2277+
"title": "Write-door parity on a packaged flow: PUT/DELETE /automation/:name must refuse the same packaged artifact that PUT /meta/flow/:name refuses (ADR-0126 §2 locked base)",
22772278
"since": "v17",
22782279
"status": "active",
2279-
"revision": 1,
2280+
"revision": 2,
22802281
"priority": "P1",
22812282
"surface": "api",
22822283
"personas": [
@@ -2289,15 +2290,15 @@
22892290
"capture the flow's full current definition BEFORE any probe (GET /api/v1/automation/showcase_urgent_task_alert) — it is the restore payload"
22902291
],
22912292
"knownGaps": [
2292-
"the acceptance is the ADR-0126 §2 PARITY PROMISE ('the packaged base is locked — in-place edit refused loudly at the write door'), not today's behavior: as of #12438 the /automation door has NO lock — PUT/DELETE /automation/:name reach registerFlow/unregisterFlow with only the manage_metadata authoring gate in front (packages/runtime/src/domains/automation.ts), and the engine has zero lock/provenance check on that path (packages/services/service-automation/src/engine.ts). Clauses 2-3 are EXPECTED FAILS: a 200 there, where /meta refuses the same artifact, is the product finding, tracked centrally in FOLLOW-UPS (#12438). Keep the parity promise as the acceptance so the item flips green when the door is locked, without a rewrite"
2293+
"the acceptance is the ADR-0126 §2 PARITY PROMISE ('the packaged base is locked — in-place edit refused loudly at the write door'). It was an expected fail when authored (#12438: the /automation door had no lock); 4b45afae (#20817) gave the /automation write doors the packaged-base lock the /meta door keeps, and on 17.6.0 all three doors answer 403 NOT_OVERRIDABLE with the same lock message. The permanent pin is packages/qa/dogfood/test/packaged-flow-write-door-parity.dogfood.test.ts"
22932294
]
22942295
},
22952296
"steps": [
22962297
"boot showcase isolated; admin session; GET /api/v1/automation/showcase_urgent_task_alert and store the full definition (the restore payload); also GET /api/v1/meta/flow/showcase_urgent_task_alert?layers=true to prove the artifact is package-backed (populated code layer, _packageId com.example.showcase)",
22972298
"control leg, the /meta door: PUT /api/v1/meta/flow/showcase_urgent_task_alert with a trivially modified copy of the definition (e.g. label suffix) — capture status, code and WHICH layer answered; repeat with ?package=com.example.showcase and capture that code too",
22982299
"probe leg 1: PUT /api/v1/automation/showcase_urgent_task_alert with the same trivially modified definition — capture status and, if 2xx, GET the flow back to prove the live registration mutated",
22992300
"probe leg 2: DELETE /api/v1/automation/showcase_urgent_task_alert — capture status and, if 2xx, confirm GET /api/v1/automation/showcase_urgent_task_alert now 404s (the shipped flow is gone from the live engine)",
2300-
"RESTORE, unconditionally: PUT /api/v1/automation/showcase_urgent_task_alert with the stored original definition (re-registering is the cheap path; a cold restart's boot flow pull is the fallback), then GET it back and diff against the stored capture — byte-identical",
2301+
"RESTORE, only if a probe mutated anything: diff GET /api/v1/automation/showcase_urgent_task_alert against the stored capture; if it differs (or 404s), restore with a cold restart over the same database (the boot flow pull re-registers the packaged body — a restore PUT is itself refused 403 once the door is locked) and diff again — byte-identical",
23012302
"verify the flow still fires: trigger its record-change mutation once and confirm a run appears (the restore must revive the trigger binding, not just the definition read)"
23022303
],
23032304
"acceptance": [
@@ -2308,13 +2309,13 @@
23082309
"evidence": "both PUT traces (with and without ?package=) + the before/after /meta reads"
23092310
},
23102311
{
2311-
"clause": "parity, update door: PUT /api/v1/automation/showcase_urgent_task_alert against the SAME packaged artifact is refused — ⚠️ EXPECTED FAIL today: the door runs only the manage_metadata authoring gate (automation.ts) and registerFlow re-registers with no lock or provenance check (engine.ts), so a 200 here while /meta refused the identical artifact IS the finding. Record the fail with both traces side by side; the defect is tracked centrally in FOLLOW-UPS (#12438) — do not re-file it per run",
2312+
"clause": "parity, update door: PUT /api/v1/automation/showcase_urgent_task_alert against the SAME packaged artifact is refused with the same lock the /meta door applies (4b45afae, #20817). A 200 here while /meta refused the identical artifact is the fail — record both traces side by side",
23122313
"oracle": "api",
23132314
"verify": "same admin session, same artifact, same-shape body at both doors; the verdicts must MATCH. A 2xx on /automation with a mutated GET read-back, paired with the /meta refusal from clause 1, is a fail of this clause and the expected present-day outcome",
23142315
"evidence": "the /automation PUT trace + the mutated (or unchanged) GET read-back, paired with clause 1's refusal"
23152316
},
23162317
{
2317-
"clause": "parity, delete door: DELETE /api/v1/automation/showcase_urgent_task_alert is refused for the same reason — ⚠️ EXPECTED FAIL today (unregisterFlow, engine.ts, removes the shipped flow from the live engine with no check; 'delete first, refuse second' is the exact shape the #10145 measurement recorded at this door before the capability gate existed, and the lock half is still missing)",
2318+
"clause": "parity, delete door: DELETE /api/v1/automation/showcase_urgent_task_alert is refused for the same reason (4b45afae, #20817) — a 200 that removes the shipped flow from the live engine is the fail ('delete first, refuse second' is the shape the #10145 measurement recorded at this door before the lock existed)",
23182319
"oracle": "api",
23192320
"verify": "DELETE answers >=400 and the flow still serves; a 200 followed by a 404 on GET /api/v1/automation/showcase_urgent_task_alert is the fail (and the deletion this item's restore step exists to undo)",
23202321
"evidence": "the DELETE trace + the follow-up GET"
@@ -2349,7 +2350,8 @@
23492350
"date": "2026-08-26",
23502351
"change": "new — #12438 measured that PUT/DELETE /automation/:name re-register/unregister a PACKAGED flow on manage_metadata alone while PUT /meta/flow/:name refuses the same artifact: the ADR-0126 §2 lock is unimplemented at the /automation door. Authored as a door-parity item with the parity promise as the acceptance and the present-day 200 recorded as an expected fail (tracked centrally in FOLLOW-UPS), plus a mandatory restore step because the probe mutates a live engine registration",
23512352
"ref": "#12438"
2352-
}
2353+
},
2354+
{ "revision": 2, "date": "2026-10-02", "change": "retired the expected-fail framing: 4b45afae (#20817) locked the /automation write doors against the packaged base, and the 17.6.0 release-verification run measured PUT and DELETE /automation/showcase_urgent_task_alert answering 403 NOT_OVERRIDABLE with the /meta door's lock message, the flow unchanged and still firing. Title, knownGaps and clauses 2-3 now state the parity as holding; the restore step is conditional (a restore PUT is itself refused), and the knownGap names the permanent pin packaged-flow-write-door-parity.dogfood.test.ts", "ref": "#21330" }
23532355
]
23542356
},
23552357
{

0 commit comments

Comments
 (0)