Skip to content

fix(driver-mongodb)!: refuse a { $field } cross-field reference instead of sending it to MongoDB as a literal (#19949) - #20182

Merged
objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/issue-19949-mongodb-field-ref-refused
Sep 27, 2026
Merged

objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/issue-19949-mongodb-field-ref-refused

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #19949

Clause-②: no (narrowing)

What changed

translateFilter in @objectstack/driver-mongodb now refuses a { $field } cross-field reference in any comparand position. The refusal is INVALID_FILTER / 400, the envelope every other filter refusal on this driver uses. Before, the driver sent the reference to MongoDB as a literal sub-document.

This is the maintainer's ruling B on the card (triage comment 5807932277), quoted verbatim: 「维护者答:「19949 B」。」 The driver refuses the reference and does not implement it: there is no $expr column-to-column lowering. The driver-mongodb investment freeze (#5499) stays in force for everything else.

Rows: base 3bd28e2b2e vs head 00612182e9

MongoDB selection was read through mingo 7.2.4 as the proxy (the query-semantics library driver-memory uses), over the card's rows r1 {s:'a', t:'a'} and r2 {s:'a', t:'b'}, where ref is { $field: 't' }. A live mongod is NOT MEASURED: this container cannot fetch the binary, and the package's mongodb-memory-server suites are opt-in and were skipped. The readings were taken with a scratch script outside the suite, because mingo is not a dependency of this package.

shape base head
s $ne ref (what record.s != record.t lowers to) translated as a literal: selects r1, r2 400
s $eq ref (what record.s == record.t lowers to) translated as a literal: selects none 400
$gt / $gte / $lt / $lte ref literal: none 400
$in: [ref] literal: none 400
$nin: [ref] literal: selects r1, r2 400
$between: [ref, 'z'] / ['a', ref] literal bound: none 400
$and: [s $ne ref] / $or: [s $ne ref, s='zz'] selects r1, r2 400
$or: [{}, s $ne ref] TRUE identity, {}: r1, r2 400 (the gate is on the walk)
$not: {s $eq ref} (lowered to $nor) selects r1, r2 400
$not: {s $ne ref} none 400
s $ne {$field:'t', addDays:1} selects r1, r2 400
s $eq {$field:'t', addDays:{$field:'n'}} none 400
s $ne {$field:42} (malformed) selects r1, r2 400
$notContains: ref regex on the text of a plain object: selects r1, r2 400
$contains / $startsWith / $endsWith ref the same regex: none 400
$exists: ref lowered to $eq: null: none 400
bare { s: ref } already 400 (unknown operator $field, message names the field) 400, the cross-field message
$icontains: ref / $null: ref / $ne: [ref] already 400 (their own comparand gates) 400, the cross-field message
$nor at node level already 400 (undeclared combinator) unchanged
CONTROL s $ne 'a' / s $eq 'a' none / r1, r2 identical document, identical selection
CONTROL t $ne 'b' / t $eq 'b' r1 / r2 identical
CONTROL t $in ['b'] / t $nin ['b'] / t $between ['a','a'] / $not {t $eq 'b'} r2 / r1 / r1 / r1 identical

The RLS read path fails closed

Measured end to end with a scratch script. The path was compileCelToFilter, then the real RLSCompiler, the real SecurityPlugin with a rowLevelSecurity policy (operation: 'all'), ObjectQL, and the real MongoDBDriver over a stub Db whose collection selects with mingo. The caller is a MEMBER holding the permission set.

  • compileCelToFilter('record.s != record.t') and 's != t' both return { s: { $ne: { $field: 't' } } }. RLSCompiler.compileFilter keeps it, because s and t are declared.
  • Base: under using: 's != t', find returned r1, r2, count returned 2, and findOne({ id: 'r1' }) returned r1, the row the policy excludes. The server received {"s":{"$ne":{"$field":"t"}}}.
  • Head: find, findOne and count each throw INVALID_FILTER / 400, and the collection is asked 0 times. The refusal is not swallowed and the read never falls back to the unfiltered set. using: 's == t' (base: 0 rows, the wrong answer in the fail-closed direction) is also refused.
  • Controls, identical at base and head: using: 't == "a" gives r1 / count 1, using: 't != "a" gives r2, and using: 's == "a" gives r1, r2.

Every driver door reads where through translateFilter (find, findOne, count, updateMany, deleteMany, aggregate via buildAggregationPipeline, explain), so the one gate covers them all. The suite pins each door and the engine rethrow. It uses an ObjectQL middleware that composes the policy the way the security middleware does, because @objectstack/plugin-security is not a dependency of this package.

Collateral

Tests at 00612182e9

  • pnpm --filter @objectstack/driver-mongodb test: 28 files passed, 5 skipped (the opt-in live-mongod suites); 630 tests passed, 147 skipped.
  • pnpm --filter @objectstack/driver-mongodb exec vitest run --maxWorkers=2 src/mongodb-field-reference-refusal.test.ts: 39 passed.
  • pnpm --filter @objectstack/driver-mongodb typecheck: tsc --noEmit, then check:test-typecheck with 0 errors. tsc -p tsconfig.test.json --listFiles includes the new test file among 33 test files.
  • Reverse verification (ablation), run after the fix was committed at a53249d5e5. node scripts/ablation-replace.mjs deleted the gate line (anchor 1 to 0, blob 7b33578be43b to 343b8c5aa323). With the gate gone, the new file went 33 failed / 6 passed: every refusal red, all six controls green. The tool then restored the file (blob back to 7b33578be43b, the HEAD blob, and git diff HEAD empty). No dist/ step was needed, because the suite imports the translator by relative path from src.
  • The CJS entry dist/index.js loads and refuses, built at 00612182e9.

Gates

  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 59 commands from this diff at 00612182e9. All 59 were run, and the --ran reconciliation reports 57 run and 2 NOT MEASURED.
  • 57 exited 0.
  • pnpm check:dual-build-cjs-loads exited 3: PREREQUISITE NOT MET, because it needs the whole workspace built. NOT MEASURED. Narrowed instead: this package's CJS entry loads, as above.
  • pnpm check:type-check-debt exited 3: PREREQUISITE NOT MET, because the ledgered packages' closure is not built. NOT MEASURED. This package carries no DEBT entry, and its own test-layer typecheck is green.
  • Also run: node scripts/check-issue-citations.mjs --base 3bd28e2b2e (exit 0, 5 citations resolve), plus the five artifact-roster gates whose rosters sit under this diff's directories: check:authz-resolver, check:error-code-casing, check:filter-alias-parity, check:object-def-param-keys and check:tenant-chokepoint, all exit 0.
  • check-changeset-no-major and check-adr-0087-registration both pass on the changeset. The first accepts the minor level, and the second reads the not-required (no-migration-prescription) disposition.
  • Lint was narrowed to the two changed TypeScript files: eslint --no-inline-config --format json reports 2 files, 0 errors, 0 warnings. Both files are in the config's population, because --print-config gives each one a rule set. The narrowing cannot hide anything: the config enables no type-aware linting (no parserOptions.project or projectService), so this diff cannot change the verdict on any other file. The full pnpm lint is left to CI.

Acceptance notes

  • Not in scope, per the ruling: implementing field-to-field comparison on MongoDB (a $expr lowering). A policy that needs it cannot be enforced on this driver. It is now refused instead of read without the restriction.
  • An observation, not a card. $exists with a non-boolean comparand still translates to { $eq: null } ("has no value") on this driver, because the arm tests value === true. Measured on translateFilter at head: 'yes', 1, null and 'false' all give $eq: null. It has no comparand gate like $null's, and the engine's comparand-type door lists $exists as a scalar operator. Only a reference is refused here. Whether a public door delivers this shape, and what the other drivers answer, was not measured. It is left alone under the driver-mongodb freeze. Carrier: none.

Generated by Claude Code

… comparand position

translateFilter emitted the reference as a literal sub-document, so an
RLS using clause such as s != t matched every row. The shape walk now
refuses it with the INVALID_FILTER / 400 envelope, field and operator
withheld.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
…t both doors

Also refuses a reference listed in the implicit-equality position, so a
list holding one answers the same refusal as $eq with that list.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation tests tooling labels Sep 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

5 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 54 of 207 client-bound route-ledger rows — the other 153 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 153: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 98 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 6 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 455dcc060d5c712cc4fbb161b132827a0900e3ac → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 92e10884d995531c08848fdea68308f5275833e2 — the merge of head 00612182e92d42f7efaeb99ee2b51cad01672c68 into base 455dcc060d5c712cc4fbb161b132827a0900e3ac, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 92e10884d995531c08848fdea68308f5275833e2 && git checkout 92e10884d995531c08848fdea68308f5275833e2
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 455dcc060d5c712cc4fbb161b132827a0900e3ac 00612182e92d42f7efaeb99ee2b51cad01672c68 && git checkout -B drift-repro 455dcc060d5c712cc4fbb161b132827a0900e3ac && git merge --no-ff 00612182e92d42f7efaeb99ee2b51cad01672c68

node scripts/docs-audit/affected-docs.mjs --json 455dcc060d5c712cc4fbb161b132827a0900e3ac

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 00612182e92d42f7efaeb99ee2b51cad01672c68

Scope: PR #20182 for card #19949 (priority:p2, security), the maintainer's ruling B (triage 5807932277, verbatim 「维护者答:「19949 B」。」): refuse a { $field } reference in driver-mongodb translateFilter, fail closed on the RLS read path, ⛔ no $expr lowering, ⛔ no other driver-mongodb defect, ⛔ not packages/spec / compileCelToFilter / the RLS compiler. Diff: 3 files, +426/−0 (mongodb-filter.ts +87, new mongodb-field-reference-refusal.test.ts +303, new .changeset/19949-mongodb-field-reference-refused.md +36). Merge base with origin/main = 3bd28e2b2ea81206dc9f5507de205b564bd97097; origin/main at review = 836aad2a19. Measured in detached worktrees of head and base with the driver-mongodb / plugin-security / formula / driver-memory closures built; mingo 7.2.4 (the version pnpm-lock.yaml pins for driver-memory) as the selection proxy over r1 {s:'a',t:'a'}, r2 {s:'a',t:'b'}. A live mongod: NOT MEASURED (no binary in this container). Inputs read: the card, comments 5807932277 / 5814595680 / 5853096324 / 5853343199 (the last as a claim to check), PRs #19947 and #19882, the PR body, diff and check-runs; AGENTS.md, contract-review.md, landing-operations.md from origin/main.

① Derived judgments

  • The rows (base → head), through translateFilter + mingo 7.2.4. 34 reference shapes measured; 34/34 refused at head with INVALID_FILTER / 400, all with the one cross-field message. Base cells, each matching the PR table: s $ne ref → {"s":{"$ne":{"$field":"t"}}} selects r1,r2; $eq / $gt / $gte / $lt / $lte ref → none; $in [ref], $in [x,ref] → none; $nin [ref] → r1,r2; $between [ref,z] / [a,ref] → literal bound, none; $and [s $ne ref], $or [s $ne ref, s zz] → r1,r2; $or [{}, s $ne ref] → {} (TRUE identity), r1,r2; $not {s $eq ref} → $nor, r1,r2; $not {s $ne ref} → none; three combinators deep → translated; {$field:'t', addDays:1} under $ne → r1,r2; {$field:'t', addDays:{$field:'n'}} under $eq → none; malformed {$field:42} and {$field:null} under $ne → r1,r2; $contains/$startsWith/$endsWith ref → regex on [object Object], none; $notContains ref → r1,r2; $exists ref → {$eq:null}, none; bare {s:[ref]} and $eq:[ref] → passed through, none; s {$gte 'a', $ne ref} → r1,r2. Already refused at base and still refused at head (wording moved, envelope same): bare {s: ref}, $icontains: ref, $null: ref, $ne: [ref]. $nor at node level: 400 unknown-combinator at both, unchanged. Judgment: correct — the ruling's two cases and every reachable position fail closed.
  • Literal controls: 21/21 byte-identical documents and identical selections base vs head, including s $ne/$eq 'a', t $ne/$eq 'b', bare t:'b', t $in/$nin ['b'], t $between ['a','a'], $not {t $eq 'b'}, $and/$or with a literal, $gt, $contains/$notContains, $exists true, $null false, the [finding] the comparand-SHAPE face declares it closes the door "for every driver at once", but an array in the IMPLICIT-EQUALITY slot passes it — and driver-mongodb alone answers it, as an exact-array match #19757 equality-slot array pin, the fix(formula, driver-mongodb): refuse == / != against a list literal at the CEL lowering and $ne arrays at the mongodb face #19947 $ne-array refusal, the empty-constraint and $where refusals. Judgment: no collateral movement.
  • One non-reference shape checked and unchanged: { s: { nested: { $field: 't' } } } translates at both base and head as the nested-document exact match (none). Not a comparand-position reference under FieldReferenceSchema; correct, recorded as an observation.
  • The RLS read path fails closed, on the real chain. compileCelToFilter('record.s != record.t') and ('s != t') → { s: { $ne: { $field: 't' } } }; ('record.s == record.t') → $eq. RLSCompiler.compileFilter with declared {id,s,t} keeps both. Through the real SecurityPlugin (rowLevelSecurity, operation: 'all', MEMBER) → ObjectQL → real MongoDBDriver over a mingo-backed stub Db that records every call: base using 's != t' — find → [r1,r2], findOne({id:'r1'}) → r1, count → 2, aggregate count → 2, bulk update → 2, bulk delete → 2; the collection received {"s":{"$ne":{"$field":"t"}}} (7 calls). using 's == t' — 0 rows / null / 0. Head — all six doors throw INVALID_FILTER / 400, 0 collection calls, and the engine logs at WARN then rethrows. Controls identical at base and head: t == "a" → [r1] / count 1, t != "a" → [r2], s == "a" → [r1,r2]. Judgment: fails closed; not swallowed.
  • Every driver door. Driver-level, with the engine's $and composition: base find/findOne/count/updateMany/deleteMany/aggregate/explain all reached the collection (7 calls); head all seven refused 400, 0 calls. Every where read in mongodb-driver.ts goes through translateFilter; no door bypasses it. CJS entry dist/index.js built at head also refuses (INVALID_FILTER 400).
  • Disclosure. The message is one constant string: names the unsupported feature (does not support field-to-field comparison ({ "$field": … })) and the remedy; carries no field name, no referenced field, no operator, no path. Same posture as PR fix(formula, driver-mongodb): refuse == / != against a list literal at the CEL lowering and $ne arrays at the mongodb face #19947's arrayNotEqualComparandError. The four rewordings are not a regression: at base three of them disclosed the field and position and the fourth named the operator; at head all four disclose strictly less and name the actual defect. No test pinned the old wording for a reference.
  • The pins discriminate. At head intact: mongodb-field-reference-refusal.test.ts 39/39 passed. Gate line deleted (anchors 1 → 0): 33 failed / 6 passed of 39; the 33 reds are every refusal case plus the driver-door and both RLS using pins; the 6 greens are exactly the six controls. Restored with git checkout HEAD -- <path>: blob 7b33578be43be5911fce83a9766e3edb741f6ac2 = HEAD: blob, git status --porcelain empty. Matches the dev's count.
  • Scope and merge. git diff --name-only 3bd28e2b2e..00612182e9 = the 3 files only; packages/spec, packages/formula (compileCelToFilter), packages/plugins/plugin-security (RLS compiler) untouched; no deleted lines in mongodb-filter.ts; $expr appears in the file only in prose refusing it. git merge-tree --write-tree origin/main 00612182e9 → clean, exit 0.
  • PR body, sentence by sentence against head — FALSE sentence named: Collateral: "Three shapes were already refused and now get the cross-field message instead of their old one" — four (bare {s: ref}, $icontains: ref, $null: ref, $ne: [ref]); the fourth is stated in the preceding bullet and in the rows table, so the count is wrong and the content is disclosed. Every other checkable sentence held.
  • Changeset, sentence by sentence — imprecision named: line 34, "Before this change it returned every row." — true for the negated forms (!=, $nin, $notContains, $eq under $not), false as a generalisation: an == policy returned no rows. Line 26 of the same changeset states both directions correctly, so the reader has the accurate statement; recorded as an imprecision, not a contract falsity. Every other sentence held.
  • Pending changesets (grep of .changeset/ at head for mongodb / $field, 25 files): none reads FALSE after this PR. Noted, not false: [finding] a { $field } reference as a $between endpoint is refused by the schema door and accepted by the runtime door — the #19071 shape, one endpoint spelling over, already ruled on the schema side #19377's changeset recommends the two-bound { "$gte": { "$field": … }, "$lte": … } spelling as the "position that compiles on every face" — on driver-mongodb that spelling now gets this 400 (before, it was silently mistranslated); carrier: none, see ③.
  • Published surfaces touched by the narrowing, not edited here: content/docs/references/data/filter.mdx (generated from the spec's EQ_DESCRIPTION / NE_DESCRIPTION) can be read as a { $field } reference lowering on MongoDB; it never did and is now refused. Fixing it is a packages/spec describe() edit the claim and ruling exclude. See ③.

② Semver level

  • Declaration Clause-②: no (narrowing) is correct under AGENTS.md's closed pair: no key is added to any published payload (FieldReferenceSchema unchanged; no export, type or schema moves), and the accept set of one driver's filter-compile door shrinks — (narrowing) ⇒ BREAKING. @objectstack/driver-mongodb: minor with the **BREAKING** banner and fix(driver-mongodb)!: is the launch-window level. No other package's published source moves.
  • ADR-0087 disposition not-required (no-migration-prescription): correct. Nothing authorable is removed or renamed, no stored policy can be converted to keep its meaning on this driver, and the author-facing remedy is stated in the changeset.
  • node scripts/check-changeset-no-major.mjs --base 3bd28e2b2e --head HEAD --event <pull_request payload carrying the PR's body>, exit 0: "✓ This diff introduces no major bump." / "✓ LEVEL AXIS: this PR declares clause-② no (narrowing), and no package whose packages/**/src/** it moves is graded patch." / "· direction arm: narrowing — a BREAKING change; during the launch window it ships minor".
  • node scripts/check-adr-0087-registration.mjs --base 3bd28e2b2e --head HEAD, exit 0: "✓ check-adr-0087-registration: 1 declared-breaking changeset(s), each carrying an ADR-0087 disposition. .changeset/19949-mongodb-field-reference-refused.md [BREAKING+bang+clause-②-narrowing] not-required (no-migration-prescription)".

③ Boundary flags

  • Dev open_questions: none. Dev deviations, each answered: (a) followed the repo's os-dev.md over a stale copy — accepted. (b) the gate covers the bare/implicit-equality position and list members beyond the ruling's literal "operator value" wording — accepted as inside ruling B: the implicit slot is $eq by declaration, list members are the values of $in/$nin/$between, and the ruling's ⛔ is 「不再当作普通值下发」. (c) four already-refused shapes reworded — no regression. (d) the suite's engine-level pin uses an ObjectQL middleware stand-in — accepted; the real SecurityPlugin chain was measured outside the suite and agrees.
  • Dev out-of-scope observation ($exists with a non-boolean comparand → $eq: null, carrier none): concur it is an observation under the [裁决] driver-memory / driver-mongodb 投入冻结 —— 维护者 2026-08-05 口径(跨单锚点) #5499 freeze.
  • Reviewer flag 1 (not blocking; spec lane, outside this claim): the generated filter.mdx rows for $eq / $ne read as if a { $field } comparand lowers on MongoDB; after this PR that is an explicit 400. Same for the two-bound prescription in [finding] a { $field } reference as a $between endpoint is refused by the schema door and accepted by the runtime door — the #19071 shape, one endpoint spelling over, already ruled on the schema side #19377's changeset.
  • Reviewer flag 2 (observation): { field: { nested: { $field } } } still passes as a nested-document literal at head — correct per the spec's comparand-position definition; no action.
  • Live mongod: NOT MEASURED here and by the dev; mingo 7.2.4 is the proxy in both readings.
  • CI at head, read last, all 34 check runs complete: 31 success, 3 skipped, 0 failure. The seven required contexts are all success: Lint & Repo Gates, TypeScript Type Check, Test Core, Dogfood Regression Gate, Build Core, Temporal Conformance (live PG + MySQL), Governed Surface Queue Guard. Also success: Check Changeset.

Implemented-by: claude/issue-19949-mongodb-field-ref-refused
Reviewed-by: session_01Bvd69VPa6puiNzzPUroDBx

VERDICT: PASS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants