From 70c5f8552ce2e7c763db88683fd84051b066ceec Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 22:21:14 +0000 Subject: [PATCH 1/3] wip(docs): predicate compilation convergence design skeleton Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude --- docs/design/predicate-compilation-convergence.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 docs/design/predicate-compilation-convergence.md diff --git a/docs/design/predicate-compilation-convergence.md b/docs/design/predicate-compilation-convergence.md new file mode 100644 index 00000000000..f6799ec4de0 --- /dev/null +++ b/docs/design/predicate-compilation-convergence.md @@ -0,0 +1,3 @@ +# Design / Cost Account — Predicate Compilation Convergence + +Work in progress: census, measured tax, candidate shape, cost account, recommendation. From a3c75d3399a811c1b394f1d45500599e3a7456d7 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 22:40:18 +0000 Subject: [PATCH 2/3] wip(docs): predicate convergence census and measured tax Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude --- .../predicate-compilation-convergence.md | 227 +++++++++++++++++- 1 file changed, 226 insertions(+), 1 deletion(-) diff --git a/docs/design/predicate-compilation-convergence.md b/docs/design/predicate-compilation-convergence.md index f6799ec4de0..2c409bdf2ae 100644 --- a/docs/design/predicate-compilation-convergence.md +++ b/docs/design/predicate-compilation-convergence.md @@ -1,3 +1,228 @@ # Design / Cost Account — Predicate Compilation Convergence -Work in progress: census, measured tax, candidate shape, cost account, recommendation. +**Card**: [#5930](https://github.com/objectstack-ai/objectstack/issues/5930) — the investigation the maintainer approved in principle on 2026-08-14 (comment `5293341484`), dispatched 2026-09-29. +**Governed by**: [ADR-0053](../adr/0053-date-and-datetime-semantics.md) (D-A1, D-D1, D-E3) · [ADR-0058](../adr/0058-expression-and-predicate-surface.md) (D1, D6) · [ADR-0112](../adr/0112-error-code-vocabulary-and-ledger.md) (refusal envelopes) +**Status**: proposal. This document measures and recommends. ⛔ It decides nothing: the build decision is the maintainer's. +**Anchor**: every count, `file:line` and probe reading below was taken on objectstack commit `3711e0b763` (`origin/main` at dispatch, 2026-09-29) unless the line names another commit. + +The scope, verbatim from the maintainer's approval and the triage wake that started it: + +> **the predicate-compilation convergence INVESTIGATION is approved in principle** — scope: evaluate shape A (shared semantic tree, dialect-only compilers) using the #5977-validated pattern as the candidate form, deliverable = design + cost account, not a build. + +> **交付物**:一份**设计 + 成本账**,⛔ 不是实现 +> +> **评估对象**:形态 A —— 共享语义树,各编译器只管方言。用 #5977 已验证过的那个模式(把守卫下沉进树结构后,3 个消费者零改动同时对齐)作候选形态 + +--- + +## 0. The answer in brief + +- **Census (§1).** Eleven faces turn a `FilterCondition` into SQL, a query document or a boolean, through 15 source files. One of them (F10) builds its own tree and compiles it three times. The card counted 5 faces plus one half in August. +- **The refusal half has already converged; the lowering half has not (§1.3).** About ten shared doors in `@objectstack/spec`, `@objectstack/core` and `@objectstack/objectql` decide what a filter may say, and they run at six seams. What a filter *means* is still hand-copied. The NULL-polarity helpers exist in 4 copies, and the whole-day upper-bound rule is called from 10 face files (41 call sites). +- **The measured tax (§2).** + - The `$empty` ruling took 4 cards, 4 PRs and 13 dev rounds, and edited 13 distinct face files. + - The last-supported-day ruling took 2 cards, 2 PRs and 3 rounds, and edited 10 distinct face files. + - The temporal-comparand ruling was enforced at a shared seam. It took 1 PR and 1 round, and edited 0 face files. + - In the 46 days since the approval, 22 commits on `main` edited 3 or more face files. By subject, 18 of them are filter semantics. +- **#6050 re-check (§2.4).** Measured at each face's own entry point, the `undefined`-comparand split is still there. Four faces answer an `undefined` comparand, each in its own way, and `driver-mongodb` passes it to the wire. Every production path I traced now refuses it at a shared door, except the `driver-memory` cube face. That face is a published export, and it runs no door. +- **Shape A holds (§3).** #5977 put the NULL guard into service-analytics' shared tree, and its three compilers aligned with no edits. The same package keeps its whole-day rule in the emitters, and PR #20643 had to edit three of its files for that rule. +- **Recommendation (§5): a smaller first slice.** Build one shared semantic lowering (`FilterCondition` → `FilterCondition`) beside the existing doors, and run it at the seams that already run them. It would hold `$between`, the whole-day bound including the last supported day, and the NULL-polarity guards. The per-face copies would then be deleted lane by lane. This is not the full typed-IR build, and it adds no new gate. It does require an amendment to ADR-0053 D-D1. + +--- + +## 1. Census — every face that turns a `FilterCondition` into a predicate (H1) + +### 1.1 The faces + +Vocabulary is counted against the 19 field operators an author can write. That is the 17 of `FILTER_OPERATORS` plus `$like` / `$ilike`, which `StringOperatorSchema` declares while they are staged out of that array. + +| # | Face | Entry (`file:line`) | Read path it serves | Field operators | Shared comparand doors it runs itself | +|---|---|---|---|---|---| +| F1 | `driver-sql` | `SqlDriver.applyFilters` → `compileFilters` → `applyFilterCondition` (`sql-driver.ts:16350` / `16462` / `16789`) | `find`/`findOne`, `count`, `aggregate`, `distinct`, `updateMany`, `deleteMany`, `findWithWindowFunctions`, `analyzeQuery` on PostgreSQL, MySQL and SQLite. `driver-sqlite-wasm` and `driver-turso` local/replica mode inherit it (`extends SqlDriver`). | 19/19, plus `$and`/`$or`/`$not` and `$field` | none (relies on the engine seam); calls `reduceFilterVerdict`, `isAcceptedFilterComparand`, `expandEmptyOperator` | +| F2 | `driver-turso` remote | `TursoDriver.toRemoteFilter` (`turso-driver.ts:2566`, a lowering pass) → `RemoteTransport.buildWhereSQL` / `compileWhereSQL` (`remote-transport.ts:2852` / `2896`) | remote-mode `find`, `count`, `aggregate`, `distinct`, `updateMany`, `deleteMany` | 18/19 (`$between` is lowered before the transport) | none; calls `isAcceptedFilterComparand`, `expandEmptyOperator` | +| F3 | `driver-memory` query path | `InMemoryDriver.convertToMongoQuery` (`memory-driver.ts:1334`) → mingo `Query`, behind `assertFilterConditionShape` (`filter-refusal.ts`) | `find`/`findOne`, `count`, `distinct`, `aggregate`, `updateMany`, `deleteMany` | 19/19 | none (relies on the engine seam) | +| F4 | `driver-memory` reference matcher | `match()` (`memory-matcher.ts:43`) | **no production caller.** It is not exported from the package index, and 20 test files import it. | 19/19 | none | +| F5 | `driver-memory` cube face | `MemoryAnalyticsService.query` / `generateSql` → `normalizeFilters` (`memory-analytics.ts:910` / `1389` / `1501`) → mingo `$match` and echo SQL | the published `@objectstack/driver-memory` export. No in-repo door constructs it (recorded on #20661). | 12/19 (no `$between`, `$startsWith`, `$endsWith`, `$null`, `$empty`, `$like`, `$ilike`); `$and` is its only combinator | **none** — see §2.5 | +| F6 | `driver-mongodb` | `translateFilter` (`mongodb-filter.ts:805`); the aggregation `$match` reuses it (`mongodb-aggregation.ts:556`) | all CRUD verbs and `aggregate` | 17/19 (no `$like`/`$ilike`) | none (relies on the engine seam) | +| F7 | `formula` | `matchesFilterCondition` (`matches-filter.ts:322`) | the RLS `check` on a write's post-image (`security-plugin.ts:3215`), the tenant check (`:3519`), the explain engine (`explain-engine.ts:964`), and F8's scalar comparisons (`having-filter.ts:1155`) | 19/19 | none (the RLS compile seam runs the doors first) | +| F8 | `objectql` `having` / per-aggregation filter | `applyHaving` / `matchesHaving` / `matchesAggregationFilter` (`having-filter.ts:1167` / `1190` / `1244`) | `ObjectQL.aggregate`: `having` (`engine.ts:16608`, `16652`) and in-memory `aggregations[i].filter` | 17/19 (no `$like`/`$ilike`) | the engine runs both doors on `having` and on each `aggregations[i].filter` first (`engine.ts:16387`, `16470`) | +| F9 | `service-analytics` read scope | `compileScopedFilterToSql` (`read-scope-sql.ts:747`) | the RLS/tenant scope merged into the NativeSQL statement (`native-sql-strategy.ts:668`) and into the `/analytics/sql` echo (`objectql-strategy.ts:578`); also a published export | 17/19 (no `$like`/`$ilike`) | `assertListComparandShapes`, `normalizeFilterComparandTypes` (`read-scope-sql.ts:982`) | +| F10 | `service-analytics` `where` → tree | `normalizeAnalyticsFilterTree` (`filter-normalizer.ts:2356`) builds a `NormalizedFilterNode`, which three compilers read (F10a–c) | widget and dataset `where` | 17/19 (`$like`/`$ilike` refused) | both, through `normalizeWhereComparands` (`filter-normalizer.ts:2218`) | +| F10a | ↳ NativeSQL | `compileFilterNode` / `buildFilterClause` (`native-sql-strategy.ts:1061` / `1114`) | the SQL that executes | (the tree's) | — | +| F10b | ↳ echo | `renderFilterNodeSql` / `buildFilterClauseSql` (`objectql-strategy.ts:1651` / `1260`) | the `/analytics/sql` echo | (the tree's) | — | +| F10c | ↳ engine hand-off | `filterNodeToCondition` (`objectql-strategy.ts:1602`) | a `FilterCondition` handed back to the engine, which F1, F3 or F6 then compile a second time | (the tree's) | — | +| F11 | `service-analytics` draft preview | `evaluateAnalyticsQueryOverRows` → `matchesWhere` (`preview-evaluator.ts:640` / `325`) | the draft-preview branch of `queryDataset` (`analytics-service.ts:1819`), reached through REST `?preview=` (`rest-server.ts:5872`) | 10/19 (`$eq $ne $gt $gte $lt $lte $between $in $nin $contains`); the rest are refused | both, through `normalizeWhereComparands` (`preview-evaluator.ts:665`) | + +The 15 source files are `sql-driver.ts`, `turso-driver.ts`, `remote-transport.ts`, `memory-driver.ts`, `filter-refusal.ts`, `memory-matcher.ts`, `memory-analytics.ts`, `mongodb-filter.ts`, `matches-filter.ts`, `having-filter.ts`, `read-scope-sql.ts`, `filter-normalizer.ts`, `native-sql-strategy.ts`, `objectql-strategy.ts` and `preview-evaluator.ts`. §2 uses this list as its "face files". + +### 1.2 Against the card's table + +- **Named by the card:** + - F1; + - F2's transport half; + - F9; + - F10, which the drivers seat's correction (`5205970996`) split into F10a / F10b, plus F10c; + - F7; + - F8, named as "the sixth and a half". +- **Never named:** + - **F2's lowering half.** `TursoDriver.toRemoteFilter` puts comparands into storage form, applies the whole-day rule and lowers `$between` before the transport sees the filter. Its own comment says "`RemoteTransport` stays the dumb SQL builder it is documented to be". It is a small shape A that already exists. + - **F3, F4 and F5.** `driver-memory` has three faces, not one. + - **F6.** The card named it only to exclude it. + - **F11.** The preview evaluator appears neither in the card nor in the correction. + - **F8's second half.** This is the per-aggregation `filter` walker (#20122). +- **Out of the census, because they are not faces:** + - front ends that *produce* a `FilterCondition`: `compileCelToFilter`, `parseFilterAST`, the REST view-filter-rule lowering and `rls-compiler.ts`; + - static analysers that model the faces: `lint`'s `validate-rls-predicate-enforceability.ts`; + - filters that are not a `FilterCondition`: knowledge-memory's metadata equality, realtime subscription options and metadata watch filters. +- **NOT MEASURED:** the sibling repositories (`objectui`, `cloud`). No checkout was available in this container. + +### 1.3 The shared layer that exists today — refusal is shared, meaning is not + +Since August, most of the *refusal* half has moved into shared modules: + +| Rule | One definition | Where it runs | +|---|---|---| +| boolean identities (`$and: []`, `$or: []`, `$not: {}`) | `reduceFilterVerdict` (spec `filter-verdict.ts`, #5659) | inside F1, F3/F4 and F6 | +| comparand shape (list operators, the equality slot, the `$ne` slot) | `assertListComparandShapes` (spec `filter-comparand-shape.ts`) | the engine seam, the RLS compile seam, the read-scope door, the analytics `where` door, the preview door, `parseFilterAST` and the save door | +| comparand type, including `undefined` (ruling #7872) | `normalizeFilterComparandTypes` (spec `filter-comparand-type.ts`) | the same seams | +| a temporal comparand the column can read | core `isUninterpretableTemporalComparand` + objectql `temporal-comparand-door.ts` | the engine seam (plus the analytics raw-SQL decline) | +| number and text declared-type doors | spec + objectql door modules | the engine seam | +| `$empty`'s per-type row | `expandEmptyOperator` / `isEmptyFilterValue` (spec `filter-empty-operator.ts`) | called inside 9 faces | +| the `$like` pattern language, the ASCII fold | helpers in spec `filter.zod.ts` | called inside the faces | +| retired operators | `RETIRED_FILTER_OPERATORS` | the refusal text of 5 faces | +| the calendar day and the last supported day | `nextUtcCalendarDay` / `isUnboundedAbove` (spec `calendar-day.ts`, re-exported by core) | called inside 10 faces, 41 call sites | + +The engine seam is already a *lowering* seam, not only a gate. `lowerWhereFilterArray` (`engine.ts:947`) runs six of these doors in sequence and returns a rewritten `where`: `narrowNumberComparands` and `normalizeFilterComparandTypes` rewrite comparands, copy-on-write. + +The *meaning* half is still hand-copied, and these counts are measured: + +- **The NULL-polarity quartet** (`nullValueSatisfiesOperator`, `operatorIsNullTotal`, `nullGuardForFieldSpec`, `nullSafeNegationOperand`) exists in 4 copies: + - `sql-driver.ts:5052`, 67 non-comment code lines; + - `remote-transport.ts:425`, 69 lines; + - `read-scope-sql.ts:2158`, 67 lines; + - `filter-normalizer.ts:1477`, 90 lines. + + That is 293 lines. The JS faces reach the same answer through two-valued evaluation, and F4 has its own `noValueSatisfiesNegation`. +- **The `undefined` refusal beside the door.** `assertDefinedComparands` exists in `sql-driver.ts:4682`, `remote-transport.ts:4365`, `read-scope-sql.ts:1607` and `filter-normalizer.ts:970`. +- **The whole-day upper bound.** It is called from 10 face files: `memory-driver` 8 call sites, `sql-driver` 5, `memory-analytics` 5, `native-sql-strategy` 4, `preview-evaluator` 4, `having-filter` 4, `mongodb-filter` 4, `turso-driver` 3, `objectql-strategy` 2 and `matches-filter` 2. F9 never calls it (§2.5). +- **The operator→SQL tables.** F10a's `opMap` / `likeShape` and F10b's `SCALAR_SQL_OPS` / `LIKE_SQL_OPS` are a declared pair: "they move together". +- **The `$between` lowering.** It is written separately in F2's lowering pass, in F10's normalizer, and in F1's own arm plus `calendarDayBetweenRewrite`. + +### 1.4 The #5499 freeze does not put anything out of scope + +The wake's census line excludes `driver-memory` / `driver-mongodb` 「按 #5499」. That freeze was dissolved on 2026-08-11: + +- for `driver-mongodb` in comment `5249019855` on #5499; +- for `driver-memory` in comment `5252526378`: "this card's freeze is fully dissolved". + +The spec records the same fact in `aggregation-conformance.ts`, and #5499's own thread logs six sightings of the dissolved freeze being cited as live. The census therefore includes F3–F6. F5 is a live face: PR #20714 edited it on the day of this dispatch. + +Whether a build includes them is the maintainer's scope call (§6). For the recommended slice, the question mostly dissolves: F3 and F6 sit behind the engine seam, so they would receive the lowered filter without being edited. + +--- + +## 2. The measured tax (H2) + +### 2.1 Three recent rulings, counted face by face + +Definitions used in this section: + +- A **round** is one `os-dev-report` comment on the card, that is, one dev run that delivered a report. +- **Face files** are the 15 census files of §1.1, edited in source. Test files are not counted. + +Everything below is read from the merged commits and the cards' threads. + +| Ruling | Cards | PRs (commits) | Rounds | Face-file edits (distinct) | Shared files edited | What the rounds were spent on | +|---|---|---|---|---|---|---| +| `$empty` — ruling A on #20399 (#20311 declares it → #20445 analytics → #20444 engine faces → #20446 flip) | 4 | #20442 (7), #20498 (4), #20523 (7), #20570 (18) | 2 + 2 + 1 + 8 = **13** | 4 + 9 + 8 = 21 (**13 distinct**) | `filter-empty-operator.ts` and 5 other spec files | The operator stayed staged out of `FILTER_OPERATORS` until "one compile-surface lane card per face" had landed. #20446's rulings in rounds 2 and 3 were about *which faces hold a field declaration*: federated objects, an analytics host with no `sourceFieldMeta`, and harness tables no driver had declared. | +| Last supported day — #20600, then #20661 | 2 | #20643 (19), #20714 (3) | 2 + 1 = **3** | 10 + 1 = 11 (**10 distinct**) | spec `calendar-day.ts`, `temporal-conformance.ts`; core `datetime.ts` | Round 1 failed at tier because the new sentinel was typed twice across two declaration files. The fix then had to be narrowed at "17 direct call sites in 9 files, plus Turso's indirect one" (review `5894730557`). The same work found a tenth face whose own order of operations was wrong: F5 converted to storage form before widening. That became #20661. | +| Temporal comparand door — #20549 + #20480 | 2 | #20668 (2) | **1** | **0** | core `temporal-comparand.ts`, objectql `temporal-comparand-door.ts`, `record-validator.ts` | None. The rule lives at the engine seam, so the faces changed only in tests. | + +And the pattern this investigation evaluates: #5977 (PR #6004, 7 files, 1 commit) moved the #5298 NULL guard into service-analytics' shared tree. It aligned three compilers with no edits to them. + +### 2.2 The natural experiment + +In the same week, in the same lanes: + +- A rule enforced once at a shared seam cost 1 round and 0 face files. +- Two rules implemented at the emitters cost 3 and 13 rounds and 10 and 13 distinct face files. + +Inside one package the contrast is sharper still. service-analytics keeps the #5298 NULL guard in its tree (`filter-normalizer.ts`), and #5977 cost its three compilers nothing. It keeps the whole-day rule in its emitters, and #20643 had to edit three of its files for that rule: `native-sql-strategy.ts`, `objectql-strategy.ts` and `preview-evaluator.ts`. + +### 2.3 How often the tax is paid + +`node scripts/pm/git-history.mjs log --since=2026-08-14 --ref=3711e0b763` gave 5,364 first-parent commits. The receipt reads "complete clone (no fetch)". Of those commits: + +| edited face files | commits | +|---|---| +| ≥ 1 | 221 | +| ≥ 3 | 22 | +| ≥ 5 | 5 | +| face-file edits in all | 331 | + +Of the 22 commits that edited three or more face files, 18 change filter-predicate semantics. The other 4 are driver-door, docs or aggregation work that happens to live in the same large files. That split is a reading of the commit subjects, not a measurement. The list is in Appendix B, and §4.3 classifies the 18 by what shape A would have done with them. + +### 2.4 The #6050 re-check: an `undefined` comparand, face by face + +Triage asked whether the split recorded on 2026-08-07 (`5214990870`) still holds. On that day `formula` read `undefined` as key-absent, `read-scope-sql` compiled `= NULL` and so answered zero rows, and `driver-memory` read `undefined` as `null`. + +I ran #6125's fixture through every face at its **own** entry point, with no door in front. In the fixture, `d` holds a value on rows 1–2, is `null` on row 3, and is absent from row 4. The probe is described in Appendix A. + +| Face (entry called directly) | `{d: undefined}` | `{d: {$eq: undefined}}` | `{d: {$ne: undefined}}` | `{$not: {d: {$ne: undefined}}}` | `{d: {$in: ['v1', undefined]}}` | +|---|---|---|---|---|---| +| spec door `normalizeFilterComparandTypes` | refused 400 | refused 400 | refused 400 | refused 400 | refused 400 | +| F1 `SqlDriver.find` (SQLite) | refused 400 | refused 400 | refused 400 | refused 400 | refused 400 | +| F2 `RemoteTransport.buildWhereSQL` | refused 400 | refused 400 | refused 400 | refused 400 | refused 400 | +| F3 `InMemoryDriver.find` | `[3,4]` | `[3,4]` | `[1,2]` | `[3,4]` | `[1]` | +| F5 `MemoryAnalyticsService.query` | `[3,4]` | `[3,4]` | `[1,2]` | refused 400 (no `$not`) | `[1]` | +| F6 `translateFilter` | passed through: `{"d":undefined}` | passed through | passed through | `{"$nor":[{"d":{"$ne":undefined}}]}` | passed through | +| F7 `matchesFilterCondition` | `[4]` | `[4]` | `[1,2,3]` | `[4]` | `[1,4]` | +| F8 `matchesHaving` (row walker alone) | `[3,4]` | `[3]` | `[1,2]` | `[3,4]` | `[1]` | +| F9 `compileScopedFilterToSql` | refused 500 `READ_SCOPE_COMPILE_FAILED` | refused 500 | refused 500 | refused 500 | refused 500 | +| F10 `normalizeAnalyticsFilterTree` | refused 400 | refused 400 | refused 400 | refused 400 | refused 400 | +| F11 `evaluateAnalyticsQueryOverRows` | refused 400 | refused 400 | refused 400 | refused 400 | refused 400 | + +Every "refused 400" is `INVALID_FILTER` / 400. + +**Reading:** + +1. **Inside the faces, the split is still there, and it is wider than the August record.** + - F7 still reads `undefined` as key-absent, as it did in August. + - F3 still reads it as `null`, as it did in August. + - F8 and F5 are two more readings, and F8 disagrees with F3 on `$eq`. + - F6 hands `undefined` to the BSON encoder. How the server answers was NOT MEASURED, because there was no `mongod` here. + - F9 moved from `= NULL` to a refusal (PR #6390). +2. **On the production paths, the split is masked by doors.** Since #7872 the comparand-type door runs at each of these seams: + - the engine seam: `where`, `aggregations[i].filter` and `having`; + - the RLS compile seam, `judgeCompiledComparands` (`rls-compiler.ts:284`, #20212), which serves both `using` and `check`, so F7 never receives a policy comparand the door has not judged; + - the analytics `where` and preview doors; + - the read-scope door. + + I traced those paths. I did **not** exhaustively trace direct driver callers, the security tenant check (`security-plugin.ts:3519`) or the explain engine. For direct driver callers, the spec's comparand-type door says those drivers "inherit it by receiving already-validated input". +3. **One published face has no door in front of it: F5.** Its reading, `{d: undefined}` ≡ `{d: null}`, is confirmed by controls: no `where` answers `[1,2,3,4]`, `{d: 'v1'}` answers `[1]`, and `{d: null}` answers `[3,4]`. It is §2.5 item 2. + +So the question's answer is: **yes, the faces still answer each in its own way, and the platform now hides it by refusing at the seams.** That is the state this document names "refusal converged, meaning not converged". A new entry point that skips a door, as F5 does, re-exposes the difference at once. + +### 2.5 Divergences the census found + +These were measured at the faces' published or internal entry points. None was measured at an HTTP door. They are reported to the seat for filing and are **not** filed from here. + +1. **F9 has no whole-day upper bound, and it binds temporal comparands exactly as written.** `compileScopedFilterToSql({t: {$lte: '2026-07-28'}}, 't')` compiles to `"t"."t" <= ?` and binds `'2026-07-28'`. I ran it on SQLite over ISO-text rows at `2026-07-27T10:00Z`, `2026-07-28T10:00Z`, `2026-07-29T10:00Z` and `null`: + + | face | rows kept | + |---|---| + | F9 read scope | only the 07-27 row, `[1]` | + | F1 `find` (column declared `datetime`) | `[1,2]` | + | F7 | `[1,2]` | + | F11 | `[1,2]` | + + One RLS `using` policy would therefore hide the named day's rows in NativeSQL analytics while CRUD reads and the policy's own `check` admit them. + + ADR-0053 D-A1 says: "any surface that binds a filter comparand into raw SQL (analytics `NativeSQLStrategy` today, and any future raw-SQL strategy) **must** coerce through the driver's dialect-aware temporal coercion". D-D1's list of emitters does not include F9. I found no in-repo policy that writes a bare-day bound, and PostgreSQL and MySQL were NOT MEASURED. +2. **F5 runs neither shared comparand door.** Its `where` goes straight into `normalizeFilters` (`memory-analytics.ts:1501`). Neither `assertListComparandShapes` nor `normalizeFilterComparandTypes` is called anywhere in the file. The measured consequence is the F5 row of §2.4. +3. **F11's `$contains` folds case.** The spec's `$contains` description says: "CASE-SENSITIVELY … answered case-exactly on every JS evaluation face the platform ships". F11's row is `String(value).toLowerCase().includes(String(expected).toLowerCase())` (`preview-evaluator.ts:179`). I measured it through `evaluateAnalyticsQueryOverRows` over the rows `ACME Corp` / `acme ltd` / `Beta`: + - `{name: {$contains: 'acme'}}`: preview `[1,2]`, F7 `[2]`; + - `{name: {$contains: 'ACME'}}`: preview `[1,2]`, F7 `[1]`. + + So a drafted chart would count rows the published chart excludes. `FILTER_TEXT_CASES` is not wired into F11's suites. The REST `?preview=` door was NOT MEASURED. +4. **F4 has no production caller** (§1.1), yet rulings keep paying for it. #20444 edited it, #13356 is the no-value fix, and #13166 aligned it. It is a test oracle carried as a face. From c814d601647d87ce3417144441a6256b9402fe0b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 22:45:15 +0000 Subject: [PATCH 3/3] =?UTF-8?q?docs(design):=20predicate=20compilation=20c?= =?UTF-8?q?onvergence=20=E2=80=94=20census,=20measured=20tax,=20shape=20A?= =?UTF-8?q?=20and=20cost=20account?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude --- .../predicate-compilation-convergence.md | 352 +++++++++++++++++- 1 file changed, 347 insertions(+), 5 deletions(-) diff --git a/docs/design/predicate-compilation-convergence.md b/docs/design/predicate-compilation-convergence.md index 2c409bdf2ae..5ddcd742e70 100644 --- a/docs/design/predicate-compilation-convergence.md +++ b/docs/design/predicate-compilation-convergence.md @@ -63,7 +63,7 @@ The 15 source files are `sql-driver.ts`, `turso-driver.ts`, `remote-transport.ts - F9; - F10, which the drivers seat's correction (`5205970996`) split into F10a / F10b, plus F10c; - F7; - - F8, named as "the sixth and a half". + - F8, named as 「第六个半」. - **Never named:** - **F2's lowering half.** `TursoDriver.toRemoteFilter` puts comparands into storage form, applies the whole-day rule and lowers `$between` before the transport sees the filter. Its own comment says "`RemoteTransport` stays the dumb SQL builder it is documented to be". It is a small shape A that already exists. - **F3, F4 and F5.** `driver-memory` has three faces, not one. @@ -82,7 +82,7 @@ Since August, most of the *refusal* half has moved into shared modules: | Rule | One definition | Where it runs | |---|---|---| -| boolean identities (`$and: []`, `$or: []`, `$not: {}`) | `reduceFilterVerdict` (spec `filter-verdict.ts`, #5659) | inside F1, F3/F4 and F6 | +| boolean identities (`$and: []`, `$or: []`, `$not: {}`) | `reduceFilterVerdict` (spec `filter-verdict.ts`, #5659) | inside F1, F4 and F6 (and the flow linter) | | comparand shape (list operators, the equality slot, the `$ne` slot) | `assertListComparandShapes` (spec `filter-comparand-shape.ts`) | the engine seam, the RLS compile seam, the read-scope door, the analytics `where` door, the preview door, `parseFilterAST` and the save door | | comparand type, including `undefined` (ruling #7872) | `normalizeFilterComparandTypes` (spec `filter-comparand-type.ts`) | the same seams | | a temporal comparand the column can read | core `isUninterpretableTemporalComparand` + objectql `temporal-comparand-door.ts` | the engine seam (plus the analytics raw-SQL decline) | @@ -135,14 +135,14 @@ Everything below is read from the merged commits and the cards' threads. | Ruling | Cards | PRs (commits) | Rounds | Face-file edits (distinct) | Shared files edited | What the rounds were spent on | |---|---|---|---|---|---|---| | `$empty` — ruling A on #20399 (#20311 declares it → #20445 analytics → #20444 engine faces → #20446 flip) | 4 | #20442 (7), #20498 (4), #20523 (7), #20570 (18) | 2 + 2 + 1 + 8 = **13** | 4 + 9 + 8 = 21 (**13 distinct**) | `filter-empty-operator.ts` and 5 other spec files | The operator stayed staged out of `FILTER_OPERATORS` until "one compile-surface lane card per face" had landed. #20446's rulings in rounds 2 and 3 were about *which faces hold a field declaration*: federated objects, an analytics host with no `sourceFieldMeta`, and harness tables no driver had declared. | -| Last supported day — #20600, then #20661 | 2 | #20643 (19), #20714 (3) | 2 + 1 = **3** | 10 + 1 = 11 (**10 distinct**) | spec `calendar-day.ts`, `temporal-conformance.ts`; core `datetime.ts` | Round 1 failed at tier because the new sentinel was typed twice across two declaration files. The fix then had to be narrowed at "17 direct call sites in 9 files, plus Turso's indirect one" (review `5894730557`). The same work found a tenth face whose own order of operations was wrong: F5 converted to storage form before widening. That became #20661. | +| Last supported day — #20600, then #20661 | 2 | #20643 (19), #20714 (3) | 2 + 1 = **3** | 10 + 1 = 11 (**10 distinct**) | spec `calendar-day.ts`, `temporal-conformance.ts`; core `datetime.ts` | Round 1 failed at tier because the new sentinel was typed twice across two declaration files. The fix then had to be narrowed at "17 direct call sites in 9 files, plus Turso's indirect one" (the seat's ACCEPT record `5894771815`, summarizing review `5894730557`). The same work found a tenth face whose own order of operations was wrong: F5 converted to storage form before widening. That became #20661. | | Temporal comparand door — #20549 + #20480 | 2 | #20668 (2) | **1** | **0** | core `temporal-comparand.ts`, objectql `temporal-comparand-door.ts`, `record-validator.ts` | None. The rule lives at the engine seam, so the faces changed only in tests. | And the pattern this investigation evaluates: #5977 (PR #6004, 7 files, 1 commit) moved the #5298 NULL guard into service-analytics' shared tree. It aligned three compilers with no edits to them. ### 2.2 The natural experiment -In the same week, in the same lanes: +Over the same two days (2026-09-28 and 09-29): - A rule enforced once at a shared seam cost 1 round and 0 face files. - Two rules implemented at the emitters cost 3 and 13 rounds and 10 and 13 distinct face files. @@ -192,7 +192,7 @@ Every "refused 400" is `INVALID_FILTER` / 400. - F8 and F5 are two more readings, and F8 disagrees with F3 on `$eq`. - F6 hands `undefined` to the BSON encoder. How the server answers was NOT MEASURED, because there was no `mongod` here. - F9 moved from `= NULL` to a refusal (PR #6390). -2. **On the production paths, the split is masked by doors.** Since #7872 the comparand-type door runs at each of these seams: +2. **On the production paths, the split is masked by doors.** The comparand-type door (ruling #7872) now runs at each of these seams: - the engine seam: `where`, `aggregations[i].filter` and `having`; - the RLS compile seam, `judgeCompiledComparands` (`rls-compiler.ts:284`, #20212), which serves both `using` and `check`, so F7 never receives a policy comparand the door has not judged; - the analytics `where` and preview doors; @@ -226,3 +226,345 @@ These were measured at the faces' published or internal entry points. None was m So a drafted chart would count rows the published chart excludes. `FILTER_TEXT_CASES` is not wired into F11's suites. The REST `?preview=` door was NOT MEASURED. 4. **F4 has no production caller** (§1.1), yet rulings keep paying for it. #20444 edited it, #13356 is the no-value fix, and #13166 aligned it. It is a test oracle carried as a face. + +--- + +## 3. Candidate shape A — the shared semantic tree (H3) + +### 3.1 The #5977 pattern, stated precisely + +#5977 did four things, and each is a design rule: + +1. **The rule became structure in a tree every compiler already reads.** `fieldLeaves` emits a negative-polarity leaf as `or(notSet, comparison)`. The `$not` operand gets one more `{col: {$null: false}}` conjunct. The compilers compile what they are handed. +2. **Which leaves get the rule was decided once.** The deciding table is the polarity pair `nullValueSatisfiesOperator` / `operatorIsNullTotal`, and it is asked about the *comparand*, never about a list of operator names. This is why `$ne: null` stays `IS NOT NULL` and is not widened. +3. **No compiler needed to know the rule.** F10a, F10b and F10c aligned with no edit. +4. **Applying it twice is harmless.** The engine path guards twice (`c IS NULL OR (c IS NULL OR c <> v)`), and that is the same predicate. Idempotence is what lets the rest of a migration go face by face. + +Two more in-repo instances already work at the `FilterCondition` level. `nullSafeNegationOperand` rewrites a `$not` operand before emission, in each of its four copies. F2's `TursoDriver.toRemoteFilter` rewrites `$between`, the whole-day bound and storage form before the transport sees the filter. + +### 3.2 What the shared tree would hold + +| Rule | Today | In the tree | +|---|---|---| +| comparand shape and type, `undefined` included (#6050, #7872); whether a temporal comparand is interpretable (#20549) | shape and type: 1 shared door, plus 4 local `assertDefinedComparands` copies. Temporal: the engine seam only. | the existing doors run first, so the lowered output never holds a refused comparand. A temporal comparand leaves the tree as a calendar-day or ISO string, and storage form stays with the driver (§3.3). | +| boolean identities (#5322, #5134) | shared `reduceFilterVerdict`, called by each face with its own hooks | TRUE / FALSE are resolved once; an empty combinator is gone or is a constant | +| NULL polarity (#5146 `$not`; #5298 `$ne` / `$nin` / `$notContains`) | 4 hand copies, 293 code lines | a negative-polarity leaf becomes `{$or: [{f: {$null: true}}, {f: {op: v}}]}`; a `$not` operand is made total leaf by leaf, exactly as the copies do | +| `$between` | lowered separately in F1, F2 and F10 | two conjuncts, `$gte` and the upper bound below | +| whole-day upper bound (ADR-0053 D-D1), last supported day (#20600) | 10 face files, 41 call sites; F9 has none | a `$lte` on a bare day, or a `$between` maximum on one, becomes `$lt` `nextUtcCalendarDay(day)` in the **calendar-string** domain; on `9999-12-31` it becomes `{$null: false}` | +| order of widening and storage form (ADR-0053 D-E3) | kept by convention in each face; F5 broke it (#20661) | structural: the tree runs before any face converts to storage form, so the #20661 class cannot recur | +| `$exists` (#13529, "has a value") | read per face | one spelling of presence (`$null`) | +| `$empty` (#20311, declared row) | `expandEmptyOperator` called inside 9 faces | the `null` and `''` rows as structure; the empty-list test stays a dialect leaf (§3.3) | +| one operator per conjunct | operator maps, where a lowered key can collide with a sibling (df18120502; Turso remote, noted on #20643) | the lowering emits `$and` conjuncts, so it cannot create a collision | + +### 3.3 What stays per dialect — the last inch + +- **Storage form of comparands and columns.** ADR-0053 D-A1 keeps the driver as "the single source of dialect truth" (`temporalFilterValue` / `temporalFilterColumnSql`). The tree hands a calendar string, and the driver converts it. +- **Text-match constructs.** + - `LIKE` or `GLOB`, and `ESCAPE`; + - the ASCII-fold construct per dialect (#6518, #15684, #16020); + - the `$like` → `GLOB` translation, which is already one shared helper. +- **JSON-column constructs.** Membership for `$contains` on a JSON column (#17590), and the empty-list test for `$empty` on a multi-value column. +- **Surface mechanics.** Identifier quoting, aliasing, bind placeholders and parameter renumbering; Mongo `$nor` / `$regex`; mingo documents. +- **Refusal envelopes, which depend on context.** + - `INVALID_FILTER` / 400 against `READ_SCOPE_COMPILE_FAILED` / 500; + - the SQL drivers' withheld-diagnostic redaction (#7929, #8220). +- **Gates that read the physical column map.** Examples are the JSON-column operator refusal and the non-text-column constant (#14079). They could move up once declarations reach the seam. The slice leaves them where they are. + +### 3.4 The output vocabulary — the constraint the census exposes + +Every face that consumes the lowering must accept its output. Today's intersection is too small: + +- F5 accepts 12 operators and only `$and`. +- F11 accepts 10 operators, with no `$null` and no `$exists`. +- Together that leaves `$eq $ne $gt $gte $lt $lte $in $nin $contains` and `$and`. + +That set has no null test and no `$or`. So neither the NULL guard nor the last-day `{$null: false}` could be handed to F5 or F11 as they stand. + +The design therefore fixes a **closed output vocabulary**: `$eq $ne $gt $gte $lt $lte $in $nin`, the text family, `$null` (boolean) and `$and $or $not`. It widens F5 (`$or`, `$not`, `$null`) and F11 (`$null`) as part of the migration. Both faces evaluate in two-valued mingo/JS, which express all three natively. + +The alternative is to skip the guard at the JS seams, because it is redundant under two-valued evaluation. That would re-create a per-face exception, which is the thing being removed. + +### 3.5 Two representations + +- **T1 — `FilterCondition` → `FilterCondition`,** with each operator split into its own `$and` conjunct. + - Strengths: + - Every face already reads it. + - The engine seam already returns a rewritten `where`. + - Both in-repo precedents (§3.1) are T1. + - Faces keep their own copies at first, harmlessly, and delete them later. + - Limits: + - TRUE / FALSE are spelled `{}` / `{$not: {}}` rather than typed. + - There is no new leaf kind: "empty list" has no spelling. + - A face can still misread a shape the lowering emits. +- **T2 — a typed tree** with kinds `const` / `and` / `or` / `not` / `leaf{field, op, value}`, compiled by every face. + - Service-analytics' `NormalizedFilterNode` is the in-repo prototype, with Cube operator names. + - Constants and leaf lists are explicit, and "compilers own only dialect" is literally true. + - The cost is a rewrite of every compile entry in §1.1. + +T1 is a step toward T2, not a detour, provided it lives where T2 would live: T2 is T1's output parsed into kinds. + +### 3.6 Where it runs, and where it lives + +**It runs at the seams that already run the doors**, so no new entry point is created. The seam positions, seven in five files: + +| Seam | Position | +|---|---| +| engine `where` | `lowerWhereFilterArray`, after `normalizeFilterComparandTypes` (`engine.ts:1034`) | +| engine `aggregations[i].filter` | `engine.ts:16387` | +| engine `having` | `engine.ts:16470` | +| RLS compile seam, serving `using` and `check` | `judgeCompiledComparands` (`rls-compiler.ts:284`) | +| analytics `where` and preview door, serving F10 and F11 | `normalizeWhereComparands` (`filter-normalizer.ts:2218`) | +| read scope | the entry of `compileScopedFilterToSql` (`read-scope-sql.ts:747`) | +| F5 — a new door | `normalizeFilters` (`memory-analytics.ts:1501`) | + +**Where it lives is a decision (§6 D3).** + +- **Beside the doors in `@objectstack/spec/data`.** That follows the precedent of `filter-verdict.ts`, `filter-comparand-*.ts`, `filter-empty-operator.ts` and `calendar-day.ts`, and adds no dependency edge. `@objectstack/formula` depends on `@objectstack/spec` alone. +- **In `@objectstack/core`.** That honours Prime Directive #2 ("No business logic in `packages/spec`") literally, but adds a `formula → core` edge. + +### 3.7 What shape A does not remove + +A ruling that adds a new predicate *kind* or a dialect construct still costs one arm per compiler. Examples are JSON membership (#17590), the per-dialect case folds (#15684, #16020) and the empty-list test. The tree removes the tax for rulings that are *compositions* of leaves every face already compiles. §4.3 counts both kinds. + +### 3.8 Governance + +- **ADR-0053 D-D1 would be reversed in placement.** It says "Operator-sensitive translation lives at the comparison EMITTERS". Moving the whole-day rewrite into a shared lowering reverses where that translation lives, so it needs a new ADR or an amended status line (Prime Directive #13). `docs/adr/**` is a Tier H surface. +- **D-A1 and D-E3 stand.** The driver still owns storage form, and widening still happens before conversion. The lowering makes D-E3 structural. +- **ADR-0058 D1 / D6 are compatible.** `FilterCondition` stays the canonical pushdown shape. The lowering sits after it and before the backends. +- **No ADR records where the NULL polarity or the doors live.** Those are rulings recorded in code (#5146, #5298, #7872), so moving them needs no ADR. + +--- + +## 4. Cost account (H4) + +### 4.1 One-time cost of the slice, step by step + +- **S1 — the lowering module, T1.** It is a pure function. Its pins are its own unit table: + - each rule of §3.2, input to output; + - idempotence, `lower(lower(x)) === lower(x)`; + - closure of the output vocabulary (§3.4). + + `FILTER_LOGIC_CASES` and `TEMPORAL_CASES` can be driven through `lower` and then F7. This is an ordinary suite, not a gate. +- **S2 — the seam calls.** Seven positions in five files (§3.6). The F5 call also installs the two comparand doors F5 lacks (§2.5 item 2). +- **S3 — widen F5 and F11 to the output vocabulary** (§3.4). +- **S4 — the ADR-0053 D-D1 amendment,** by the maintainer's hand (Tier H). +- **S5 — deletions.** One lane card per group. Each is proven by that face's existing suites staying green before and after, with an ablation. + +| Face | Receives the lowered filter via | Edited by the slice (S2 / S3) | Deleted later (S5) | Suites that hold it today | Risk | +|---|---|---|---|---|---| +| F1 `driver-sql` | the engine seam | no | polarity quartet (67 lines), `assertDefinedComparands`, the `calendarDay*Rewrite` calls (5 sites) — only under D4 (b) | `FILTER_LOGIC`, `FILTER_TEXT`, `TEMPORAL`, `FILTER_COMPARAND_TYPE` on SQLite, plus the PostgreSQL/MySQL live matrix | direct callers (D4); a 21,103-line file | +| F2 `driver-turso` remote | the engine seam, then `toRemoteFilter` | no | `toRemoteFilter`'s `$between` / whole-day arms (3 sites), the transport's polarity copy (69 lines) | turso filter-logic (local and remote), local/remote NULL parity | a live remote server was NOT MEASURED here | +| F3 `driver-memory` query | the engine seam | no | whole-day calls (8 sites) | memory filter-logic, temporal, text | — | +| F4 reference matcher | — (no production caller) | no | keep as test oracle, or retire (D6) | 20 test files | — | +| F5 cube face | the new `normalizeFilters` door | **yes**: doors + lowering, and widen `$or` / `$not` / `$null` | whole-day calls (5 sites) | its own suites; not in `check:driver-conformance` | an accept-set widening, so a changeset with its Clause-② line | +| F6 `driver-mongodb` | the engine seam | no | whole-day calls (4 sites) | mongodb filter-logic, text, temporal, comparand-type | the server answer was NOT MEASURED here | +| F7 `formula` | the RLS compile seam (policies); the engine (via F8) | no | `lteBound` (2 sites), only under D4 (b): F7 is also a public evaluator (explain engine, test doubles) | matches-filter not-null-safe, or-semantics, temporal | — | +| F8 `having` | the engine seam | no | whole-day calls (4 sites) | having filter-logic, text, temporal | — | +| F9 read scope | the new call at its entry | **yes**: the lowering call | polarity quartet (67 lines), `assertDefinedComparands` | read-scope-sql conformance, read-scope not-null-safe | fixes §2.5 item 1, which changes the rows a scope returns, so it needs a changeset | +| F10 `where` → tree | the analytics `where` door | no | polarity quartet (90 lines), `assertDefinedComparands`, the `$between` lowering | native-sql filter-logic, text, temporal | echo fidelity (R6) | +| F11 preview | the analytics door it shares with F10 | **yes**: widen `$null` | `lteBound` (4 sites) | preview-temporal conformance | — | + +**Measured sizes:** + +- 7 seam positions in 5 files; +- 3 faces edited by the slice; +- 293 code lines in 4 polarity copies; +- 4 local `assertDefinedComparands` copies; +- 41 whole-day call sites in 10 face files. + +**Effort — an estimate, NOT a measurement.** Using §2.1's measured rounds as the yardstick (a card that touches one shared module and at most three faces took 1–2 rounds): + +- S1–S3 are about one engine-lane card and one analytics/memory card; +- S5 is about four lane cards; +- in all, about 6 cards and 8–12 rounds, plus the maintainer's ADR amendment. + +### 4.2 The conformance net that would hold it + +| Case-set | Non-spec test files wiring it today | +|---|---| +| `FILTER_LOGIC_CASES` (36 cases) | 33 (the card counted 11 hand-copied wirings in August) | +| `FILTER_TEXT_CASES` | 19 | +| `TEMPORAL_CASES` | 10 | +| `TEMPORAL_TIME_CASES` | 9 | +| `FILTER_COMPARAND_TYPE_CASES` | 8 | + +`node scripts/check-driver-conformance.mjs` answers OK: "50 covered cell(s), 0 in the DEBT ledger, 0 exempt". It scores the five driver packages only. Its header leaves `formula` and service-analytics "to their own suites". + +That scope is how two of §2.5's findings stayed invisible: + +- F9 wires no `TEMPORAL_CASES`, which is how item 1 survived; +- F11 wires no `FILTER_TEXT_CASES`, which is how item 3 survived. + +The slice needs **no new gate**. It needs the lowering's own unit table. It would also benefit from two ordinary suites: `TEMPORAL_CASES` through F9, and `FILTER_TEXT_CASES` through F11. + +### 4.3 The recurring tax the slice removes + +The 18 filter-semantics commits of §2.3, classified by what shape A would have done with them. The classification is my reading of each commit's subject and diff, not a measurement. + +| Class | Commits | Shape A's effect | +|---|---|---| +| structure over leaves every face compiles | `1a75e39d4a` (#20600 last day), `a646120dc7` (text operator over a stored non-string value, which needs declarations at the seam), `9dac1ae017` (`$exists` = has a value), `178325bcbe` (F4 no-value) | **4** — one edit instead of N | +| partly structure | `fb386074f5`, `f1e921ab8e`, `2b53993aca` (`$empty`: the `null` and `''` rows are structure; the empty-list test is a dialect leaf) | **3** — fewer face edits | +| refusal at a door (the pattern that has already converged) | `8a44ce7182` (`$like` NUL), `5c7cbe37e6` (vacating read scopes), `51efbf1168` (text operator over a temporal type) | 3 — unaffected | +| dialect construct | `54bb2f125f`, `fd014b1713`, `dcad825d46` (case-exact and ASCII-fold constructs per dialect) | 3 — stays per dialect by design | +| front end, engine capability or analytics vocabulary | `6936d0755b` (per-aggregation filter), `0da638cd9f`, `86c5052869` (`dateRange`), `7c1039b388` (read-scope placeholders) | 4 — outside the lowering | +| clobber inside a face's own lowering | `df18120502` | 1 — T2 only | + +So, over those 46 days, T1 would have turned 4 of the 18 multi-face commits into single edits and shortened 3 more, and T2 would have added one. Seven of the 18 are dialect or front-end work, and convergence does not touch them. + +The larger saving is not in that count. It is in the divergences that per-face copies keep producing between rulings. §2.5 found three, and #20661 was a fourth, found by the #20600 work. + +### 4.4 Order of work + +1. **The ADR-0053 D-D1 amendment first, in its own PR.** `docs/adr/**` is Tier H, and a mixed diff is governed whole, so keeping it separate keeps the code slice off Tier H. The code slice waits on the amendment (Prime Directive #13). +2. **S1 + S2 on the engine and RLS seams.** This is one card in the engine lane. F1, F2, F3, F6, F7 and F8 then receive the lowered filter with no edits of their own. +3. **S2 on the analytics seams, S3, and F5's door.** This covers the analytics lane and `driver-memory`. §2.5 items 1 and 2 close here. +4. **S5 deletions,** one lane card each, each proven by the face's existing suites and an ablation. +5. **T2 — only on measured need.** Measure the next filter-semantics ruling against §2.1's baseline first. + +### 4.5 Risks + +- **R1 — double application.** Idempotence is measured for the NULL guard (#5977). The whole-day rewrite emits `$lt`, which no face rewrites again, and `$between` is gone before any face sees it. Both need an idempotence pin (S1). +- **R2 — provenance marks (#8220).** Rewritten nodes are new objects. A refusal raised inside one resolves as ambiguous, and so stays withheld from the author. That is fail-closed, and F1 already behaves this way under its own `$not` rewrite. The lowering must copy the marks forward to keep today's author-visible refusals. +- **R3 — declarations at the seam.** Scoping the whole-day rule to `datetime` and reading `$empty`'s row both need field types. + - The engine seam has them. + - F9 has `declaredValueShape`. + - F7 and F11 hold none, so they get the type-blind reading, which D-D1 already sanctions: `< next-day` ≡ `<= day` for `date` text. +- **R4 — direct driver callers bypass the seams.** Decision D4. +- **R5 — accept sets move.** F9's whole-day fix changes the rows a scope returns. F5's and F11's widening admits shapes they refuse today. Each needs a changeset and its Clause-② line. +- **R6 — echo fidelity.** F10b must reproduce F10a. Both read the lowered tree, so they agree by construction. +- **R7 — hot files.** `sql-driver.ts` (21,103 lines) and `remote-transport.ts` (4,771 lines) collide with parallel lane work. The slice leaves both untouched until S5. + +--- + +## 5. Recommendation — build, a smaller first slice, or not now + +The three options: + +- **A — build shape A in full.** T2, compiled by all 14 compile entries of §1.1. +- **B — a smaller first slice.** T1 at the existing seams, as costed in §4.1, followed by the S5 deletions. +- **C — not now.** Keep the accepted process half and the doors, and pay the tax per ruling. The process half is the per-ruling compiler-face checklist of 2026-08-07, now `.claude/skills/pm-dispatch/references/compile-surfaces.md`. + +The analysis follows the four fixed axes of the dispatch frame. + +**实际业务需求 — real, measured need.** + +The authors whose filters pass through these faces are real: + +- every stored view filter rule, lowered by REST and read through F1, F3 or F6; +- every RLS policy (F1 on CRUD reads, F7 on writes, F9 in analytics); +- every widget and dataset `where` (F10, and F11 in draft preview); +- every `having` (F8). + +The measured cost is 18 filter-semantics commits that each edited 3 or more face files in 46 days, 13 rounds for the `$empty` ruling alone, and three live divergences found by one census (§2.5). Those divergences reach authors as *different rows for one predicate*. + +- B targets exactly the absorbable classes (7 of the 18) and closes all three divergences. +- A adds, over B, one commit class (the clobber) and typed constants. No measured demand exists for more. +- C leaves the tax and the divergences in place. The checklist existed throughout the time F5's ordering defect (#20661) and §2.5's three divergences sat undetected. It has also drifted from the code it lists, measured against §1.1: + - it names F4, which has no production caller, as the `driver-memory` face; + - it omits F3, F5, F11, F10c and F2's lowering half; + - it still calls `having` 「唯一没有 conformance 表覆盖的面」, though `having-filter.test.ts` now runs `FILTER_LOGIC_CASES`; + - its line numbers point to the August tree (`applyFilterCondition` at `:7083`; it is at `:16789` today). + +**项目长远合理性 — long-term soundness.** + +- A is the end state. +- B is on the path to it, because T2 is T1's output parsed into kinds (§3.5). +- C keeps N dialects of *meaning*. That is the "one concept, one implementation" line `filter-verdict.ts` already crossed for the boolean identities. + +B keeps ADR-0053 D-A1 intact and makes D-E3 structural. It costs one honest ADR amendment (D-D1). Its interim state — faces holding idempotent copies until S5 — is declared, and each copy has a deletion card, so it is not a workaround. + +**防 AI 写错 — making the AI mistake structurally harder.** + +An author, human or AI, cannot see which face a filter will land on. One RLS policy is read by F1, F7 and F9, and §2.5 item 1 shows those faces answering it differently. Per-face meaning cannot be learned. + +- Under B, a new face inherits the absorbed rules at its seam. It is correct by construction, not by someone remembering a checklist. F5 shows what happens to a face nobody remembered: it has no door at all. +- B adds no consumer tolerance. It lowers once and refuses at the seams. Its widening of F5 and F11 implements operators the spec already declares (「声明而未兑现是实现缺口,补实现」); it is not leniency toward off-spec input. +- C relies on a checklist, which is guidance an agent reads, not enforcement. + +**创业阶段不扩散需求 — no scope creep at the startup stage.** + +- **A** is a multi-week rewrite across three lanes. It is the expansion this axis defaults against. +- **B** is bounded: about 6 cards, as estimated in §4.1. It deletes more code than it adds — 293 polarity lines, 41 whole-day call sites and 4 `undefined` copies. It creates no new authorable surface and **recommends no new gate**, per 「新增门禁默认否」. + + Its staging (S2 before S5) is an internal work order. It exposes no dual spelling to authors. To honour 「过渡也从紧」, the S5 cards should queue as soon as S2 lands, not be left open-ended. +- **C** costs nothing now, and then costs every future ruling. + +**The axes conflict only on A.** Long-term soundness favours it; the startup axis refuses it now. + +**Recommendation: B**, starting with the ADR-0053 D-D1 amendment. **A: not now.** Revisit it only after the next filter-semantics ruling is measured against §2.1's baseline under B. **C: not recommended**, because the tax and the divergences are measured, not predicted. + +This is a proposal. The build decision is the maintainer's. + +--- + +## 6. Decisions for the maintainer + +| # | Question | Options | Recommendation | +|---|---|---|---| +| D1 | Build? | A full build · B first slice · C not now | **B** (§5) | +| D2 | The slice's representation | T1 (`FilterCondition` → `FilterCondition`, `$and` conjuncts) · T2 (typed tree) | **T1** now; T2 only on measured need | +| D3 | Where the lowering lives | `@objectstack/spec/data`, beside the doors · `@objectstack/core` | **spec**: the precedent is the doors themselves, and it adds no dependency edge. `formula` depends on spec alone, so core would add a `formula → core` edge. The tension with Prime Directive #2 is named, not hidden. | +| D4 | Direct driver callers, which bypass the seams | (a) faces keep their local guards permanently, so the copies survive and S5 shrinks to analytics, F5 and F11 · (b) drivers receive seam-lowered input only, as #7872 chose for comparand types ("inherit it by receiving already-validated input") | **(b)**, following #7872. The in-repo direct driver callers named on #20446 (`5886202626`) are `metadata`'s loaders and history cleanup and the CLI's `secret-reference-union`. Whether any of them builds a filter the lowering would change was NOT MEASURED. | +| D5 | ADR-0053 D-D1 | amend its placement · write a new ADR that supersedes it | amend the placement; keep D-A1 and D-E3 as they are | +| D6 | F4, the reference matcher with no production caller | keep it as a test oracle, declared not-a-face and owed no ruling arms · retire it and point its 20 test files at F7 or F3 | **retire** it | +| D7 | Scope of F3–F6, given the #5499 freeze dissolved on 2026-08-11 | in · out | **in**. Under B, F3 and F6 are seam-fed at no cost, and F5 needs its door either way. | + +--- + +## Appendix A — method, reproduction, and what was NOT MEASURED + +**Probes.** They are scratch scripts, not committed. + +- Worktree sources ran under `tsx`. Package imports resolved to a `dist` built with `OS_SKIP_DTS=1` for `spec`, `types`, `core`, `observability`, `formula`, `driver-sql` and `driver-turso`, under the shared verify lock. +- F1 and F2 ran as plain Node against their built `dist`. F1 used `better-sqlite3` `:memory:`. F2 called `buildWhereSQL` with a stub client. +- §2.4: the fixture `{id:'1',d:'v1'}`, `{id:'2',d:'v2'}`, `{id:'3',d:null}`, `{id:'4'}`, with the five shapes shown. Each face was called at its own entry point, with no door in front. F5's controls are no `where`, `{d:'v1'}` and `{d:null}`. +- §2.5 item 1: rows `t` = `2026-07-27T10:00:00.000Z`, `2026-07-28T10:00:00.000Z`, `2026-07-29T10:00:00.000Z` and `null`. The filter is `{t: {$lte: '2026-07-28'}}`. F1's column was declared `datetime`, and F9's compiled SQL was executed on SQLite. +- §2.5 item 3: rows `ACME Corp`, `acme ltd` and `Beta`, with `$contains` `'acme'` and `'ACME'`. + +**Counts.** + +- **Face files per commit:** `git show --name-only` against the 15-file list of §1.1. +- **The window:** `node scripts/pm/git-history.mjs log --since=2026-08-14 --ref=3711e0b763`. Its receipt reads "complete clone (no fetch)". +- **Polarity copies:** non-comment lines from `nullValueSatisfiesOperator` to the next top-level declaration after `nullSafeNegationOperand`. +- **Whole-day call sites:** `git grep -c -E "nextUtcCalendarDay\(|isUnboundedAbove\("` over non-test `src`. +- **Case-set wiring:** `git grep -l` over non-spec test files. +- **Rounds:** the number of `os-dev-report` comments on each card, read over REST. +- **The driver conformance matrix:** `node scripts/check-driver-conformance.mjs`. + +**NOT MEASURED:** + +- PostgreSQL and MySQL for §2.4 and §2.5. Only SQLite was measured. +- A live `mongod`: §2.4 reports F6's translation only. +- A live Turso remote server: F2 was measured at compile time only. +- Every HTTP door: the REST `?preview=` route, the analytics routes and the data routes. +- The sibling repositories `objectui` and `cloud`. +- The `undefined` path through direct driver callers, the security tenant check and the explain engine. +- The effort figures in §4.1, which are estimates. + +## Appendix B — the 22 commits that edited three or more face files (2026-08-14 → `3711e0b763`) + +| Face files | Commit | Subject (abridged) | §4.3 class | +|---|---|---|---| +| 10 | `1a75e39d4a` | a datetime `$lte` / `$between` maximum on 9999-12-31 includes the whole last day (#20600) | structure | +| 9 | `fb386074f5` | the engine's filter faces answer the staged `$empty` (#20444) | partly structure | +| 8 | `f1e921ab8e` | `$empty` joins `FILTER_OPERATORS`; `is_empty` lowers to it (#20446) | partly structure | +| 8 | `a646120dc7` | `FILTER_TEXT_CASES` declares a text operator over a stored non-string value | structure | +| 5 | `6936d0755b` | per-aggregation filter on `engine.aggregate` | engine capability | +| 4 | `8a44ce7182` | a `$like` / `$ilike` pattern holding U+0000 is refused | door | +| 4 | `54bb2f125f` | the case-sensitive text family compiled per SQL dialect | dialect | +| 4 | `2b53993aca` | both analytics filter faces answer `$empty` by declared type (#20445) | partly structure | +| 4 | `0da638cd9f` | the closed `dateRange` preset vocabulary lowered once | front end | +| 3 | `fd014b1713` | the `unknown` dialect arm folds `$icontains` portably | dialect | +| 3 | `f6fa22ce1a` | a boolean aggregand column in the aggregation fixture | not filter semantics | +| 3 | `df18120502` | a lowered operator key stops clobbering a sibling | clobber (T2 only) | +| 3 | `dcad825d46` | the `$icontains` ASCII fold per dialect | dialect | +| 3 | `be5c60291c` | eight more `IDataDriver` doors publish their declared type | not filter semantics | +| 3 | `9dac1ae017` | `$exists` means HAS A VALUE on three faces | structure | +| 3 | `96d8b20ee4` | docs: retire the dissolved #5499 freeze claims | not filter semantics | +| 3 | `86c5052869` | a `dateRange` array that is not a two-bound window is refused once | front end | +| 3 | `7c1039b388` | NativeSQL and the echo resolve a read-scope placeholder with the caller's context | front end | +| 3 | `5c7cbe37e6` | refuse vacating read scopes on the echo and NativeSQL merge sites | door | +| 3 | `5ba2ec3ca4` | a transport can declare it has no transactions | not filter semantics | +| 3 | `51efbf1168` | type-gate a text operator over a declared temporal column | door | +| 3 | `178325bcbe` | a no-value row satisfies `$nin` / `$notContains` in the reference matcher | structure |