Repository navigation
fix(driver-turso)!: the remote filter compiler refuses the JSON-column family and answers $contains by membership (#21178) - #21208
Conversation
… gate and answers $contains by membership
RemoteTransport.buildWhereSQL now refuses every operator in @objectstack/core's
JSON_COLUMN_INCOMPATIBLE_OPERATORS on a column the driver stores as JSON text,
with the shared jsonColumnOperatorRefusalText sentence, and answers
$contains / $notContains there through jsonMembershipPredicate('sqlite'), the
negated form NULL-safe. The JSON-column population is the driver's own
jsonFields registry, handed down through a new optional
RemoteTransport.setJsonColumnResolver that TursoDriver wires to the inherited
SqlDriver.isJsonColumn, beside setDeclaredValueShapeResolver.
A local/remote parity suite holds both faces to one answer over the whole
shared set, the membership pair, the bare and null equality spellings and a
scalar text-field control.
Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp
Co-authored-by: Claude <noreply@anthropic.com>
…-refusal seam table; add the changeset The seam enumeration requires every refusal method that goes through the withheld seam to have a row: jsonColumnOperator gets one per position (operator map, bare value, bare null), on a half-2 transport told that exactly one column is JSON-stored. The class is read from jsonColumnOperatorRefusalText, never spelled in the test. The bare-null position of buildWhereSQL now asks the gate too, as the local face's bare-value positions do. Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check8 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 7 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 7213aa27b0367f41d157a772dad1e41ddc8a7626 && git checkout 7213aa27b0367f41d157a772dad1e41ddc8a7626
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 0d421041d14cfc716e85e9bdd1da3d57d4eb3aa1 c67a136e94b876497ea177a6b49cd12fd3b3eefe && git checkout -B drift-repro 0d421041d14cfc716e85e9bdd1da3d57d4eb3aa1 && git merge --no-ff c67a136e94b876497ea177a6b49cd12fd3b3eefe
node scripts/docs-audit/affected-docs.mjs --json 0d421041d14cfc716e85e9bdd1da3d57d4eb3aa1 |
Contract reviewServed-tier: Inputs read: card #21178 (body and all five comments: triage 5933993910, claim 5934703166, os-dev-report 5934971162, claim amendment 5935032674, os-dev-report 5935977677); PR #21208 body, file list (5 files, +553/-0) and the net diff; the check-runs on the head. Verified against ① Derived judgmentsAccept-set changes, all on the REMOTE face of
Public-surface changes:
Pins:
Check-runs on ② Semver level
Clause-②: yes (narrowing) Read as ③ Boundary flags
Implemented-by: VERDICT: PASS Generated by Claude Code |
…es a REST caller whole (objectstack-ai#21213) Fixes objectstack-ai#21067 Clause-②: no ## What this changes `jsonColumnOperatorRefusalText` (`packages/core/src/utils/json-column-operator-refusal.ts`) is the one builder of the `INVALID_FILTER` / 400 refusal a filter gets for a scalar comparison or text operator on a multi-value or JSON field. It is rewritten once, for both of the card's reasons together (seat 2's carried note folded in: one rewrite, not two passes): 1. **Under the REST bound.** The withheld message was 748 characters. The `/data` door cuts a 4xx message of 500 or more to 499 plus an ellipsis, so the wire ended `…Refused rather than compiled because the answ…`, and no caller read the sentence saying the field and the operator were withheld. It is now **486** characters, one constant text. 2. **True on every face.** The old reason named `driver-sql`'s storage ("a field this driver stores as a JSON TEXT column") and the two wrong answers SQL used to give. Both are untrue on the engine's per-aggregation `filter` and on `driver-memory`, which print the same text. The reason is now the field's declaration: `it aims a scalar comparison or text operator at a multi-value or JSON field, which it cannot test for one member.` 3. **The null comparand's repair (added at the PM's note on this card).** The refused set also catches `{ f: null }`, `$eq: null` and `$ne: null`, which ask whether the field has a value; `$contains` cannot express that. One constant clause, with no branch on the comparand, names the presence spellings: `For no value, use "$null" or "$empty".` Both answer on a multi-value or JSON field on every face (they are outside the refused set), and the REST pin now proves `$null: true`, `$null: false` and `$empty: true` answer on both the `where` and the per-aggregation faces. The new message, in full: ```text A constraint in this filter WAS NOT APPLIED: it aims a scalar comparison or text operator at a multi-value or JSON field, which it cannot test for one member. Use "$contains" for membership ({ "FIELD": { "$contains": "a" } }), or an $or of "$contains" for any-of ({ "$or": [{ "FIELD": { "$contains": "a" } }, { "FIELD": { "$contains": "b" } }] }). For no value, use "$null" or "$empty". The field and the operator are withheld from the message; the full diagnostic is in the server log. ``` The diagnostic (server log, and the author-disclosed wire text, see below) shares that reason, names the operator in it (`it aims "$in", a scalar comparison or text operator, at …`; the bare spelling's operator as `=`, as before), names the field, and spells the remedy with the field's name, presence clause included. It drops the same storage and SQL history. One `refusalReason()` and one `containsRemedy()` (which ends in `PRESENCE_REMEDY`) serve both texts, so the two cannot drift. The SQL mechanism and the measured wrong answers stay in the builder's docblock. **Unchanged:** `code`, `status`, the refused operator set (`JSON_COLUMN_INCOMPATIBLE_OPERATORS`, 27 spellings), the `$contains` remedy, no new export, no new code. **Faces.** Re-derived at `cb45469e` by name and by text: every face that prints this sentence calls the builder, and none builds its own copy. They are `driver-sql` `jsonColumnOperatorError` (`sql-driver.ts`), `objectql` `having-filter.ts` (the judgment and the per-row backstop, three call sites), and `driver-memory` `jsonStoredFieldOperatorError` (`filter-refusal.ts`). No face file changes except one stale docblock in `sql-driver.ts`. `driver-turso`'s `remote-transport.ts` and `driver-mongodb` share phrases with other refusals, not this sentence. No docs page or skill quotes it (`content/docs/**`, `skills/**`, `docs/**`, `apps/docs/**`, by builder name and by seven distinctive phrases: zero hits). ## Measured, before and after | reading | `BASE` `cb45469e` | this PR | |:--|:--|:--| | withheld message, constant | 748 chars | 486 chars | | its wire `error`, SQLite and live PostgreSQL 16.14, `mapDataError` | 499 plus an ellipsis, ending `Refused rather than compiled because the answ…` | the whole message | | author-disclosed diagnostic, field `owners`, `$in` | 643 chars | 402 chars | | that, at `POST /api/v1/data/:object/query` through the real `SecurityPlugin` | 499 plus an ellipsis, ending `Refused rather than compiled beca…` | the whole diagnostic (measured at 377 before the presence clause; the 402 text is pinned whole through `mapDataError`) | | field-name length from which the diagnostic is cut on the wire | every length | 27 characters (the presence clause goes first) | | field-name length from which the cut takes the any-of example | 10 characters | 37 characters | ### A dispatch hypothesis this falsified (H5) The dispatch read the diagnostic as server-log text the envelope never bounds. Measured, it is also a wire text. `driver-sql`'s `'author'` provenance arm (`resolveWithheldFilterRefusal`) swaps the refusal for the diagnostic when the predicate is the caller's own, and `plugin-security` marks the caller's verbatim `where` `'author'` (`security-plugin.ts`, the `markFilterSubtreeProvenance(callerWhere, 'author')` call). Through `POST /api/v1/data/:object/query` with a real `SecurityPlugin`, a member caller sending `{ owners: { $in: ['u1'] } }` received the diagnostic: cut to 500 at `BASE`, whole (377) at `caf0e3c7`, before the presence clause raised it to 402. That was a one-off measurement file, run twice and not committed; the `BASE` leg rebuilt `core`'s `dist/` with the old text and restored it, with dist preflight proofs both ways. The diagnostic was in the rewrite already, because its reason clause was the same untrue `driver-sql` mechanism seat 2's note names. All four conditions of the bounded in-place fix hold: same defect class (a refusal cut at the envelope), a mechanical fix in the shape triage pinned, the claimed file, and the same gate families. So it is fixed here, not filed. Its content is kept: field and operator named, remedy with the field's name. ## Pins New, compared with the builder's output and the bound's own function (`truncateClientMessage` / `mapDataError` from `@objectstack/types`), never with a copied sentence or a retyped 500. `CLIENT_MESSAGE_MAX` itself is module-private in `packages/types/src/data-error-classification.ts`, so it is not imported; `truncateClientMessage` is already exported there for `rest`, and nothing new is exported. A sibling that rewords nothing and only calls the builder cannot flip these. - `packages/core/src/utils/json-column-operator-refusal.test.ts`: re-captured hashes and lengths. For every refused spelling, `truncateClientMessage(message)` returns it unchanged. The message carries the one-member, any-of and no-value remedy and ends with the withheld sentence. Neither text names a storage form or a backend's wrong answer. The diagnostic gives the same reason with the operator named. - `packages/drivers/driver-sql/src/sql-driver-json-column-refusal-wire-bound.test.ts` (new, `DIALECT_CELLS`): for 14 operators plus bare equality, on a multi-value lookup, a `tags` field and a `json` field, `mapDataError(err).body.error` equals the shared message (unmarked, carrying the presence clause) and the shared diagnostic (author-marked), with the remedy spelling. SQLite always; **PostgreSQL and MySQL in CI's `Temporal Conformance (live PG + MySQL)` job**, which runs this package's whole suite with both URLs set. - `packages/rest/src/aggregation-filter-json-column-refusal.test.ts`: through `POST /api/v1/data/:object/query`, the `where` twin's body equals the builder's message whole, not just the per-aggregation face's (`toBe`, plus the any-of remedy, the presence clause and the withheld sentence), on the null-comparand rows too; three new controls show `$null: true`, `$null: false` and `$empty: true` answer with equal counts on both faces. Flipped, each to the new substance: - `sql-driver-json-column-operator-refusal.test.ts`: `'JSON TEXT column'` becomes the operator-named reason, and the remedy is asserted with the field's own name. - `sql-driver-target-field-provenance.test.ts`: the class fragment that survives redaction, `'JSON TEXT column'`, becomes `'at a multi-value or JSON field'`. - `sql-driver-json-column-refusal-shared-text.test.ts`: the docblock's "did not change by one byte" claim. ADR-0112 `code` plus `status` assertions are untouched everywhere. ### Reverse verification At `fefb6e1f`, `BASE`'s builder was written to disk (tree only, never staged) and its landing checked by grep: old text 1, new 0. Then: - `sql-driver-json-column-refusal-wire-bound.test.ts` (SQLite plus live PostgreSQL 16.14): **90 failed** (45 per cell), 1 skipped (MySQL). The red direction, as expected. - The core pin: **8 failed**, 4 passed. Restored with `git checkout HEAD -- PATH` under an `EXIT INT TERM` trap. The blob is `2f17d7f3`, equal to HEAD's, and `git diff HEAD` is empty. ## Verification (on `d352319a`, the final head: the branch, one merge of `origin/main` `0d421041`, which touches none of these packages, and the presence-clause commit) The targeted files below ran on `d352319a`, with `OS_TEST_POSTGRES_URL` set to a private PostgreSQL 16.14: - core: 12 passed - driver-sql, the five JSON-column files: 360 passed, 2 skipped (the MySQL cells) - rest `aggregation-filter-json-column-refusal`: 136 passed, 68 skipped (MySQL) - driver-memory `memory-20444-*` and `memory-21066-*`: 125 passed - objectql `engine-aggregate-filter-json-column-refusal`, `engine-aggregate-filter`, `engine-aggregate-filter-array-membership` and `engine-cascade-delete-multivalue-probe`: 263 passed Full suites, on the pre-merge commits (before the presence clause; that commit touches only the builder, its unit pin and the two wire pins above, which were re-run): - `@objectstack/core` local: 74 files, 2107 passed - `@objectstack/driver-sql`, SQLite: 208 files passed, 11 skipped; 3469 passed, 192 skipped - `@objectstack/rest` local: 255 files, 4834 passed, 301 skipped Typecheck: `core`, `driver-sql` and `rest` all pass, and `tsc --listFiles` confirms the four touched driver-sql test files are in its program. **Gates** (`node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands`, re-derived and re-run on `d352319a`): 65 derived, 65 exit 0, 0 NOT MEASURED. `--ran` with exit codes reconciles to 65 accounted, 0 unrun. - `check:dual-build-cjs-loads` measured this time: 105 require entries across 66 packages load. The first run on `8450f66b` was exit 3 `PREREQUISITE NOT MET`, since a whole-repo build was missing then. - `check:driver-conformance`, before the first edit and after the last commit: 50 covered, 0 DEBT, 0 exempt; dialect axis 8 suites, 0 in the DIALECT ledger. The ledger did not move. **Lint**, narrowed: 1. The population comes from eslint's own config: `--format json` reports every listed file with 0 warnings, so none was ignored. 2. On `d352319a`, the 8 `.ts` files of this diff were linted, with 0 errors and 0 warnings. 3. `eslint.config.mjs` enables no type-aware linting (no `parserOptions.project`), so this diff cannot move the verdict on any untouched file. The repo-wide `pnpm lint` is CI's to run. **Patch round 1 (`89209290`).** `Lint & Repo Gates` was red at `d352319a` on `pnpm check:live-db-isolation`: `packages/rest/src/aggregation-filter-json-column-refusal.test.ts:258 use names the literal "$null"`. The fix is test-only: the presence clause is asserted in two pieces. On `89209290`: - `pnpm check:live-db-isolation`: exit 0, 34 live-server files scanned. - That test file: 136 passed, 68 skipped, with SQLite and live PostgreSQL 16.14 running and MySQL a named skip. - The 65 derived families re-run: 65 exit 0. `check:dual-build-cjs-loads` gave exit 3 PREREQUISITE NOT MET until another family built the missing `dist/`, then exit 0 on re-run. `--ran` reconciles to 65 run, 0 NOT MEASURED. - `check:live-db-isolation` is a declared WIDE-population family, so per-card derivation never names it. ## Siblings that print this sentence (both landed before this PR; seat edit) Both siblings this PR named as later landers merged first, so they are earlier landers now: - objectstack-ai#21178 → `862f12c0b` (PR objectstack-ai#21208): the `driver-turso` remote face. It prints the builder's output and pins it by equality with the builder, so this rewrite does not flip it. - objectstack-ai#20822 group 3b → `e18fea6dc` (PR objectstack-ai#21196): new `having-filter.ts` callers of the builder. This branch is behind both. The merge queue rebuilds it onto `main`, so their pins run against this text in the merge group. ## Acceptance notes - **The diagnostic's boundary.** It names the field four times, so its length grows with the name. It is whole on the wire up to 26-character field names (measured over `$in`, `$startsWith` and bare equality). From 27 characters the cut takes the presence clause first, and from 37 the any-of example; the one-member remedy and both names come before both. Field names declare no maximum length, so no text that repeats the name can be bounded. The withheld message, the card's subject, is constant and bounded. - **A gate false positive, on the test's spelling.** `check:live-db-isolation`'s statement-head needle matches the verb USE followed by a quoted operand (`STATEMENT` in `scripts/check-live-db-isolation.mjs`). It read the assertion string `'For no value, use "$null" or "$empty".'` at `packages/rest/src/aggregation-filter-json-column-refusal.test.ts:258` as a MySQL USE naming a database `"$null"`. Fixed in `89209290` by asserting the clause in two pieces; the gate and the product sentence are unchanged. - `packages/objectql/src/engine-cascade-delete-multivalue-probe.test.ts` keeps a test double whose refusal paraphrases the old wording ("is stored as a JSON TEXT column"). It asserts only `code` and `status`, so it is not a pin of this text. Left as is. - `rest`'s own PostgreSQL and MySQL cells of `aggregation-filter-json-column-refusal.test.ts` are still provisioned by no CI job (its header says so). The PostgreSQL wire pin that CI does run is the new `driver-sql` file. - The local PostgreSQL leg ran against a private PostgreSQL 16.14 started for this run on a random port (UTC server, so `driver-sql`'s temporal files were not run against it). It was stopped afterwards. --- _Generated by [Claude Code](https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…its counter from the stored MAX on SQLite (objectstack-ai#21163) (objectstack-ai#21206) Fixes objectstack-ai#21163 Clause-②: no ## What changed `SqlDriver.scanMaxNumericTail` (`packages/drivers/driver-sql/src/sql-driver.ts`) is the one read of the highest counter already stored under an autonumber prefix. Its two callers are the cold bootstrap in `getNextSequenceValue` and the objectstack-ai#5495 re-seed `resyncSequenceToDataMax`. It escapes the rendered prefix with `escapeLikePrefix` (a backslash before `\`, `%` and `_`). It then compiled `where(field, 'like', pattern)` through Knex, which declares no `ESCAPE` clause on any dialect. The scan now reads: ```ts .whereRaw('?? like ? escape ?', [field, `${this.escapeLikePrefix(prefix)}%`, LIKE_ESCAPE_CHARACTER]) ``` `LIKE_ESCAPE_CHARACTER` is the driver's existing bound escape character, the one the filter compiler's `LIKE ... ESCAPE ?` already binds. It is bound and never written as a literal, because MySQL applies C escape syntax inside string literals. There is one escape helper (`escapeLikePrefix`, unchanged) and one declared escape character, the same on every dialect. The method's docblock records why. One file of production code changed. The Turso remote face's own statement (`turso-driver.ts`, which already declares `ESCAPE` with a backslash) is untouched. ## Why SQLite's `LIKE` has no escape character unless one is declared. Postgres and MySQL use a backslash by default. So on the SQLite faces, a prefix carrying `_`, `%` or `\` matched no stored row. Measured at `cb45469e` with this PR's pins and the pre-fix builder restored: | format | stored before the create | cold create issued | create after rows 2..30 land above a warm counter | |:---|:---|:---|:---| | `SO_{0000}` | `SO_0007` | `SO_0001` | refused: `UNIQUE constraint failed` (on `SO_0005`, retries spent) | | `SO%{0000}` | `SO%0007` | `SO%0001` | refused, same | | `SO\{0000}` | `SO\0007` | `SO\0001` | refused, same | | `{region}-{0000}`, region `north_east` | `north_east-0007` | `north_east-0001` | not pinned | | `SO-{0000}` (control) | `SO-0007` | `SO-0008` | `SO-0031` | The `{region}` row shows that the trigger does not need an unusual format. The prefix is rendered, so a `_` can come from a field value such as a snake_case machine name. ## Pins - `packages/drivers/driver-sql/src/sql-driver-21163-autonumber-prefix-like-escape.test.ts`: one suite per dialect cell through `declareDialectCell`. SQLite always runs. Live Postgres and MySQL run where provisioned (`Temporal Conformance (live PG + MySQL)` runs this whole package with `OS_EXPECT_LIVE_DIALECT_MATRIX=1`) and are declared un-run otherwise. Each cell covers `_`, `%` and `\` prefixes plus a plain control, each cold and on the objectstack-ai#5495 re-seed, plus the `{field}`-borne `_`. - `packages/drivers/driver-turso/src/turso-autonumber-prefix-like-escape.test.ts`: the LOCAL face (`url: ':memory:'`) and the REMOTE face (over `makeLibsqlSqliteStub`, a real SQLite behind the libsql client), each with `_`, `%` and a plain control, cold and on re-seed. The REMOTE legs passed before this fix because that statement already declared its escape. They hold both faces to one answer from the one helper. The assertion is the issued autonumber as stored, read back through the driver's `findOne` by an id the test chose. On MySQL `create` returns the insert id rather than the row, so its return value is not a reading every cell can make. Never a SQL string. ## Reverse verification (fix committed first, at `9b3f7d0a`) Mutation: `node scripts/ablation-replace.mjs` replaced the new `whereRaw(...)` with the pre-fix `where(field, 'like', ...)` (anchor 1 to 0, blob `cb63656d` to `5f8edc7a`). Restore: blob equal to HEAD, `git diff HEAD` empty, whole-tree `git status --porcelain` clean. - driver-sql pin over src: **7 failed / 2 passed**. All three escaped prefixes failed cold and on re-seed, and so did the `{field}` case (`expected 'SO_0001' to be 'SO_0008'`, `UNIQUE constraint failed`). The two controls passed. - driver-sql rebuilt, and `node scripts/ablation-dist-preflight.mjs @objectstack/driver-sql '?? like ? escape ?' --absent` confirmed the fix was absent from all 6 built files. Then the turso pin: **4 failed / 8 passed**. LOCAL `_` and `%` failed cold and on re-seed. LOCAL control and all 6 REMOTE legs passed. - Restore leg: rebuilt, and preflight `'where(field, "like"' --absent` confirmed the mutation was gone from dist (tree clean). Pins: driver-sql 9 passed / 2 skipped (live cells unprovisioned), turso 12 passed. The pristine dist reading (default mode) found `?? like ? escape ?` present in `dist/index.js` and `dist/index.mjs`. ### Live Postgres (local PG 16, `initdb` + `pg_ctl` on a scratch port, torn down after) - With the fix: 18 passed / 1 skipped (9 sqlite + 9 live postgres; mysql un-run). - With the pre-fix builder restored: 7 failed, all in the sqlite cell. 0 failed in live postgres. So declaring the escape does not change behavior on Postgres, which already used a backslash by default. **MySQL: NOT MEASURED locally** because this container has no MySQL server. The same file's live mysql cell runs in `Temporal Conformance (live PG + MySQL)`. ## Verification (at HEAD `7b7fcafb`) - **Patch round 1 (`5b83098b`, seat edit):** the live MySQL cell failed on `create` return values (insert id `0` on MySQL). The pin now reads the stored row. The whole `driver-sql` suite, CI-shaped against a local live PostgreSQL 16 and MySQL 8.0.46: 219 files, 5108 passed / 1 skipped. Reverse verification on the reworked pin: 7 failed, all in the SQLite cell, 0 in live PostgreSQL and live MySQL. - `pnpm --filter @objectstack/driver-sql test`: exit 0, 208 files passed / 11 skipped, 3433 tests passed / 192 skipped. - `pnpm --filter @objectstack/driver-turso test`: exit 0, 84 files passed, 2243 tests passed / 33 skipped. - `typecheck` on both packages: exit 0. `tsc --listFilesOnly` includes both new test files in each package's program. - `pnpm check:driver-conformance`: identical before the first edit (at `cb45469e`) and after the last commit: 50 covered cells, 0 in the DEBT ledger, 0 exempt. Dialect axis: 8 suites, 0 in the DIALECT ledger. - `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` (no paths) derived 63 commands. 61 exited 0. `check:type-check-debt` hit a 280 s per-command cap on the first run and exited 0 on a re-run (re-measure OK in 211 s, self-test OK). Two exited 3 (PREREQUISITE NOT MET, NOT MEASURED): `check:dual-build-cjs-loads` needs a whole-repo build, and `check:lean-entry-closure` needs objectql's dist, which is outside this diff's closure. For the first one, both changed packages' CJS builds were loaded directly with `require` and loaded OK. `--ran` reconciliation: 63 derived, 61 run, 2 NOT-MEASURED, 0 UNRUN. - Lint was narrowed to the 3 changed TS files: `eslint --no-inline-config --format json` gave 3 files, 0 errors, 0 warnings. Population: `eslint.config.mjs` lints `**/*.{ts,...}` outside its never-linted build dirs, and no file reported as ignored. Invariance: the config enables no type-aware linting (no `parserOptions.project`), so this diff cannot move the verdict on any untouched file. Repo-wide `pnpm lint` is CI's. ## Acceptance notes - `escapeLikeComparand` (the filter compiler's module function) and `escapeLikePrefix` (this method) escape the identical set: a backslash before each of `\`, `%` and `_`. They were not merged. Triage named `escapeLikePrefix` as this path's one helper, and nothing here needed a merge. - The Turso remote face writes its escape as an SQL literal. That is correct, because it only ever talks to SQLite. Its new REMOTE legs are the guard that keeps the two faces' escape characters equal. - `SqliteWasmDriver` extends `SqlDriver` and inherits the scan. A one-off probe after the fix (not committed) issued `SO_0008` over a stored `SO_0007` on that face. That face has no permanent pin here, and its pre-fix answer was not measured. - Counters that an earlier release already seeded too low are not rewritten by this change. The next collision on such a counter now re-seeds it from the stored MAX, as on any other prefix. - Patch round 2 (`f71f31b2`, seat edit): the branch merged `origin/main` at `097ef802`, a clean merge with the same 4 files in the net diff. It carries the sibling objectstack-ai#21208 (`862f12c0b`, `driver-turso`), and it re-ran the `Type Check · source gates` lane that had timed out at `5b83098b`. Every check is green on this head. --- _Generated by [Claude Code](https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #21178
Clause-②: yes (narrowing)
What changes
RemoteTransport.buildWhereSQL(@objectstack/driver-turso, the filter compiler every remote-modeTursoDriverread and filtered write uses) now applies the JSON-column half of the filter contract exactly as the local face (SqlDriver, which local and replica mode inherit) does, from the same shared home in@objectstack/core:JSON_COLUMN_INCOMPATIBLE_OPERATORSis refused withINVALID_FILTER/ 400 before any statement runs: in an operator map (at the top of the per-operator loop, ahead of every arm), in the bare{ field: value }spelling, and in the bare{ field: null }spelling. The message and the withheld diagnostic arejsonColumnOperatorRefusalText's output, byte for byte the local face's, through this transport's existing withheld-refusal seam ([A of #7929] a spec-declared provenance mark set at both read-scope merge boundaries, so the driver can restore the author-facing cross-field diagnostic without re-disclosing policy #8220 provenance, diagnostic sink).$contains/$notContainson such a column answer membership throughjsonMembershipPredicate('sqlite', ...)(libSQL is SQLite); the negated form sits insidenullSafeNegative, so a row with no value satisfies$notContains. A scalar string column keeps the substring test.driver-turso; no remote-only dialect.The widening (for the contract review)
The transport keeps no schema, so it learns "is this a JSON-stored column" the way it learns every other declared fact, through an injected resolver. This adds ONE optional public method on the exported
RemoteTransportclass, and one type:RemoteTransport.setJsonColumnResolver(resolver: JsonColumnResolver): voidtype JsonColumnResolver: a function taking(object: string, field: string)and returningboolean, exported fromremote-transport.tsand NOT re-exported fromindex.ts, like its siblingsNonTextColumnResolverandDeclaredValueShapeResolver; it reaches the published.d.tsonly as the method's parameter type.TursoDriver's constructor wires it to the inheritedSqlDriver.isJsonColumn, besidesetDeclaredValueShapeResolver(constructor wiring only inturso-driver.ts; theupsertregions are untouched). In remote moderegisterRemoteFieldMetadatacallsregisterExternalObject, which fills the samejsonFieldsregistry the local face's gate and membership reading ask, so both faces read one population. ARemoteTransportdriven standalone without the resolver treats no column as JSON-stored and compiles as before. This is the fourth sibling ofsetFilterColumnSql,setNonTextColumnResolverandsetDeclaredValueShapeResolver; the last of those shipped as "New optional API" in commitfb386074's changeset. The seat's answer to the fork is on the card (claim amendment 5935032674, option A, open to the maintainer's veto).Why (measured at base
0b12b9ea, on the libsql SQLite stub harness)Over a
multiple: truelookup holding["u1","u2"]r1,["u2"]r2,["u3","u1"]r3,["u10"]r4 and null r5, the remote face answered, while the local face answered the right-hand column:$contains: 'u1'$notContains: 'u1'$nin: ['u1'],$ne: 'u1'INVALID_FILTER/ 400$eq,$in, bare equalityINVALID_FILTER/ 400$lt/$lte'u1'INVALID_FILTER/ 400$startsWith: '[',$endsWith: ']'INVALID_FILTER/ 400$icontains: 'u1'INVALID_FILTER/ 400{ owners: null }INVALID_FILTER/ 400jsonfield,$contains: 'u1'Pins
turso-local-remote-json-column-parity.test.ts(new): one fixture on BOTH faces; every case asserts remote equals local AND the canonical answer. It iterates the refused set fromJSON_COLUMN_INCOMPATIBLE_OPERATORSas it stands (27 members at base), assertingcode+status+ equality withjsonColumnOperatorRefusalText(...).message(never the literal words); the bare infix spellings in an operator map stay refused on both faces. Also: bare equality,nullequality ({f: null},$eq: null,$ne: null), the gate under$and/$or/$not, the population (atagsfield and ajsonfield),count(); membershipu1vs["u10"],u10,$notContainscomplement with the NULL row,tags, thejsonobject, bind alignment beside sibling predicates,count(); the scalar text-field control ($contains,$nin,$eq, bare and null equality,$startsWithunchanged);$null/$existsstill answered.remote-transport-compile-refusal-seam.test.ts: the enumeration requires every seam refusal method to have a row, sojsonColumnOperatorgets one per position (operator map, bare value, bare null) on a half-2 transport told exactly one column is JSON-stored. Policy / author / unmarked / merged-arm provenance all hold.Ablation (one-time proofs, via
scripts/ablation-replace.mjs, restore proven by blob hash and emptygit diff HEAD)The suite imports
./turso-driver.jsfrom source (vitest), so nodist/leg applies.turso-driver.ts(this.remoteTransport.setJsonColumnResolver(replaced by a no-op call; anchor 1 to 0, bloba9affc72tob90200ef): the parity suite goes 24 failed / 19 passed of 43, as predicted. Red: all 14 operator-map refusals, bare equality, depth, population, bothcount()pins,u1membership,$notContains, thejsonobject, bind alignment, null equality. Still green, as predicted: the 13 bare-infix rows (both faces refuse as an object comparand whatever the gate), the set check,u10andtagsmembership (substring coincides), the scalar controls, presence.remote-transport.ts(pushJsonMembershipanswers false): 5 failed / 38 passed, exactly the five membership pins whose substring answer differs; every refusal pin stays green.git diff HEADempty, blob equals HEAD in both legs.Filter-semantics compile surfaces, one conclusion per face
driver-sqlapplyFilterCondition(sql-driver.ts): already conformant — it is the contract; the parity suite's local column is its answer, anddriver-sqlite-wasmand local/replicadriver-tursoinherit it. Not edited.driver-tursoRemoteTransport.buildWhereSQL(remote-transport.ts): changed (this PR).service-analyticscompileScopedFilterToSql(read-scope-sql.ts): out of scope — another face; it already reaches the shared membership construct throughcontains-membership-sql.ts(importsjsonMembershipPredicate).service-analyticslowerAnalyticsWhere(strategies/filter-normalizer.ts): out of scope — another face, same shared construct as 3.formulamatchesFilterCondition(matches-filter.ts): out of scope — another face, and seat 2's in-flight #5930 step 4 (domain:engine): the engine-fed faces delete their hand-copied filter meaning (driver-sql, turso remote, memory query, mongodb, formula,having); the memory reference matcher retires (D6) #20822 group 3b owns it.objectqlapplyHaving/matchesHaving(having-filter.ts): out of scope — another face (it already imports the shared set and sentence), and seat 2's #5930 step 4 (domain:engine): the engine-fed faces delete their hand-copied filter meaning (driver-sql, turso remote, memory query, mongodb, formula,having); the memory reference matcher retires (D6) #20822 group 3b owns it.driver-memoryquery path (refuses the family since45ce12a4,filter-refusal.tsimports the shared set) anddriver-mongodbtranslateFieldOperators(mongodb-filter.ts): out of scope — other faces, not touched.Tests and gates (all on HEAD
c67a136eunless noted)pnpm --filter @objectstack/driver-turso exec vitest run --maxWorkers=2: 84 files passed, 2286 tests passed, 33 skipped (exit 0, viaos-verify-lock).pnpm --filter @objectstack/driver-turso typecheck: exit 0;tsc --listFilesincludes both edited test files.pnpm check:driver-conformance: before (base0b12b9ea) 50 covered / 0 DEBT / 0 exempt; after (c67a136e) 50 covered / 0 DEBT / 0 exempt. The ledger did not move.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commandsderived 63 commands atc67a136e; all 63 were run and reconciled with--ran(each line recording its exit code): 62 run, 1 NOT MEASURED.pnpm check:dual-build-cjs-loadsexited 3 (PREREQUISITE NOT MET: it reads every package'sdist/, and a repo-wide build is CI's); narrowed in its place,requireofdriver-turso's CJS build andimportof its ESM build both load and exposesetJsonColumnResolver.check-plugin-teardown-shape --self-testandcheck:lean-entry-closurefirst exited 3 on prerequisites (a fixture commit outside the shallow clone; objectql'sdist/) and exited 0 after fetching that commit and building objectql's closure.check-adr-0087-registration --base origin/main: the changeset reads as declared-breaking with one disposition,not-required (no-migration-prescription);check-changeset-no-major: no major bump.eslint --no-inline-config --format jsonread 4 files, 0 errors, 0 warnings. All four are ineslint .'s population (--print-configresolves each). The config never enables type-aware linting (the resolvedparserOptionsareecmaVersionandsourceTypeonly, andeslint.config.mjssays so in its header), so this diff cannot move a verdict on any untouched file. The fullpnpm lintis CI's.cli,qa/dogfood,runtime,service-datasource) are declared to CI: none referencesRemoteTransport's API, and the one remote-mode consumer test (date-bucket-parity-turso) aggregates without a JSON-field filter.Changeset
.changeset/21178-remote-json-column-gate.md:@objectstack/driver-tursominor, BREAKING banner,Clause-②: yes (narrowing), ADR-0087 dispositionnot-required (no-migration-prescription), the new optional API named, and the migration text (write$containsfor "holds this member", an$orof$containsfor any-of,$notaround either for the exclusion,$null/$exists/$emptyfor presence).Siblings
json-column-operator-refusal.ts; this PR edits neither that file nor that branch, and its pins compare against the builder's output, so the reword cannot flip them. Whichever of the two lands second mergesmainand re-runs its pins.upsertregions ofremote-transport.tsandturso-driver.ts; this PR is region-disjoint. Whichever lands second mergesmain.Acceptance notes
{ f: null },$eq: null,$ne: null) althoughIS NULLis a well-formed presence question there; this PR matches it on the remote face (that is the parity invariant), and$null/$exists/$emptyremain the presence spellings on both. Noted, not filed: no declared contract says otherwise ($eqis a declared member of the set). Carrier: none.{ owners: { $in: [{ ... }] } }reads the JSON-column sentence remotely and the comparand sentence locally. Both areINVALID_FILTER/ 400.=,in, …) in an operator map are refused on both faces as an object comparand, with different wording per face; pre-existing, not JSON-specific.Generated by Claude Code