Skip to content

Commit f8178ff

Browse files
fix(driver-memory): $contains on a multi-valued or JSON-stored field is membership, on every face (#20984)
Fixes #20874 Clause-②: yes (widening) ## What changes `driver-memory` now answers `$contains` and `$notContains` on a **declared JSON-stored field** by whole-element membership, the reading `driver-sql` compiles on all three dialects. A field counts as JSON-stored when it is `multiple: true`, a `multiselect` / `checkboxes` / `tags` field, or a `STRUCTURED_JSON_TYPES` member such as `json`. A scalar text column keeps the case-exact substring test. The change covers every face of the package: - the query path's `$`-spelling (`normalizeFieldOperators`) and its AST spelling (`convertConditionToMongo`): `find`, `count`, and every verb that goes through `convertToMongoQuery`; - the analytics (cube) face's mingo `$match`; - the analytics face's SQL echo (`generateSql`), which now renders SQLite's `json_each` membership construct for such a column, so the echoed statement still returns the rows the chart was drawn from. One rule backs all of them: `InMemoryDriver.filterContainsTest(object, field, value)`. On a JSON-stored field it returns `{ $elemMatch: { $in: members, $not: { $type: 'array' } } }`, and on any other field `{ $regex: filterSubstringPattern(value) }`. `$notContains` is `$not` over that test, on both faces. The members come from the comparand's text, read the way `driver-sql`'s `jsonMembershipCandidates` reads it: `'1'` names the string or the number 1, `'1.50'` names 1.5, and `'true'` / `'false'` / `'null'` name the string or the JSON literal. The `$contains` docblock in `packages/spec/src/data/filter.zod.ts` lost its stale "driver-memory DOES NOT ANSWER IT YET" bullet. Its dead tracker link went with it (that card answers 404). The replacement text states the measured status, and states that the other text operators over a stored array are not ruled by that section. ## Measured on `origin/main` `f6ccca4a` before the change, and after (HEAD `10656601`) Fixture: `driver-sql`'s #17590 fixture plus a multi-valued lookup `owners` (`['u1','u2']`, `['u10']`, `['u3','u1']`, `[]`). | `where` | memory before | SQLite (`driver-sql`) | memory after | |:--|:--|:--|:--| | `{ owners: { $contains: 'u1' } }` | 1, **2**, 3 | 1, 3 | 1, 3 | | `{ owners: { $notContains: 'u1' } }` | 4 (**2 dropped**) | 2, 4 | 2, 4 | | `{ tags_: { $contains: 'red' } }` | 1, **2** | 1 | 1 | | `{ nums: { $contains: '1' } }` (members are numbers) | **none** | 1 | 1 | | `{ label: { $contains: 'red' } }` (scalar control) | 1, 2 | 1, 2 | 1, 2 | | the cube face, the same filters | the query path's rows | n/a | the query path's rows | `engine.find` on a real `InMemoryDriver` with the nested-relation filter `{ owners: { region: 'NA' } }` (owner `u1` NA, `u10` EU; this is PR #20872's lowering): `d1, d3, d5` before and `d1, d3` after. Its `$not` gave `d2, d4` before and `d2, d4, d5` after, which matches the rest suite's SQLite rows. That was a one-off run (a throwaway probe file, not committed): this package may not import the engine and the engine's packages may not import this driver (`check:driver-memory-census`). What is pinned instead is the driver input the engine sends, an `$or` of one `$contains` per related id, which objectql's `engine-nested-relation-lowering.test.ts` asserts. ## Mechanism hypotheses (order Zone 2) - **H1 confirmed.** The `$contains` arm lowered to an escaped `$regex`, and mingo applies a `$regex` to each element of an array value. Reproduced on the fixture above before any edit. - **H2: the fork is by the DECLARED field, not the row's runtime shape.** Memory has the declaration (`valueShapes`, recorded by `syncSchema` since the `$empty` work), and SQL forks on its JSON-column registry, which is also filled from the declaration. The population is the spec's JSON-stored classes: `STRUCTURED_JSON_TYPES` plus `isMultiValueField`, the two halves `driver-sql`'s registry is built from. The two readings DO give different rows on one fixture, so the choice goes to the PM in the report's `open_questions`: a declared `json` field holding the scalar string `'u1'`. SQLite answers no member, because its constructs are array-only. A per-row-shape fork would answer it by substring. The declared fork matches SQLite, and the contract text says the question is "selected by the COLUMN", declared metadata. Undeclared fields (an object never passed through `syncSchema`) keep the substring reading, as `SqlDriver.isJsonColumn` answers `false` for a table it was never told about. - **H3 confirmed and pinned.** Number members answered nothing; `'1'`, `'2'`, `'10'`, `'0'` and `'1.50'` now give SQLite's rows. - **H4 confirmed: `$notContains` diverged.** It is the mirror arm of the same defect, changed under os-dev rule 3's bounded in-place exemption. All four conditions hold: 1. Same defect class. 2. Mechanical, with its shape pinned by SQL's `col IS NULL OR NOT (…)`: `$not` over the test admits null and missing rows, pinned on both fixtures. 3. `memory-driver.ts` is held by no other claim; the sibling `$exists` card is kept off it by its own claim. 4. Same tests, no new gate. **The claim's surface should be amended to include it.** - **H5 confirmed.** The cube face borrowed `filterSubstringPattern` and wrapped it in its own `$regex`, so it had the same defect. It now takes the driver's whole test. The echo reads its member set off that same test, so the chart and its echo cannot name different sets. - **H6 confirmed.** See the `engine.find` run above. ## Compile-surface conclusions | # | face | conclusion | |---|---|---| | 1 | `driver-sql` `applyFilterCondition` (`driver-sqlite-wasm`, `driver-turso` local inherit it) | **already compliant**: `applyJsonMembership` emits membership on JSON columns. Evidence: the #17590 suite now carries a multi-valued lookup column and the `u1`/`u10` case, green on SQLite here. Its live PostgreSQL/MySQL cells are unprovisioned locally; the required live job runs the whole driver-sql suite. The two inheriting drivers were not separately run. | | 2 | turso `RemoteTransport.buildWhereSQL` | **out of scope** (another package). By reading, its `$contains` arm emits `pushLike` (a GLOB substring) on every column, JSON columns included, so remotely `u1` would match `["u10"]` while the local transport answers membership. Not measured. Reported as a finding. | | 3 | service-analytics `read-scope-sql` `compileScopedFilterToSql` | **out of scope** (another package). **Measured**: `{ owners: { $contains: 'u1' } }` on a declared `lookup` + `multiple: true` field compiles to `instr("t"."owners", ?) > 0` on SQLite and `LIKE '%u1%'` on PostgreSQL/MySQL, a substring over the stored JSON text. This is an RLS read-scope face, so it is reported as a security-relevant finding. | | 4 | service-analytics `filter-normalizer` `lowerAnalyticsWhere` | **out of scope**. By reading, it lowers `$contains` to the cube `contains` operator, which the native SQL strategy renders as `LIKE`, a substring. Reported. | | 5 | `formula` `matchesFilterCondition` | **out of scope**. By reading, the arm is `typeof actual === 'string' && actual.includes(v)`, so a stored array never matches (fail-closed). Reported. | | half | objectql `having-filter` | **out of scope by the claim** (serial behind another card). Untouched. | | unfrozen | `driver-memory` query path and cube face (`memory-analytics.ts`), behind `filter-refusal.ts` | **changed** (this PR). `filter-refusal.ts` and the `$exists` arm are untouched (sibling card). | | unfrozen | `driver-mongodb` `translateFieldOperators` | **out of scope** (another package). By reading, it lowers `$contains` to a native `$regex`, which MongoDB applies per array element: the same defect. Not measured. Reported. | ## Why the shared pin is a mirrored literal table, not `FILTER_TEXT_CASES` (order Zone 3, not taken) `FILTER_TEXT_CASES` has no array column, so adding one changes the fixture of all five enrolled drivers. `driver-mongodb` imports every row of it and still carries the per-element defect, so its suite would go red. The table's own rule 2 says rows join a driver's suite in the PR that ends that driver's gap. Doing it here would also widen the spec touch beyond the one declared docblock. A new sibling case-set would add DEBT rows to a ledger that only goes down. So the new memory file mirrors `driver-sql`'s #17590 fixture row for row and asserts the same literal row sets. That is how the #17590 file already holds its three dialect cells to one answer. ## Files - `packages/drivers/driver-memory/src/memory-driver.ts`: the population (`isJsonStoredField`), the members, the one test (`filterContainsTest`), both query-path spellings. - `packages/drivers/driver-memory/src/memory-analytics.ts`: the `$match` rows take the driver's test; the echo renders membership (`sqliteMembershipPredicate`). - `packages/drivers/driver-memory/src/memory-20874-contains-membership.test.ts` (new): two fixtures. The first is the #17590 fixture plus `owners`. The second holds the stored shapes SQLite decides: a scalar, an object, a nested array, `[null]`, `[[null]]`, `[true, 1.5]`, `['']` and a NULL row, each answer measured on `driver-sql`/SQLite first. Every case runs on `find` in both spellings, on `count`, and on the cube's `$match`. The cube's echo is EXECUTED on sql.js over the same rows. The file also covers the nested-relation driver input and the fork. - `packages/drivers/driver-sql/src/sql-driver-17590-json-column-membership.test.ts`: an `owners` multi-valued lookup column and the `u1`/`u10` case (the claim's SQLite pin). - `packages/spec/src/data/filter.zod.ts`: the one declared docblock. - `.changeset/20874-memory-contains-membership.md`: **`minor`**, not the `patch` the order suggested. `filterContainsTest` is a new public method on the exported `InMemoryDriver`. It ships in `dist/index.d.ts` (the existing `filterSubstringPattern` appears there as the positive control), and the level ruling quoted in `pr-automation.yml` (WHICH LEVEL) grades an additive widening of a published package's public surface at least `minor`. `Clause-②: yes (widening)`: the new public method widens the published surface; no filter key or operator is added. - **Sweep, beyond the claim's listed surface.** The order's pin sweep asks for every same-semantics statement to be flipped in one round. Each item below is an edit to a statement this change makes false: - `packages/rest/src/data-nested-object-door.test.ts` (a header comment that cited the gap and the removed docblock clause); - `.changeset/20802-nested-relation-filter-served.md`, one sentence of a **pending** release note that said the in-memory driver still matches per element. ## Confirmation requested: a pending release note is corrected This PR changes `.changeset/20802-nested-relation-filter-served.md`, a pending release note this PR did not add. This is the DELIBERATE CORRECTION class `check-empty-changeset.mjs` names, so `Check Changeset` stays **red** on purpose. It is not a required context. `skip-changeset` must not be applied. The note's last sentence said: > On the in-memory driver, a multi-valued relation's `$contains` still matches a stored id by substring per element, so there an id that is a substring of another stored id (`u1` inside `u10`) also matches; SQLite and PostgreSQL match the element. This PR makes that false, so it now reads: > SQLite, PostgreSQL and the in-memory driver match the element of a multi-valued relation, so an id that is a substring of another stored id (`u1` inside `u10`) does not match it. Please confirm the correction on this PR. If the release that consumes that note ships before this PR lands, the edit should be dropped on rebase. ## Tests (HEAD `10656601`) - `pnpm --filter @objectstack/driver-memory typecheck` passes, and `vitest run` gives **68 files / 1553 tests passed**. Both typecheck programs include the new test file (`--listFiles`). - The new file alone gives **113 passed**. - `pnpm --filter @objectstack/driver-sql typecheck` passes (its program includes the edited test). `sql-driver-17590-json-column-membership.test.ts` gives **22 passed, 2 skipped**; the skips are the live PostgreSQL/MySQL cells, unprovisioned here, NOT MEASURED locally. - `pnpm --filter @objectstack/spec typecheck` passes. `spec build` and then `check:generated` report "All 15 generated artifacts are up to date". **Reverse verification.** The code change was committed first. Then `memory-driver.ts` and `memory-analytics.ts` were checked out at BASE under an EXIT/INT/TERM restore trap. The landed mutation was verified on disk before the run: `filterContainsTest` count 0, two `escapeRegex(val)` arms back, and both files byte-equal to BASE. The predicted direction was that the cases where per-element substring and membership disagree go red and the agreeing ones stay green. Measured: **69 failed / 44 passed**. The failures were every disagreeing case, on all four faces, plus the executed echo, the relation-lowering input and the fork pin. The `label` controls and the `redwood`, `ab`, `u10` and `'0'` cases stayed green. The restore was proven by blob hashes equal to HEAD (`d376dadb…` / `9de9198b…`), an empty `git diff HEAD` and clean porcelain. **Guard ablation** (`scripts/ablation-replace.mjs`, anchor 1 to 0). Dropping `$not: { $type: 'array' }` turned exactly the nested-array cases red: 12, the json `'u1'`, its complement and `'null'`, on all four faces. The executed echo stayed green, since SQLite is the oracle there. The file was restored to the HEAD blob with an empty `git diff HEAD`. **Lint (narrowed, a measurement).** 1. Population: `eslint.config.mjs` lints `**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` minus `NEVER_LINTED`. The six changed TS files are all in it; the two changesets are not. 2. `eslint --no-inline-config --format json` over those six files reports 6 files, 0 errors, 0 warnings at `10656601`. 3. The config never enables type-aware linting (no `parserOptions.project`, no typed rules; it says so itself), so this diff cannot move any untouched file's verdict. **Driver conformance ledger**: `node scripts/check-driver-conformance.mjs` reads "50 covered cell(s), 0 in the DEBT ledger, 0 exempt" both at `f6ccca4a` (before) and at `10656601` (after). **Gates**: `dispatch-gates --commands` was re-derived at `10656601`. That gives 84 commands, six more than the dispatch list: `engine-double-contract`, `objectql-double-limit`, `query-options-erasure`, `type-check-coverage`, `type-check-debt` and `where-matcher`. The run also covered the eight roster gates flagged as sharing a directory with these paths. All exit 0 except: - `check-empty-changeset`: exit 1, the deliberate correction above. - `check:dual-build-cjs-loads` and `check:type-check-debt`: exit 3, `PREREQUISITE NOT MET` (they need the whole workspace built). **NOT MEASURED**; CI's Build/Lint jobs own those. - `check:lean-entry-closure`: exit 3 at first. It passed after building objectql. ## Acceptance notes (observed, not filed by this PR) - The other text operators over a stored array are not ruled by the membership section and still differ: `{ owners: { $startsWith: 'u1' } }` gives `d1, d3, d5` on memory (per element) and no rows on SQLite (GLOB over the serialized text). The spec docblock now says so. - objectql `engine.ts`'s delete-probe docblock (the `$contains` pushdown) still says every backend answers a substring SUPERSET. That is stale for `driver-sql` since #17590 and for memory after this PR. The code stays correct because it narrows exactly through `storedReferenceIncludes`. Comment only, no carrier. - A field `driver-memory` holds no declaration for keeps the substring reading, and mingo still applies that `$regex` per element of an array value. This is the deliberate counterpart of `driver-sql`'s `isJsonColumn` returning false for an unknown table. --- _Generated by [Claude Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent a5bce40 commit f8178ff

8 files changed

Lines changed: 676 additions & 43 deletions

‎.changeset/20802-nested-relation-filter-served.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,4 @@ Measured through `POST /api/v1/data/:object/query` on SQLite and PostgreSQL 16 (
2525
| `{ $not: { owner: { region: "NA" } } }` | `INVALID_FILTER` / 400 | `d2`, `d4` |
2626
| `{ owner: { region: "APAC" } }` (no owner matches) | `INVALID_FILTER` / 400 | no rows |
2727

28-
On the in-memory driver, a multi-valued relation's `$contains` still matches a stored id by substring per element, so there an id that is a substring of another stored id (`u1` inside `u10`) also matches; SQLite and PostgreSQL match the element.
28+
SQLite, PostgreSQL and the in-memory driver match the element of a multi-valued relation, so an id that is a substring of another stored id (`u1` inside `u10`) does not match it.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
"@objectstack/driver-memory": minor
3+
---
4+
5+
fix(driver-memory): `$contains` / `$notContains` on a multi-valued or JSON-stored field answer by membership, as the SQL drivers do
6+
7+
Clause-②: yes (widening) — one new public method on the exported `InMemoryDriver` class, `filterContainsTest`; its return type `MemoryContainsTest` is not re-exported from the package entry. No accepted filter key or operator is added: `$contains` and `$notContains` keep their declared shape.
8+
9+
On a field whose declaration makes it JSON-stored (`multiple: true` on a `lookup`, `user`, `select`, `radio`, `file` or `image` field, a `multiselect`, `checkboxes` or `tags` field, or a structured type such as `json`), the in-memory driver now answers `{ field: { $contains: v } }` by whole-element membership: some element of the stored array equals `v`. It used to match each element by substring, so `u1` matched a row storing `['u10']` and `'red'` matched a row storing `['redwood']`. A number member answered nothing: `{ nums: { $contains: '1' } }` missed `[1, 2]`. `$notContains` is the exact complement, and a row with no value still satisfies it. A scalar text column keeps the case-exact substring test.
10+
11+
`driver-sql` gives the same answer on SQLite, PostgreSQL and MySQL; the two drivers were measured over the same fixture. The answer holds on every face of this driver:
12+
13+
- `find()` and `count()`, in both filter spellings;
14+
- the nested-relation filter on a multi-valued relation, which the engine lowers to one `$contains` per related id;
15+
- `MemoryAnalyticsService`'s query, and its SQL echo, which now renders SQLite's `json_each` membership construct for such a column.
16+
17+
The comparand is still a string. A number or boolean member is named by its text: `'1'` matches the stored number `1` (and `'1.50'` the number `1.5`), `'true'` matches the boolean `true`, and `'null'` matches a `null` member. A field the driver holds no declaration for, such as a field on an object never passed through `syncSchema`, keeps the substring reading.
18+
19+
New: `InMemoryDriver.filterContainsTest(object, field, value)` returns the one test every face above lowers `$contains` to. It is a narrow seam for the analytics face, beside `filterSubstringPattern` and `filterComparandStorageForm`. The added public method is why this entry is `minor`.
20+
21+
**If your tests relied on the old answer:** on the in-memory driver, a filter that matched an id by prefix or a tag by substring now returns only the member rows. That is what SQL already returned in production. Write `$contains` with the whole member value.

‎packages/drivers/driver-memory/src/memory-20874-contains-membership.test.ts‎

Lines changed: 339 additions & 0 deletions
Large diffs are not rendered by default.

‎packages/drivers/driver-memory/src/memory-analytics.ts‎

Lines changed: 87 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ import {
2020
// call: `isFilterAST` gates the shape, `parseFilterAST` lowers it. See
2121
// {@link lowerWhereFilterArray}. ⛔ No second FilterArray parser in this face.
2222
import { isFilterAST, parseFilterAST, VALID_AST_OPERATORS } from '@objectstack/spec/data';
23-
import type { InMemoryDriver } from './memory-driver.js';
23+
import type { InMemoryDriver, MemoryContainsTest } from './memory-driver.js';
2424
import {
2525
Logger,
2626
createLogger,
@@ -238,15 +238,23 @@ interface MongoPredicateInput {
238238
/** The operands as authored. For operands that are not comparands. */
239239
readonly raw: readonly unknown[];
240240
/**
241-
* A comparand as a literal-substring pattern, built by the DRIVER's own rule
242-
* (`filterSubstringPattern`) rather than re-derived here.
241+
* The test `$contains` asks of this member's column, built by the DRIVER's
242+
* own rule (`filterContainsTest`) rather than re-derived here — `$notContains`
243+
* wraps it in `$not`, as the live query path does.
243244
*
244245
* [#7723] Case-EXACT, because that rule is: `filterSubstringPattern` carried
245246
* an `i` flag until #7723 took it off, putting the `$contains` family on the
246247
* #4706 Q2 = A answer across every face of this package. Borrowing the rule
247248
* rather than restating it is what made that one edit reach this face too.
249+
*
250+
* [#20874] Borrowed WHOLE now, not as a pattern. On a declared JSON-stored
251+
* field the rule asks MEMBERSHIP (an `$elemMatch`), not substring, and this
252+
* face used to wrap `filterSubstringPattern` in a `$regex` of its own — so
253+
* `u1` matched a stored `['u10']` here exactly as it did on `find()`. Taking
254+
* the predicate rather than a piece of it is what makes the membership reading
255+
* reach this face with the same edit, the lesson #7723 records above.
248256
*/
249-
readonly substring: (value: unknown) => RegExp;
257+
readonly containment: (value: unknown) => MemoryContainsTest;
250258
/**
251259
* [#6520] A comparand as an ASCII-case-insensitive literal-substring pattern —
252260
* `$icontains`' fold, which is NOT {@link substring}'s.
@@ -335,17 +343,19 @@ const CUBE_OPERATOR_TO_MONGO_PREDICATE: Readonly<Record<CubeOperator, MongoPredi
335343
// table while `find()` answered with none of it.
336344
in: ({ comparands }) => ({ $in: [...comparands] }),
337345
notIn: ({ comparands }) => ({ $nin: [...comparands] }),
338-
// A pattern, not a comparand: `raw`, and the driver's own substring rule.
339-
contains: ({ raw, substring }) => ({ $regex: substring(raw[0]) }),
346+
// A pattern, not a comparand: `raw`, and the driver's own rule — membership on
347+
// a declared JSON-stored field, the substring everywhere else (#20874).
348+
contains: ({ raw, containment }) => containment(raw[0]),
340349
// [#6520] The case-INSENSITIVE twin, folding ASCII and nothing else. It takes
341350
// `asciiSubstring`, not `substring`: the neighbour above folds Unicode, so
342351
// reusing it here would answer `CAFÉ` for `café` on this face while the SQL
343352
// family answered no rows — the divergence #6520 closed.
344353
icontains: ({ raw, asciiSubstring }) => ({ $regex: asciiSubstring(raw[0]) }),
345354
// The fix this issue is about. `{$not: <scalar>}` constrains nothing; the
346355
// negation has to wrap a pattern, which is exactly what the live query path
347-
// builds for `$notContains` (`memory-driver.ts` `normalizeFieldOperators`).
348-
notContains: ({ raw, substring }) => ({ $not: { $regex: substring(raw[0]) } }),
356+
// builds for `$notContains` (`memory-driver.ts` `normalizeFieldOperators`) —
357+
// and, since #20874, the SAME test it builds, membership or substring.
358+
notContains: ({ raw, containment }) => ({ $not: containment(raw[0]) }),
349359
// [#13195] A presence flag, not a comparand — and "present" means HAS A
350360
// VALUE (`!= null`), never key presence: #5298 leg 3 / #5369, landed in PR
351361
// #5962, ruled onto this face 2026-08-30. It used to emit `{$exists: <bool>}`
@@ -386,6 +396,52 @@ interface SqlPredicateInput {
386396
* See {@link globSubstringPattern} for why GLOB and not LIKE.
387397
*/
388398
readonly globSubstring: (value: unknown) => string;
399+
/**
400+
* [#20874] The stored members a `$contains` comparand names on this column,
401+
* or `null` when the column asks the SUBSTRING question — read off the
402+
* DRIVER's test (`filterContainsTest`), the one its `$match` twin executes.
403+
* See {@link sqliteMembershipPredicate}.
404+
*/
405+
readonly members: (value: unknown) => readonly unknown[] | null;
406+
}
407+
408+
/**
409+
* [#20874] The SQLite rendering of a `$contains` MEMBERSHIP test — `driver-sql`'s
410+
* own SQLite construct (`jsonMembershipPredicate`), with literals where that
411+
* one binds.
412+
*
413+
* The echo's job is reproducing execution ({@link globSubstringPattern}). Once
414+
* the `$match` twin asks membership on a declared JSON-stored column, a
415+
* `GLOB '*u1*'` echo over the stored text `["u10"]` would return the row the
416+
* chart excludes — so this renders the question that ran:
417+
*
418+
* - `json_each` over the column, guarded by `json_valid` so a cell holding
419+
* bare text has no members instead of raising — the guard `driver-sql` keeps
420+
* for the same reason;
421+
* - `typeof(os_member.key) = 'integer'` keeps it array-only: an array element
422+
* has an INTEGER key, an object member a TEXT key and a scalar root a NULL
423+
* one — the same "a scalar or object has no member" the `$elemMatch` twin
424+
* answers;
425+
* - each element's JSON TEXT compared with each member's, the `CASE` spelling
426+
* the three JSON literals by type name because SQLite surfaces `true` as the
427+
* INTEGER 1 and `json_quote` would render it `1`.
428+
*
429+
* The members' JSON texts are exactly the texts `driver-sql` binds, because
430+
* both sides read the comparand the same way (`containsMemberCandidates` in
431+
* `memory-driver.ts`, `jsonMembershipCandidates` in `driver-sql`).
432+
*/
433+
function sqliteMembershipPredicate(
434+
column: string,
435+
members: readonly unknown[],
436+
literal: (value: unknown) => string,
437+
): string {
438+
const texts = members.map((member) => literal(JSON.stringify(member))).join(', ');
439+
return (
440+
`EXISTS (SELECT 1 FROM json_each(CASE WHEN json_valid(${column}) THEN ${column} ELSE '[]' END) AS os_member ` +
441+
`WHERE typeof(os_member.key) = 'integer' AND CASE os_member.type ` +
442+
`WHEN 'true' THEN 'true' WHEN 'false' THEN 'false' WHEN 'null' THEN 'null' ` +
443+
`ELSE json_quote(os_member.value) END IN (${texts}))`
444+
);
389445
}
390446

391447
type SqlPredicateBuilder = (input: SqlPredicateInput) => string;
@@ -537,10 +593,20 @@ const CUBE_OPERATOR_TO_SQL_PREDICATE: Readonly<Record<CubeOperator, SqlPredicate
537593
comparands.length === 0
538594
? '1 = 1'
539595
: `(${column} IS NULL OR ${column} NOT IN (${comparands.map(literal).join(', ')}))`,
540-
// A pattern, not a comparand: `raw`, and the shared GLOB substring rule.
541-
contains: ({ column, raw, globSubstring }) => `${column} GLOB ${globSubstring(raw[0])}`,
542-
notContains: ({ column, raw, globSubstring }) =>
543-
`(${column} IS NULL OR ${column} NOT GLOB ${globSubstring(raw[0])})`,
596+
// A pattern, not a comparand: `raw`, and the shared GLOB substring rule — or,
597+
// on a declared JSON-stored column, the membership the `$match` twin runs
598+
// (#20874, {@link sqliteMembershipPredicate}). The negation stays null-safe
599+
// either way: `NOT EXISTS` is never UNKNOWN, but the NULL row still has to be
600+
// admitted by name, since `json_each(NULL)` is not a row set to negate.
601+
contains: ({ column, raw, globSubstring, members, literal }) => {
602+
const set = members(raw[0]);
603+
return set ? sqliteMembershipPredicate(column, set, literal) : `${column} GLOB ${globSubstring(raw[0])}`;
604+
},
605+
notContains: ({ column, raw, globSubstring, members, literal }) => {
606+
const set = members(raw[0]);
607+
const test = set ? `NOT ${sqliteMembershipPredicate(column, set, literal)}` : `${column} NOT GLOB ${globSubstring(raw[0])}`;
608+
return `(${column} IS NULL OR ${test})`;
609+
},
544610
// [#6520] The case-INSENSITIVE twin. SQLite's `lower()` folds ASCII and
545611
// nothing else — measured in #6518: `lower('CAFÉ')` is `'cafÉ'` — so it is
546612
// `$icontains`' fold (#4706 Q1 = A) rather than the Unicode one, and it goes
@@ -1595,10 +1661,11 @@ export class MemoryAnalyticsService implements IAnalyticsService {
15951661
// predicate over `is_active` or `closed_at` selects the same rows
15961662
// `find()` selects instead of none / all of them.
15971663
const storageForm = this.storageFormFor(cube, filter.member);
1664+
const table = this.extractTableName(cube.sql);
15981665
const predicate = this.mongoPredicateBuilder(filter.operator)({
15991666
comparands: filter.values.map(storageForm),
16001667
raw: filter.values,
1601-
substring: (value) => this.driver.filterSubstringPattern(value),
1668+
containment: (value) => this.driver.filterContainsTest(table, fieldPath, value),
16021669
// [#6520] `$icontains`' fold, from the spec's shared definition rather
16031670
// than from the driver's Unicode-folding `filterSubstringPattern`.
16041671
asciiSubstring: (value) => new RegExp(asciiCaseInsensitiveRegexSource(String(value))),
@@ -1663,12 +1730,19 @@ export class MemoryAnalyticsService implements IAnalyticsService {
16631730
}
16641731
const fieldPath = this.resolveFieldPath(cube, entry.member);
16651732
const storageForm = this.storageFormFor(cube, entry.member);
1733+
const table = this.extractTableName(cube.sql);
16661734
clauses.push(this.sqlPredicateBuilder(entry.operator)({
16671735
column: fieldPath,
16681736
comparands: entry.values.map(storageForm),
16691737
raw: entry.values,
16701738
literal: (value) => this.toSqlLiteral(value),
16711739
globSubstring: (value) => this.toSqlLiteral(globSubstringPattern(value)),
1740+
members: (value) => {
1741+
// [#20874] Read off the very test the `$match` exit runs, so the echo
1742+
// and the chart can never name two different member sets.
1743+
const test = this.driver.filterContainsTest(table, fieldPath, value);
1744+
return '$elemMatch' in test ? test.$elemMatch.$in : null;
1745+
},
16721746
}));
16731747
}
16741748
return clauses;

0 commit comments

Comments
 (0)