diff --git a/.changeset/19757-dispatch-cases-array-where-id-retired.md b/.changeset/19757-dispatch-cases-array-where-id-retired.md new file mode 100644 index 00000000000..b944cd2a7e4 --- /dev/null +++ b/.changeset/19757-dispatch-cases-array-where-id-retired.md @@ -0,0 +1,17 @@ +--- +"@objectstack/metadata-core": patch +--- + +`ENGINE_DELETE_DISPATCH_CASES` and `ENGINE_UPDATE_DISPATCH_CASES` retire their three ARRAY `where.id` rows (#19757) + +The engine-double conformance tables no longer carry these three rows: + +- delete's `array id, no multi` +- update's `array id, no multi` +- update's `a SCALAR data.id beside an ARRAY where.id` + +Each row puts `where: { id: ['a', 'b'] }` in the equality slot. Since this release's `@objectstack/spec` change, the shared comparand-shape face refuses an array in that slot with `INVALID_FILTER` / 400. The face runs at the engine's lowering seam, which every verb crosses before the dispatch runs, so the real engine never reaches the dispatch predicate with such an input. A row claiming a dispatch verdict for it would pin a branch the engine cannot reach. It was measured red against the real engine: `ObjectQL.delete` / `ObjectQL.update` refused the input with the face's words, not the dispatch's. + +The predicates themselves are unchanged. `resolveEngineDeleteDispatch` / `resolveEngineUpdateDispatch` and the `assert*` helpers still answer an array `where.id` with `reject`, and `scalarDeleteId` / `scalarUpdateId` still treat an array as not-an-id. A test double bound to them therefore still refuses such a call, with the dispatch's sentence. No double runs the shared filter face, for this shape or for any other face refusal. The `$in` rows keep the "a non-scalar `where.id` is not an id" coverage, including the #11230 refusal beside a scalar payload id. + +If you run these tables against your own engine double, it has three fewer cases to answer. Nothing else changes. diff --git a/.changeset/19757-equality-slot-array-refused.md b/.changeset/19757-equality-slot-array-refused.md new file mode 100644 index 00000000000..5cd08308867 --- /dev/null +++ b/.changeset/19757-equality-slot-array-refused.md @@ -0,0 +1,63 @@ +--- +"@objectstack/spec": minor +--- + +fix(spec)!: the shared comparand-shape face refuses an ARRAY in the equality slot — `{ field: [...] }` and `{ field: { $eq: [...] } }` — for every driver at once (#19757) + +**BREAKING** — an accept-set narrowing at the runtime filter doors, shipped as `minor` under the repo's launch-window convention for accept-set narrowings. Ruled on #19757 (record 5793368540, letter 乙, 「217 同意」): an array in the implicit-equality slot is refused at the shared face, for every driver at once — no alias, no grace window. The hand-migration prescription is registered under protocol major 18 as `filter-equality-array-comparand-refused`. + +## What changes + +`parseFilterAST` lowers `['tags', 'equals', ['a']]` — and the same triple on `=`, `==` and `eq` — to the implicit form `{ tags: ['a'] }`. That shape, and its explicit spelling `{ tags: { $eq: ['a'] } }`, now get `INVALID_FILTER` / 400 from the shared comparand-shape face (`assertListComparandShapes` in `@objectstack/spec/data`). That face runs inside `parseFilterAST` and at the engine's lowering seam on both engine doors, so the refusal lands before any driver runs, at any depth under `$and` / `$or` / `$not`. The empty array is refused too. The message names the field, the path, and the two operators a list in that slot was standing in for: `{"$in": […]}` for "one of these values" (authoring spelling `in`), and `{"$contains": "…"}` for "the stored list holds a value" on a multi-value field (authoring spelling `contains`), with an `$or` of those for any-of. + +How each backend answered the lowered `{ tags: ['a'] }` before this change. Each was run for this change beside a scalar and an `$in` control: + +| backend | before | how it was measured | +|:--|:--|:--| +| `driver-sql` (SQLite) | **refused**, 400, at the top level. Nested under `$and` / `$or` / `$not` it answered **500 `DATABASE_ERROR`**: SQLite could not bind the list. | `SqlDriver.find` on better-sqlite3 | +| `driver-memory` | **refused**, 400, at every depth | `InMemoryDriver.find` | +| `@objectstack/formula` | **no row**, including a row storing exactly `['a']` | `matchesFilterCondition` | +| `driver-mongodb` | **answered**. `translateFilter` emits the array unchanged. MongoDB equality on an array operand selects a stored array **equal to** `['a']` **or holding** `['a']` as an element. | `translateFilter`, then mingo 7.2.4 as the named proxy for the server. Over `['a']`, `'a'`, `['a','b']`, `['b','a']`, `[['a'],'x']`, `[['a']]`, `'b'` and `[]`, it selected `['a']`, `[['a'],'x']` and `[['a']]`. | +| `service-analytics` filter normalizer | **answered as membership**. The FilterArray form `[['stage', '=', ['won', 'lost']]]` charted as `stage IN ('won', 'lost')`. | `normalizeAnalyticsFilterTree` | + +⚠️ NOT MEASURED: a live `mongod`, MySQL, PostgreSQL, and a live Turso server. `driver-turso` and `driver-sqlite-wasm` are built on `driver-sql` and were not run separately. + +After this change, every row above that goes through a platform door gets the 400. That covers `parseFilterAST`, the engine's lowering seam on both doors (every engine verb's `where` passes through it; measured on `find` and `count`), and the analytics normalizer's FilterArray form. The drivers themselves are untouched, so a caller that hands a raw `FilterCondition` straight to a driver, without `parseFilterAST`, still gets that driver's own answer. + +## What does NOT change + +- **`$ne` carrying an array is not judged.** The ruling names implicit and explicit equality. `$ne` measured the same split (refused by `driver-sql` and `driver-memory`, answered by `driver-mongodb`) and is left to its own ruling. +- The other scalar operators carrying an array (`$gt`, `$contains`, `$like`, …) are not judged here either. +- The list operators keep their arrays: `$in`, `$nin` and `$between`, including `$in: []` / `$nin: []`. +- Every scalar equality comparand is untouched. That includes `null`: `{ field: null }` and `{ field: { $eq: null } }` are the has-no-value predicate. +- A `{ $field }` reference on an equality spelling still lowers to `$eq` and passes. +- A field spec with no `$` key (`{ author: { tags: ['a'] } }`) is still not descended into. +- The schema doors are not touched. `FilterConditionSchema` still parses `{ field: [...] }`, because `FieldOperatorsSchema.$eq` is `z.any()`. A document carrying the shape therefore still publishes, and is refused when it is queried. +- `ViewFilterRule` already refused an array on every scalar view operator at authoring time. + +## FROM → TO + +| you wrote | write instead | +|:--|:--| +| `{ tags: ['a', 'b'] }` / `[['tags', 'equals', ['a', 'b']]]`, meaning "one of these values" | `{ tags: { $in: ['a', 'b'] } }` / `[['tags', 'in', ['a', 'b']]]` | +| `{ tags: ['a'] }`, meaning "the stored list holds `a`" on a multi-value field | `{ tags: { $contains: 'a' } }` / `[['tags', 'contains', 'a']]` | +| `{ tags: ['a', 'b'] }`, meaning "the stored list holds `a` or `b`" | `{ $or: [{ tags: { $contains: 'a' } }, { tags: { $contains: 'b' } }] }` | +| `{ tags: ['a'] }`, meaning one value | `{ tags: 'a' }` | +| `{ tags: { $eq: [...] } }` | any of the rows above | + +On `driver-mongodb`, check what the query is supposed to return, and do not assume the old rows were right. The old answer was MongoDB array equality, and neither `$in` nor `$contains` gives the same rows. A dashboard or dataset filter written as the FilterArray sugar with an array on equality used to chart as membership. It is now refused, and `$in` is the spelling that charts the same rows. + +## Who is affected, measured + +Nothing in this repository's examples, seeds, docs or published skills authors the shape. The repo was grepped for the FilterArray triple on `=` / `==` / `equals` / `eq` carrying an array, for `$eq` carrying an array, and for filter / where objects whose field value is an array. The hits are tests and the engine-double conformance tables. The full suites of `@objectstack/spec`, `objectql`, `driver-memory`, `driver-sql`, `driver-mongodb`, `driver-turso`, `driver-sqlite-wasm`, `formula`, `service-analytics`, `metadata-protocol`, `metadata-core`, `plugin-sharing` and `lint` were run, and four things went red. Each was re-judged, not rewritten by rote: + +- The comparand-shape suite pinned `{ tags: ['a','b'] }` and `$eq: ['a','b']` as shapes the face passes through. Both rows are inverted, and the shapes now live in the arm's refusal section. +- The field-reference lowering suite pinned `['stage', '=', ['a','b']]` lowering to the implicit form. What that row proved still holds, because an array is not promoted to `$eq`. The row now asserts the refusal, which names the implicit slot and not `$eq`. +- Two probe helpers passed a two-element array through every AST spelling to find its `$` operator. One is in this package's comparand-shape suite, the other in `driver-memory`'s vocabulary suite. Each assumed the array could never trip the face. The equality spellings now refuse it, so each helper reads that refusal as `undefined`, which is the answer the helper always gave those spellings. +- `@objectstack/metadata-core`'s engine-double dispatch tables carried three ARRAY `where.id` rows. The real engine now refuses that input at the face before its dispatch runs, so the rows are retired. Their own changeset explains why. + +`FILTER_COMPARAND_TYPE_CASES` gains three `door-refusal` rows (implicit, `$eq`, and nested under `$or`). Every driver suite that consumes the table runs them through `parseFilterAST`. + +Clause-②: no (narrowing) — nothing is widened. No key is added, removed or renamed, no exported symbol moves, and the operator vocabulary is unchanged. The runtime accept set narrows: one comparand shape in one slot, which the face now refuses the way `driver-sql` and `driver-memory` already did. + + diff --git a/packages/drivers/driver-memory/src/memory-filter-ast-vocabulary.test.ts b/packages/drivers/driver-memory/src/memory-filter-ast-vocabulary.test.ts index 5501511d58b..1e0f84fe300 100644 --- a/packages/drivers/driver-memory/src/memory-filter-ast-vocabulary.test.ts +++ b/packages/drivers/driver-memory/src/memory-filter-ast-vocabulary.test.ts @@ -93,10 +93,21 @@ describe('InMemoryDriver filter vocabulary ↔ VALID_AST_OPERATORS', () => { * gets a list here without anyone remembering to edit this line. * * The probe uses a two-element array, which is legal for all three list - * operators, so it never trips the shape door it is used to satisfy. + * operators, so it never trips the shape door it is used to satisfy. The + * EQUALITY spellings (`=`, `==`, `equals`, `eq`) do refuse it, since the + * 2026-09-23 equality-slot ruling (#19757) — and they answer `undefined` + * either way: before that ruling they lowered it to the implicit form, which + * carries no `$` key. So a refusal here reads as `undefined`, exactly the + * answer this helper always gave them, and `valueFor` still hands them a + * scalar. */ const loweredOperatorOf = (op: string): string | undefined => { - const lowered = parseFilterAST([['probe', op, ['a', 'b']]]) as Record | undefined; + let lowered: Record | undefined; + try { + lowered = parseFilterAST([['probe', op, ['a', 'b']]]) as Record | undefined; + } catch { + return undefined; + } const spec = lowered?.probe; if (spec === null || typeof spec !== 'object' || Array.isArray(spec)) return undefined; return Object.keys(spec).find((key) => key.startsWith('$')); diff --git a/packages/drivers/driver-mongodb/src/mongodb-equality-array-comparand-refusal.test.ts b/packages/drivers/driver-mongodb/src/mongodb-equality-array-comparand-refusal.test.ts new file mode 100644 index 00000000000..5ebacc9c948 --- /dev/null +++ b/packages/drivers/driver-mongodb/src/mongodb-equality-array-comparand-refusal.test.ts @@ -0,0 +1,196 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#19757] An ARRAY in the EQUALITY slot is refused at the shared face BEFORE + * `translateFilter` sees it — this driver's pin on the 2026-09-23 ruling. + * + * ## What this driver did with the shape, measured + * + * `parseFilterAST([['tags', 'equals', ['a']]])` lowers to the implicit form + * `{ tags: ['a'] }` (so do `=`, `==` and `eq`). Until the ruling that shape + * passed both shared comparand doors, and this driver was the one backend that + * ANSWERED it rather than refusing: `translateFilter` has no gate for the slot + * and emits it unchanged (the reverse-direction pin at the bottom keeps that + * visible), and `MongoDBDriver.find()` hands its output to the server with + * nothing in between. On the server that is MongoDB's equality on an array + * operand, which selects a stored array EQUAL to `['a']` or HOLDING `['a']` as + * one of its elements — never a row storing the scalar `'a'`. + * + * That server reading is taken through **mingo 7.2.4, the named proxy** (the + * MongoDB query-semantics library `driver-memory` hands its filters to), over + * the rows `['a']`, `'a'`, `['a','b']`, `['b','a']`, `[['a'],'x']`, + * `[['a']]`, `'b'` and `[]`: `{ tags: ['a'] }` and `{ tags: { $eq: ['a'] } }` + * each selected `['a']`, `[['a'],'x']` and `[['a']]`, and `{ tags: [] }` + * selected `[]`. ⚠️ **A live `mongod` is NOT MEASURED** — this fleet cannot + * fetch a mongod binary (#5517), and mingo is not a dependency of this package, + * so the proxy reading was taken outside this suite, recorded here and in the + * PR, and is not re-run by it. Every other backend refused the same shape + * (`driver-sql`, `driver-memory`) or excluded every row (`@objectstack/formula`), + * so one stored filter answered silently on exactly this backend. + * + * ## What the ruling changed, and what it deliberately did not + * + * The comparand-SHAPE face (`@objectstack/spec/data`, `assertListComparandShapes`) + * now refuses the shape — implicit and `$eq` — with `INVALID_FILTER` / 400, and + * it runs inside `parseFilterAST` and at the engine's lowering seam on both + * engine doors. ⛔ This driver's source is NOT edited: the ruling's point is + * that the one shared door answers for every driver at once, so a second, + * driver-local copy of the rule would be the "one declared contract, one + * implementation per backend" shape the face exists to end. + * + * ## Why the translator and a recording engine, not a live mongod + * + * The same reason as `mongodb-null-comparand-refusal.test.ts`: `translateFilter` + * is a pure function whose output IS the query document MongoDB receives, and a + * suite that needs `mongodb-memory-server` is skipped whenever its download is + * blocked — a test of a ruling that can be skipped is not a test of the ruling. + * The measurement here is at the compile face: every door a query crosses + * refuses the shape, and `translateFilter` is never called. + */ + +import { describe, it, expect, beforeEach } from 'vitest'; +import { parseFilterAST } from '@objectstack/spec/data'; +import { ObjectQL } from '@objectstack/objectql'; +import { translateFilter } from './mongodb-filter.js'; + +interface WireBearingError extends Error { + code?: string; + status?: number; +} + +const deal = { + name: 'deal', + label: 'Deal', + fields: { + id: { name: 'id', type: 'text' as const, primaryKey: true }, + stage: { name: 'stage', type: 'text' as const }, + tags: { name: 'tags', type: 'text' as const }, + }, +}; + +/** + * A driver whose read path is THIS package's translator and nothing else: every + * `where` the engine hands it is passed to `translateFilter`, and every call is + * counted. "translateFilter never saw it" is therefore a count, not an + * inference from a thrown error. + */ +function makeTranslatingDriver() { + const translated: unknown[] = []; + const translate = (ast: { where?: unknown } | undefined): unknown => { + const doc = translateFilter(ast?.where); + translated.push(doc); + return doc; + }; + const driver = { + name: 'translating-mongodb-double', + version: '0.0.0', + supports: {}, + async connect() {}, + async disconnect() {}, + async checkHealth() { return true; }, + async execute() { return null; }, + async find(_o: string, ast: { where?: unknown }) { translate(ast); return []; }, + async findOne(_o: string, ast: { where?: unknown }) { translate(ast); return null; }, + async count(_o: string, ast: { where?: unknown }) { translate(ast); return 0; }, + async aggregate(_o: string, ast: { where?: unknown }) { translate(ast); return []; }, + async create(_o: string, data: Record) { return { ...data }; }, + async update() { return null; }, + async updateMany(_o: string, ast: { where?: unknown }) { translate(ast); return 0; }, + async delete() { return true; }, + async deleteMany(_o: string, ast: { where?: unknown }) { translate(ast); return 0; }, + async bulkCreate(_o: string, rows: Record[]) { return rows; }, + async beginTransaction() { return { commit: async () => {}, rollback: async () => {} }; }, + async commit() {}, + async rollback() {}, + }; + return { driver, translated }; +} + +const refusalOf = async (p: Promise): Promise => + p.then(() => null, (e: unknown) => e as WireBearingError); + +describe('[#19757] the shared face refuses an array in the equality slot BEFORE translateFilter', () => { + describe('the face a direct caller composes: parseFilterAST, then translateFilter', () => { + const composed = (where: unknown): { translated: boolean; error: WireBearingError | null } => { + let translated = false; + try { + const lowered = parseFilterAST(where); + translated = true; + translateFilter(lowered); + return { translated, error: null }; + } catch (e) { + return { translated, error: e as WireBearingError }; + } + }; + + it.each([ + ['the FilterArray sugar on "equals"', [['tags', 'equals', ['a']]], 'where.tags'], + ['the FilterArray sugar on "="', [['tags', '=', ['a', 'b']]], 'where.tags'], + ['the implicit object form', { tags: ['a'] }, 'where.tags'], + ['an EMPTY array', { tags: [] }, 'where.tags'], + ['explicit $eq', { tags: { $eq: ['a'] } }, 'where.tags.$eq'], + ['nested under $or', { $or: [{ stage: 'won' }, { tags: ['a'] }] }, 'where.$or[1].tags'], + ])('%s — refused with INVALID_FILTER / 400, translateFilter never reached', (_label, where, path) => { + const { translated, error } = composed(where); + expect(translated).toBe(false); + expect(error?.code).toBe('INVALID_FILTER'); + expect(error?.status).toBe(400); + expect(error?.message).toContain(`at ${path}.`); + expect(error?.message).toContain('{"$in": […]}'); + expect(error?.message).toContain('{"$contains": "…"}'); + }); + + it('LIT CONTROL — a scalar, $in and $eq: null still reach translateFilter and translate as before', () => { + expect(translateFilter(parseFilterAST([['tags', 'equals', 'a']]))).toEqual({ tags: 'a' }); + expect(translateFilter(parseFilterAST([['tags', 'in', ['a', 'b']]]))).toEqual({ tags: { $in: ['a', 'b'] } }); + expect(translateFilter(parseFilterAST({ tags: { $eq: null } }))).toEqual({ tags: { $eq: null } }); + }); + }); + + describe('both engine doors, with this driver\'s translator as the only read path', () => { + let engine: ObjectQL; + let translated: unknown[]; + + beforeEach(async () => { + const double = makeTranslatingDriver(); + translated = double.translated; + engine = new ObjectQL(); + engine.registerDriver(double.driver as never, true); + await engine.init(); + engine.registry.registerObject(deal as never, 'test'); + }); + + it.each([ + // Door 2 carrying the FilterArray sugar — lowered by `parseFilterAST`. + ['Door 2, the FilterArray form', () => engine.find('deal', { where: [['tags', 'equals', ['a']]] } as never)], + // The OBJECT form: what Door 1 (the protocol face) hands the engine after + // its own lowering, and what a direct object-form engine call carries. + ['the object form (Door 1\'s hand-off)', () => engine.find('deal', { where: { tags: ['a'] } } as never)], + ['the object form under $eq', () => engine.find('deal', { where: { tags: { $eq: ['a'] } } } as never)], + ['count, nested under $or', () => engine.count('deal', { where: { $or: [{ tags: ['a'] }] } } as never)], + ])('%s — refused at the engine seam, translateFilter called ZERO times', async (_label, run) => { + const err = await refusalOf(run()); + expect(err?.code).toBe('INVALID_FILTER'); + expect(err?.status).toBe(400); + expect(err?.message).toMatch(/^(find|count)\('deal'\): /); + expect(translated).toHaveLength(0); + }); + + it('LIT CONTROL — the same doors hand a scalar and an $in list to translateFilter', async () => { + await engine.find('deal', { where: [['tags', 'equals', 'a']] } as never); + await engine.find('deal', { where: { tags: { $in: ['a'] } } } as never); + expect(translated).toEqual([{ tags: 'a' }, { tags: { $in: ['a'] } }]); + }); + }); + + it('REVERSE DIRECTION — handed the shape directly, translateFilter still emits it unchanged', () => { + // Kept visible on purpose, as `mongodb-comparand-type-conformance.test.ts` + // keeps the raw silent-edit visible: this driver has NO gate of its own for + // the slot (⛔ no driver source edit, by the ruling), so the shared face is + // the only thing standing between an embedder's filter and MongoDB's array + // equality. If this ever throws, a driver-local copy of the rule landed — + // read that as a second implementation to reconcile, not as a pass. + expect(translateFilter({ tags: ['a'] })).toEqual({ tags: ['a'] }); + expect(translateFilter({ tags: { $eq: ['a'] } })).toEqual({ tags: { $eq: ['a'] } }); + }); +}); diff --git a/packages/metadata-core/src/engine-delete-dispatch.ts b/packages/metadata-core/src/engine-delete-dispatch.ts index 1ffa07034c7..c0e0a9ce0b4 100644 --- a/packages/metadata-core/src/engine-delete-dispatch.ts +++ b/packages/metadata-core/src/engine-delete-dispatch.ts @@ -292,7 +292,17 @@ export const ENGINE_DELETE_DISPATCH_CASES: readonly EngineDeleteDispatchCase[] = // accepted it, and what a running server answers 500 to. { what: 'predicate on a non-id column, no multi', options: { where: { rule_id: 'r1' } }, expect: 'reject' }, { what: '$in over ids, no multi (an operator object is NOT an id)', options: { where: { id: { $in: ['a', 'b'] } } }, expect: 'reject' }, - { what: 'array id, no multi', options: { where: { id: ['a', 'b'] } }, expect: 'reject' }, + // [#19757] An 'array id, no multi' row — `where: { id: ['a', 'b'] }` — sat + // here. Since the 2026-09-23 ruling the shared comparand-shape face + // (`@objectstack/spec/data`) refuses an ARRAY in the equality slot at the + // engine's lowering seam, which every verb crosses BEFORE this dispatch runs: + // the real engine now answers that input with the face's INVALID_FILTER / 400, + // so no dispatch verdict for it is observable any more, and a row claiming + // one would pin a branch the engine never reaches. Retired rather than + // re-spelled — the `$in` row above keeps the "a non-scalar is not an id" + // coverage, and `scalarDeleteId`'s own array pin stays. A double bound to this + // predicate still refuses an array id with the dispatch sentence: no double + // runs the face, for this shape or any other face refusal. { what: 'null id, no multi', options: { where: { id: null } }, expect: 'reject' }, // The two shapes objectstack#5747 was filed for: a fake pinned to // `assertEngineDeleteDispatch` ACCEPTED both until this case-set could diff --git a/packages/metadata-core/src/engine-update-dispatch.ts b/packages/metadata-core/src/engine-update-dispatch.ts index 2a10f964b06..ac8c2754c91 100644 --- a/packages/metadata-core/src/engine-update-dispatch.ts +++ b/packages/metadata-core/src/engine-update-dispatch.ts @@ -625,7 +625,12 @@ export const ENGINE_UPDATE_DISPATCH_CASES: readonly EngineUpdateDispatchCase[] = // inherits the refusal rather than each one re-deriving it. { what: 'a SCALAR data.id beside an $in where.id and multi:true — refused; the row SET and the declared bulk intent were BOTH silently dropped (#11230 reverses the remaining half of the #5748 pin)', data: { id: 'rec_1', title: 'x' }, options: { where: { id: { $in: ['a', 'b'] } }, multi: true }, expect: 'reject' }, { what: 'a SCALAR data.id beside an $in where.id, no multi — refused (#11230)', data: { id: 'rec_1', title: 'x' }, options: { where: { id: { $in: ['a', 'b'] } } }, expect: 'reject' }, - { what: 'a SCALAR data.id beside an ARRAY where.id — refused (#11230)', data: { id: 'rec_1', title: 'x' }, options: { where: { id: ['a', 'b'] } }, expect: 'reject' }, + // [#19757] A 'SCALAR data.id beside an ARRAY where.id' row sat here. The + // shared comparand-shape face now refuses an ARRAY in the equality slot at + // the engine's lowering seam, BEFORE this dispatch runs (ruled 2026-09-23), + // so the real engine answers it with the face's INVALID_FILTER / 400 and no + // #11230 verdict for it is observable. Retired, not re-spelled: the two `$in` + // rows above carry the #11230 "declared non-scalar where.id" refusal. { what: 'a SCALAR data.id beside a NULL where.id — refused (#11230)', data: { id: 'rec_1', title: 'x' }, options: { where: { id: null } }, expect: 'reject' }, // [#11230] The boundary that does NOT move: a FALSY scalar `where.id` IS a // scalar, so it is not this refusal's shape at all and keeps the #11142 @@ -653,7 +658,12 @@ export const ENGINE_UPDATE_DISPATCH_CASES: readonly EngineUpdateDispatchCase[] = // by hand tends to accept, and a running server answers 500 to. { what: 'predicate on a non-id column, no multi', data: { title: 'x' }, options: { where: { tenant: 't1' } }, expect: 'reject' }, { what: '$in over ids, no multi (an operator object is NOT an id)', data: { title: 'x' }, options: { where: { id: { $in: ['a', 'b'] } } }, expect: 'reject' }, - { what: 'array id, no multi', data: { title: 'x' }, options: { where: { id: ['a', 'b'] } }, expect: 'reject' }, + // [#19757] An 'array id, no multi' row (`where: { id: ['a', 'b'] }`) sat + // here — retired for the reason the #11230 block above gives: the shared + // face refuses that input before this dispatch runs. The `$in` row above + // keeps the "a non-scalar is not an id" coverage; `scalarUpdateId`'s own + // array pin stays, and a double bound to this predicate still refuses an + // array id with the dispatch sentence (no double runs the face). { what: 'null id, no multi', data: { title: 'x' }, options: { where: { id: null } }, expect: 'reject' }, { what: 'falsy scalar where.id (0), no multi', data: { title: 'x' }, options: { where: { id: 0 } }, expect: 'reject' }, { what: 'empty where, no multi', data: { title: 'x' }, options: { where: {} }, expect: 'reject' }, diff --git a/packages/spec/src/data/filter-comparand-shape.test.ts b/packages/spec/src/data/filter-comparand-shape.test.ts index b901bd281a4..9482bb5823c 100644 --- a/packages/spec/src/data/filter-comparand-shape.test.ts +++ b/packages/spec/src/data/filter-comparand-shape.test.ts @@ -21,7 +21,12 @@ import { describe, it, expect } from 'vitest'; import { StandardErrorCode } from '../api/errors.zod'; -import { parseFilterAST, RangeOperatorSchema, VALID_AST_OPERATORS } from './filter.zod'; +import { + FieldOperatorsSchema, + parseFilterAST, + RangeOperatorSchema, + VALID_AST_OPERATORS, +} from './filter.zod'; import { assertListComparandShapes } from './filter-comparand-shape'; type Refusal = Error & { code?: string; status?: number }; @@ -39,10 +44,17 @@ const refusalOf = (run: () => unknown): Refusal => { * Which `$` operator an authoring spelling lowers to, derived by LOWERING one * rather than by reading a table this file would then be a second copy of. * A two-element array is legal for all three list operators, so the probe never - * trips the door it is used to find. + * trips the door it is used to find. The EQUALITY spellings do refuse it, since + * the 2026-09-23 arm (#19757) — and they answer `undefined` either way: before + * that arm they lowered it to the implicit form, which carries no `$` key. */ const loweredOperatorOf = (op: string): string | undefined => { - const lowered = parseFilterAST([['probe', op, ['a', 'b']]]) as Record | undefined; + let lowered: Record | undefined; + try { + lowered = parseFilterAST([['probe', op, ['a', 'b']]]) as Record | undefined; + } catch { + return undefined; + } const spec = lowered?.probe; if (spec === null || typeof spec !== 'object' || Array.isArray(spec)) return undefined; return Object.keys(spec).find((key) => key.startsWith('$')); @@ -462,6 +474,159 @@ describe('the list-comparand shape door (#5869) runs inside parseFilterAST (#922 .toMatch(/^Filter comparand at where\.n\.\$gt is undefined/); }); + // ── the EQUALITY-slot arm, ruled 2026-09-23 (#19757) ─────────────────── + // + // Ruling 5793368540, letter 乙: an array in the implicit-equality slot is + // refused at this face, for every driver at once; ⛔ no alias, ⛔ no grace + // window. Before it, `{ tags: ['a'] }` passed this door and was refused by + // the SQL family and `driver-memory`, excluded every row on + // `@objectstack/formula`, and was ANSWERED by `driver-mongodb` as MongoDB's + // array equality — the pins below were each measured RED on the face as it + // stood (see the PR's firing control). + + it.each([ + ['the lowered array form, "equals"', [['tags', 'equals', ['a']]], 'where.tags'], + ['the lowered array form, "="', [['tags', '=', ['a', 'b']]], 'where.tags'], + ['the object passthrough — implicit', { tags: ['a'] }, 'where.tags'], + ['an EMPTY array — still an array in this slot', { tags: [] }, 'where.tags'], + ['the object passthrough — explicit $eq', { tags: { $eq: ['a'] } }, 'where.tags.$eq'], + ['explicit $eq with an EMPTY array', { tags: { $eq: [] } }, 'where.tags.$eq'], + ['nested under $and (lowered)', ['and', ['amount', '>', 5], ['tags', 'eq', ['a']]], 'where.$and[1].tags'], + ['nested under $or', { $or: [{ stage: 'won' }, { tags: ['a'] }] }, 'where.$or[1].tags'], + ['nested under $not', { $not: { tags: { $eq: ['a'] } } }, 'where.$not.tags.$eq'], + ])('refuses an ARRAY in the equality slot — %s', (_label, where, path) => { + const err = refusalOf(() => parseFilterAST(where)); + // ADR-0112 class 1, both halves: on the face as it stood, every one of + // these RETURNED — there was no error at all to carry either field. + expect(err.code).toBe(StandardErrorCode.enum.INVALID_FILTER); + expect(err.status).toBe(400); + expect(err.message).toContain(`at ${path}.`); + }); + + it('the implicit-equality refusal names the slot and prescribes $in and $contains by their spec spellings', () => { + const err = refusalOf(() => parseFilterAST([['tags', 'equals', ['a', 'b']]])); + // The leading sentence is driver-memory's own for this condition, kept + // verbatim (one condition, one wording across packages). + expect(err.message).toMatch( + /^The implicit-equality comparand on field "tags" requires a single comparable value, but received an array \(\["a","b"\]\) at where\.tags\./, + ); + // The ruling's prescription: the declared list operator and the + // array-containment operator, spec spelling AND authoring spelling. + expect(err.message).toContain('"one of these values" use {"$in": […]} (authoring: in)'); + expect(err.message).toContain( + '"the stored list holds a value" on a multi-value field, {"$contains": "…"} (authoring: contains)', + ); + expect(err.message).toContain('an $or of those for any-of'); + expect(err.message).toMatch(/The filter was NOT applied, .*UNFILTERED result set\.$/); + }); + + it('the $eq refusal names the operator it was written with, and the same two remedies', () => { + const err = refusalOf(() => parseFilterAST({ tags: { $eq: ['a'] } })); + expect(err.message).toMatch( + /^Operator "\$eq" on field "tags" requires a single comparable value, but received an array \(\["a"\]\) at where\.tags\.\$eq\./, + ); + expect(err.message).toContain('{"$in": […]} (authoring: in)'); + expect(err.message).toContain('{"$contains": "…"} (authoring: contains)'); + // A caller-supplied context keeps its prefix, as on every sibling arm. + expect(refusalOf(() => assertListComparandShapes({ tags: ['a'] }, "find('deal')")).message) + .toMatch(/^find\('deal'\): The implicit-equality comparand on field "tags"/); + }); + + it('every AST spelling that lowers to EQUALITY refuses an array — the vocabulary, read at source', () => { + // Derived by LOWERING a scalar rather than from a hand list: a spelling is + // an equality spelling when `[f, op, 'x']` lowers to the implicit form. + // (`in` / `between` refuse a scalar probe outright — they are list + // operators, so a throw here is a "no", not a failure.) + const equality = [...VALID_AST_OPERATORS].filter((op) => { + try { + const lowered = parseFilterAST([['probe', op, 'x']]) as Record | undefined; + return lowered?.probe === 'x'; + } catch { + return false; + } + }); + // Guards the loop from passing vacuously. + expect(equality.sort()).toEqual(['=', '==', 'eq', 'equals']); + for (const op of equality) { + const err = refusalOf(() => parseFilterAST([['tags', op, ['a']]])); + expect(err.code, op).toBe(StandardErrorCode.enum.INVALID_FILTER); + expect(err.status, op).toBe(400); + expect(err.message, op).toMatch(/^The implicit-equality comparand on field "tags"/); + } + }); + + it('the two prescribed operators are DECLARED — the refusal invents no spelling', () => { + // The message spells `$in` / `in` and `$contains` / `contains` by hand + // (`filter.zod.ts` imports the door, so deriving them would be a cycle). + // This is what keeps them the vocabulary's own: both are keys of the + // enforced operator schema, and each authoring spelling lowers to its `$` + // spelling through the one AST table. + const declared = Object.keys(FieldOperatorsSchema.shape); + expect(declared).toContain('$in'); + expect(declared).toContain('$contains'); + expect(loweredOperatorOf('in')).toBe('$in'); + expect(loweredOperatorOf('contains')).toBe('$contains'); + // …and the prescribed spellings actually lower and pass this door. + expect(parseFilterAST([['tags', 'in', ['a', 'b']]])).toEqual({ tags: { $in: ['a', 'b'] } }); + expect(parseFilterAST([['tags', 'contains', 'a']])).toEqual({ tags: { $contains: 'a' } }); + expect(parseFilterAST({ $or: [{ tags: { $contains: 'a' } }, { tags: { $contains: 'b' } }] })) + .toEqual({ $or: [{ tags: { $contains: 'a' } }, { tags: { $contains: 'b' } }] }); + }); + + it('LIT CONTROL — every scalar equality comparand keeps passing, null predicate first', () => { + // `{ f: null }` and `$eq: null` ARE the has-no-value predicate (#5332); + // the arm is array-shaped and nothing wider. + expect(parseFilterAST({ tags: null })).toEqual({ tags: null }); + expect(parseFilterAST({ tags: { $eq: null } })).toEqual({ tags: { $eq: null } }); + expect(parseFilterAST([['tags', 'equals', null]])).toEqual({ tags: null }); + expect(parseFilterAST({ tags: 'a' })).toEqual({ tags: 'a' }); + expect(parseFilterAST({ n: 0 })).toEqual({ n: 0 }); + expect(parseFilterAST({ on: false })).toEqual({ on: false }); + expect(parseFilterAST({ tags: { $eq: '' } })).toEqual({ tags: { $eq: '' } }); + const day = new Date('2026-07-01T00:00:00.000Z'); + expect(parseFilterAST({ at: day })).toEqual({ at: day }); + expect(parseFilterAST({ at: { $eq: day } })).toEqual({ at: { $eq: day } }); + // #7597's promotion: a `{ $field }` comparand on an equality spelling is + // lowered to `$eq` and is not an array — untouched. + expect(parseFilterAST([['amount', '=', { $field: 'budget' }]])) + .toEqual({ amount: { $eq: { $field: 'budget' } } }); + }); + + it('LIT CONTROL — every array-valued operator the vocabulary declares keeps its array', () => { + // Read off the enforced schema rather than listed: the operators whose + // declared comparand ACCEPTS an array. `$eq` / `$ne` are in that set only + // because both are `z.any()` there — `$eq` is this arm's subject and + // `$ne` is not judged at this door (see the todo below) — so the loop + // covers exactly the list operators, and a fourth array-valued operator + // added to the schema lands in it without an edit here. + const arrayValued = Object.keys(FieldOperatorsSchema.shape).filter((op) => + FieldOperatorsSchema.safeParse({ [op]: ['a', 'b'] }).success); + expect(arrayValued.sort()).toEqual(['$between', '$eq', '$in', '$ne', '$nin']); + for (const op of arrayValued.filter((o) => o !== '$eq' && o !== '$ne')) { + expect(parseFilterAST({ tags: { [op]: ['a', 'b'] } }), op).toEqual({ tags: { [op]: ['a', 'b'] } }); + } + // The empty lists stay the declared predicates they are. + expect(parseFilterAST({ tags: { $in: [] } })).toEqual({ tags: { $in: [] } }); + expect(parseFilterAST({ tags: { $nin: [] } })).toEqual({ tags: { $nin: [] } }); + // `$contains` is declared with a STRING comparand — the array-containment + // operator takes ONE member, which is why the refusal says `"…"`. + expect(FieldOperatorsSchema.safeParse({ $contains: ['a'] }).success).toBe(false); + }); + + it('an array INSIDE a no-$-key field spec is still not descended into', () => { + // The nested-relation / deep-equality boundary this door has always kept: + // the arm judges the field's own slot, never the inside of a nested + // condition it does not walk. + expect(parseFilterAST({ author: { tags: ['a'] } })).toEqual({ author: { tags: ['a'] } }); + }); + + it.todo( + '$ne carrying an ARRAY — equality\'s negation measured the same split (refused by the SQL family ' + + 'and driver-memory, answered by driver-mongodb) but the 2026-09-23 ruling names implicit and ' + + 'explicit equality only. Reported for its own ruling; ⛔ not pinned green here, because a green ' + + 'pin would read as a ruling nobody made', + ); + // ── the wording contract (#5346 / #5348), unchanged by the move ──────── it('names the operator, the field, what arrived, where, and the fix', () => { @@ -520,6 +685,14 @@ describe('the list-comparand shape door (#5869) runs inside parseFilterAST (#922 { close_date: { $lte: null } }, { close_date: { $gt: null } }, { close_date: { $lt: null } }, + // The 2026-09-23 equality-slot arm (#19757): two prescriptions plus the + // received list. The long list is the tallest this arm assembles — its + // preview is cut at the shared 60-char bound — so it is the row that + // proves the "NOT applied" tail survives the wire. + { close_date: ['a'] }, + { close_date: { $eq: ['a'] } }, + { close_date: ['aaaaaaaaaaaaaaaaaaaa', 'bbbbbbbbbbbbbbbbbbbbbbb', 'cccccccccccccccccccc'] }, + { $or: [{ close_date: [] }] }, ]) { const err = refusalOf(() => parseFilterAST(where, "find('deal')")); expect(err.message.length, JSON.stringify(where)).toBeLessThan(500); @@ -608,12 +781,12 @@ describe('the list-comparand shape door (#5869) runs inside parseFilterAST (#922 // comparand, not an operator bag — descending into one would invent a // contract no backend agrees with. expect(parseFilterAST({ author: { name: 'x' } })).toEqual({ author: { name: 'x' } }); - // A scalar operator carrying an array is answered per driver, not here. - expect(parseFilterAST({ tags: { $eq: ['a', 'b'] } })).toEqual({ tags: { $eq: ['a', 'b'] } }); - // An implicit-equality array comparand is passed through for the driver to - // answer; what it MEANS is not this door's ruling (see the comparand door's - // own list of the cases it deliberately does not rule). - expect(parseFilterAST({ tags: ['a', 'b'] })).toEqual({ tags: ['a', 'b'] }); + // ⚠️ Two rows used to sit HERE, pinning `{ tags: { $eq: ['a', 'b'] } }` and + // `{ tags: ['a', 'b'] }` as shapes this door passes through for the driver + // to answer. Both are INVERTED by the 2026-09-23 ruling (#19757) — they + // pinned exactly the open slot that ruling closes — and now live, refused, + // in the equality-slot section below. ⛔ Not re-spelled into some other + // passing shape: what they asserted is no longer true of this door. // An unknown `$` key at node level belongs to the by-name refusals // downstream, which carry the specific prescription. expect(parseFilterAST({ $wat: [{ stage: { $in: ['won'] } }] })) diff --git a/packages/spec/src/data/filter-comparand-shape.ts b/packages/spec/src/data/filter-comparand-shape.ts index d958b5e72a1..ef772a616c0 100644 --- a/packages/spec/src/data/filter-comparand-shape.ts +++ b/packages/spec/src/data/filter-comparand-shape.ts @@ -5,6 +5,12 @@ * operators — the one place that decides whether `$in` / `$nin` / `$between` * received a list at all, for every driver. * + * [#19757] And its mirror, one slot over: the one place that decides whether + * an EQUALITY comparand — implicit (`{ field: V }`) or `$eq` — received ONE + * value rather than a list, for every driver. Two arms, one door: "a list + * operator takes a list" and "an equality comparand is not one". The second + * is the 2026-09-23 ruling's, recorded in its own section below. + * * `FieldOperatorsSchema` (`./filter.zod.ts`) declares three operators whose * comparand is a LIST rather than a scalar: * @@ -64,6 +70,18 @@ * delegating wrapper that supplies the engine's `find('deal')` context prefix; * there is exactly one implementation of "a list operator takes a list". * + * That sentence is true of TWO slots, and each is named because it was once + * true of only one: + * + * - **The list-operator slot** (`$in` / `$nin` / `$between`) — a scalar where a + * list belongs, since #9228 moved the rule here. + * - **The equality slot** (implicit `{ field: V }` and `$eq`) — a list where one + * value belongs, since the 2026-09-23 ruling (#19757). Until then this face + * judged only the first slot, and the door stood open in the second: the + * SQL family and `driver-memory` refused the shape, `@objectstack/formula` + * excluded every row, and `driver-mongodb` alone ANSWERED it, as MongoDB's + * array equality. That is the arm's section below. + * * ## Both engine doors, because only one of them carries an array * * The engine calls this on the LOWERED condition on both of its branches. Door @@ -209,6 +227,62 @@ * prescription — so this check changes the verdict only for pairs this door * accepts today. * + * ## Refused BY RULING, 2026-09-23: an ARRAY in the EQUALITY slot (#19757) + * + * The ruling (letter 乙, record 5793368540): "an array in the implicit-equality + * slot is refused at the shared face, for every driver at once" — ⛔ no alias, + * ⛔ no grace window. {@link parseFilterAST} lowers + * `['tags', 'equals', ['a']]` (and `=` / `==` / `eq`) to the IMPLICIT form + * `{ tags: ['a'] }`, and before this arm that shape passed both shared doors + * and got one answer per backend, measured on the lowered node: + * + * | backend | `{ tags: ['a'] }` | + * |:--|:--| + * | SQL family (`driver-sql`; `driver-sqlite-wasm` / `driver-turso` built on it) | refused, 400 — top level only: nested in `$and` / `$or` / `$not` it reached SQLite and came back a 500 | + * | `driver-memory` | refused, 400, at every depth | + * | `@objectstack/formula` | no row, including a row storing exactly `['a']` | + * | `driver-mongodb` | ANSWERED — `translateFilter` emits it unchanged, and MongoDB's equality on an array operand selects a stored array EQUAL to `['a']` or HOLDING `['a']` as an element (mingo 7.2.4, the named proxy; a live mongod is NOT measured) | + * + * One stored filter, a 400 on most backends and a silent, differently-shaped + * row set on one — the silent direction on exactly one backend. Refusing here + * makes every one of those cells unreachable through a platform door. + * + * The scope is the EQUALITY slot, and it is spelled out because the next slot + * over looks the same: + * + * - **Implicit equality** — `{ field: [...] }`, the lowering of every `$eq` + * authoring spelling given an array, at any depth under `$and` / `$or` / + * `$not`. An EMPTY array is still an array: `{ tags: [] }` is refused too. + * - **`$eq`** — `{ field: { $eq: [...] } }`, the explicit spelling of the same + * comparison. + * - ⛔ **`$ne` is NOT judged here.** It is equality's negation, not equality, + * and the ruling names implicit and explicit equality. It measured the same + * split (refused on the SQL family and `driver-memory`, answered by + * `driver-mongodb`), which makes it its own card needing its own ruling — + * absorbing it silently here is the move this family exists to refuse. The + * other scalar operators (`$gt`, `$contains`, …) carrying an array are + * likewise not this ruling's. + * - **The list operators keep their lists**, `$in: []` / `$nin: []` included, + * and every scalar — `null` above all, since `{ field: null }` and + * `$eq: null` ARE the has-no-value predicate (#5332) — keeps passing. + * - **A field spec with no `$` key** (`{ author: { name: 'x' } }`) is still not + * descended into; an array sitting INSIDE one is its nested condition's + * business, not this slot's. + * + * The prescription names the two operators an author holding a list was + * reaching for, by their spec spellings, never an invented one: `$in` + * ("one of these values", authoring spelling `in`) and `$contains` + * ("the stored list holds this value" on a multi-value field — the MEMBERSHIP + * reading `FieldOperatorsSchema` declares for a `multiple: true` / JSON-stored + * column, authoring spelling `contains`). Both are hand-spelled for the import + * cycle {@link LIST_COMPARAND_OPERATORS} records, and + * `filter-comparand-shape.test.ts` reconciles them against `FieldOperatorsSchema` + * and the AST vocabulary. + * + * The leading sentence is `driver-memory`'s own for the same condition + * (`arrayComparandError`), kept verbatim so one condition keeps one wording + * across the platform (#5240's rule, applied across packages). + * * ## Refusal envelope * * Every refusal carries `code: 'INVALID_FILTER'` and `status: 400` (ADR-0112 @@ -220,6 +294,7 @@ * * @see https://github.com/objectstack-ai/objectstack/issues/5869 (the rule) * @see https://github.com/objectstack-ai/objectstack/issues/9228 (the move) + * @see https://github.com/objectstack-ai/objectstack/issues/19757 (the equality-slot arm) */ /** @@ -259,6 +334,15 @@ const ORDERING_COMPARAND_OPERATORS: ReadonlyMap = new ['$lte', ['<=', 'lte', 'less_than_or_equal', 'lessthanorequal', 'lessorequal']], ]); +/** + * [#19757] The ARRAY-CONTAINMENT operator the equality-slot refusal prescribes, + * with its authoring spelling: `$contains`, which `FieldOperatorsSchema` + * declares as a MEMBERSHIP test on a `multiple: true` / JSON-stored column + * ("the stored list holds this value"). The other half of the prescription is + * `$in`, read off {@link LIST_COMPARAND_OPERATORS}. Reconciled by pin. + */ +const ARRAY_CONTAINMENT_OPERATOR = { op: '$contains', spellings: ['contains'] } as const; + /** What a caller most likely meant when they wrote a scalar. */ const SCALAR_ALTERNATIVE: ReadonlyMap = new Map([ ['$in', '"=" ($eq)'], @@ -580,17 +664,61 @@ function nullOrderingComparandError( ); } +/** + * An ARRAY as an EQUALITY comparand — implicit (`{ field: [...] }`, `op` + * omitted) or `$eq` — refused BY RULING, 2026-09-23 (#19757); see the module + * note's fifth "Refused BY RULING" section. + * + * The leading sentence is `driver-memory`'s `arrayComparandError` verbatim, so + * the one condition keeps one wording across packages. What follows is this + * door's own: the two operators an author holding a list was reaching for, by + * their spec spellings and their authoring spellings, then the "NOT applied" + * sentence — front-loaded and inside the 500-char client bound (#5423) like + * every sibling. The corrected shapes are written WITHOUT the field wrapper and + * with `…` for the value: the field is already the subject of the first + * sentence, and the bound buys more as prescription than as a second and third + * echo of the field name or the received list. For the same reason the four + * equality spellings (`=`, `==`, `equals`, `eq`) are not listed: "the + * implicit-equality comparand" names the slot, and a received list of 60 + * characters leaves no room for them inside the bound. + */ +function arrayEqualityComparandError( + context: string | undefined, + field: string, + value: unknown, + path: string, + op?: '$eq', +): Error { + const position = op + ? `Operator "${op}" on field "${field}"` + : `The implicit-equality comparand on field "${field}"`; + const inSpellings = LIST_COMPARAND_OPERATORS.get('$in') ?? []; + return invalidFilterComparandError( + context, + `${position} requires a single comparable value, but received an array ` + + `(${shapePreview(value)}) at ${path}. For "one of these values" use {"$in": […]} ` + + `(authoring: ${inSpellings.join(', ')}); for "the stored list holds a value" on a ` + + `multi-value field, {"${ARRAY_CONTAINMENT_OPERATOR.op}": "…"} (authoring: ` + + `${ARRAY_CONTAINMENT_OPERATOR.spellings.join(', ')}), an $or of those for any-of. ` + + `The filter was NOT applied, and an unapplied filter would have returned the ` + + `UNFILTERED result set.`, + ); +} + /** * Walk one `FilterCondition` and refuse every list-shaped operator whose * comparand cannot be one — and, since the two null rulings (2026-08-31, - * 2026-09-01), the null comparand positions those rulings carved out. + * 2026-09-01), the null comparand positions those rulings carved out; and, + * since the 2026-09-23 ruling (#19757), every EQUALITY comparand (implicit or + * `$eq`) that IS a list. * * Read-only and allocation-free on the overwhelmingly common path (a filter * with no list operator walks its own keys and returns). Runs on every engine * read and write and inside every {@link parseFilterAST} call, so it stays a * walk rather than a schema parse — that cost is now the whole reason, and this * gate deliberately enforces only the three list declarations the drivers - * genuinely cannot agree on, plus the null carve-outs ruled onto the same door. + * genuinely cannot agree on, the equality slot's mirror of them, plus the null + * carve-outs ruled onto the same door. * * @param node the LOWERED `FilterCondition` — never the authoring array. * @param context optional caller prefix (`find('deal')`), the engine's #5346 @@ -644,13 +772,29 @@ function assertFieldListComparands( path: string, ): void { // A spec that is not a plain object is a comparand (implicit equality) and - // carries no operator to check. + // carries no operator to check — unless it is a LIST, which the equality + // slot does not take (2026-09-23 ruling, #19757). Empty included: `[]` is + // an array in this slot like any other, and only `$in: []` / `$nin: []` are + // declared predicates. + if (Array.isArray(spec)) { + throw arrayEqualityComparandError(context, field, spec, path); + } if (!isFilterNode(spec)) return; const keys = Object.keys(spec); // No `$` key at all → a deep-equality comparand or a nested-relation // condition. Not descended into; see the module note. if (!keys.some((key) => key.startsWith('$'))) return; for (const op of keys) { + // The explicit spelling of the same equality slot (#19757). Strictly + // `$eq`: `$ne` is equality's negation and not this ruling's — see the + // module note — and `null` / every scalar / a `{ $field }` reference are + // not arrays, so they pass exactly as before. + if (op === '$eq') { + if (Array.isArray(spec[op])) { + throw arrayEqualityComparandError(context, field, spec[op], `${path}.${op}`, '$eq'); + } + continue; + } // The ordering carve-out (2026-09-01 ruling, #14080). Strictly `null`: // `undefined` keeps the TYPE door's own sentence, and every other // comparand type in these slots is that door's question, not this one's. diff --git a/packages/spec/src/data/filter-comparand-type-conformance.ts b/packages/spec/src/data/filter-comparand-type-conformance.ts index 9bb71dfcad4..4f8e9d99f08 100644 --- a/packages/spec/src/data/filter-comparand-type-conformance.ts +++ b/packages/spec/src/data/filter-comparand-type-conformance.ts @@ -48,6 +48,19 @@ * (`$icontains: 42` is refused per-operator) are `FILTER_TEXT_CASES`'; storage * forms are `TEMPORAL_CASES`'. The same one-axis bar the sibling tables set. * + * [#19757] One row family sits on that line and lives HERE: an ARRAY as an + * EQUALITY comparand (implicit `{ field: [...] }`, or `$eq`). An array is + * none of the six accepted types, and the row is the direct sibling of "a + * plain object in a scalar slot" below — the same question, "is this ONE + * comparable value", one JS type over. It is ANSWERED by the comparand-SHAPE + * door (`assertListComparandShapes`, ruled 2026-09-23), which runs inside + * `parseFilterAST` before the type door does; a `door-refusal` row asks only + * that `parseFilterAST` refuse before any driver runs, which is the whole of + * what the ruling promises every driver, and this is the one table every + * driver suite already runs through that door. `driver-mongodb` was the backend + * that ANSWERED the shape; the nested row is the one `driver-sql` answered with + * a 500 rather than its 400. + * * @see https://github.com/objectstack-ai/objectstack/issues/7872 (the ruling) * @see https://github.com/objectstack-ai/objectstack/issues/7956 (the matrix) */ @@ -255,6 +268,32 @@ export const FILTER_COMPARAND_TYPE_CASES: readonly ComparandTypeCase[] = [ note: 'The SQL family already refused this ("cannot be bound"); the door makes the answer uniform ' + 'instead of deep-equality-on-two-drivers, refusal-on-three.', }, + { + name: 'an ARRAY in the implicit-equality slot is refused (#19757)', + filter: () => ({ label: ['alpha'] as unknown as string }), + verdict: 'door-refusal', + code: 'INVALID_FILTER', + mustMention: ['at where.label.', '{"$in": […]}', '{"$contains": "…"}'], + note: 'Ruled 2026-09-23: refused at the shared face for every driver at once. Before it, the SQL ' + + 'family and driver-memory refused it, formula excluded every row, and driver-mongodb ANSWERED ' + + 'it with MongoDB\'s array equality (a stored array equal to the list, or holding it as an element).', + }, + { + name: 'an ARRAY under $eq is refused (#19757)', + filter: () => ({ label: { $eq: ['alpha'] as unknown as string } }), + verdict: 'door-refusal', + code: 'INVALID_FILTER', + mustMention: ['Operator "$eq"', 'at where.label.$eq.', '{"$in": […]}'], + }, + { + name: 'an ARRAY in the equality slot is refused nested in $or too (#19757)', + filter: () => ({ $or: [{ qty: 100 }, { label: ['alpha'] as unknown as string }] }), + verdict: 'door-refusal', + code: 'INVALID_FILTER', + mustMention: ['at where.$or[1].label.'], + note: 'The nested form is the cell driver-sql answered with a 500 DATABASE_ERROR (SQLite could not ' + + 'bind the list) where its top-level twin got the 400 — the door now answers both alike.', + }, { name: 'a bigint beyond ±2^53 is refused — precision loss must not answer silently', filter: () => ({ qty: { $eq: (BigInt(2) ** BigInt(53) + BigInt(1)) as unknown as number } }), diff --git a/packages/spec/src/data/filter-comparand-type.ts b/packages/spec/src/data/filter-comparand-type.ts index fdb633a2a47..163674e7dc1 100644 --- a/packages/spec/src/data/filter-comparand-type.ts +++ b/packages/spec/src/data/filter-comparand-type.ts @@ -88,7 +88,11 @@ * `driver-memory` refuse it, each with its own message; `driver-mongodb` * hands it to MongoDB and inherits that engine's array semantics); the matrix * did not measure it and the ruling does not name it, so the door leaves it - * to the layers that already answer it. + * to the layers that already answer it. [#19757] For the EQUALITY slot — + * implicit and `$eq` — the layer that answers it is now the comparand-SHAPE + * door one file over (ruled 2026-09-23), which runs first inside + * `parseFilterAST` and at the engine seam, so no array reaches this walk + * there; `$ne` and the other scalar operators stay per driver. * - **An operator outside the declared vocabulary** (`$wat`, retired `$regex`): * the unknown-/retired-operator refusals downstream carry the specific * prescriptions (`RETIRED_FILTER_OPERATORS`), which a generic type refusal diff --git a/packages/spec/src/data/filter-field-reference-lowering.test.ts b/packages/spec/src/data/filter-field-reference-lowering.test.ts index 3dc9888ab3b..037f4101b1c 100644 --- a/packages/spec/src/data/filter-field-reference-lowering.test.ts +++ b/packages/spec/src/data/filter-field-reference-lowering.test.ts @@ -98,7 +98,22 @@ describe('[#7597] equality triples with a `{ $field }` comparand', () => { expect(parseFilterAST(['amount', '=', 5])).toEqual({ amount: 5 }); expect(parseFilterAST(['stage', 'equals', 'won'])).toEqual({ stage: 'won' }); expect(parseFilterAST(['stage', '=', null])).toEqual({ stage: null }); - expect(parseFilterAST(['stage', '=', ['a', 'b']])).toEqual({ stage: ['a', 'b'] }); + // ⚠️ An ARRAY literal used to be pinned here lowering to `{ stage: ['a', 'b'] }`. + // Since the 2026-09-23 ruling (#19757) the shared face refuses an array in + // the equality slot, so the lowering is no longer observable as a return + // value — re-judged rather than dropped, because what this row proved still + // holds: the array is NOT promoted to `$eq` the way a reference is. The + // refusal names the IMPLICIT slot (`where.stage`), never `$eq`. + let refusal: (Error & { code?: string; status?: number }) | undefined; + try { + parseFilterAST(['stage', '=', ['a', 'b']]); + } catch (e) { + refusal = e as Error & { code?: string; status?: number }; + } + expect(refusal?.code).toBe('INVALID_FILTER'); + expect(refusal?.status).toBe(400); + expect(refusal?.message).toMatch(/^The implicit-equality comparand on field "stage"/); + expect(refusal?.message).toContain('at where.stage.'); }); it('an object comparand that is NOT a field reference keeps implicit equality', () => { diff --git a/packages/spec/src/migrations/entries/semantic/18.filter-equality-array-comparand-refused.ts b/packages/spec/src/migrations/entries/semantic/18.filter-equality-array-comparand-refused.ts new file mode 100644 index 00000000000..d6f055a3634 --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.filter-equality-array-comparand-refused.ts @@ -0,0 +1,73 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +// The RUNTIME door's half of the question the sibling entry +// view-filter-rule-scalar-operator-array-refused answered at the view-rule +// schema door: that entry refuses an array on a scalar view operator when the +// rule is authored; this one refuses the lowered shape itself, at the shared +// comparand-shape face every query crosses, whichever vocabulary produced it. +// Recorded as its own entry because the surface is different (the $ dialect +// and the FilterArray sugar, not ViewFilterRule) and because its scope stops at +// the EQUALITY slot, where the view entry covers every scalar operator. +export const entry: SemanticMigration = { + id: 'filter-equality-array-comparand-refused', + // No backticks in `surface` — build-upgrade-guide renders it inside a code + // span already, and a nested backtick would close it. + surface: + 'data.FilterCondition — an ARRAY as an EQUALITY comparand, at the runtime filter doors (the ' + + 'shared comparand-shape face that parseFilterAST and the engine lowering seam both run): the ' + + 'implicit form { field: [...] } — which the FilterArray sugar ["field", "equals", [...]] ' + + 'lowers to, and likewise "=", "==" and "eq" — and the explicit form { field: { $eq: [...] } }, ' + + 'at any depth under $and / $or / $not, the empty array included', + replacement: + 'the operator the list was standing in for. "One of these values" is $in: ' + + '{ field: { $in: ["a", "b"] } } (authoring spelling "in"). "The stored multi-value field ' + + 'holds this value" is $contains with ONE member: { field: { $contains: "a" } } (authoring ' + + 'spelling "contains"), and an $or of those for any-of. A filter that meant a single value ' + + 'writes that value: { field: "a" }. The list operators ($in / $nin / $between) keep their ' + + 'arrays, empty lists included; every scalar equality comparand, null above all (the ' + + 'has-no-value predicate), is untouched; and $ne is NOT judged by this entry', + reason: + 'Maintainer ruling on #19757 (record 5793368540, batch 217 item 3, letter 乙, 「217 同意」): ' + + 'an array in the implicit-equality slot is refused at the shared face, for every driver at ' + + 'once — no alias, no grace window. The comparand-shape face declared that moving a rule ' + + 'to it 「closes that door for every driver at once」, and before this change it judged ' + + 'only the list-operator slot; the equality slot passed both shared doors and each backend ' + + 'answered it alone. Measured on the lowered node { tags: ["a"] } at this release, beside a ' + + 'scalar and an $in control. driver-sql on SQLite REFUSED it with INVALID_FILTER / 400 at ' + + 'the top level, and nested under $and / $or / $not answered 500 DATABASE_ERROR instead ' + + '(driver-turso and driver-sqlite-wasm are built on driver-sql and were not run separately). ' + + 'driver-memory REFUSED it with INVALID_FILTER / 400 at every depth. The formula matcher ' + + 'returned no row, including a row storing exactly ["a"]. driver-mongodb ANSWERED it: its ' + + 'translateFilter emits the array unchanged, and MongoDB equality on an array operand ' + + 'selects a stored array equal to ["a"] or holding ["a"] as an element — mingo 7.2.4, the ' + + 'named proxy, over ["a"], "a", ["a","b"], ["b","a"], [["a"],"x"], [["a"]], "b" and [] ' + + 'selected ["a"], [["a"],"x"] and [["a"]]. The service-analytics filter normalizer read the ' + + 'FilterArray form as MEMBERSHIP: ["stage", "=", ["won", "lost"]] charted as stage IN ' + + '(won, lost); that form now gets the refusal too, while its OBJECT form, which that ' + + 'normalizer does not route through the shared face, still reads as membership. A live mongod, MySQL, PostgreSQL and a live ' + + 'Turso server were NOT measured. So one stored filter was a 400 on most backends and a ' + + 'silent, differently-shaped row set on one. The shared face now refuses it with ' + + 'INVALID_FILTER / 400 before any driver runs, naming the field, the path and both remedies. ' + + 'The ruling records the hosted product as running on the SQL family, where the top-level ' + + 'shape was already a 400, so the population that can observe a change is self-hosted ' + + 'driver-mongodb, plus any filter nested under a combinator on the SQL family (a 500 ' + + 'becomes a 400). $ne carrying an ' + + 'array measured the same split and is deliberately left to its own ruling. Metadata AT ' + + 'REST is not rewritten and this entry adds no D2 conversion: an array on equality has no ' + + 'single honest value, and choosing between $in and $contains is the author\'s call, not ' + + 'the platform\'s. ADR-0049 / ADR-0087 / ADR-0112.', + acceptanceCriteria: + 'Grep stored filters, dataset and widget filters, flow node filters and code that builds ' + + 'a where for a field whose value is an array — { field: [...] }, { field: { $eq: [...] } }, ' + + 'or a FilterArray triple on =, ==, eq or equals carrying an array — then decide per filter ' + + 'what it meant: one of these values ($in), the stored list holds a value ($contains, an ' + + '$or of them for several), or one value. Each is refused at query time with INVALID_FILTER ' + + '/ 400 naming the field and the path, so a test suite that exercises the query finds ' + + 'every one. A dashboard or dataset filter written as the FilterArray sugar with an array on ' + + 'equality charted as membership through the analytics normalizer; $in is the spelling that ' + + 'charts the same rows. On driver-mongodb re-check what the query is supposed to return rather than ' + + 'assuming the old rows were right: the old answer was MongoDB array equality, which ' + + 'neither $in nor $contains reproduces.', +}; diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index 0d0cc134a12..8165cd2f14f 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -8699,6 +8699,75 @@ const step18: MigrationStep = { + 'authoring schema door (SET_MEMBER_DESCRIPTION); they are outside THIS entry\'s transition ' + 'and are worth sweeping in the same pass.', }, + // The RUNTIME door's half of the question the sibling entry + // view-filter-rule-scalar-operator-array-refused answered at the view-rule + // schema door: that entry refuses an array on a scalar view operator when the + // rule is authored; this one refuses the lowered shape itself, at the shared + // comparand-shape face every query crosses, whichever vocabulary produced it. + // Recorded as its own entry because the surface is different (the $ dialect + // and the FilterArray sugar, not ViewFilterRule) and because its scope stops at + // the EQUALITY slot, where the view entry covers every scalar operator. + { + id: 'filter-equality-array-comparand-refused', + // No backticks in `surface` — build-upgrade-guide renders it inside a code + // span already, and a nested backtick would close it. + surface: + 'data.FilterCondition — an ARRAY as an EQUALITY comparand, at the runtime filter doors (the ' + + 'shared comparand-shape face that parseFilterAST and the engine lowering seam both run): the ' + + 'implicit form { field: [...] } — which the FilterArray sugar ["field", "equals", [...]] ' + + 'lowers to, and likewise "=", "==" and "eq" — and the explicit form { field: { $eq: [...] } }, ' + + 'at any depth under $and / $or / $not, the empty array included', + replacement: + 'the operator the list was standing in for. "One of these values" is $in: ' + + '{ field: { $in: ["a", "b"] } } (authoring spelling "in"). "The stored multi-value field ' + + 'holds this value" is $contains with ONE member: { field: { $contains: "a" } } (authoring ' + + 'spelling "contains"), and an $or of those for any-of. A filter that meant a single value ' + + 'writes that value: { field: "a" }. The list operators ($in / $nin / $between) keep their ' + + 'arrays, empty lists included; every scalar equality comparand, null above all (the ' + + 'has-no-value predicate), is untouched; and $ne is NOT judged by this entry', + reason: + 'Maintainer ruling on #19757 (record 5793368540, batch 217 item 3, letter 乙, 「217 同意」): ' + + 'an array in the implicit-equality slot is refused at the shared face, for every driver at ' + + 'once — no alias, no grace window. The comparand-shape face declared that moving a rule ' + + 'to it 「closes that door for every driver at once」, and before this change it judged ' + + 'only the list-operator slot; the equality slot passed both shared doors and each backend ' + + 'answered it alone. Measured on the lowered node { tags: ["a"] } at this release, beside a ' + + 'scalar and an $in control. driver-sql on SQLite REFUSED it with INVALID_FILTER / 400 at ' + + 'the top level, and nested under $and / $or / $not answered 500 DATABASE_ERROR instead ' + + '(driver-turso and driver-sqlite-wasm are built on driver-sql and were not run separately). ' + + 'driver-memory REFUSED it with INVALID_FILTER / 400 at every depth. The formula matcher ' + + 'returned no row, including a row storing exactly ["a"]. driver-mongodb ANSWERED it: its ' + + 'translateFilter emits the array unchanged, and MongoDB equality on an array operand ' + + 'selects a stored array equal to ["a"] or holding ["a"] as an element — mingo 7.2.4, the ' + + 'named proxy, over ["a"], "a", ["a","b"], ["b","a"], [["a"],"x"], [["a"]], "b" and [] ' + + 'selected ["a"], [["a"],"x"] and [["a"]]. The service-analytics filter normalizer read the ' + + 'FilterArray form as MEMBERSHIP: ["stage", "=", ["won", "lost"]] charted as stage IN ' + + '(won, lost); that form now gets the refusal too, while its OBJECT form, which that ' + + 'normalizer does not route through the shared face, still reads as membership. A live mongod, MySQL, PostgreSQL and a live ' + + 'Turso server were NOT measured. So one stored filter was a 400 on most backends and a ' + + 'silent, differently-shaped row set on one. The shared face now refuses it with ' + + 'INVALID_FILTER / 400 before any driver runs, naming the field, the path and both remedies. ' + + 'The ruling records the hosted product as running on the SQL family, where the top-level ' + + 'shape was already a 400, so the population that can observe a change is self-hosted ' + + 'driver-mongodb, plus any filter nested under a combinator on the SQL family (a 500 ' + + 'becomes a 400). $ne carrying an ' + + 'array measured the same split and is deliberately left to its own ruling. Metadata AT ' + + 'REST is not rewritten and this entry adds no D2 conversion: an array on equality has no ' + + 'single honest value, and choosing between $in and $contains is the author\'s call, not ' + + 'the platform\'s. ADR-0049 / ADR-0087 / ADR-0112.', + acceptanceCriteria: + 'Grep stored filters, dataset and widget filters, flow node filters and code that builds ' + + 'a where for a field whose value is an array — { field: [...] }, { field: { $eq: [...] } }, ' + + 'or a FilterArray triple on =, ==, eq or equals carrying an array — then decide per filter ' + + 'what it meant: one of these values ($in), the stored list holds a value ($contains, an ' + + '$or of them for several), or one value. Each is refused at query time with INVALID_FILTER ' + + '/ 400 naming the field and the path, so a test suite that exercises the query finds ' + + 'every one. A dashboard or dataset filter written as the FilterArray sugar with an array on ' + + 'equality charted as membership through the analytics normalizer; $in is the spelling that ' + + 'charts the same rows. On driver-mongodb re-check what the query is supposed to return rather than ' + + 'assuming the old rows were right: the old answer was MongoDB array equality, which ' + + 'neither $in nor $contains reproduces.', + }, // One entry for two doors on purpose: the two vocabularies spell one operator // and the rows being answered are one pair. Splitting it would put half the // prescription in front of an author who wrote the other spelling. diff --git a/scripts/doc-authoring-prose-id.baseline.json b/scripts/doc-authoring-prose-id.baseline.json index 2b904e13d03..5231ed50a58 100644 --- a/scripts/doc-authoring-prose-id.baseline.json +++ b/scripts/doc-authoring-prose-id.baseline.json @@ -341,7 +341,7 @@ "packages/metadata-core/src/engine-update-dispatch.ts": { "#11009": 6, "#11142": 5, - "#11230": 6, + "#11230": 5, "#5748": 4 }, "packages/metadata-core/src/object-schema-fls-contract.ts": {