feat(spec): declare the staged $empty filter operator and its per-type expansion (#20311)

.changeset/20311-empty-filter-operator-staged.md

@@ -0,0 +1,34 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec): declare the `$empty` filter operator — what 「is empty」 means, once, per field type — staged ahead of its executors (#20311)
+
+Clause-②: yes (widening) — a declared operator slot and five exports are added. `FieldOperatorsSchema.parse({ $empty: true })` used to strip the undeclared key and now keeps it. The one refusal that comes with the declared type sits on a key nothing writes (see below).
+
+**⚠️ Authoring `$empty` today is refused at query time.** The operator is declared but STAGED: it is deliberately absent from `FILTER_OPERATORS`, so no query executor answers it yet. A hand-written `{ "f": { "$empty": true } }` gets `INVALID_FILTER` / 400 from `driver-sql` (and the drivers that inherit its compiler), `driver-turso`'s remote transport, `driver-memory`, `driver-mongodb`, objectql `having` and the analytics `where` compiler; `READ_SCOPE_COMPILE_FAILED` / 500 (fail-closed) from the analytics read-scope SQL compiler; and `@objectstack/formula`'s write-side `matchesFilterCondition` answers `false` for every record, its fail-closed posture for an operator it has no arm for. Until each of those faces has its arm, write 「is empty」 with the view operator `is_empty`, which is unchanged.
+
+**What the operator means.** Its description is the ruled per-type table (ruling B on #20311, spelled as an operator by ruling A on #20399):
+
+| field type | `$empty: true` matches |
+|---|---|
+| text-like (`STRING_VALUE_TYPES`: text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) | null or `''` |
+| multi-value (`isMultiValueField`: multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with `multiple: true`) | null or `[]` |
+| every other type | null only |
+
+`$empty: false` is the exact complement. A face that holds no field declaration (the formula matcher, objectql `having`) judges by the value: null, `''` and `[]` are empty.
+
+**The one expansion every face calls**, exported from `@objectstack/spec/data`:
+
+- `expandEmptyOperator(field)` — keyed on the field DEFINITION (type plus `multiple`), because a `lookup` is `null_only` and a `lookup` with `multiple: true` is `multi_value`. Returns one of the frozen `EMPTY_OPERATOR_ARMS` rows: `{ arm, emptyString, emptyList }` (`EmptyOperatorArm`, `EmptyOperatorExpansion`).
+- `isEmptyFilterValue(value, expansion?)` — the value-level half: with an expansion, the declared row; without one, the by-value reading for the declaration-free faces.
+
+**What does not change.**
+
+- The `is_empty` / `is_not_empty` view operators still lower to `{ "$null": true | false }`. A later change flips that lowering to `$empty` once every face answers it; no stored filter changes result in this release.
+- An empty list is still refused as an equality comparand: `{ "tags": [] }` and `{ "tags": { "$eq": [] } }` keep their refusal. The multi-value row lives in the operator precisely because it cannot be spelled as a lowered equality.
+- `FILTER_OPERATORS` is unchanged, so every executor that derives its accepted set from it (`driver-memory`'s gate among them) keeps refusing `$empty` rather than dropping it.
+
+**One new refusal, on a key nothing writes.** A NON-boolean `$empty` (`"true"`, `1`, `null`) is refused where the declared boolean flags `$null` / `$exists` already are: at the operator slot, and at the save door (`FilterConditionSchema` and the analytics filter carriers that share its slot check), in the flags' own first sentence. `$empty` appears nowhere in this repository or in objectui's `main` before this change (0 occurrences in either).
+
+**Stored sharing rules** (ruling B's landing measurement): the criteria sharing rules in this repository's examples and objectui's fixtures that use 「is empty」 are 0, and this release changes no lowering, so none changes result. Production sharing rules are NOT MEASURED: they are unreadable from here.

content/docs/references/data/filter.mdx

@@ -120,6 +120,7 @@ const result = ComparisonOperatorSchema.parse(data);
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
 
 ### Nested Shape: `FieldOperators.$gt`
 
@@ -246,6 +247,7 @@ Type: `[FilterArray](#filterarray)[]`
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
 
 ### Nested Shape: `NormalizedFilter.$or[number][string]`
 
@@ -269,6 +271,7 @@ Type: `[FilterArray](#filterarray)[]`
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
 
 ### Nested Shape: `NormalizedFilter.$not[string]`
 
@@ -292,6 +295,7 @@ Type: `[FilterArray](#filterarray)[]`
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
 
 
 ---
@@ -338,6 +342,7 @@ Type: `[FilterArray](#filterarray)[]`
 | :--- | :--- | :--- | :--- |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
 
 
 ---

packages/spec/api-surface/data.json

@@ -232,11 +232,14 @@
     "DriverVocabularyEntry (interface)",
     "DroppedFieldsEvent (type)",
     "DroppedFieldsEventSchema (const)",
+    "EMPTY_OPERATOR_ARMS (const)",
     "ENGINE_UPDATE_UPSERT_REMOVED (const)",
     "ESignatureConfig (type)",
     "ESignatureConfigParsed (type)",
     "ESignatureConfigSchema (const)",
     "EffectiveApiMethods (interface)",
+    "EmptyOperatorArm (type)",
+    "EmptyOperatorExpansion (interface)",
     "EnableLike (interface)",
     "EngineAggregateOptions (type)",
     "EngineAggregateOptionsSchema (const)",
@@ -789,6 +792,7 @@
     "driverSupportsTransactions (function)",
     "effectiveOperationsArray (function)",
     "emptyGroupValueFor (function)",
+    "expandEmptyOperator (function)",
     "fieldForm (const)",
     "filterSubtreeProvenanceOf (function)",
     "foldAsciiCase (function)",
@@ -821,6 +825,7 @@
     "isCurrentUserDefaultToken (function)",
     "isDateMacroToken (function)",
     "isDateRangePresetName (function)",
+    "isEmptyFilterValue (function)",
     "isExpressionEnvelopeDefault (function)",
     "isFileIdToken (function)",
     "isFilterAST (function)",

packages/spec/authorable-surface/data.json

@@ -419,6 +419,7 @@
     "data/FieldMaskingKeep:keepTail",
     "data/FieldOperators:$between",
     "data/FieldOperators:$contains",
+    "data/FieldOperators:$empty",
     "data/FieldOperators:$endsWith",
     "data/FieldOperators:$eq",
     "data/FieldOperators:$exists",
@@ -948,6 +949,7 @@
     "data/ShardingConfig:shardingStrategy",
     "data/SortNode:field",
     "data/SortNode:order",
+    "data/SpecialOperator:$empty",
     "data/SpecialOperator:$exists",
     "data/SpecialOperator:$null",
     "data/SqliteConfig:autoMigrate",

packages/spec/export-origins/data.json

@@ -227,11 +227,14 @@
     "DriverVocabularyEntry": "src/data/driver/config-registry.zod.ts#DriverVocabularyEntry (interface)",
     "DroppedFieldsEvent": "src/data/data-engine.zod.ts#DroppedFieldsEvent (type)",
     "DroppedFieldsEventSchema": "src/data/data-engine.zod.ts#DroppedFieldsEventSchema (const)",
+    "EMPTY_OPERATOR_ARMS": "src/data/filter-empty-operator.ts#EMPTY_OPERATOR_ARMS (const)",
     "ENGINE_UPDATE_UPSERT_REMOVED": "src/data/data-engine.zod.ts#ENGINE_UPDATE_UPSERT_REMOVED (const)",
     "ESignatureConfig": "src/data/document.zod.ts#ESignatureConfig (type)",
     "ESignatureConfigParsed": "src/data/document.zod.ts#ESignatureConfigParsed (type)",
     "ESignatureConfigSchema": "src/data/document.zod.ts#ESignatureConfigSchema (const)",
     "EffectiveApiMethods": "src/data/api-derivation.ts#EffectiveApiMethods (interface)",
+    "EmptyOperatorArm": "src/data/filter-empty-operator.ts#EmptyOperatorArm (type)",
+    "EmptyOperatorExpansion": "src/data/filter-empty-operator.ts#EmptyOperatorExpansion (interface)",
     "EnableLike": "src/data/api-derivation.ts#EnableLike (interface)",
     "EngineAggregateOptions": "src/data/data-engine.zod.ts#EngineAggregateOptions (type)",
     "EngineAggregateOptionsSchema": "src/data/data-engine.zod.ts#EngineAggregateOptionsSchema (const)",
@@ -776,6 +779,7 @@
     "driverSupportsTransactions": "src/data/driver.zod.ts#driverSupportsTransactions (function)",
     "effectiveOperationsArray": "src/data/api-derivation.ts#effectiveOperationsArray (function)",
     "emptyGroupValueFor": "src/data/aggregation-policy.ts#emptyGroupValueFor (function)",
+    "expandEmptyOperator": "src/data/filter-empty-operator.ts#expandEmptyOperator (function)",
     "fieldForm": "src/data/field.form.ts#fieldForm (const)",
     "filterSubtreeProvenanceOf": "src/data/filter-subtree-provenance.ts#filterSubtreeProvenanceOf (function)",
     "foldAsciiCase": "src/data/filter.zod.ts#foldAsciiCase (function)",
@@ -808,6 +812,7 @@
     "isCurrentUserDefaultToken": "src/data/default-value-tokens.ts#isCurrentUserDefaultToken (function)",
     "isDateMacroToken": "src/data/date-macros.zod.ts#isDateMacroToken (function)",
     "isDateRangePresetName": "src/data/date-range-presets.ts#isDateRangePresetName (function)",
+    "isEmptyFilterValue": "src/data/filter-empty-operator.ts#isEmptyFilterValue (function)",
     "isExpressionEnvelopeDefault": "src/data/default-value-shape.ts#isExpressionEnvelopeDefault (function)",
     "isFileIdToken": "src/data/field-value.zod.ts#isFileIdToken (function)",
     "isFilterAST": "src/data/filter.zod.ts#isFilterAST (function)",

packages/spec/src/data/filter-comparand-type.test.ts

@@ -73,7 +73,7 @@ describe('the accepted set (#7872 ruling)', () => {
     const judgedScalar = [
       '$eq', '$ne', '$gt', '$gte', '$lt', '$lte',
       '$contains', '$notContains', '$startsWith', '$endsWith', '$icontains',
-      '$like', '$ilike', '$null', '$exists',
+      '$like', '$ilike', '$null', '$exists', '$empty',
     ];
     const judgedList = ['$in', '$nin', '$between'];
     expect([...judgedScalar, ...judgedList].sort()).toEqual(declared);

packages/spec/src/data/filter-comparand-type.ts

@@ -184,7 +184,7 @@ const SCALAR_COMPARAND_OPERATORS: ReadonlySet<string> = new Set([
   '$eq', '$ne', '$gt', '$gte', '$lt', '$lte',
   '$contains', '$notContains', '$startsWith', '$endsWith', '$icontains',
   '$like', '$ilike',
-  '$null', '$exists',
+  '$null', '$exists', '$empty',
 ]);
 
 const LIST_COMPARAND_OPERATORS: ReadonlySet<string> = new Set([