Repository navigation
fix(objectql)!: engine aggregate asks the field-type table for every row — min / max / avg over a refused type answer INVALID_FIELD / 400 on every driver - #21037
Conversation
…type table for every judged row, not only count_distinct engine.aggregate now looks up each aggregation's (function, declared field type) pair in AGGREGATE_FIELD_TYPE_COMPATIBILITY, with the isMultiValueField declaration half beside it, and refuses a pair the table refuses with INVALID_FIELD / 400 before any driver: max / min over a json or tags field answered a document in memory, a string on SQLite and a 500 on PostgreSQL. The door is count-distinct-json-stored-door.ts generalized and renamed (aggregate-field-type-door.ts); the count_distinct words are unchanged. The sum row is held back: the census found one authored sum over a refused type (a formula column summary in the published hotcrm stack), which goes back to triage. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…n no longer reaches, and pin the multi-valued lookup under a groupBy engine-nested-object-door.test.ts reached having through a max over a master_detail and over a json field; the aggregate door now refuses both pairs one door earlier, so those two cases become refusal pins at that door. The lookup with multiple: true under groupBy ['title'] is pinned beside the json / tags pins. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…ject/query on SQLite and live PostgreSQL max / min over a json, tags, multiple select and multiple lookup field, a max over a multiple lookup under groupBy title and an avg over a datetime answer 400 INVALID_FIELD before any read; the accepted pairs are served by the driver unchanged. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
… door as a reader of every row, and add the changeset The table's module TSDoc, and isAggregateCompatibleWithFieldType's, said the engine's aggregate door reads only the count_distinct row; it now asks the table for every row it judges. Comment-only: the table is unchanged. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…out spreading an optional array Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…gregate-door-whole-table
📓 Docs Drift CheckThis PR changes 2 package(s): 9 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 4 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 138 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin aac4ffd49c9ab27bcb4532a9e3a40cbe55228be4 && git checkout aac4ffd49c9ab27bcb4532a9e3a40cbe55228be4
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 2f2fa11d756f665a4c06160480c1dce15b9d67a4 9497f4c614e8939c3fad25bf25df08393a54c5af && git checkout -B drift-repro 2f2fa11d756f665a4c06160480c1dce15b9d67a4 && git merge --no-ff 9497f4c614e8939c3fad25bf25df08393a54c5af
node scripts/docs-audit/affected-docs.mjs --json 2f2fa11d756f665a4c06160480c1dce15b9d67a4
|
Contract reviewServed-tier: Inputs, and nothing else: card #20914 (body and its five comments 5916785741, 5922062362, 5922620655, 5923896529, 5923929098), PR #21037 (body, nine-file list, net diff against the merge-base ① Derived judgments
② Semver level
③ Boundary flagsDev deviations (os-dev-report 5923896529), each answered:
PR acceptance notes, each answered:
Reviewer's own flags:
Implemented-by: VERDICT: PASS |
…al limit (objectstack-ai#21085) Fixes objectstack-ai#21039 Clause-②: no ## What this changes Docs only, two hand-written pages. Both said the SQL drivers refuse `count_distinct` with `501 NOT_IMPLEMENTED` and that its SQL lowering is "scheduled". It has been lowered since objectstack-ai#6409; both callouts now state what `main` does. ## Before / after | page | before | after | |:--|:--|:--| | `content/docs/data-modeling/queries.mdx` (warn callout under the function table) | "`count_distinct` is not yet lowered by the SQL drivers ... refuse it as a capability gap, `501 NOT_IMPLEMENTED` ... its SQL lowering is scheduled" | Computed on every backend: SQL drivers (SqlDriver, both Turso transports) lower it to `COUNT(DISTINCT field)`; MongoDB, the in-memory driver and the engine's in-memory fallback count distinct non-null values; no `field` is `400 INVALID_QUERY`. The one real limit: a JSON-stored field (structured-JSON types, `multiselect` / `checkboxes` / `tags`, or `multiple: true`) is `400 INVALID_FIELD` at the engine before any driver runs. One sentence on the merged objectstack-ai#21037 effect (min / max / avg over a refused type answer the same `400 INVALID_FIELD`). | | `content/docs/protocol/objectql/query-syntax.mdx` (callout after "Schema enum", old lines 971-982) | "Only count, sum, avg, min, max are portable today. count_distinct ... not yet by the SQL drivers ... 501 ... Its SQL lowering is scheduled" | All six functions computed on every backend, same backend list, `median` still `400 INVALID_QUERY`, same JSON-stored limit. Callout type warn to info. This also removes the page's self-contradiction with its own line 118 (`COUNT(DISTINCT field)` on both SQL faces since objectstack-ai#6409). | ## Code anchors (origin/main a75311d) - SQL lowering table: `packages/drivers/driver-sql/src/sql-driver.ts:1524` (`['count_distinct', { sql: 'count', distinct: true }]`); emission `:10252-10254` (`count(distinct ??)`); fieldless refusal `:10222` calling `refuseDistinctAggregateWithoutField` (`:1827`, `INVALID_QUERY` / 400). - Turso remote transport: `packages/drivers/driver-turso/src/remote-transport.ts:677` and `:1408`. Local Turso `extends SqlDriver` (`turso-driver.ts:631`). - Memory driver: `packages/drivers/driver-memory/src/memory-driver.ts:2164` (null and undefined excluded). Engine in-memory fallback: `packages/objectql/src/in-memory-aggregation.ts:234`. MongoDB: `packages/drivers/driver-mongodb/src/mongodb-aggregation.ts:406`. - Engine refusal: `packages/objectql/src/aggregate-field-type-door.ts:236` (`assertAggregationFieldTypesAccepted`; `INVALID_FIELD` at `:261`, status 400 at `:262`), called at `packages/objectql/src/engine.ts:16891`. Table row: `packages/spec/src/data/aggregate-field-type-compatibility.ts:214` (`count_distinct: DISTINCT_COMPARABLE_FIELD_TYPES`; JSON-stored list `:190-196`). - Pins: `packages/objectql/src/engine-json-stored-group-distinct-door.test.ts:189`; `packages/rest/src/data-json-stored-group-distinct-door.test.ts:219` (read; its SQLite cell always runs, PostgreSQL / MySQL cells are skips without `OS_TEST_POSTGRES_URL` / `OS_TEST_MYSQL_URL`). - objectstack-ai#21037 at write time: MERGED (2026-10-01T04:51:07Z, head 9497f4c). Its `aggregate-field-type-door.ts` is on this base, so the one sentence about min / max / avg is true on main. - Not run locally: the pins above (docs-only change; the claims are read from the code and the pin files, not re-executed). ## Census of hand-written pages mentioning `count_distinct` - `data-modeling/queries.mdx`: stale, fixed here. - `protocol/objectql/query-syntax.mdx`: stale at the callout, fixed here; lines 104, 118, 134 and 1067 current (118 states the lowering correctly; 134 is the interface union; 1067 points to the aggregation). - `ai/natural-language-queries.mdx` (lines 18, 27): current. Lists `count_distinct` among the `aggregate_records` functions, which the tool declares (`packages/mcp/src/mcp-http-tools.ts:821`); no capability claim. - `api/data-api.mdx` (line 418): current. The `_count_distinct` measure suffix is the one the analytics service names (`packages/services/service-analytics/src/analytics-service.ts:2882`). - `deployment/validating-metadata.mdx` (lines 210-213): current. States exactly the table row: every type except the structured-JSON types and `multiselect` / `checkboxes` / `tags` (`aggregate-field-type-compatibility.ts:190-214`). - `kernel/contracts/data-engine.mdx` (lines 152, 498, 509): current. "every face computes" holds for the vocabulary on all backends above; the JSON-stored limit is a field-type refusal, not a face gap. - `ui/dashboards.mdx` (line 328): current. Function table row, no capability claim. - `content/docs/references/**` and `content/docs/releases/**`: not touched. ## Gates `node scripts/pm/dispatch-gates.mjs --commands`: 40 derived; `--ran` reports 40 run, 0 NOT-MEASURED, 0 UNRUN. Five gates (two in `@objectstack/lint`, `check:skill-examples`, `check:docs-transcript-drift`) first answered exit 3 PREREQUISITE NOT MET and `check:docs` exit 1 before the spec / lint / client-react packages were built; after the builds all exit 0. No changeset (docs only). --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01VDtqoecgES7ScQYGbFVDRv --------- Co-authored-by: Claude <noreply@anthropic.com>
…n reads the object its lookup field declares (objectstack-ai#20986) (objectstack-ai#21088) Fixes objectstack-ai#20986 Clause-②: yes (narrowing) ## What this changes A relationship-path hop the cube declares no join for now reads the object its lookup field DECLARES as its target, the field's `reference`. Before, such a hop fell back to its ALIAS, the lookup field's own name. An inferred cube declares no join, so a dotted path through a lookup named differently from its target (`owner` referencing a person object) named no object at all. The door admitted, and refused, the field's name as if it were an object, and the strategies joined and read a table of that name. One resolver, `service-analytics/src/hop-object.ts` (`resolvePathHops`), names a hop's object in three tiers, first answer wins: 1. the cube's declared join at that path, keyed by the path with its dots as `__`. An authored cube that declares its join keeps it; 2. the relationship field's declared `reference`, asked of the host's `relationshipResolver`. `AnalyticsServicePlugin` already wires it from the data engine's object schema, through `referenceCarrierOf`, for dataset compilation; 3. the alias itself, when the host cannot answer. Every reader takes its answer from there, with the same function: the door's field gate and its admitted and scoped set (`fieldsOfColumnSql`, `queryObjects`), and both strategies through the context's new `relationshipReference`, which the service sets to the very function its own gate uses. There is no second resolution. - `NativeSQLStrategy`: each registered join records the object it reads (`StatementJoins`), and the read scope applied to the alias is that recorded object's. The join table and the scoped object are one value. - `ObjectQLStrategy`: the cross-object plan and the FK-expand read (and its read scope) take the hop's object from the resolver. A lookup whose declared target is the base object itself, a self-reference such as `parent`, still plans as a cross-object hop. ## Per site (H2), file:line on base `8f784959cf` and on head `e75208968e` | site | base | role | head | |---|---|---|---| | `analytics-service.ts` `fieldsOfColumnSql` (via `namedQueryFields`) | `:428` `joins?.[alias]?.name`, else alias | the field gate, AND the admitted and scoped set (`queryObjects` `:1612`, `assertFieldsReadable` `:1682`) | `:432` `resolvePathHops`; callers `:1652`, `:1722` pass `this.hopReference` | | `analytics-service.ts` `cubeObjects` | `:1634` `j?.name ?? alias` | admits and scopes the DECLARED joins only | unchanged `:1674`; `?? alias` is unreachable, `CubeJoin.name` is required by the spec | | `native-sql-strategy.ts` `crossFieldComparisonIn` fallback | `:364` `cube.joins?.[alias]?.name ?? alias` | reads scopes for the decline, declared joins only, only for a context without the door's set | unchanged `:388`; the door always passes its set | | `native-sql-strategy.ts` `generateSql` read-scope loop | `:686` `cube.joins?.[alias]?.name ?? alias` | scopes each joined alias | `:751` `join.object`, the object the join recorded | | `native-sql-strategy.ts` `qualifyAndRegisterJoin` | `:848` `cube?.joins?.[alias]?.name ?? alias` | joins | `:904` `resolvePathHops` | | `native-sql-strategy.ts` `resolveStorageTarget` | `:1044` `cube.joins?.[joinAlias(relPath)]?.name ?? relPath` | reads (declared type, temporal coercion) | `:1104` `columnObjectOf` | | `native-sql-strategy.ts` `canHandle` external decline | `:169` declared join targets | reads (routing), declared joins only | unchanged `:193`; see Acceptance notes | | `objectql-strategy.ts` `isCrossObjectField` | `:724` `cube.joins?.[alias]?.name ?? alias` | reads (plans the FK-expand or the refusal) | `:736` `resolvePathHops` | | `objectql-strategy.ts` `planCrossObject` cross dimension | `:1054` `refObject: cube.joins?.[alias]?.name ?? alias` | reads and scopes (the FK-expand `executeAggregate` and its `getReadScope`) | `:1070` `resolvePathHops` | | `objectql-strategy.ts` `resolveStorageTarget` | `:1456` `... ?? relPath` | reads (declared type for the lowering and the echo) | `:1478` `columnObjectOf` | | `structured-json-dimension-door.ts` `columnOf` | `:192` declared join only, stands down otherwise | judges a grouped column's class | unchanged; outside this claim, see Acceptance notes | | `dataset-compiler.ts` | `:648` `relationshipResolver` per `include` | joins (compiled, declared) | unchanged: already tier 2's source, read into tier 1 | **H3, where the declared reference is reachable.** At the door, `AnalyticsServiceConfig.relationshipResolver`. `plugin.ts` answers it from `engine.getObject(base).fields[rel]` for a `lookup` or `master_detail` field, through `referenceCarrierOf`. At the strategies it was not reachable on base: `StrategyContext` carried a field's type and value shape (`declaredFieldType`, `declaredValueShape`), never its reference. It is now, as `DatasetScopedStrategyContext.relationshipReference`, which is package-internal. The context type is not exported. The member's one hit in the built `dist/index.d.ts` is prose in the doc comment of the private `hopReference` field, against 19 hits for `StrategyContext`. A host returning a `RelationshipTarget` is read by its `object`. **H4, the divergence check, both before and after.** On base, every site resolved the alias (the reads site resolved a dotted `relPath`), so the object admitted and the object joined were the same name. After, the six changed sites call `resolvePathHops` with the same function. The native join and the scope applied to it are one recorded value. The unchanged sites read declared joins only (tier 1), which the resolver returns verbatim. No site uses the alias where another uses the reference, so there is no `security` stop. Measured on the route pin: the security spy shows `canReadObject` asked only for the base object and the target. ## Per row (H1), measured in the shipped composition Fixture: real `SecurityPlugin` and `AnalyticsServicePlugin` over `ObjectQL` on SQLite. A member who may read everything except two objects. The "before" column was read at base `8f784959cf`. The decoy row's "before" was read on the ablation leg below, whose resolver is the base one. The "after" column was read at the fix commit `bf58903995`. The member is the caller. At head `e75208968e`, the two pins re-ran green over these rows: the readable dimension and filter, the unreadable target, the self-reference, and the controls. | strategy | lookup | target | position | before | after | |---|---|---|---|---|---| | native | named differently (`owner`) | readable | dimension | 403 naming `owner` (system caller: 500 `DATABASE_ERROR`) | rows, equal to a declared join's | | native | named differently | readable | filter member | 403 naming `owner` (system: 500) | rows, equal to a declared join's and to the nested form `{ owner: { region } }` | | native | named differently (`keeper`) | unreadable | dimension, filter | 403 naming `keeper` | 403 naming the target | | native | named after its target (control) | readable / unreadable | dimension | rows / 403 naming it | unchanged | | native | self-reference (`parent`) | the base object | dimension, filter | 403 naming `parent` | rows | | native | named after ANOTHER object (decoy) | unreadable | dimension | answered from the other object's table | 403 naming the target | | ObjectQL | named differently | readable | dimension | 403 naming `owner` (system: 500) | rows (FK-expand), equal to a declared join's | | ObjectQL | named differently | readable | filter member | 403 naming `owner` (system: 400) | 400 `INVALID_FIELD`, the strategy's own capability refusal, equal to a declared join's | | ObjectQL | named differently | unreadable | dimension, filter | 403 naming `keeper` | 403 naming the target | | ObjectQL | named after its target (control) | readable / unreadable | dimension | rows / 403 naming it | unchanged | | ObjectQL | self-reference | the base object | dimension | 403 naming `parent` | rows (FK-expand) | | ObjectQL | named after another object (decoy) | unreadable | dimension | answered from the other object | 403 naming the target | **`Clause-②: yes (narrowing)`, measured.** Widening: the first rows move from refused to served. Narrowing: the decoy rows move from answered to `403`. A lookup whose name is also the name of ANOTHER object used to read that other object's rows, by the ids of the records the field points to. The dispatch expected a plain `yes`; the measured grammar adds the arm. Line 2 reads `declared · yes · narrowing` through `scripts/pm/clause2-line.mjs`'s `readClause2Line`. The changeset is `minor`, carries the BREAKING banner, and carries its ADR-0087 marker (`not-required (no-migration-prescription)`). ## Pins, red first, and the ablation - **Unit pin** `service-analytics/src/__tests__/hop-object-reference-resolution.test.ts` (28 cases), on both strategies. It covers five positions: an inferred dimension, a filter member, a time-dimension window, an authored member over an undeclared relationship, and a two-hop path's second hop. For each, an unreadable target is refused by name before anything runs, on the read and on the echo. It also pins: - admission and the read scope ask one set, carrying the target and never the field's name; - the field gate judges the column on the target; - the native join and its scope name the target; - the native strategy asks the target's declared column type; - the ObjectQL FK-expand reads the target under its scope; - the self-reference on both strategies; - and two controls: a lookup named after its target, and a declared join is kept. - **Route pin** `packages/rest/src/analytics-hop-object-reference.test.ts` (10 cases), once per strategy. Every answer is compared with the same question through a DECLARED join and checked absolutely. It covers: - readable dimension: rows, with only the base and the target admitted (security spy); - readable filter: rows on native, equal to the nested form; ObjectQL's own 400; - unreadable dimension and filter: 403 naming the target, on the read and the echo; - the control. - **Red first.** Pins committed at `f31ca819e2`, before the fix at `bf58903995`: unit 24 failed / 4 passed (the 4 are the controls), route 8 failed / 2 passed (the controls). Every failure sits at the assertion naming the target, never at a reference assertion. - **Ablation A1, the resolver falls back to the alias again.** Predicted in writing before the run: exactly the red set above. - `scripts/ablation-replace.mjs` replaced tier 2's `const reference = referenceOf?.(from, field);` with `const reference = undefined as string | undefined;`: anchor 1 to 0, blob `489bc64f35f0` to `a783c0f62a1c`. The package was rebuilt, and `ablation-dist-preflight --absent` found the marker gone from all 6 built files. The pristine build carried it in `dist/index.js` and `dist/index.cjs`, 1 each. - Result: unit 24 failed / 4 passed, route 8 failed / 2 passed. The failing names are identical, as sorted lists, to the pins-first red set. - Restore: by absolute path, under an EXIT/INT/TERM trap. The blob equals the HEAD blob (`489bc64f35f0`), `git diff HEAD` is empty, and porcelain is 0. After a rebuild, the preflight found the marker present in 2 built files, and the tree clean. The pins then read unit 28/28, route 10/10. ## Tests and gates, head `e75208968e` Each command below ran under `os-verify-lock`, in ONE locked sequential script, on head `e75208968e`. The script printed `git rev-parse --short HEAD` at its start and its end, with a clean porcelain both times. Each exit code was captured before any pipe. - **Build:** `turbo run build --filter=!@objectstack/docs --concurrency=1`, 72 of 72 tasks, exit 0. - **Gate union:** `dispatch-gates --commands --repo objectstack-ai/objectstack` derives 62 commands from this diff's 8 paths, the same 62 as on the first merge. All 62 exit 0. - `dispatch-gates --ran`: `62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN`. - Verdict lines include `check:nul-bytes` (OK, 9671 files, no raw control bytes), `check:cross-package-test-inputs` ("29 package(s) read outside themselves, all declared"), `check:engine-double-contract`, `check:dual-build-cjs-loads`, `check:type-check-debt` (no entry above its count), `check-adr-0087-registration` (the `no-migration-prescription` exemption read from this changeset) and `check-empty-changeset`. - `check-changeset-no-major`'s clause-② axis reads the PR payload, so it is NOT APPLICABLE locally and is CI's. - **The four ⛔ roster families the derivation names, run anyway:** `check-changeset-fixed`, `check:authz-resolver`, `check:error-code-casing` and `check:filter-alias-parity`. All exit 0. - **`@objectstack/service-analytics`:** `vitest run --maxWorkers=2`: 155 files, 3526 passed, 10 skipped. `typecheck` exits 0, and `--listFiles` counts 152 of the 152 `__tests__` files, the unit pin among them, in the program. - **The rest analytics pins:** every `src/analytics-*.test.ts` plus `data-nested-relation-permission.test.ts`, 20 files: 243 passed, 8 skipped. `@objectstack/rest` `typecheck` exits 0, with `check:test-typecheck` OK; the route pin is in `tsconfig.test.json`'s program (`--listFiles`). - **The client census** `envelope-caller-census.test.ts`: 20 of 20. Neither pin calls the censused spelling: the pins call `service.query(`, never `analytics.query(`. - **ESLint, a proven narrowing.** - The population is the 7 changed `.ts` files (`git diff --name-only 2821e9f...HEAD`). Per eslint's own config, `isPathIgnored` is false for all 7, and `parserOptions.project` and `projectService` are unset for all 7. - `lintFiles` gives 7 results, 0 errors, 0 warnings. - With no type-aware lint, this diff cannot move an untouched file's verdict. - **On the first merge (`cff14ac80f`)**, the same union was 62 of 62 exit 0, and the four rosters were green. Analytics was 154 files / 3521 passed, and the rest pins 19 files / 242 passed. ## Surface Within the claim's regions: `fieldsOfColumnSql` and `queryObjects` in `analytics-service.ts`; the hop-object sites in both strategies; the new module; the pins; and the changeset. ⛔ Not touched: the native `execute`, `buildFilterClause`, `convertFilter`, `packages/objectql/src/**` or `packages/spec/src/**`. Four further lines carry the resolver to the strategies, outside the claim's listed regions: - `analytics-service.ts`: the `hopReference` class field, `baseCtx.relationshipReference`, the `assertFieldsReadable` call to `namedQueryFields`, and the `relationshipResolver` config doc; - `strategies/types.ts`: one member, `relationshipReference`, on the package-internal context type. No in-flight sibling PR touches them. `main` was merged twice, with no rebase: `5dbeb7d7b7` brought PR objectstack-ai#21036 (`convertFilter`), and `2821e9f15b` brought PR objectstack-ai#21040 (`execute`) and PR objectstack-ai#21037. Both merges were clean, and the branch delta against `main` stayed the same 8 files. ## Acceptance notes - **The ObjectQL strategy still refuses a dotted FILTER or time window through any relationship** with its own `400 INVALID_FIELD`, as it does through a declared join. The engine serves the nested form there. Lowering the dotted filter onto the nested form for the engine belongs to `convertFilter`, outside this claim. The dimension position is served (FK-expand). - **The grouped-column door** (`structured-json-dimension-door.ts` `columnOf`) judges declared joins only and stands down on a path the cube does not declare. A multi-value or structured-JSON column reached through an undeclared path is not judged there. That was already true for a lookup named after its target, and is now reachable for one named differently. NOT MEASURED. Outside this claim. - **The native external-object decline** (`canHandle`) reads declared joins only, so an undeclared hop to a federated object is not declined. That holds for same-named lookups on base, and now for differently named ones. NOT MEASURED: there is no federated fixture. - **The ObjectQL SQL echo for a self-reference** prints an unaliased self-join. It describes a statement that is not the FK-expand the query runs. NOT MEASURED by a pin; it affects the echo only. - **A null FK** buckets as `(restricted)` on the ObjectQL FK-expand and as null on native. This is pre-existing, and the same for a lookup named after its target. - **A hop no tier answers** (a field that declares no reference, or a host with no resolver) reads the alias, as before. For a multi-hop path that fallback is now spelled as the join alias (`a__b`) on the two reads sites, where it was the dotted path. Either spelling names no object. - **`relationshipResolver`'s warn** for a field whose `reference` is not a string is now reachable per query hop, not only at dataset compilation. NOT MEASURED. - **The dataset door** with a selection naming an undeclared, differently named path is NOT MEASURED. The route pin covers the cube read and the echo. - **Drivers.** SQLite only. The resolution is computed before any driver. - **CI is not awaited.** These lanes are CI's: the test shards, dogfood, temporal conformance, build core, the workspace type-check lanes, and the wide-population and roster families the derivation lists outside its total. --- _Generated by [Claude Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…refused type answers INVALID_FIELD / 400 on every driver (objectstack-ai#21103) Fixes objectstack-ai#20914 Clause-②: no (narrowing) The follow-up round of this card, after PR objectstack-ai#21037 landed as `a75311dd2` (`Part of`, the `sum` row held). It executes triage's answer 5924445782 (A), whose execution line reads: "`'sum'` leaves `ROWS_HELD_FOR_TRIAGE`, and the held pins flip. If the constant is then empty, delete it." With this PR the engine's aggregate door asks every row of `AGGREGATE_FIELD_TYPE_COMPATIBILITY`, so the card's direction (the whole table) is complete. ## What this changes - `packages/objectql/src/aggregate-field-type-door.ts`: `'sum'` leaves `ROWS_HELD_FOR_TRIAGE`. The set is then empty, so the constant is deleted with its one read. `assertAggregationFieldTypesAccepted` now judges all six rows (`count` refuses no type). ⛔ No second table and no per-pair exemption: the door still carries no accept list of its own. In the module header, the section "The row the census held back" becomes "Every row is judged — `sum` included" and records the release. It also gains the measured `sum` table, and "a held row" leaves the not-judged list. - The refusal words are the door's existing machinery (`FUNCTION_WORDS.sum`: "sums" / "does not sum"). The route is read off the table's `sum` row: `sum accepts a field of type number, currency, rating, slider, progress, summary, boolean or toggle: aggregate a field of one of those types, or count the rows with count.` It sits inside the 500 characters the REST door keeps. - `packages/spec/src/data/aggregate-field-type-compatibility.ts`, TSDoc only. Review 5924038187 ① 7 named three published sentences; each now states the rows the engine door judges: - the module header: "on every row: `count_distinct` since objectstack-ai#20808 …, and `sum`, `avg`, `min` and `max` since objectstack-ai#20914 (`count` refuses no type, so its row never refuses there)". The pointer to an objectql source file a spec consumer cannot see is gone. - the predicate's TSDoc: "on every row of the table". - the "does NOT do" section: "at query time, on every row too". - `radio` joins the multi-capable list in the JSON-stored paragraph (`MULTI_CAPABLE_TYPES` holds it). - A comment-stripped compare (the TypeScript printer with `removeComments`) of the file at base and head is byte-identical, sha256 `22d8df93…` on both sides: ⛔ the table and the predicate did not move. The new clause is in the built `dist/data/index.d.ts`, and "other rows since objectstack-ai#20914" occurs in 0 built `.d.ts`. - `.changeset/20914-release-sum-row.md`: - `@objectstack/objectql: minor` with a BREAKING banner and `Clause-②: no (narrowing)`. It carries the FROM → TO per type class, what to write instead, and who is affected. - Exactly one ADR-0087 marker: `not-required (already-registered dataset-measure-aggregate-field-type-refused)`. That registered id's prescription already covers `sum` / `avg` over every class the table refuses; the selecting id covers only `min` / `max`. - It names the earlier entry's `radio` omission (review ① 3) in its own text, and that entry's "Not judged yet: `sum`" paragraph as superseded. ⛔ The landed file is not edited. - `@objectstack/spec: patch` for the TSDoc. ## FROM → TO, measured on three drivers (H2) Through `engine.aggregate`, two rows, on the real `InMemoryDriver`, on `SqlDriver` over SQLite, and on a private PostgreSQL 16.14. Before = base `2821e9f15`; after = this branch's built `dist`. | `sum` over | before: memory / SQLite / PostgreSQL | after, all three | |:--|:--|:--| | `json` | `0` / `0` / **500** `function sum(json) does not exist` | 400 `INVALID_FIELD` | | `text` | `0` / `0` / **500** `function sum(text) does not exist` | 400 `INVALID_FIELD` | | single-value `select` | `0` / `0` / **500** `function sum(character varying) does not exist` | 400 `INVALID_FIELD` | | `formula` | `0` / 400 `INVALID_FIELD` (driver: no column) / 400 (driver: no column) | 400 `INVALID_FIELD`, the engine's words | | `tags` | `0` / `0` / **500** | 400 `INVALID_FIELD` | | `datetime` | `0` / `4052` (the years added) / **500** | 400 `INVALID_FIELD` | | `percent` (refused by the table: a rate does not add) | `30` / `30` / `30` | 400 `INVALID_FIELD` | | controls: `currency`, `number`, `boolean`; `max` `number`; `count` `json` | `30`, `3`, `1`, `2`, `2` on all three | unchanged | The `percent` row is the one whose old answer agreed across drivers. The table refuses it on the semantic ground its TSDoc names (`isIncoherentAggregate`), and it is in the changeset's FROM → TO with its route (`avg`). ## The census, re-run before the flip (H1) Query: every line carrying a `'sum'` / `"sum"` literal, plus the unquoted and backtick spellings. Each hit was resolved by hand to its object and the field's declared type, then asked of the table. - **`examples/**` at `2821e9f15`: 17 lines, 0 refused pairs.** They are dataset measures, roll-up `summaryOperations`, a cube measure, a `kpi` metric and an `ObjectChart` aggregate. Every operand is a `currency`, `number` or `summary` field (11 distinct object.field pairs). - **hotcrm (`objectstack-ai/hotcrm` cloned at its head `fb408a73`): 33 lines in `src`.** One is a comment (a `derived` op), which leaves 32 authored `sum` aggregations over 18 distinct object.field pairs. 31 are over `currency` or `number`. **1 refused pair:** `src/sales/views/forecast.view.ts:28`, `crm_forecast.expected_amount` (`Field.formula`), the list-column `summary: 'sum'`. That is the known hit: triage-answered and carried by objectstack-ai/hotcrm#1980, which has not changed it yet. **No other hit, so the flip stands.** Outside `src`: 2 lines in `test`, and 3 in docs and the changelog (prose). - The query is its own positive control: it finds the known hotcrm hit. ## Pins: the held `sum` pins flipped, ⛔ not deleted - `packages/objectql/src/engine-aggregate-field-type-door.test.ts`: - The CONTROL's two held cases (`sum meta`, `sum title`, which reached the driver while the row was held) move into a new refusal test. It covers `sum` over `json`, `text`, `select`, `formula`, `tags`, `percent`, `datetime` and a `select` with `multiple: true`, with envelope `INVALID_FIELD` / 400 / `httpStatus` 400, `field` / `fields` / `object` / `param`, the message head and the route, and no read. - The CONTROL gains `sum` over a `currency` and a `boolean`. - The GUARD that asserted "no verdict for `sum` × every type" now walks every row read off the table, so a row the door skipped would turn it red. It asserts the key set is the six functions, with a `sum` floor beside the others. - The fail-closed GUARD covers `sum` too. - `packages/rest/src/data-aggregate-field-type-door.test.ts`: a `sum` refusal cell over `json`, `text`, `select`, `tags` and `formula` at `POST /api/v1/data/:object/query` (SQLite always; PostgreSQL where `OS_TEST_POSTGRES_URL` is set). The CONTROL gains `sum` over a `currency` (60) and a `boolean` (2).⚠️ As before, no CI job provisions the PostgreSQL / MySQL cells; the PostgreSQL evidence is the local run below. - `packages/objectql/src/engine-json-stored-group-distinct-door.test.ts`, its GUARD clause: `sum` moves from "passes, held for triage" to the refused set beside `avg` / `min` / `max` over `json`, in the same envelope at the same position. `count` still passes. ## Ablation (from committed code, `1519ed6bd`) - **Mutate.** `node scripts/ablation-replace.mjs` (WRAP, trap-restored) put `sum` back in the held set: the table-vocabulary `continue` became `… || (fn === 'sum' ? 'ablation-20914-sum-A1' : '')) continue;`. Anchor 1 → 0, blob `9a2bc89178af` → `adcfb213d670`. objectql was rebuilt, and `ablation-dist-preflight.mjs` found the marker in 4 built files. - **Predicted:** red on the flipped `sum` pins only. - **Observed, objectql:** 3 failed / 28 passed — the new `sum` refusal test, the every-row GUARD (`sum × text: expected null`), and the group-distinct GUARD clause. Every other pin was green. - **Observed, REST:** 2 failed / 14 passed / 8 skipped (MySQL) — the `sum` cell on SQLite and on PostgreSQL. The controls and the `min` / `max` / `avg` / `count_distinct` cells were green. - **Restore.** Blob == HEAD (`9a2bc89178af`), `git diff HEAD` 0 bytes, rebuilt. `--absent` found the marker in 0 of 14 built files and the tree clean. The same pins were green again: objectql 31 / 31, REST 16 passed + 8 skipped. ## Tests (at `1519ed6bd`, the merge of `origin/main` `b3d7a7086`; all with a private PostgreSQL 16.14 where a cell reads it) - objectql full: 353 files, 6889 passed. `pnpm --filter @objectstack/objectql typecheck` exit 0; test layer 40 files / 234 errors / 65 pinned, held. - rest full (`OS_TEST_POSTGRES_URL` set): 259 files, 5124 passed / 52 skipped. typecheck exit 0; test layer 0 / 0 / 0. - metadata-protocol full: 196 files passed, 3 skipped; 2930 tests passed, 19 skipped. - service-analytics full: 154 files, 3498 passed / 10 skipped. - Every other test file in the workspace that mentions a `sum` aggregation and reaches the engine: runtime 4 files / 25 tests, plugin-security 2 / 297, platform-objects 2 / 56, mcp 1 / 17, dogfood 2 / 12. All passed. - Driver conformance before and after: 50 covered, 0 DEBT, 0 exempt; the two readings are identical. - Lint, narrowed and proven at `1519ed6bd`: - eslint `--no-inline-config --format json` over the 6 changed `.ts` files: 6 results, 0 errors, 0 warnings; - ESLint `isPathIgnored` is false for all 6; - `calculateConfigForFile` shows no `parserOptions.project` / `projectService` on any of them. No type-aware lint is on, so this diff cannot move a verdict on an untouched file. - Repo-wide `pnpm lint` is CI's. - `pnpm --filter @objectstack/spec check:generated`: all 15 generated artifacts up to date. ## Gates `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, run with no paths, derived 88 commands at `1519ed6bd`. All 88 were run, each exit code recorded before any pipe, and all exited 0. `--ran` reconciles: 88 derived, 88 run, 0 NOT-MEASURED, 0 UNRUN. Among them: `check:adr-0087-registration` (accepted, `not-required (already-registered)`), `check:changeset-no-major`, `check:empty-changeset`, `check:doc-authoring`, `check:nul-bytes`, `check:engine-double-contract` and `check:api-surface`. NOT MEASURED locally (CI-owned): the 5 path-scheduled CI jobs, the workspace type-check lanes beyond objectql and rest, the MySQL cells (no server here), and repo-wide `pnpm lint`. `origin/main` moved 13 commits after this merge. A local `git merge-tree` of the head with it is clean. One of them, `88b484e00`, touches `engine.ts` where metadata collections load (`picklists`), not at the aggregate call site. The merge queue rebuilds on current `main`. ## Deviations from the claim's file surface, each named - `packages/objectql/src/engine.ts`, **one comment** at the aggregate door's call site. "The `sum` row is held back by that card's census (see the door's header)" would have been false after this PR. It now says every row is asked, and adds the measured `sum` answer. No code line moved. - `radio` added to two comment lists of the multi-capable types: the door's DECLARATION paragraph, and the spec TSDoc's JSON-stored paragraph. This is the same omission review ① 3 named in the landed changeset. The door's own behaviour already held `radio` (it asks `isMultiValueField`). ## Acceptance notes - `sum` × `percent` answers with the door's generic reason ("accepts only the pairs every backend answers alike, and it refuses this one"), which is true but not the table's ground for that pair (a rate does not add). Its route lists the accepted types, not `avg`; the changeset carries the `avg` route. A rate-specific reason would be a words change on the door, so it was left out of this PR. Carrier: none. - The hotcrm exhibit (`crm_forecast.expected_amount`, `summary: 'sum'` on a `formula`) stays with objectstack-ai/hotcrm#1980. It is a client-side list footer and does not reach `engine.aggregate`, so this PR does not change what that view shows. --- _Generated by [Claude Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_ Co-authored-by: Claude <noreply@anthropic.com>
…on a declared JSON-stored field, in where's words (objectstack-ai#21097) Fixes objectstack-ai#21007 Clause-②: yes (widening) A per-aggregation `filter` now refuses a scalar comparison on a declared JSON-stored field (`$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$between`, `$in`, `$nin`, implicit equality) with `INVALID_FILTER` / 400, in the words `where` refuses the same filter in. It no longer counts rows the stored arrays cannot support. The operator set and the refusal text move from `driver-sql` to `@objectstack/core`, byte for byte, so both faces read one set and one sentence. **Clause-② has two halves.** It is `yes (widening)` because `@objectstack/core`'s root gains three exports (`JSON_COLUMN_INCOMPATIBLE_OPERATORS`, `jsonColumnOperatorRefusalText` and its return type `JsonColumnOperatorRefusalText`), and `applyInMemoryAggregation` gains an optional trailing `reportWithheld` parameter. It also narrows: `@objectstack/objectql` refuses queries it used to answer 200, at `engine.aggregate` and at the published `applyInMemoryAggregation` given a field map. The changeset therefore carries `minor` for objectql with a BREAKING banner and one ADR-0087 marker, `minor` for core, and `patch` for driver-sql, whose output is unchanged. The seat answer on the card (`## Seat answer — objectstack-ai#21007`, comment 5924546829) amended the claim to this surface and this `Clause-②`. ## What was wrong (measured before this change, `d1f8ce865`) `POST /api/v1/data/:object/query` on SQLite and a live PostgreSQL 16.14, over the card's six rows (`owners` is a `multiple: true` lookup, and `d1` and `d3` hold `u1`). Both dialects answered identically: | filter | `where` twin | per-aggregation `m` before | now | |:--|:--|:--|:--| | `owners $in ['u1','u9']` (the card) | 400 `INVALID_FILTER` | 0 | 400, same body | | `owners $nin ['u1','u9']` (the card) | 400 | 6, with `d1` and `d3` counted | 400, same body | | `owners $eq 'u1'` / `{ owners: 'u1' }` | 400 | 0 | 400 | | `owners $ne` / `$gt` / `$lte` / `$between` | 400 | 6 / 4 / 1 / 5 | 400 | | `tags $eq 'red'` | 400 | 1 (`['red']` loosely `==` `'red'`) | 400 | | `meta` (json) `$eq` / `$in` | 400 | 0 / 0 | 400 | | `owners $contains 'u1'` (the prescribed spelling) | 2 | 2 | 2 (unchanged) | | `title $in` / `$nin` / `$eq` (controls) | 2 / 4 / 1 | 2 / 4 / 1 | unchanged | ## What changed - **`@objectstack/core`:** a new `src/utils/json-column-operator-refusal.ts`, exported from the root beside `temporal-storage-form.js`. The `export *` publishes three names: `JSON_COLUMN_INCOMPATIBLE_OPERATORS` (driver-sql's 22 spellings, member for member), `jsonColumnOperatorRefusalText(field, op, bare)`, and its return type `JsonColumnOperatorRefusalText` (`{ message, diagnostic }`). These are the two strings `jsonColumnOperatorError` built, and nothing else. Each face keeps its own error constructor: driver-sql keeps its objectstack-ai#8220 provenance seam, and objectql keeps its ADR-0112 envelope. - **`driver-sql`** (`sql-driver.ts`): the module-private set and the two template strings are gone. `jsonColumnOperatorError` keeps its name and signature, and now calls the core builder (hunk at `:3298`). One import line (with its comment) sits at `:153`–`:156`, after the top import block. It is outside the declared `:3376`–`:3460` region on purpose, so as not to touch the `@objectstack/core` import block that objectstack-ai#20988 and objectstack-ai#20987 edit. `assertOperatorAppliesToColumn` (in objectstack-ai#20988's former region) is untouched: it reads the imported set under the same name. - **`objectql`** (`having-filter.ts`): `assertAggregationFilterIsEvaluable` gains `assertAggregationFilterSparesJsonStoredFields`. It runs once on the filter after the reference rule, against `declaredJsonStoredFields(declared.fields)`, and before any driver is asked for a row, so an empty table refuses too (objectstack-ai#20122's rule). It walks `$and` / `$or` / `$not` and refuses implicit equality (reported as `=`, bare, as driver-sql does) and every operator in the shared set, whatever the comparand (`null` and `[]` included). The withheld message is thrown, and the diagnostic, with the aggregation position, goes to `reportWithheld`, as objectstack-ai#20148 does. `$contains` / `$notContains` / `$exists` / `$null` / `$empty` keep answering. `checkCondition` carries the same refusal above its no-value exit, but only as a backstop for a row that reaches the arm. Its reach is the row's: an empty row set never gets there, and spec lowering rule 3 puts a `$null` arm ahead of every negation that the walker's `$or` short-circuit takes first. The gate is the complete door, and the docblocks now say so (round 2, from the review's ①.5). - **`objectql`** (`in-memory-aggregation.ts`, round 2, the review's F10 (b)): the published `applyInMemoryAggregation(rows, ast, timezone, fields, reportWithheld?)` now calls the same `assertAggregationFilterSparesJsonStoredFields` (exported from `having-filter.ts`, not from the package root) once per `aggregations[i].filter` when it is handed `fields`, before any row is judged. Before this, a direct caller reached only the backstop and got a row-dependent answer. This entry point holds no logger, so the closest seam is a new optional trailing `reportWithheld(diagnostic)`, which receives the field, operator and position; without it the diagnostic is dropped and the 400 is unchanged. `engine.aggregate` passes none, because it has already judged and logged the same filter. The gate's declaration parameter is narrowed to just the `fields` and `reportWithheld` members of `AggregationFilterDeclaration`, since `object` is the reference rule's. - **`objectql`** (`engine.ts`): one comment block and the `reportWithheld` log line at `assertAggregationFilterIsEvaluable`'s call site. The log line now reads "as it is for the same refusal in a where" instead of "…cross-field comparison…", since it carries two refusals now. ## Measured findings behind the shape (H1–H5) - **H1, the premise, holds.** `SET_MEMBER_DESCRIPTION`, the `$in` / `$nin` entries of `FILTER_OPERATORS` and the `$contains` docblock give no per-element reading. driver-sql's `where` refuses (it does not answer membership), so triage's "membership, as driver-sql does" misread it. - **H2, the set.** All ten operators, plus null and empty-list comparands, were answered with a wrong count; none was already refused. The bare infix spellings (`in`, `=`, `nin`) are refused earlier at both positions by the nested-relation door, so the evaluator only meets the `$` forms. - **H3, the home.** None existed; per the seat answer, the home is `@objectstack/core`. - **H4, where it fires.** The engine's in-memory lowering is the only evaluator of `aggregations[i].filter`: every driver's aggregate face refuses a per-aggregation filter 501, and the analytics ObjectQL strategy hands measure filters to `engine.aggregate`. - **H5, `having`.** After objectstack-ai#21037 (landed, merged here), `min` / `max` over a multi-valued field is refused `INVALID_FIELD` at the aggregate door. Measured on InMemoryDriver at the merged head: `max(owners)` with or without `having` gives 400 `INVALID_FIELD`. So `having` cannot meet a JSON-stored column, and it is left alone. ## Tests Round 1 numbers were read at `6e541b101`; round 2 numbers are marked with the head `f66bed950` (main merged). - `@objectstack/core` `json-column-operator-refusal.test.ts`: 6 passed. It pins the set member for member, and the message and three diagnostics by SHA-256 and length against what driver-sql printed at `8f784959c`. Hashes avoid a third literal copy of the sentence. - `driver-sql` `sql-driver-json-column-refusal-shared-text.test.ts`, run beside the existing JSON-column, compile-refusal-seam and provenance suites: 248 passed. It checks every `FILTER_OPERATORS` member against the shared set: the driver's thrown message and withheld diagnostic equal the core builder's output. - **Byte identity of the move.** A scratch capture through the built driver-sql (`SqlDriver` over SQLite) covered all 22 spellings plus bare equality, unmarked and author-marked, message and diagnostic, 46 entries. Before (`8f784959c`) and after: `cmp` identical, sha256 `dbcf32f5…534b2` on both. driver-sql's `dist` no longer contains the sentence. - `objectql` `engine-aggregate-filter-json-column-refusal.test.ts` (engine-level cell over the `find()` read shape): 74 passed. It covers 18 family cases on each of `owners`, `tags` and `meta` (`code`, `status`, the `$contains` / `$or` prescription, the field absent from the message, field and operator in the logged diagnostic, and the driver never asked for a row), an empty table (pure and grouped), the logged position, 11 answered cases (membership, null predicates, `title` controls) and the per-row floor. - `rest` `aggregation-filter-json-column-refusal.test.ts`: 52 per cell. SQLite passes and a live PostgreSQL 16.14 passes locally; MySQL is a named skip. Every family case asserts that the per-aggregation 400 body's `error` is **the same string** as its `where` twin's. objectstack-ai#21004's `aggregation-filter-array-membership.test.ts` still passes beside it. - **Full suites.** Read at merge `1a226419e`: objectql local 6948 passed, rest local 5072 passed / 247 skipped, core 1809 passed. Read before the first merge: driver-sql 3285 passed / 188 skipped. `typecheck` passed for core, driver-sql, objectql and rest. At head `6e541b101`: core, driver-sql (refusal suites), objectql `engine-aggregate*` (571 passed) and rest `aggregation-filter*` (150 passed with PostgreSQL) re-ran green. - **Ablation A: the engine gate call deleted** (`ablation-replace`, plus a rebuilt objectql `dist`, plus `ablation-dist-preflight --absent`): - objectql suite: 56 of 74 red. The 54 family cases, the empty table and the logged position failed; the 11 answered cases and the 7 floor cases stayed green. - rest suite: 92 of 104 red (46 per dialect). Populated `owners` / `tags` cases still got a 400 from the per-row floor, without the logged diagnostic. `meta` negations (`$ne`, `$nin`, `$nin []`, `$not $in`, where `meta` is null on every row) answered 200 `{ n: 6, m: 6 }`; the mechanism was not traced. The empty table answered 200. - Restored: blob equals HEAD, `git diff HEAD` empty, objectql rebuilt, preflight shows the marker present in 4 dist files with a clean tree, and both suites green again (74 and 104). - **Ablation B: the per-row operator floor replaced by a no-op** (src, engine suite): 5 red, the 4 operator floor cases and the no-value row; restored blob equals HEAD. - **Round 2: the direct-caller pins** (`engine-aggregate-filter-json-column-refusal.test.ts`, 92 passed). For `meta` (json) `$ne`, `$nin` and `$not $in`, each on four cells: an empty row set, an empty grouped row set, `meta` null in every row with the filter as spec `lowerFilterCondition` lowers it (the shape that carries the `$null` arm), and the same rows with the filter as written. Each must refuse 400 `INVALID_FILTER` with exactly `engine.aggregate`'s message, and hand the diagnostic (field, operator, `At aggregations[1].filter.…`) to `reportWithheld`. Also pinned: no reporter means the same refusal; no field map means nothing judged (`m: 0`, as before); and `$contains` still answers. - **Ablation C: the new `applyInMemoryAggregation` call deleted** (`ablation-replace`, anchor 1 to 0, blob `c65412761a90` to `f530761c0559`): 13 of 92 red. - The 9 empty, empty-grouped and lowered-null cells, plus the no-reporter case, answered instead of refusing. That is the backstop's 200. - The 3 as-written null-row cells were refused by the backstop but with no diagnostic reported. - Restored: blob equals HEAD `c65412761a90`, `git diff HEAD` empty, 92 passed again. - **Round 2 at `f66bed950`:** objectql local full suite 7008 passed (356 files), rest `aggregation-filter*` 150 passed / 61 skipped with a live PostgreSQL 16.14, core refusal pin 6 passed, driver-sql refusal pins 141 passed. - **Driver conformance ledger:** 50 covered cells, 0 in the DEBT ledger, 0 exempt, both before and after, in both rounds. ## Gates - `node scripts/pm/dispatch-gates.mjs --commands` (no paths) derived 70 families at `6e541b101`. - **68 ran, exit 0.** Among them: `check:adr-0087-registration`, `check:changeset-no-major`, `check:engine-double-contract`, `check:nul-bytes`, `check:doc-authoring`, `check:driver-conformance`, `check:driver-memory-census`, `check:query-options-erasure` and `check:test-source-alias`. - **2 NOT MEASURED (exit 3, prerequisite not met):** `check:dual-build-cjs-loads` and `check:type-check-debt`. Both need the whole workspace built; two attempts at that build timed out in the shared verify-lock queue. CI's lint job builds first. - `--ran` reconciliation: 70 derived, 68 run, 2 NOT MEASURED, 0 unrun. - **Round 2 at `f66bed950`:** re-derived with no paths, the same 70 families; 68 ran with exit 0 and the same 2 were NOT MEASURED (exit 3). `--ran`: 70 derived, 68 run, 2 NOT MEASURED, 0 unrun. `typecheck` passed for core and objectql. Narrowed lint: 10 changed `.ts` files, 0 errors, 0 warnings. - An earlier run caught one real finding, fixed in `e7bd7f667` ("type the shared-text pin's find options"): `check:query-options-erasure`'s test surface grew 236 to 237 because of an `as any` on a `find` options bag in the new driver-sql test. - **Lint, narrowed and declared:** `eslint --no-inline-config --format json` over the 9 changed `.ts` files reports 9 files, 0 errors, 0 warnings. `eslint.config.mjs` sets no `parserOptions.project` and registers no typed rule, so linting is not type-aware and this diff cannot move a verdict on an untouched file. The full `pnpm lint` is CI's. ## Acceptance notes - **Round 2, from the at-tier review (5926186339).** - F10 (b): `applyInMemoryAggregation` is gated (above). - F10 (a): the backstop docblocks are corrected. - F12 and ①.6: the export count is three, and the changeset's "Who is affected" names `applyInMemoryAggregation` direct callers and the new optional `reportWithheld`. - **The "flip the `m: 0` / `m: 6` pins" step had nothing to flip.** No suite on `main` pinned a per-aggregation `$in` / `$nin` count on a JSON-stored field (objectstack-ai#21004's two suites pin only `$contains` / `$notContains`). The full objectql, rest and driver-sql runs found no other pin that this change turns. The refusal pins are new files beside objectstack-ai#21004's. - **A `{ $field }` comparand on a JSON-stored field** (`{ owners: { $eq: { $field: 'title' } } }`) is refused by this gate in the JSON-column words. driver-sql's `where` refuses it through its cross-field class rule, in that rule's words. Both answers are `INVALID_FILTER` / 400 with the field withheld, so the two faces disagree only on which sentence they print. - **The REST envelope truncates the shared message at 500 characters** on both faces, so it ends "…because the answ…". That is unchanged here by direction, and filed separately by the seat. - **Findings for the seat, not filed here:** - driver-memory's `where` answers the family per element on a multi-valued field, while the SQL family refuses it (engine-level measurement). The seat files it. - service-analytics' native measure-filter compiler has no JSON-column gate (read at source, not measured). Carrier: objectstack-ai#20987. --- _Generated by [Claude Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…e table for every measure (objectstack-ai#21044) (objectstack-ai#21128) Fixes objectstack-ai#21044 Clause-②: no (narrowing) ## What changed A configured cube's `max` over a `text` column was served at the cube door (`POST /api/v1/analytics/query`) by the native-SQL strategy, with the column's text, while the same response's `fields[]` declared the measure `number`. The dataset door refuses that pair at compile by the spec table `AGGREGATE_FIELD_TYPE_COMPATIBILITY`, and the cube door consulted nothing. Triage's direction (`5924500811`) is carried out as ruled: the cube door asks the one table, and no second table exists. - **The judgment** (`packages/services/service-analytics/src/cube-measure-field-type-door.ts`, new, beside `measure-result-type.ts`): `assertCubeMeasureFieldTypesAccepted` refuses the first `measures` entry whose aggregate the table refuses for its column's declared type, `INVALID_FIELD` / 400 through `invalidMemberError`, with `member`, `param: 'measures'`, `cube`, `field` and `object` on the error. The verdict is `isAggregateCompatibleWithFieldType`'s; the accepted set the words name is read off the exported table. It judges every row of the table, as the dataset door does since decision batch objectstack-ai#127, with no scope condition on top of it. `count_distinct` keeps its own door (objectstack-ai#20912, `structured-json-dimension-door.ts`), which asks the same row plus the `multiple: true` declaration, so one pair has one verdict and one wording. - **Where it runs** (`analytics-service.ts`): `assertMeasureFieldTypes`, called in `ensureCube` on all three paths (inferred cube, augmented cube, declared cube), right after the objectstack-ai#20807 / objectstack-ai#20912 door and before the `where` gate. That is ahead of `callCtx` and strategy selection, so both strategies see it once and nothing is read before it answers. The same placement covers the dry run (`POST /api/v1/analytics/sql`) and every query `DatasetExecutor` runs through `queryIn`. - **The column description** (`analytics-service.ts`): `withMeasureResultTypes`, applied at the result seam in `queryIn` beside `withDeclaredMeasureFormats`, asks the dataset door's one rule, `measureResultType`, with the cube measure's aggregate and the declared type of the column it reads, and writes only what the rule answers. A `min` / `max` over `date` / `datetime` / `time` is now described `time`. ⛔ No copy of the rule in `buildFieldMeta`: both strategies are untouched. - **`measure-result-type.ts`**: TSDoc only, one paragraph naming the cube door as the rule's second reader. - **`.changeset/21044-cube-measure-field-type-table.md`**: `@objectstack/service-analytics` minor, BREAKING banner, `Clause-②: no (narrowing)`, ADR-0087 `not-required (already-registered dataset-measure-selecting-aggregate-field-type-refused, dataset-measure-aggregate-field-type-refused)`. `check:adr-0087-registration` accepts it. ### The four measured questions of the dispatch - **H1, the card's reading on `main`.** Confirmed through the real dispatcher route at base `2821e9f15b`, SQLite and PostgreSQL 16.13, both strategies (table below). The native face served every refused `min` / `max` pair with the column's text under `fields[]` `number`. Since PR objectstack-ai#21037 the ObjectQL face already answered `400 INVALID_FIELD`, from the engine's aggregate door, after the strategy had begun: the words name the engine's position (`aggregate('…'): aggregations[0].field takes the max of 'note', a declared text field…`), and the error carries no `member`. `sum` over the same text column answered `0` on SQLite on both faces and `500 DATABASE_ERROR` on PostgreSQL; `avg` answered `0` / `500` on the native face and `400` on the ObjectQL face. - **H2, where the door learns the source field type.** From the resolver this file already uses for measure columns: `declaredMemberEntry(cube, member, 'measure')` (the one `withDeclaredMeasureFormats` reads) gives the cube measure, its `sql` is the column when it is a bare identifier, and the type is `sourceFieldMeta(object, column).type`, the base-object declaration the objectstack-ai#20807 / objectstack-ai#20912 door and `compile()` already read. ⛔ No second resolution of a member to a field. A relationship-path column is not judged (the declaration read is the base object's), which is the dataset door's tier. - **H3, where the refusal goes and its code.** In `ensureCube`, as above. The code is `INVALID_FIELD` / 400, not the dataset door's `DATASET_INVALID` / 400, for the reason `dataset-refusal.ts`'s header gives: `DATASET_INVALID` is a verdict about a dataset document, and `/analytics/query` carries none; a verdict about one member the request named is the `INVALID_FIELD` family. It is also the code the cube door's three source-field gates and its objectstack-ai#20912 `count_distinct` door answer, and the code the engine's door already answered for this very pair on the ObjectQL face, so that face's wire code does not move. The dataset door keeps `DATASET_INVALID` at compile and never reaches this door for a pair it refuses. - **H4, `fields[]` for an accepted non-numeric pair.** By `measureResultType`, read at the cube door's result seam without widening the claimed surface: `min` / `max` over a temporal column is `time`; over a `boolean` column the rule declines (three readings disagree), so the producer's `number` stands, which is what SQLite (`1`) and the ObjectQL face on PostgreSQL (`1`) answer. Triage's second sentence, "`buildFieldMeta` stops minting `number` for a non-numeric `min` / `max`", is delivered at the seam both strategies' results leave through rather than inside `buildFieldMeta`, which has no field types to read: a refused pair never reaches a descriptor, a temporal one is re-described `time` by the one rule, and a boolean one keeps `number` by that rule's own verdict. ## Per-row readings, before and after | driver | strategy | measure | pair | before (`2821e9f15b`): status, value, `fields[]` type | after (`d85b700030`) | |:--|:--|:--|:--|:--|:--| | SQLite | native SQL | config `max_note` | `max` over note (text) | 200, `"y"`, `number` | 400 `INVALID_FIELD` | | SQLite | native SQL | config `min_note` | `min` over note (text) | 200, `"x"`, `number` | 400 `INVALID_FIELD` | | SQLite | native SQL | config `max_status` | `max` over status (select) | 200, `"won"`, `number` | 400 `INVALID_FIELD` | | SQLite | native SQL | config `max_opened` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | SQLite | native SQL | config `min_due` | `min` over due_on (date) | 200, `"2026-01-15"`, `number` | 200, `"2026-01-15"`, `time` | | SQLite | native SQL | config `max_flag` | `max` over flag (boolean) | 200, `1`, `number` | 200, `1`, `number` | | SQLite | native SQL | config `max_amount` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | SQLite | native SQL | config `sum_note` | `sum` over note (text) | 200, `0`, `number` | 400 `INVALID_FIELD` | | SQLite | native SQL | config `avg_note` | `avg` over note (text) | 200, `0`, `number` | 400 `INVALID_FIELD` | | SQLite | native SQL | config `cd_note` | `count_distinct` over note (text) control | 200, `2`, `number` | 200, `2`, `number` | | SQLite | native SQL | inferred `note_max` | `max` over note (text) | 200, `"y"`, `number` | 400 `INVALID_FIELD` | | SQLite | native SQL | inferred `amount_max` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | SQLite | native SQL | inferred `opened_at_max` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | SQLite | native SQL | augmented `note_max` | `max` over note (text) | 200, `"y"`, `number` | 400 `INVALID_FIELD` | | SQLite | ObjectQL | config `max_note` | `max` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | SQLite | ObjectQL | config `min_note` | `min` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | SQLite | ObjectQL | config `max_status` | `max` over status (select) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | SQLite | ObjectQL | config `max_opened` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | SQLite | ObjectQL | config `min_due` | `min` over due_on (date) | 200, `"2026-01-15"`, `number` | 200, `"2026-01-15"`, `time` | | SQLite | ObjectQL | config `max_flag` | `max` over flag (boolean) | 200, `1`, `number` | 200, `1`, `number` | | SQLite | ObjectQL | config `max_amount` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | SQLite | ObjectQL | config `sum_note` | `sum` over note (text) | 200, `0`, `number` | 400 `INVALID_FIELD` | | SQLite | ObjectQL | config `avg_note` | `avg` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | SQLite | ObjectQL | config `cd_note` | `count_distinct` over note (text) control | 200, `2`, `number` | 200, `2`, `number` | | SQLite | ObjectQL | inferred `note_max` | `max` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | SQLite | ObjectQL | inferred `amount_max` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | SQLite | ObjectQL | inferred `opened_at_max` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | SQLite | ObjectQL | augmented `note_max` | `max` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | native SQL | config `max_note` | `max` over note (text) | 200, `"y"`, `number` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | native SQL | config `min_note` | `min` over note (text) | 200, `"x"`, `number` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | native SQL | config `max_status` | `max` over status (select) | 200, `"won"`, `number` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | native SQL | config `max_opened` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | PostgreSQL 16.13 | native SQL | config `min_due` | `min` over due_on (date) | 200, `"2026-01-15"`, `number` | 200, `"2026-01-15"`, `time` | | PostgreSQL 16.13 | native SQL | config `max_flag` | `max` over flag (boolean) | 500 `DATABASE_ERROR` | 500 `DATABASE_ERROR` | | PostgreSQL 16.13 | native SQL | config `max_amount` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | PostgreSQL 16.13 | native SQL | config `sum_note` | `sum` over note (text) | 500 `DATABASE_ERROR` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | native SQL | config `avg_note` | `avg` over note (text) | 500 `DATABASE_ERROR` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | native SQL | config `cd_note` | `count_distinct` over note (text) control | 200, `2`, `number` | 200, `2`, `number` | | PostgreSQL 16.13 | native SQL | inferred `note_max` | `max` over note (text) | 200, `"y"`, `number` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | native SQL | inferred `amount_max` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | PostgreSQL 16.13 | native SQL | inferred `opened_at_max` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | PostgreSQL 16.13 | native SQL | augmented `note_max` | `max` over note (text) | 200, `"y"`, `number` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | ObjectQL | config `max_note` | `max` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | ObjectQL | config `min_note` | `min` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | ObjectQL | config `max_status` | `max` over status (select) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | ObjectQL | config `max_opened` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | PostgreSQL 16.13 | ObjectQL | config `min_due` | `min` over due_on (date) | 200, `"2026-01-15"`, `number` | 200, `"2026-01-15"`, `time` | | PostgreSQL 16.13 | ObjectQL | config `max_flag` | `max` over flag (boolean) | 200, `1`, `number` | 200, `1`, `number` | | PostgreSQL 16.13 | ObjectQL | config `max_amount` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | PostgreSQL 16.13 | ObjectQL | config `sum_note` | `sum` over note (text) | 500 `DATABASE_ERROR` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | ObjectQL | config `avg_note` | `avg` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | ObjectQL | config `cd_note` | `count_distinct` over note (text) control | 200, `2`, `number` | 200, `2`, `number` | | PostgreSQL 16.13 | ObjectQL | inferred `note_max` | `max` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | | PostgreSQL 16.13 | ObjectQL | inferred `amount_max` | `max` over amount (number) control | 200, `32`, `number` | 200, `32`, `number` | | PostgreSQL 16.13 | ObjectQL | inferred `opened_at_max` | `max` over opened_at (datetime) | 200, `"2026-03-04T05:06:07.000Z"`, `number` | 200, `"2026-03-04T05:06:07.000Z"`, `time` | | PostgreSQL 16.13 | ObjectQL | augmented `note_max` | `max` over note (text) | 400 `INVALID_FIELD` | 400 `INVALID_FIELD` | "Before" is base `2821e9f15b`; "after" is this branch's head `d85b700030` (two merges of `main` in, the second carrying objectstack-ai#21098's 401 for an anonymous analytics caller, so the probe signs its caller in). Both were read through the real `dispatcher-plugin` mount of `POST /api/v1/analytics/query`, over `AnalyticsServicePlugin` composed on a real `ObjectQL` engine and `SqlDriver`, by a scratch probe that was deleted after each run. Two rows of a ledger: `note` `x` / `y` (text), `status` `open` / `won` (select), two instants and two days, `flag` `true` / `false`, `amount` `10` / `32`. "Inferred" is an unregistered cube name (the object's), "augmented" a suffix-inferred measure on the configured cube. The same 56 readings were taken again after the first merge of `main` (`0abe2120fb`): no row differs from the head's. Since objectstack-ai#21103 landed (in the second merge) the engine's door also refuses `sum` over a refused type, so on the ObjectQL face the `sum` rows would answer `400` without this change too; this door answers first. ## Pins (red first), the ablation - **Red, on the tree committed as `be5c1a69b2`** (pins only, no fix; base `2821e9f15b`), SQLite and a private PostgreSQL 16.13: - `service-analytics` `src/__tests__/cube-measure-field-type-door.test.ts` (new) and the flipped `native-sql-measure-number-presentation.test.ts`: 16 failed, 16 passed, 1 skipped. Failures: `max_note must not be served: expected { rows: [ { max_note: 'y' } ], …(1) } to be undefined` (native), `max_note: expected undefined to be 'max_note'` (ObjectQL: the engine's refusal carries no `member`), `max_opened is described time: expected 'number' to be 'time'`, `expected the query to be refused, but it resolved` (dry run). - `runtime` `src/analytics-cube-measure-field-type-door.test.ts` (new): 6 failed, 6 passed. Native `max_note` answered `200`; both faces described `max_opened` `number`. The ObjectQL face's `400 INVALID_FIELD` and both faces' `number` controls were green already. - **Fixture triage.** `native-sql-measure-number-presentation.test.ts` (objectstack-ai#20889) read a configured cube's `max` over its `code` text column back as text, as the second half of its "keyed on the declared function, never on the value" control. That pin held exactly the served pair this card refuses, so it is flipped, not deleted: the case now asserts `INVALID_FIELD` / 400 with no statement run, keeps its text-dimension half, and the cube read above it no longer asks for `max_code`. - **Green, at `d85b700030`:** the same files 32 passed, 1 skipped (the PostgreSQL native `max` over `boolean` cell, a named skip: an accepted pair that is a 500 there, see Acceptance notes) and 12 passed. - **Ablation** (the cube door's table check removed), from committed code at `5d69394a60`. Predicted before running, in `progress.log`: service-analytics 12 red (per dialect: the native refusal, the ObjectQL refusal, both inferred-measure cases, the dry run and the flipped objectstack-ai#20889 case), 20 green, 1 skipped; runtime 2 red (the native refusal on both dialects), 10 green, because the engine's door still answers the ObjectQL face's `400 INVALID_FIELD` on the wire. - The mutation went through `scripts/ablation-replace.mjs` (WRAP mode, trap-restored, absolute path): `if (isAggregateCompatibleWithFieldType(aggregate, declared)) continue;` gained `|| String(aggregate) !== 'ABLATION-21044'`, so every pair passes. Anchor 1 to 0, marker 0 to 1, blob `6c7bdce48e43` to `f49be3058c84`. A second `trap` in the outer script restored by absolute path too. - `service-analytics` was rebuilt, and `ablation-dist-preflight.mjs` found the marker in 2 built files (`dist/index.cjs`, `dist/index.js`). - Observed: service-analytics 12 failed, 20 passed, 1 skipped; runtime 2 failed, 10 passed. Exactly as predicted. - Restore: blob after restore `6c7bdce48e43` equals HEAD, `git diff HEAD` empty, whole-tree porcelain 0 lines. Then rebuilt, and `ablation-dist-preflight.mjs --absent`: marker absent from all 6 built files and the tree clean. The pins are green again at both later heads. ## Tests (at `d85b700030`, after `pnpm install --frozen-lockfile` and a full `turbo run build`, 73 tasks) - `@objectstack/service-analytics`, full suite with `OS_TEST_POSTGRES_URL` set (no PostgreSQL cell skipped): 156 files, 3558 passed, 1 skipped (the named cell above). `typecheck` exit 0; `tsc --noEmit --listFiles` lists both touched test files. - `@objectstack/rest`, `src/analytics-*` and `rest-hook-refusal-message-parity`, with PostgreSQL: 20 files, 279 passed. - `@objectstack/runtime`, `src/analytics-*`, `src/dispatcher*`, `src/http-dispatcher*`, `src/domains/analytics*` and the other analytics-route suites: 51 files, 843 passed. `typecheck` exit 0, `check:test-typecheck` holds its ledger (27 files, 190 errors, 68 signatures; the new file adds none). - Not run locally, declared to CI: the whole `rest` and `runtime` suites, and `packages/qa/dogfood` (`Dogfood Regression Gate`). ## Gates (at `d85b700030`, as ONE sequential script under the shared verify lock, each exit code captured before any pipe) - **Derived families:** `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` names 62. `--ran` reconciles: `62 derived famil(ies) accounted for — 62 run, 0 NOT-MEASURED (a DERIVED zero — all 62 recorded an exit code and none of them is 3)`. All 62 exit 0. `check:dual-build-cjs-loads` and `check:type-check-debt` answered `PREREQUISITE NOT MET` (exit 3) on the first sweep at `0abe2120fb`, which had built only the dependency closure; after a full `turbo run build` (73 tasks) both exit 0, and both are 0 in the sweep at this head. - **The five roster families the derivation marks ⛔ (a roster in a directory this diff touches):** `node scripts/check-changeset-fixed.mjs`, `pnpm check:authz-resolver`, `pnpm check:error-code-casing`, `pnpm check:filter-alias-parity`, `pnpm check:route-ledger-census`. All exit 0. - `check:adr-0087-registration` reads the changeset as `[BREAKING+bang+clause-②-narrowing] not-required (already-registered)`. `check-changeset-no-major`: no `major` bump. `check:nul-bytes`: 9734 text files, no raw control bytes. ## ESLint, a declared narrowing (the repo-wide `pnpm lint` is CI's) - Touched files: `eslint --no-inline-config --format json` over the 6 touched `.ts` files at `0abe2120fb` (the two later commits are a merge of `main` and a test-only harness change of 7 lines). From the JSON: 6 files, 0 errors, 0 warnings, 0 ignored. - Population: `eslint.config.mjs`'s `files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']` minus `NEVER_LINTED`, which contains all 6 (none came back ignored). - Invariance: the config enables no type-aware linting (no `parserOptions.project` or `projectService` in any block), and this diff does not touch the config, so no untouched file's verdict can move. ## Beyond the claimed file surface - **The route pin lives in `packages/runtime/src/`, not beside `packages/rest/src/analytics-*.test.ts`.** The cube door, `POST /api/v1/analytics/query`, is mounted by `@objectstack/runtime`'s `dispatcher-plugin`; `RestServer` mounts only `/analytics/dataset/query`, so a pin in `rest` cannot reach this route. The new file is test-only and sits beside the sibling analytics route pins there. `rest`'s analytics suites were run as well (above). - **`sum` / `avg` are judged by the same door.** The claim priced the narrowing as `min` / `max`; the door asks the table for every row, as the dataset door has since decision batch objectstack-ai#127, because a `min` / `max`-only condition would be a second scope on top of the one table, the shape `dataset-compiler.ts` records retiring. Measured, those rows answered `0` on SQLite and `500` on PostgreSQL before (table above), and the changeset prices them. No shipped cube authors either. ## Acceptance notes - **NOT MEASURED: MySQL** (no server here). The door runs before any driver, so its verdict cannot depend on the dialect; the `time` description reads metadata only. - **NOT MEASURED locally: `packages/qa/dogfood`** (`Dogfood Regression Gate` is CI's). The one shipped cube, `examples/app-showcase`'s `showcase_delivery`, declares `count` over `*` and `sum` / `avg` over `estimate_hours` (`Field.number`): every pair accepted. Every other `min` / `max` / `sum` / `avg` in `examples/**` is a dataset measure, which the dataset door already judged. - **NOT MEASURED: the console.** A console widget that sends a suffix-inferred `FIELD_max` / `FIELD_sum` to `/analytics/query` over a refused field now gets `400 INVALID_FIELD`; the sibling repository was not read (dispatch order). - **Relationship-path measures are not judged here, measured at the head:** a configured measure `{ type: 'max', sql: 'account.name' }` over a related `text` field is still served on the native face (`200`, `"zeta"`, `fields[]` `number`, SQLite and PostgreSQL), and `{ type: 'max', sql: 'account.revenue' }` over a related `number` field answers the string `"250.000000000000000000000000000000"` on PostgreSQL's native face under `fields[]` `number`. The ObjectQL face refuses both as a cross-object measure (`400 INVALID_FIELD`). Judging them needs the declaration on the hop's object, which is the hop-object resolution this dispatch fences off (objectstack-ai#20986's sites); the door stands down on a dotted column rather than guess, the dataset door's tier. This is the second position the seat's comment `5924234751` names; reported to the seat, not fixed here. - **PostgreSQL's native face answers `500 DATABASE_ERROR` for `max` over a `boolean` column**, a pair the table ACCEPTS (`function max(boolean) does not exist`); SQLite and the ObjectQL face on PostgreSQL answer `1`. Unchanged by this PR (measured before and after); the boolean pin skips that one cell by name. Reported to the seat. - `main` advanced three commits after the second merge (`a11faeecb3`, `99398542b3`, `53ed3d1093`: objectql's aggregation-filter door, driver-mongodb and the showcase's security set). None touches `service-analytics`, the dispatcher or the spec table; CI and the merge queue read the merged generation. - The private PostgreSQL 16.13 cluster used for every live cell runs on `127.0.0.1` from `/tmp`; it is stopped and its directory removed with this delivery. --- _Generated by [Claude Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Part of #20914
Clause-②: no (narrowing)
#20914 remains open for one row: the census found an authored
sumover a type the table refuses, so thesumrow is held back here and goes back to triage (section "The census" below). Every other row of the table is judged by this PR.What this changes
engine.aggregatenow asksAGGREGATE_FIELD_TYPE_COMPATIBILITY(@objectstack/spec/data, throughisAggregateCompatibleWithFieldType) for EVERY aggregation that names a declared field, not only forcount_distinct, with theisMultiValueFielddeclaration half beside it. A pair the table refuses answersINVALID_FIELD/400before any driver is resolved. No second table: the door reads the table's rows and names the row's accepted set in its words, read off the table.count-distinct-json-stored-door.tsgeneralized and renamedaggregate-field-type-door.ts(assertAggregationFieldTypesAccepted), at the same call site inengine.ts, right after thegroupBydoor. The old name would have lied about what it judges.count_distinct's refusal words are byte-identical, so its [finding] two more JSON-stored columns as a group or distinct key answer 500 on PostgreSQL:groupByon amultiple: trueselect, andcount_distincton ajsonfield #20808 pins are unchanged (the one clause of its GUARD pin that said "no verdict for any other function" is flipped, see Pins).multiple: trueholds the same value as the multi-option types (a list in a JSON column), so it takes the verdict the row gives that class. Rows refusing any multi-option type (count_distinct,avg,min,max) refuse it;count, which accepts every type, accepts it.FieldType(a driver alias such asstring/integer), a fieldless aggregation or'*', and a function outside the table's vocabulary all get no verdict. Pinned.FROM → TO, measured on three drivers
Through
engine.aggregate(the door the REST query route, flows, hooks, roll-up recompute and the analytics ObjectQL strategy reach), two rows, realInMemoryDriver,SqlDriveron SQLite and a private PostgreSQL 16.14. Before =origin/maindfe5a0863; after = this branch's builtdistat9497f4c61.maxoverjson{"a":1}/"{\"b\":1}"(a string) / 500function max(json) does not existINVALID_FIELDminoverjson{"a":1}/"{\"a\":1}"/ 500INVALID_FIELDmax/minovertagsINVALID_FIELDmaxoverselectwithmultiple: true["a","b"]/"[\"a\"]"/ 500INVALID_FIELDavgoverdatetimenull/2026/ 500INVALID_FIELDavgovertagsnull/0/ 500INVALID_FIELDmaxovertext,minover singleselect,maxoveremail(string-class rows, as ruled)"y"/"y"/"y"INVALID_FIELDmaxnumber,mindatetime,avgpercent,maxboolean,sumnumber,count_distincttextsumoverjson/text/ singleselect(held row)0/0/ 500The words:
aggregate('OBJ'): aggregations[0].field takes the max of 'meta', a declared json field — a structured-JSON value, which the engine does not take the max of. The query was NOT run. max accepts a field of type number, currency, percent, rating, slider, progress, summary, date, datetime, time, boolean or toggle: aggregate a field of one of those types, or count the rows with count.followed by the reason. The route lands inside the 500 characters the REST door keeps. The thrown error carriescode,status,httpStatus,field,fields,object,param.The census (and the held
sumrow)Query: every line carrying an aggregate-function literal
'min' | 'max' | 'sum' | 'avg'(either quote), acrossexamples/**and the published hotcrm stack, each hit resolved by hand to its object and the field's declared type, then asked ofisAggregateCompatibleWithFieldType.examples/**atdfe5a0863(crm, todo, multi-package, showcase, embed-objectql): 21 lines (dataset measures, roll-upsummaryOperations, akpimetric, anObjectChartaggregate, two cube measures). 0min/max. Everysum/avgis overcurrency,number,progressorsummary: 0 refused pairs.objectstack-ai/hotcrmcloned at4ca8e2d4bb, source read, deps not installed): 41 lines insrc/appsplus 2 intest. 0min/max. 1 refused pair:src/sales/views/forecast.view.ts:28, the grouped list viewall_forecastsoncrm_forecastdeclaressummary: 'sum'onexpected_amount, aField.formula. The table refusessum×formula(a formula is virtual in SQL storage).packages/**, tests excluded): 0 refused pairs.packages/services/service-analytics/src/__tests__/aggregate-datetime-measure-refusal.test.tsit finds that file's authoredavgover adatetime.Per triage's direction ("A hit stops that row and goes back to triage"), the
sumrow is held:ROWS_HELD_FOR_TRIAGEin the door, named in its header. Evidence for triage: on the base,sumover aformulaalready answers 400INVALID_FIELDon SQLite and PostgreSQL (the SQL driver has no column) and0in memory; and at objectuibe0ad00a list view's columnsummaryis a client-side footer, while the server header query readsobject-grid.aggregations, so this hotcrm pair does not reachengine.aggregatethrough the console today. Releasing the row is deleting that one entry and flipping thesumpins.Other doors on the engine path (H2)
No other door enforces a row of the table. The
groupBydoor (group-by-structured-json-door.ts) judges a group KEY, not a function's operand, and stays beside this one; the number-comparand, temporal-comparand, text-operator and no-operator-object doors judge filter comparands. So there is one door asking the table, and one verdict per pair.Pins
packages/objectql/src/engine-aggregate-field-type-door.test.ts(new, recording driver, so the in-memory cell by construction):max/minoverjson,tags,address, amultiple: trueselect;maxover alookupwithmultiple: trueundergroupBy: ['title'](the shape a sibling card measured as PG 500 / SQLite serialized text / memory array);avgoverdatetimeandjson;maxovertext,minoverselect; first-offending-position across functions; scalar controls reach the driver;sumheld for every type; a GUARD that asks every judged row × everyFieldTypeagainst the table with per-row floors; the fail-closed tiers. Each refusal assertscode+status+httpStatusand the field, declaration, function and position in the message.packages/rest/src/data-aggregate-field-type-door.test.ts(new,POST /api/v1/data/:object/queryoverSqlDriver): SQLite always, live PostgreSQL whereOS_TEST_POSTGRES_URLis set, MySQL a named skip. No CI job sets those variables for this package; the local PostgreSQL 16.14 run is below.engine-json-stored-group-distinct-door.test.ts's GUARD clause "no verdict for any other function" now assertscountand the heldsumpass andavg/min/maxoverjsonare refused in the same envelope.engine-nested-object-door.test.tsreachedhavingthrough amaxover amaster_detailand ajsonfield; those two cases now pin the earlier refusal at this door (INVALID_FIELD, nohaving.words, no read). Itslookupgroup-key case is unchanged.Ablation (from committed code)
node scripts/ablation-replace.mjs(WRAP, trap-restored) replaced the held-row check inaggregate-field-type-door.tswith one that also skips every function butcount_distinct(markerablation-20914-A1): anchor 1 to 0, blob67c76fa23d15to80100111e881.pnpm --filter @objectstack/objectql build, thenablation-dist-preflight.mjsfound the marker in 4 built files. Predicted red on the new pins only. Observed: objectql 8 failed / 22 passed (the new suite, the flipped GUARD clause and the flippedhavingfixture; everycount_distinctpin green), REST 4 failed / 10 passed / 7 skipped (the SQLite and PostgreSQL refusal cells; controls and the #20808 cells green). Restore: blob equals HEAD,git diff HEADempty, rebuilt,--absentfound the marker in 0 of 14 built files and the tree clean, pins green again (30 / 30, 14 passed + 7 skipped).Tests (at
9497f4c61, the merge oforigin/main2f2fa11d7)pnpm --filter @objectstack/objectql exec vitest run --project local: 349 files, 6830 passed.OS_TEST_POSTGRES_URL=(private PG 16.14) pnpm --filter @objectstack/rest exec vitest run --project local: 252 files, 5055 passed, 42 skipped (MySQL cells).pnpm --filter @objectstack/service-analytics exec vitest run: 150 files, 3458 passed.@objectstack/metadata-protocoland@objectstack/plugin-securityfull suites ate37028132, before the merge oforigin/main(not re-run after it; the merge moved metadata-protocol's flow read path, not an aggregate path): 194 files / 2886 passed, and 150 files / 3262 passed.typecheckfor objectql and rest: exit 0, test-typecheck ledgers held (objectql 40 / 234 / 65, rest 0).pnpm --filter @objectstack/spec buildandcheck:generated: all 15 generated artifacts up to date.allowInlineConfig: false) over the 7 changed.tsfiles: 0 ignored, 7 results, 0 errors, 0 warnings; no file hasparserOptions.project/projectService, so type-aware linting is off and this diff cannot move a verdict on any untouched file.Gates
node scripts/pm/dispatch-gates.mjs --commandsat9497f4c61: 88 derived, 88 run, all exit 0;--ranwith exit codes: 0 NOT-MEASURED.check:dual-build-cjs-loadsandcheck:type-check-debtfirst answered PREREQUISITE NOT MET (exit 3), then 0 afterturbo run buildover every package (71 tasks). Also run:check-changeset-fixed,check:authz-resolver,check:error-code-casing,check:filter-alias-parity,check:error-status-conformance, all exit 0.check:adr-0087-registrationacceptsnot-required (already-registered dataset-measure-selecting-aggregate-field-type-refused, dataset-measure-aggregate-field-type-refused).Beyond the claimed file surface
packages/spec/src/data/aggregate-field-type-compatibility.ts: TSDoc only, three sentences that ship indist/data/index.d.tsand said the engine door reads only thecount_distinctrow. The table's value is untouched (the diff has no non-comment line).packages/objectql/src/engine-nested-object-door.test.ts: the fixture triage above.Acceptance notes
no-operator-object-door.ts'shavingwords for an aggregated column that "carries a" relation or JSON type were reached only throughmin/maxover such a field. The door now refuses those pairs first, so that branch is unreachable throughengine.aggregatefor a declared field. Not touched here..changeset/20783-groupby-structured-json-refused.md(pending) lists a structured-JSON field as an aggregatedmin/maxcolumn as unchanged; this PR's changeset says it is the later word on that shape, as the [finding] two more JSON-stored columns as a group or distinct key answer 500 on PostgreSQL:groupByon amultiple: trueselect, andcount_distincton ajsonfield #20808 changeset did for its two shapes.content/docs/data-modeling/queries.mdxstill sayscount_distinctis not lowered by the SQL drivers; it is (measured2on SQLite and PostgreSQL). Not made false by this change.Generated by Claude Code