diff --git a/.changeset/20914-release-sum-row.md b/.changeset/20914-release-sum-row.md new file mode 100644 index 00000000000..055089bdfa8 --- /dev/null +++ b/.changeset/20914-release-sum-row.md @@ -0,0 +1,30 @@ +--- +"@objectstack/objectql": minor +"@objectstack/spec": patch +--- + +fix(objectql)!: the engine's `aggregate` judges the `sum` row of the aggregate × field-type table too, so `sum` over a type the table refuses answers `INVALID_FIELD` / 400 on every driver instead of `0` in memory and on SQLite and a 500 on PostgreSQL + +Clause-②: no (narrowing) + + + +**BREAKING** (`@objectstack/objectql`): this narrows what `aggregate` accepts, on every driver and for every caller that reaches the engine — the REST query door, a flow or hook, a roll-up summary's recompute, and the analytics strategy that lowers a cube query onto `engine.aggregate`. Shipped as `minor` under the launch-window convention for accept-set narrowings. No export or published type changes. + +This completes the change of this same release that made the engine's `aggregate` ask the table for `min`, `max` and `avg`, and that held the `sum` row back. Its paragraph "Not judged yet: `sum`" is superseded: this entry is the later word, and the door now asks every row of the table. + +FROM → TO, per aggregation `{ function: 'sum', field }` naming a declared field: + +- `sum` over a type outside `number`, `currency`, `rating`, `slider`, `progress`, `summary`, `boolean` and `toggle` — a `percent` (a rate does not add), the temporal types (`date`, `datetime`, `time`), the string family (`text`, `email`, `url`, `phone`, …), the option and reference types (`select`, `radio`, `lookup`, `master_detail`, `tree`, `user`), `autonumber`, the file family, the structured-JSON types (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`), the multi-option types (`multiselect`, `checkboxes`, `tags`) and `formula` — and over any `select`, `radio`, `lookup`, `user`, `file` or `image` declared `multiple: true`: FROM whatever the driver answered TO `400 INVALID_FIELD`. Measured through `engine.aggregate` over two rows: a `json`, `text`, `select` or `tags` field summed to `0` in memory and on SQLite and answered 500 `DATABASE_ERROR` on PostgreSQL 16 (`function sum(json) does not exist`); a `datetime` field summed to `0` in memory, to the years added on SQLite and a 500 on PostgreSQL; a `formula` field summed to `0` in memory and was already refused `400 INVALID_FIELD` by both SQL drivers, which have no column for it; a `percent` field added the rates on all three. + +**What an author sees now.** `400 INVALID_FIELD`, naming the position (`aggregations[0].field`), what the function does and the field with its declaration (`sums 'meta', a declared json field — a structured-JSON value`), saying the query was not run, and naming the types `sum` accepts, read off the table, inside the first 500 characters the REST door keeps. The thrown error carries `field`, `fields` (every offending aggregation), `object` and `param` (`aggregations`). + +**What to write instead.** Sum a field of a type `sum` accepts: `number`, `currency`, `rating`, `slider`, `progress`, `summary`, `boolean` or `toggle`. A rate stored as a `percent` is averaged (`avg` accepts it), or the quantity it is a rate of is summed. A value computed by a `formula` is stored in a numeric field of its own when it must be summed on the server. A question that was counting in disguise is `count`. + +**Who is affected.** A caller that asked `sum` of such a field on the in-memory driver or SQLite and read the `0` as a real total, and a caller that summed a `percent` field on any driver. On PostgreSQL the other measured pairs were already refused (a 500, or a 400 for a `formula`), and on SQLite so was a `formula`. Metadata that lowers onto `engine.aggregate` takes the same verdict at run time: a roll-up summary (`summaryOperations`) whose `sum` names such a child field records a failed recompute, a grouped list view's server-side header summary is refused, and a chart or metric component's `sum` over such a field is refused. No example app authors such a pair. One published stack authors a `sum` list-column summary over a `formula` field; that summary is computed client-side and does not reach `engine.aggregate`. + +**Unchanged.** Every pair the table accepts, `sum` over the eight types above included; `count` over any field; an aggregation that names no field; an undeclared name or a relationship path, which this door does not judge (the REST door answers an unknown name `INVALID_FIELD` before the engine is reached); a field whose declared type is outside `FieldType`. + +**A correction to the earlier entry of this release.** It listed the multi-capable types declared `multiple: true` as `select`, `lookup`, `user`, `file` or `image`; the list is `select`, `radio`, `lookup`, `user`, `file` and `image` (`MULTI_CAPABLE_TYPES`). A `radio` declared `multiple: true` was refused by `min` / `max` / `avg` there all the same, by its type's own row, and it is refused by `sum` here. + +`@objectstack/spec`: the TSDoc of `AGGREGATE_FIELD_TYPE_COMPATIBILITY` and `isAggregateCompatibleWithFieldType` states that the engine's `aggregate` door asks every row of the table, where it said "the other rows" while one was held, and names `radio` among the multi-capable types. The table and the predicate are unchanged. diff --git a/packages/objectql/src/aggregate-field-type-door.ts b/packages/objectql/src/aggregate-field-type-door.ts index 67c76fa23d1..9a2bc89178a 100644 --- a/packages/objectql/src/aggregate-field-type-door.ts +++ b/packages/objectql/src/aggregate-field-type-door.ts @@ -36,6 +36,16 @@ * | `avg` over a `datetime` field | 200, `null` | 200, `2026` | **500 `DATABASE_ERROR`** | * | `max` over a `number`, `min` over a `datetime`, `avg` over a `percent` (the controls) | one answer | the same | the same | * + * [#20914] measured the `sum` row on `origin/main` `2821e9f15`, the same way: + * + * | aggregation | InMemoryDriver | SqlDriver, SQLite | SqlDriver, PostgreSQL 16 | + * |:--|:--|:--|:--| + * | `sum` over a `json`, `text`, single-value `select` or `tags` field | 200, `0` | 200, `0` | **500 `DATABASE_ERROR`** (`function sum(json)` — or `sum(text)`, `sum(character varying)` — `does not exist`) | + * | `sum` over a `datetime` field | 200, `0` | 200, `4052` (the years added) | **500 `DATABASE_ERROR`** | + * | `sum` over a `formula` field | 200, `0` | 400 `INVALID_FIELD` (no column) | 400 `INVALID_FIELD` (no column) | + * | `sum` over a `percent` field (refused by the table: a rate does not add) | 200, the rates added | the same | the same | + * | `sum` over a `number`, `currency` or `boolean` (the controls) | one answer | the same | the same | + * * ## Whose verdict it is * * The TYPE half is the spec table's: `AGGREGATE_FIELD_TYPE_COMPATIBILITY` @@ -46,25 +56,28 @@ * nothing. The refusal names the row's accepted set, read off the table. * * The DECLARATION half is `isMultiValueField`: a multi-capable type flagged - * `multiple: true` (`select`, `lookup`, `user`, `file`, `image`) is stored in - * the JSON column the multi-option types (`MULTI_OPTION_TYPES`) are stored in - * and holds the same value — a list — but a per-TYPE table cannot see the - * flag. So the declaration takes the verdict the row gives that class: a row - * that refuses any multi-option type refuses a multi-value declaration too - * (`count_distinct`, `sum`, `avg`, `min`, `max`), and a row that accepts the - * whole class accepts it (`count`, which compares no value). A field is - * refused if either half refuses it. + * `multiple: true` (`MULTI_CAPABLE_TYPES`: `select`, `radio`, `lookup`, + * `user`, `file`, `image`) is stored in the JSON column the multi-option types + * (`MULTI_OPTION_TYPES`) are stored in and holds the same value — a list — but + * a per-TYPE table cannot see the flag. So the declaration takes the verdict + * the row gives that class: a row that refuses any multi-option type refuses a + * multi-value declaration too (`count_distinct`, `sum`, `avg`, `min`, `max`), + * and a row that accepts the whole class accepts it (`count`, which compares + * no value). A field is refused if either half refuses it. * - * ## The row the census held back: `sum` + * ## Every row is judged — `sum` included * * [#20914] The triage direction sent the door to the whole table "census * first": an authored pair the table refuses, in `examples/**` or a published - * stack, stops that row and goes back to triage. The census found one, in the - * published hotcrm stack: a grouped list view whose column summary sums a - * `formula` field (`sum` × `formula`, refused by the table — a formula is - * virtual in SQL storage). So `sum` is in {@link ROWS_HELD_FOR_TRIAGE} and is - * not judged here; releasing it is deleting that entry and flipping its pins. - * Every other row is judged. + * stack, stops that row and goes back to triage. The census found one `sum` + * pair, in the published hotcrm stack: a grouped list view whose column + * summary sums a `formula` field (`sum` × `formula`, refused by the table — a + * formula is virtual in SQL storage). The row was held for one landing, then + * released by triage's answer: that summary is a client-side list footer that + * never reaches `engine.aggregate`, and both SQL drivers already refused the + * pair, while `sum` over a `json`, `text` or `select` field answered a + * plausible `0` or a 500. ⛔ No row is held and no pair is exempted: the door + * asks the table's row whole for all six functions. * * ## Where it stands, and what it judges * @@ -79,12 +92,11 @@ * **Not judged** (no verdict, the aggregation passes on as it came): an * aggregation that names no field (a fieldless `count`, or `'*'`), a function * outside the table's vocabulary (that is the query schema's refusal, not a - * field-type one), a held row, an undeclared name (a relationship path - * included), a registry-less host (no field map, no verdict), and a declared - * type outside `FieldType` (a driver-internal alias such as `string` or - * `integer` on an introspected object): the table is fail-closed on - * vocabulary, and "cannot answer, do not block" is this consumer's tier, as - * the table's own TSDoc says. + * field-type one), an undeclared name (a relationship path included), a + * registry-less host (no field map, no verdict), and a declared type outside + * `FieldType` (a driver-internal alias such as `string` or `integer` on an + * introspected object): the table is fail-closed on vocabulary, and "cannot + * answer, do not block" is this consumer's tier, as the table's own TSDoc says. * * `INVALID_FIELD`, the code the `groupBy` door beside it answers: the verdict * is about the NAMED field's type at a position. @@ -107,13 +119,6 @@ import { /** The declared `FieldType` vocabulary — the only types the table can answer for. */ const DECLARED_FIELD_TYPES: ReadonlySet = new Set(FieldType.options); -/** - * Rows of the table this door does not judge yet, each held by the census - * that preceded it — see the module header. ⛔ A row is held, never trimmed: - * the door asks the table's row whole or not at all. - */ -const ROWS_HELD_FOR_TRIAGE: ReadonlySet = new Set(['sum']); - /** Does the table's `fn` row accept the multi-value class — every multi-option type? */ function rowAcceptsMultiValue(fn: string): boolean { for (const type of MULTI_OPTION_TYPES) { @@ -152,7 +157,6 @@ function refusedAggregations( const fn = (agg as { function?: unknown }).function; if (typeof fn !== 'string') continue; if (!Object.prototype.hasOwnProperty.call(AGGREGATE_FIELD_TYPE_COMPATIBILITY, fn)) continue; - if (ROWS_HELD_FOR_TRIAGE.has(fn)) continue; const field = (agg as { field?: unknown }).field; if (typeof field !== 'string' || field === '*') continue; if (!Object.prototype.hasOwnProperty.call(fields, field)) continue; diff --git a/packages/objectql/src/engine-aggregate-field-type-door.test.ts b/packages/objectql/src/engine-aggregate-field-type-door.test.ts index 186d760e348..bc2cef77708 100644 --- a/packages/objectql/src/engine-aggregate-field-type-door.test.ts +++ b/packages/objectql/src/engine-aggregate-field-type-door.test.ts @@ -16,6 +16,15 @@ * | `avg` over a `datetime` field | 200, `null` | 200, `2026` | 500 `DATABASE_ERROR` | * | `max` over a `number`, `min` over a `datetime`, `avg` over a `percent` (controls) | one answer | the same | the same | * + * …and the `sum` row, released after one landing held it for triage's census + * answer, measured on `origin/main` `2821e9f15` the same way: + * + * | aggregation | InMemoryDriver | SqlDriver, SQLite | SqlDriver, PostgreSQL 16 | + * |:--|:--|:--|:--| + * | `sum` over a `json`, `text`, `select` or `tags` field | 200, `0` | 200, `0` | 500 `DATABASE_ERROR` | + * | `sum` over a `formula` field | 200, `0` | 400 `INVALID_FIELD` (no column) | the same | + * | `sum` over a `number`, `currency` or `boolean` (controls) | one answer | the same | the same | + * * The InMemoryDriver cell is this suite's recording driver by construction: * the door answers before a driver is resolved, so no read runs. The SQL cells * over a real driver live in `@objectstack/rest`'s @@ -48,6 +57,7 @@ const PROBE = { fields: { title: { name: 'title', type: 'text' }, amount: { name: 'amount', type: 'number' }, + price: { name: 'price', type: 'currency' }, pct: { name: 'pct', type: 'percent' }, due: { name: 'due', type: 'datetime' }, flag: { name: 'flag', type: 'boolean' }, @@ -57,6 +67,7 @@ const PROBE = { picks: { name: 'picks', type: 'select', multiple: true, options: OPTIONS }, owners: { name: 'owners', type: 'lookup', multiple: true, reference: 'aggregate_type_target' }, ship_to: { name: 'ship_to', type: 'address' }, + expected: { name: 'expected', type: 'formula', expression: 'record.amount * 2' }, }, }; @@ -100,10 +111,12 @@ const ENVELOPE = { code: 'INVALID_FIELD', status: 400, httpStatus: 400 }; const envelopeOf = (err: Thrown) => ({ code: err?.code, status: err?.status, httpStatus: err?.httpStatus }); /** The function's words in the refusal — a control must never be answered in them. */ -const DOES_NOT = { min: 'does not take the min of', max: 'does not take the max of', avg: 'does not average' } as const; -const DOES = { min: 'takes the min of', max: 'takes the max of', avg: 'averages' } as const; +const DOES_NOT = { min: 'does not take the min of', max: 'does not take the max of', avg: 'does not average', sum: 'does not sum' } as const; +const DOES = { min: 'takes the min of', max: 'takes the max of', avg: 'averages', sum: 'sums' } as const; const ACCEPTS_MIN_MAX = 'accepts a field of type number, currency, percent, rating, slider, progress, summary, ' + 'date, datetime, time, boolean or toggle: aggregate a field of one of those types, or count the rows with count.'; +const ACCEPTS_SUM = 'accepts a field of type number, currency, rating, slider, progress, summary, boolean or toggle: ' + + 'aggregate a field of one of those types, or count the rows with count.'; describe('[#20914] the engine\'s aggregate door asks the aggregate × field-type table for every row', () => { let engine: ObjectQL; @@ -173,6 +186,33 @@ describe('[#20914] the engine\'s aggregate door asks the aggregate × field-type expect(reads).toHaveLength(0); }); + it('refuses sum over a json, text, select, formula, tags, percent and datetime field, and a select with multiple: true — the row the census held, released — with INVALID_FIELD / 400 naming the accepted types — no read', async () => { + // Flipped from the CONTROL below, where `sum meta` and `sum title` reached + // the driver while the row was held: in memory they answered `0`. + const cases: ReadonlyArray = [ + ['meta', 'json field — a structured-JSON value'], + ['title', 'text field'], + ['status', 'select field'], + ['expected', 'formula field'], + ['labels', 'tags field — a multi-value field'], + ['pct', 'percent field'], + ['due', 'datetime field'], + ['picks', 'select field with multiple: true — a multi-value field'], + ]; + for (const [field, declared] of cases) { + const label = `sum(${field})`; + const err = await refusalOf(engine.aggregate(OBJECT, { aggregations: agg('sum', field) })); + expect(envelopeOf(err), label).toEqual(ENVELOPE); + expect({ field: err?.field, fields: err?.fields, object: err?.object, param: err?.param }, label) + .toEqual({ field, fields: [field], object: OBJECT, param: 'aggregations' }); + expect(err!.message, label).toMatch(new RegExp( + `^aggregate\\('${OBJECT}'\\): aggregations\\[0\\]\\.field sums '${field}', a declared ${declared}, which the engine does not sum\\. The query was NOT run\\.`, + )); + expect(err!.message, label).toContain(`sum ${ACCEPTS_SUM}`); + } + expect(reads, 'every refusal precedes the driver').toHaveLength(0); + }); + it('names the first offending position across functions, and lists every offender', async () => { const err = await refusalOf(engine.aggregate(OBJECT, { groupBy: ['status'], @@ -191,19 +231,18 @@ describe('[#20914] the engine\'s aggregate door asks the aggregate × field-type expect(reads).toHaveLength(0); }); - it('CONTROL a pair the table accepts reaches the driver, never this refusal — and so does every sum (the held row)', async () => { + it('CONTROL a pair the table accepts reaches the driver, never this refusal — sum over a number, a currency and a boolean included', async () => { const shapes: ReadonlyArray = [ ['max amount', { aggregations: agg('max', 'amount') }], ['min due', { aggregations: agg('min', 'due') }], ['avg pct', { aggregations: agg('avg', 'pct') }], ['max flag', { aggregations: agg('max', 'flag') }], ['sum amount', { aggregations: agg('sum', 'amount') }], + ['sum price', { aggregations: agg('sum', 'price') }], + ['sum flag', { aggregations: agg('sum', 'flag') }], // `count` compares no value, so a JSON-stored column is countable. ['count meta', { aggregations: agg('count', 'meta') }], ['count picks', { aggregations: agg('count', 'picks') }], - // The held row: not judged here until triage releases it. - ['sum meta', { aggregations: agg('sum', 'meta') }], - ['sum title', { aggregations: agg('sum', 'title') }], ['grouped max amount', { groupBy: ['status'], aggregations: agg('max', 'amount') }], ]; for (const [label, query] of shapes) { @@ -225,7 +264,7 @@ describe('[#20914] the engine\'s aggregate door asks the aggregate × field-type expect(reads).toHaveLength(0); }); - it('GUARD the door asks the spec table for every judged row × every FieldType, and the declaration half for every flagged multi-capable type', () => { + it('GUARD the door asks the spec table for every row × every FieldType, and the declaration half for every flagged multi-capable type — no row held', () => { const judged = (fn: string, def: Record) => { try { assertAggregationFieldTypesAccepted(OBJECT, { fields: { f: def } }, agg(fn, 'f')); @@ -235,7 +274,11 @@ describe('[#20914] the engine\'s aggregate door asks the aggregate × field-type } }; const refusedPerRow: Record = {}; - for (const fn of ['count', 'count_distinct', 'avg', 'min', 'max']) { + // Every row of the table, read off the table: a row the door skipped would + // answer null where the table refuses, and turn this red. + const rows = Object.keys(AGGREGATE_FIELD_TYPE_COMPATIBILITY); + expect([...rows].sort()).toEqual(['avg', 'count', 'count_distinct', 'max', 'min', 'sum']); + for (const fn of rows) { // A row that refuses any type-level multi-value type refuses the declaration too. const rowRefusesMulti = !isAggregateCompatibleWithFieldType(fn, 'tags'); refusedPerRow[fn] = 0; @@ -257,18 +300,18 @@ describe('[#20914] the engine\'s aggregate door asks the aggregate × field-type expect(refusedPerRow).toEqual({ count: 0, count_distinct: refusedByTable('count_distinct'), + sum: refusedByTable('sum'), avg: refusedByTable('avg'), min: refusedByTable('min'), max: refusedByTable('max'), }); expect(refusedPerRow.max).toBeGreaterThan(30); - // The held row: no verdict for any declared type. - for (const type of FieldType.options) expect(judged('sum', { type }), `sum × ${type}`).toBeNull(); + expect(refusedPerRow.sum).toBeGreaterThan(30); }); it('GUARD no verdict without a field map, for an undeclared name or a path, an off-vocabulary type, a fieldless aggregation or an off-vocabulary function', () => { const judge = (schema: unknown, aggregations: unknown) => () => assertAggregationFieldTypesAccepted(OBJECT, schema, aggregations); - for (const fn of ['max', 'min', 'avg', 'count_distinct']) { + for (const fn of ['max', 'min', 'avg', 'sum', 'count_distinct']) { expect(judge(undefined, agg(fn, 'meta')), fn).not.toThrow(); expect(judge({}, agg(fn, 'meta')), fn).not.toThrow(); expect(judge(PROBE, agg(fn, 'nope')), fn).not.toThrow(); diff --git a/packages/objectql/src/engine-json-stored-group-distinct-door.test.ts b/packages/objectql/src/engine-json-stored-group-distinct-door.test.ts index 9b318abe665..639a03edb15 100644 --- a/packages/objectql/src/engine-json-stored-group-distinct-door.test.ts +++ b/packages/objectql/src/engine-json-stored-group-distinct-door.test.ts @@ -286,14 +286,12 @@ describe('[#20808] a groupBy on a multi-value field, and a count_distinct on a J // A driver-internal alias on an introspected object: the table cannot answer, so no block. expect(judge({ fields: { f: { type: 'object' } } }, distinct('f'))).not.toThrow(); expect(judge({ fields: { f: { type: 'integer' } } }, distinct('f'))).not.toThrow(); - // `count` compares no value, so its row accepts a JSON-stored field; `sum` - // is the row the #20914 census held back for triage. - for (const fn of ['count', 'sum']) { - expect(judge(PROBE, [{ function: fn, field: 'meta', alias: 'n' }]), fn).not.toThrow(); - } - // [#20914] Flipped: the door asks every row now, and the `avg`, `min` and - // `max` rows refuse a `json` field — the same envelope, at the same position. - for (const fn of ['avg', 'min', 'max']) { + // `count` compares no value, so its row accepts a JSON-stored field. + expect(judge(PROBE, [{ function: 'count', field: 'meta', alias: 'n' }]), 'count').not.toThrow(); + // [#20914] Flipped: the door asks every row now, and the `sum`, `avg`, + // `min` and `max` rows refuse a `json` field — the same envelope, at the + // same position. (`sum` passed here while the census held its row.) + for (const fn of ['sum', 'avg', 'min', 'max']) { let thrown: Thrown = null; try { judge(PROBE, [{ function: fn, field: 'meta', alias: 'n' }])(); } catch (e) { thrown = e as Thrown; } expect(envelopeOf(thrown), fn).toEqual(ENVELOPE); diff --git a/packages/objectql/src/engine.ts b/packages/objectql/src/engine.ts index 2adf30e99e3..2d1b17f8adf 100644 --- a/packages/objectql/src/engine.ts +++ b/packages/objectql/src/engine.ts @@ -17013,8 +17013,8 @@ export class ObjectQL implements IObjectQLEngine { // the `count_distinct` row (memory counted equal documents apart, SQLite // compared serialized text, PostgreSQL answered 500); [#20914] took the // other rows — `max` over a `json` field answered a document in memory, a - // string on SQLite and a 500 on PostgreSQL. The `sum` row is held back by - // that card's census (see the door's header). + // string on SQLite and a 500 on PostgreSQL; `sum` over one answered `0`, + // `0` and a 500. Every row of the table is asked (see the door's header). assertAggregationFieldTypesAccepted(object, this._registry.getObject(object), query.aggregations); // [#10576] The per-aggregation `filter` (`AggregationNodeSchema.filter`, // the contract half of #10413) is a second filter position on this verb, diff --git a/packages/rest/src/data-aggregate-field-type-door.test.ts b/packages/rest/src/data-aggregate-field-type-door.test.ts index 2939d0a35fa..a0e0fbb234c 100644 --- a/packages/rest/src/data-aggregate-field-type-door.test.ts +++ b/packages/rest/src/data-aggregate-field-type-door.test.ts @@ -17,6 +17,15 @@ * | `avg` over a `datetime` field | 200, `null` | 200, `2026` | 500 `DATABASE_ERROR` | * | `max` over a `number`, `min` over a `datetime`, `avg` over a `percent`, `max` over a `boolean` (the controls) | one answer | the same | the same | * + * …and the `sum` row, which one landing held back for triage's census answer + * and the next released, measured on `origin/main` `2821e9f15`: + * + * | query | InMemoryDriver | SQLite | PostgreSQL 16 | + * |:--|:--|:--|:--| + * | `sum` over a `json`, `text`, `select` or `tags` field | 200, `0` | 200, `0` | 500 `DATABASE_ERROR` (`function sum(json) does not exist`, `sum(text)`, `sum(character varying)`) | + * | `sum` over a `formula` field | 200, `0` | 400 `INVALID_FIELD`, the driver's words (no column) | the same | + * | `sum` over a `number`, `currency` or `boolean` (the controls) | one answer | the same | the same | + * * The refusals sit in the engine, in front of every driver, so one verdict * holds on each cell. InMemoryDriver's row is `@objectstack/objectql`'s * `engine-aggregate-field-type-door.test.ts` by construction (the door answers @@ -52,22 +61,25 @@ const LEDGER = { fields: { title: { name: 'title', type: 'text' as const }, amount: { name: 'amount', type: 'number' as const }, + price: { name: 'price', type: 'currency' as const }, pct: { name: 'pct', type: 'percent' as const }, due: { name: 'due', type: 'datetime' as const }, flag: { name: 'flag', type: 'boolean' as const }, + status: { name: 'status', type: 'select' as const, options: OPTIONS }, meta: { name: 'meta', type: 'json' as const }, labels: { name: 'labels', type: 'tags' as const }, picks: { name: 'picks', type: 'select' as const, multiple: true, options: OPTIONS }, owners: { name: 'owners', type: 'lookup' as const, multiple: true, reference: TARGET }, + expected: { name: 'expected', type: 'formula' as const, expression: 'record.amount * 2' }, }, }; const TARGET_OBJECT = { name: TARGET, label: 'Target 20914', fields: { name: { name: 'name', type: 'text' as const } } }; const ROWS = [ - { id: 'd1', title: 'x', amount: 1, pct: 10, due: '2026-01-01T00:00:00.000Z', flag: true, meta: { a: 1 }, labels: ['p', 'q'], picks: ['a'], owners: ['t1'] }, - { id: 'd2', title: 'x', amount: 2, pct: 20, due: '2026-02-01T00:00:00.000Z', flag: false, meta: { b: 1 }, labels: ['p'], picks: ['a', 'b'], owners: ['t1', 't2'] }, - { id: 'd3', title: 'y', amount: 3, pct: 30, due: '2026-03-01T00:00:00.000Z', flag: true, meta: { a: 2 }, labels: ['q'], picks: ['b'], owners: ['t2'] }, + { id: 'd1', title: 'x', amount: 1, price: 10, pct: 10, due: '2026-01-01T00:00:00.000Z', flag: true, status: 'a', meta: { a: 1 }, labels: ['p', 'q'], picks: ['a'], owners: ['t1'] }, + { id: 'd2', title: 'x', amount: 2, price: 20, pct: 20, due: '2026-02-01T00:00:00.000Z', flag: false, status: 'b', meta: { b: 1 }, labels: ['p'], picks: ['a', 'b'], owners: ['t1', 't2'] }, + { id: 'd3', title: 'y', amount: 3, price: 30, pct: 30, due: '2026-03-01T00:00:00.000Z', flag: true, status: 'a', meta: { a: 2 }, labels: ['q'], picks: ['b'], owners: ['t2'] }, ]; const agg = (fn: string, field: string, alias = 'v'): EngineAggregateOptions['aggregations'] => @@ -77,6 +89,10 @@ const agg = (fn: string, field: string, alias = 'v'): EngineAggregateOptions['ag const MIN_MAX_ROUTE = 'accepts a field of type number, currency, percent, rating, slider, progress, summary, ' + 'date, datetime, time, boolean or toggle: aggregate a field of one of those types, or count the rows with count.'; +/** The route the refusal names for sum, read off the table's `sum` row — inside the same 500-character bound. */ +const SUM_ROUTE = 'sum accepts a field of type number, currency, rating, slider, progress, summary, boolean or toggle: ' + + 'aggregate a field of one of those types, or count the rows with count.'; + interface Cell { id: 'sqlite' | 'pg' | 'mysql'; label: string; @@ -206,6 +222,33 @@ for (const cell of CELLS) { expect(reads.n - before, 'no read of the object').toBe(0); }); + it('sum over a json, text, select, tags and formula field answers 400 INVALID_FIELD in the engine\'s words, naming the types sum accepts — no read', async () => { + // The row one landing held for the census, released: these pairs + // reached the driver then (0 on SQLite, a 500 on PostgreSQL; a formula + // the driver's own "no column" 400). + const before = reads.n; + const cases: ReadonlyArray = [ + ['meta', 'json field — a structured-JSON value'], + ['title', 'text field'], + ['status', 'select field'], + ['labels', 'tags field — a multi-value field'], + ['expected', 'formula field'], + ]; + for (const [field, declared] of cases) { + const label = `sum(${field})`; + const res = await query({ aggregations: agg('sum', field) }); + expect(res.status, `${label}: ${JSON.stringify(res.body)}`).toBe(400); + expect(res.body.code, label).toBe('INVALID_FIELD'); + expect(res.body.error, label).toContain( + `aggregations[0].field sums '${field}', a declared ${declared}, which the engine does not sum. The query was NOT run.`, + ); + expect(res.body.error, label).toContain(SUM_ROUTE); + const err = await engine.aggregate(OBJECT, { aggregations: agg('sum', field) }).then(() => null, (e: any) => e); + expect({ code: err?.code, status: err?.status }, `engine.aggregate, ${label}`).toEqual({ code: 'INVALID_FIELD', status: 400 }); + } + expect(reads.n - before, 'no read of the object — every refusal precedes the driver').toBe(0); + }); + it('CONTROL a pair the table accepts is served unchanged, from the driver', async () => { const before = reads.n; const answers: Record = {}; @@ -213,6 +256,8 @@ for (const cell of CELLS) { ['max', 'amount', 'max amount'], ['avg', 'pct', 'avg pct'], ['sum', 'amount', 'sum amount'], + ['sum', 'price', 'sum price'], + ['sum', 'flag', 'sum flag'], ['max', 'flag', 'max flag'], ['min', 'due', 'min due'], ]; @@ -225,8 +270,10 @@ for (const cell of CELLS) { 'max amount': Number(answers['max amount']), 'avg pct': Number(answers['avg pct']), 'sum amount': Number(answers['sum amount']), + 'sum price': Number(answers['sum price']), + 'sum flag': Number(answers['sum flag']), 'max flag': Number(answers['max flag']), - }).toEqual({ 'max amount': 3, 'avg pct': 20, 'sum amount': 6, 'max flag': 1 }); + }).toEqual({ 'max amount': 3, 'avg pct': 20, 'sum amount': 6, 'sum price': 60, 'sum flag': 2, 'max flag': 1 }); expect(new Date(answers['min due'] as string).toISOString()).toBe('2026-01-01T00:00:00.000Z'); expect(reads.n - before, 'the driver was asked, once per query').toBe(shapes.length); }); diff --git a/packages/spec/src/data/aggregate-field-type-compatibility.ts b/packages/spec/src/data/aggregate-field-type-compatibility.ts index f00ba98fbcc..e8d53830086 100644 --- a/packages/spec/src/data/aggregate-field-type-compatibility.ts +++ b/packages/spec/src/data/aggregate-field-type-compatibility.ts @@ -10,9 +10,9 @@ * in the dataset compiler (#16099) and the authoring-time lint rule — so the * two cannot drift into two accounts of one pair. The table has a third * reader, the engine's `aggregate` door, which refuses a query-time pair the - * table refuses — the `count_distinct` row since #20808 (the JSON-stored rows - * below), the other rows since #20914; objectql's `aggregate-field-type-door.ts` - * names the rows it judges. + * table refuses, on every row: `count_distinct` since #20808 (the JSON-stored + * rows below), and `sum`, `avg`, `min` and `max` since #20914 (`count` refuses + * no type, so its row never refuses there). * * ## Why this exists * @@ -95,8 +95,8 @@ * refused the statement (`could not identify an equality operator for type * json`, a 500), on every member of both classes. `count` is untouched: * counting rows compares no value. The one other JSON-stored shape, a - * multi-capable type flagged `multiple: true` (`select`, `lookup`, `user`, - * `file`, `image`), is invisible to a per-TYPE table, so the engine's + * multi-capable type flagged `multiple: true` (`select`, `radio`, `lookup`, + * `user`, `file`, `image`), is invisible to a per-TYPE table, so the engine's * aggregate door refuses it by the declaration (`isMultiValueField`) beside * this row's verdict. ⛔ No per-backend JSON distinctness is defined to * make the pair answerable: no caller of it was measured. @@ -138,8 +138,7 @@ * * It refuses nothing itself. The refusals are the consumer legs — the dataset * compile leg and the authoring lint leg on every row, and the engine's - * `aggregate` door at query time (#20808, #20914 — the door names the rows it - * judges); a + * `aggregate` door at query time, on every row too (#20808, #20914); a * consumer that cannot resolve a field's type (a relationship PATH it has no * metadata for) must NOT call the predicate with a guess — "cannot answer, do * not block" is the consumer's tier, not this table's. @@ -226,7 +225,7 @@ export const AGGREGATE_FIELD_TYPE_COMPATIBILITY: Readonly