fix(core,driver-sql,service-analytics): the analytics native-SQL path answers measures declared number as numbers (#20889) - #21040
Conversation
…mbers on the analytics native-SQL path (red) Red-first pins for #20889. On PostgreSQL the analytics native-SQL path answers count / count_distinct / sum / avg and numeric min / max as strings while fields[] declares number; SQLite answers numbers. - core: aggregate-answer.test.ts pins AGGREGATE_ANSWER_KIND and the 'number' presenter (red: the module does not exist yet). - service-analytics: the cube read and the dataset door over a real engine and SqlDriver, SQLite and a named-skip PostgreSQL cell. - rest: POST /api/v1/analytics/dataset/query on both cells, and the lenient row_count pin (number or string) tightened to a number. Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H Co-authored-by: Claude <noreply@anthropic.com>
… answers measures declared number as numbers (#20889) On PostgreSQL, NativeSQLStrategy returned the SQL client's rows unshaped, so count / count_distinct / sum / avg and numeric min / max came back as strings while fields[] declared number. driver-sql's own aggregate() has presented those answers since #20335, through a private table and presenter. - core: utils/aggregate-answer.ts receives AGGREGATE_ANSWER_KIND (its docblock moved byte-identical) and the 'number' presenter, presentAsNumber (the body of presentReadValue's 'number' arm). Exported from the root beside compensatedSum. - driver-sql: the definition becomes an import; presentReadValue's 'number' arm calls presentAsNumber. No behaviour change. - service-analytics: NativeSQLStrategy.execute presents each measure column keyed on its declared aggregate function, and min / max only over a column declared numeric (declaredFieldType). Expression measures are untouched. Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 3 package(s): 6 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 3 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 35 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 57fed1f5f61445066b13c8b00611ecfa264fe3d0 && git checkout 57fed1f5f61445066b13c8b00611ecfa264fe3d0
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 2f2fa11d756f665a4c06160480c1dce15b9d67a4 e827c237d1a18f469e33955474d81e8c3dcc9e56 && git checkout -B drift-repro 2f2fa11d756f665a4c06160480c1dce15b9d67a4 && git merge --no-ff e827c237d1a18f469e33955474d81e8c3dcc9e56
node scripts/docs-audit/affected-docs.mjs --json 2f2fa11d756f665a4c06160480c1dce15b9d67a4
|
Contract reviewServed-tier: Inputs: card #20889 (body and all 8 comments, the route ruling ① Derived judgments
② Semver level
③ Boundary flagsThe build report's Dev flags, from the acceptance notes and
Implemented-by: VERDICT: PASS |
Fixes #20889
Clause-②: yes (widening)
What changed
On PostgreSQL, the analytics native-SQL path answered every measure its response declares
numberas a string:count: "2",sum: "500.000000000000000000000000000000". SQLite answered numbers.driver-sql's ownaggregate()has presented these answers since #20335, but through a private table and presenter thatservice-analyticscould not reach. The seat's route ruling (5922780640) moves that one rule into@objectstack/core, and both faces now call it.@objectstack/core(newsrc/utils/aggregate-answer.ts, one export line inindex.tsbesidecompensatedSum):AGGREGATE_ANSWER_KIND, with its precision-policy docblock, moved fromsql-driver.ts. The text is byte-identical apart from theexportkeyword; the proof is below.presentAsNumber(value), the body ofpresentReadValue's'number'arm as a named function. A stringNumber()reads as a number becomes that number. Any other value is returned as given.@objectstack/driver-sql,sql-driver.ts, the two ruled regions only:AGGREGATE_ANSWER_KINDdefinition becomes an import, and a pointer comment marks where it was;presentReadValue's'number'arm returnspresentAsNumber(value).The import is a separate statement beside the
AggregationFunctionimport (inserted after base line 23; line 27 at HEAD). It does NOT join the@objectstack/coreimport block at base lines 94-100, which PR fix(plugin-security,driver-sql,driver-turso): lower type-blind at the RLS seam without a guard, then delete the F1/F2 whole-day and NOT-rewrite copies (#5930 step 4, group 2) #20988 rewrites. That hunk starts at base line 91, 67 lines below the insertion. No behaviour change: the full driver-sql suite is green on SQLite and PostgreSQL (below).@objectstack/service-analytics,NativeSQLStrategy.executeonly. After the raw statement runs, each measure column is presented by the measure's DECLARED aggregate function (lookupMember(cube, m, 'measure').type), never by whether a value looks numeric:count,count_distinct,sumandavg(the table's'number'kind) always takepresentAsNumber;min/max(the'column'kind) take it only whendeclaredFieldType(object, column)is inNUMERIC_VALUE_TYPES. That is the ruledriver-sql'sreadPresentationKindapplies to a declared numeric column. A host that cannot answer, or a relationship-path column, leaves the value as given;number/string/boolean) are left as they are (ruling Q3).One point covers both doors: the cube read (
POST /api/v1/analytics/query) and the dataset door (POST /api/v1/analytics/dataset/query, throughDatasetExecutor)..changeset/20889-analytics-native-measure-number.md:@objectstack/coreminor,@objectstack/driver-sqland@objectstack/service-analyticspatch,Clause-②: yes (widening)for the two new root exports.Per-measure answers, before and after
Fixture:
textcategory;ratingstars (an integer column);numberamount and frac (the exact-decimal column);currencyprice. Groupaholds 2 rows.7fa67dada3, throughAnalyticsService.query(what the route relays verbatim). The red pins at719b98795eread the same strings.e827c237d1through the realdispatcher-pluginmount ofPOST /api/v1/analytics/query, by a scratch probe that was deleted after the run.count*22"2"2count_distincttext)22"2"2sumrating, integer)77"7"7avgrating, integer)3.53.5"3.5000000000000000"3.5sumnumber, decimal)500500"500.000000000000000000000000000000"500avgnumber, decimal)250250"250.000000000000000000000000000000"250minnumber)100100"100.000000000000000000000000000000"100maxcurrency)20.520.5"20.500000000000000000000000000000"20.5maxtext), the control"y""y"(left as the client gave it)sum900719925474099390071992547409929007199254740992"9007199254740993.000000000000000000000000000000"9007199254740992row_count(count)*22"2"2filtered_count, a group the statement answered*11"1"1numberSUM(amount) / 2250250"250.000000000000000000000000000000"fields[]declarednumberfor every measure on both dialects, before and after. On the ObjectQL face of the same route, every answer was already a number on both dialects, and it is unchanged.Move proof
move-proof.py(scratch) reads the block from BASE2f2fa11d75'ssql-driver.tsand from HEAD'saggregate-answer.ts:The table and its docblock are byte-identical apart from
export. The presenter's eight body lines, comment included, are identical once de-indented. In core, a new file header says why the rule lives there. The moved docblock still speaks ofSqlDriver.aggregateandformatOutputin the driver's voice, and the header says so.Pins (red first), the ablation, and the local PostgreSQL run
719b98795e(pins only, no fix), on a private PostgreSQL 16.13 (initdb, 127.0.0.1, trust):@objectstack/coreaggregate-answer.test.ts: 1 file failed,Cannot find module './aggregate-answer'.service-analyticsnative-sql-measure-number-presentation.test.ts: 5 failed, all PostgreSQL; 5 passed, all SQLite. First failure:a row_count is a number, never "2": expected 'string' to be 'number'.rest, the newanalytics-dataset-measure-number-door.test.tsand the tightenedanalytics-dataset-json-dimension-door.test.ts: 2 failed, both PostgreSQL; 6 passed. Failures:a row_count is a number, never "2"andexpected [ [ 'x', '2' ], [ 'y', '1' ] ] to deeply equal [ [ 'x', 2 ], [ 'y', 1 ] ].e827c237d1: core 5/5; service-analytics 10/10 (SQLite and PostgreSQL); rest 8/8 (SQLite and PostgreSQL).scripts/ablation-replace.mjs(WRAP mode, with its restore trap) on the absolute path. The guardif (numberMeasures.length > 0 && Array.isArray(rows))gained&& String(numberMeasures) === 'ABLATION-20889', so it never holds. Anchor count went 1 to 0, blob3bce7bb073to8f36f62dfe.service-analyticswas rebuilt, andablation-dist-preflight.mjsfound the marker in 2 built files.3bce7bb073),git diff HEADis 0 bytes, and porcelain is 0 lines. Then a rebuild,ablation-dist-preflight.mjs --absent(marker absent from all 6 built files, tree clean), and a re-run: all green.driver-sql's full suite ate827c237d1, SQLite and live PostgreSQL,TZ=America/New_York, with the server atAsia/Shanghaias CI runs it: 210 files passed, 3 skipped; 4061 tests passed, 95 skipped.sql-driver-20335-aggregate-numeric-presentation.test.ts15 tests (1 skipped, the MySQL cell), andsql-driver-20387-aggregate-double-accumulation.test.ts11 (1 skipped). A first run on a UTC server failed only the four suites whose precondition asserts a non-UTC server.OS_TEST_POSTGRES_URLforservice-analyticsorrest, so their PostgreSQL cells are named skips there (skipped: set OS_TEST_POSTGRES_URL to run this cell). The core pin (string to number) and every SQLite cell run in CI. The private cluster was stopped and its data directory removed after the runs.Gates
All at HEAD
e827c237d1, as ONE sequential script under the shared verify lock, each exit code captured before any pipe.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstacknames 67.--ranreconciles:67 derived famil(ies) accounted for — 65 run, 2 NOT-MEASURED, 0 unrun. All 65 exit 0.check:dual-build-cjs-loadsandcheck:type-check-debt. Reason: both exit 3,PREREQUISITE NOT MET. They read the built output of every package, and this run built only the dependency closure it tests. CI'slint.ymlbuilds the full closure before them.check-changeset-fixed.mjs,check:authz-resolver,check:error-code-casing,check:filter-alias-parity. All exit 0.@objectstack/coretest: 62 files, 1803 tests passed. Typecheck exit 0, and itscheck:test-typecheckcovers the new test.@objectstack/driver-sql: typecheck exit 0; the full suite as above.@objectstack/service-analyticsfull suite withOS_TEST_POSTGRES_URLset: 151 files, 3468 tests passed. Typecheck exit 0;tsc --listFilesincludes the new pin.@objectstack/rest: the two pin files, SQLite and PostgreSQL, 8/8. Typecheck exit 0, withcheck:test-typecheckovertsconfig.test.json.pnpm lintis CI's):eslint --no-inline-config --format jsonover the 8 touched.tsfiles. From the JSON: 8 files, 0 errors, 0 warnings, 0 ignored.eslint.config.mjs'sfiles: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']minusNEVER_LINTED, which contains all 8.eslint.config.mjs:326-328, noparserOptions.project), and this diff does not touch the config, so no untouched file's verdict can move.Acceptance notes
DECIMALstring would be presented the same way. Separately, a read-only inference, also unmeasured:plugin.ts:500returnsengine.execute's result as the rows whenever it is an array, andknex.rawon mysql2 resolves to a[rows, fields]pair, so on MySQL the native path may read that pair as its rows.formatOutput's inline copy of the presenter (sql-driver.ts, the numeric pass at about:20303) is not folded ontopresentAsNumberhere, per the ruling. It is the same rule written a second time, againstpresentReadValue's "never a re-derivation" docblock. Fordomain:engine.sql-driver-20335-aggregate-numeric-presentation.test.ts:31andsql-driver-20387-aggregate-double-accumulation.test.ts:7sayAGGREGATE_ANSWER_KINDlives insql-driver.ts. These are driver-sql test docblocks outside this card's two regions. Fordomain:engine, or whoever next touches those files.type: 'number' | 'string' | 'boolean') are untouched:fields[]still declaresnumberwhile PostgreSQL answers"250.000…", text or a JS boolean. [Decision] analytics field gate (#20917): an authored cube member whosesqlis an expression — keep the stand-down, judge its identifiers, refuse it, or retire expressions #20943 (PR feat(spec)!: an analytics cube member's sql is a column reference, and the showcase done rate moves to its dataset (#20943) #20998) retires SQL expressions in cube membersql.min/maxover a relationship-path column (account.revenue) stay as the client gave them.declaredFieldTypelooks a column up on the base object only, the same "cannot answer, do not block" tiermeasureResultTypetakes.0.1 + 0.2answers0.3on PostgreSQL native, against0.30000000000000004elsewhere). It was measured once through the route, as the ruling asked, and is reported to the seat. No pin enshrines either side of it.analytics.query(call sites (envelope-caller-census.test.ts) is unchanged: the new pins drive the route or call the service under another name, so no ledger row or declared input moved.Generated by Claude Code