[codex] Prepare sorted q2 global ranks lazily

[snissn]

Summary

• Prepare sorted grouped-distinct q2 global rank maps per local dictionary instead of per-row GlobalCodes.
• Translate local codes to global ranks at the sorted iterator/reducer boundary.
• Split sorted q2 post-prepare diagnostics so dictionary/rank build and local-rank map fill are visible separately.
• Keep the production dense q2 rank-fill behavior untouched.

Refs #3324.

Claim Boundary

This is no-aggregate one_shot_end_to_end typed-column q2 setup work only. It is not metadata acceleration, not a hot-prepared headline result, and not a TreeDB-vs-ClickHouse claim.

Important: current 1M JSONBench production q2 still selects the dense grouped-count-distinct path, not this PR's sorted grouped-distinct lazy/local-rank path. Treat the 1M production cell below as current-head correctness/no-regression evidence and dense-path frontier evidence, not as a sorted-path runtime win.

Validation

Current head: 20e74d45515b6cfc1ed3f03be7d895e3845bc5d0, merged with current main after #3412.

Local validation passed:

• GOWORK=off go test ./TreeDB/collections -run 'TestTypedColumnQ2|Q2|q2' -count=1
• GOWORK=off go test ./cmd/unified_bench -count=1
• git diff --check

Earlier sorted-path validation also passed:

• GOWORK=off go test ./TreeDB/collections -run "TestTypedColumnQ2SortedGroupedDistinct(Streaming1950|LocalDictionariesAndEmptyValues1950|Fallback1950|PrefixMismatchFallback1950)$" -count=1
• GOWORK=off go test ./TreeDB/collections -run "^$" -bench "^BenchmarkTypedColumnQ2SortedGroupedDistinct1950/prepared/sorted_prefix$" -benchtime=1x -count=1
  • smoke result: 1103717 ns/op, 608 B/op, 5 allocs/op; not a long-run performance claim.

Current 1M q2 Evidence

Run used this PR plus JSONBench draft PR #40 for the local-rank reporting fields.

Artifacts:

• report JSON: /mnt/fast4tb/gomap-profiles/q2-pr3413-current-head-20260630_171446/jsonbench_q2_1m_full_prepared_noagg/report.json
• report markdown: /mnt/fast4tb/gomap-profiles/q2-pr3413-current-head-20260630_171446/jsonbench_q2_1m_full_prepared_noagg/report.md
• raw result: /mnt/fast4tb/gomap-profiles/q2-pr3413-current-head-20260630_171446/jsonbench_q2_1m_full_prepared_noagg/1m_column-store-full-prepared_one_shot_end_to_end_no_aggregate_metadata_json_full_q2/result.json

1M q2 one_shot_end_to_end / no_aggregate_metadata:

• total/setup/post-prepare/run/render-hash: 57.607 / 40.626 / 16.865 / 16.938 / 0.043 ms
• rows scanned/matched/reduced: 1,000,000 / 954,611 / 954,611
• result groups/hash: 13, b63b8e1013c918fcda7c299ef64beb20c4988854cf0671a83803b6951d795860
• aggregate metadata used: false
• storage durable bytes, WAL excluded: 135,830,268
• physical bytes scanned / decoded payload bytes: 15,991,989 / 32,925,252
• row/doc materializations: 0 / 0; JSON reconstruction: false

Path caveat:

• dense_group_count_distinct_used=true
• sort_layout=time_us, fallback_reason=none
• sorted global-local-rank fields are zero/absent in this production cell because the sorted path was not selected
• dense split: dense distinct global rank 15.934 ms, collect refs 5.102 ms, build shards 10.832 ms, part local rank 0.872 ms, refs 627,646, global ranks 180,100

Notes

• This does not touch WAL, value-log, durability, metadata acceleration, or on-disk format.
• The production q2 frontier remains dense grouped-count-distinct rank/ref/build work unless a production cell is made to select sorted grouped-distinct.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ce70c521-c815-4cc7-a1d5-9d62a0178f35

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/3324-q2-sorted-rank-setup

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[snissn]

Current-main refresh status:

• Head 20e74d45515b6cfc1ed3f03be7d895e3845bc5d0 is now green across latest-head GitHub checks.
• PR remains draft by design. The current 1M production q2 evidence still selects dense grouped-count-distinct, so this is sorted-path algorithmic/reference work, not a production q2 runtime claim.
• JSONBench reporting companion remains draft at [codex] report q2 global local rank diagnostics JSONBench#40.

[snissn]

Closing this stale sorted-path draft in favor of the refreshed current-base draft #3433. The q2 post-prepare diagnostic split has already landed on main, and #3433 now carries the sorted grouped-distinct lazy/local-rank algorithm boundary plus current JSONBench context. Keeping this closed avoids two open drafts for the same sorted-path lane.