refactor(dfir_lang): remove stratum, add push codegen, test sort, sort_by_key

[MingweiSamuel]

No description provided.

[cloudflare-workers-and-pages]

Deploying hydro with    Cloudflare Pages

Latest commit:	110b674
Status:	✅  Deploy successful!
Preview URL:	https://a97ebb0e.hydroflow.pages.dev
Branch Preview URL:	https://mingwei-newpush-sort.hydroflow.pages.dev

View logs

[Copilot]

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

This PR refactors sort / sort_by_key in dfir_lang to remove the stratum-based delay requirement and to support push-side code generation, adding new push-path tests and updating graph snapshots accordingly.

Changes:

• Add push-path surface tests for sort and sort_by_key.
• Update sort / sort_by_key operator codegen to support both pull and push pipelines (and remove DelayType::Stratum reliance).
• Refresh many Graphviz snapshot expectations to reflect the new graph structure (notably removal of intermediate handoff nodes and subgraph reshaping).

Reviewed changes

Copilot reviewed 61 out of 61 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
dfir_rs/tests/surface_push_blocking.rs	Adds new push-path surface tests covering sort and sort_by_key.
dfir_rs/tests/snapshots/surface_sort__sort_by_key@graphvis_mermaid.snap	Updates expected graph output after sort pipeline refactor.
dfir_rs/tests/snapshots/surface_sort__sort_by_key@graphvis_dot.snap	Updates expected graph output after sort pipeline refactor.
dfir_rs/tests/snapshots/surface_sort__sort@graphvis_mermaid.snap	Updates expected graph output after sort pipeline refactor.
dfir_rs/tests/snapshots/surface_sort__sort@graphvis_dot.snap	Updates expected graph output after sort pipeline refactor.
dfir_rs/tests/snapshots/surface_singleton__reduce_singleton_push@graphvis_mermaid.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_singleton__reduce_singleton_push@graphvis_dot.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_singleton__reduce_singleton@graphvis_mermaid.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_singleton__reduce_singleton@graphvis_dot.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_singleton__fold_singleton_push@graphvis_mermaid.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_singleton__fold_singleton_push@graphvis_dot.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_singleton__fold_singleton@graphvis_mermaid.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_singleton__fold_singleton@graphvis_dot.snap	Snapshot refresh reflecting new operator placement/subgraphs.
dfir_rs/tests/snapshots/surface_reduce_keyed__reduce_keyed_tick@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce_keyed__reduce_keyed_tick@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce_keyed__reduce_keyed_static@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce_keyed__reduce_keyed_static@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce_keyed__reduce_keyed_infer_basic@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce_keyed__reduce_keyed_infer_basic@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce__reduce_tick@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce__reduce_tick@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce__reduce_sum@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce__reduce_sum@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce__reduce_static@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_reduce__reduce_static@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_persist__persist_push@graphvis_mermaid.snap	Snapshot refresh reflecting push classification changes and edge rewiring.
dfir_rs/tests/snapshots/surface_persist__persist_push@graphvis_dot.snap	Snapshot refresh reflecting push classification changes and edge rewiring.
dfir_rs/tests/snapshots/surface_persist__persist_pull@graphvis_mermaid.snap	Snapshot refresh reflecting removal of intermediate handoff node.
dfir_rs/tests/snapshots/surface_persist__persist_pull@graphvis_dot.snap	Snapshot refresh reflecting removal of intermediate handoff node.
dfir_rs/tests/snapshots/surface_persist__persist_basic@graphvis_mermaid.snap	Snapshot refresh reflecting removal of intermediate handoff node.
dfir_rs/tests/snapshots/surface_persist__persist_basic@graphvis_dot.snap	Snapshot refresh reflecting removal of intermediate handoff node.
dfir_rs/tests/snapshots/surface_handoff__singleton_with_fold@graphvis_mermaid.snap	Snapshot refresh reflecting simplified graph connections/subgraphs.
dfir_rs/tests/snapshots/surface_handoff__singleton_with_fold@graphvis_dot.snap	Snapshot refresh reflecting simplified graph connections/subgraphs.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_typed_basic@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_typed_basic@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_tick@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_tick@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_static@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_static@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_infer_basic@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold_keyed__fold_keyed_infer_basic@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold__fold_tick@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold__fold_tick@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold__fold_static_join@graphvis_mermaid.snap	Snapshot refresh reflecting rewired tee/handoff structure and subgraphs.
dfir_rs/tests/snapshots/surface_fold__fold_static_join@graphvis_dot.snap	Snapshot refresh reflecting rewired tee/handoff structure and subgraphs.
dfir_rs/tests/snapshots/surface_fold__fold_static@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold__fold_static@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold__fold_sort@graphvis_mermaid.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_fold__fold_sort@graphvis_dot.snap	Snapshot refresh reflecting removal of implicit handoff edges.
dfir_rs/tests/snapshots/surface_difference__diff_static@graphvis_mermaid.snap	Snapshot refresh reflecting simplified diff-to-sort edge flow.
dfir_rs/tests/snapshots/surface_difference__diff_static@graphvis_dot.snap	Snapshot refresh reflecting simplified diff-to-sort edge flow.
dfir_rs/tests/snapshots/surface_difference__diff_multiset_tick_static@graphvis_mermaid.snap	Snapshot refresh reflecting simplified handoff usage/subgraphs.
dfir_rs/tests/snapshots/surface_difference__diff_multiset_tick_static@graphvis_dot.snap	Snapshot refresh reflecting simplified handoff usage/subgraphs.
dfir_rs/tests/snapshots/surface_difference__diff_multiset_static_tick@graphvis_mermaid.snap	Snapshot refresh reflecting simplified handoff usage/subgraphs.
dfir_rs/tests/snapshots/surface_difference__diff_multiset_static_tick@graphvis_dot.snap	Snapshot refresh reflecting simplified handoff usage/subgraphs.
dfir_rs/tests/snapshots/surface_difference__diff_multiset_static@graphvis_mermaid.snap	Snapshot refresh reflecting simplified handoff usage/subgraphs.
dfir_rs/tests/snapshots/surface_difference__diff_multiset_static@graphvis_dot.snap	Snapshot refresh reflecting simplified handoff usage/subgraphs.
dfir_rs/tests/snapshots/surface_cross_singleton__union_defer_tick@graphvis_mermaid.snap	Snapshot refresh reflecting push classification changes and cluster reshaping.
dfir_rs/tests/snapshots/surface_cross_singleton__union_defer_tick@graphvis_dot.snap	Snapshot refresh reflecting push classification changes and cluster reshaping.
dfir_lang/src/graph/ops/sort_by_key.rs	Implements push-side codegen path and removes stratum delaytype behavior.
dfir_lang/src/graph/ops/sort.rs	Implements push-side codegen path and removes stratum delaytype behavior; updates doc wording.

Comments suppressed due to low confidence (2)

dfir_rs/tests/surface_push_blocking.rs:1

• These tests validate push-side sorting for a single finite input sequence, but they don't exercise tick boundaries (the key behavioral contract for these blocking operators). Add a push-side test that feeds items across multiple ticks and asserts each tick is sorted independently (e.g., two ticks producing two separately sorted batches).
  dfir_rs/tests/surface_push_blocking.rs:1
• These tests validate push-side sorting for a single finite input sequence, but they don't exercise tick boundaries (the key behavioral contract for these blocking operators). Add a push-side test that feeds items across multiple ticks and asserts each tick is sorted independently (e.g., two ticks producing two separately sorted batches).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.