feat(hydro_lang): allow for_each on unordered/retried streams with annotations#2953
Draft
shadaj wants to merge 2 commits into
Draft
feat(hydro_lang): allow for_each on unordered/retried streams with annotations#2953shadaj wants to merge 2 commits into
for_each on unordered/retried streams with annotations#2953shadaj wants to merge 2 commits into
Conversation
Deploying hydro with
|
| Latest commit: |
854c9f4
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://445c3d20.hydroflow.pages.dev |
| Branch Preview URL: | https://sandbox-6fa9c2dc-6c21-4fc3-a.hydroflow.pages.dev |
shadaj
added a commit
that referenced
this pull request
Jun 17, 2026
…annotations Changed `Stream::for_each` from requiring `O: IsOrdered` + `R: IsExactlyOnce` to using `ValidCommutativityFor<O>` / `ValidIdempotenceFor<R>` with `AggFuncAlgebra<C, I>`, matching the pattern used by `fold` and `reduce`. - On `TotalOrder + ExactlyOnce` streams, no annotations needed (backward compatible). - On unordered streams, developer provides `commutative = ...` proof. - On streams with retries, developer provides `idempotent = ...` proof. - Added `maybe_observe_for_mut` in the ForEach IR emit path for mut ref handling. - Added compile-fail test verifying the error on NoOrder streams without annotation. Co-authored-by: Infinity 🤖 <infinity@hydro.run> PR: #2953
shadaj
force-pushed
the
sandbox-6fa9c2dc-6c21-4fc3-a7d7-f772462c3c6f
branch
from
0f3cc69 to
987d3f1
Compare
shadaj
force-pushed
the
sandbox-72d22213-017d-4173-b9a8-831d53b21403
branch
from
fe9d437 to
86dfa72
Compare
shadaj
added a commit
that referenced
this pull request
Jun 18, 2026
…annotations Changed `Stream::for_each` from requiring `O: IsOrdered` + `R: IsExactlyOnce` to using `ValidCommutativityFor<O>` / `ValidIdempotenceFor<R>` with `AggFuncAlgebra<C, I>`, matching the pattern used by `fold` and `reduce`. - On `TotalOrder + ExactlyOnce` streams, no annotations needed (backward compatible). - On unordered streams, developer provides `commutative = ...` proof. - On streams with retries, developer provides `idempotent = ...` proof. - Added `maybe_observe_for_mut` in the ForEach IR emit path for mut ref handling. - Added compile-fail test verifying the error on NoOrder streams without annotation. Co-authored-by: Infinity 🤖 <infinity@hydro.run> PR: #2953
shadaj
force-pushed
the
sandbox-6fa9c2dc-6c21-4fc3-a7d7-f772462c3c6f
branch
from
987d3f1 to
f7cd2a9
Compare
luckyworkama
approved these changes
Jun 18, 2026
shadaj
added a commit
that referenced
this pull request
Jun 18, 2026
…annotations Changed `Stream::for_each` from requiring `O: IsOrdered` + `R: IsExactlyOnce` to using `ValidCommutativityFor<O>` / `ValidIdempotenceFor<R>` with `AggFuncAlgebra<C, I>`, matching the pattern used by `fold` and `reduce`. - On `TotalOrder + ExactlyOnce` streams, no annotations needed (backward compatible). - On unordered streams, developer provides `commutative = ...` proof. - On streams with retries, developer provides `idempotent = ...` proof. - Added `maybe_observe_for_mut` in the ForEach IR emit path for mut ref handling. - Added compile-fail test verifying the error on NoOrder streams without annotation. Co-authored-by: Infinity 🤖 <infinity@hydro.run> PR: #2953
shadaj
force-pushed
the
sandbox-6fa9c2dc-6c21-4fc3-a7d7-f772462c3c6f
branch
from
f7cd2a9 to
aaccd5d
Compare
shadaj
force-pushed
the
sandbox-72d22213-017d-4173-b9a8-831d53b21403
branch
from
86dfa72 to
5d602c8
Compare
shadaj
added a commit
that referenced
this pull request
Jun 18, 2026
…annotations Changed `Stream::for_each` from requiring `O: IsOrdered` + `R: IsExactlyOnce` to using `ValidCommutativityFor<O>` / `ValidIdempotenceFor<R>` with `AggFuncAlgebra<C, I>`, matching the pattern used by `fold` and `reduce`. - On `TotalOrder + ExactlyOnce` streams, no annotations needed (backward compatible). - On unordered streams, developer provides `commutative = ...` proof. - On streams with retries, developer provides `idempotent = ...` proof. - Added `maybe_observe_for_mut` in the ForEach IR emit path for mut ref handling. - Added compile-fail test verifying the error on NoOrder streams without annotation. Co-authored-by: Infinity 🤖 <infinity@hydro.run> PR: #2953
shadaj
force-pushed
the
sandbox-72d22213-017d-4173-b9a8-831d53b21403
branch
from
5d602c8 to
1384133
Compare
shadaj
force-pushed
the
sandbox-6fa9c2dc-6c21-4fc3-a7d7-f772462c3c6f
branch
from
aaccd5d to
d5836cc
Compare
…ct inputs in simulator When a closure (map, flat_map, filter, filter_map, inspect, partition) captures a singleton via `by_mut()` and operates on a non-strict input (unordered or at-least-once), the ordering of elements through the closure is non-deterministic. Previously the simulator blindly allowed this without exploring different orderings. - Added `ClosureExpr::has_mut_ref()` helper. - Added `CollectionKind::is_strict()` and `CollectionKind::strict_kind()` helpers to check/create TotalOrder+ExactlyOnce versions of a kind. - Added `observe_for_mut` to the `DfirBuilder` trait (identity in prod, delegates to `observe_nondet` with strict output kind in sim). - Added `maybe_observe_for_mut` free function to deduplicate the emit logic. - In `emit_core`, for Map/FlatMap/FlatMapStreamBlocking/Filter/FilterMap/ Inspect/Partition: if the closure has any mut singleton ref and the input's collection kind is not strict, emit `observe_for_mut` before the operator. - Added passing test `sim_map_with_mut_on_unordered_explores_multiple_states`. - Added ignored test `sim_map_with_mut_on_unordered_top_level` for the top-level bounded case (#2950). Co-authored-by: Infinity 🤖 <infinity@hydro.run> PR: #2951
…annotations Changed `Stream::for_each` from requiring `O: IsOrdered` + `R: IsExactlyOnce` to using `ValidCommutativityFor<O>` / `ValidIdempotenceFor<R>` with `AggFuncAlgebra<C, I>`, matching the pattern used by `fold` and `reduce`. - On `TotalOrder + ExactlyOnce` streams, no annotations needed (backward compatible). - On unordered streams, developer provides `commutative = ...` proof. - On streams with retries, developer provides `idempotent = ...` proof. - Added `maybe_observe_for_mut` in the ForEach IR emit path for mut ref handling. - Added compile-fail test verifying the error on NoOrder streams without annotation. Co-authored-by: Infinity 🤖 <infinity@hydro.run> PR: #2953
shadaj
force-pushed
the
sandbox-72d22213-017d-4173-b9a8-831d53b21403
branch
from
1384133 to
8df21db
Compare
shadaj
force-pushed
the
sandbox-6fa9c2dc-6c21-4fc3-a7d7-f772462c3c6f
branch
from
d5836cc to
854c9f4
Compare
Base automatically changed from
sandbox-72d22213-017d-4173-b9a8-831d53b21403
to
main
June 19, 2026 18:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changed
Stream::for_eachfrom requiringO: IsOrdered+R: IsExactlyOncetousing
ValidCommutativityFor<O>/ValidIdempotenceFor<R>withAggFuncAlgebra<C, I>,matching the pattern used by
foldandreduce.TotalOrder + ExactlyOncestreams, no annotations needed (backward compatible).commutative = ...proof.idempotent = ...proof.maybe_observe_for_mutin the ForEach IR emit path for mut ref handling.Fixes #2949
Co-authored-by: Infinity 🤖 infinity@hydro.run