Small update - #69
Open
JasonChamorro wants to merge 18 commits into
Open
Conversation
Feature/throughput
…t in previous merge)
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends SAGA’s online scheduling experiment and policy suite to support additional stochastic rescheduling behaviors (notably 1%/5% random rescheduling and checkpoint-based rescheduling), and augments experiment outputs to record additional metrics.
Changes:
- Clamp WFCommons-generated network sizes to a minimum of 2 nodes to avoid downstream CCR scaling failures on single-node traces.
- Add new online rescheduling policies (random 1%/5% and checkpoint-based), and track
reschedule_countin online environments/policies. - Update throughput experiment scripts to emit Throughput/Makespan/RescheduleCount and generate separate heatmaps for throughput vs makespan; add a helper script to backfill new checkpoint policies.
Reviewed changes
Copilot reviewed 10 out of 66 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/saga/schedulers/stochastic/data/wfcommons.py | Clamp generated network node counts to >= 2 for stochastic WFCommons traces. |
| src/saga/schedulers/data/wfcommons.py | Clamp generated network node counts to >= 2 for deterministic WFCommons traces. |
| src/saga/schedulers/online/policy/reschedule.py | Add new random (1%/5%) and checkpoint reschedule policies; increment reschedule_count across policies. |
| src/saga/schedulers/online/policy/init.py | Re-export newly added policies from the online policy package. |
| src/saga/schedulers/online/environment/frontier.py | Reset reschedule_count in frontier environment resets. |
| src/saga/schedulers/online/environment/init.py | Add and reset reschedule_count in the base online environment. |
| src/saga/init.py | Add TaskGraph.width() utility using transitive closure + bipartite matching. |
| scripts/experiments/throughput_experiment/run.py | Expand policy grid, record Makespan/RescheduleCount, and adjust config selection by branch. |
| scripts/experiments/throughput_experiment/run_new_policies.py | New script to append results for new checkpoint policies into existing stochastic CSVs. |
| scripts/experiments/throughput_experiment/analyze.py | Produce separate throughput/makespan heatmaps using normalized ratios. |
Comments suppressed due to low confidence (2)
src/saga/schedulers/online/policy/reschedule.py:433
CheckpointRescheduleMidmutatesself.intervalsas it triggers checkpoints, but doesn't overridereset(). If a policy instance is reused across multiple runs, it won't reschedule after the first run. Also, the docstring currently describes quarterly checkpoints even though this policy only triggers at 50%.
src/saga/schedulers/online/policy/reschedule.py:486CheckpointReschedule10mutatesself.intervalsbut doesn't overridereset(), so reusing a policy instance across runs will disable rescheduling after the first run. The docstring also describes 25/50/75% checkpoints even though this policy uses 10% increments; adding a small guard for empty task graphs avoids ZeroDivisionError.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+376
to
+387
| """Reschedules at checkpoint intervals 0% (initial schedule), 25%, 50%, 75%""" | ||
| def __init__(self): | ||
| super().__init__() | ||
| self.intervals = [0.25,0.50,0.75] | ||
|
|
||
| def evaluate_reschedule(self, environment: "Environment") -> bool: | ||
| if not self.intervals: | ||
| return False | ||
| if len(environment.finished_tasks)/len(environment.task_graph.tasks) >= self.intervals[0]: | ||
| self.intervals.pop(0) | ||
| return True | ||
| return False |
Comment on lines
+9
to
13
| plus reschedule/conditional/random10/random25/random50/random5/random1/checkpoint_quarterly/ | ||
| checkpoint_mid/checkpoint_10 in the stochastic regime (offline it equals static). | ||
| FastestNode and MaxTP are standalone heuristics run with no policy layered on top. | ||
|
|
||
| Evaluation is parallelized across instances with a small process pool (capped at 4). |
Comment on lines
+66
to
+82
| def run(branch: str, n_instances: int, n_seeds: int, workers: int) -> None: | ||
| out = resultsdir / f"{branch}_stochastic.csv" | ||
| lock_path = out.with_suffix(".csv.lock") | ||
|
|
||
| # Resume: skip any (Workflow, CCR, Instance) that already has every seed x new-config | ||
| # row, so a crash or a second invocation of this backfill doesn't duplicate rows. | ||
| new_names = new_config_names(branch) | ||
| expected_per_instance = n_seeds * len(new_names) | ||
| finished_keys: set = set() | ||
| if out.exists(): | ||
| prev = pd.read_csv(out) | ||
| prev_new = prev[prev["Scheduler"].isin(new_names)] | ||
| counts = prev_new.groupby(["Workflow", "CCR", "Instance"]).size() | ||
| finished_keys = {k for k, n in counts.items() if n >= expected_per_instance} | ||
| logging.warning("resuming: %d instances already have the new policies in %s", | ||
| len(finished_keys), out.name) | ||
|
|
…, for MaxTP construction
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adding results for 1% and 5% rescheduling
Type of change
Checklist
mainand up to date with it (no conflicts)uv run pytest tests/ --timeout=120passes locallyuv run mypy src/saga --ignore-missing-importsis cleanuv run ruff check src/sagaanduv run ruff format --check src/sagaare cleanpyproject.toml/uv.lockchanges, or version bumpNotes for reviewers