Add PEFT scheduler - #67
Merged
Merged
Conversation
PEFT (Predict Earliest Finish Time; Arabnejad & Barbosa, 2014) is a list scheduler that uses an Optimistic Cost Table (OCT) for lookahead: it ranks tasks by average OCT and places each on the processor minimizing optimistic EFT (earliest finish time plus the OCT lookahead). Brings the implementation from the earlier PEFT branch onto current main, registers it, and adds it to the parametrized scheduler suite.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new PEFTScheduler implementation to Saga’s scheduler suite, exposing it via saga.schedulers and exercising it in the standard scheduler test parametrization.
Changes:
- Introduces
src/saga/schedulers/peft.pyimplementing PEFT with OCT-based ranking and optimistic-EFT processor selection. - Registers
PEFTSchedulerinsrc/saga/schedulers/__init__.pyfor public import. - Adds
PEFTScheduler()to the parametrized scheduler list intests/test_schedulers.py.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/test_schedulers.py | Adds PEFT to the shared scheduler test matrix. |
| src/saga/schedulers/peft.py | Implements PEFT (OCT table + ranking + processor selection). |
| src/saga/schedulers/init.py | Exports PEFTScheduler from the schedulers package. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+109
to
+113
| if schedule is not None: | ||
| start_time = schedule.get_earliest_start_time( | ||
| task=task_name, node=processor.name, append_only=False | ||
| ) | ||
| start_time = max(start_time, min_start_time) |
Comment on lines
+104
to
+108
| best_start_time = 0.0 | ||
| best_end_time = 0.0 | ||
| best_rank = np.inf | ||
| best_processor = None | ||
| for processor in network.nodes: |
| from saga import Schedule, Scheduler, ScheduledTask, TaskGraph, Network, NetworkNode | ||
|
|
||
|
|
||
| def OCT_table(task_graph: TaskGraph, network: Network) -> Dict[str, Dict[str, float]]: |
The paper uses the edge's average communication cost in the OCT (its examples assume uniform inter-processor links). SAGA models heterogeneous per-link speeds, and the OCT is indexed by (task, processor), so both transfer endpoints are known; this implementation uses the actual link cost between the two processors. That reduces to the paper on uniform-communication networks and is the faithful extension to fully-heterogeneous ones. Documented so the divergence reads as intentional.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the PEFT scheduler (Predict Earliest Finish Time; Arabnejad & Barbosa, 2014, https://doi.org/10.1109/TPDS.2013.57), a list-scheduling heuristic that uses an Optimistic Cost Table (OCT) for lookahead:
Unlike HEFT (which is greedy on EFT alone), PEFT's OCT gives it one-step lookahead into downstream cost. It does not use task duplication.
This brings the implementation from the earlier PEFT branch onto current
main(the original PR's branch was deleted), registers it inschedulers/__init__.py, and adds it to the parametrized scheduler suite so it is exercised on the standard DAGs.Type of change
Tests
PEFTScheduleris now intests/test_schedulers.py, run across the diamond/chain/fork graphs. Full suite: 124 passing. Clean under ruff and mypy.Notes for reviewers
Original author: @jasonlopez5. Worth a close read against the paper's OCT recurrence and the optimistic-EFT selection rule.
Note on communication cost in the OCT (intentional divergence from the paper)
PEFT computes the OCT using the edge's average communication cost, because the paper's examples assume uniform inter-processor links (the widely-used reference implementation notes the paper does not even provide a heterogeneous communication matrix). Under uniform links, average and per-link cost are identical.
SAGA models heterogeneous per-link speeds, and the OCT is indexed by
(task, processor), so both endpoints of every transfer are known at computation time. This implementation therefore uses the actual link cost between the two processors rather than the average. It reduces exactly to the paper on uniform-communication networks and is the faithful extension to fully-heterogeneous ones; collapsing SAGA's real link heterogeneity into a single average would blunt PEFT's defining lookahead and unfairly handicap it against schedulers that see the full network. Documented inOCT_table's docstring.