Make actor runtime device-agnostic (support XPU alongside CUDA)#4307
Open
songhappy wants to merge 1 commit into
Open
Make actor runtime device-agnostic (support XPU alongside CUDA)#4307songhappy wants to merge 1 commit into
songhappy wants to merge 1 commit into
Conversation
Remove three CUDA-only assumptions from Layer 1 so Monarch's actor runtime works on any accelerator supported by torch.accelerator: - device_utils.py: replace /dev/nvidia* scanning with torch.accelerator.device_count() - proc_mesh.py: add XPU env vars (ZE_AFFINITY_MASK, etc.) to env-change monitoring; generalize _torch_cuda_already_initialized to dispatch by torch.accelerator.current_accelerator() - job.py: wrap distributed_telemetry imports in try/except so ProcessJob loads on actors-only builds (USE_TENSOR_ENGINE=0) - setup.py: gate distributed_sql_telemetry behind USE_TENSOR_ENGINE No new environment variables. No Layer 2 changes. CUDA remains the default. Includes test_xpu.py with 17 tests (6 unit/mocked + 11 integration requiring XPU hardware). Tested on Intel Data Center GPU Max (PVC) with torch 2.12.0+xpu, xccl backend, 12 tiles. All upstream tests pass unmodified.
meta-cla
Bot
added
the
CLA Signed
Author
|
@shayne-fletcher would you please kindly have somebody review this PR? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Remove three CUDA-only assumptions from the actor runtime (Layer 1) so Monarch works on any
accelerator supported by torch.accelerator:
by torch.accelerator.current_accelerator()
builds
No Layer 2 changes. CUDA remains the default. 5 files, +600/-31.
Test plan
SPMD elastic env, GRPO convergence, FSDP2 over xccl
Run: pytest python/tests/test_xpu.py -v