Problem
The "ray" joblib backend uses a fixed pool of num_cpus=0 actors. On an
elastic cluster, those actors expose no CPU demand to the autoscaler and remain
alive for the lifetime of the pool.
Proposed behavior
Add an experimental autoscale=True mode to the existing backend rather than a
second joblib backend.
- Autoscaling actors request one CPU, so pending placements drive cluster
scale-up.
- Batches wait in a FIFO queue and are submitted only to actors that have
completed startup.
- Pending actors express resource demand but do not own work. This allows
progress when max_size exceeds the cluster's maximum CPU capacity.
- Idle actors are reaped down to
min_size, allowing cluster scale-down.
n_jobs remains joblib's concurrency limit; max_size may lower but not
raise it.
apply_async and map_async remain asynchronous; result ordering and error
propagation continue through the existing AsyncResult/ResultThread.
autoscale=False remains the default fixed-pool behavior.
Example:
from ray.util.joblib import register_ray
register_ray(autoscale=True, max_size=64, idle_timeout_s=60)
Implementation status
Implemented in #64957 with a single dispatcher thread:
- startup, ready, and running actor states are dispatcher-owned;
- ObjectRef callbacks only wake the dispatcher;
- no separate controller or second backend is introduced.
Local validation covers constrained CPU capacity, asynchronous APIs, error
propagation and reuse, shutdown, idle reaping, AutoscalingCluster scale-up, and
KubeRay worker scale-up/scale-down.
Problem
The
"ray"joblib backend uses a fixed pool ofnum_cpus=0actors. On anelastic cluster, those actors expose no CPU demand to the autoscaler and remain
alive for the lifetime of the pool.
Proposed behavior
Add an experimental
autoscale=Truemode to the existing backend rather than asecond joblib backend.
scale-up.
completed startup.
progress when
max_sizeexceeds the cluster's maximum CPU capacity.min_size, allowing cluster scale-down.n_jobsremains joblib's concurrency limit;max_sizemay lower but notraise it.
apply_asyncandmap_asyncremain asynchronous; result ordering and errorpropagation continue through the existing
AsyncResult/ResultThread.autoscale=Falseremains the default fixed-pool behavior.Example:
Implementation status
Implemented in #64957 with a single dispatcher thread:
Local validation covers constrained CPU capacity, asynchronous APIs, error
propagation and reuse, shutdown, idle reaping, AutoscalingCluster scale-up, and
KubeRay worker scale-up/scale-down.