Skip to content

[joblib] Add opt-in autoscaling to the default 'ray' actor pool #64959

Description

@OneSizeFitsQuorum

Problem

The "ray" joblib backend uses a fixed pool of num_cpus=0 actors. On an
elastic cluster, those actors expose no CPU demand to the autoscaler and remain
alive for the lifetime of the pool.

Proposed behavior

Add an experimental autoscale=True mode to the existing backend rather than a
second joblib backend.

  • Autoscaling actors request one CPU, so pending placements drive cluster
    scale-up.
  • Batches wait in a FIFO queue and are submitted only to actors that have
    completed startup.
  • Pending actors express resource demand but do not own work. This allows
    progress when max_size exceeds the cluster's maximum CPU capacity.
  • Idle actors are reaped down to min_size, allowing cluster scale-down.
  • n_jobs remains joblib's concurrency limit; max_size may lower but not
    raise it.
  • apply_async and map_async remain asynchronous; result ordering and error
    propagation continue through the existing AsyncResult/ResultThread.
  • autoscale=False remains the default fixed-pool behavior.

Example:

from ray.util.joblib import register_ray

register_ray(autoscale=True, max_size=64, idle_timeout_s=60)

Implementation status

Implemented in #64957 with a single dispatcher thread:

  • startup, ready, and running actor states are dispatcher-owned;
  • ObjectRef callbacks only wake the dispatcher;
  • no separate controller or second backend is introduced.

Local validation covers constrained CPU capacity, asynchronous APIs, error
propagation and reuse, shutdown, idle reaping, AutoscalingCluster scale-up, and
KubeRay worker scale-up/scale-down.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions