Skip to content

[YUNIKORN-3506] Fix child queue preemptable resource calculation in quota preemption - #1189

Closed
hedger9487 wants to merge 1 commit into
apache:masterfrom
hedger9487:YUNIKORN-3506
Closed

hedger9487 wants to merge 1 commit into
apache:masterfrom
hedger9487:YUNIKORN-3506

Conversation

@hedger9487

Copy link
Copy Markdown
Contributor

Description

In getChildQueuesPreemptableResource(), the first pass computes each child queue's preemptable resource above its guaranteed quota (allocated - guaranteed), which had three issues:

  1. Premature skipping via StrictlyGreaterThanOrEqualsOnlyExisting: Checking guaranteed.StrictlyGreaterThanOrEqualsOnlyExisting(allocated) only inspects resource types configured in guaranteed. If a child queue meets its guaranteed quota on configured resource types (e.g., cpu) while using an unguaranteed resource type (e.g., gpu), the entire child queue was skipped.
  2. Dropping unguaranteed resource types: Computing resources.SubOnlyExisting(guaranteed, allocated) only retained keys present in guaranteed, silently dropping any allocated resource types not configured in guaranteed.
  3. Treating under-guaranteed usage as preemptable: For resource types where allocated < guaranteed, guaranteed - allocated > 0 was copied directly into preemptableResource instead of being clamped/removed, causing a child queue below its guaranteed quota to be treated as over-quota.

This PR fixes getChildQueuesPreemptableResource() by computing resources.SubOnlyExisting(allocated, child.GetGuaranteedResource()) and removing any resource types that are either not present in parentPreemptableResource or have a non-positive excess (v <= 0).

Type of change

  • Bug Fix
  • Improvement
  • Feature
  • Refactoring
  • Documentation

Jira issue

Jira ID : https://issues.apache.org/jira/browse/YUNIKORN-3506

  • I have created a Jira issue for this pull request.
  • The Jira ID is part of the title of this pull request.

AI Tooling

If an AI tool was used:

  • The PR includes the phrase "Generated by Antigravity", where Antigravity is the name of the AI tool used.
  • My use of AI contributions follows the ASF legal policy.

Check https://www.apache.org/legal/generative-tooling.html for details.

How has this been tested?

  • Added table-driven unit test TestGetChildQueuesPreemptableResource_UnderGuaranteedAndUnguaranteedTypes in pkg/scheduler/objects/quota_preemptor_test.go.
  • Full repo unit tests & deadlock detection passed with make test (-race -tags deadlock).
  • Clean lint verified with CI linter (GOTOOLCHAIN=go1.26.8 tools/golangci-lint-v2.10.1/golangci-lint run).
  • 100% Patch Coverage & 100% Mutation Score verified locally.

Questions:

  • The change needs documentation, a pull request for apache/yunikorn-site repository will be created.
  • There is breaking changes for older versions: jira is tagged with release-notes label.
  • The licenses files needs to be updated.

Screenshots or other details

Generated by hedger9487 with assistance from Antigravity.

@codecov

codecov Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.31%. Comparing base (0ef9fd7) to head (bab2625).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1189      +/-   ##
==========================================
- Coverage   83.31%   83.31%   -0.01%     
==========================================
  Files         104      104              
  Lines       14652    14646       -6     
==========================================
- Hits        12208    12202       -6     
  Misses       2146     2146              
  Partials      298      298              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@manirajv06 manirajv06 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants