[YUNIKORN-3483] Reinstate Partition state transitions

[manirajv06]

Description

Reinstated the partition object state transitions. State would be marked as "draining" before heading to deletion process which could prevent any activity imposed on it. Activity could be either add app or node, update allocation etc. In case state already marked as "draining", updating the properties of the same partition object would reverse the state to "active".

Removed "stopped" state as it is not really used anymore. Updated Queue object state based on the same approach.

Type of change

Please delete options that are not relevant.

• Bug Fix
• Improvement
• Feature
• Refactoring
• Documentation

Jira issue

Jira ID : https://issues.apache.org/jira/browse/YUNIKORN-3483

• I have created a Jira issue for this pull request.
• The Jira ID is part of the title of this pull request.

AI Tooling

If an AI tool was used:

• The PR includes the phrase "Generated by <tool>", where <tool> is the name of the AI tool used.
• My use of AI contributions follows the ASF legal policy.

Check https://www.apache.org/legal/generative-tooling.html for details.

How has this been tested?

• New unit tests were added to cover new or changed code paths.
• make test_all was run, and no failures reported.
• A pull request will be opened for new e2e tests (apache/yunikorn-k8shim repository).

Questions:

• The change needs documentation, a pull request for apache/yunikorn-site repository will be created.
• There is breaking changes for older versions: jira is tagged with release-notes label.
• The licenses files needs to be updated.

Screenshots or other details

[rjgoyln]

I had three notes below: one blocking concern around draining queues no longer being scheduled, and two smaller points on the partition state handling.

Separately, Stop/Stopped are still defined in object_state.go and tested in object_state_test.go, although there don't seem to be any callers left. Removing them from there would make the implementation consistent with the description.

Just some thoughts from my side — happy to be corrected if I’m missing something.

[wilfred-s]

Lets not remove the stopped state. It might not fully fit in for the partition at the moment but I logged YUNIKORN-3490 to expose the stopped state as it was supposed to be exposed for the queues.

[manirajv06]

@rjgoyln @wilfred-s

Summary of the changes:

Pending allocations in Queue would be processed as earlier even when it is draining. Partition would be marked as draining when deletion happens and proceed with the actual clean up process - Queue, Nodes etc. Retained IsStopped() for Partition as is but it would not be used or set for now. We MAY set in future but no plans as of now unlike setting it for Queue through YUNIKORN-3490.

[rjgoyln]

LGTM! Just two small nits on the test coverage.

[codecov]

Codecov Report

❌ Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 83.26%. Comparing base (c99a01f) to head (4f23a87).

Files with missing lines	Patch %	Lines
pkg/scheduler/objects/queue.go	66.66%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1174      +/-   ##
==========================================
- Coverage   83.27%   83.26%   -0.02%     
==========================================
  Files         104      104              
  Lines       14460    14463       +3     
==========================================
+ Hits        12041    12042       +1     
- Misses       2131     2132       +1     
- Partials      288      289       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wilfred-s]

LGTM