Skip to content

shiftstack/liveliness: skip TLS verify for dev-install CA workaround - #83891

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
shiftstack:workaround_liveliness
Aug 26, 2026
Merged

shiftstack/liveliness: skip TLS verify for dev-install CA workaround#83891
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
shiftstack:workaround_liveliness

Conversation

@dlaw4608

@dlaw4608 dlaw4608 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Liveliness checks have been failing on mecha-central since the shiftstack/tooling:scripts image picked up Python 3.14, which enforces stricter TLS validation. The cloud's simpleca CA was generated before dev-install shiftstack/dev-install#238 and doesn't mark Basic Constraints as critical, so the liveliness step fails before it can do any real health checks.

This change sets verify: false on clouds.yaml in the liveliness step only, so monitoring works again while we regenerate certs on affected clouds. This is a workaround that will be reverted once mecha-central has compliant certs and the CI secret is updated.

Summary by CodeRabbit

The ShiftStack CI liveliness check now sets verify: false for the configured OpenStack cloud in clouds.yaml. This prevents Python 3.14 TLS validation from rejecting affected SimpleCA certificates during the liveliness step.

  Python 3.14 in shiftstack/tooling:scripts enforces strict CA validation,
  breaking liveliness on clouds with pre-dev-install#238 simpleca certs.
  Set verify: false on clouds.yaml in the liveliness step only until certs
  are regenerated.

Signed-off-by: Daniel Lawton <dlawton@redhat.com>
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: f511825a-e5be-46a0-979c-0d9e65e6f899

📥 Commits

Reviewing files that changed from the base of the PR and between 0822a32 and 5d3235f.

📒 Files selected for processing (1)
  • ci-operator/step-registry/shiftstack/liveliness/shiftstack-liveliness-commands.sh

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.


Walkthrough

The ShiftStack liveliness command updates the selected OpenStack cloud in clouds.yaml to disable TLS certificate verification before the liveliness check runs.

Changes

ShiftStack liveliness

Layer / File(s) Summary
Disable TLS verification
ci-operator/step-registry/shiftstack/liveliness/shiftstack-liveliness-commands.sh
The command sets the selected cloud’s verify setting to false before running the liveliness check.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 5d323

The change is narrowly scoped to the liveliness step, and no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring check was indeterminate for this PR — some files could not be analyzed in time. Not blocking.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR changes only one shell script, adding a yq update and comment; the exact diff introduces no Ginkgo test titles or dynamic test-name content.
Test Structure And Quality ✅ Passed The patch changes only a Bash liveliness command script and adds a yq update; it introduces no Ginkgo It blocks or test setup, waits, or assertions.
Microshift Test Compatibility ✅ Passed The committed patch modifies only a ShiftStack shell script and adds no Ginkgo e2e tests or MicroShift-incompatible API references.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR changes only one shell script to update clouds.yaml; the diff adds no Ginkgo e2e tests or multi-node assumptions.
Topology-Aware Scheduling Compatibility ✅ Passed The PR changes only a ShiftStack liveliness shell step to edit clouds.yaml TLS verification; it adds no manifests, controllers, or scheduling constraints.
Ote Binary Stdout Contract ✅ Passed The patch changes only a Bash liveliness step. It adds an in-place yq update and no OTE binary, main/suite setup, or stdout write in Go code.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The PR changes only one liveliness shell script and adds no Ginkgo e2e tests, IPv4 assumptions, or external connectivity requirements.
No-Weak-Crypto ✅ Passed The diff only adds a scoped OpenStack TLS certificate-verification bypass; it introduces no MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB, custom crypto, or secret comparison.
Container-Privileges ✅ Passed The PR adds only a yq TLS setting in a shell script. The changed diff and associated manifests add no privileged, host namespace, SYS_ADMIN, root, or allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs ✅ Passed The committed change only edits clouds.yaml with yq; it adds no logging and does not print passwords, tokens, identifiers, hostnames, or customer data.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the ShiftStack liveliness change and the TLS verification workaround.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from gryf and mandre August 24, 2026 11:45
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@dlaw4608: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
periodic-ci-shiftstack-ci-main-liveliness-mecha-central N/A periodic Registry content changed

Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-ci

openshift-ci Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

@dlaw4608: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@IlanZuckerman IlanZuckerman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Please make sure to open a follow up Jira for a permanent fix, in which you state that this line needs to be removed.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 25, 2026
fi

# TODO: Remove once dev-install clouds have Python 3.13+ compatible CA certs (shiftstack/dev-install#238).
yq --yaml-output --in-place ".clouds[\"${OS_CLOUD}\"].verify = false" "${SHARED_DIR}/clouds.yaml"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might actually be better to modify the secrets directly for the impacted clouds, so that we avoid disabling cert validation on all environments.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mandre I might be wrong, but from what i saw, today only liveliness-mecha-central uses this step, so the PR doesn’t touch other clouds. So this temporal WA is narrower until the certs are fixed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mandre that is a good point, and @IlanZuckerman you are also correct, at the moment only one test config references the workflow: shiftstack-liveliness step in the release repo. So I think as this is a temporary fix, and as far as I know no other jobs are being configured to use the same workflow: shiftstack-liveliness eg liveliness-hwoffload, we should be good to keep the current workaround as is? WDYT

@mandre mandre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@openshift-ci

openshift-ci Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dlaw4608, IlanZuckerman, mandre

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 26, 2026
@dlaw4608

Copy link
Copy Markdown
Contributor Author

/pj-rehearse ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@dlaw4608: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Aug 26, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit cb6624b into openshift:main Aug 26, 2026
10 checks passed
@openshift-merge-bot
openshift-merge-bot Bot deleted the workaround_liveliness branch August 26, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants