shiftstack/liveliness: skip TLS verify for dev-install CA workaround - #83891
Conversation
Python 3.14 in shiftstack/tooling:scripts enforces strict CA validation, breaking liveliness on clouds with pre-dev-install#238 simpleca certs. Set verify: false on clouds.yaml in the liveliness step only until certs are regenerated. Signed-off-by: Daniel Lawton <dlawton@redhat.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review. WalkthroughThe ShiftStack liveliness command updates the selected OpenStack cloud in ChangesShiftStack liveliness
Estimated code review effort: 1 (Trivial) | ~2 minutes Merge Risk: ⚪ Minimal · up to The change is narrowly scoped to the liveliness step, and no actionable merge-blocking risk remains beyond normal checks and review. 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[REHEARSALNOTIFIER]
Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals. Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@dlaw4608: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
IlanZuckerman
left a comment
There was a problem hiding this comment.
/lgtm
Please make sure to open a follow up Jira for a permanent fix, in which you state that this line needs to be removed.
| fi | ||
|
|
||
| # TODO: Remove once dev-install clouds have Python 3.13+ compatible CA certs (shiftstack/dev-install#238). | ||
| yq --yaml-output --in-place ".clouds[\"${OS_CLOUD}\"].verify = false" "${SHARED_DIR}/clouds.yaml" |
There was a problem hiding this comment.
It might actually be better to modify the secrets directly for the impacted clouds, so that we avoid disabling cert validation on all environments.
There was a problem hiding this comment.
@mandre I might be wrong, but from what i saw, today only liveliness-mecha-central uses this step, so the PR doesn’t touch other clouds. So this temporal WA is narrower until the certs are fixed.
There was a problem hiding this comment.
@mandre that is a good point, and @IlanZuckerman you are also correct, at the moment only one test config references the workflow: shiftstack-liveliness step in the release repo. So I think as this is a temporary fix, and as far as I know no other jobs are being configured to use the same workflow: shiftstack-liveliness eg liveliness-hwoffload, we should be good to keep the current workaround as is? WDYT
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dlaw4608, IlanZuckerman, mandre The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/pj-rehearse ack |
|
@dlaw4608: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Liveliness checks have been failing on
mecha-central since theshiftstack/tooling:scripts image picked up Python 3.14, which enforces stricter TLS validation. The cloud's simpleca CA was generated before dev-install shiftstack/dev-install#238 and doesn't mark Basic Constraints as critical, so the liveliness step fails before it can do any real health checks.This change sets
verify: falseon clouds.yaml in the liveliness step only, so monitoring works again while we regenerate certs on affected clouds. This is a workaround that will be reverted once mecha-central has compliant certs and the CI secret is updated.Summary by CodeRabbit
The ShiftStack CI liveliness check now sets
verify: falsefor the configured OpenStack cloud inclouds.yaml. This prevents Python 3.14 TLS validation from rejecting affected SimpleCA certificates during the liveliness step.