Repository navigation
fix(restapi): serve /shutdown only on a loopback listener (VLN-4882) - #195
Open
persona-cloudflare-app-oracle-se[bot] wants to merge 1 commit into
Open
persona-cloudflare-app-oracle-se[bot] wants to merge 1 commit into
persona-cloudflare-app-oracle-se[bot] wants to merge 1 commit into
Conversation
persona-cloudflare-app-oracle-se
Bot
requested a review
from a team
as a code owner
October 26, 2026 22:52
/shutdown was registered on the :8080 probe server, which listens on all interfaces, so any pod in the cluster could drain and stop ProxySQL with one unauthenticated request (VLN-4882). /shutdown now lives on a separate server bound to 127.0.0.1 on api.shutdown_port (default 8081) and accepts POST only. The probe server keeps /healthz/* on :8080 unchanged. Both servers are stopped after the drain completes. The preStop hook must switch from httpGet to an exec hook on the proxysql container that POSTs to the loopback endpoint. Refs: https://withpersona.atlassian.net/browse/VLN-4882 Oracle-Chat: https://oracle.withpersona.ai/editor/b741a8370f198022bbac60190d4f5cd745b4abbe0fa363d88c43d9d69356754a Co-authored-by: Oracle Editor <oracle-selfedit[bot]@users.noreply.github.com>
persona-cloudflare-app-oracle-se
Bot
force-pushed
the
kyle.barton+editor/2026-10-26/loopback-shutdown-b741a837
branch
from
October 26, 2026 22:56
89707d2 to
bb24149
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On behalf of: @sideshowbandana
Summary
Fixes VLN-4882 (unauthenticated ProxySQL shutdown).
The agent registered
/shutdownon the same HTTP server as the kubelet probes. That server listens on:8080on all interfaces, so any pod in the cluster could reach it on the pod IP. The handler did no auth, method or source check, so one request ranPreStopShutdown()(drain,PROXYSQL PAUSE,PROXYSQL SHUTDOWN) and stopped the pod for good. A source comment wrongly said the API was "only accessible inside the pod itself".Changes
api.port, default 8080, all interfaces): serves/healthz/started|ready|liveonly. The paths, port and handlers are unchanged./shutdownnow returns 404 here.127.0.0.1:api.shutdown_port, default 8081): servesPOST /shutdownonly, and returns 405 for any other method. The bind host is hard-coded to127.0.0.1and cannot be configured. Only the port can be set, viaapi.shutdown_port,AGENT_API_SHUTDOWN_PORTor--api.shutdown_port. Validation rejects ports outside 1–65535 and a port equal toapi.port.SetHTTPServerbecameSetHTTPServers(...).nosemgrepstays onListenAndServewith an accurate justification..github/workflows/golangci-lint.yml: pin setup-go togo.mod(see Testing).config.yaml, CHANGELOG (marked breaking for deployments), CLAUDE.md, and the OrbStack smoke manifests (now using the exec hook).Design: why no agent CLI subcommand
The preStop hook has to stay on the
proxysqlcontainer. The agent runs as a native sidecar, so the kubelet runs the agent container's own preStop only afterproxysqlhas exited, which is too late to drain. An exec hook therefore runs in theproxysql/proxysqlimage, not the agent image (debian:stable-slim, no curl or wget). Theproxysql/proxysqlimage already shipswget: this repo's OrbStack smoke test runskubectl exec -c proxysql -- wget …against the agent. Aproxysql-agent shutdownsubcommand would not be available in that container, so this PR doesn't add one.Recommended preStop hook (on the
proxysqlcontainer):Rollout order (persona-kubernetes)
httpGetpreStop on/shutdowngets a 404 and pods stop without draining. Change the order like this:vX.Y.Z). The tag triggers goreleaser, which publishesghcr.io/persona-id/proxysql-agent:vX.Y.Z.lib/proxysql-stack-{green,blue}/deployment-{core,satellite}.libsonnet,lib/proxysql-blue/…). Switch the hook only after a release containing this change is deployed. Moving it earlier points the hook at a port the old agent doesn't serve.Testing
/shutdownon every method and starts no shutdownshutdownAddralways resolves to loopbackPreStopShutdown(200, or 500 when it fails), GET and PUT return 405,/healthz/*returns 404shutdownHTTPServerstops every registered serverapi.shutdown_portgofmtran locally. TheRun Testsandgolangci-lintworkflows on this PR are the compile, test and lint gate. On the first push (89707d2),Run Testspassed.go-version: stable. That now resolves to a Go newer than golangci-lint v2.13.2 supports, so lint failed in typecheck before it checked any code (export data version 5 is greater than maximum supported version 4). The workflow now pinsgo-version-file: go.mod.🤖 Generated with Oracle
Co-authored-by: kyle.barton kyle.barton@withpersona.com
Co-authored-by: Claude noreply@anthropic.com