Skip to content

Gateway Programmed=True does not detect when a GatewayConfiguration change fails to reach the LB Deployment #255

Description

@zolug

Problem

Gateway Programmed=True is only re-asserted after a successful reconcile. If
reconcileLBDeployment's r.Update() on the Deployment fails with a plain
(non-conflict, non-permanentDeploymentError) error — e.g. a persistent API
server issue — the reconciler returns early and step 7 (setting Programmed)
is never reached. Programmed keeps its last True value indefinitely; nothing
on the Gateway object signals that a GatewayConfiguration change never
propagated to the Deployment.

Gateway.Status.Conditions[Programmed].ObservedGeneration tracks
Gateway.Generation, not GatewayConfiguration's generation, so it cannot
detect this: Gateway.Generation doesn't change when only the referenced
GatewayConfiguration is edited.

Note: this is Deployment-object drift (spec never applied), not Pod-rollout
lag — the existing "Pods may still be initializing" caveat on the Programmed
message already covers the rollout-lag case and is unaffected by this issue.

Not proposed

  • Flipping Programmed=False on any reconcileLBDeployment error (including
    transient ones) — too noisy/incorrect; Programmed=True is meant to reflect
    "traffic-handling infra in place," and a passing API hiccup shouldn't flap it.
  • A status condition on GatewayConfiguration itself — a GatewayConfiguration
    can be referenced by multiple Gateways in the same namespace
    (LocalParametersReference has no Namespace field, so cross-namespace
    reference is impossible, but same-namespace multi-Gateway reuse is), so a
    single condition on it can't represent multiple Gateways' per-reconcile state.
  • Repurposing Gateway API's Ready condition type — spec-reserved, "should
    not be used by implementations."
  • A new field on GatewayStatus, or on metav1.Condition, to carry the
    GatewayConfiguration generation — both are fixed by the Gateway API/
    Kubernetes API conventions; there's no room to add a field.
  • An annotation on the Gateway recording the observed GatewayConfiguration
    generation — the Gateway's metadata lifecycle belongs to the user, not the
    controller; writing to it is not viable (same objection as for other
    controller-authored metadata on user-owned objects).

Directions to evaluate

  • Include the referenced GatewayConfiguration's generation in the Programmed
    message itself, so staleness is detectable without a new condition type or
    new field. Downside: makes the message content-variable, with some cost to
    tests currently asserting on a literal message.
  • A new, Meridio-2-specific condition type carrying
    GatewayConfiguration-generation currency. No Gateway API
    compatibility constraint on custom condition types/semantics.
  • Longer-term/separate scope: whether Programmed (or a new condition) should
    ever reflect actual LB Pod status. Not investigated here — no confirmed
    survey of how other Gateway API implementations handle this scope.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions