[rawadhossain]

Description

This PR adds the Controller Internals section, fifth of five planned dashboard sections for node-readiness-controller.

Questions it answers:

• Is the controller up and reachable?
• Is it keeping up with incoming reconcile work?
• Are workqueues building up or taking too long to process?
• Are reconciliations failing?
• Which controller version is currently running?

Most of these panels use metrics from controller-runtime and kube-state-metrics.

Panels in this Section

• Controller Up - controller availability.
• Workqueue Depth - queued reconcile work per controller.
• Workqueue Latency (p99) - p99 time items spend waiting in the queue.
• Reconcile Errors - failed reconcile attempts per second.
• Bootstrap Completions - total bootstrap completions since the controller started.
• Build Info - deployed version and pod identity.

Screenshot

Video demo

Section.5.mp4

Related to Issue #459

Type of Change

/kind feature

Testing

• Verified all panels render correctly against live data in a kind + kwok test cluster
• Confirmed the setup is reproducible from a clean cluster

[netlify]

✅ Deploy Preview for node-readiness-controller canceled.

Name	Link
🔨 Latest commit	9dcb599
🔍 Latest deploy log	https://app.netlify.com/projects/node-readiness-controller/deploys/6a9fc8afc77dd0000832cae1

[rawadhossain]

/cc @AvineshTripathi

[AvineshTripathi]

initial look! I will need some time here to understand. Too many variables here!

[rawadhossain]

Hey @AvineshTripathi, here's a quick breakdown so its easier to skim.

{
  "id": 10,                    → unique panel id, ignore it
  "type": "timeseries",          → line graph (vs "stat" = big number, "table" = table)
  "title": "Workqueue Depth",
  "gridPos": {...},               → x/y position + size on the grid, just layout
  
  "targets": [
    { "expr": "sum by (controller) (workqueue_depth{job=\"$job\"})" }
  ],                 → the actual PromQL query — this is the only line that defines what the panel shows
 
 "fieldConfig": {
    "defaults": {
      "unit": "short",              → how the number's displayed
      "min": 0, "max": 1200,   → axis bounds
      "thresholds": {
        "steps": [            → color bands, kicks in once value crosses these
          green → null (default),
          yellow → 500,
          red → 1000
        ]
      }                        
    },
  
  "overrides": [
      "node" line → solid,
      "nodereadiness-controller" line → dashed
    ]               → per-series exceptions, so the two lines are visually distinguishable even if same color
  },
  "options": {...}        → position, tooltip behavior, cosmetic only
}

so overall, it's just: one query per panel, wrapped in a bit of formatting (colors, units, layout) that repeats the same way across all of them.

[AvineshTripathi]

am not an expert in this but it LGTM

[ajaysundark]

/lgtm
/approve

[kubernetes-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ajaysundark, rawadhossain

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [ajaysundark]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment