Generated `sk-driver` pods cannot be configured to use Azure Workload Identity.

[ghuang0]

Describe the bug

Generated sk-driver pods cannot be configured to use Azure Workload Identity.

The Simulation driver configuration does not support selecting a Kubernetes service account or adding pod labels. Therefore, a driver pod cannot use an annotated service account or set the required azure.workload.identity/use: "true" label.

Steps to reproduce

1. Create a service account configured for Azure Workload Identity:

   kubectl create serviceaccount sk-driver-sa -n simkube
   
   kubectl annotate serviceaccount sk-driver-sa \
     -n simkube \
     azure.workload.identity/client-id=<CLIENT_ID>

2. Run a simulation using an Azure Blob trace:

   skctl run \
     --trace-path azure://<CONTAINER>/<TRACE_PATH> \
     --driver-namespace simkube

3. Inspect the generated driver Job:

   kubectl get jobs -n simkube -o yaml

4. Observe that there is no way to configure the generated pod with:

   spec:
     serviceAccountName: sk-driver-sa

   or:

   metadata:
     labels:
       azure.workload.identity/use: "true"

Expected behavior

The Simulation driver configuration should support a service account and pod labels:

spec:
  driver:
    serviceAccount: sk-driver-sa
    podLabels:
      azure.workload.identity/use: "true"

sk-ctrl should copy these values into the generated driver Job:

spec:
  template:
    metadata:
      labels:
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: sk-driver-sa

Logs or Metrics

The issue is visible in the generated Job manifest. No specific log line is required.

Additional Info

Kubernetes Version: Any
Environment: Kubernetes with Azure Workload Identity enabled
Affected components: sk-ctrl, sk-driver, skctl

[linear-code]

SK-278

[drmorr0]

Hi, Gilbert! Thanks for the report -- the normal way to give the driver access to blob storage is through the driver secrets field in the Simulation CRD. You can pass in the --driver-secrets field to skctl and it will mount in a secret with that name into the driver pod using envFrom. So if your secret has

AZURE_STORAGE_ACCOUNT_NAME: asdf
AZURE_STORAGE_ACCOUNT_KEY: hjkl

those will get passed into the driver pod as environment variables (this is obliquely mentioned in the docs, but it could definitely be more clear!). We unfortunately don't support other methods of auth yet, I would need to think about the right way to add support for this.

[ghuang0]

@drmorr0

Yeah, I noticed the mounted secret could be one way for this. I think for AWS, the integration is much ergonomic compared with Azure, thus filed this issue about it so you could consider adding that support.