[sammaji]

Summary

Adds config schema and Helm chart support for AWS SageMaker real-time endpoints as a provider. Users can now declare a sagemaker provider with a sagemaker_key_config block in config.schema.json and the Helm chart's values.schema.json. Without this, the provider block is rejected by schema validation.

Changes

• Added a sagemaker provider entry to transports/config.schema.json, backed by a new provider_with_sagemaker_config definition. It supports the same provider-level options as the other providers: network_config, concurrency_and_buffer_size, proxy_config, raw request/response flags, ignore_provider_cost, injected_tools, passthrough, and custom_provider_config. At least one key is required.
• Added a sagemaker_key definition that extends base_key with a sagemaker_key_config object. Its fields:
  • region is required.
  • access_key, secret_key and session_token are optional. If the access key and secret key are left empty, the default AWS credential chain is used.
  • role_arn, external_id and session_name configure AssumeRole.
  • model_name sets the value sent as the body model to the container. Empty sends "", because SageMaker routes by endpoint name and vLLM rejects any other name.
  • endpoints.runtime overrides the runtime host, for example an interface VPC endpoint or a FIPS or dual-stack host. It must contain a .. Region is still required because it sets the token's credential scope.
• Mirrored sagemaker_key_config in the Helm chart's values.schema.json.
• Added a commented SageMaker example to the Helm values.yaml. It shows models as endpoint names in the form sagemaker/<endpoint>[/<inference-component>], and shows using the default AWS credential chain (IRSA or instance role) by omitting the keys.
• Design notes:
  • Bifrost mints the SageMaker bearer token from the configured AWS credentials. A plain key value is sent verbatim as a pre-minted sagemaker-api-key-... token.
  • A model id can override the region with a prefix, for example sagemaker/us-west-2/my-endpoint.
  • Both key config objects set additionalProperties: false, so unknown fields are rejected.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Validate a config that uses the new provider against the schema, and lint the Helm chart with an example values file.

# Core/Transports
go version
go test ./...

# UI
cd ui
pnpm i || npm i
pnpm test || npm test
pnpm build || npm run build

Example config to validate against transports/config.schema.json (or via Helm values):

sagemaker:
  keys:
    - name: "sagemaker-key"
      models: ["my-qwen3-endpoint"]
      weight: 1
      sagemaker_key_config:
        region: "us-east-1"
        access_key: "env.AWS_ACCESS_KEY_ID"
        secret_key: "env.AWS_SECRET_ACCESS_KEY"

Expected outcomes:

• The config above validates.
• A config with sagemaker_key_config missing region fails validation.
• A config with unknown fields in sagemaker_key_config fails validation.
• An endpoints.runtime value without a . fails validation.

helm lint helm-charts/bifrost

Screenshots/Recordings

N/A

Breaking changes

• Yes
• No

Related issues

N/A

Security considerations

• Credentials (access_key, secret_key, session_token, role_arn, external_id, session_name, and the runtime host) can be supplied through the env. prefix, so secrets need not be stored in config files.
• Omitting static credentials lets deployments rely on the default AWS credential chain (IRSA or instance roles).
• A key value is sent verbatim as a pre-minted token, so it should be handled like any other secret.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: maximhq/bifrost
• Branch: 10-24-chore&#95;updates&#95;config&#95;schema&#95;and&#95;helm&#95;schema&#95;for&#95;sagemaker
• File: .coderabbit.yaml
• Commit: 0d81e11
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Team
• Run ID: c455cace-ed02-497c-8ba3-dea900190760

Commits

Reviewing files that changed from the base of the PR and between 56dd8e7 and 0d81e11.

Files selected for processing: 3

• helm-charts/bifrost/values.schema.json
• helm-charts/bifrost/values.yaml
• transports/config.schema.json

Files skipped from review as they are similar to previous changes: 1

• helm-charts/bifrost/values.yaml

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

📝 Summary

Summary by CodeRabbit

• New Features
  • Added configuration support for AWS SageMaker real-time endpoints, including endpoint routing, AWS region and credentials, optional role assumption, and runtime host overrides.
  • Added a Helm chart example for configuring a SageMaker provider with credentials supplied through environment variables. Explicit credentials can be omitted to use the default AWS credential chain.

The change adds SageMaker provider and key configuration schemas. They support AWS credentials, region, role-assumption settings, and an optional runtime host. It also adds a commented Helm values example for a SageMaker real-time endpoint.

Changes: SageMaker configuration

Layer / File(s)	Summary
Provider configuration schema transports/config.schema.json	Registers SageMaker as a provider and adds its key and provider schemas. The key schema requires a region and allows AWS credentials, role-assumption settings, and an optional runtime host override.
Helm chart configuration helm-charts/bifrost/values.schema.json, helm-charts/bifrost/values.yaml	Adds the SageMaker key schema to the Helm values schema and a commented endpoint example. The example includes a model, weight, region, and environment-referenced credentials. Its comments describe using the default AWS credential chain when explicit credentials are omitted.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• maximhq/bifrost#9004: Adds SageMaker key-management tests and documents the key configuration contract in OpenAPI.
• maximhq/bifrost#9002: Implements the SageMaker provider and consumes the configuration fields declared here.
• maximhq/bifrost#8952: Provisions SageMaker endpoints and emits provider configuration that these schemas define.

Merge Risk: Low · previous review 0d81e

SageMaker configuration does not expose a model-name override. Confirm whether that capability is intended before depending on it.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title clearly and concisely describes the main change: adding SageMaker provider configuration support to the Helm chart and configuration schemas.
Description check		The description is complete and aligned with the template. It explains the purpose, changes, affected areas, testing steps, security considerations, and breaking-change status. It includes expected va…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description is complete and aligned with the template. It explains the purpose, changes, affected areas, testing steps, security considerations, and breaking-change status. It includes expected validation outcomes, although it does not report completed test results.

Finishing Touches: 💡 1

🛠️ Fix failing CI checks: 💡

• Commit to this branch
• Create a new PR

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[sammaji]

• docs: add sagemaker provider key config to openapi schema and e2e tests #9004
• chore: add sagemaker provider config to helm chart and config schemas #9003 👈 (View in Graphite)
• feat: add sagemaker passthrough for InvokeEndpoint and response streaming #9037
• feat: add aws sagemaker provider #9002
• feat: add terraform module for SageMaker on AWS #8952
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Pratham-Mishra04]

Merge activity

• Oct 27, 5:04 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Oct 27, 5:14 PM UTC: Graphite couldn't merge this pull request because a downstack PR feat: add sagemaker passthrough for InvokeEndpoint and response streaming #9037 failed to merge.
• Oct 27, 5:32 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Oct 27, 5:36 PM UTC: Graphite rebased this pull request as part of a merge.
• Oct 27, 5:40 PM UTC: @Pratham-Mishra04 merged this pull request with Graphite.

The base branch was changed.