Repository navigation
Conversation
…t it to 16 MiB (#8126) ## Problem On a ClickHouse logs store, team-scoped users with a big team get this on every chart: ``` code: 62, message: Max query size exceeded (can be increased with the `max_query_size` setting) ``` `logScope` widens team-data scope to every member user id and every team-owned VK id. `queryscope.InSet` binds those as a single array parameter on Postgres and SQLite, but ClickHouse falls through to `IN ?`, so the driver writes each UUID into the SQL text. ClickHouse parses at most 256 KiB of query by default. That's roughly 6.7k ids, and a large team gets past it. The failure isn't specific to the model histogram. Every DAC-scoped read for that user goes through the same predicate. ## Fix `buildClickHouseDSN` now sends `max_query_size`, the same way it already sends `final`, `mutations_sync` and `prefer_column_name_to_alias`. It defaults to 16 MiB, which is about 400k ids, and you can set it from: - config.json: `logs_store.config.max_query_size` (bytes) - Helm: `storage.logsStore.clickhouse.maxQuerySize`, which renders into the field above Query text and results don't change. Only the parser cap moves, and small queries cost the same as before. If a ClickHouse profile puts a constraint on this setting, the store fails at startup instead of partway through. Setting a value below the cap fixes it. I didn't bind the list as a server-side `{p:Array(String)}` parameter. Over the HTTP protocol clickhouse-go sends parameters in the URL, so a large scope would trip `http_max_uri_size` or a proxy's URL limit instead. ## Testing - `TestBuildClickHouseDSN` checks the default and an override - `helm template` renders `max_query_size` when the value is set, leaves it out when it isn't, and the values schema rejects a non-integer - `validate-helm-config-fields.sh` passes, 353/353 - Not yet run against a real ClickHouse with a scope of about 20k ids
…nfig type (#8130) ## Overview Adds an optional `disabled` boolean to the MCP client object nested in the `VirtualKeyMCPConfig` TypeScript type. The UI can now read and type-check the disabled state of an MCP client attached to a virtual key. Without it, the field would be inaccessible or need a cast. ## Changes - Added `disabled?: boolean` to the MCP client shape in `VirtualKeyMCPConfig` in `ui/lib/types/governance.ts`. - The field is optional so existing payloads that omit it still type-check, and no existing consumers need changes. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test Confirm the UI type-checks and builds with the new field. ```sh # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Expected: the build and type-check pass, and `disabled` is available on the MCP client object within a virtual key's MCP config. No new configs or environment variables are introduced. ## Screenshots/Recordings N/A. This is a type-only change with no visual difference. ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations None. This is a type definition change only and does not alter runtime behavior, auth, or data handling. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…)` accessors (#7975) ## Summary Deprecates a set of raw `BifrostContextKey` constants that store request attribution data (user, team, customer, business unit, project, budget, and rate limit identifiers) directly on the context. These keys are superseded by the structured `ctx.Grant()` API, which provides typed access to the same information. ## Changes - Moved the deprecated context key constants (`BifrostContextKeyGovernanceVirtualKeyID`, `BifrostContextKeyGovernanceTeamID`, `BifrostContextKeyGovernanceCustomerID`, `BifrostContextKeyGovernanceBusinessUnitID`, `BifrostContextKeyGovernanceProjectID`, `BifrostContextKeyGovernanceBudgetIDs`, `BifrostContextKeyGovernanceRateLimitIDs`, `BifrostContextKeyUserID`, `BifrostContextKeyUserName`, `BifrostContextKeyUserEmail`, and related name/plural variants) into a dedicated `const` block with `Deprecated` godoc comments pointing callers to the equivalent `ctx.Grant()` accessor. - The constants retain their original string values so existing readers continue to work without any runtime change; only the documentation and grouping change. - New code is directed to use `ctx.Grant().Identity()`, `ctx.Grant().Limits()`, and their sub-accessors rather than reading raw context values. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test No behavioral change is introduced; the constants are identical in value. Verify the build continues to pass and that no existing references break. ```sh go build ./... go test ./... ``` ## Breaking changes - [ ] Yes - [x] No The string values of all constants are unchanged. Existing code that reads these keys from the context will continue to work. The deprecation notices are informational only. ## Security considerations No security impact. The change is purely organizational — no credentials, PII handling, or auth logic is modified. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…utright (#8100) ## Overview A request whose session is bound to a key now moves to another eligible key when the provider rejects the bound key outright (a credential, quota or model-access failure). Previously the session's key was the only candidate for every attempt, so a dead bound key failed the request with `502 upstream_credentials_exhausted` even when healthy sibling keys were available. ## Changes - **Session pool keeps the rest of the eligible keys.** `selectKeyFromProviderForModelWithPool` now returns the session's key first, followed by the other eligible keys. `canRotate` stays `false`, so the session's key is still used for every attempt until it is marked dead. - **Fallback once the session's key is dead.** In `requestWorker`, the fixed-key provider returns the session's key while it is alive. After it is marked dead, the provider chooses among the remaining live keys, in this order: - Keys not yet rate-limited during this request are preferred. - The configured `keyPoolFilter` (for example, the circuit breaker) is applied. A filter error falls back to the unfiltered keys, and an empty result returns `errAllKeysFiltered`. - `keySelector` picks from what remains, as it would for a request with no session. - If no live key remains, `errAllKeysDead` is still surfaced. - **Explicit pins are unchanged.** A key pinned by API key ID or name has nothing behind it, so a dead pinned key still returns `errAllKeysDead` immediately. - **Session rebinding.** After a sibling serves the request, the session binds to that key. - **Single-key lookups keep the session's key.** `SelectKeyForProviderRequestType` (used by realtime/WebSocket connections that need one key up front) now returns the first key when `canRotate` is false. Without this, the extra keys in the session pool would have been passed to the key selector, which could override the session's key. - **Docs.** The `SessionAffinity.ResolveKey` contract and related comments now say that the returned key is used until the provider rejects it outright, after which the request continues on another eligible key. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./core/... -run 'Session' go test ./... ``` The new tests in `core/sessionaffinity_test.go` cover these cases: - **Rejected session key:** key-a returns 401, key-b serves the request, and the session is rebound to key-b. - **Rate-limited sibling:** with three keys, key-a returns 401 and key-b returns 429. The request goes a → b → c and is served by key-c. - **Key pool filter:** when the filter excludes key-b and key-a is rejected, the request goes a → c and key-b is never contacted. - **Single-key lookup:** `SelectKeyForProviderRequestType` returns the session's key-a even when the selector would pick a different key. `TestSelectKeyFromProviderForModel_SessionStickiness` and `TestSelectKeyFromProviderForModel_SessionStickinessNoRotation` were updated to expect the session's key first, with the rest of the pool behind it. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations Fallback only considers keys already in the eligible pool for the provider and model, so disabled or model-incompatible keys cannot be selected. The key pool filter (such as the circuit breaker) still applies to the key a request moves to. Explicitly pinned keys are never replaced. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…provider `x-bifrost-*` headers (#8101) ## Summary The gateway's request ID could not be reliably read from the `x-request-id` response header. Provider response headers are forwarded to the caller, and OpenAI answers with its own `x-request-id`, which replaced Bifrost's value. An OpenAI caller therefore had no way to learn the ID its request was logged under (`GET /api/logs/{id}`). This PR adds `x-bifrost-request-id` as the dependable header for the gateway's request ID. It also stops provider responses from overwriting any `x-bifrost-*` header the gateway writes itself. ## Changes - The tracing middleware now sets `x-bifrost-request-id` on every traced response, next to `x-bifrost-trace-id`. Its value is the caller's `x-request-id`, or the generated UUID if none was sent, and it is also the ID of the request's log entry. - `x-request-id` is still written for compatibility and is now documented as deprecated for reading Bifrost's request ID. It carries the same value unless the provider sends its own `x-request-id` (OpenAI does), in which case the provider's value wins, as before. - The panic recovery path restores `x-bifrost-request-id`, `x-request-id` and `x-bifrost-trace-id` on the recovered 500. - Added `lib.IsGatewayOwnedResponseHeader` and `lib.ForwardProviderResponseHeaders`. Provider response headers whose names start with `x-bifrost-` (case-insensitive) are no longer forwarded. - This matters when the upstream is another Bifrost gateway. Its `x-bifrost-*` headers describe its own hop, so they must not replace this gateway's request ID, trace ID or routing headers, or add values this hop never set. - The provider's values are still available in `extra_fields.provider_response_headers` on routes that return `extra_fields`. - Every place that copied provider headers now uses the shared helper: the inference handlers, streaming and non-streaming integration router paths, stream and error responses, large-response streaming, realtime client secrets, and `ApplyBifrostResponseHeaders`. - Passthrough responses (non-stream and stream) skip gateway-owned headers as well. - Docs: the request ID option and the "Correlation response headers" table are updated, and a note explains which provider headers are forwarded. - E2E: the existing Postman checks and the video costing runner now read `x-bifrost-request-id` as the log row ID. A new collection folder, "167. Bifrost request id survives the provider's own x-request-id", covers these cases: - OpenAI non-streaming, streaming and the `/openai` drop-in. - Anthropic, where `x-request-id` still echoes the caller's ID. - OpenAI with a generated ID. ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Run the unit tests for the touched packages: ```sh go test ./transports/bifrost-http/lib/... ./transports/bifrost-http/handlers/... ./transports/bifrost-http/integrations/... ``` Expected: the new tests pass, covering the correlation headers set by tracing, restoration after a panic, and `x-bifrost-*` provider headers not being forwarded. Manual check against a running gateway with OpenAI configured: ```sh curl -si http://localhost:8080/v1/chat/completions \ -H 'Content-Type: application/json' \ -H 'x-request-id: my-test-id' \ -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"ok"}],"max_tokens":8}' ``` Expected: - `x-bifrost-request-id: my-test-id`. - `x-request-id` is OpenAI's own ID. - `GET /api/logs/my-test-id` returns the log entry. Omit the `x-request-id` header to confirm a generated UUID is returned in `x-bifrost-request-id` and matches the log entry. The Postman collection can be run with `FEATURE="bifrost-request-id"` (needs provider keys). ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No `x-request-id` is still returned with the same behavior. One behavior change: provider response headers named `x-bifrost-*` are no longer forwarded to callers. This only affects chained Bifrost setups, where those headers previously overwrote the downstream gateway's own values. ## Related issues N/A ## Security considerations Blocking forwarding of `x-bifrost-*` provider headers prevents an upstream from spoofing or overwriting this gateway's correlation IDs and routed-identity headers. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ider configs, and routing targets to prevent weighted-random panic (#8102) ## Overview Weighted selection could be driven by weights it cannot draw by, and the API accepted some of them. This PR fixes both. - **Provider key selection:** `WeightedRandom` summed key weights as ints (`int(weight * 100)`). A negative weight, or one too large for an int, could push the total below zero, and `rand.Intn` then panicked inside the provider's request worker. That worker has no recover, so one request on such a key pool could take down the whole gateway. The int conversion also silently dropped weights below 0.01. - **Virtual key load balancing:** A negative provider-config weight pulled the summed weight below the draw. No candidate matched, so the pick fell through to the first configured provider whatever the other weights said. - **Routing rule targets:** `config.schema.json` and the dashboard require every target weight to be greater than 0, but the API accepted 0, which is a target that is never drawn. ## Changes - `WeightedRandom` now sums and draws with `float64` (`rand.Float64() * totalWeight`). Small weights are no longer truncated, and large ones cannot overflow. - A new `usableWeight` helper makes selection treat a negative, NaN or infinite weight as 0. Write paths reject these, but `config.json` and older rows can still carry them, so selection stays safe either way. Where every key resolves to zero, the existing uniform-random fallback applies. - The load balancer's `getWeight` in the governance plugin applies the same rule. A nil weight still defaults to 1.0, and an explicit 0 is still honored. - A new `validateWeight` rejects negative and non-finite weights. It is applied on: - provider key create and update (on the merged key for update) - virtual key create - both virtual key update paths, for existing and new provider configs - Virtual key provider-config weights of `null` (opts out of load balancing) and `0` remain accepted. Key weights of `0` also remain accepted. - `validateRoutingTargets` now rejects any weight that is not greater than 0, including NaN, and includes the offending value in the error message. - Tests added: - a table test for `WeightedRandom` covering zero, sub-0.01, negative, NaN, Inf and overflow-sized weights - a load balancer test showing a negative-weight provider is never selected - handler tests for key weights, virtual key provider-config weights and routing target weights - three `[EXPECT-400]` rows (folder 168) in the provider harness collection, covering key weights, virtual key provider configs and routing rule targets. They run against a run-scoped throwaway provider that never serves a request, and they clean up after themselves. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Key selection (includes the negative/NaN/Inf/overflow cases) go test ./core/keyselectors/ -run TestWeightedRandom go test ./core/keyselectors/ -bench WeightedRandom -benchmem # Load balancer go test ./plugins/governance/ -run TestLoadBalanceProvider # API validation go test ./transports/bifrost-http/handlers/ -run 'TestProviderKeyWeight_RejectsNegative|TestVirtualKeyProviderConfigWeight_RejectsNegative|TestValidateRoutingTargetsWeights' ``` For the e2e harness, run the provider-harness collection with `FEATURE="weight-validation"`. Expected results: - Creating or updating a key with a negative weight returns 400, and the error names the weight. - Creating or updating a virtual key provider config with a negative weight returns 400, and the stored weight is unchanged. - Creating a routing rule with a zero-weight target returns 400 with "greater than 0". No new configs or environment variables. ## Screenshots/Recordings N/A ## Breaking changes - [x] Yes - [ ] No The API now returns 400 for the following inputs: - a negative or non-finite weight on a provider key - a negative or non-finite weight on a virtual key provider config - a routing rule target with weight 0 Clients sending those values must change them. For routing targets, remove the zero-weight target and make the remaining weights sum to 1. Existing stored negative or non-finite weights no longer cause panics or skew selection, because they count as 0 at selection time. They are only rejected when written again. ## Related issues N/A ## Security considerations A negative or overflowing key weight could crash the gateway with a panic in the request worker, so anyone able to set key weights, or any bad value in `config.json`, could cause a denial of service. Selection no longer panics on such weights, and the key and virtual key APIs reject them. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…og resolver (#8103) ## Overview This PR trims the code comment above the fallback-population logic in the model catalog resolver plugin's `PreRequestHook`. The removed sentence said an explicit fallback list, even a deliberately empty one, is always respected. The code does not distinguish an empty list from an unset one: the check is `len(existingFallbacks) == 0 && len(candidates) > 1`. The comment therefore described behavior the code does not implement. ## Changes - Removed the sentence "Only fires when the caller passed none; an explicit fallback list (even an empty one set deliberately) is always respected" from the comment in `plugins/modelcatalogresolver/main.go`. - Kept the rest of the comment: the cross-provider resilience rationale, the parallel with the governance and load balancing plugins, and the note that model refinement isn't needed because `GetProvidersForModel` only returns providers that already serve the exact model string. - No logic changes. Fallbacks are still populated from the remaining catalog candidates when none exist and more than one candidate is available. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test This is a comment-only change, so runtime behavior is unchanged. Confirm the plugin still builds and its tests pass: ```sh # Core/Transports go version go build ./plugins/modelcatalogresolver/... go test ./plugins/modelcatalogresolver/... ``` Expected outcome: the build succeeds and all existing tests pass. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations None. This change only touches a code comment. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
…iority and print scope ID rather than pointer (#8104) ## Summary Creating or updating a routing rule with a priority already used by another rule in the same scope returned a 500 "Failed to create routing rule" (or "Failed to update routing rule in database"). That reads as a server failure a client might retry, when the real problem is a conflict in the request. For rules scoped to a team, customer, or virtual key, the error message also printed the memory address of the scope ID pointer (e.g. `0xc000...`) instead of the ID. This PR makes a duplicate priority a 409 Conflict with a message that names the priority and the real scope ID. ## Changes - The config store now wraps the duplicate-priority error with `ErrAlreadyExists` on both create and update, so callers can detect it with `errors.Is`. - The error message now dereferences the scope ID, so it reads `scope 'virtual_key' with scope ID 'vk-1'` rather than printing a pointer. - The `createRoutingRule` and `updateRoutingRule` handlers check for `ErrAlreadyExists` and answer `409 Conflict`. The message is the store error plus "; use a different priority". Other errors still return 500. - Added a store test covering global and scoped duplicates on create and update. It also checks that the same priority in a different scope or under a different scope ID is not a conflict, and that no pointer address appears in the message. - Added a handler test confirming that duplicate create and update both return 409 with the priority in the body. It also confirms the conflicting write stores nothing and doesn't trigger a routing reload. - Added two e2e cases to the provider harness collection (feature `routing-priority-conflict`): - a global-scope duplicate on create and on PUT - a virtual-key-scoped duplicate whose message names the virtual key ID The harness rules match only a header no request sends, and each row deletes what it created. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./framework/configstore -run TestRDBConfigStore_RoutingRuleDuplicatePriority go test ./transports/bifrost-http/handlers -run TestRoutingRuleDuplicatePriorityIsAConflict go test ./... ``` Manually: 1. Create a routing rule with priority `N` in the global scope. 2. POST a second rule with priority `N`. Expect `409` with a message containing `priority N already exists for scope 'global'`. 3. Create a rule at a different priority and PUT `{"priority": N}`. Expect `409`. 4. Repeat with `scope: virtual_key` and a `scope_id`. Expect `409` with the virtual key ID in the message and no `0x...` address. To run the e2e cases, run the provider harness collection with `FEATURE="routing-priority-conflict"`. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No Clients that treated this case as a 500 will now see a 409. The response body is still an error with a message. ## Related issues N/A ## Security considerations The error message no longer exposes a pointer address, which is an internal memory detail. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…guish added providers from edited ones (#8105) ## Overview `ReloadProvider` now receives an `isNew` flag that tells it whether the write that triggered the reload added a provider or edited an existing one. Until now the reload had no way to tell the two apart. A server that wraps `BifrostHTTPServer` can use the flag to treat an operator's edit differently from a fresh add. The reload itself behaves the same either way. ## Changes - `ReloadProvider` on `ServerCallbacks`, `ModelsManager`, and `GovernanceManager` takes an extra `isNew bool` argument. `BifrostHTTPServer.ReloadProvider` accepts it and documents its purpose, but the built-in reload logic ignores it. - The add endpoint's reload (`reloadProviderAfterCreate`) passes `true`. - The update endpoint passes `false` on both of its reload paths: the keyless-provider reload in `updateProvider`, and the reload in `attemptModelDiscovery`. Discovery runs after an update, which edits a provider the add endpoint already created. - The mocks in `providers_test.go` and `pricing_override_test.go` and the existing `ReloadProvider` tests in `server_test.go` use the new signature. The `providers_test.go` mock also records the flag it received on each reload. - New test `TestProviderWrites_TellTheReloadWhetherTheyAdded` checks that: - an update to a provider with keys reloads with `isNew=false`; - an update to a keyless provider reloads with `isNew=false`; - an add reloads with `isNew=true`. - Design note: the flag is a plain boolean on the existing method rather than a new method or callback. This keeps a single reload path and leaves existing behavior unchanged. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the handler and server tests, including the new test that checks which `isNew` value each endpoint passes: ```sh # Core/Transports go version go test ./transports/bifrost-http/handlers/... -run 'TestProviderWrites_TellTheReloadWhetherTheyAdded|TestAddProvider|TestUpdateProvider' go test ./transports/bifrost-http/server/... -run TestReloadProvider go test ./... ``` Expected result: all tests pass. The add request records `[true]` and both update requests record `[false]`. ## Screenshots/Recordings N/A (no UI changes). ## Breaking changes - [x] Yes - [ ] No The `ReloadProvider` signature changed on `ServerCallbacks`, `ModelsManager`, and `GovernanceManager`. Any external implementer or wrapper of these interfaces must add the `isNew bool` parameter. Callers should pass `true` when the reload follows a provider being added and `false` otherwise. HTTP API behavior is unchanged. ## Related issues N/A ## Security considerations None. No auth, secrets, or request handling changes. The flag is internal and does not alter reload behavior. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ting (#8106) ## Summary Stops a caller's direct key (`x-bf-direct-key: true` plus their own provider key, enabled by `allow_direct_keys`) from being sent to providers it doesn't belong to. Previously, a model listing that named no provider polled every configured provider and substituted the direct key for each provider's own keys. A single SDK `models.list()` call could therefore send a caller's OpenAI key to Anthropic, Gemini, and every custom provider's upstream. ## Changes - `ListAllModels` in core now returns a `400` when the context carries a direct key, before any provider is contacted. The error says the request must name its provider. - Drop-in model listing routes now resolve a direct-key listing that names no provider to the route's own provider: - `/openai/v1/models` lists OpenAI - `/anthropic/v1/models` lists Anthropic - `/genai/v1beta/models` lists Gemini - If the route's provider isn't configured on the gateway, the route answers `400` and tells the caller to name a configured provider with `x-bf-model-provider`. - `x-bf-model-provider: all` with a direct key is refused with `400`, as is `GET /v1/models` without `?provider=`. `x-bf-model-provider` can still name a different provider. - Added `IsProviderConfigured` to the `HandlerStore` interface, implemented on `Config` as a lookup in the providers map. Test stores were updated to satisfy the interface. - Documented the behavior in the request options docs, next to the direct key warning. - Tests: - A core test verifies that the aggregate refuses a direct key and reaches no upstream, while a named-provider listing sends the key to that provider only. - A router test covers the OpenAI and Anthropic routes, the `all` refusal, and the unconfigured-provider refusal. - A new Postman/e2e harness section (`direct-key-model-listing`) uses throwaway keys, so the provider's own 401 shows which provider was asked. - `allow_direct_keys` is enabled in the Python integration test config. Design note: refusing the fan-out outright was chosen over silently dropping the key for other providers, because a listing mixing the caller's key and operator keys would give misleading results and hide the misconfiguration. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test ```sh # Core/Transports go version go test ./core/... -run TestListAllModelsRefusesADirectKey go test ./transports/bifrost-http/integrations/... -run TestListModelsDirectKeyStaysOnTheRouteProvider go test ./... ``` Expected outcomes, with `client.allow_direct_keys` enabled: - `GET /v1/models` with `x-bf-direct-key: true` and no `?provider=` returns `400` ("must name its provider"). - `GET /openai/v1/models` or `/anthropic/v1/models` with a direct key reaches only that provider. A bogus key yields that provider's own 401. - `x-bf-model-provider: all` with a direct key returns `400`. - A drop-in route whose provider isn't configured returns `400`. The e2e harness rows can be run with `FEATURE="direct-key-model-listing"`. ## Screenshots/Recordings N/A ## Breaking changes - [x] Yes - [ ] No Callers who relied on a direct key to list models across all providers now get a `400`. They should name a provider with `?provider=` or `x-bf-model-provider`. Drop-in routes now list only their own provider rather than every provider when a direct key is used. ## Related issues N/A ## Security considerations This closes a credential leak. A caller's direct key could be forwarded to every configured provider's upstream, including custom provider endpoints, on a single model listing request. Direct keys now reach only the provider they are meant for, and refused requests contact no provider. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…l so the primary's error isn't returned with the last fallback's headers (#8107) ## Summary When the primary provider and every fallback fail, Bifrost returns the primary's error. However, each fallback attempt overwrote the provider response headers on the request context with its own provider's headers, so the transport forwarded the last fallback's headers with the primary's error body. For example, an OpenAI 404 went out with Anthropic's request-id, organization, workspace, and rate-limit headers. This PR makes the primary's error go out with the primary's own headers. ## Changes - Capture the primary attempt's provider response headers right after the primary request returns, in both `handleRequest` and `handleStreamRequest`. - Add a `restoreProviderResponseHeaders` helper in `core/utils.go` that puts those headers back on the context just before the primary error is returned after all fallbacks are exhausted. If the primary had no headers, the key is cleared so none of the fallback's headers leak through. - Add a core test, `TestExhaustedFallbacksKeepThePrimaryProviderHeaders`, covering streaming and non-streaming. It uses fake OpenAI and Anthropic upstreams that each return a 404 tagged with an `X-Upstream` header, and it asserts that the forwarded headers belong to the primary. - Add an e2e harness collection entry (`exhausted-fallback-headers`) that sends a missing OpenAI model with a missing Anthropic fallback. It runs on `/v1/chat/completions` and the `/openai` drop-in, and it asserts a 404 carrying OpenAI's error with no Anthropic-specific headers. Design note: the headers are restored only on the path where the primary error is returned. If a fallback succeeds, its headers stay on the context, since its response is the one being returned. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the core test suite, including the new test: ```sh # Core/Transports go version go test ./core/... -run TestExhaustedFallbacksKeepThePrimaryProviderHeaders go test ./... ``` To check end to end, run the provider harness with `FEATURE="exhausted-fallback-headers"`. Alternatively, send a request to `/v1/chat/completions` or `/openai/v1/chat/completions` with a nonexistent OpenAI model and `"fallbacks": ["anthropic/claude-harness-missing-model"]`. Expected result: a 404 with OpenAI's `model_not_found` error and no `anthropic-organization-id`, `anthropic-workspace-id`, or `anthropic-ratelimit-requests-limit` headers. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations This prevents one provider's response headers (request-id, organization and workspace IDs, rate-limit information) from being exposed alongside another provider's error response. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… refused instead of collapsing into 502 `upstream_credentials_exhausted` (#8108) ## Summary When a caller supplies their own provider key with `x-bf-direct-key`, and the provider refuses it, the gateway could turn the refusal into a synthetic `502 upstream_credentials_exhausted`. This happened whenever `max_retries` left a retry in the budget. That response tells the caller the gateway's own keys ran out, and it invites a retry with the same bad key. This PR makes a refused direct key return the provider's own error (for example OpenAI's 401) regardless of `max_retries`. ## Changes - In `executeRequestWithRetries`, a direct key is now detected through `BifrostContextKeyDirectKey` on the context. When the key pool is dead and the key was supplied by the caller, the provider's error is returned as-is. Previously this applied only when the attempt fell outside the retry budget. - A single configured key pinned within the retry budget still collapses into `502 upstream_credentials_exhausted`, because that credential isn't the caller's. - The comments on `errAllKeysDead` and the surrounding branch now describe the direct-key exception. - Added a unit test, `TestExecuteRequestWithRetries_DirectKeyRefusalIsTheCallersError`. It covers `max_retries` of 0, 1, and 3 with a direct key. Each case expects the provider's 401 and exactly one upstream call. It also checks that a pinned configured key within the budget still gets the 502. - Added an e2e harness row, `172. ... (direct-key-refusal)`. It creates a run-scoped OpenAI-backed custom provider with `max_retries: 1` and no keys, then sends a chat completion with a throwaway direct key. It expects OpenAI's own 401 and a single entry in the log row's attempt trail, then deletes the provider. It costs nothing, because OpenAI rejects the key before any tokens are used. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Core unit test go test ./core -run TestExecuteRequestWithRetries_DirectKeyRefusalIsTheCallersError go test ./core/... # E2E harness (requires a running gateway with client.allow_direct_keys enabled) # Run the provider-harness Postman collection with FEATURE="direct-key-refusal" ``` Expected: a chat completion sent with `x-bf-direct-key: true` and an invalid key returns the provider's 401 ("Incorrect API key"), not `upstream_credentials_exhausted`. The attempt trail contains exactly one attempt. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No Callers using direct keys that previously got a 502 `upstream_credentials_exhausted` on a refused key will now get the provider's own error status. ## Related issues N/A ## Security considerations Returning the provider's own error for a caller-supplied key stops the gateway from implying that its own credentials are exhausted. No new secrets are handled. The e2e row uses a throwaway generated key. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ct-key requests and from moving requests pinned to the head provider's key (#8109) ## Summary Fixes two ways session affinity could route a request carrying the caller's own credentials to the wrong provider. 1. **Direct key leaking to a fallback provider.** A direct key (`x-bf-direct-key`) is only tried on the first route of a chain. Fallbacks run on the gateway's keys. When a fallback served such a request, the session was bound to that fallback. The next request put the fallback first, so the caller's key was sent to a provider it was never meant for. 2. **Caller key pin moved off its provider.** When the caller pinned a key with `x-bf-api-key-id` or `x-bf-api-key` on the provider heading the routed chain, the session could still promote another provider. The pin then reached a provider that has no such key, so that attempt failed at key selection and the fallback ran without the pin. ## Changes - **Direct key outcome handling** - `RouteOutcome` gains a `DirectKey` field. - `handleRequest` and `handleStreamRequest` read whether a direct key is present at the start of the request, because the first fallback clears it from the context. They pass the value to `observeSessionOutcome`. - In `Observe`, a fallback serving a direct-key request no longer writes a route binding. - Any route binding the request followed is deleted, so the next request follows the routing decision. A routing-trail entry is added and `session-affinity` is listed among the engines used. - The key binding on the fallback's own provider is still written, since that key really did serve. - **Caller key pin handling** - A new `callerPinsKeyOf` check runs in `resolveSessionRoute`. If the caller's key ID or name matches a key on the head provider, the session leaves the chain alone and logs why. This treats the pin like a provider-prefixed model. - A routing rule's pin is not treated as a caller pin. It still moves with its route, because key selection reads the committed api-key-id first. - A pin on another provider's key, an unknown key, or a direct key still leaves the decision to the session. - **Tests** - Existing `observeSessionOutcome` calls are updated for the new argument. - New unit tests cover the `Observe` direct-key cases. - A new end-to-end Go test runs against fake OpenAI and Anthropic upstreams, for both streaming and non-streaming, and checks that the caller's key never reaches the fallback provider. - The old "a caller's own pins are left alone" test is replaced with tests for each pin scenario. - Two new provider-harness collection rows (173 and 174) cover the caller pin and direct key behavior against a live gateway. - **Docs** - The request-options and session-affinity pages document the new behavior, the new table row, the trail messages, and the limit that a direct key goes to whichever provider comes first. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test ```sh # Core/Transports go version go test ./core/... -run 'SessionAffinity|ResolveSessionRoute|ObserveSessionOutcome|SessionKey|SelectKeyFromProviderForModel' go test ./... ``` Expected results: - A direct-key request served by a fallback leaves no route binding. The second turn starts on the routed provider again, and the caller's key is never seen by the fallback provider. - A request pinning a key on the head provider is served by that provider as primary, not as a fallback. - A request pinning a key on the promoted provider is still moved there by the session. Provider harness rows 173 (`FEATURE="session-caller-pin"`) and 174 (`FEATURE="session-direct-key"`) can be run against a live gateway. Each creates and deletes its own run-scoped providers and routing rule. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No Affinity no longer rebinds to a fallback that served a direct-key request, and no longer reorders a chain away from a provider the caller pinned a key on. Both were unintended behaviors. ## Related issues N/A ## Security considerations A caller's direct key could previously be sent to a provider it was never meant for, because a fallback that served the request became the session's first provider on the next turn. After this change the direct key only goes to the first provider of the routing decision. Caller key pins are also no longer carried to a provider that has no such key. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…after a billed primary attempt (#8111) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…nd correct the session affinity docs (#8112) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…8113) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…headers (#8114) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Make A2A push handling quieter during idle periods, ensure every accepted callback configuration can be reproduced by the gateway, and surface the resulting push and Agent data in the management API and UI. ## Changes - Stop exporting successful `a2a.push.maintenance.list-due` traces when a relay poll finds no deliveries. Error, cancellation, non-empty poll, prune, ingress, and delivery traces are still exported. Suppressed traces are returned to the tracer pools through a new `ReleaseTrace` method on `schemas.Tracer`, which makes cleanup part of the `EndTrace` ownership contract. Custom tracer implementations must add this method. - Validate callback authentication before contacting the upstream Agent. Push configs may omit authentication or use Basic or Bearer with non-empty credentials. Unsupported or incomplete authentication fails immediately instead of producing an undeliverable registration. - Preserve the client-supplied tenant in durable push state, local get/list responses, the management push-config view, and the push config sent upstream. A configstore migration adds the tenant column. - Send downstream callbacks with `Content-Type: application/a2a+json`. - Add `GET /api/agents/history/top-agents`, which returns per-Agent operation count, errors, and average latency for a filter, through the log store, hybrid store, and HTTP handler. - UI: - The push configurations table shows a Tenant column. - The Agent Catalog page is titled "Agent Catalog" and the sidebar item "MCP Gateway" is now "MCP Servers". - In Agent Logs sheets, the Formatted / Raw JSON toggles inside events use the compact tab style of the allowed/blocked models tabs. The main Conversation, Events, and similar tab bars match the MCP logs sheet. - New API types and an RTK Query hook for the top-Agents endpoint. - Add focused coverage for trace export and release decisions, callback-auth validation, tenant persistence, forwarding and management output, and downstream callback headers. ### Why The push relay polls its durable outbox every few seconds so it can resume work after restarts and pick up deliveries created by another node. A successful empty poll is expected idle behavior, but each poll previously exported a trace: ```text relay timer -> query due deliveries -> zero rows -> export successful maintenance trace -> send recurring noise to trace connectors ``` The relay must keep polling because push can be configured dynamically. This change ends the trace locally when the query succeeds with no work, without stopping the poller: ```text query result | +-- error or cancellation -> export error trace +-- one or more rows -> export maintenance trace +-- zero rows -> end trace, release to pools, no export ``` The callback configuration also crosses two HTTP hops. Bifrost needs enough durable information to make the second hop without asking the upstream Agent for client credentials: ```text client registration tenant: acme URL: https://client.example/push token: client-token auth: Bearer client-secret | v Bifrost stores the client-facing values | +--> upstream registration | tenant: acme | URL: https://gateway.example/agents/a2a/research/push/callback | token: Bifrost ingress token | auth: omitted | +<-- upstream posts event to Bifrost | v Bifrost posts the event to the client A2A-Notification-Token: client-token Authorization: Bearer client-secret Content-Type: application/a2a+json ``` Previously, Bifrost accepted arbitrary authentication schemes but only reconstructed Basic and Bearer during delivery. A Digest registration, for example, succeeded, then every callback lacked usable authorization and retried until dead-lettering. The new validation rejects that configuration before the upstream call, so acceptance means the gateway can perform delivery from its own stored state. Tenant was dropped from durable push state and from the management view. It now survives registration, persistence, reads, and the rewritten upstream request, while callback URLs and credentials stay separated between the two hops. The usage views could only show Agent totals because no endpoint grouped Agent logs by Agent. The top-Agents endpoint provides the same shape the MCP top-tools endpoint does, so per-Agent tables can be built the same way. ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Push trace lifecycle, registration, and delivery behavior go test ./core/agent -run 'TestPush|TestListStoredPushConfigs' -count=1 # Push config persistence and migrations go test ./framework/configstore -count=1 # Tracer pooling and Agent history handlers go test ./framework/tracing -count=1 go test ./transports/bifrost-http/handlers -run 'AgentGatewayHistory' -count=1 # UI types cd ui && npx tsc --noEmit -p . ``` Expected outcomes: - an empty successful due-delivery poll closes its trace, releases it to the pools, and does not export it; - due work, errors, cancellations, and prune operations still export traces; - no-auth, Basic, and Bearer callback registrations are accepted when valid; - unsupported schemes and missing credentials fail before the upstream call; - tenant survives explicit and embedded push registration, persistence, local reads, and the management list; - downstream callbacks include the client token, configured authorization, and `application/a2a+json` media type; - the top-Agents endpoint returns one row per Agent for the given filters. No new configuration or environment variables are required. ## Screenshots/Recordings Not attached. ## Breaking changes - [x] Yes - [ ] No - `schemas.Tracer` gains `ReleaseTrace(*Trace)`. Custom tracer implementations outside this repository must add it. The built-in tracer and no-op tracer already do. - Unsupported callback authentication schemes now fail during registration instead of being accepted and failing asynchronously during delivery. These are configurations Bifrost could not previously deliver. ## Related issues None. ## Security considerations - Client callback tokens and credentials remain local to Bifrost and are not forwarded upstream. - The upstream Agent receives a separate Bifrost ingress URL and notification token. - Push-config reads, including the management list, redact tokens and authentication credentials. Tenant is routing metadata and is returned. - Rejecting unsupported callback authentication prevents registrations that appear valid but cannot be authenticated during delivery. - The top-Agents endpoint reuses the existing Agent history query scope and filters. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… off governed teams (#8156) ## Summary Fixes three config.json governance bugs found while testing access profiles attached to teams, customers and business units. | Mode | Bug | Effect | |---|---|---| | `source_of_truth: config.json` | The budget prune kept only ids listed in `governance.budgets` | Budgets declared inline on a team or customer were deleted on every other restart, leaving the entity unfunded. Inline virtual key and provider config budgets were deleted for good. | | `source_of_truth: config.json` | The rate limit prune kept only ids listed in `governance.rate_limits` | An inline rate limit still linked by its team or customer failed the prune on the foreign key: `failed to prune governance config: ... FOREIGN KEY constraint failed`, which stops startup. | | split | A team governed by an access profile whose entry was rewritten was saved with its inline budgets | gorm wrote them as the team's own budgets beside the profile, so its keys were capped twice. Governed customers were already handled. | ## Changes - `markInlineGovernanceLimits` adds the ids of budgets and rate limits declared inline on teams, customers, virtual keys and provider configs to the prune's keep set (`governanceLimitsToKeep`). - Governed teams are created and updated with `Budgets = nil`, the same way governed customers already are. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./transports/bifrost-http/lib -run 'TestSQLite_Team_MigratedToProfile|TestSQLite_SourceOfTruthConfigJSON_KeepsInlineLimitsAcrossRestarts' ``` Each new test fails without the fix: | Test | Without the fix | |---|---| | `TestSQLite_Team_MigratedToProfile_SplitRewriteSkipsItsInlineBudgets` | the new inline budget is written beside the profile | | `TestSQLite_SourceOfTruthConfigJSON_KeepsInlineLimitsAcrossRestarts/budgets` | team, customer, key and provider config budgets pruned on restart 2 | | `TestSQLite_SourceOfTruthConfigJSON_KeepsInlineLimitsAcrossRestarts/rate_limits` | the prune fails on the foreign key | ## Breaking changes - [ ] Yes - [x] No ## Related issues Enterprise end-to-end coverage for the access profile flows is in a companion PR in bifrost-enterprise. ## Security considerations None. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…gets from config.json (#8161) ## Summary Follow-up to #8156. Addresses the review comment there about retained provider-config budgets. Inline budgets on a virtual key, or on one of its provider configs, were written only when the key or provider config was **created**. On every later reload, `reconcileVirtualKeyAssociations` saved the provider config's columns and key associations, and `UpdateVirtualKey` saved the key's columns, but neither touched `budgets`: | Edit in config.json | Before | After | |---|---|---| | Same budget id, new `max_limit` | Ignored | Applied, spend kept | | New budget id added | Never created | Created under its key / provider config | | Budget id removed or swapped | Old row kept (split); under `source_of_truth: config.json` the old row was pruned and the new one never created, leaving the provider config with no budget | Old row deleted, new one created | | Budget moved between the key and a provider config | key → provider config: deleted and recreated; provider config → key: missing until the next restart | Deleted and created fresh under the new owner in the same reload, both directions | ## Changes - New `ConfigStore` methods `GetVirtualKeyBudgets` and `GetVirtualKeyProviderConfigBudgets` return the budgets an owner holds directly through `virtual_key_id` / `provider_config_id`, leaving out rows a model config has taken over. - `reconcileVirtualKeyAssociations` reconciles inline budgets in two passes: 1. `deleteUndeclaredBudgets`: the key and every provider config it has delete the budgets they hold that the file no longer declares on them, inline or in `governance.budgets`. A provider config the file drops is included, so its later deletion cannot cascade away a budget the key just wrote. 2. `upsertInlineBudgets`: an existing id goes through `UpdateBudget`, which keeps `current_usage`, `last_reset`, active overrides and model-config ownership. The stored owner columns and `created_at` are carried onto the file's row, since `UpdateBudget` saves the whole row. A new id is created under its owner. - `updateGovernanceConfigInStore` takes the ids `governance.budgets` declares, so a budget declared there is not treated as stale. It runs wherever `reconcileVirtualKeyAssociations` already runs: for changed keys in split mode, and for every declared key on every boot under `source_of_truth: config.json`. In split mode, a budget-only edit still does not reach the store unless something else in the key's entry changes, because the virtual key hash does not cover inline budgets. Adding them would make every key look changed on the first boot after upgrade and rewrite it from the file, so that is left as is. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./transports/bifrost-http/lib -run TestSQLite_VirtualKeyInlineBudgetsFollowTheFile ``` The test reloads one key in both modes: same id with a new `max_limit` (stored value, spend and owner checked, including a budget its model config owns), a budget moved to the provider config and back, then budgets added and swapped, with a `governance.budgets` row on the same key. Without the fix it fails in both modes. ## Breaking changes - [ ] Yes - [x] No ## Related issues Review comment on #8156. ## Security considerations None. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…8164) ## Summary Callers that copy the live fasthttp client and set `DisablePathNormalizing` (for example, to send an escaped path segment like `%2F` in a Bedrock guardrail ARN) were not getting that behavior. The live client forwards each request to an inner client, and fasthttp applies the setting from the client that actually writes the request. The inner client's value therefore overwrote the caller's, and the escaped segment was normalized before it was sent. This PR makes the inner client honor the caller's setting. ## Changes - Inner fasthttp clients are now cached by a `fasthttpClientKey` made of the client purpose and a `rawPath` flag. Before, the cache was keyed by purpose alone. - `liveFasthttpTransport.RoundTrip` now reads `DisablePathNormalizing` from the outer client. It then dispatches to the matching inner client, which is built with `DisablePathNormalizing` set when `rawPath` is true. - The raw-path variant is built lazily on first use. It lives in the same map as the default variant, so `UpdateProxyConfig` and `ApplyOptions` drop it along with everything else. A raw-path request therefore never reuses a client built for a previous proxy config. - Design note: the setting has to be applied per inner client rather than per request, because fasthttp reads it from the client that sends the request. The cost is at most one extra inner client per purpose, created only when a caller asks for it. - `currentFasthttpClient(purpose)` still returns the default variant, so existing callers are unaffected. - Added tests: - `TestLiveFasthttpClientHonorsDisablePathNormalizing` runs against a local server. A copy of the live client with `DisablePathNormalizing` sends `%2F` as written, while the default client still gets fasthttp's normalized path. - `TestRawPathInnerClientFollowsProxyUpdates` checks that the raw-path inner client has the flag set, that the default one does not, and that the raw-path client is rebuilt after `UpdateProxyConfig`. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./core/network/... -run 'TestLiveFasthttpClientHonorsDisablePathNormalizing|TestRawPathInnerClientFollowsProxyUpdates' go test ./... ``` Expected outcome: all tests pass. The new test confirms that a client with `DisablePathNormalizing` sends `.../guardrail%2Ftest/...` unchanged, while the default client does not. No new configs or environment variables are introduced. ## Screenshots/Recordings N/A (no UI changes) ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No auth, secrets, or PII are affected. Raw path preservation is opt-in per caller. The default behavior, including path normalization, is unchanged for everyone else. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ence is saved (#8160) ## Summary Fixes a bug where switching an OAuth credential to an env or vault reference was treated as a masked placeholder, so the update never wrote it. Previously, any reference value counted as "redacted". As a result, the old plaintext secret was silently kept on update. Adding OAuth with a reference to an agent that had no stored OAuth config also failed with "cannot preserve missing upstream OAuth credentials". Now only a real masked placeholder (such as `<REDACTED>`) means "keep the stored credential". Env and vault references are treated as new values and are written as submitted. ## Changes - Agent manager update: the upstream OAuth `ClientID` and `ClientSecret` checks now use `IsMaskedPlaceholder()` instead of `IsRedacted()`. These checks decide both whether to preserve the stored value and whether to reject an update that has nothing to preserve. - MCP client update handler: the token exchange `ClientID` and `ClientSecret` merge uses the same `IsMaskedPlaceholder()` check, so it matches the agent manager. Comments in both places now explain that an env/vault reference is never a placeholder. - Added `TestUpdateWritesAnOAuthSecretSwitchedToAReference`, which covers three cases: - A masked placeholder keeps the stored secret. - Switching a stored plaintext secret to an `env.` reference writes the reference and resolves to the new value. - Adding OAuth with an env reference to an agent that had none is accepted. - Design note: the nil checks on the submitted values were dropped from the agent manager conditions. This assumes `IsMaskedPlaceholder()` is safe to call on nil values. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the new regression test and the surrounding agent manager tests: ```sh # Core/Transports go version go test ./core/agent/... -run 'TestUpdateWritesAnOAuthSecretSwitchedToAReference|TestUpdateTransitionsAndRedactedSecretPreservation' go test ./... ``` Manual check: 1. Create an agent with upstream OAuth using a plaintext client secret. 2. Update it, round-tripping the redacted secret. The stored secret should be unchanged. 3. Update it again with `env.SOME_VAR` as the client secret. The stored value should now be the env reference and resolve to the variable's value. 4. For an agent with no OAuth config, update it to add OAuth with an env reference. The update should succeed. 5. Repeat steps 2 and 3 for MCP client token exchange credentials. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations - Env and vault references are no longer mistaken for masked placeholders. A user who rotates a credential to a reference now gets the new value, instead of the old plaintext staying in place. - Masked placeholders from redacted GET responses still resolve to the stored credential, so secrets are never overwritten with the mask. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…le cannot be created, instead of skipping them (#8115) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…8116) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds support for the >100k-token pricing tier in the cost engine. Claude Haiku 5.5 bills every token category at a higher rate once a prompt exceeds 100,000 tokens (input $0.50, output $2.50, 5m cache write $0.625, 1h cache write $1, cache read $0.05 per MTok, versus $0.10 / $0.50 / $0.125 / $0.20 / $0.01 at or under 100k). The datasheet already publishes these as `*_above_100k_tokens` keys, but the cost engine dropped them. Long Haiku 5.5 prompts were therefore billed at the base rate, about one fifth of the real cost. ## Changes - Added five nullable pricing fields across the pricing table, datasheet entry/options types, and the pricing sync column list so they survive re-syncs: - `input_cost_per_token_above_100k_tokens` - `output_cost_per_token_above_100k_tokens` - `cache_creation_input_token_cost_above_100k_tokens` - `cache_read_input_token_cost_above_100k_tokens` - `cache_creation_input_token_cost_above_1hr_above_100k_tokens` - Added a configstore migration (`add_100k_token_pricing_columns`) that adds the columns, with a rollback that drops them. The columns are nullable, so models without 100k rates keep their base rates. - Added a `TokenTierAbove100K` constant. The tiered input, output, cache read, cache creation and 1h cache creation rate resolvers now fall back to the 100k rate after checking the higher tiers (272k, 200k, 128k). The highest matching tier still takes precedence. - Applied the same tier logic to batch text cost, so batch pricing stacks with the above-100k rates. - Custom pricing overrides (`patchPricing`) can now set the 100k rates. - Exposed the new fields in the OpenAPI schemas (`openapi.json`, `governance.yaml`), the UI pricing override fields and `PricingOverridePatch` type, and the custom pricing and model catalog docs. The docs now list the supported tiers as 100k, 128k, 200k and 272k. - The boundary is strict: a prompt of exactly 100,000 tokens uses base rates, and 100,001 uses the above-100k rates. ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [x] Docs ## How to test Unit tests cover the new behavior: - `TestComputeTextCost_Haiku55Tiered100k` checks the exact-100k boundary, one token over, and 150k. It also checks the cost breakdown, `inference_geo` stacking and batch stacking. - `TestPatchPricing_Above100kRates` checks that overrides apply to the new fields. - `TestUpsertModelPricesBatch_Above100kColumns_SurviveResync` checks that the new columns survive a pricing re-sync. - The UI `pricingFields` test now expects 117 fields and the token unit for the new keys. ```sh # Core/Transports go version go test ./framework/configstore/... ./framework/modelcatalog/... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` An e2e case (#183 in the provider harness collection) sends chat completions to `anthropic/claude-haiku-5-5` with a short prompt and with a prompt over 100k tokens. It then asserts that the logged `cost_breakdown.total_cost` matches the published rates for each tier. It needs Anthropic credentials, and the over-100k case fails without this fix. ## Screenshots/Recordings N/A. The UI change only adds new fields to the pricing override list. ## Breaking changes - [ ] Yes - [x] No The new columns are nullable and additive. Models without 100k rates are unaffected, and older binaries ignore the columns. ## Related issues N/A ## Security considerations No auth, secrets, or PII changes. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…unit tests for backoff, fixed keys and filtered pools (#8117) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…responses and governance refusals (#8118) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…d fallback coverage (#8119) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ing fallbacks, key restrictions, streaming rate limits and team budgets (#8120) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Overview
Claude on Bedrock rejects `oneOf`, `anyOf`, and `allOf` at the root of a tool `input_schema` ("input_schema does not support oneOf, allOf, or anyOf at the top level"). Bedrock's default Converse egress forwarded `toolSpec.inputSchema` verbatim, so tools with a root composition, such as Codex's `automation_update`, failed on `bedrock/` but worked on `bedrock_mantle/`. This PR flattens those schemas for Claude models on Converse, the same way the Anthropic provider already does.
## Changes
- Exported `NormalizeAnthropicToolInputSchema` so the Bedrock provider can reuse the Anthropic normalizer. Existing Anthropic callers and tests were updated to the new name.
- The Bedrock Chat builder (`convertToolConfigFromFiltered`) and Responses builder (`ToBedrockResponsesRequest`) now normalize a tool's schema when it has a root `oneOf`/`anyOf`/`allOf` and the model is in the Anthropic family. This also covers aliases that resolve to an Anthropic model family.
- The normalizer returns a new schema and does not mutate the caller's request.
- Schemas without a root composition skip normalization, so the common case adds no overhead.
- Non-Claude Converse models (Nova, Llama, Qwen) accept root compositions and still receive them unchanged.
- `convertToolConfigFromFiltered` now returns an error. Schemas that exceed the normalizer's branch-count or depth limits fail the request with a caller-facing invalid-request error (HTTP 400) instead of being sent to Bedrock. The legacy `convertToolConfig` helper ignores this error as before.
- Added Go unit tests for both the Chat and Responses paths:
- root `oneOf` flattening, with `required` as the intersection of the alternatives
- alias-based `anyOf` flattening
- root `allOf` flattening, with `required` as the union of the members
- non-Claude passthrough
- the too-complex error
- a check that the caller's schema is not mutated
- Added `BenchmarkConverseToolSchemaClaude`, which compares a plain object schema against a flattened root `oneOf`.
- Added e2e folder "222. Bedrock Converse Claude tool-schema-compatibility" to the provider harness collection. It covers these routes:
- `/v1/responses`
- `/openai/v1/responses` (streaming)
- `/v1/chat/completions`
- `/anthropic/v1/messages`
- native `/bedrock` converse and invoke
- the gateway-side 400 for the branch limit
- the Nova passthrough case
## Type of change
- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
Run the Go tests and the new benchmark:
```sh
# Core/Transports
go version
go test ./...
# Targeted
go test ./core/providers/anthropic/... ./core/providers/bedrock/... -run 'ToolSchema|NormalizeAnthropicToolInputSchema'
go test ./core/providers/bedrock/ -bench BenchmarkConverseToolSchemaClaude -run '^$'
# UI
cd ui
pnpm i || npm i
pnpm test || npm test
pnpm build || npm run build
```
For the e2e check, run folder 222 of `tests/e2e/api/collections/provider-harness.json` against a gateway with Bedrock credentials. Expected results:
- Claude requests with root `oneOf`/`anyOf`/`allOf` tools return 200, and `raw_request` shows no root composition keyword.
- The over-limit schema returns a 400 mentioning "branch count".
- The Nova request keeps its root `oneOf`.
No new configs or environment variables.
## Screenshots/Recordings
N/A, no UI changes.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
N/A
## Security considerations
No auth, secrets, or PII changes. Oversized or deeply nested tool schemas are rejected locally with a 400 before they reach Bedrock, which uses the normalizer's existing complexity limits.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
## Overview The GPT Live e2e suite (`tests/live`, fake upstream) no longer runs on every pull request. It now runs as a gating job in the release pipeline, against the gateway binary the pipeline has already built. The sideband attach test is also tightened to cover session ownership by virtual key. ## Changes - **Release pipeline:** Added a `test-live` job that downloads the prebuilt `bifrost-http-binary` artifact and runs `test-live.sh fake` with `LIVE_PREBUILT=1`. It runs on the live profile, so no provider keys are needed. It installs `uv` so the OpenAI Python SDK cases run instead of skipping, and it uploads `test-reports/live.xml` and `tmp/live-gateway.log` as an artifact. It only runs when the gateway needs a release and tests aren't skipped. - **Release gating:** `test-live` is now in the `needs` list and the success/skipped condition of every downstream release job: core, framework, plugins, bifrost-http, and the Docker jobs. The final job that depends on the test results also waits on it. A live-suite failure now blocks a release. - **Live Tests workflow:** Removed the `pull_request` trigger and the PR-specific concurrency key and job condition. The workflow now runs only the nightly long sessions and manual dispatches in either mode. - **`test-live.sh`:** With `LIVE_PREBUILT=1` and an executable `tmp/bifrost-http` present, it uses that binary. Otherwise it builds from the checkout as before, so local and nightly usage is unchanged. - **Sideband attach test:** - `TestAttach_SidebandIsHeldToTheVirtualKey` now checks that a sideband attach from a different virtual key, even one allowed on the same provider, gets a `404` before any key is selected or dialed. The provider connection count stays unchanged. - The opener's sideband is asserted to dial with the key its own virtual key allows. - `findSidebandLog` takes a `wantStatus` argument, so the refused (`error`) and successful (`success`) rows can be told apart when both are logged under the same virtual key. Its failure message now names the status. - **Trade-off:** The suite no longer gives PR feedback, but it gates every release. The 17-minute sessions stay nightly-only. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the fake suite locally, then confirm the workflow behavior in GitHub Actions. ```sh # Core/Transports go version go test ./... # Live suite (builds the gateway from the checkout) ./.github/workflows/scripts/test-live.sh fake # Live suite using an already-built binary LIVE_PREBUILT=1 ./.github/workflows/scripts/test-live.sh fake # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Expected outcomes: - A pull request no longer triggers the Live Tests workflow. - A release pipeline run with a gateway release shows a `test-live` job. The release jobs wait on it. - The `live-test-report` artifact is uploaded even when the job fails. - The nightly schedule and manual dispatch of Live Tests still work. New environment variable: `LIVE_PREBUILT=1` makes `test-live.sh` use `tmp/bifrost-http` instead of building. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations The new job uses `contents: read` permissions, runs with Harden Runner in egress-block mode with an explicit allowlist, and does not persist checkout credentials. It needs no provider secrets because it uses the fake upstream. The updated attach test verifies that a sideband can only attach to a session opened by the same virtual key. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…nflated (#8431) ## Overview Buffered request decompression was refusing ordinary traffic. Each gzipped request reserved its full expansion allowance from the in-flight budget up front. Any body more than cap/100 compressed (about 1 MiB at the default 100 MiB cap) has the whole cap as its allowance. So with a 100 MiB cap and a 400 MiB budget, the fifth concurrent multi-MB SDK request got a `503`, even though the process held only a small share of the budget. The budget is now charged for the bytes a body actually inflates to, not the ceiling it could have reached. ## Changes - Added a `budgetedReader` that charges each decompressed byte to the process-wide budget as it is produced. If the budget cannot cover the next chunk, decoding stops with `errDecompressionBudgetExhausted`, which the middleware answers with `503`. - `decodeRequestBodyWithLimit` now takes the budget and returns how many bytes were charged. The middleware releases that amount with a `defer`, so budget is returned on success, on the `413` expansion refusal, and on corrupt or truncated streams. - Left the other limits as they were: - The per-body expansion bound (100× compressed size, 1 MiB floor, cap as ceiling) still returns `413`. - The budget is still four times the request body cap. - Streaming bodies still take no charge. - Updated the docs for `max_request_body_size_mb` (`client.mdx`) and the runtime contract to say that each body counts what it actually inflates to. - Updated the e2e harness description for the decompression-expansion case. It notes that concurrent refusal is covered by a Go test, because the harness sends requests one at a time. Trade-off: a body can now begin decompressing when the budget is nearly full and be refused partway through. Its partial charge is released immediately. Concurrent buffers still cannot exceed the budget. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Run the decompression middleware tests and benchmarks: ```sh # Core/Transports go version go test ./transports/bifrost-http/handlers/ -run 'TestRequestDecompressionMiddleware' -race go test ./transports/bifrost-http/handlers/ -run xxx -bench 'BenchmarkRequestDecompressionMiddleware' ``` Expected outcomes: - `ConcurrentMultiMBGzipNotRefused`: eight concurrent ~2 MiB-inflated gzip bodies under a 10 MiB cap are all admitted and return 200 with the body intact. - `InFlightBudget`: four bodies that each inflate to the full 1 MiB cap fill the 4 MiB budget, and the next request gets `503`. - `BudgetReleasedOnFailedDecode`: after repeated `413` (bomb) and `400` (truncated gzip) responses, four full-size bodies are still all admitted, so no budget leaked. - `BenchmarkRequestDecompressionMiddlewareParallel` reports `503/op`. It should be 0 for a 10 MiB SDK-like body at the concurrency levels tested. No new configuration or environment variables. ## Screenshots/Recordings N/A, no UI changes. ## Breaking changes - [ ] Yes - [x] No Requests that were previously refused with `503` because of over-reservation are now accepted. Clients that depended on that behavior are not expected. ## Related issues N/A ## Security considerations The protections against decompression bombs are unchanged. The per-body expansion bound still returns `413` before routing and auth. The process-wide in-flight cap of four times the body limit is still enforced, now against real decompressed bytes, so memory use stays bounded. Failed or aborted decodes release their charge, so malformed requests cannot exhaust the budget over time. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…nvalid UTF-8 and jsonb-rejected escapes (#8428) ## Summary A single log entry containing a NUL byte, invalid UTF-8, or a JSON escape that Postgres `jsonb` refuses could fail the writer's whole multi-row `INSERT` (SQLSTATE 22021 / 22P05 / 22P02). The writer then fell back to slower per-row inserts. This PR sanitizes string columns before they reach Postgres, so one bad request can no longer fail a batch. It also keeps the stored JSON columns castable to `jsonb`, so one bad row can't break dashboard queries. ## Changes - Added GORM create and update callbacks, installed only by the Postgres log store, that clean every persisted `string` and `*string` column of a row before it is written. They run after the model's `BeforeCreate`/`BeforeSave` hooks, because `SerializeFields` is what produces the JSON column strings. - They cover slice and single-row creates, struct updates (including `Updates(&Log{...})`), and `map[string]interface{}` updates. - SQLite and ClickHouse accept these bytes, so they don't get the callbacks. - Added `pgSafeText`, which drops NUL bytes and replaces invalid UTF-8 with U+FFFD. A clean string is returned as-is with no allocation. - Columns that hold serialized JSON are also passed through `sanitizeJSONForJSONB`. This matters because GIN expression indexes cast `metadata` and the owner ID columns to `::jsonb` during the `INSERT` itself, and ranking queries cast `error_details`, `attempt_trail` and others on read. - Rewrote `sanitizeJSONForJSONB` to walk the string escape by escape instead of using blind string replacement: - `\u0000` is dropped. - Unpaired surrogate escapes become `\ufffd`. - Valid surrogate pairs are kept. - An escaped backslash followed by the text `u0000` is left intact. The old replace could turn that into a different escape or invalid JSON. - Some columns are never cast to `jsonb`, so they only get the TEXT-level cleanup and otherwise keep their text exactly as written: `raw_request`, `raw_response`, `passthrough_request_body`, `passthrough_response_body`, `content_summary` and `routing_engine_logs`. `routing_engine_logs` always starts with `[`, and `content_summary` can start with `{` or `[` when a message starts with JSON, so the JSON sanitizer would wrongly rewrite them. - Pointer fields are replaced with a new pointer rather than written through, because the old pointee may be shared with caller state. - Added a benchmark for the callback on clean rows, including one with a 1 MiB input history. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Unit tests cover `pgSafeText`, its zero-allocation behavior on clean input, and the escape-aware JSON sanitizer. The integration tests need a Postgres instance, since they use the existing Postgres test setup. The new Postgres tests verify the following: - A batch of 67 rows with one poisoned row inserts successfully. The poison cases are a raw NUL, invalid UTF-8, a NUL escape or lone surrogate in GIN-indexed `metadata`, and a NUL in `team_ids`. - `error_details`, `output_message` and `params` still cast to `jsonb` after a NUL or lone surrogate reaches them. - Literal `\u0000` text in message content round-trips unchanged. - Struct updates, map updates and MCP tool logs are sanitized. - Plain-text columns keep literal `\u0000` and `\ud800` text. ```sh # Core/Transports go version go test ./framework/logstore/... go test ./framework/logstore/ -run 'TestPGSafeText|TestSanitizeJSONForJSONB|TestPGTextSafety' -v go test ./framework/logstore/ -bench BenchmarkPGTextSafetyRow -run '^$' go test ./... ``` ## Screenshots/Recordings N/A (no UI changes). ## Breaking changes - [ ] Yes - [x] No Stored values change only for data Postgres would have rejected. NUL bytes are removed, and invalid UTF-8 and unpaired surrogates become U+FFFD or `\ufffd`. ## Related issues N/A ## Security considerations No auth or secrets are involved. Request and response content is still stored, but NUL bytes are removed and invalid byte sequences are replaced. Plain-text debug columns otherwise keep their text exactly as written. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…8429) ## Summary Large-response previews stored for logging could be cut in the middle of a multi-byte UTF-8 character. The preview is truncated by byte count (1 MiB max, or whatever was prefetched), so the cut can land inside a rune. The resulting invalid UTF-8 is rejected by Postgres with SQLSTATE 22021, which fails the log writer's batch INSERT. This PR makes the preview always end on a character boundary. ## Changes - Added a `trimIncompleteRune` helper in `core/providers/utils/largeresponse.go`. It checks at most the last `utf8.UTFMax` bytes and drops a trailing partial UTF-8 sequence. - `FinalizeResponseWithLargeDetection` now applies the helper to the preview in both paths: the one that has the full body bytes and the one that uses the prefetch buffer. - Design decision: bytes that were never valid UTF-8 (for example a stray `0xff` in a binary body) are left alone. Only an incomplete sequence at the cut point is trimmed, so binary previews are not altered. - Added tests in `core/providers/utils/streamingclient_test.go`: - `TestFinalizeResponseWithLargeDetection_PreviewEndsOnRuneBoundary` serves a body of repeated 3-byte `€` runes and checks that the stored preview is valid UTF-8. It covers two cases: a known `Content-Length` with a 64KB prefetch, and a chunked body cut at the threshold. Neither size is a multiple of 3, so the cut lands mid-rune. - `TestTrimIncompleteRune` covers empty input, ASCII, complete multi-byte runes, cuts after 1 or 2 of 3 bytes, a cut after 3 of 4 bytes, and a binary byte that must be kept. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the new tests for the utils package, then the full suite: ```sh # Core/Transports go version go test ./core/providers/utils -run 'TestFinalizeResponseWithLargeDetection_PreviewEndsOnRuneBoundary|TestTrimIncompleteRune' go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Expected: all tests pass, and the stored preview is valid UTF-8 even when the body is made of multi-byte characters and is cut mid-character. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No security impact. The change only trims the logged preview, and it prevents invalid input from causing log write failures. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Overview
Binary data in log body fields (audio passthrough responses, multipart uploads, Bedrock event streams) was being written to the logs table as raw bytes. Postgres rejects NUL bytes and invalid UTF-8 in `TEXT` columns with SQLSTATE 22021, which failed the entire batch `INSERT` and dropped every other log in that batch. This PR replaces binary bodies with a short size placeholder before they are written, so one binary body can no longer take down a whole batch.
## Changes
- The logging writer now checks `RawRequest`, `RawResponse`, `PassthroughRequestBody` and `PassthroughResponseBody` on both inserted logs and log updates before they are batched. Any body that holds binary data is replaced with `<binary content omitted: N bytes>`.
- A body counts as binary if it contains a NUL byte or is not valid UTF-8. Text bodies, including multi-byte text such as `café ☕`, are stored unchanged.
- **Design decision:** a partial UTF-8 sequence at the very end of a body is treated as a preview cut by byte count, not as binary data. The partial character is dropped and the text is kept, so truncated JSON previews are not replaced by a placeholder.
- **Trade-off:** binary bodies are no longer viewable in the logs, even on stores that would have accepted them. Raw bytes rendered as garbage there anyway.
- Added unit tests for the text/binary decision, covering:
- empty and JSON bodies
- multi-byte text
- cuts mid-rune (2-byte and 4-byte)
- NUL bytes
- invalid bytes
- Added a batch-level test that processes an insert and an update containing audio, multipart, and event-stream bodies. It verifies that the stored rows hold placeholders, and that a text passthrough request body is untouched.
- Added an e2e collection request to the provider harness. `POST /openai_passthrough/v1/audio/speech` returns MP3 bytes, then the test polls `/api/logs/{id}` and asserts that `passthrough_response_body` is a placeholder.
## Type of change
- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
Run the new unit tests for the logging plugin:
```sh
# Core/Transports
go version
go test ./plugins/logging/... -run 'TestOmitBinaryBody|TestProcessBatchOmitsBinaryBodies'
go test ./...
# UI
cd ui
pnpm i || npm i
pnpm test || npm test
pnpm build || npm run build
```
For the e2e check, run the provider-harness collection against a Bifrost instance backed by Postgres and with an OpenAI key configured. Run the "POST /openai_passthrough/v1/audio/speech: binary response body logged as placeholder" request in section 9. Expected results:
- The speech request returns 200 with an `x-bifrost-request-id` header.
- The log row for that ID eventually appears.
- Its `passthrough_response_body` matches `<binary content omitted: N bytes>`.
No new configs or environment variables are required.
## Screenshots/Recordings
Not applicable. There are no UI changes.
## Breaking changes
- [ ] Yes
- [x] No
Binary bodies that were previously stored as raw bytes (on stores that accepted them) are now stored as a placeholder string. Text bodies are unaffected.
## Related issues
N/A
## Security considerations
No auth or secret handling changes. Dropping binary payloads from stored logs also means less raw upload and audio content is retained in the database.
## Checklist
- [x] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
## Summary This PR fixes a bug where a Postgres log update could silently keep the old column value. It happened when the new value contained only NUL bytes. `pgSafeText` stripped the NULs and returned `""`. GORM's struct `Updates` treats `""` as a zero value and skips the field, so the previous value stayed in place. If that field was the only one being updated, GORM also reported that no row was updated. ## Changes - `pgSafeText` now returns `U+FFFD` instead of an empty string when the input is non-empty but cleans down to nothing, such as a value made only of NUL bytes. - Clean strings are still returned as-is without allocating. - Design decision: the replacement character is stored in place of the empty string. This keeps the write from being skipped, and the column shows that the original value was unusable instead of keeping stale data. - Added a `nul_only_stays_non_empty` case to `TestPGSafeText`. - Added `TestPGTextSafety_StructUpdateNULOnlyValueIsWritten`. It creates a log with `RawRequest` set to `"previous body"`, updates it with a NUL-only `RawRequest` through a struct update, and checks that the stored value is `U+FFFD`. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the log store tests. The Postgres-backed text safety tests need a Postgres instance, as in the existing setup. ```sh # Core/Transports go version go test ./framework/logstore/... -run 'TestPGSafeText|TestPGTextSafety' go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Expected outcome: all tests pass. A struct update with a NUL-only value replaces the previous column value with `U+FFFD`. ## Screenshots/Recordings N/A, no UI changes. ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No auth, secrets, or sandboxing changes. Sanitization of NUL bytes and invalid UTF-8 is unchanged. The only difference is that a NUL-only value is now stored as `U+FFFD`, so the update is not dropped and stale data is not kept. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Overview Adds a setting, `enforce_vk_object_ownership`, that controls whether provider objects are restricted to the virtual key that created them. Until now the restriction was always on. Operators who share files, batches, containers and similar objects across virtual keys had no way to turn it off. The setting is on by default, so existing deployments behave as before. ## Changes - **New setting:** `client.enforce_vk_object_ownership` in `config.json` and the config schema, plus a **Restrict Provider Objects to the Creating Virtual Key** switch under **Settings > Security**. It defaults to `true`. - **Behavior when off:** - Every virtual key can reach every provider object. - List pages are not narrowed. - The ownership refusals are skipped, including those made when no config store is present. - Live sessions follow the same switch. - **Creators are still recorded while off.** Turning the setting back on therefore also binds objects created in the meantime. Automatic `code_interpreter` containers keep their marker so their creator is still recorded. - **Live sessions:** a failed owner write, or a missing config store at admission, no longer ends a session while the setting is off. The failure is logged instead. - **Applies without a restart.** The governance plugin and the live-session handler read the setting through a pointer into the live client config. Saving it in the UI or via `PUT /api/config` affects the next request. - **Storage and defaults:** - A new migration adds `config_client.enforce_vk_object_ownership` and backfills existing rows to `true`. - A `config.json` that omits the field is read as on. - Only the opt-out is written into the config hash, so existing configs keep their hash on upgrade. - Because the column defaults to `true` and the update path deletes and recreates the row, an explicit `false` is forced through after the write so it is not lost. - **UI:** the switch reads an omitted value as on, so a gateway that predates the setting does not show it as off, and an unrelated save does not turn it off. - **Docs:** the OpenAI SDK files-and-batch page now describes the setting and what happens when it is off. - **Tests:** - Unit tests for the migration, the config round trip and hash, governance, and live sessions. - A Postman provider-harness flow that turns the setting off, confirms another key can retrieve the file, then restores the original value. - Playwright specs for the omitted-as-on case and for persisting the switch. - Benchmarks now cover both the on and off paths. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [x] Docs ## How to test ```sh # Core/Transports go version go test ./... # Targeted go test ./framework/configstore/... -run "EnforceVKObjectOwnership" go test ./plugins/governance/... -run "OwnershipFollowsTheClientSetting" go test ./transports/bifrost-http/handlers/... -run "LiveSessionOwnershipFollowsTheClientSetting" # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Manual check: 1. With a virtual key A, upload a file through `/v1/files?provider=openai`. 2. Retrieve it with virtual key B. Expect `404` while the setting is on (the default). 3. Turn off **Restrict Provider Objects to the Creating Virtual Key** under **Settings > Security**, or set `"enforce_vk_object_ownership": false` in the client config. 4. Retrieve it again with key B. Expect `200`, and expect list pages to show all objects. 5. Turn the setting back on. Expect objects created while it was off to be bound to their creators again. New config: - `client.enforce_vk_object_ownership` (boolean, default `true`). ## Screenshots/Recordings UI change: one new switch on the Security settings page, with no other layout changes. ## Breaking changes - [ ] Yes - [x] No The default is on, and the migration backfills existing rows to `true`, so current behavior is unchanged unless an operator opts out. ## Related issues Link related issues and discussions here. ## Security considerations - The setting is on by default, and the migration, the JSON default and the UI all treat a missing value as on, so an upgrade or a partial config cannot silently weaken isolation. - Turning it off lets any virtual key allowed on a provider read, list, download, delete and edit objects created by other keys. Operators should only do this when keys are mutually trusted. - Requests without a virtual key are not restricted, with or without this setting. - Creators are still recorded while it is off, so re-enabling it takes effect immediately for all objects. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Overview Brings the end-to-end provider harness collection (Postman) back in line with current gateway behavior, so rows that were failing or asserting stale behavior now check what the gateway actually does. ## Changes **Response content check** - The non-empty-content check now recognizes four more reply shapes: - Responses `compaction` items (`POST /v1/responses/compact`, e.g. xAI). - MCP tool results (`role: tool`, from `/v1/mcp/tool/execute`). - JSON-RPC replies from the `/mcp` endpoint, where a `result` counts as content and an `error` does not. - Realtime client secrets (`value` plus `expires_at`). **OpenAI prompt caching (#8334)** - Bifrost no longer forces `prompt_cache_options.mode=explicit`, because the breakpoint is honoured in OpenAI's default implicit mode. - Rows 63.7, 64.1.5, 64.1.6, 64.8.5, the Bedrock row and the Astra row now assert that `prompt_cache_options` is absent. `prompt_cache_breakpoint` is still expected on the wire. **Astra reasoning** - Row 115.1 now expects Astra chat to be routed to Responses (#7806), so the normalized effort is read from `reasoning.effort`. **Rerank** - The usage assertion now reads `usage.search_units` instead of `completion_tokens_details.num_search_queries`. **Setup-token auth rows** - The provider-endpoint-updates rows (proxy URL change, Azure key endpoint) and the MCP log deletion rows now expect the setup lock's 401. The previous 403 "authenticated admin session" check no longer matches. - These rows strip the injected `X-Bifrost-Setup-Token` header in a pre-request script, since a request carrying the token is authenticated and would be allowed. Folder descriptions explain this. **Header and ID renames** - Virtual key rows use `x-bf-vk` (value `sk-bf-test-key`) instead of `X-Bifrost-VK`. - Request ID checks read `x-bifrost-request-id` instead of `x-request-id`. **Models and fixtures** - Cohere v2 chat rows (including LangChain, LiteLLM and PydanticAI) move from `command-r-plus` to `command-a-03-2025`. The Python integration config's alternatives list is updated to match. - The Cohere drop-in "list models" row (a POST with an empty body to `/cohere/v1/models`) is removed. - The `cohere/command-a-vision-07-2025` decisions row is removed. - `parasail-deepseek-v3` becomes `parasail-deepseek-v4-flash`. - The web-search cross-cut uses `openai/gpt-5-search-api`. - GenAI invalid-parameter rows use the `gemini/gemini-2.5-flash` model path. - The Gemini Responses function-call row raises `max_output_tokens` from 20 to 512. - Bedrock Cohere embed row 150.6 now sends an unknown `truncate` value (`MIDDLE`) instead of `END`. - Bedrock Cohere v3 embedding rows send `truncate` at the top level rather than under `extra_params`. - Anthropic `/v1/complete` row 151.13 now targets `claude-opus-4-7` with `top_k`. - The container create row passes `provider` in the body instead of the query string. - The `tool_removal` row uses an `input_text` block instead of `text`. **Code mode budget rows** - The integer-result row now builds a large integer by repeated squaring (`1 << 128`, then six squarings) instead of `1 << 10000`. - The raised-steps row now sets `result = 'done'` after its loop. **Streaming leak checks** - Rows 166.2 and the custom-tool stream row strip `extra_fields.raw_response` before checking for leaked Responses or function-call event names. The raw upstream echo no longer causes false positives. **Expectation tags** - Rows that intentionally expect errors are tagged so the harness treats them correctly. Tags used are `[EXPECT-4XX]`, `[EXPECT-400]`, `[EXPECT-404]` and `[EXPECT-204]`. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the provider harness against the default profile (`tests/integrations/python/config.json`, dashboard auth unconfigured, setup token set). Folders 138 and 163 should be run against that profile and will fail on an auth-enabled one by design. ```sh # Example: run the setup-lock folders in order FEATURE=mcp-log-delete-auth PARALLEL=0 <harness runner> FEATURE=provider-endpoint-updates <harness runner> ``` Expected outcomes: - The setup-lock rows return 401 mentioning `X-Bifrost-Setup-Token`. - The prompt-cache rows show no `prompt_cache_options` in `raw_request`. - The Cohere, MCP, compaction and realtime rows pass the content check. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues Relates to #8334, #7806, #7769, #6328 ## Security considerations The management-plane rows (provider endpoint updates, MCP log deletion) still verify that an unauthenticated caller is refused before any state is changed. They now assert the setup-lock 401 and remove the setup token header so the refusal path is actually exercised. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary The GPT Live session panel in the log detail sheet now respects the sheet's Messages role filter. Before, toggling roles filtered the regular message views but left the live transcript and delegation items untouched. This PR also makes the backend response ids and tool results in delegation cards easier to read. ## Changes - The log detail sheet passes its active role filter to `LiveSessionView`. The filter is passed only when some roles are hidden, so the default view is unchanged. - The live transcript is filtered by speaker. Delegation items are filtered by the same role mapping the Responses view uses: - A function call and its result both fall under the tool role. - The answer falls under assistant. - Reasoning keeps its own role. - When the filter hides every item in a delegation, the card shows "Hidden by the message filter" instead of the misleading "No answer". - Backend response ids in delegation cards are shortened to the head and tail of the id (for example `resp_08dc…f426`) so they fit on one line. Hovering shows a tooltip with the full id and a label such as "Backend response 1 of 2". The chips are right-aligned. A function call spans two backend responses, so the card lists every id the delegation billed. - A function call result row now shows the name of the tool it answers instead of the bare call id. `ResponsesItemRow` takes an optional `callName` prop, and the delegation card resolves it from the matching `function_call` item. Rows fall back to the call id when no name is found. - The filtering and id helpers (`visibleTranscript`, `visibleDelegationItems`, `shortId`, `toolNamesByCall`) live in a new `liveSessionView.utils.ts` file with unit tests. ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Open a log for a GPT Live session that has a transcript and delegations, including a delegation with a function call. 2. In the Messages filter, deselect roles one at a time. The transcript lines and delegation items should show or hide to match the selected roles. 3. Select only the tool role. The function call and its result should remain, and the transcript should be empty. 4. Hide every role present in a delegation. The card should read "Hidden by the message filter". 5. Hover a response id chip. The tooltip should show the full id with its "Backend response N of M" label, and clicking should copy the full id. 6. Check that a function call result row is labeled with the tool name rather than the call id. ```sh # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` ## Screenshots/Recordings Before/after screenshots of the live session panel with the Messages filter applied and the response id tooltip can be added here. ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations None. This is a display-only change and adds no new data handling. Redaction mapping is still applied to the content. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…esponses routes (#8421) ## Summary A provider error that arrives after a stream has started on the OpenAI-compatible Responses routes (`/openai/v1/responses`, `/openai/responses`, Azure `/openai/deployments/{d}/responses`, and the litellm / langchain / pydanticai copies) was written as a bare `data: {...}` SSE frame with no `event:` line. The native `/v1/responses` handler sends `event: error`, which is what the OpenAI SDKs read as a stream error. ## Changes - `transports/bifrost-http/integrations/router.go`: new `streamErrorEventType` picks `error` for OpenAI-type routes when the error's request type is a Responses stream (`ResponsesStreamRequest` / `ResponsesRetrieveStreamRequest`). It reads the error's request type rather than the path because the Azure deployments route serves chat and responses on one path; errors with no request type fall back to a `/responses` path check. - The payload is unchanged (the same BifrostError JSON the native handler sends). - Chat and text-completion streams keep bare `data:` frames. Anthropic routes keep their pre-formatted `event: error` frames. Bedrock and GenAI are untouched. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh cd transports && go test ./bifrost-http/integrations/ -run TestStreamErrorEventFramingPerRoute -v ``` `TestStreamErrorEventFramingPerRoute` (8 cases: `/openai/v1/responses`, `/openai/responses`, GET retrieve stream, Azure deployments with a responses error, a responses error with no request type, `/openai/v1/chat/completions`, Azure deployments with a chat error, `/anthropic/v1/messages`). - Red on the parent commit: all 5 responses cases failed, the wire was `data: {"is_bifrost_error":false,"status_code":502,...,"request_type":"responses_stream"}}` with no `event:` line. - Green: all 8 pass; the whole `integrations` package passes. **Provider harness: exempt.** No request reliably makes a live provider fail after the stream has started (validation errors arrive before the stream as normal HTTP errors, and the profile has no fault-injection knob), so a harness case would pass before and after this change. The behaviour is pinned at unit level. **Cost:** none on successful chunks. The new code runs only on the stream-terminating error path: one type comparison and a switch, no allocations. ## Breaking changes - [ ] Yes - [x] No ## Related issues Found while triaging a full provider-harness run. ## Security considerations None. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ive retrieval tools, candidateCount and logprobs (#8422) ## Summary Three places where the Gemini Responses conversion path (used by `/v1/responses`, the Go SDK, Vertex and bare-model `/genai`) lost or mis-sent request fields that the chat path or the native Gemini API handles: 1. JSON mode (`responseMimeType: application/json` + schema) was sent together with function tools to pre-Gemini-3 models, and Google rejects it: `Function calling with a response mime type: 'application/json' is unsupported`. The chat converter already guarded this; the Responses converter did not. 2. `/genai` → Vertex silently dropped Gemini-native tools with no neutral Responses form (`retrieval` incl. `vertexAiSearch` / `vertexRagStore`, `googleSearchRetrieval`, and also `enterpriseWebSearch`, `googleMaps`, `urlContext`, `computerUse`). 3. Bare-model `/genai` dropped `candidateCount`, `logprobs` and `responseLogprobs` (and `seed`) on the way out, and flattened multiple candidates into one on the way back. ## Changes - **JSON mode + tools:** the chat guard is now a shared helper, `dropJSONResponseFormatWithToolsIfUnsupported` (`core/providers/gemini/utils.go`), called from both the chat path and `convertParamsToGenerationConfigResponses` right after the text config is applied. - **Native-only tools:** `geminiNativeOnlyTools` carries them verbatim through `ExtraParams["gemini_native_tools"]`; `ToGeminiResponsesRequest` removes that key from wire ExtraParams and appends the tools to `geminiReq.Tools` without mutating the source slice (retry-safe). No mixed-tool dropping is applied to them, so invalid combinations are rejected by Google visibly instead of being dropped. - **Generation config:** `TopLogProbs` → `generationConfig.logprobs` (not clamped, so out-of-range values reach Google and are rejected there), a positive value also sets `responseLogprobs`; `candidate_count`, `response_logprobs` and `seed` map into generationConfig and are added to `responsesGenerationConfigExtraParamKeys` so they don't leak as unknown snake_case fields. - **Multiple candidates:** the neutral output is built from candidate 0; extra candidates and candidate 0's `logprobsResult` go to `ProviderExtraFields` (`additionalCandidates`, `logprobsResult`) and are restored by `ToGeminiResponsesResponse` for `/genai` clients. Trade-off: `/v1/responses` with `candidateCount > 1` now returns candidate 0 as the output, with the other candidates under `provider_extra_fields.additionalCandidates`. Previously every candidate's parts were flattened into one output list. Streaming with multiple candidates is not handled. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh cd core && go test ./providers/gemini/ ./providers/vertex/ ./schemas/ cd ../transports && go test ./bifrost-http/integrations/ ``` Each test was red on the parent commit and green after: - `TestResponsesStructuredOutputWithToolsConflict` (gemini_test.go): red `expected: "" actual: "application/json"` for 2.5 + json_schema / json_object; 3.x and no-tools cases keep the format. - `TestGenAINativeRetrievalToolsSurviveNeutralRoundTrip` (responses_test.go): red, only `codeExecution` survived; green, each native tool reaches the wire once across two attempts and no carrier key leaks. - `TestGenAICandidateCountAndLogprobs_ReachGenerationConfig`, `TestToGeminiResponsesRequest_TopLogProbsMapsToLogprobs`, `TestGenAIMultipleCandidatesAndLogprobsResult_PersistThroughBifrostRoundTrip`: red `expected: 2 actual: 0`, nil logprobs, and `should have 1 item(s), but has 2`. Provider harness: - New folder **222. Structured output alongside function tools on pre-3 models (structured output with tools)**: gemini `/v1/responses`, vertex `/v1/responses`, gemini `/v1/responses` streaming, bare-model `/genai`. - Existing cases now expected green: 151.10, 151.11, 151.12, 156.4 (Vertex retrieval tools reach Google and are rejected), 147.7 (candidateCount=2), 157.2 (logprobs out of range reaches Google). **Cost:** one bool and one string compare per Gemini Responses request with a text config; native-tool carry allocates only when such tools are present; a few ExtraParams lookups only when ExtraParams is non-nil. No new allocation on the common path. ## Breaking changes - [ ] Yes - [x] No `/v1/responses` output shape changes only for requests that set `candidateCount > 1` (see trade-off above). ## Related issues Found while triaging a full provider-harness run. ## Security considerations None. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… tier on chat bridge, bare sonar routing, mcp server_label, legacy prompt framing (#8423) ## Summary Six independent request-fidelity bugs, each reproduced with a failing unit test first: 1. **Cohere replay:** a replayed conversation put a `thinking` item after `text` in the same assistant message; Cohere returns 400 `content items of type 'thinking' must come before...`. 2. **`/v1/decisions`:** emulation always sent `tool_choice: required`; Cohere reasoning models reject `tool_choice is not supported for this model`. 3. **`service_tier`** (with `speed`, `inference_geo`) was lost when a chat request is served through Responses, and in the reverse direction and on streams. 4. **Perplexity:** bare `sonar` was routed to `/v1/responses`, which rejects it (`model "sonar" is not supported`); only `sonar-*` was excluded. 5. **MCP `server_label`:** `ResponsesMCPListTools.ServerLabel` and `ResponsesMCPToolCall.ServerLabel` were both embedded in `ResponsesToolMessage` with the same JSON tag, so the field was ambiguous and dropped in both directions; replaying `mcp_list_tools` without it gets a 400 from OpenAI. 6. **Legacy `/v1/complete`:** a prompt already framed as `\n\nHuman: ...\n\nAssistant:` was wrapped again. Bedrock's Claude text path reuses the converter and had the same bug. ## Changes - `core/providers/cohere/responses.go`: pending thinking is prepended when an assistant turn is flushed; reasoning after an assistant turn with text/tool calls starts a new turn; reasoning, text and function_call items fold back into one assistant message (thinking + text in content, calls in `tool_calls`); a function_call_output flushes the open assistant turn first. This also fixes a second assistant message overwriting the first and text + function_call being emitted after the tool result. - `core/decisionemulation.go`: `decisionToolChoice` returns nil (omitted) when `ResolveModelCaps(...).SupportsToolChoice(...)` is false; the no-record fallback is the new `cohere.DefaultSupportsToolChoice` (false for `-reasoning` models). - `core/schemas/mux.go`: copy `ServiceTier`, `Speed`, `InferenceGeo` in Responses→chat, chat→Responses, and Responses stream chunk → chat chunk. - `core/providers/perplexity/responses.go`: `isPerplexityResponsesSupported` strips `perplexity/`, lowercases, and excludes exact `sonar` as well as `sonar-*`. - `core/schemas/responses.go`: `ServerLabel` hoisted to a single field on `ResponsesToolMessage` (next to `ItemCaller`); the empty `ResponsesMCPToolCall` type removed; deep copies and Anthropic MCP converters updated. JSON wire shape unchanged. - `core/providers/anthropic/text.go`: `frameAnthropicTextPrompt` keeps an already-framed prompt and only appends `\n\nAssistant:` if missing. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh cd core && go test . ./schemas/ ./providers/cohere/ ./providers/perplexity/ ./providers/anthropic/ ./providers/bedrock/ ./providers/openai/ cd ../framework && go test ./streaming/ ``` Red on the parent commit, green after: - `TestCohereReplayKeepsThinkingBeforeText` (7 subtests): red `messages[1].content[2] is thinking after a non-thinking block`, the dropped turn, and `5 items, want 4`. - `TestEmulateDecisionToolChoicePerSurface` (+3 cases): red `want it omitted` x2. - `TestResponsesChatBridgeCarriesServedTier`: red `responses->chat dropped service_tier: <nil>`. - `TestResponses_RoutesSonarFamilyToChatCompletions`: red `path = "/v1/responses", want "/chat/completions"` for `sonar`, `perplexity/sonar`, `SONAR`. - `TestResponsesMCPItemsKeepServerLabel`: red `sonic dropped server_label` / `encoding/json dropped server_label` for `mcp_list_tools`, `mcp_call`, `mcp_approval_request`. - `TestToAnthropicTextCompletionRequestFraming`, `...PromptArray`, `TestToBedrockTextCompletionRequestClaudeFraming`: red double frame `"\n\nHuman: \n\nHuman: Say hi\n\nAssistant:\n\nAssistant:"`. Two failures in these packages are pre-existing on dev and unrelated: bedrock `TestAnthropicIngressMantleReplayUsesResponsesInputShapes`, openai `TestRealtimeWebRTCUpstreamErrorCarriesRetryHint`. Provider harness: new folder **223. Replay and routing fidelity (replay-routing-fidelity)**: - 223.1 cohere replay with the whole transcript in the request, asserting thinking comes first in every assistant message on `raw_request`. - 223.2 streaming chat served through Responses carries `service_tier`. - 223.3 / 223.4 bare `sonar` on `/v1/responses`, non-stream and stream. - 223.5 / 223.6 capture and replay `mcp_*` items, asserting `server_label` reaches OpenAI. Existing cases covering the rest: `/v1/decisions - matches jev anchors - cohere/command-a-reasoning-08-2025`, `... - perplexity/sonar`, `128.3` (non-stream service_tier), `152.1` (mcp tool). `/v1/complete` framing is pinned at unit level only: no live model serves the legacy endpoint natively (with compat text→chat on, the request goes through Messages). Open: after the Cohere fix, a turn with text → reasoning → tool call goes out as two consecutive assistant messages; 223.1 confirms live whether Cohere accepts that. **Cost:** Cohere request conversion only: one closure and a prepend slice per assistant turn with reasoning. Decisions path only: one caps lookup. Three pointer copies on the chat/Responses bridge. One `ToLower` per Perplexity Responses call. One pointer copy per tool item in deep copies. No new allocation on the common path. ## Breaking changes - [ ] Yes - [x] No `ResponsesMCPToolCall` (an empty embedded type) is removed from `core/schemas`; Go SDK users who referenced it directly should use `ResponsesToolMessage.ServerLabel`. The JSON shape is unchanged. ## Related issues Found while triaging a full provider-harness run. ## Security considerations None. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ale harness cases (#8424) ## Summary Once the `sse_mcp` fixture (examples/mcps/remote-test-server on :3012) is running during harness runs, the harness profile's `sse_mcp` client auto-injects `echo` / `add` / `greet` plus its server instructions into every chat and responses request. In a full run that broke about 190 unrelated cases: inflated tool counts, Gemini/Vertex server tools evicted by the injected functions, structured output turning into prose, Nova failing with `Model produced invalid sequence as part of ToolUse`, thinking lost behind auto-executed tool turns, and shifted prompt-cache markers. This PR scopes MCP auto-injection in the harness while keeping injection itself under test, and fixes a batch of stale or wrong harness cases found in the same run. No gateway code changes. ## Changes **MCP injection scope** - `tests/integrations/python/config.json`: `mcp.tool_manager_config.disable_auto_tool_inject: true`. - `disable_auto_tool_inject` applies to the whole gateway (no per-client or per-VK switch), so the auto-inject path is tested in a `[SERIAL]` window that turns injection on through `PUT /api/config` and restores it, the same read/modify/restore pattern as code-mode-limits. - New folder **224. MCP tool auto-injection scope (mcp-tool-injection) [SERIAL]** (14 rows): (c) no header gets no `sse_mcp-*` tools and no `<mcp_server>` instructions on the wire; (b) `x-bf-mcp-include-clients: sse_mcp` puts tools + instructions on the wire and echo auto-executes; (a) with auto-inject on and no header, tools + instructions reach the wire on chat and responses and echo auto-executes (proved by the tool result key and a token in the answer); then restore and re-check (c). - No existing case relied on auto-injection. Provider `injected_tools` (folder 198) resolve through the MCP manager directly and are unaffected. **Stale / wrong cases** - OAuth DCR cases (folders 112, 114, 139): `https://harness.example/cb` → `http://127.0.0.1/cb` (8 occurrences); #7960 only accepts loopback, cursor callbacks, or `allowed_redirect_uris`. - `Code mode configurable limits` and `138. Provider endpoint updates` tagged `[SERIAL]`: they are stateful and their rows were split across parallel shards and ran out of order. - `[Costing]` cases: inline `openai/gpt-4o-mini`, `openai/text-embedding-3-small`, `openai/whisper-1` (the `{{model}}`-style variables are only defined by the inference runner) and use `tests/e2e/api/fixtures/sample.mp3`. - Gemini-shape assertions skip `thought: true` parts (8 cases). - 187.1 / 187.2 use `openai/gpt-5.1-codex-mini` (Responses-only), since `convert_chat_to_responses` only converts Responses-only models. - 168.1–168.3 no longer require `reasoning.summary === 'auto'`; nothing sets that default for OpenAI. - codex-desktop UA case parses SSE frames instead of searching raw text (which matched `chat.completion.chunk` inside `extra_fields.raw_response`). - Live named-provider refusal asserts on the parsed `error.message`. - `__skipShape` also skips `/.well-known/`. - 147.9 video input asserts `fileData` / `video/mp4` on the wire and a description of the video instead of `prompt_tokens > 200` (Gemini reports only TEXT tokens for https video URLs). - DeepSeek structured-output rows use `json_object` (DeepSeek returns `This response_format type is unavailable now` for `json_schema`); 48.3.C fixed the same way. - Gemini stop-sequence cases: prompt that emits the words in order, stop on `three`, assert `two` present and `three` / `four` absent; thinking disabled. `Gemini: stop sequences` and `Vertex Gemini: stop sequences` move from 2.5-pro to 2.5-flash because 2.5-pro cannot disable thinking. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs Test collection and harness profile only. ## How to test Structural validation (no live calls): ```sh node tests/e2e/api/runners/augment-provider-harness.mjs --source tests/e2e/api/collections/provider-harness.json --out tmp/harness-augmented.json node tests/e2e/api/runners/filter-collection.mjs --source tmp/harness-augmented.json --out tmp/filtered.json --feature "mcp-tool-injection" node tests/e2e/api/runners/filter-collection.mjs --source tmp/harness-augmented.json --out tmp/serial.json --only-serial ``` `--only-serial` returns the code-mode-limits, 138 and 224 folders whole and in order; `--exclude-serial` contains none of them. Profile schema errors are unchanged (3 pre-existing, none from the new field). Live: the injection folder needs the `sse_mcp` fixture listening on :3012. ## Breaking changes - [ ] Yes - [x] No ## Related issues Found while triaging a full provider-harness run. ## Security considerations OAuth harness cases now register loopback callbacks, matching the redirect allow-list. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…cache points (#8425) ## Summary Three bugs in the MCP agent loop (auto-executed tools on chat / responses): 1. **Earlier turns' content was dropped.** `executeAgent` returned only the last iteration's response. Usage was merged across turns but content was not, so a turn-1 answer followed by an auto-executed tool call came back to the client as empty content, and turn-1 thinking/reasoning was missing even though its reasoning tokens were billed. 2. **Follow-up turns re-ran routing.** Each follow-up turn re-entered `handleRequest`: the requested-route stamp was overwritten with the already-rewritten model, and routing / session routing / the fallback chain ran again, so a turn that turn 1 had served from a fallback went back to the failing primary (surfaced as a 404). 3. **Prompt-cache points counted the gateway's MCP instructions.** `addServerInstructionsToRequest` prepends the server instructions as a system message before `promptCacheChatRequest` places cache points, so every index shifted by one and the auto marker landed on `<mcp_server ...>` instead of the caller's content. ## Changes - **Carry earlier content** (`core/mcp/agentadaptors.go`, `core/mcp/agent.go`): `carryForward` / `applyCarriedContent` for both adapters. Chat: text and reasoning from earlier turns go before the final turn's; reasoning details keep their signatures and are re-indexed; executed tool calls are not handed back. Responses: earlier output items are kept whole (so `encrypted_content` survives) and placed first; executed `function_call` items are left out. - **Pin continuation turns** (`core/bifrost.go`, `core/utils.go`): the loop records turn 1's `RoutingInfo` on ctx under the reserved `BifrostContextKeyMCPAgentServedRoute` (cleared on exit). `continueOnServedRoute` keeps the requested-route stamp, pins the provider/model that served turn 1, drops fallbacks, skips PreRequestHooks and session routing (PreLLMHooks still run), keeps the fallback index/identity when turn 1 was a fallback, and gives each fallback turn a fresh `FallbackRequestID` so the logging plugin no longer overwrites turn 1's row. Key pins from turn 1 already on ctx carry over; otherwise normal key selection runs within the pinned provider/model. - **Cache points** (`core/mcp/toolmanager.go`, `core/providers/utils/promptcache.go`, `core/bifrost.go`): the injected instructions text is recorded on ctx (`BifrostContextKeyMCPInstructionsText`, a pointer). New `InjectChatCacheBreakpointsSkipping` / `InjectResponsesCacheBreakpointsSkipping` place points as if the injected message were absent, then map positions back; the old functions call them with -1 and are unchanged. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh cd core && go test . ./mcp/... ./providers/utils/ ./schemas/ make test-mcp ``` Red on the parent commit, green after: - `TestAgent_EarlierTurnTextAndReasoningReachClient_ChatFormat` / `_ResponsesFormat` (core/internal/mcptests/agent_basic_test.go): red `"" does not contain "1706.03762"`, `final message has no assistant fields`, `"[]" should have 2 item(s), but has 0`. - `TestAgentLoopContinuationKeepsRequestedRoute` (core/bifrost_test.go, real Bifrost + in-process echo tool + mock upstream): red `final RoutingInfo requested route = ("openai", "gpt-4o"), want (openai, requested-alias)`. - `TestAgentLoopContinuationStaysOnServingFallback`: red `primary was called 2 times, want 1`. - `TestPromptCacheChatRequest_SkipsInjectedMCPInstructions` (4 subtests): red `the auto marker landed on the gateway's MCP instructions`, `index 1 should name the caller's second message`. - `TestParseAndAddTools_RecordsInjectedInstructionsForCachePlacement`: red `the injected instructions were not recorded on the context`. `make test-mcp`: 738 tests pass. Provider harness: new folder **225. MCP agent loop keeps earlier turns, the turn-1 route and caller cache points (mcp-agent-continuation)**, 6 cases, all opting in with `x-bf-mcp-include-clients: sse_mcp` (auto-injection is off in the profile after the parent PR): - anthropic claude-sonnet-4-6 chat with thinking: final content non-empty, reasoning present, echo call not handed back. - anthropic claude-sonnet-4-6 responses: reasoning item before the message, no echo `function_call`. - openai gpt-4o-mini requested-route: same header and body assertions as folder 128. - openai gpt-4o-mini provider-only fallback: same assertions as #7538 (is-fallback, fallback-index 1). - anthropic_pc_auto chat: `mcp_server` on the wire but unmarked; the auto marker on the caller's system prompt. - anthropic_pc_edge chat: exactly the `ed-last` and `ed-idx1-user` markers, none on `mcp_server`. The first four depend on the model calling echo; the prompt-cache cases depend on the fixture advertising server instructions. Both need the `sse_mcp` fixture listening on :3012. **Cost** - Requests without MCP: one extra ctx lookup in `handleRequest`; the cache-placement lookup only runs when prompt-cache injection is already on. `BenchmarkAgentContinuationChecksOnPlainRequest` (`-benchmem`): 38–44 ns/op, 0 B/op, 0 allocs/op (new code; there was no prior equivalent). - Agent-loop requests only: one `RoutingInfo` copy per loop, the carried-content slices, one fresh fallback ID per fallback turn, one routing log line per turn. - Instructions skip: one copy of the input slice, only when MCP instructions were injected and prompt-cache injection is on. - Retained memory: nothing outlives the request. ctx holds two pointers (the turn-1 `RoutingInfo` and the instructions string the request already holds); the served-route pointer is cleared when the loop exits. ## Breaking changes - [ ] Yes - [x] No Clients of the agent loop now receive earlier turns' text/reasoning before the final turn's content, where they previously received only the final turn. ## Related issues Found while triaging a full provider-harness run. ## Security considerations Two new internal context keys (`BifrostContextKeyMCPAgentServedRoute`, `BifrostContextKeyMCPInstructionsText`) hold small pointers set by the gateway only. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…8432) ## Summary DeepSeek's chat endpoint rejects `response_format` of type `json_schema` with HTTP 400 ("This response_format type is unavailable now") and only accepts `json_object`. Structured-output requests sent to DeepSeek through Bifrost therefore failed. This PR rewrites those requests so they succeed and still return JSON in the requested shape. ## Changes - When the provider is DeepSeek and the request carries a `json_schema` response format, `ToOpenAIChatRequest` now sends `{"type":"json_object"}` instead. - The schema is passed to the model as an added system message, which also satisfies DeepSeek's requirement that the word "json" appear in the prompt. The message contains: - an instruction to respond with only a valid json object - the schema name and description, when present - the schema bytes verbatim - The new system message goes after any leading system/developer messages, so the client's own system prompt stays at the top. - The downgrade is not gated on `SupportsResponseSchema`. The datasheet marks DeepSeek rows `supports_response_schema=true` because they support JSON mode, and the limitation belongs to the DeepSeek API surface, not to individual models. - Only requests with `json_schema` pay for the rewrite. `json_object` requests on DeepSeek, requests without a response format, and `json_schema` requests on other providers (OpenAI, Groq, xAI) are left untouched. - The caller's request is never mutated. The response format is replaced by pointer and the messages are inserted into a new slice, since the original request is shared with fallbacks and post-hooks. - The `/v1/responses` route is covered too. DeepSeek serves it through `ToChatRequest`, where `text.format` arrives as an in-memory `response_format`. - DeepSeek is removed from the delegating-providers list in `TestResponseFormatSurvivesDelegatingProviders`, since its schema is now intentionally moved into the prompt. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Unit tests cover the conversion, the Responses route, the scope of the downgrade, and the full provider path against a capturing test server. A benchmark compares DeepSeek conversion with and without a `json_schema` format. ```sh # Core/Transports go version go test ./core/providers/openai/... ./core/providers/deepseek/... go test ./core/providers/openai -bench BenchmarkToOpenAIChatRequest_DeepSeekResponseFormat -run '^$' # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` E2E: a new collection group, "226. DeepSeek structured output downgrades json_schema to json_object", in `tests/e2e/api/collections/provider-harness.json` requires a DeepSeek key. It covers: - `/v1/chat/completions` (non-streaming) - `/v1/chat/completions` (streaming) - `/v1/responses` Each case expects a 200 and content that parses as JSON with a string `name` and a numeric `age`. Without the fix, DeepSeek returns a 400. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No auth or secrets changes. The caller-supplied schema is now included in a system message sent to DeepSeek. It was already being sent as part of the request. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… byte-faithfully on the Responses path (#8433) ## Overview Fixes two places where Gemini-native data was lost when a `/genai` request or response went through Bifrost's Responses path: - `candidates[].urlContextMetadata` was dropped on responses. Clients using the `urlContext` tool could see `groundingMetadata` but not which URLs were fetched. - Sub-fields of Gemini-native tools that Bifrost doesn't model were dropped on requests. For example, `retrieval.dynamicRetrievalConfig` collapsed to `{"retrieval":{}}`, so an invalid request became an ungrounded 200 instead of being rejected by Google. ## Changes - **`urlContextMetadata` round trip (non-streaming):** The metadata is stored in `ProviderExtraFields["urlContextMetadata"]` when converting Gemini to Bifrost Responses. It is restored onto the candidate when converting back, and when merging terminal candidates the first non-nil value wins. - **`urlContextMetadata` round trip (streaming):** - The metadata can arrive on any chunk, not only the terminal one, so the stream state now keeps it and attaches it to `response.completed`. - It is rebuilt on the GenAI stream egress. - It is cleared in `flush()` so a pooled stream state doesn't leak one stream's metadata into the next. - **Extraction helper:** `extractGeminiURLContextMetadata` accepts either the in-memory pointer or a JSON-decoded map, so the data survives `ProviderExtraFields` passing through JSON. - **Verbatim native tool forwarding:** - `Retrieval`, `GoogleSearchRetrieval`, `EnterpriseWebSearch`, `GoogleMaps`, `URLContext` and `ToolComputerUse` now keep the raw ingress bytes. Their `MarshalJSON` emits those bytes as-is. - Typed fields are still decoded for Bifrost's own reads, and values built in code still marshal from their fields. - Google, not Bifrost, now decides whether an unknown or newer sub-field is valid. - Only requests that carry one of these tools pay for the byte copy. - **Tests and benchmarks:** - Unit tests cover native tool sub-field preservation (including a JSON round trip of the `ExtraParams` carrier), the Vertex request body build, and `urlContextMetadata` in both non-streaming and streaming paths, including no leakage across a flushed state. - New benchmarks compare plain function-tool requests and plain responses against the native-tool and `urlContext` cases. - A new e2e Postman case, `152.14b`, checks that `streamGenerateContent` with `urlContext` returns `urlContextMetadata` with a successful retrieval in some chunk. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the Gemini and Vertex provider tests, then the benchmarks to check that the common (function tool, plain response) path doesn't regress: ```sh # Core/Transports go version go test ./... # Targeted go test ./core/providers/gemini/... ./core/providers/vertex/... \ -run 'URLContextMetadata|NativeToolUnknownSubFields|NativeToolSubFieldsReachWire' go test ./core/providers/gemini/ -run xxx -bench 'GenAIToolsResponsesEgress|GeminiResponsesResponseRoundTrip' -benchmem # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Expected outcomes: - A `/genai` `generateContent` or `streamGenerateContent` call with `tools: [{"urlContext": {}}]` returns `candidates[].urlContextMetadata.urlMetadata` with `URL_RETRIEVAL_STATUS_SUCCESS`. - A request with `{"retrieval":{"dynamicRetrievalConfig":{...}}}` reaches Google with that body unchanged. - The e2e case `152.14b` in `tests/e2e/api/collections/provider-harness.json` passes (needs a real `genaiKey`). No new configs or environment variables. ## Screenshots/Recordings N/A (no UI changes). ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations Native tool entries are now forwarded to the provider byte-for-byte instead of being re-marshalled from typed structs. Bifrost no longer filters unmodelled sub-fields on these tools, and Google validates them. The raw bytes only come from the client's own request and are not logged or exposed elsewhere. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…lbacks; forward raw gemini logprobs (#8434) ## Summary Stops Bifrost from sending a request to providers the caller never named when the primary provider has already rejected that request as malformed. It also makes the Gemini raw `generateContent` path forward logprobs to Google instead of silently stripping them. Previously, when the model-catalog resolver promoted every other provider serving a bare model to a fallback, a 400 from the primary (for example, `logprobs` out of range) could be answered by one of those providers. The caller got a 200 from a provider they never asked for, and the error that explained the problem was hidden. ## Changes - Added an `Inferred` flag to `schemas.Fallback`. It marks fallbacks Bifrost added on its own, as opposed to ones named by the caller or a routing config. It is never serialized (`json:"-"`). - The model-catalog resolver now sets `Inferred: true` on the fallbacks it generates. - Session-affinity reordering preserves `Inferred`, so a reordered inferred chain stays inferred. - `shouldTryFallbacks` now takes the context. For an inferred chain, if the primary error is classed `FailureClassCallerFault`, it skips the fallbacks and returns the primary error. Both the regular and streaming orchestrator loops use this. The decision is logged to the routing engine log. - Caller-fault errors are a 400/409/413/415/422 with a provider error body, or a 404 naming a request field. - Still falls back: model-not-found 404s (`FailureClassModelAccess`), rejected keys, rate limits, 5xx and network failures, since another provider can succeed. - Unchanged: caller-named and routing-config chains still fall back on any error. - The Gemini raw `generateContent` normalization no longer strips `generationConfig.responseLogprobs` and `generationConfig.logprobs`. They now reach Google as sent, matching the typed path, so Google validates the range and returns `logprobsResult`. `presencePenalty`, `frequencyPenalty` and `fallbacks` are still stripped, because the raw path has no per-model drop-params check. - Cost: one bool check on the first fallback decision. Classification runs only on the inferred-chain path. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./core/... ./plugins/modelcatalogresolver/... # Fallback decision benchmark go test ./core -run '^$' -bench BenchmarkShouldTryFallbacks ``` Expected outcomes: - `TestScenarioInferredFallbacksStopOnCallerFault` passes for streaming and non-streaming requests: - An inferred chain returns the primary's 400/422 and makes no fallback call. - An inferred chain still falls back on 503, 429, 401 and 404 `model_not_found`. - A caller-named chain still falls back on the same 400. - The session-affinity test confirms a reordered inferred chain stays inferred. - The Gemini tests confirm `responseLogprobs` and `logprobs` are kept and the penalties are removed. - The e2e Postman collection has two new cases against `gemini/gemini-2.5-flash`: - 157.2b: `logprobs: 99` returns a 4xx from Google that mentions logprobs, and no other provider answers. - 157.3b: `responseLogprobs: true` with `logprobs: 3` returns 200 with `logprobsResult`. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No Behavior changes only for automatically inferred fallback chains. A request the primary rejects as malformed now returns that error instead of being answered by another provider. Chains named by the caller or a routing config are unaffected. ## Related issues N/A ## Security considerations No auth or secret handling changes. Requests the primary rejects as malformed are no longer sent to additional providers the caller did not name, which reduces unintended data exposure. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Overview The Postgres index builders in the log store could skip building an index they needed. Their validity checks looked up an index by name and table name only, without restricting the lookup to the current schema. If another schema in the same database had its own `logs` or `mcp_tool_logs` table with a valid index of the same name (another deployment, or a test package), the check reported the index as present and valid. The builder then skipped the build, leaving this schema's table unindexed. As a result, cost and latency sorts fell back to sequential scans. ## Changes - The metadata GIN index, the team/business-unit/customer GIN indexes, the logs and MCP histogram covering indexes, and the performance indexes now check validity through the existing schema-aware `pgIndexValid` helper. They no longer run their own inline `pg_class`/`pg_index` queries. - Added a small `pgDropIndexConcurrently` helper that drops an INVALID remnant from an interrupted `CREATE INDEX CONCURRENTLY` before the rebuild. It replaces the inline `DROP INDEX CONCURRENTLY IF EXISTS` calls and uses one error message format. - Added `TestIndexBuildersIgnoreSameNamedIndexInOtherSchema`. It creates a shadow schema with valid same-named indexes on `logs`, `mcp_tool_logs`, and every performance index table. It removes those indexes from the test schema, runs the builders, and asserts that each index is built and valid in the test schema. - Design note: the inline queries used `COALESCE(bool_and(indisvalid), false)`, so a missing index read as not valid. `pgIndexValid` keeps that behavior, so the skip/drop/rebuild flow is the same apart from the schema scoping. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test The new test needs a Postgres-backed log store test database, as with the other Postgres migration tests. ```sh # Core/Transports go version go test ./framework/logstore -run TestIndexBuildersIgnoreSameNamedIndexInOtherSchema go test ./framework/logstore/... ``` Expected: all tests pass. The new test confirms that every index is built and valid in the test schema even when a same-named valid index exists in another schema. No new configs or environment variables. ## Screenshots/Recordings Not applicable (no UI changes). ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No auth, secrets, or PII are affected. The index names and table names passed to the helpers are internal constants, not user input, so identifier interpolation in the `DROP INDEX` statement remains safe. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
#8435) ## Overview File, batch, and video list pages look up ownership for every id on the page. Each id is fanned out across every provider name that shares the provider's API, so a 10,000-item page can pass more ids than SQLite (32,766) or Postgres (65,535) allow as bind parameters. Vertex list pages also run one `LIKE` term per short id, which can exceed SQLite's expression tree depth limit of 1000. In both cases the lookup failed and took down the whole page. This PR makes those lookups work for arbitrarily large id lists. ## Changes - `GetProviderObjectsByIDs`, `GetProviderJobsByIDs`, and `GetProviderJobsByFileIDs` now bind lists longer than `providerIDListInlineMax` (1000) as a single parameter through `queryscope.InStrings`. For `GetProviderJobsByFileIDs`, the list is bound once per column (input, output, error file). - Lists of 1000 ids or fewer keep the plain `IN ?` clause. This is the common case, and SQLite plans it faster than a `json_each` scan. - `GetProviderObjectsByObjectIDSuffixes` and `GetProviderJobsByJobIDSuffixes` now read suffix patterns in chunks of `suffixPatternChunkSize` (500). Each `LIKE` term deepens the OR chain by one, so chunking keeps every statement well under SQLite's depth cap of 1000. Results are deduplicated by row ID across chunks. - `providerObjectOwnersOf` in the governance plugin deduplicates unresolved ids through a set once a request carries more than 8 ids, replacing a quadratic `slices.Contains` scan. Single-object requests still allocate no set. - Added `TestInListLimitProviderObjectsAndJobsByIDs` and `TestInListLimitProviderObjectsAndJobsBySuffixes`, which run id lists and 10,000 suffixes against the stores and expect only the real rows back. - Added `TestHiddenProviderObjectsReadsAPageOverTheBindParameterLimit`, a governance test with 40,002 ids. - Added `BenchmarkGetProviderObjectsByIDs`, which measures a typical 20-id page on SQLite to confirm the short-list path is not slowed down. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the new in-list limit tests, the governance test, and the benchmark, then the full suite: ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Targeted runs: ```sh go test ./framework/configstore -run 'TestInListLimitProviderObjectsAndJobs' go test ./plugins/governance -run 'TestHiddenProviderObjectsReadsAPageOverTheBindParameterLimit' go test ./framework/configstore -run '^$' -bench BenchmarkGetProviderObjectsByIDs ``` Expected: the over-limit lookups return only the matching rows without error, and the benchmark shows the 20-id case still using the plain `IN` path. ## Screenshots/Recordings Not applicable. There are no UI changes. ## Breaking changes - [ ] Yes - [x] No ## Related issues None linked. ## Security considerations Ownership filtering is unchanged. The governance test confirms another key's file is still hidden on a page over the bind-parameter limit, while the caller's own files and unrecorded files stay visible. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…8436) ## Overview Deleting or updating an MCP client with an id that doesn't exist returned a 500 from the HTTP API. The config store's "not found" error was a plain formatted string, so the handlers couldn't tell it apart from a real database failure. Both operations now return a 404 for an unknown id. ## Changes - `UpdateMCPClientConfig` and `DeleteMCPClientConfig` in the config store now wrap `ErrNotFound` when no client row matches the id. The message still names the id, so existing error text stays useful. - `deleteMCPClient` returns 404 when the store reports `ErrNotFound`. It returns before the runtime MCP manager is called, since there is nothing to remove. It also skips the error-level log for this case. - `updateMCPClient` maps `ErrNotFound` to 404 and keeps 500 for all other store errors. - Tests: - A store test checks that delete and update on an unknown id both satisfy `ErrNotFound` and mention the id. - A handler test checks that an unknown id returns 404 and never reaches the manager. It also checks that a known id still returns 200 and is removed from both the store and the manager. - A new e2e request in the provider harness collection (section 137) sends `DELETE /api/mcp/client/<random uuid>` and expects a 404 with "not found" in the body. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the new store and handler tests, then the e2e collection against a harness instance. ```sh # Core/Transports go version go test ./framework/configstore/... -run 'MCPClientConfigUnknownID|UpdateMCPClientTools' go test ./transports/bifrost-http/handlers/... -run 'TestDeleteMCPClient_UnknownIDIsNotFound' go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Manual check: `curl -i -X DELETE http://localhost:8080/api/mcp/client/does-not-exist` should return `404` with a body containing "not found". Before this change it returned `500`. No new configs or environment variables. ## Screenshots/Recordings Not applicable. There are no UI changes. ## Breaking changes - [ ] Yes - [x] No Clients that treated the previous 500 as the response for an unknown MCP client id will now receive a 404. All other responses are unchanged. ## Related issues None linked. ## Security considerations None. The error message still contains only the id the caller supplied, and no additional information is exposed. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary This PR fixes three separate problems. A complexity analyzer config in `config.json` could overwrite the classifier chosen in the dashboard. A race in `abandonDelivery` could dereference a message the worker had already released. A new SQLite test covers read-then-write transactions failing on a stale snapshot with "database is locked". ## Changes - **Complexity analyzer config merge (`transports/bifrost-http/lib/config.go`)** - A `config.json` that omits the classifier section has no opinion on it. Previously the file config was normalized before hashing and merging, which injected a default classifier. That default was hashed and then applied over the classifier selected in the dashboard, so a keyword-only edit in the file reset it. - `complexityAnalyzerConfigFromFile` now returns both the normalized config and the file as written. The hashes and the split-mode merge use the file as written, so omitted sections are treated as "no opinion". - Source-of-truth mode still persists the normalized config. - Added `TestMergeGovernanceConfig_FileWithoutClassifierKeepsDashboardClassifier`. It checks that a keyword edit in the file leaves a dashboard-selected classifier and its empty classifier hash intact. - **Delivery abandonment race (`core/bifrost.go`)** - Once the CAS in `abandonDelivery` succeeds, the worker owns the message and may release it and nil its fields concurrently. The function was still reading `m.Context` after the swap. - `m.Context` is now read before the CAS. The doc comment says that nothing on the abandoning side may touch the message after the swap. - **SQLite transaction locking (`framework/configstore/sqlite_test.go`)** - Added `TestSQLiteTransactionsTakeTheWriteLockOnBegin`. In WAL mode, a deferred transaction that reads and then writes after another pooled connection commits fails immediately with "database is locked", because the busy timeout does not apply to a stale snapshot. - The test verifies that a concurrent writer queues behind an open read-then-write transaction and that both writes land in order. This matters because startup workers write to the store while the first request-driven write runs. - **OpenAI realtime WebRTC test (`core/providers/openai/realtime_test.go`)** - The upstream error test now expects the upstream 429 status and message to be carried through, instead of a synthetic 502. This keeps the retry hint (`RetryAfter` of 7000) next to the status that gives it meaning. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the Go tests, focusing on the new and updated ones: ```sh # Core/Transports go version go test ./... # Targeted go test ./transports/bifrost-http/lib -run TestMergeGovernanceConfig go test ./framework/configstore -run TestSQLiteTransactionsTakeTheWriteLockOnBegin go test ./core/providers/openai -run TestRealtimeWebRTCUpstreamErrorCarriesRetryHint go test -race ./core/... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Expected: all tests pass. With a `config.json` that omits the classifier, editing only keywords leaves the dashboard-selected classifier unchanged. ## Screenshots/Recordings Not applicable. There are no UI changes. ## Breaking changes - [ ] Yes - [x] No ## Related issues None linked. ## Security considerations No auth, secrets, or PII are affected. Reading the context before the CAS removes a potential use-after-release on the message. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
✨ Features
POST /api/agents,POST /api/agents/inspect, or the new top-levelagentsarray in config.json, reconciled by name at startup) and clients reach it at/agents/a2a/<name>/.well-known/agent-card.json,/agents/a2a/<name>/json-rpc(JSON-RPC and SSE streaming) and/agents/a2a/<name>/rest/..., plus an optional shared gRPC listener routed by<agent>.<a2a_grpc_base_domain>:<a2a_grpc_port>. The runtime caches cards and clients, refreshes a stale interface on 404/405/410/426, supports header, OAuth and TLS upstream auth, runs A2A plugin hooks, and delivers push notifications through a durable, retried outbox (/agents/a2a/<name>/push/callback, authenticated with a per-config token). Push callbacks to loopback or private addresses stay blocked unless the newserver.a2a_allow_private_push_callbacksis on (default off). Access is governed per virtual key withagent_grants(omit to keep,[]to clear, unknown agents rejected atomically) andallow_by_defaultper agent; ungranted or disabled agents return403. Every A2A operation and stream event gets its own log row (relational, ClickHouse, or hybrid with object-store payloads) in a new Agent Logs workspace with session grouping, raw capture and DAC-bounded filters, LLM and MCP logs carry the A2A correlation id, and A2A client spans plus thea2a.client.operation.durationmetric reach connectors. The dashboard adds an agent sheet with auth fields and a card preview, push-config UI and an Agent Gateway settings page; Helm addsbifrost.agents[],bifrost.client.a2aExternalClientUrl,bifrost.server.a2aGrpcBaseDomainandbifrost.server.a2aGrpcPort(feat: add agent gateway core contracts #7772, feat: persist agent registrations and push state #7773, feat: implement agent gateway runtime #7774, feat: expose agent gateway transports #7775, feat: govern agent access #7776, feat: add agent activity logging #7777, feat: trace agent gateway operations #7778, feat: add agent logs workspace #7779, feat: add agent catalog and push management #7780, feat: configure agent gateway settings #7781, feat: package agent gateway configuration #7782, fix(agent-logs): bound filter dropdowns by the DAC dimension ceiling #8088, fix: harden A2A push configuration and tracing #8125, feat: add opt-in private push callback delivery #8204, ui: a2a ui fixes #8336, feat: add session grouping and raw capture, refine log views #8364)query_logs,count_logs,get_log_detail,get_request_trace,query_metrics,query_usage_by,query_model_performance,render_chart,describe_filter_space,describe_virtual_key,describe_user_limits,ask_user). It asks multiple-choice clarifying questions, draws inline charts, links to matching rows on the Logs page, resolves "today", "yesterday" and bare dates in the asker's time zone, and filters by agent name andx-bf-agent-correlation-id. Admins set a default model plus up to 20 more models users can switch between,temperature,reasoning_effortand max iterations (2 to 20) at/api/warp/config; chat streams from/api/warp/chatwith history under/api/warp/conversations(100-conversation cap). Warp calls the gateway in-process, its calls are logged and governed under the app name Warp, signed-in users without an access profile are served, and it only reads rows the asker may see. Governance refusals return their own error codes (access_denied,budget_exceeded,rate_limited,model_blocked), and the chat panel is a resizable dock. Gated by thewarpfeature flag, on by default in OSS and off by default in Enterprise (Adds Odin config storage and settings API #6212, Adds Odin settings page #6213, Adds Odin read-only query tools #6214, Adds Odin agent endpoint #6215, Adds Odin topbar button and fixed-width dock #6216, Adds Odin chat surface #6217, Gives every panel the same card border, and polishes Odin's chrome #6218, Adds Odin chat history #6219, Scopes Odin's queries to whose traffic the question is about #6220, Odin asks before it answers, with pickable options #6221, minor warp fixes #6882, feat(warp): query tool gaps #7068, feat(warp): utc offset #7070, feat(warp): prompt and settings fixes #7086, feat(warp):parallel tool calls #7151, feat(warp):context overflow fix #7183, feat(warp): add warp behind feature flag #7418, warp-fixes #7511, feat(warp): call the gateway in-process and remove base_url #7562, fix(warp): query scoping and logs links #7667, fix(warp): links that cannot open are no longer clickable #7736, fix(warp): nudges oown traffic hint and whose keys a ranking covers #7746, feat(warp): read a key and a person the way the dashboard does #7750, feat(warp): an unrestricted caller's questions cover the whole deployment #7752, feat(warp): themes related questions sample logs instead of using semantic search #7756, feat(warp): feature flag on by default #7757, feat(warp): make dock panel resizable #7765, feat(warp): allow admins to set up multiple chat models #7814, feat(warp): let deployments report row-level caller restriction #7972, fix(warp): resolve calendar days in asker timezone #7978, fix(warp): explain refused model access #7995, feat(warp): serve users without access profile #8049, feat(warp):improved error codes #8176, fix(warp): reopened failed turns keep their error code #8184, fix(warp): say its own queries carry the asker's attribution, and never claim configuration #8186, fix(ui): move warp dock bottom spacing from inner container to resizable panel #8360, feat(ui): add motion and smooth streaming animations to Warp chat #8380, feat: add agent name and correlation id filters to warp logs tools #8388, fix(warp ui): max iterations ui #8401, docs(warp): add warp docs #8416)semantic_search_logsfinds conversations by meaning. The embedding provider, model, dimension and namespace are set in Warp settings; new logs are indexed live, and older logs by a resumable, concurrent background backfill (/api/warp/log-index/backfill,/backfill/status,/backfill/cancel,/api/warp/log-index/status). Restricted callers get results prefiltered to rows they can see, the similarity threshold is one 0 to 1 scale on every vector store (default 0.5, Weaviate included), and a finished backfill is hidden once the embedding space changes (feat(warp): persist log embedding configuration #6847, feat(warp): index completed logs in vector store #6848, feat(warp): backfill log embeddings with sidekiq #6849, feat(warp): expose embedding backfill APIs #6850, feat(warp): add semantic log search #6851, feat(warp): configure embeddings and backfill in UI #6852, feat(warp): fix broken urls and indexing optimizations #7186, dix(warp): hide a finished backfill once the embedding space changes #7740, feat(warp): update warp indexer to the new embedding input shape #7805, feat(warp): prefilter semantic search by caller visibility #7973, fix(warp): one similarity scale for semantic search, 0.5 default, multi-phrasing fan-out #8180, fix(warp): treat Weaviate scores as raw cosine in semantic search like every other vector store #8339)/v1/live/sessions, with a sideband/v1/live/sessions/{session_id}/attachWebSocket and recording downloads at/v1/live/sessions/{session_id}/content(logged aslive_content). Voice time is billed in 30-second windows frominput_cost_per_secondon alivepricing row and backend Responses calls are billed per turn, with governance checked at each window and one session log row carrying transcript, delegations and costs. Live has its ownliveallowed request type (realtimedoes not imply it). Sessions are bound to the virtual key that opened them, keys are picked only from those governance allows, later billing units on an excluded key return403, and sideband attachments run through the plugin pipeline aslive_sidebandwith their own log row. Failed backend delegations are recorded on the delegation instead of failing the session, an upstream that ends withoutsession.closedgives the client502 server_error, a failed WebRTC setup still records the 15-second minimum,x-bf-direct-keyworks on WebSocket upgrades, and?provider=orx-model-providerpicks the provider on attach and content routes (feat: gpt live core structs #7721, fix: cost changes for gpt-live and key selection #7722, fix: gpt live sessions handling #7723, feat: gpt live webrtc handling #7724, feat: attach and sip routes #7725, feat: content route gpt live #7726, feat: logging for gpt-live api #7727, tests: live api #7728, feat: live content request #7999, fix: cr fixes for live handling #8011, fix: gpt live tests and direct key #8133, fix: live in openai custom provider UI #8203, fix: governance key narrowing in live #8293, fix: failed delegation error #8294, fix: invalid request error in realtime #8295, fix: session close #8296, fix: key admission in unit #8297, fix: 15 seconds charge on webrtc initialization of live session #8298, fix: delegation error #8299, fix: alias resolution in live #8319, fix(live): bind a live session to the virtual key that opened it #8331, fix(live): billing via webrtc channel #8346, fix: ws close in live #8372, fix: live provider key header #8376, fix: sideband request logs #8381, fix(replicate): keep file list pagination on the configured file endpoint (#8290) #8405, fix: live sheet ui #8414)cloudflareprovider for chat completions (streaming included), Responses, embeddings and model listing against Workers AI. Every key needscloudflare_key_config.account_id(supportsenv.references); the base URL defaults tohttps://api.cloudflare.com/client/v4. Clef decision models (clef,clef-flash) are served natively on/v1/decisionswith image input, and the Cloudflare{result, success, errors}envelope is unwrapped. A migration dialog in the UI moves an existing custom provider namedcloudflareonto the new provider (feat(decisions): add Cloudflare Workers AI envelope unwrapping for Clef support #8007, feat(providers): adds cloudflare as a new provider #8342, feat(decisions): add native Clef decision support to the Cloudflare provider #8344, feat(openapi): add Cloudflare Workers AI provider and key config to docs #8345, feat(providers): add migration dialog for custom Cloudflare providers and dedupe Cloudflare model list #8351, docs(helm): add Cloudflare provider and expanded decision model providers to v2.1.44 changelog #8378)injected_tools, a list of{name, mcp_client_name, tool_name}entries. Bifrost adds each MCP tool to every chat and Responses request for that provider (past client and request tool filters), runs it server side when the model calls it (non-streaming, streaming chat, streaming Responses and the Anthropic passthrough path), and returns only the final answer with usage summed across turns. An entry namedweb_searchreplaces any native web search the client sends; any other name replaces a client tool of the same name. Configure it in the provider's new Tools tab, the provider API or config.json. Depth is capped bymcp_agent_depth(feat: add injected_tools provider config field #8021, feat: accept injected_tools on the provider API and config schema #8022, feat: let provider-injected MCP tools past tool filters #8023, feat: rewrite requests to carry provider-injected tools #8024, refactor: split chat and responses dispatch out of the provider handlers #8025, feat: execute provider-injected tools server side for non-streaming requests #8026, feat: execute provider-injected tools in streaming chat #8027, feat: execute provider-injected tools in streaming responses #8028, feat: serve injected tools on the Anthropic passthrough path #8029, feat(ui): add a web search tab to provider configuration #8030, feat(injected-tools): inject any number of named MCP tools per provider #8366, feat(ui): replace the provider Web Search tab with a Tools table #8367)openaiand on custom providers with anopenaibase can setoauth_key_configinstead of a staticvalue. Bifrost mints a bearer from the token endpoint with theclient_credentialsgrant (HTTP Basic orauth_style: "body") or thejwt_bearergrant (RFC 7523, RSA/EC signed assertions), caches it per credential, refreshes it a minute before expiry and backs off on rejected credentials. A non-emptyvaluealways wins. Available in the API, config.json, Helm and a new "OAuth (minted token)" tab in the key form. Databricks OAuth M2M and GitHub Copilot now mint through the same shared token cache, so a rejected Databricks service principal is backed off instead of hitting the token endpoint on every request (feat(core): shared token cache for OAuth-minted provider credentials #8254, refactor(githubcopilot): mint through the shared token cache #8255, feat(core): oauth_key_config mints bearers for OpenAI-compatible providers #8256, feat(transport): persist, validate and expose oauth_key_config #8257, feat(ui): OAuth-minted keys for OpenAI-compatible providers #8258, docs: oauth_key_config for custom OpenAI-compatible providers #8259)POST /v1/decisionsnow takesinput(text, messages with image parts, or structured data) and an orderedquestionslist, each withtype(predicate,choice,score), optionalname,instructions,criteria,choicesorlevels, plussafety_identifier. Answers come back as a list in question order, with Laya answer fields, responseroutingand state/truncation usage fields passed through. New/openai/v1/decisionsand/openai/decisionsroutes (also under/langchain,/litellm,/pydanticai) accept the OpenAI SDK's native request; an OpenAI-based provider's body is relayed unchanged and others are rebuilt into OpenAI's response shape (feat(decisions): add Laya-specific typed fields to decision answers, response and usage #7982, feat(decisions)porting v1/decisions to normalised openai shape #8181, feat(decisions): add OpenAI-format /openai/v1/decisions route #8192)/v1/decisionsnow reaches native decision endpoints on OpenAI (gpt-6-luna), Perplexity (pplx-decider-*), Ollama (local System One modelsnimble,tev1,clef,clef-flash,layaon/v1/systemone, withkeep_aliveand base64 images) and OpenRouter (TypeSafe Jev models and any model the datasheet markssupports_decisions, listed live with response headers forwarded). These providers returnedunsupported_operationbefore ([feat]: OpenRouter provider - native decisions for TypeSafe jev models #7589, feat(decisions): add OpenAI native /v1/decisions support for gpt-6-luna #8191, feat(openrouter): route decisions by datasheet supports_decisions row #8301, feat(decisions): add native Perplexity decider model support #8354, feat(openrouter): send decision images as plain state items, list live decision models, and forward response headers #8355, feat(decisions): add native System One decision support to the Ollama provider #8363, docs(decisions): document Cloudflare, Perplexity, Ollama and self-hosted decision models and rework decision emulation docs #8397) (thanks @hcavarsan!)unsupported_operationwith a message naming the model andconvert_decision_to_responses. Enable emulation with the new compat settingconvert_decision_to_responses(UI: Compatibility, "Emulate Decisions API on non-decision models") or per request withx-bf-compat: ["convert_decision_to_responses"](feat(compat): make decision-to-responses emulation opt-in via convert_decision_to_responses #8288, harness and docs changes for emulation -> responses in compact #8289)base_provider_type: "typesafe", so a second TypeSafe-compatible endpoint (for example self-hosted Laya or Nimble) runs with its own keys, name,allowed_requestsandrequest_path_overridesfordecisionsandlist_models, with keyless listing. The dashboard offers TypeSafe as a base format and adds a Decisions toggle to allowed requests (core: allow typesafe as a custom provider base type #7948, ui: offer typesafe as a custom provider base format #7949, docs: document typesafe as a custom provider base type #7950)input_cost_per_token_decisionsandoutput_cost_per_token_decisionsprice decisions served natively by chat-row models such asgpt-6-luna; rows without them fall back to the normal token rates. Decision requests on variable-price models no longer bill a negative cost (feat(pricing): add decision-specific rates for chat-row models and skip negative variable-price rates #8249)gpt-6-lunaalso work), as the primary classifier or as the semantic classifier's fallback. Only user turns are sent (previous_message_count,timeout), per-tier guidance (definition, signals, examples) can be edited with shipped defaults returned asjev_defaultson the status endpoint, and decision calls are billed under the routing cost with aRequestType.GET /api/models?decisions=truelists only models that serve decisions natively,decisionsis a request type in pricing overrides and routing CEL rules, decision logs show their questions, and the LLM fallback classifier no longer sendstemperature: 0. The dashboard has a stepped classifier setup and settings sheets (feat(complexity): add Typesafe Jev classifier as primary and semantic fallback #7485, feat(complexity): add Jev decision cost tracking, RequestType field, and stepped classifier UI #7605, [docs] : docs for jev router #7647, feat(complexity): add per-tier Jev criteria overrides with UI guidance editors and jev_defaults status endpoint field #7690, docs: updated docs for jev router #7749, feat(complexity): add decision-model classifier with per-tier guidance criteria and settings sheet #8059, feat(complexity): add Laya, Nimble, and Clef provider icons and unblock Next without a decision provider #8080, feat(complexity): support OpenAI decision models like gpt-6-luna in the complexity router #8193, feat(complexity): list datasheet-marked decision models for OpenRouter in the complexity router #8303, fix(complexity-router): ignore model-listing failures for decision providers and open the model selector with the full list #8385)framework/queuepackage: a durable queue with consumer groups, at-least-once delivery, per-key ordering, retries, dead-lettering and lease heartbeats. Backends: in-memory, and alogstoretype that keeps queue tables in the logs database on SQLite (single process), Postgres (multi-instance,SKIP LOCKEDclaims,LISTEN/NOTIFYwakeups) and ClickHouse. Available to framework consumers; the gateway does not use it yet (feat(framework/queue): add queue interfaces, options, store SPI and registry #8217, feat(framework/queue): add generic store engine and in-memory store #8218, feat(framework/queue): add SQL store and the logstore queue type for SQLite #8219, feat(framework/queue): add the Postgres dialect for the logstore queue #8220, feat(framework/queue): add the ClickHouse store for the logstore queue #8221, feat(framework/queue): wake consumers across instances with Postgres LISTEN/NOTIFY #8222)/v1/embeddingsinput now accepts content parts (text,image,audio,video,file,tokens) and per-itemparamsoverrides, served by Cohere, Gemini (including:batchEmbedContents) and Vertex (includinggs://URIs,multimodalembedding@001video segments andembedContentConfig.documentOcr/audioTrackExtraction). Bedrock gains Nova multimodal (amazon.nova-2-multimodal-embeddings-v1:0) and Titan multimodal (amazon.titan-embed-image-v1) embeddings,encoding_formatmaps to Cohereembedding_types, and Parasail serves embeddings. OpenAI-compatible providers get onlyencoding_formatanddimensions(Mistral no longer returns422ontask_type), and the semantic cache hashestask_type,titleandauto_truncate. Embedding inputs are stored in a newembedding_inputlog column and shown in the log view (feat: add multimodal embedding support for cohere, gemini, and vertex #2559, docs: add multimodal embeddings guide and update embedding input schema examples #3367, feat: addembedding_inputcolumn and ui display for embedding request inputs #7001, feat: add embedding support for parasail provider #7096, feat: add titan multimodal embedding support and mapencoding_formatto cohereembedding_types#7459, fix: allow gs:// uri for gemini and vertex embeddings #7549, feat: add nova multimodal embedding support for bedrock #7550, feat: update the embedding span attributes to support the new shape #7804, fix: move batch embeddings into single embeddings when params are same #8140, fix: send image-only bedrock cohere embeddings as images[] with input_type image #8143, fix: embedding params handling for openai, cohere, and semanticcache #8146, fix: nest documentOcr and audioTrackExtraction under embedContentConfig for gemini and vertex embeddings #8199)shelltool withshell_call/shell_call_outputitems (streaming included) and theapply_patchtool withapply_patch_call/apply_patch_call_outputitems and diff stream events.allowed_callersis translated between OpenAI'sprogrammaticand Anthropic's versionedcode_execution_*callers on chat and Responses. GeminiexecutableCode/codeExecutionResultparts map tocode_interpreter_callitems, andcode_interpreterreaches Gemini ascodeExecution. Shell, code interpreter, image generation and computer tools are dropped for Bedrock and Bedrock Mantle, and the semantic cache treatsshell_callandapply_patch_callas tool calls (feat: addshelltool andshell_call/shell_call_outputmessage types with streaming support #7292, feat: translateallowed_callersbetween anthropic and openai on chat and responses paths #7293, feat: addapply_patchtool andapply_patch_callmessage type support across providers #7301, fix drop shell and code-interpreter tools on Bedrock providers in responses filter #7668, feat: add gemini code execution support for responses api (streaming and non-streaming) #7711, fix: treat shell and apply_patch calls as tool calls in semanticcache #8093)GET /v1/models/{model}returns one model's metadata, implemented for OpenAI, Anthropic (with alias resolution), Cohere, Gemini, Mistral and Ollama, withshutdown_datewhere the provider announces one. Gated by themodel_retrieveallowed request and exempt from budgets and rate limits (feat: retrieve model method #7419, feat: model retrieve for other providers #7716)/v1/realtime/client_secrets. Browser clients can authenticate with thexai-client-secret.<token>subprotocol as well asopenai-insecure-api-key.<token>; an empty, whitespace-containing or conflicting subprotocol credential returns400before the upgrade, and mint requests that carry session settings are rejected (send them withsession.update). Responses streaming keeps the provider stop reason from the newest chunk that has one, and the logging plugin marksrefusal,content_filterandsafetystops as errors of typemodel_refusal(go mod patches #8200)/bedrock_passthroughroute forwards the three Bedrock operations Bifrost has no native route for: InvokeAgent (streamed as AWS sends it), knowledge-base Retrieve, and ApplyGuardrail. Any other path, method or malformed identifier returns400. InvokeAgent and Retrieve are SigV4-signed with the key's AWS credentials (an API-key-only key is refused with400), and AWS status, body andX-Amzn-*headers come back unchanged. Azureprompt_filter_resultsand per-choicecontent_filter_resultsare now passed through on chat responses (adds all missing harness tests #7907)vertex_key_config.aws_workload_identity(audience,service_account_email,token_lifetime_seconds,aws_region,aws_role_arn) to authenticate from EKS IRSA, Pod Identity, ECS task roles or EC2 instance profiles with no stored GCP or AWS key. gcloud-generated AWSexternal_accountJSON now also resolves AWS credentials through the AWS SDK chain. New "Workload Identity (AWS)" tab in the key form (feat(core): vertex aws workload identity federation #7705, feat(framework,transports): persist and validate vertex aws workload identity #7706, feat(ui): vertex workload identity (aws) auth tab #7707, docs: aws workload identity federation for vertex #7708, feat(core): route aws external_account json through the sdk credential chain #7709)usage.tool_usage.web_search.num_requests(top-leveltool_usageon Responses) for Anthropic, Gemini, Perplexity and OpenAI Responses, and billed with the new pricing fieldweb_search_cost_per_request. Existing values are copied over by migration, and datasheet rows still carryingsearch_context_cost_per_queryare read as the new field (feat: addToolUsage.WebSearchfield and deprecateNumSearchQueriesin token detail #7624, feat: replacesearch_context_cost_per_querywithweb_search_cost_per_requestpriced fromtool_usage.web_search.num_requests#7687)ignore_provider_costsetting (UI: provider Pricing tab, defaultfalse) discards theusage.costa provider reports, for providers such as Cortecs that report credits, and prices requests from the model catalog and pricing overrides for logs, budgets, telemetry and streamed totals. The client response keeps the provider's figure (feat(configstore): add ignore_provider_cost provider config #8052, feat(modelcatalog): price from catalog when ignore_provider_cost is on #8053, feat(logging): store bifrost cost breakdown for providers that ignore reported cost #8054, feat(ui): add pricing tab with ignore provider cost toggle #8056, docs(providers): documentignore_provider_costoption #8057, fix(tracing): honor ignore_provider_cost and request pricing scopes on streamed cost #8165)force_reasoning_only_models_to_responsessends chat completion requests with tools to the Responses API when the datasheet marks the model as not supportingreasoning_with_tool_calls, and converts the answer back to chat format, so reasoning is kept instead of turned off. Defaults totrue, also available per request throughx-bf-compat(feat: addforce_reasoning_only_models_to_responsescompat toggle to route chat+tools requests to Responses API for models withoutreasoning_with_tool_callssupport #7806, chore: update compat plugin docs forforce_reasoning_only_models_to_responses#7807)reasoning.mode(standardorpro) on OpenAI or Azure is served through the Responses API soprois honoured; other providers report it as a dropped compat param. Chat requests can also setreasoning.summary(auto,concise,detailed), forwarded when chat is served through the Responses API (adds missing tool level transalation for anthropic -> openai caching (#7885) #7955, fix: preserve reasoning.summary in chat-to-responses conversion #8214)ServerSideFallbackModelfor pricing and routing info, and a datasheetserver_side_modelcovers the Anthropic-compatible endpoint, which echoes the requested ID (fix: stampServerSideFallbackModelwhen deepseek serves a different model than requested #7811, fix: resolve datasheet based server side model for deepseek #7822)failure_classon the attempt trail (credential,quota,model_access,model_gone,region_blocked,rate_limit,transient,caller_fault,timeout,cancelled,unknown), read from provider error codes rather than status alone. Credential and quota failures hold a key out for every model; keys refused for the model, a retired model or a region block are rotated past, and permanent per-key failures walk the key pool even atmax_retries: 0. Attempts Bifrost ends itself are recorded astimeout(504) orcancelled(499). Provider retry hints (Retry-After,retry-after-ms, GeminiRetryInfo) are exposed asextra_fields.retry_after_ms, clamped to 1s to 5m (feat: addCoversAllModelstoFailureClassand pin classifier outputs for key-wide failures #7684, fix: record timeout and cancellation classes on the attempt trail for attempts Bifrost ends #8231, e90eaf4, 00ab24c)ttft_timeout_ms(1 to 300000). A streaming attempt that does not produce its first token in time is cut off and handed to the next fallback without cancelling the request; the last attempt is never cut off, and non-streaming requests are unaffected. A cut-off attempt fails with codettft_timeoutand status504on the attempt trail. The rule editor has a "Time to first token cutoff (ms)" field per target, also in Helm (feat(core): per-attempt first-token (TTFT) abort for stream attempts #7503, feat(core): enforce TTFT deadline per stream attempt with fallback handover #7504, feat(routing): ttft_timeout_ms on routing rules #7505, feat(ui): TTFT cutoff field on routing rules #7506, docs(routing): document routing rule TTFT cutoff #7507, fixes single target ttf field ui capture #7996)routing_info.requested_provider/requested_modeland in thex-bifrost-routing-info-requested-providerandx-bifrost-routing-info-requested-modelresponse headers (feat: addrequested_provider/requested_modeltoRoutingInfoand response headers so the caller's pre-routing route is visible after a routing rule rewrites it #7683)x-bifrost-request-id, the ID the request is logged under (GET /api/logs/{id}), next tox-bifrost-trace-id. Provider response headers whose names start withx-bifrost-are no longer forwarded to the caller (feat: addx-bifrost-request-idresponse header and stop forwarding providerx-bifrost-*headers #8101)GET /api/modelswithout aquerynow leaves out deprecated models and their counts; with aquerythey are kept but sorted last. Passinclude_deprecated=trueto get them all (feat(models): hide deprecated models from unsearched listings, sink them below live ones in search results #7257)provider_objectstable. Retrieve, content, delete, cancel, results and remix on another key's object return404, list pages are narrowed to the caller's own objects, and indirect references (a batch on another key's input file, a container withfile_ids, a request namingcached_content) are checked the same way under every id form the provider accepts. A failed ownership lookup returns500instead of an unchecked page, and id lists over 1000 no longer fail large list pages. A new Restrict Provider Objects to the Creating Virtual Key switch under Settings > Security (client.enforce_vk_object_ownership, defaulttrue, applied without a restart) turns the restriction off; creators are still recorded while it is off (governance: bind batch lookups to the creating virtual key #7892, fix(governance): bind provider objects to the virtual key that created them #8292, fix(governance): bind provider objects under every id form the provider accepts #8317, adds toggle for virtul key level objct ownership #8427, fix(configstore): bind large provider-object id lists as one parameter #8435)input_file_idorinput_blobis now judged by what the provider actually runs. On OpenAI and Azure (and custom providers built on them), which run the models named inside the file, the virtual key must allow every model of the provider. On Gemini, Vertex and Bedrock the requestmodelmust be present and allowed. A batch create that sends a file or blob together with inlinerequestsreturns400(governance: require a model for file-based batches under a model-restricted key #7891, fix(governance): classify file batches by what the provider executes #8318)/openai_passthrough,/anthropic_passthrough,/azure_passthrough,/genai_passthroughand siblings) now forward onlyPOSTrequests to a fixed allowlist of inference paths per provider (chat, completions, responses, embeddings, moderations, audio, images,generateContent,countTokens,predictand similar). Paths that could change the upstream host (not rooted,//, userinfo@, schemes, backslashes,.., control characters, percent-encoded forms included) are refused, the final URL must stay on the provider's host, andx-model-providercan only confirm the route's provider. The model is read strictly from the JSON body, and governance checks every model a body names, including each task of a Runware task array (fix(core,transports): validate passthrough paths and provider header #7855, integrations: read the passthrough model strictly from the forwarded body #7890, fix: constrain passthrough to inference routes #7959, fix(governance): check every model a passthrough body names #8321)config_store.enabled: false, browser requests to/api/*(and/wsand/metricswithout a config store) from an origin that is not localhost, the gateway's own origin, or listed inclient.allowed_origins(patterns such ashttps://*.example.comwork, a bare*does not count) now return403. Requests without anOriginheader (CLI, SDKs), whitelisted routes and inference routes are unchanged, and enabling dashboard auth lifts the auth-off check (auth: refuse unlisted browser origins on the management API while auth is disabled #7893, fix(http): apply the management API cross-origin rule without a config store #8300)403when they: useenv./vault.references in MCP client credential fields (headers, connection string, OAuth, token exchange, TLS) onPOST /api/mcp/clientorPUT /api/mcp/client/{id}; create, re-point or redeliver to a webhook endpoint withallow_private_network; set an outbound endpoint (collector_url,metrics_endpoint,push_gateway_url,endpoint,url,base_url) or anyenv./vault.reference on a built-in plugin (OTEL, telemetry); set, change or remove a Vertex key'svertex_key_config.auth_credentials(the JSON names the hosts the Google client dials); or delete MCP tool logs viaDELETE /api/mcp-logs. config.json provisioning is not affected (webhooks: require an admin session for private-network endpoints #7886, mcp: require an admin session for secret references in client fields #7887, [fix]: Require authenticated management access to delete MCP tool logs #7913, fix(providers): treat the Vertex credential JSON as a key dial target #8330, fix(plugins): gate built-in plugin endpoints and secret references behind real auth #8335) (thanks @xujiantop-crypto!)PUT /api/confignow requirescurrent_password(the stored admin password) or a validsetup_tokenin the request. The Security page asks for the current password in that case (fix(transports): verify credentials on auth config update #7854)client.max_request_body_size_mbceiling), and a process-wide in-flight budget of four times the body cap bounds buffered decompression. The budget is charged for bytes actually inflated, so ordinary concurrent gzip traffic is not refused (fix(transports): bound request decompression by what the caller sent #8333, fix(transports): charge the decompression budget for bytes actually inflated #8431)business_unit_id), and a business unit holds at most one budget. config.json acceptsaccess_profileongovernance.teams,governance.customersandgovernance.business_units, roles accept the newbusiness-unit-dataandcustomer-datadata access scopes, and the dashboard hides a key's own limit editors when a profile governs it (feat(grant): access profiles can be held by teams, business units and customers #7251, feat(governance): a virtual key can be owned by a business unit #7253, feat(ui): the enterprise customer sheet no longer edits legacy limits #7254, feat(ui): assign a virtual key to a business unit, and show when a profile governs it #7255, feat(governance): refuse an entity's own budgets when an external governs it #7386, fix(governance): leave a governed entity's stored limits alone when config.json still declares them #7401, feat: addLegacyLimitHolderBusinessUnitconstant for business unit governance #7421, feat: hide "Add Budget" button whenmaxLineslimit is reached for Business units #7432, fix: persistbusiness_unit_idwhen updating a virtual key #7616, fix: resolve business unit name in virtual keys table assigned-to label #7619, fixes access profile summary UI #7642, feat(governance): addproxy_config, access profile on entities, andbusiness-unit-data/customer-dataDAC scopes to config.json docs and schema #7792, fix(config): skip a governed team's or customer's limits at startup, not only after it #7794, feat(config): teams and customers carry access_profile, and LoadConfig reports what it wrote #7795, docs(dac): customer-data grants only what sits under the caller's customers #8086, fix: skip config.json limits on virtual keys governed by an access profile #8332, docs: data access control docs and add teams, business units & customers guide #8375, docs: access profile docs update for teams customers and bu #8379)disable_content_logging: omitted ornullinheritsclient.disable_content_logging,truekeeps that key's request and response content out of the log store and strips it from OpenTelemetry export (MCP arguments included),falsekeeps content in the log store even when the client setting turns it off. The per-requestx-bf-disable-content-loggingheader still wins. The virtual key editor has an Inherit / Off / On select, and the CLI has--disable-content-logging(feat(governance): add disable_content_logging column to virtual keys #7476, feat(governance): stamp the virtual key's content-logging decision on the request context #7477, feat(logging): honor the virtual key's content-logging decision in the log store #7478, feat(otel): strip exported content for virtual keys that turned content logging off #7479, feat(governance): expose disable_content_logging on the virtual key API #7480, feat(ui): content logging control on the virtual key editor, with e2e and docs #7481)delete_after_expire(requiresexpires_at,400otherwise). A daily job deletes flagged keys once they expire and posts one notification with the counts. The expiry picker has a matching option (feat(governance): auto-delete expired virtual keys with a daily sidekiq job #7525, feat(ui): delete-after-expire option on the virtual key expiry picker #7526, docs(governance): document delete_after_expire for virtual keys #7527, refactor(virtual-keys): simplify expiry cleanup notifications to counts only #7828)proxy_configin config.json (enabled,url,username,password,no_proxy,timeout,skip_tls_verify,enable_for_scim,enable_for_inference,enable_for_api) and in Helm. Withenable_for_apion, MCP connections, MCP OAuth discovery, model catalog sync, OpenTelemetry export, webhooks, skill fetches and plugin downloads go through it, each keeping its own SSRF policy. Withenable_for_inferenceon, providers without their ownproxy_confignow actually send inference through it (the flag was stored but never read before), and multi-entryno_proxylists match on both fasthttp and net/http paths (fix(transports): apply the global proxy to providers without their own #7568, feat: build the HTTP client factory in OSS and route webhooks, skills and plugin downloads through it #7574, feat: route MCP, OAuth, catalog sync and OpenTelemetry through the global API proxy #7575, feat(config): the global proxy, with its SCIM, inference and API toggles, in config.json #7799)bifrost_alb_prefix (caller headers cannot spoof them) and hidden from the log filter sidebar and metadata grid. Docs cover held keys, incidents and the/api/adaptive-routingendpoints (feat: addBifrostContextKeyLoadBalancerAttemptandLoadBalancerMetadataPrefixto propagate enterprise load balancer routing decisions into log metadata, with system-key filtering andcluster_node_idcomposite index #7456, docs: add Held Keys section to adaptive load balancing, replace Mermaid diagrams with images #7531, fix: remove wrapper padding from adaptive routing page and addLoadBalancerHistorycache tag #7645, feat: add downstream-extensible logs registry for routing panel, labelled metadata filters, and reserved key prefixes; hide all load-balancer metadata keys from filter sidebar and detail grid #7734, feat(adaptive-routing): add Incidents page and sidebar nav entry #7974, docs: update adaptive load balancing endpoints to /api/adaptive-routing and clarify release response semantics #8233, docs(adaptive-routing): document the Incidents page #8272, docs(adaptive-routing): document the incident list on the issues endpoint #8274)BusinessUnitspermission; newBackgroundJobsRBAC resource; Warp follows RBAC; create, edit, delete and rotate controls follow the viewer's permissions with a tooltip explaining why they are disabled; the customers list shows a server-side team count (feat(ui): dac updates for permissions on ui #7404, feat(ui): addDisabledReasontooltip wrapper to governance action buttons #7675, feat(ui): org chart route, governance sidebar entry and fallback #7954, feat(customers): add server-side team count to customer list instead of fetching all teams #7998, fix(ui): business units sidebar entry checks the BusinessUnits permission #8046, feat: adds BackgroundJobs rbac permissions #8132, f6f43df, 18b4b3f)attributeProjectMappings(add users to existing projects when an IdP attribute matches a value or glob; memberships granted by a rule are revoked when it stops matching) andbulkSyncInterval(directory reconcile interval, default24h, from1hto30d), in config.json and Helm. The generic SSO provider accepts a logindisplayName, and a new identity sync guide aligns the IdP setup docs (chore: addattributeProjectMappingsandbulkSyncIntervalto config schema #7803, feat(schema): generic SSO provider accepts a login displayName #8045, docs: identity sync guide and align IdP setup docs #8373)audit_logs.omit_ip_addresses(HelmomitIpAddresses, defaultfalse) stops recording client IPs in audit logs. New events store norequest_ipor initiator host, older IPs are hidden from APIs, exports and archives (not deleted), and IP search and filters are disabled. HMAC signatures stay valid (feat(audit-logs): addomit_ip_addressesoption to suppress client IP recording #7671, docs(audit-logs): document the new event type URI #8269)BifrostContextKeyGovernanceVirtualKeyID,...TeamID,...CustomerID,...BusinessUnitID,...ProjectID,...BudgetIDs,...RateLimitIDs,BifrostContextKeyUserID,...UserName,...UserEmailand related variants) are deprecated in favour of thectx.Grant()accessors. Values and behavior are unchanged (refactor: deprecate attribution context keys in favour ofctx.Grant()accessors #7975)require_public_targetflag, and every later dial (reconnects, per-call connections, proxied hops) is held to public addresses, not only the registration-time DNS check. Existing clients default tofalse(fix(core,transports): validate mcp client targets at connect time #7849)client.oauth2_server_config.allowed_redirect_uris(loopback andcursor://anysphere.cursor-mcpare always allowed), re-checks existing registrations at authorization and token exchange, and shows the exact callback on the consent page. Oversize DCR/authorize fields (client_name,scope,redirect_uris,state,code_challenge) return400before anything is stored (fix(transports,framework): tighten mcp oauth2 server validation #7851, fix: validate OAuth callback registration and consent #7960)client.mcp_oauth_private_allowlist(ormcp.tool_manager_config.oauth_private_allowlist, which wins) and the MCP OAuth chain (discovery, dynamic client registration, token exchange, refresh) may reach them for clients registered by an authenticated admin or declared in config.json. Link-local and cloud metadata addresses stay blocked, invalid entries stop boot, and edits apply to the next connection (adds allow localhost oauth servers for mcp #8347)instructionsan upstream MCP server returns from itsinitializehandshake are now captured, persisted with discovered tools and refreshed with them. Bifrost forwards them on its own/mcpgateway and injects them into MCP-bearing chat (as a leading system message) and Responses (appended toinstructions) requests as tagged<mcp_server name="...">blocks, scoped to the servers the request can see. Caps aremcp.tool_manager_config.max_instructions_per_client(default 4096 bytes),max_instructions_total(default 16384 bytes) and a per-clientmax_instructions_lengthoverride. In code mode the instructions appear in the.pyistubs (feat: capture and persist upstream mcp server instructions #7516, feat: forward server instructions on the MCP gateway #7517, feat: inject mcp server instructions into LLM requests #7518, fix: include server instructions in on-demand MCP tool refresh #7519, feat: add per-mcp server override for max instructions length #7766, fix: carry instruction fields through the scoped MCP client refetch #7818, feat: update server instructions forwarding when code mode is on #8371)POST /api/mcp/client/{id}/refresh-tools(and a Refresh tools action in the client table) re-discovers a client's tools immediately, for every client type including per-user and per-call clients. Tool calls that fail because the server dropped its session ("session terminated", "session not found", "need to re-initialize") trigger an immediate reconnect and, where safe, a retry; a failed live-connection check now triggers the documented reconnect, and a failed make-before-break re-dial keeps the live connection. Health probes retry timeouts but treat dead sessions and auth rejections as permanent.POST /api/mcp/client/{id}/reregister(dashboard: "Reauthorize with a new client") registers a new dynamic OAuth client and runs consent against it when the upstream forgot the oldclient_id(fix: preserve live connection on failed make-before-break re-dial instead of tearing it down #6921, fix: trigger reconnect after failed live-connection check instead of re-probing the same dead session #6922, feat: extend dead-session detection and recovery toexecuteToolInternal, renamingattemptAuthFailureRecoverytoattemptCallFailureRecovery#6923, fix: retry probe timeouts but treat dead sessions and auth failures as permanent inProbeRetryConfig#6924, feat: addRefreshMCPClientToolsfor on-demand tool rediscovery across all client types #6927, feat: add "Refresh tools" action to MCP client table for immediate tool re-discovery #6928, docs: addrefresh-toolsendpoint and Manual Tool Refresh section across MCP connection docs #6929, docs: overhaul health monitoring docs to reflect adaptive cadence, automatic reconnect recovery, and simplified config knob #6931, feat: addReregisterDynamicClientandPOST /api/mcp/client/{id}/reregisterto recover from evicted DCR registrations #7228, feat: add "Reauthorize with a new client" action to re-register OAuth client when provider rejects the existingclient_id#7229, docs: addreregisterendpoint and "Reauthorizing with a new client" section for invalid/dropped DCR client recovery #7230)mcp.tool_manager_config.code_mode_limits(alsoclient.mcp_code_mode_limitsthrough the config API, the MCP settings page and Helm) setsmax_source_bytes(default 64 KiB),max_steps(1,000,000),max_memory_bytes(64 MiB),max_log_bytes(64 KiB),max_tool_calls(64),max_value_bytes(1 MiB, minimum 1024) andmax_nesting_depth(64, maximum 1000). Cyclic or oversized results are rejected before serialization (fix: bound code execution resources #7962)mcp_configsentry on a virtual key now shows its MCP client'sdisabledvalue, and the docs explain that a disabled client offers no tools even when the key lists it (feat(governance): add optionaldisabledfield to virtual key MCP config type #8130, docs(governance): document disabled MCP clients on virtual keys #8270)identity_scopecannot claim another identity's scope. The storedcache_keyis now the raw value the caller sent (thevk:/raw:vk:rewriting is gone), soDELETE /api/cache/clear-by-key/{cacheKey}clears every identity's entries under that key. Newdisable_identity_scopingoption (defaultfalse, UI switch "Scope Cache by Identity") shares entries across identities (feat(semanticcache): scope cache entries by identity, with adisable_identity_scopingopt-out #8337)tool_calls, Responsesfunction_call,custom_tool_call,computer_call,local_shell_call,mcp_call,mcp_approval_request) are no longer written to the semantic cache unlesscache_tool_call_responsesistrue. Server-side tool items such as web search are still cached (fix(plugins): scope semantic cache entries per virtual key #7852)x-bifrost-cache-*headers mirroringcache_debug:x-bifrost-cache-hit(always sent when a lookup ran), plus-id,-hit-type,-hit-latency-ms,-requested-provider,-requested-model,-similarity,-threshold,-provider-used,-model-usedand-input-tokenswhen known. This works on streams and on the Anthropic, GenAI, Bedrock and Cohere routes, whose bodies have noextra_fields(feat: emit x-bifrost-cache-* response headers for semantic cache lookups, including streams #8340)bifrost.error.type,error.type,gen_ai.error,gen_ai.error.codeand the HTTP status, including for requests rejected before dispatch, and streaming requests get an end-to-end time-to-first-token attribute that includes queueing, plugins and key selection (fix: export failure classification and end-to-end TTFT on traces #8404)export_raw_payloads: trueon an OTel profile (or BigQuery) to attachbifrost.raw_request/bifrost.raw_responseto LLM spans. It requiresstore_raw_request_responseon the provider, is suppressed bydisable_content_logging, and drops bodies over 256 KB. Failed requests export the raw request and the provider's error body. Exposed in the OTel profile form and Helm (feat: export raw payloads to connectors #7391, feat: add raw payloads export support #7753, chore: update helm charts for excluded attributes and raw payloads for connectors #7758, fix: export raw request and provider error body for failed requests #8227)excluded_attributeson OTel profiles (and the Datadog, Kafka, Pub/Sub, Splunk and BigQuery connectors) drops listed span attributes by exact name, such asgen_ai.request.tools, before export. BigQuery takes column names (for exampletools);trace_idandtimestampcannot be excluded. Unknown names are ignored (feat: per-connector attribute denylist #7754, docs: add exclude attributes and raw payloads in connectors #7759)bifrost.cost.*(input/output/additional sides with text, audio, image, cached read/write, request, reasoning, citation, search queries, guardrail, MCP, semantic cache and routing).gen_ai.usage.costis still emitted. Plugins can callctx.CalculateCostBreakdown(resp)for the same split (feat: export the cost breakdown to connectors #7371)User-Agentinto anappvalue (Claude Code,Cursor,Codex CLI,Cline,Windsurf,API,Other), exported as thebifrost.appspan attribute and anapplabel on request-level Prometheus metrics (feat: export app to connectors #7390)metric_dimensionslistsx-bf-dim-*request dimension names to add as DogStatsD metric tags. Dimensions already reached Datadog spans; metrics stay untagged unless listed, to bound cardinality (feat: add metric_dimensions config for x-bf-dim-* metric tags #7389)http.request.method(unknown methods become_OTHERwithhttp.request.method_original),http.response.status_code,http.route(the matched route template),url.path,url.schemeanduser_agent.original. The deprecatedhttp.method,http.url,http.user_agentandhttp.status_codeare still emitted (feat: update OTEL to semconv 1.44.0 #7810)HTTPTransportResponseHeadersHookruns once, in reverse registration order, just before response headers are sent, for buffered and streaming responses alike, so plugins can set headers on SSE streams.Content-Length,Transfer-EncodingandConnectioncannot be changed and the status code is read-only. Native.soplugins may export the symbol optionally (feat: add HTTPTransportResponseHeadersHook in plugins #7537)session_idunder the session's earliest root request, with expandable session rows, session totals and a redesigned expander (tree lines, turn and fallback labels).GET /api/logsgainsgroup_sessions=true(requiresroots_only=true; ignored whensession_id,parent_request_idorrequest_idis set), and collapsed rows carrysession_child_count,session_total_costandsession_total_tokens(feat(logs): collapse sessions to earliest chain root with per-session rollup aggregates #7298, feat(logs): addroots_onlyandgroup_sessionsquery params with aggregated child/session fields #7299, feat(logs): collapse sessions in grouped view with expandable session rows #7300, feat(logs): redesign grouped view expander with tree lines, turn/fallback labels, and kind markers #7970, docs(logs): document the grouped view #8273)GET /api/sidekiq/jobsandPOST /api/sidekiq/jobs/{id}/cancellist every pending, running or recently finished background job (cost recalculation, Warp log indexing) with progress, and stop one. The notification panel has a Background jobs tab with count badges, Cancel / Stop job and Clear finished. Access uses View and Update onBackgroundJobs(sidekiq jobs in notifications #7820, feat(ui): move background jobs into a separate tab in the notification panel #8211, docs(logs): document background jobs in the notification panel #8275)logs_store.object_storage_exclude_request_typeslists request types (the logobjectvalue, for examplelist_models,embedding,chat_completion_stream) whose logs stay in full in the logs database instead of being offloaded to S3/GCS. Streaming and non-streaming variants are separate types. Logs with hidden content are still cleared from the DB row and not uploaded (feat(logstore): add option to exclude request types from object storage offloading #8047, docs(logs): document object_storage_exclude_request_types for log exports #8048)*_above_100k_tokensrates (input, output, cache read, 5m and 1h cache write) once a prompt exceeds 100,000 tokens, as Claude Haiku 5.5 bills. Higher tiers (128k, 200k, 272k) still take precedence. The custom pricing override form has the new fields (feat: 100k tier pricing #8144)detections: phase, rule, guardrail, reason and sanitized categories, never the matched text) alongside judge calls. Docs add the A10 Guardrails integration, regexallowed_values, profile execution order, and the prompt guardrails judge model requirements (max_output_tokensdefault 1000) (feat(schemas): addBifrostGuardrailDetectionfor detect-only guardrail policy matches #7697, docs: add A10 Guardrails provider integration docs #7755, docs(guardrails): clarify prompt guardrails judge model requirements and update max_output_tokens default to 1000 #8002, docs(guardrails): document regex allowed_values and profile execution order #8410)ProviderSelectorand oneModelSelector, with icons, server-side search and paging, multi-select chips and custom-provider icon fallback. Provider lists in tables collapse into a+Nchip (feat(ui): add responsiveProviderListcomponent with overflow chip #7247, feat: model selector component #7258, ui: model selector usage #7259, chore: remove unused model multiselect #7272, feat(ui): addProviderSelectorcomponent #7408, feat(ui): replace ad-hoc provider selects with a unifiedProviderSelectorcomponent #7410, docs(agents): addProviderSelector/ModelSelectorusage guidelines #7427)mcp_enable_temp_token_authis off, visitors opening an OAuth consent or MCP per-user-headers link are sent to/loginand back instead of seeing "invalid link". A401now carrieserror.codetemp_token_auth_disabledortemp_token_rejectedwhen relevant (feat: add sign in option when temp tokens is disabled #7664)bifrostCLI covers the published API:api(generated catalog of OpenAPI operations with search and tags), resource commands (models,responses,videos,providers, ...),chat,infer,request/http,context,auth,usage,keys,configure/unconfigure(reversible native-agent setup with receipts and rollback) andcompletion. It authenticates to the gateway (SSO in the interactive launcher and authenticated harness launches, with its own browser auth identity). Output is--output table|json|yaml|raw, and destructive operations require--yes(feat(cli): add authenticated gateway transport #7279, feat(cli): add OpenAPI operations and structured output #7280, feat(cli): support authenticated harness launches #7281, feat(cli): add management and inference commands #7282, feat(cli): integrate SSO into the interactive launcher #7283, docs(cli): document authentication and management workflows #7284, fix(cli): separate CLI browser auth identity #7579)proxy_config, entityaccess_profile, thebusiness-unit-data/customer-datascopes, routingttft_timeout_ms,storage.configStore/storage.logsStorestatementTimeoutandidleInTransactionSessionTimeout,bifrost.client.compat.forceReasoningOnlyModelsToResponses,bifrost.client.deleteExpiredVirtualKeys,bifrost.client.oauth2ServerConfig.allowedRedirectUris, virtual keydelete_after_expire/disable_content_logging/business_unit_id,bifrost.mcp.clientConfigs[].requirePublicTarget, semantic cachecache_tool_call_responses, object-form routing rulefallbacks,bifrost.governance.complexityAnalyzerConfig.jev.criteriaandbifrost.mcp.toolManagerConfig.maxInstructionsPerClient/maxInstructionsTotal. NewtopologySpreadConstraintsspreads replicas across nodes and zones, andextraContainersadds sidecars (native sidecars go ininitContainerswithrestartPolicy: Always, refused on Kubernetes older than 1.29) (feat(helm): addproxy_config,access_profileon entities,business-unit-data/customer-dataDAC, andttft_timeout_ms#7791, chore(helm): sync chart with config.schema.json for v2.1.44 #8187, feat(helm): support topologySpreadConstraints on the Bifrost pod #8188, feat(helm): add extraContainers and guard native sidecars #8189, 0d25eca)⚙️ Upgrade Notes
These entries change a default, a status code, a response shape or a Go SDK type. The matching entries above describe each change in full.
client.a2a_external_client_urlfor push and stable card URLs, andserver.a2a_grpc_base_domain+server.a2a_grpc_port(plus wildcard DNS) for gRPC. Customschemas.Tracerimplementations must addReleaseTrace.feature_flags.flags.warp.enabled: falseto keep it hidden.cloudflareis reserved; migrate a custom provider namedcloudflarewith the dashboard dialog or recreate it withcloudflare_key_config.account_id./v1/decisionsbodies are deprecated (still served); update Go SDK code to the newBifrostDecisionRequestfields.unsupported_operation; setcompat.convert_decision_to_responses: trueor sendx-bf-compat: ["convert_decision_to_responses"]to keep emulation.BifrostEmbeddingRequest.Inputis now[]EmbeddingInputItem; HTTP bodies are unchanged.model_refusalerrors; exclude that error type from error-rate alerts if needed.search_context_cost_per_querytoweb_search_cost_per_requestin config.json overrides; read counts fromusage.tool_usage.web_search.num_requests.compat.force_reasoning_only_models_to_responses: falseto keep the old behavior./openaichat route stripsreasoning.summary; send it throughextra_paramsthere.404instead of502 upstream_credentials_exhausted; update alerts on that error and onfail_reasonvalues.x-bifrost-request-id; upstreamx-bifrost-*headers are no longer forwarded.include_deprecated=truetoGET /api/modelsto keep deprecated models.client.enforce_vk_object_ownership: falseif virtual keys share files, batches, containers or cached contents.*model grant for OpenAI/Azure file batches, send an allowedmodelfor Gemini/Vertex/Bedrock file batches, and never mix a file with inlinerequests.POSTinference paths with rooted paths, the route's own provider and one stringmodelkey.client.allowed_originsor enable dashboard auth.X-Bifrost-Setup-Tokenfor MCPenv./vault.references, private webhooks, plugin endpoints, Vertexauth_credentialsandDELETE /api/mcp-logs.auth_config.current_passwordorsetup_tokenwhen re-enabling auth or rotating admin credentials while auth is off.413; send them uncompressed or raiseclient.max_request_body_size_mb.enable_for_inference: falseto keep inference direct, and list direct hosts inproxy_config.no_proxy.typeURIvalues.ctx.Grant().http,sseorstdioasconnection_type.client.oauth2_server_config.allowed_redirect_urisbefore upgrading.max_instructions_per_client,max_instructions_totalor per-clientmax_instructions_length.mcp.tool_manager_config.code_mode_limitsfor scripts that exceed the new budgets.disable_identity_scoping: trueto share entries across identities.cache_tool_call_responses: trueto keep caching them.applabel to recording rules or alerts that match exact label sets.http.request.method,http.response.status_code,http.routeandurl.path; the span name andhttp.urldrop the query string.HTTPTransportPluginmust addHTTPTransportResponseHeadersHook(a no-op is fine).bifrost.client.oauth2ServerConfig.issuerUrlwhenmcpServerAuthModeisoauthorboth, orhelm templatefails.file://.base_url.?provider=orx-bf-model-provideron direct-key model listings.409as a request error instead of retrying on500.id: "other"likeunassigned.404, not500, for an unknown MCP client.prompt_cache_options.mode: "explicit"on the request to opt into explicit mode.400.502.aftercursor exactly as returned; others get400 invalid pagination cursor.api_error.status_code="unknown"and adderror_typeto OTel queries.tls_ca_certor keepinsecure: truefor self-signed collectors.bifrost_instance_idinstead ofservice_instance_id, and review telemetrycustom_labelsfor cardinality.statement_timeoutandidle_in_transaction_session_timeoutto"0"to keep server defaults.logs_store.config.max_query_sizeif a ClickHouse settings profile constrains it.thresholdto2 * old - 1to keep the old hit rate.🐞 Fixed
PUT /api/configno longer echo transport details (fix(framework,transports): use http client for outbound fetches #7848)x-bf-vkis now treated as a credential and redacted in captured request headers, so observability connectors no longer export it (fix(tracing): redact the virtual key header in captured request headers #8304)base_urlon create and update must be an origin with an optional path; URLs with credentials, a query (including a bare?) or a fragment are rejected before anything is stored or dialed (fix: validate provider endpoint URL components #7958)x-bf-direct-keyis no longer sent to every configured provider on an unscoped model listing, and drop-in routes (/openai/v1/models,/anthropic/v1/models,/genai/v1beta/models) list only their own provider. A refused direct key returns the provider's own error (for example OpenAI's401) instead of502 upstream_credentials_exhausted, and no session key binding is created when the direct key serves (fix: stop direct keys from fanning out to every provider on model listing #8106, fix: return provider's own error when a caller-supplied direct key is refused instead of collapsing into 502upstream_credentials_exhausted#8108, fix(core): bind no session key when the caller's direct key serves #8113)proxy_configinstead of connecting directly.https://proxies are reached over TLS (system roots,ca_cert_pem, orskip_tls_verify), proxies are dialed dual-stack so IPv6 proxies work, proxy handshakes are bounded at 30s, runtime proxy changes reach already-created clients, andDisablePathNormalizingis honoured through the proxy-aware client (fix(core): route every provider egress stack through proxy_config #7567, fix(core): dial proxies dual-stack so IPv6 proxies work #7569, refactor(core): run provider auth and URL fetch on fasthttp #7570, feat(core): live proxy clients in the HTTP client factory #7573, feat(core): reach https:// proxies over TLS on every provider stack #7576, fix(network): honor DisablePathNormalizing in live fasthttp client #8164)max_limit, added, removed or moved budgets) apply on reload instead of only at create. Undersource_of_truth: config.json, inline budgets and rate limits on teams, customers, keys and provider configs survive the prune (a linked inline rate limit no longer stops startup with a foreign key error), calendar-aligned virtual key budgets keep their alignment after a restart, and a governed team is no longer saved with its own inline budgets (fix(governance): keep inline limits through the config.json prune and off governed teams #8156, fix(governance): reconcile virtual key and provider config inline budgets from config.json #8161, fix: store config.json virtual key limits on model configs #8306)PUT /api/governance/virtual-keys/{id}withoutkey_idsno longer setsallow_all_keysto false and detaches every key; an omittedkey_idsnow means unchanged (fix(governance): preserve VK provider-config key associations when PUT omits key_ids #7362) (thanks @xiechimon!)PUT /api/providers/{provider}/keys/{key_id}now keeps fields left out of the body (value,weight, provider configs) instead of resetting them; fields sent explicitly, even as empty values, are still applied (fix(provider-keys): preserve omitted fields on partial PUT update #7679)409 Conflictwith the priority and the real scope ID, instead of500with a pointer address (fix: return 409 conflict instead of 500 for duplicate routing rule priority and print scope ID rather than pointer #8104)502 upstream_credentials_exhausted; keys pinned withx-bf-api-key/x-bf-api-key-idstill fail immediately. A session no longer binds to a fallback that served a request sent with the caller's direct key, no longer moves a request pinned withx-bf-api-key-id/x-bf-api-keyoff the head provider, and drops its binding to a key removed from a single-key pool (fix: fall back to sibling keys when a session-bound key is rejected outright #8100, fix: keep session affinity from binding to fallbacks that served direct-key requests and from moving requests pinned to the head provider's key #8109, fix: make the key blacklist tests fail when blacklists are ignored, and correct the session affinity docs #8112, fix(sessionaffinity): drop session key bindings to keys removed from single-key pools #8384)400/409/413/415/422from the primary (for examplelogprobsout of range) is now returned to the caller instead of being answered by a provider the caller never named. Caller- and routing-named fallbacks are unchanged. The Gemini rawgenerateContentpath now forwardslogprobsinstead of stripping them (fix(core): keep caller-fault errors from running catalog-inferred fallbacks; forward raw gemini logprobs #8434)DeepInfra-Embeddingsis found at warmup (fix(routing): keep custom provider name case in complexity router config #7590) (thanks @MohammadHijjawi97!)Otherrow, so each tab adds up to its total and matches the Overview (fix(dashboard): count ranking rows the caller cannot name under Other #8087)listToolFiles,readToolFileandgetToolDocsnow show only the tools the request's virtual key may execute (governance: scope code-mode tool discovery to the request's grant #7888)POST /api/mcp/client/{id}/reauthorizereturns aflow_idandGET /api/oauth/config/{id}/status?flow_id=reports the live flow, so the consent popup no longer closes on the first poll. OAuth2state,scopeandclient_namecolumns aretext, so long values no longer fail withserver_erroron Postgres. Switching a token exchange client ID or secret to anenv./vault.reference is saved instead of keeping the old plaintext. A client that has not finished admin authorization stays inpending_verificationafter being enabled, a rejected identity token in token exchange is reported as rejected rather than missing, and deleting or updating an unknown MCP client returns404(fix: preservepending_verificationstate throughEnableClientand connection checker #6926, feat: addflow_idto reauthorize status polling so stale "authorized" config status is not mistaken for a completed consent #7041, fix: surface context-aware rejection message when inbound bearer token is rejected during delegated exchange #7051, fix: widen OAuth2 client-controlled columns (state,scope,client_name) fromvarchartotextto stop Postgres rejecting long values withserver_error#7405, fix: switching an MCP token-exchange or agent OAuth secret to a reference is saved #8160, fix(mcp): return 404 for deleting or updating an unknown MCP client #8436)cache_controlmarkers (Claude Code on/anthropic/v1/messagesamong them) to OpenAI, Azure, Bedrock and Bedrock Mantle were still switched toprompt_cache_options.mode: "explicit", which re-wrote the whole conversation at the cache-write rate every turn. They now stay in implicit mode, with the 20-message lookback. Markers still becomeprompt_cache_breakpoint, capped at three in implicit mode and four when the caller sets explicit mode (fix: gpt models implicit caching fix #8322, fix(openai): keep gpt-5.6+ in implicit prompt cache mode when translating cache_control #8334)encrypted_content, the strip-and-retry now also clears the itemidon messages, function, custom tool, web search, computer and MCP calls paired with the removed reasoning, so the retry succeeds instead of failing withItem 'fc_..' ... without its required 'reasoning' itemor404 Item with id 'rs_..' not found(seen from Warp on gpt-5.6-luna and load-balanced Azure keys) (fix: reasoning strip with item ids #8374)429for rate limits and quota,400for context-length errors and502otherwise, Anthropic stream errors map theirtype(including529 overloaded_error), unparseable Gemini stream errors get502, and unexpected key-selection failures return500bifrost_internal. These were all reported as400, which hid them from 5xx dashboards and stopped key rotation (fix: give stream errors and key-selection failures a real HTTP status #7611)/openai/v1/responsesand the litellm, langchain, pydanticai and Azure deployment copies is now sent asevent: error, as the OpenAI SDKs expect (fix(integrations): type mid-stream errors as event: error on openai responses routes #8421)messageStopnow returns a502error with the event count and last event type instead of a silent success, with any billed usage attached (feat(bedrock): error on incomplete converse streams missing messageStop #7797)Content-Lengthare now reported as errors instead of completing silently, and the incomplete connection is discarded (fix(core): detect truncated Content-Length response bodies #8287) (thanks @Constantine3!)aftercursor onGET /v1/filesfor Replicate is no longer used as the next request URL; only itscursorvalue is kept and requests stay on the configured file endpoint. Older full-link cursors still work when they point at the configured endpoint (fix(replicate): keep file list pagination on the configured file endpoint #8290)400 invalid_request_errorinstead of crashing on a missing key config, and a panic inside a WebSocket session (realtime, live, Responses) drops only that connection instead of the process (fix(realtime): answer unmapped ephemeral tokens on Azure with a client error and recover WebSocket session panics #8282)openai/gpt-realtimenow use therealtimepricing row, so log costs and governance budgets are no longer left at zero (fix: prefer realtime catalog rates for realtime requests #8411)should_convert_paramson, the output token cap and thinking budget are now fitted to the model each attempt actually reaches (fallbacks and aliased deployments included) instead of once before key selection, and a lowered cap no longer leaves a thinking budget above the model's own maximum (for example 65534 ongemini-2.5-flash). Each change is logged in the routing logs (fix: clamp max outputs and thinking tokens on fallbacks #8242)max_tokensis lowered to the target's datasheet ceiling (unless theoutput-300kbeta is on) andadaptivethinking becomesenabledwith a derivedbudget_tokenson Haiku 4.5, Sonnet 4.5 and Opus 4.5, instead of returning400(fix: output tokens on fallback and adaptive thinking #7732)supports_custom_tools: false(for example gpt-4o) now gets custom (freeform) tools as function tools with one stringinputparameter, with tool choice,allowed_tools, replayed custom tool calls and namespace tools rewritten to match, and calls returned to the client ascustom_tool_call(fix(openai): send custom tools as function tools when the datasheet marks them unsupported #8248)default_reasoning_effortnow decides whether a request with noreasoning.effortreasons, so models such asgpt-5.6-solthat default tononekeeptemperatureand other sampling fields (moves reasoning_effort support to datashseet #8075)false(for example Fireworks models on Azure) now have onlyreasoning.summarydropped by compat, keeping reasoning effort, instead of returning400(fix: drop reasoning.summary for models that don't support it #8408)reasoning_effortandreasoning.effortwith the same value (ai-sdk clients such as ZCode) are accepted instead of failing with400; only conflicting values are rejected ([fix]: schemas - accept mirrored reasoning shorthand+object when values agree #7648) (thanks @Daltonganger!)response_formatof typejson_schemasent to DeepSeek is now downgraded tojson_objectwith the schema passed in a system message, instead of failing with DeepSeek's400(fix(deepseek): downgrade json_schema response_format to json_object #8432)moonshotai.kimi-k3on Bedrock no longer fails on tool schemas with stringformatvalues outside its allowlist (such asformat: byte), and the schema rewrite now also runs on/v1/chat/completions(fix: tool format for kimi moonshotai #8179)input_schemahas a rootoneOf/anyOf/allOf(such as Codex'sautomation_update) are now flattened on Bedrock Converse as on Anthropic direct; schemas beyond the normalizer's limits fail with400instead of reaching Bedrock (fix: anthropic tool normalization on claude bedrock modelsz #8417)guardContentblocks in messages and system prompts are no longer dropped, so guardrails evaluate only the marked content, andguardrailConfig.tagSuffix(validated, 1 to 20 alphanumeric) is supported for inline guard tags across Converse, InvokeModel, their streaming variants, chat completions and Responses. InvokeModel ingress also reads theX-Amzn-Bedrock-Guardrail*headers (adds support for guardrails in bedrock payloads #7715)filecontent block withcache_controlon Bedrock Converse now emits itscachePoint, so cached documents are no longer billed as fresh input on every request (fix: bedrock cache control block drop #7646)redactedContent), with the original reasoning item ID kept, and astoreboolean inadditionalModelRequestFieldsnow reaches Responses targets. A chat-to-Responses conversion also reuses the reasoning item id paired with its encrypted token instead of minting a new one that upstream rejects (adds all missing harness tests #7907, Bedrock → Responses store propagation fix #8032, fix(bedrock): keep the reasoning item id when a Converse client replays redactedContent #8033, fix converse reasoning reply #8044)claude-sonnet-4-5) that governance later routes to Bedrock, fixingthinking blocks in the latest assistant message cannot be modified(ifix: bare claude model hoist thinking blocks #7789)safeguard_resultsin the finalmessage_deltafrom Bedrock and Anthropic now reach Claude Code on the/anthropicroute insidedelta, so its auto-mode classifier keeps working through Bifrost (fix: safeguards results in delta #8154, fix: preserve safeguard results in streaming responses #8252)x-anthropic-billing-headersystem line Claude Code sends is found and stripped again when an effort-only system message comes first, so guardrails no longer evaluate it and non-Anthropic fallbacks no longer receive it as prompt text; Anthropic attempts get it restored in place (adds back billing header stripping #8039)tool_additionandtool_removalblocks on system messages andtool_changeson replayedcompactionblocks are now carried through to Anthropic, streaming included, instead of being silently dropped (fixes mid conversation tool changes #8278)actionsarrays oncomputer_callitems (gpt-6.1-sol and newer) are kept and split into one Anthropictool_useper action, OpenAI's geometry-lesscomputertool maps to Anthropic's computer toolset, andcache_controlon Anthropic server tools is no longer dropped (fix: add actions array field toResponsesComputerToolCallfor newer model responses #7733, fix: preserve cache control for anthropic server tools #7979, fix: mux oai and anthropic style computer tools #7980)error.typein Anthropic error envelopes now matches the HTTP status or a documented Anthropic type instead of defaulting toapi_erroror leaking gRPC names such asINVALID_ARGUMENT, andstop_reason: "stop_sequence"with the matchedstop_sequenceis preserved, including Claude on Vertex ([fix]: Anthropic provider - normalize error.type in the Anthropic error envelope #7916, [fix]: Anthropic - preserve stop_sequence stop reason and matched sequence #7921) (thanks @jimseiwert!)status: "incomplete"is kept on OpenAI Responses, multi-part assistantoutput_textmaps to text on chat completions instead of422,cache_control, citations,prompt_cache_breakpointandcachePointsurvive content block copies, historyoutput_textwithoutlogprobsis no longer sent as"logprobs": null, and streamed Responses events keepextra_fieldson the/openai/v1/responsesdrop-in (fix responses history null logporbes handling #8036, fix: extra fields on responses stream chunks #8169, fix(core): map output_text to text on multi-part assistant turns #8208, fix(core): keep the failed tool result marker on OpenAI Responses #8209, [fix]: preserve cache_control, citations, prompt_cache_breakpoint and cachePoint in deepCopyChatContentBlock #8216) (thanks @shreyanshj10, @Shashank2985!)delta.role: "assistant"exactly once, so clients such as LangChain.js no longer drop tool calls from turns with only reasoning and function calls ([fix]: send assistant role once on converted chat streams #7699) (thanks @hmdsefi!)/v1/decisionsemulation no longer forcestool_choice: requiredon models that reject it;service_tier,speedandinference_geosurvive the chat-to-Responses bridge in both directions and on streams; bare Perplexitysonaris no longer sent to/v1/responses; MCPserver_labelround-trips onmcp_list_toolsandmcp_callitems; legacy/v1/completeprompts already framed asHuman:/Assistant:are not wrapped again (also on Bedrock) (fix: cohere replay thinking order, decisions tool_choice gate, served tier on chat bridge, bare sonar routing, mcp server_label, legacy prompt framing #8423)/v1/responses, the Go SDK, Vertex and bare-model/genai, JSON mode is no longer sent with function tools to pre-Gemini-3 models, Gemini-native tools without a Responses form (retrieval,googleSearchRetrieval,enterpriseWebSearch,googleMaps,urlContext,computerUse) are forwarded byte for byte,candidateCount,logprobs,responseLogprobsandseedare kept, multiple candidates are no longer flattened, andurlContextMetadatais carried back (fix(gemini): keep Responses-path parity for JSON mode with tools, native retrieval tools, candidateCount and logprobs #8422, fix(gemini): carry urlContextMetadata and forward native tool entries byte-faithfully on the Responses path #8433)input_schemaare declared instead of dropped and parallel tool results are reordered to call order for Vertex, tool output with a$refkey is sent as a string instead of failing with400,nmaps tocandidateCounton Gemini and Vertex chat, streaming Responses keeps output items when parts andfinishReasonshare a frame, theAuthorizationheader is dropped whenx-goog-api-keyis set for Vertex Express, and image, Imagen and transcription extra params survive retries and fallbacks (fix(gemini): strip Authorization header when setting x-goog-api-key for Vertex Express compatibility #3835, [fix]: record output items before closing open items on Gemini streaming Responses #7359, [fix]: Gemini/Vertex chat completions honor n #7394, fix: gemini rejecting tools with $ref #7713, fix: gemini parameters during fallback #7883, [fix]: Gemini provider - keep schemaless function tools and order tool results by call id #7927) (thanks @jimseiwert, @axelray-dev, @wofiporia, @SandyDRawat!)taskTypeortitlenow reaches every entry of a Gemini:batchEmbedContentsrequest instead of leaking to the top level, and transcription pricing uses the provider-reportedusage.cost(for example OpenRouter voxtral) instead of the datasheet ([fix]: use provider-reported cost for transcription pricing #7825, fix(gemini): apply request-level taskType and title to every batched embedding input #7884) (thanks @hmdsefi!)Missing required parameter: 'model') (fix: disable GenAI raw-body passthrough when falling back to non-Google providers #8230, fix: convert GenAI requests for Claude and other non-Google Model Garden models on Vertex instead of passing the raw body through #8343)POST /genai_passthrough/v1beta/interactionsis no longer refused on Gemini, Vertex keeps thev1beta1version prefix that Interactions needs, and Veo video retrieve and download poll the region named in the operation instead of the key's region (fix: vertex video retrieve uses region from the id #8083, fix: vertex passthrough version prefix #8134)state, null instructions and descriptions) are forwarded, native extensions such asimagesare kept, upstream headers (x-typesafe-request-id,Retry-After) and error detail are forwarded, and/typesafe/v1/modelscalls the endpoint's own model list (fix: typeSafe native compatibility rejects SDK-valid inputs and loses payload/response fidelity #7652)GET /v1/modelsuntil the request timeout (each poll is bounded and last-known catalog models are served), and paging through a listing that changes between pages no longer repeats or skips models (fix(core): stop dead provider blocking aggregate model list #6614, fixes listAllModels returns duplicate and skipped models when a provider listing changes between pages #7771) (thanks @novai42!)value, a422for a never-grantedcopilot_requestspermission gets its own guidance, and masked secrets no longer fail format checks when editing a Copilot provider (fix(ui): skip format validation for API-masked secret values in Copilot provider form #6840, fix(core): allow GitHub Copilot provider keys to have empty values for GitHub App bundle authentication #6841, fix(githubcopilot): distinguish 422 permission-not-granted from repository-access errors with targeted guidance #6842)status_code="unknown"on Prometheus, OTel, Datadog and Splunk, and error spans always carryhttp.response.status_code(fix: report the effective HTTP status on error metrics and spans instead of "unknown" #7451)protocol: httpto anhttps://metrics_endpointwere silently sent as plaintext and dropped, becauseinsecure(defaulttrue) forced plaintext. The endpoint scheme now picks the transport, andinsecureonly skips certificate verification (fix: stop downgrading https metrics endpoints to plain HTTP #7815)gen_ai.request.toolslike Responses requests. Prometheushttp_*series no longer disappear after a config reload, the telemetry plugin's owncustom_labelsare applied (unioned withclient_config.prometheus_labels), a lone"*"inrequest_headersis no longer swapped for the stored value, and the plugins API reports a loaded plugin's runtime status instead of masking it asdisabled(fix: correct connector export gaps surfaced by the e2e matrix #8228)user_labels_enabledand custom labels configured, metrics no longer overwrite each other's label names, which made/metricsreturn500with duplicate label errors (fix: update Bifrost labels append call to correctly work with custom labels #7739)custom_tool_call,local_shell_callandcode_interpreter_callinputs are now recorded in traces and log tool calls instead of being dropped (fix: fix custom tool calls tracing for responses #7738)metrics_enabled(defaulttrue;falsestops serving/metrics) is now accepted by the config.json schema (fix: add metrics_enabled field to config json for telemetry #7745)10,000+when the total is capped (fix(scale): postgres critical fixes for 500M logs, 100k users and VKs #7606, fix(scale): postgres high-severity fixes for 500M logs, 100k users and VKs #7607, fix(scale): postgres scale follow-ups and cap handling #7608, updates migration flow for indices #8439)jsonbrejects no longer fails the whole batch insert. Text columns are sanitized before the write (a NUL-only value is stored as U+FFFD), binary bodies (audio passthrough, multipart uploads, Bedrock event streams) are stored as<binary content omitted: N bytes>, and large-response previews end on a character boundary so Postgres no longer rejects the batch with SQLSTATE 22021 (fix(logstore): keep Postgres log batches from failing on NUL bytes, invalid UTF-8 and jsonb-rejected escapes #8428, fix(providers): end large-response log previews on a UTF-8 boundary #8429, fix(logging): store binary log bodies as a size placeholder #8430, regression fixes for pgtextsafety #8437)processingrow before eviction, and the late response updates that row (adds upsert based writes to avoid losing logs #8350)ALTER TABLEand stalling the cluster (logstore gating for smoother logstore migrations #8264)BEGIN, so a write that follows a read waits out the busy timeout instead of failing at once, for example right after startup while the pricing sync is writing (follow up harness fixes #8407)Max query size exceededon every logs chart. Bifrost now sends ClickHousemax_query_size, configurable aslogs_store.config.max_query_sizein bytes (Helmstorage.logsStore.clickhouse.maxQuerySize) (fix(logstore): make ClickHouse max_query_size configurable and default it to 16 MiB #8126)input_historyandresponses_input_historyare truncated to the last message, as on Postgres and SQLite (fix(logstore): truncate ClickHouse input histories to last message in list queries #8247) (thanks @ege-arhan!)error_detailsstays in the database instead of being offloaded, so error type, error code and status code rankings and filters work again (fix(logstore): keep error_details in the database when object storage is on #8183)log_retention_daysnow updates the running log cleaner immediately (#8243) (thanks @Constantine3!)DeepSeek-V4.1-Flashare priced from a lowercase datasheet entry (#8229)POST /api/pricing/force-syncno longer returns500when the pricing URL hangs; the fetch timeout (60s pricing, 45s model parameters) covers only the fetch and its retries, and stored pricing is used after it. Logs now name the URL that failed (#7737)PUT /api/configwith an emptypricing_urlormodel_parameters_urlresets it to the default datasheet URL instead of failing (#7449)onboarding_dismissedis no longer lost when config writes overlap at boot or across replicas (#7564)serving version "" ... has no version row(#8251) (thanks @ozenyunus35!)/v1/chat/completionsentry is no longer served to a/v1/responsescaller with the same cache key (#7561) (thanks @Atharva-Kanherkar!)semantic_cacheconfigs like config.json does. Embedding failures store an exact-match entry instead of skipping the write,clear-by-keypercent-decodes the key (bad encoding returns400), clearing a missingcache_idsucceeds on Redis, andx-bf-cache-no-storeis case-insensitive (#8338)0now renders in the log view (#7662) (thanks @G-XD!)__none__, MCP badges have dark mode colors, the rate limit placeholder reads "No limit", the virtual keys Status column is wider, the Virtual MCPs table drops its noisy warning icon, and the create MCP client form has the headers and Allowed Extra Headers fields the edit form has. Creation buttons read "Add" instead of "New", the onboarding widget renders above other overlays, enterprise pages pick up their Tailwind classes again when the UI is linked through a symlink, and charts no longer log width/height warnings on load (#7407, #7558, #7594, #7628, #7663, #7673, #7735, #7817, #7985, #8387) (thanks @basil-k-aji-dev!)🔧 Maintenance
@tanstack/react-router1.170.5,@tanstack/router-plugin1.168.7,axios1.20.0 anddompurify3.4.16 (#7796, #8178)🗄️ Database Migrations
This release adds 45 config store and 9 log store migrations. Most add nullable or defaulted columns, new tables, or indexes, and are safe to run while older pods are still serving. The items with a Note need attention before or during the upgrade.
The migration runner now reads the list of applied IDs once at startup and skips steps that are already recorded, so restarts no longer re-run finished steps (#7994). Postgres index checks are scoped to the current schema, and an INVALID index left behind by an interrupted build is dropped and rebuilt (#8439).
Config store: data moves and backfills
compat_force_reasoning_only_models_to_responsestotrueon every existing client config row. Non-reversible: the rollback does nothing. What changes: chat requests with tools to models that cannot reason with tool calls are routed to the Responses API on existing deployments after the upgrade. What to do: to keep the 2.2.6 behaviour, setclient.compat.force_reasoning_only_models_to_responses: falsein config.json (UI: Settings, Compatibility) after upgrading.search_context_cost_per_querykey toweb_search_cost_per_requestinside saved pricing overrides. Non-reversible: no rollback. What changes: pods still on 2.2.6 do not see the old key in overrides during the rollout. What to do: renamesearch_context_cost_per_querytoweb_search_cost_per_requestin any pricing override you keep in config.json.model_pricing.web_search_cost_per_requestand copies existingsearch_context_cost_per_queryrates into it. Reversible: drops the column.next_check_aton settled and unpriceable batch jobs (in batches of 10,000), then builds the partial indexidx_batch_jobs_dueconcurrently. Non-reversible: cleared values cannot be restored.config_client.enforce_vk_object_ownershipwith defaulttrueand sets existing rows totrue. Reversible: drops the column. What changes: files, batches, containers and cached content created through a virtual key can only be read, listed or deleted through that same virtual key. What to do: setclient.enforce_vk_object_ownership: falsein config.json (UI toggle under virtual key settings) if clients share provider objects across virtual keys.Config store: unique indexes on existing data
governance_virtual_keys.value_hashandsessions.token_hashvalues to NULL in batches of 10,000, then builds UNIQUE indexes on both concurrently. If duplicates already exist, it builds a non-unique index and logs a warning instead of failing. Non-reversible.mcp_oauth_flowsrows, then replacesidx_mcp_oauth_flows_statewith a UNIQUE index built concurrently. It keeps a non-unique index and warns if duplicates remain. Non-reversible: deleted rows cannot be restored.Config store: other non-reversible additions
Each of these only adds a column or table and is safe during a rolling deploy. Its rollback refuses to run because dropping the column would destroy saved settings or credentials.
config_keys.vertex_aws_workload_identity_json.oauth_*columns toconfig_keys(bounded by a 5s lock timeout).state,scopeandclient_namecolumns fromvarchartotext. No rewrite, brief lock. No rollback.warp_config.api_key_id, and clears and drops the earlierapi_keycolumn.warp_configis new in this release, so nothing is lost when upgrading from 2.2.6.governance_virtual_keys.disable_content_logging.config_client.a2a_external_client_url.tenantto the agent push config table.config_providers.ignore_provider_cost(defaultfalse).provider_objects, which records the virtual key that created each file, batch, container or cached content.Config store: reversible additions
Each of these rolls back cleanly by dropping what it added.
add_virtual_key_delete_after_expire_columnadd_client_config_delete_expired_virtual_keys_columnadd_mcp_discovered_instructions_columnadd_mcp_instruction_cap_columnsadd_mcp_client_max_instructions_length_columnadd_mcp_client_require_public_target_columnadd_warp_history_retention_days_columnadd_warp_additional_models_columnadd_ttft_timeout_ms_column_to_routing_targetsadd_mcp_code_mode_limits_client_columnadd_compat_force_reasoning_only_models_to_responses_columnadd_a2a_raw_capture_settingadd_100k_token_pricing_columnsadd_decisions_pricing_columnsadd_injected_tools_json_columnadd_compat_convert_decision_to_responses_columnadd_cloudflare_key_config_columnsadd_mcp_oauth_private_allowlist_columnadd_warp_config_tableadd_vk_provider_config_virtual_key_id_indexadd_vk_mcp_configs_mcp_client_id_indexadd_vk_provider_config_provider_and_key_indexesadd_virtual_keys_created_at_id_indexadd_batch_jobs_file_id_indexesadd_provider_objects_object_id_indexgovernance_virtual_keys.business_unit_idwith an index. The index is not built concurrently. The column starts empty, so the build is short, but it briefly blocks virtual key writes.Log store
virtual_key_id,user_id,team_id,selected_key_id,routing_rule_id,customer_id,business_unit_idandproject_idindexes onlogs(and the owner indexes onmcp_tool_logs) with(column, timestamp)composites. On Postgres a background job builds the composites concurrently and drops each old index only after its composite is valid. Reversible on SQLite. On Postgres the rollback does not restore the old indexes. What changes: a 2.2.6 pod that restarts during the rollout, or after a rollback, rebuilds the old single-column indexes in the background on a largelogstable, and 2.3.0 pods then drop them again. What to do: on large Postgres log stores, finish the rollout without restarting old pods, and expect extra index build I/O right after the upgrade.logs.agent_correlation_id,mcp_tool_logs.session_idandmcp_tool_logs.agent_correlation_id, and createsagent_logs. It takes brief locks onlogsandmcp_tool_logs, bounded by a 5s lock timeout. Its indexes are built in the background, concurrently on Postgres. Reversible only while the new table and columns are empty.agent_logs.raw_requestandraw_response. Non-reversible: the rollback would delete recorded transport bodies.warp_conversationsandwarp_messages. Reversible only while they are empty.finish_reason,total_tokens,costandquestion_jsontowarp_messages. Reversible only while no outcome is recorded.warp_conversations(updated_at DESC). Reversible.logs.embedding_input. Reversible.logs.live_session. Reversible.cached_read_tokensbackfill now walklogsin one-hour windows and run once. Completion is recorded in the migrations table aslogs_bg_invalid_metadata_cleanup_v1andlogs_bg_cached_read_tokens_backfill_v1.🐙 Closed GitHub Issues
ShellToolnot supportedoutput_tokens_details.thinking_tokens) that AWS returnsreasoning.max_tokens$ref- JSON-object tool output is forwarded as structured function_response.responsetaskType/titleapplied only to the first input of a batched/v1/embeddingsrequestlogprobs: null, causing upstream 400tool_result.is_errorno longer reaches OpenAI Responses upstreams - thestatus: "incomplete"set by the Anthropic converter is cleared inToOpenAIResponsesRequest(#7835)toolobject in the Responses conversionpopulateSkillFilesresolves the serving version viaskill.LatestVersion, whichconfigEntryToTableSkillnever sets