Skip to content

feat(byok): per-user provider config — proxy URL + pinned model - #75

Open
harshithmullapudi wants to merge 1 commit into
mainfrom
feat/byok-provider-config
Open

feat(byok): per-user provider config — proxy URL + pinned model#75
harshithmullapudi wants to merge 1 commit into
mainfrom
feat/byok-provider-config

Conversation

@harshithmullapudi

Copy link
Copy Markdown
Member

Summary

Extends the BYOK settings surface so users can configure their own proxy endpoint and pinned model per provider — not just an API key. Chats that run against any of these keys still skip the aura debit; the gate is `usedBYOK`, which stays true across all four slots.

Four provider rows on the settings card:

Slot Key Base URL Model
Anthropic required optional (defaults to `claude-haiku-4-5-20251001`)
OpenAI required optional (defaults to `gpt-5.4-mini`)
OpenAI Proxy required required optional (proxy routing key)
Ollama Cloud required optional (defaults to `https://ollama.com/v1\`) optional (defaults to `gpt-oss:120b-cloud`)

Changes

  • DB (`packages/db/prisma/schema.prisma`) — two nullable columns on `ModelKey`: `baseUrl`, `model`. Migration `20260728084341_modelkey_proxy_fields` (safe — only adds columns).
  • Store (`apps/web/src/lib/byok/store.ts`) — adds `openai_proxy` to `BYOK_PROVIDERS`. Enforces baseUrl on `openai_proxy`, drops baseUrl silently for providers that don't accept it, always accepts a pinned model. `resolveByokForUser` walks all four slots with `LLM_PROVIDER` as an explicit override and a natural priority chain (anthropic → openai_proxy → openai → ollama).
  • Chat model (`apps/web/src/lib/chat-model.ts`) — four BYOK branches. `anthropic`/`openai` stay direct against their vendor endpoints; `openai_proxy` uses `createOpenAI({ baseURL, apiKey })`; `ollama` uses `createOpenAICompatible` so a user can point at a self-hosted daemon or Ollama Cloud with their own key.
  • UI (`apps/web/src/ui/BYOKSection.tsx`) — four rows. `baseUrl` input renders for `openai_proxy` (required) and `ollama` (optional). `model` input renders for every provider. Saved rows show `url · ` and/or `model · ` beneath the status pill. Client-side validation: `http(s)://` on any provided baseUrl, and the `openai_proxy` baseUrl requirement.
  • API (`apps/web/src/app/api/byok/route.ts`) — already accepted optional `baseUrl` / `model`; store enforces per-provider rules.
  • Ollama helper (`apps/web/src/lib/ollama.ts`) — exports `DEFAULT_OLLAMA_BASE_URL` so the BYOK Ollama branch shares the same fallback.

Fallback behaviour

Unchanged: if a user has no BYOK row, `resolveByokForUser` returns `null` and `getChatModel` falls through to the platform env path (`OPENAI_BASE_URL`, `OPENAI_CHAT_MODEL`, `OPENAI_STRUCTURED_MODEL`, `ANTHROPIC_API_KEY`, etc.). Aura debits normally on that path.

Migration required

```bash
pnpm db:migrate
```

Applies `20260728084341_modelkey_proxy_fields` — two `ALTER TABLE` statements adding nullable columns. No data loss risk; existing rows keep working with `baseUrl = null, model = null` (falls back to defaults).

Test plan

  • `pnpm typecheck` — passes locally
  • Store a BYOK row per each of the four provider slots via the UI; confirm chat routes through it and no aura debit occurs
  • Verify OpenAI Proxy row refuses to save without a baseUrl (client + server validation)
  • Verify pinned model id is used verbatim by the proxy call
  • With no BYOK stored, confirm the platform env path still works and aura debits
  • Ollama BYOK: pin `gpt-oss:120b-cloud` and a self-hosted baseUrl, confirm chat streams

🤖 Generated with Claude Code

Extends the BYOK settings surface so users can configure their own
proxy endpoint + model per provider, not just an API key. Chats that
run against any of these keys still skip the aura debit — the debit
gate is `usedBYOK`, which stays true across all four slots.

Four provider rows on the settings card:

| slot           | key | base URL     | model     |
| -------------- | --- | ------------ | --------- |
| Anthropic      | req | —            | optional  |
| OpenAI         | req | —            | optional  |
| OpenAI Proxy   | req | required     | optional  |
| Ollama Cloud   | req | optional     | optional  |

- Prisma: two nullable columns on ModelKey (baseUrl, model). Migration
  20260728084341_modelkey_proxy_fields (safe: only adds columns).
- Store: adds "openai_proxy" to BYOK_PROVIDERS. Enforces baseUrl on
  openai_proxy, drops baseUrl silently for providers that don't accept
  it, always accepts a pinned model. resolveByokForUser walks all four
  slots with LLM_PROVIDER as an explicit override and a natural priority
  chain (anthropic → openai_proxy → openai → ollama).
- Chat model: four BYOK branches. anthropic/openai stay direct; openai_proxy
  uses createOpenAI({ baseURL, apiKey }); ollama uses createOpenAICompatible
  so a user can point at a self-hosted daemon or Ollama Cloud with their
  own key.
- UI: four rows. baseUrl input renders for openai_proxy (required) and
  ollama (optional). Model input renders for every provider. Saved rows
  show "url · <baseUrl>" and/or "model · <id>" beneath the status pill.
- API: existing /api/byok POST already accepts optional baseUrl / model.
  Store enforces per-provider rules; client-side validation surfaces
  http(s):// checks and the openai_proxy baseUrl requirement.

Fallback behaviour unchanged: if a user has no BYOK row, resolveByokForUser
returns null and the platform env path (OPENAI_BASE_URL / *_CHAT_MODEL /
etc.) is used with normal aura debit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant