Skip to content

provider: add Valar - #1780

Open
TomAmster wants to merge 1 commit into
Portkey-AI:mainfrom
TomAmster:provider/valar
Open

provider: add Valar#1780
TomAmster wants to merge 1 commit into
Portkey-AI:mainfrom
TomAmster:provider/valar

Conversation

@TomAmster

Copy link
Copy Markdown

What

Adds Valar as a provider under the valar slug.

Valar is an inference provider built for high-throughput agentic and asynchronous work. It serves open and frontier models behind three APIs at https://api.valarhq.ai/v1 — OpenAI Responses, OpenAI Chat Completions and Anthropic Messages — and adds completion windows, which trade turn latency for a lower rate, plus background execution with webhooks and a batch API for offline jobs. Docs: https://docs.valarhq.ai.

Endpoints covered

Gateway fn Valar endpoint
chatComplete (streaming included) POST /v1/chat/completions
createModelResponse, getModelResponse POST /v1/responses, GET /v1/responses/{id}
messages, messagesCountTokens POST /v1/messages, POST /v1/messages/count_tokens
uploadFile, listFiles, retrieveFile, deleteFile, retrieveFileContent /v1/files*
createBatch, retrieveBatch, listBatches /v1/batches*

Auth is Authorization: Bearer <key> on every surface, the Anthropic-compatible one included, so a single header covers all endpoints.

Provider-specific handling

Valar rejects parameters it does not serve rather than ignoring them, so the configs drop them at the Gateway instead of letting a generic OpenAI-shaped request 400 upstream. Each choice below tracks the published support matrix (https://docs.valarhq.ai/support):

  • max_tokensmax_completion_tokens. Valar rejects max_tokens as a deprecated field, so a client still sending it is carried over to the modern field. When a request carries both, max_completion_tokens is applied last and wins.
  • Dropped on chat completions: frequency_penalty, presence_penalty, logit_bias, seed, stop, logprobs, top_logprobs, audio, prediction, web_search_options, service_tier, functions/function_call, and store (responses are always stored; store: false is rejected).
  • n is clamped to 1 — a Valar response always carries exactly one choice.
  • No default model. Valar's catalog is served from its own model list, so model is required and nothing is invented.
  • Responses API is a deliberate subset: no streaming (long jobs use background: true plus a poll or a webhook), no server-side conversation state (previous_response_id, conversation, instructions, prompt), and scheduling comes from the completion window rather than service_tier.
  • Messages API reuses anthropic-base minus container, mcp_servers and service_tier, plus Valar's output_config extension (structured outputs and reasoning effort). Its errors come back in the OpenAI error envelope rather than Anthropic's, so the transform reads them from there.
  • createBatch omits output_expires_after: Valar's batch schema is additionalProperties: false, so forwarding it would fail the request.
  • x-portkey-valar-completion-window (also accepted unprefixed as x-valar-completion-window) sets the scheduling tier — asap / priority / standard / flex. Valar reads metadata.completion_window from the body first; the header is the fallback for clients that own the request body, such as the Claude Agent SDK.

One change outside the provider directory

src/providers/open-ai-base/index.ts had a pre-existing type error (Property 'default' does not exist on type 'ParameterConfig | ParameterConfig[]') that made ts-jest refuse to compile any test importing it — which is every test of a provider built on open-ai-base. The fix is a one-line cast inside the existing !Array.isArray(...) guard, no behaviour change. Happy to split it out if you'd rather take it separately.

Testing

src/providers/valar/valar.test.ts — 13 tests covering endpoint resolution, auth and the completion-window header, the max_tokens mapping and its precedence, the dropped/kept parameter sets, the n clamp, and both messages transforms. npm run test:gateway goes from 47 to 60 passing tests with no new failures (8 suites already fail to compile on main from unrelated type errors).

Also ran end-to-end through npm run dev:node against a local mock upstream (x-portkey-custom-host), confirming on the wire:

POST /v1/chat/completions
  authorization: Bearer vlr-test-key
  x-valar-completion-window: flex
  {"model":"zai-org/GLM-5.2","messages":[...],"max_completion_tokens":128,"n":1}

sent from a request carrying max_tokens: 128, seed: 42, store: false and n: 3 — mapped, dropped and clamped as intended. /v1/messages, /v1/responses, /v1/batches and a streamed chat completion were verified the same way.

Usage

curl http://localhost:8787/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'x-portkey-provider: valar' \
  -H 'Authorization: Bearer $VALAR_API_KEY' \
  -d '{"model":"zai-org/GLM-5.2","messages":[{"role":"user","content":"Hello!"}]}'

Valar (https://valarhq.ai) is an inference provider for high-throughput
agentic and asynchronous workloads. It serves the OpenAI Responses,
OpenAI Chat Completions and Anthropic Messages APIs at
https://api.valarhq.ai/v1, and adds completion windows that trade turn
latency for a lower rate, background execution, and a batch API.

Covers chatComplete (streaming included), createModelResponse /
getModelResponse, messages / messagesCountTokens, the files endpoints and
the batches endpoints.

Valar rejects unsupported parameters rather than ignoring them, so the
configs drop what it does not serve and map the deprecated `max_tokens`
onto `max_completion_tokens`. `x-portkey-valar-completion-window` sets the
scheduling tier for clients that own the request body.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant