provider: add Valar - #1780
Open
TomAmster wants to merge 1 commit into
Open
Conversation
Valar (https://valarhq.ai) is an inference provider for high-throughput agentic and asynchronous workloads. It serves the OpenAI Responses, OpenAI Chat Completions and Anthropic Messages APIs at https://api.valarhq.ai/v1, and adds completion windows that trade turn latency for a lower rate, background execution, and a batch API. Covers chatComplete (streaming included), createModelResponse / getModelResponse, messages / messagesCountTokens, the files endpoints and the batches endpoints. Valar rejects unsupported parameters rather than ignoring them, so the configs drop what it does not serve and map the deprecated `max_tokens` onto `max_completion_tokens`. `x-portkey-valar-completion-window` sets the scheduling tier for clients that own the request body.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds Valar as a provider under the
valarslug.Valar is an inference provider built for high-throughput agentic and asynchronous work. It serves open and frontier models behind three APIs at
https://api.valarhq.ai/v1— OpenAI Responses, OpenAI Chat Completions and Anthropic Messages — and adds completion windows, which trade turn latency for a lower rate, plus background execution with webhooks and a batch API for offline jobs. Docs: https://docs.valarhq.ai.Endpoints covered
chatComplete(streaming included)POST /v1/chat/completionscreateModelResponse,getModelResponsePOST /v1/responses,GET /v1/responses/{id}messages,messagesCountTokensPOST /v1/messages,POST /v1/messages/count_tokensuploadFile,listFiles,retrieveFile,deleteFile,retrieveFileContent/v1/files*createBatch,retrieveBatch,listBatches/v1/batches*Auth is
Authorization: Bearer <key>on every surface, the Anthropic-compatible one included, so a single header covers all endpoints.Provider-specific handling
Valar rejects parameters it does not serve rather than ignoring them, so the configs drop them at the Gateway instead of letting a generic OpenAI-shaped request 400 upstream. Each choice below tracks the published support matrix (https://docs.valarhq.ai/support):
max_tokens→max_completion_tokens. Valar rejectsmax_tokensas a deprecated field, so a client still sending it is carried over to the modern field. When a request carries both,max_completion_tokensis applied last and wins.frequency_penalty,presence_penalty,logit_bias,seed,stop,logprobs,top_logprobs,audio,prediction,web_search_options,service_tier,functions/function_call, andstore(responses are always stored;store: falseis rejected).nis clamped to 1 — a Valar response always carries exactly one choice.modelis required and nothing is invented.background: trueplus a poll or a webhook), no server-side conversation state (previous_response_id,conversation,instructions,prompt), and scheduling comes from the completion window rather thanservice_tier.anthropic-baseminuscontainer,mcp_serversandservice_tier, plus Valar'soutput_configextension (structured outputs and reasoning effort). Its errors come back in the OpenAI error envelope rather than Anthropic's, so the transform reads them from there.createBatchomitsoutput_expires_after: Valar's batch schema isadditionalProperties: false, so forwarding it would fail the request.x-portkey-valar-completion-window(also accepted unprefixed asx-valar-completion-window) sets the scheduling tier —asap/priority/standard/flex. Valar readsmetadata.completion_windowfrom the body first; the header is the fallback for clients that own the request body, such as the Claude Agent SDK.One change outside the provider directory
src/providers/open-ai-base/index.tshad a pre-existing type error (Property 'default' does not exist on type 'ParameterConfig | ParameterConfig[]') that madets-jestrefuse to compile any test importing it — which is every test of a provider built onopen-ai-base. The fix is a one-line cast inside the existing!Array.isArray(...)guard, no behaviour change. Happy to split it out if you'd rather take it separately.Testing
src/providers/valar/valar.test.ts— 13 tests covering endpoint resolution, auth and the completion-window header, themax_tokensmapping and its precedence, the dropped/kept parameter sets, thenclamp, and both messages transforms.npm run test:gatewaygoes from 47 to 60 passing tests with no new failures (8 suites already fail to compile onmainfrom unrelated type errors).Also ran end-to-end through
npm run dev:nodeagainst a local mock upstream (x-portkey-custom-host), confirming on the wire:sent from a request carrying
max_tokens: 128,seed: 42,store: falseandn: 3— mapped, dropped and clamped as intended./v1/messages,/v1/responses,/v1/batchesand a streamed chat completion were verified the same way.Usage