Add OpenRouter as a model provider - #487
Open
dsaad68 wants to merge 13 commits into
Open
Conversation
dsaad68
force-pushed
the
feat/openrouter-provider
branch
from
August 28, 2026 00:13
cd4d0a6 to
e5855e3
Compare
dsaad68
marked this pull request as ready for review
August 28, 2026 06:37
fx reaches models through three providers today: Vercel AI Gateway, Codex, and Grok. All three need either Vercel billing or a paid consumer subscription, and every base-URL override is restricted to loopback HTTP, so there is no way to point fx at another endpoint. OpenRouter adds ~400 models behind a single API key, including a set that cost nothing to run. That is the motivating outcome: a user with no Vercel billing and no ChatGPT or Grok subscription can now run fx for free. OpenRouter speaks the OpenAI Chat Completions format, which neither existing protocol module covers (vercel_protocol.zig is Vercel v3, responses_protocol.zig is the OpenAI Responses API). The new chat_completions_protocol.zig carries no vendor identity so any future OpenAI-compatible route can share it. It accumulates streamed tool calls by their `index` field and skips SSE comment lines, which OpenRouter emits as `: OPENROUTER PROCESSING` keep-alives mid-stream. Auth is a plain API key from OPENROUTER_API_KEY, following the existing ai_gateway_api_key shape rather than adding an OAuth flow. There is no stored session, so `fx logout openrouter` says so instead of falling through to the Vercel logout. Because fx calls tools on every turn, the catalog fetches only tool-capable models (?supported_parameters=tools); a model that cannot call tools breaks on the first step. Free models are identified by published pricing, sorted first, and surfaced four ways: a `Free` fact in the /model menu, a marker in `fx models`, a `free` field in its JSON, and a `--free` filter. The catalog's `is_free` flag is authoritative; the `:free` id suffix is a display convenience for surfaces that only carry id strings, and a fixture test pins the two together. Usage is exact: OpenRouter reports token counts and credit cost inline on the terminal chunk, so no deferred reconciliation is needed. The 402, 429, and 503 responses carry plain-language detail, since a negative balance blocks even free models and free-tier requests are capped at 20/min and 50/day. Ordering is this provider's own. compareModelCatalogEntries and projectPickerModelCatalog encode Vercel-specific product policy and are gateway-only, so OpenRouter is deliberately not routed through them. Verified against the live OpenRouter API and, offline, against a fake in tests/e2e/openrouter-stream.test.ts covering catalog filtering, free-first ordering, a streamed tool-call round trip, and the error paths. Unit tests cover the reducer, catalog parsing, and request serialization. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first commit wired OpenRouter through the CLI and the /model menu but missed /setup entirely, so the provider was unreachable from the TUI: the Model provider screen enumerated a hardcoded three entries and stopped at Grok. Selecting OpenRouter required `fx provider openrouter` from a shell. The provider stage now offers every ProviderId, and a test walks std.meta.tags to assert that, so a future provider cannot be added without appearing here. Connections gains an OpenRouter row reporting whether the key is present in the environment. It starts no sign-in flow, because there is none: selecting it explains that OPENROUTER_API_KEY is read from the environment. The row also appears during onboarding, where a user with no Vercel billing and no subscription most needs to learn a free path exists. credential_source_order was missing the key, so probing never recorded it and Connections could not have reported its status. Adding it exposed that the Credential source screen filtered subscriptions by listing them individually; that is now an isGatewaySource predicate, which keeps provider-scoped keys out of a screen that picks a Gateway credential and fails closed for any source added later. Verified in a real terminal: Model provider lists OpenRouter as current, Connections reports "environment" when the key is set, and selecting that row renders the guidance notice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A sweep for the same defect class as the setup-hub miss found four more places that enumerate providers by hand, where the compiler cannot help: - The ACP provider config option was a hardcoded JSON list, so an ACP client could report OpenRouter as current but never offer it as a choice. - fx status and doctor built connected_providers from three explicit checks, omitting OpenRouter even while reporting OPENROUTER_API_KEY as the auth source. - A tools-disabled host profile cleared the reviewer for three providers by name, leaving OpenRouter's active in a profile with no tools. - Missing-credential guidance fell through an if/else chain to the Gateway message, telling an OpenRouter user to run `fx login`. That chain is now missingCredentialMessage/missingInteractiveCredentialMessage, a switch, so a new provider must state its own guidance rather than inherit Vercel's. Token facts printed the raw window for values that are not round decimals. Gateway models are advertised in round numbers so this never showed, but OpenRouter reports powers of two, rendering "1048576 context · 230400 output". Non-round values now round to the nearest unit; exact multiples are untouched. Verified interactively: the /model menu shows "1M context · 230K output · Free" with the OpenRouter catalog status line, ACP lists all four providers with openrouter current, and fx status reports OpenRouter under connected_providers in both text and JSON. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The upstream draft was closed, so the rationale it carried — why a new chat-completions protocol module was needed, why the catalog filters to tool-capable models, and how free models are surfaced — would otherwise only survive in a closed pull request. Not part of the feature: it documents the change rather than shipping with it, and should be removed before any future upstream submission so it does not appear in the diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OpenRouter was environment-only: /setup listed an "OpenRouter API key" row that printed a notice telling the user to export OPENROUTER_API_KEY and restart, and `fx setup` always prompted for a Gateway key. Every other credential in fx can be pasted and saved, so OpenRouter now can be too. SecretStore gains a sibling slot rather than a second store threaded through its ~126 call sites: `openrouter` points at a store backed by its own keychain service (FX_OPENROUTER_API_KEY) and its own profile file (~/.fx/openrouter-api-key). Hosts that keep one slot leave it null and stay on the environment variable, so no existing construction changes. The environment still wins over a saved key, so a shell override keeps working, and a store that cannot be read falls through to "absent" rather than failing the whole resolution. Three defects this exposed: - The onboarding welcome screen rendered only four of the five connection choices, so the OpenRouter row was invisible. Its row block is now sized from choiceCount() instead of four hardcoded indices. - Because the root stage already counted five choices, the invisible row was still selectable, and confirming it hit `unreachable` in takePickerChoice. In ReleaseSafe that traps. Onboarding shows the connection rows on the root stage, so both stages now open key entry. - The Connections list was capped at the shared six-row picker budget, hiding the fifth entry behind a scroll. The hub screens hold a fixed, small set of rows, so they are now sized by what they contain; only the team and credential lists, which grow with the account, keep the cap. `fx provider` and `/logout` still advertised <gateway|codex|grok>, the same hand-maintained-list defect the earlier sweep found elsewhere. Verified end to end: seeded the keychain slot directly, confirmed `fx login openrouter` resolved it with no environment variable set, then removed it. Full suite shows no new failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The /model provider tabs are vendor buckets derived from the model id prefix, so an OpenRouter catalog showed no tab naming the catalog itself. A first attempt added `openrouter` as another vendor bucket matching an "openrouter/" id prefix. That never rendered: the live catalog carries 315 models across 37 vendor prefixes and none of them is "openrouter", so the tab could not match anything. The tab is not a vendor at all. It is now derived from the catalog source instead. It selects every model and is offered only while credentials.Source.openrouter_api_key is the authenticated source, which is the same field that already prints the "OpenRouter catalog: authenticated with an API key." status line. Sizing the tab by where the models came from rather than what is in them also means an empty or single-vendor OpenRouter catalog still shows it. It sits last among the named tabs, ahead of the Others catch-all. A test asserted the literal index 1 for the Anthropic tab; the enum order now decides that, so it reads the index from the enum. Also adds a test that typing filters an OpenRouter catalog, which had no coverage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An OpenRouter id already carries a vendor, so the list read "google/gemma-4-26b-a4b-it:free" with nothing saying which route serves it. Rows now read "openrouter/google/gemma-4-26b-a4b-it:free". This is display only. The id sent to OpenRouter, saved as the model preference, and matched against the catalog stays unprefixed, because OpenRouter does not know a model by that name. Display-only creates a trap, though: the prefix is on screen but was not in what the query matched, so typing what the row shows would find nothing. Matching now also considers the qualified name, so "openrouter/google" finds the row a user is looking straight at while the bare id keeps working. The prefix is not searchable on a Gateway catalog, where it is not shown. The qualified name is ellipsized as one string, so a narrow row trims the model id rather than the catalog that explains it. Verified against the live 314-model catalog: rows render qualified and "openrouter/goo" filters to the Google models. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reverts the model-list tab and the "openrouter/provider/model-name" display. The list is back to the model id as the provider publishes it, and the tab strip to its vendor buckets. Neither earned its place. The tab selected the whole catalog, so it did the same work as All whenever it was visible at all. The qualified name repeated on every row what the catalog status line already says once, and it had to be display only, which forced a matching rule so a query could still find the name it was looking at. Keeps two test changes the tab commit carried, both of which stand on their own: the provider-tab test reads its index from the enum rather than a literal, and an OpenRouter catalog now has coverage that typing filters it. Verified against the live catalog: rows read "inclusionai/ling-3.0-flash-fin:free", the strip reads [All] Anthropic OpenAI Others, and typing "gemma" filters 343 models to 7. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Saving a key from onboarding left the welcome copy on screen. Nothing said the key had taken, so the obvious response was to enter it again, which is what happened: two "Saved the API key" notices behind an unchanged screen. Every OAuth path calls closePicker on success. Key entry never did; it only restored the screen behind it, which for onboarding is the welcome screen itself. A saved key now ends the flow the same way a completed sign-in does. Outside onboarding the Gateway key still returns to Connections, so adding several credentials in one visit keeps working. An OpenRouter key goes further, matching Codex and Grok: it activates the provider through auth_transition rather than only being stored, since choosing that row is how a user asks for OpenRouter. Two related defects the same path carried: - The save recorded `.stored_key` as the credential preference whatever was saved, so storing an OpenRouter key told the Gateway route to prefer a stored Gateway key. The result now carries the source it reloaded as, and a provider-scoped key is not recorded as a Gateway preference at all, the rule the subscription sources already follow. - The notice said "Saved the API key" for either provider. It names which. The Connections row reported "environment" for the OpenRouter key. That was true when the key could only come from the environment; a stored key and an environment key now resolve to the same source, so the row cannot tell them apart and no longer claims one. Verified by driving the real flow: the row reads "configured", entry prompts for the OpenRouter key, and saving reports the key saved, switches the provider, and closes the picker. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Onboarding offered "Add an API key" and "OpenRouter API key". The first did not say which key it took, and the pair did not read as two of the same kind of choice. They are now "Add an AI Gateway API key" and "Add an OpenRouter API key". Onboarding is the one screen with no heading naming the provider, so the row has to carry it. The Connections list keeps its shorter labels, where the heading already supplies that context. Verified by rendering the welcome screen against an isolated profile with no credentials, which is the only state that shows it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The profile file is the whole key store on every non-macOS target, so the slot added beside the Gateway key needs the same 0600 guarantee the Gateway file already asserts. Both slots are now checked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rebasing onto upstream left the branch merging cleanly but not compiling. Upstream added exhaustive switches over CredentialSource and ProviderId in places this branch never touched, so the values it introduces were unhandled: requestedSource, sourcePresence, and the two credential-repair hints. That is the same hand-maintained-list defect this branch already swept for once; the compiler names every site, so each is answered rather than defaulted. Upstream also: - Guards Runtime.initInto with a field count, which the key-entry target field trips. The guard is doing its job, so initInto now sets it. - Widened activateProviderSelection with an exact credential source. Like Codex and Grok, OpenRouter pins none: its key is provider-scoped and must not become the Gateway's preferred source. - Replaced key probing with a presence check that reads store metadata and never the secret. The OpenRouter slot had no presence function, so it would have reported "unavailable" for a key that is present; presence is now slot-aware across both backends, and the OpenRouter probe no longer reads the key, which on macOS spawned a Keychain read per inventory refresh. The top-level help test asserted the old setup summary. `fx setup` takes a provider on this branch, so the assertion follows the command. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dsaad68
force-pushed
the
feat/openrouter-provider
branch
from
September 2, 2026 20:45
6e2a1bf to
63434f2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds OpenRouter as a model provider, alongside Vercel AI Gateway, Codex, and Grok.
All three existing routes need either Vercel billing or a paid consumer subscription, and every base-URL override is restricted to loopback HTTP, so there is no way to point fx at another endpoint. OpenRouter adds ~400 models behind a single API key, including a set that cost nothing to run — so a user with no Vercel billing and no ChatGPT or Grok subscription can run fx for free.
Design notes
A new wire format. OpenRouter speaks OpenAI Chat Completions, which neither existing protocol module covers (
vercel_protocol.zigis Vercel v3,responses_protocol.zigis the OpenAI Responses API).chat_completions_protocol.zigcarries no vendor identity so any future OpenAI-compatible route can share it. It accumulates streamed tool calls by theirindexfield and skips SSE comment lines, which OpenRouter emits as: OPENROUTER PROCESSINGkeep-alives mid-stream — feeding one to a JSON parser aborts an otherwise healthy stream.API key, no OAuth. Auth follows the existing
ai_gateway_api_keyshape viaOPENROUTER_API_KEY. There is no stored session, sofx logout openroutersays so rather than falling through to the Vercel logout.Tool-capable models only. fx calls tools every turn, so the catalog fetches with
?supported_parameters=tools; a model that cannot call tools breaks on the first step.Free models are surfaced deliberately. Identified by published pricing, sorted first, and shown four ways: a
Freefact in the/modelmenu, a marker infx models, afreefield in its JSON, and a--freefilter.is_freeis authoritative; the:freeid suffix is a display convenience for surfaces that only carry id strings, and a fixture test pins the two together.Exact usage. OpenRouter reports token counts and credit cost inline on the terminal chunk, so no deferred reconciliation is needed. The 402, 429, and 503 responses carry plain-language detail, since a negative balance blocks even free models and free-tier requests are capped at 20/min and 50/day.
Ordering stays provider-owned.
compareModelCatalogEntriesandprojectPickerModelCatalogencode Vercel-specific product policy and are gateway-only, so OpenRouter is deliberately not routed through them.Commits
ProviderIdplumbing./setupenumerated a hardcoded three providers, so it was unreachable from the TUI.fx status/doctor, the tools-disabled profile, and credential guidance.Where a hand-maintained list caused a miss, it was replaced with something the compiler or a test enforces:
isGatewaySource,missingCredentialMessage, and a test that walksstd.meta.tags(ProviderId)to assert every provider is reachable from the setup picker.Testing
tests/e2e/openrouter-stream.test.ts— catalog filtering, free-first ordering, the--freefilter, a streamed tool-call round trip, and the 402/429 paths. Classified incorpus.jsonas verification-only with a shard weight./setup→ Model provider and Connections,/model, and ACP config options driven in a real terminal.Known gap
FailureKindhas no payment-required variant, so a 402 maps toforbidden— correct non-retryable semantics, but it surfaces as "HTTP 403". The detail names the real status so the message cannot mislead. Happy to add apayment_requiredvariant instead if you'd prefer it fixed at the enum.Note on scope
This is a fork-driven contribution and I recognize OpenRouter is a model router, so it overlaps with AI Gateway in a way that Codex and Grok do not. Opening as a draft to check appetite before polishing further — happy to close if it is not a direction you want.
🤖 Generated with Claude Code