Skip to content

feat(llm): add OrcaRouter provider support - #927

Open
XiaoHuo888-hue wants to merge 1 commit into
evalstate:mainfrom
XiaoHuo888-hue:feat/orcarouter-provider
Open

feat(llm): add OrcaRouter provider support#927
XiaoHuo888-hue wants to merge 1 commit into
evalstate:mainfrom
XiaoHuo888-hue:feat/orcarouter-provider

Conversation

@XiaoHuo888-hue

Copy link
Copy Markdown

This registers OrcaRouter the same way the existing OpenRouter provider is wired, so the model picker, config reference and docs stay consistent. OrcaRouter is an OpenAI-compatible gateway: one ORCAROUTER_API_KEY (keys start with sk-orca-) unlocks 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax and xAI behind a single https://api.orcarouter.ai/v1 endpoint. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

I'm an engineer on the OrcaRouter team.

Changes

  • Provider.ORCAROUTER enum member, _PROVIDER_CLASS_PATHS factory mapping, OrcaRouterSettings, check_config catalog scope, and picker entry, mirroring the existing OpenRouter wiring
  • New OrcaRouterLLM provider class: defaults to openai/gpt-4o-mini and namespaces bare model ids (autoorcarouter/auto) so the gateway can route them
  • Docs: quick-reference row in llm_providers.md, an OrcaRouter section in the additional providers guide, and a provider stub page
  • Tests for default models, base URL precedence, model-string parsing, catalog entries, and check_config output

Verification

  • python scripts/lint.py — passed
  • python scripts/format.py --check — passed
  • python scripts/typecheck.py — no new diagnostics (only pre-existing diagnostics in unrelated files)
  • uv build — succeeded
  • Targeted unit tests: 275 passed; the full suite shows only the same 18 pre-existing failures that also occur on clean main (prompt_toolkit console, Windows file-permission, and websocket-timing tests)
  • L3 live test against https://api.orcarouter.ai/v1 with a real key: orcarouter.openai/gpt-4o-mini parsed to the OrcaRouter provider, resolved to OrcaRouterLLM, and the gateway returned the expected reply

Add a named OrcaRouter provider for fast-agent, an OpenAI-compatible
model routing gateway. Model ids are namespaced by upstream provider
(e.g. orcarouter.openai/gpt-4o-mini, orcarouter.orcarouter/auto).

- Provider enum member, factory mapping, config settings, picker entry,
  and check_config catalog scope
- New OrcaRouterLLM provider class defaulting to openai/gpt-4o-mini and
  namespacing bare model ids (orcarouter/auto) so the gateway can route
- Docs quick-reference row plus an OrcaRouter section in the additional
  providers guide, with a note about tool calling and the auto router
- Unit tests for default models, base URL precedence, model-string
  parsing, catalog entries, and check_config output

Co-Authored-By: Claude <noreply@anthropic.com>
@MohammedAlkindi

Copy link
Copy Markdown

Ran this rather than read it: Windows 11, CPython 3.14.7, ec68e2a against main at 610a2f5.

One thing worth settling before merge. _ensure_orcarouter_namespace() prefixes any bare model
name with orcarouter/, but by the docstring that namespace is for the router's own pseudo-models,
while real models keep their origin vendor. So the shortest thing a user is likely to type produces
an id I don't think the gateway can route:

what a user types, what gets sent
orcarouter.gpt-4o-mini        -> gpt-4o-mini        -> orcarouter/gpt-4o-mini
orcarouter.claude-sonnet-4-6  -> claude-sonnet-4-6  -> orcarouter/claude-sonnet-4-6
orcarouter.orcarouter/auto    -> orcarouter/auto    -> orcarouter/auto            (ok)

The tests cover auto, which is the one bare name where the prefix is correct. Intentional?

Smaller one: the picker ships orcarouter-auto while the comment eight lines above says the auto
pool is unreliable with always-on tool calls.

No regressions in the targeted subset. The failing set is identical, not just the count, and all
five reproduce on main without this branch. The +11 are your tests. ruff check and format are clean
at the pinned 0.16.0.

subset run, PR vs base, same machine and session
$ python -m pytest tests/unit/fast_agent/llm \
    tests/unit/fast_agent/commands/test_check_config_model_catalog.py \
    tests/unit/fast_agent/ui/test_model_picker_common.py -q

pr927   5 failed, 1596 passed
main    5 failed, 1585 passed

Key wiring needs no map entry, so leaving provider_key_manager.py untouched is right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants