Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/docs/models/llm_providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ The `default_headers` option is available for OpenAI-compatible providers (inclu
| Moonshot | `moonshot` | `MOONSHOT_API_KEY` | Native Kimi Chat Completions API |
| Aliyun | `aliyun` | `ALIYUN_API_KEY` | Additional provider; DashScope compatible-mode endpoint |
| OpenRouter | `openrouter` | `OPENROUTER_API_KEY` | Additional provider; routed upstream models |
| OrcaRouter | `orcarouter` | `ORCAROUTER_API_KEY` | Additional provider; OpenAI-compatible model routing gateway |
| Hugging Face | `hf` or `huggingface` | `HF_TOKEN` | Inference Providers router and HF MCP auth |
| Open Responses | `openresponses` | `OPENRESPONSES_API_KEY` | Additional provider; interoperable Open Responses endpoints |
| Generic | `generic` | `GENERIC_API_KEY` | Additional provider; local/self-hosted OpenAI-compatible endpoints |
Expand All @@ -67,4 +68,4 @@ See [Additional Providers](providers/additional/) for the long-tail reference wi
- [Z.ai](providers/zai/) for native GLM models, reasoning streams, tools, and structured output.
- [Moonshot](providers/moonshot/) for native Kimi K3 reasoning, streaming, tools, structured output, and vision.
- [Hugging Face](providers/huggingface/) for Inference Providers routing, curated aliases, and HF MCP authentication.
- [Additional Providers](providers/additional/) for Groq, Aliyun, OpenRouter, Open Responses, TensorZero, and generic OpenAI-compatible endpoints.
- [Additional Providers](providers/additional/) for Groq, Aliyun, OpenRouter, OrcaRouter, Open Responses, TensorZero, and generic OpenAI-compatible endpoints.
22 changes: 20 additions & 2 deletions docs/docs/models/providers/additional.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
title: Additional Providers
social:
title: Additional Providers
tagline: Configure Groq, Aliyun, OpenRouter, TensorZero, and generic endpoints.
description: Configure Groq, Aliyun, OpenRouter, TensorZero, and generic endpoints.
tagline: Configure Groq, Aliyun, OpenRouter, OrcaRouter, TensorZero, and generic endpoints.
description: Configure Groq, Aliyun, OpenRouter, OrcaRouter, TensorZero, and generic endpoints.
alt: fast-agent social card — Additional model providers
---

Expand Down Expand Up @@ -36,6 +36,7 @@ Run `fast-agent check` after adding credentials to confirm they are visible to f
| Groq | `groq` | `GROQ_API_KEY` | `https://api.groq.com/openai/v1` | `groq.openai/gpt-oss-120b` |
| Aliyun | `aliyun` | `ALIYUN_API_KEY` | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | `qwen-turbo`, `aliyun.qwen3-max` |
| OpenRouter | `openrouter` | `OPENROUTER_API_KEY` | `https://openrouter.ai/api/v1` | `openrouter.google/gemini-2.5-pro-exp-03-25:free` |
| OrcaRouter | `orcarouter` | `ORCAROUTER_API_KEY` | `https://api.orcarouter.ai/v1` | `orcarouter.openai/gpt-4o-mini` |
| Open Responses | `openresponses` | `OPENRESPONSES_API_KEY` | Your Open Responses endpoint | `openresponses.openai/gpt-oss-120b:groq` |
| Generic OpenAI-compatible | `generic` | `GENERIC_API_KEY` | `http://localhost:11434/v1` for Ollama-style local use | `generic.llama3.2:latest` |
| TensorZero | `tensorzero` | None; configure provider credentials in the TensorZero Gateway | `http://localhost:3000` | `tensorzero.test_chat` |
Expand Down Expand Up @@ -94,6 +95,23 @@ fast-agent --model openrouter.google/gemini-2.5-pro-exp-03-25:free

OpenRouter routes requests to many upstream providers. Model names and capabilities are controlled by OpenRouter and the selected upstream model.

### OrcaRouter

```yaml
orcarouter:
api_key: "${ORCAROUTER_API_KEY}"
```

```bash
fast-agent --model orcarouter.openai/gpt-4o-mini
```

[OrcaRouter](https://www.orcarouter.ai) is an OpenAI-compatible model routing gateway. Model names are namespaced by upstream provider, for example `orcarouter.anthropic/claude-sonnet-4-6` or `orcarouter.google/gemini-2.5-flash`. The `orcarouter.auto` router picks the best upstream per request.

!!! note "Tool calling and the auto router"

The `orcarouter.auto` router may select an upstream that does not support tool calling. When fast-agent runs with tools (the default), prefer an explicit namespaced model such as `orcarouter.openai/gpt-4o-mini`.

## Open Responses endpoints

Open Responses is an open standard for interoperable LLM interfaces. Use the `openresponses` provider for compatible endpoints:
Expand Down
14 changes: 14 additions & 0 deletions docs/docs/models/providers/orcarouter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
title: OrcaRouter
social:
title: OrcaRouter
tagline: Configure OrcaRouter models, routing, and credentials in fast-agent.
description: Configure OrcaRouter models, routing, and credentials in fast-agent.
alt: fast-agent social card — OrcaRouter
---

# OrcaRouter

OrcaRouter is covered in the [Additional Providers](additional.md) reference.

See that page for the config key, API key environment variable, default endpoint, model string examples, and provider-specific notes.
5 changes: 5 additions & 0 deletions src/fast_agent/cli/commands/check_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,10 @@ class ProviderCatalogScope:
display_name="OpenRouter",
providers=(Provider.OPENROUTER,),
),
"orcarouter": ProviderCatalogScope(
display_name="OrcaRouter",
providers=(Provider.ORCAROUTER,),
),
}

_PROVIDER_CATALOG_SCOPE_ALIASES: dict[str, str] = {
Expand All @@ -147,6 +151,7 @@ class ProviderCatalogScope:
"xai",
"metaai",
"openrouter",
"orcarouter",
"responses",
"codexresponses",
)
Expand Down
25 changes: 25 additions & 0 deletions src/fast_agent/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -1556,6 +1556,28 @@ class OpenRouterSettings(BaseModel):
model_config = ConfigDict(extra="allow", arbitrary_types_allowed=True)


class OrcaRouterSettings(BaseModel):
"""Settings for using OrcaRouter models via its OpenAI-compatible API."""

api_key: str | None = Field(default=None, description="OrcaRouter API key")
base_url: str | None = Field(
default=None,
description="Override API endpoint (default: https://api.orcarouter.ai/v1)",
)
default_model: str | None = Field(
default=None,
description=(
"Default model when OrcaRouter provider is selected without an explicit model"
),
)
default_headers: dict[str, str] | None = Field(
default=None,
description="Custom headers for all API requests",
)

model_config = ConfigDict(extra="allow", arbitrary_types_allowed=True)


class AzureSettings(BaseModel):
"""Settings for using Azure OpenAI Service in the fast-agent application."""

Expand Down Expand Up @@ -2357,6 +2379,9 @@ def _migrate_legacy_mcp_settings(cls, values: Any) -> Any:
openrouter: OpenRouterSettings | None = None
"""Settings for using OpenRouter models in the fast-agent application"""

orcarouter: OrcaRouterSettings | None = None
"""Settings for using OrcaRouter models in the fast-agent application"""

generic: GenericSettings | None = None
"""Settings for using Generic models in the fast-agent application"""

Expand Down
1 change: 1 addition & 0 deletions src/fast_agent/llm/model_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ def __call__(self, **kwargs: Any) -> FastAgentLLMProtocol: ...
"MetaAIResponsesLLM",
),
Provider.OPENROUTER: ("fast_agent.llm.provider.openai.llm_openrouter", "OpenRouterLLM"),
Provider.ORCAROUTER: ("fast_agent.llm.provider.openai.llm_orcarouter", "OrcaRouterLLM"),
Provider.TENSORZERO: (
"fast_agent.llm.provider.openai.llm_tensorzero_openai",
"TensorZeroOpenAILLM",
Expand Down
14 changes: 14 additions & 0 deletions src/fast_agent/llm/model_selection.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,20 @@ class ModelSelectionCatalog:
Provider.ZAI: (_builtin_entry("zaiglm", display_label="GLM 5.2"),),
Provider.MOONSHOT: (_builtin_entry("kimik3", display_label="Kimi K3"),),
Provider.OPENROUTER: (),
Provider.ORCAROUTER: (
CatalogModelEntry(
alias="orcarouter-gpt4o-mini",
display_label="GPT-4o Mini (OrcaRouter)",
model="orcarouter.openai/gpt-4o-mini",
fast=True,
),
CatalogModelEntry(
alias="orcarouter-auto",
display_label="OrcaRouter Auto",
model="orcarouter.orcarouter/auto",
description="smart routing (best model per request)",
),
),
Provider.ALIYUN: (
_builtin_entry("qwen-turbo", fast=True),
_builtin_entry("qwen3-max"),
Expand Down
55 changes: 55 additions & 0 deletions src/fast_agent/llm/provider/openai/llm_orcarouter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import os

from fast_agent.llm.provider.openai.llm_openai_compatible import OpenAICompatibleLLM
from fast_agent.llm.provider_types import Provider
from fast_agent.types import RequestParams

ORCAROUTER_BASE_URL = "https://api.orcarouter.ai/v1"
# OrcaRouter is an OpenAI-compatible routing gateway; model ids are namespaced
# (e.g. "openai/gpt-4o-mini", "anthropic/claude-sonnet-4-6", "orcarouter/auto").
# The default is pinned to a fixed chat model rather than the "orcarouter/auto"
# router because fast-agent sends tool calls on every request and some upstreams
# in the auto pool do not support tool calling.
DEFAULT_ORCAROUTER_MODEL = "openai/gpt-4o-mini"


def _ensure_orcarouter_namespace(model_name: str) -> str:
"""Prefix a bare model name with the OrcaRouter namespace.

OrcaRouter routes on the full namespaced model id and rejects bare names
(e.g. "auto" -> 503). Only bare names are prefixed; names that already
contain a namespace (e.g. "openai/gpt-4o-mini") are passed through.
"""
if "/" in model_name:
return model_name
return f"orcarouter/{model_name}"


class OrcaRouterLLM(OpenAICompatibleLLM):
"""LLM provider for OrcaRouter, an OpenAI-compatible model routing gateway."""

def __init__(self, **kwargs) -> None:
kwargs.pop("provider", None)
super().__init__(provider=Provider.ORCAROUTER, **kwargs)

def _initialize_default_params(self, kwargs: dict) -> RequestParams:
"""Initialize OrcaRouter default parameters."""
base_params = self._initialize_default_params_with_model_fallback(
kwargs, DEFAULT_ORCAROUTER_MODEL
)
if base_params.model:
base_params.model = _ensure_orcarouter_namespace(base_params.model)
return base_params

def _provider_base_url(self) -> str:
"""Retrieve the OrcaRouter base URL from env/config or use the default."""
base_url = os.getenv("ORCAROUTER_BASE_URL", ORCAROUTER_BASE_URL)
config = self.context.config

# Check config file for override
if config and config.orcarouter:
config_base_url = config.orcarouter.base_url
if config_base_url:
base_url = config_base_url

return base_url
1 change: 1 addition & 0 deletions src/fast_agent/llm/provider_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ def config_name(self) -> str:
GOOGLE = ("google", "Google") # For Google GenAI native library
OPENAI = ("openai", "OpenAI")
OPENROUTER = ("openrouter", "OpenRouter")
ORCAROUTER = ("orcarouter", "OrcaRouter") # OpenAI-compatible model routing gateway
TENSORZERO = ("tensorzero", "TensorZero") # For TensorZero Gateway
AZURE = ("azure", "Azure") # Azure OpenAI Service
ALIYUN = ("aliyun", "Aliyun") # Aliyun Bailian OpenAI Service
Expand Down
1 change: 1 addition & 0 deletions src/fast_agent/ui/model_picker_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
Provider.BEDROCK,
Provider.ALIYUN,
Provider.OPENROUTER,
Provider.ORCAROUTER,
Provider.FAST_AGENT,
)

Expand Down
11 changes: 11 additions & 0 deletions tests/unit/fast_agent/commands/test_check_config_model_catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,17 @@ def test_show_provider_model_catalog_moonshot_includes_curated_model(capsys) ->
assert "moonshot.kimi-k3" in _collapse(output)


def test_show_provider_model_catalog_orcarouter_includes_curated_models(capsys) -> None:
show_provider_model_catalog("orcarouter")

output = capsys.readouterr().out
assert "OrcaRouter model catalog (curated)" in output
assert "orcarouter-gpt4o-mini" in output
assert "orcarouter-auto" in output
assert "orcarouter.openai/gpt-4o-mini" in _collapse(output)
assert "orcarouter.orcarouter/auto" in _collapse(output)


def test_show_models_overview_includes_provider_args_and_named_aliases(
tmp_path: Path,
capsys,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
OpenAISettings,
OpenResponsesSettings,
OpenRouterSettings,
OrcaRouterSettings,
Settings,
)
from fast_agent.constants import DEFAULT_MAX_ITERATIONS
Expand All @@ -34,6 +35,7 @@
from fast_agent.llm.provider.openai.llm_huggingface import HuggingFaceLLM
from fast_agent.llm.provider.openai.llm_openai import OpenAILLM
from fast_agent.llm.provider.openai.llm_openrouter import OpenRouterLLM
from fast_agent.llm.provider.openai.llm_orcarouter import OrcaRouterLLM
from fast_agent.llm.provider.openai.openresponses import OpenResponsesLLM
from fast_agent.llm.provider.openai.responses import ResponsesLLM
from fast_agent.llm.provider_types import Provider
Expand Down Expand Up @@ -157,6 +159,55 @@ def test_openrouter_provider_base_url_prefers_config_over_env() -> None:
os.environ["OPENROUTER_BASE_URL"] = original_env


def test_orcarouter_provider_default_model_used_when_model_missing() -> None:
llm = OrcaRouterLLM(context=Context(config=Settings()), model="")

assert llm.default_request_params.model == "openai/gpt-4o-mini"


def test_orcarouter_provider_config_default_model_used_when_model_missing() -> None:
settings = Settings(orcarouter=OrcaRouterSettings(default_model="anthropic/claude-sonnet-4-6"))
llm = OrcaRouterLLM(context=Context(config=settings), model="")

assert llm.default_request_params.model == "anthropic/claude-sonnet-4-6"


def test_orcarouter_provider_bare_model_gets_namespace() -> None:
llm = OrcaRouterLLM(context=Context(config=Settings()), model="auto")

assert llm.default_request_params.model == "orcarouter/auto"


def test_orcarouter_provider_namespaced_model_is_preserved() -> None:
llm = OrcaRouterLLM(
context=Context(config=Settings()),
model="google/gemini-2.5-flash",
)

assert llm.default_request_params.model == "google/gemini-2.5-flash"


def test_orcarouter_provider_base_url_prefers_config_over_env() -> None:
original_env = os.environ.get("ORCAROUTER_BASE_URL")
os.environ["ORCAROUTER_BASE_URL"] = "https://env-orcarouter.example/v1"
try:
settings = Settings(orcarouter=OrcaRouterSettings(base_url="https://config.example/v1"))
llm = OrcaRouterLLM(context=Context(config=settings), model="openai/gpt-4o-mini")

assert llm._base_url() == "https://config.example/v1"
finally:
if original_env is None:
os.environ.pop("ORCAROUTER_BASE_URL", None)
else:
os.environ["ORCAROUTER_BASE_URL"] = original_env


def test_orcarouter_provider_default_base_url_used_when_unset() -> None:
llm = OrcaRouterLLM(context=Context(config=Settings()), model="openai/gpt-4o-mini")

assert llm._base_url() == "https://api.orcarouter.ai/v1"


def test_huggingface_provider_default_model_used_with_provider_suffix() -> None:
settings = Settings(
hf=HuggingFaceSettings(
Expand Down
21 changes: 21 additions & 0 deletions tests/unit/fast_agent/llm/test_model_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,27 @@ def test_full_model_strings():
assert config.reasoning_effort == exp_effort


def test_orcarouter_dotted_model_string_parses_to_namespaced_model() -> None:
config = ModelFactory.parse_model_string("orcarouter.openai/gpt-4o-mini")

assert config.provider == Provider.ORCAROUTER
assert config.model_name == "openai/gpt-4o-mini"


def test_orcarouter_auto_dotted_model_string_parses_to_auto_router() -> None:
config = ModelFactory.parse_model_string("orcarouter.orcarouter/auto")

assert config.provider == Provider.ORCAROUTER
assert config.model_name == "orcarouter/auto"


def test_orcarouter_slash_model_string_parses_to_bare_auto_router() -> None:
config = ModelFactory.parse_model_string("orcarouter/auto")

assert config.provider == Provider.ORCAROUTER
assert config.model_name == "auto"


def test_deprecated_reasoning_suffix_is_rejected() -> None:
with pytest.raises(ModelConfigError, match=r"Use '\?reasoning=<value>' instead"):
ModelFactory.parse_model_string("openai.o1.high")
Expand Down
10 changes: 10 additions & 0 deletions tests/unit/fast_agent/llm/test_model_selection_catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,16 @@ def test_groq_current_aliases_drop_deprecated_kimi_entry() -> None:
assert "qwen3-32b" in aliases


def test_orcarouter_catalog_exposes_default_and_auto_router() -> None:
aliases = ModelSelectionCatalog.list_current_aliases(Provider.ORCAROUTER)

assert aliases == ["orcarouter-gpt4o-mini", "orcarouter-auto"]
assert ModelSelectionCatalog.list_current_models(Provider.ORCAROUTER) == [
"orcarouter.openai/gpt-4o-mini",
"orcarouter.orcarouter/auto",
]


@pytest.mark.parametrize(
"provider",
(
Expand Down
Loading