Skip to content

docs: add use-with guide for Thunderbolt (MZLA's AI client) #487

Description

@njbrake

What

Add docs/use-with-thunderbolt.md showing how to point Thunderbolt at Otari, following docs/use-with-opencode.md (Thunderbolt speaks OpenAI-compatible /v1/chat/completions).

Thunderbolt is a self-hostable AI client (MPL-2.0) whose settings accept any OpenAI-compatible model provider, so Otari should work with no code change on either side.

Unlike the other use-with guides, this one needs the integration verified against a running Thunderbolt build before the page is written. See "Verify first" below; there is a known streaming risk that decides whether this is a docs task or a gateway bug.

What is already known (from reading their source, not yet run)

Recorded so whoever picks this up does not redo the recon. All paths are in thunderbird/thunderbolt at main:

  • Base URL handling: src/lib/openai-base-url.ts exports normalizeOpenAiBaseUrl, which strips trailing slashes and appends /v1 when absent. So http://localhost:8000 and http://localhost:8000/v1 should both work. Worth confirming and documenting, since it is the opposite of the base-URL footgun in some other clients.
  • Transport dispatch: resolveOpenAiCompatConnection in src/ai/fetch.ts sends loopback addresses direct from the app and routes everything else through their backend proxy. This is the most likely source of surprises: a local Otari is a direct fetch (CORS applies), while a remote Otari needs their backend to have network reach to us.
  • Catalog and connection test: their settings UI has a "Load Models" fetch and a Test Connection step (src/settings/models/). model-policy.ts requires a connection test for every provider except their own thunderbolt provider, so ours will go through it. Both paths hit our GET /v1/models, so our response shape needs to satisfy their catalog loader.
  • Their server-side inference path (backend/src/inference/) is separate from the custom-provider path: a hardcoded map of five models over three providers (fireworks, mistral, anthropic), each with a literal baseURL and an env-var key. Not what this guide targets, but relevant to the follow-ups below.

Verify first

Do this before writing any prose. If it passes, the guide is straightforward. If it fails, close this as a docs issue and open a gateway bug instead.

  1. Run Otari locally, run Thunderbolt from source, add Otari as a custom OpenAI-compatible provider.
  2. Confirm Test Connection passes and Load Models populates from our GET /v1/models.
  3. Send a chat turn and confirm streaming renders correctly.
  4. Confirm reasoning content round-trips. Their issue #8 documents reasoning chunks not reaching their UI: OpenRouter sends reasoning chunks while Fireworks sends <think> tags, and their @ai-sdk/openai-compatible path mishandled one of them. Any gateway sitting in front of multiple providers can hit this. Test with a reasoning model and check whether our stream matches what their AI SDK path expects.
  5. Try both a loopback and a non-loopback Otari address, since those take different transports (see above).

Steps

  1. Copy docs/use-with-opencode.md to docs/use-with-thunderbolt.md.
  2. Fill in each section:
    • Intro: Thunderbolt speaks OpenAI-compatible chat completions; name the Otari endpoints it hits (/v1/models, /v1/chat/completions)
    • Quick start: cover both Connected to otari.ai and Standalone, with the settings fields Thunderbolt actually asks for (base URL, API key, model)
    • Choosing a model: the selector format Thunderbolt passes through, and how it interacts with our provider:model form and with their Load Models catalog
    • Gotchas: at minimum the loopback-vs-proxy transport split and the /v1 auto-append; add whatever else verification turns up
    • See also: link to docs/quickstart.md and provider setup
  3. Do not re-explain provider setup. Link to it.
  4. Link the new page from docs/index.md.

Acceptance criteria

  • Integration verified end to end against a real Thunderbolt build, including reasoning/streaming behavior
  • New file at docs/use-with-thunderbolt.md following the opencode exemplar
  • Both "Connected to otari.ai" and "Standalone" variants covered
  • Model selector format explained with concrete examples
  • Loopback-vs-proxy transport gotcha documented
  • Linked from docs/index.md
  • Any gateway-side bug found during verification filed separately and linked here

Out of scope (recorded for later, do not scope into this issue)

Reading their source surfaced larger opportunities. Each is bigger than a docs change and needs its own issue:

  1. Otari behind their backend/src/inference proxy. Would replace a hardcoded three-provider client factory with our provider matrix and give them usage and budget tracking, letting them delete provider code they currently maintain.
  2. Central key and budget management for multi-user deployments. src/dal/models.ts documents their model API keys as living in a local-only secrets table, so each user pastes a provider key on each device: no central issuance, revocation, budget ceiling, or per-user spend visibility. We already have keys, users, budgets, usage, and pricing. Their model-policy.ts already treats the thunderbolt provider as server-authenticated and preconfigured, which is a pattern an otari provider kind could follow.
  3. Anthropic-shaped traffic. They call api.anthropic.com/v1 directly; our /v1/messages endpoint could front that path too.
  4. Their CLI. cli/ has openai-compat-model.ts and an ACP harness, so a CLI variant of this guide is close to free once the client guide exists.

Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/apiAPI design and behaviordocumentationImprovements or additions to documentationgood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions