feat: add LiteLLM API base and API key config#1043
Conversation
|
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60eb16dd0a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Add first-class LiteLLM embedding routing configuration for Basic Memory: - add semantic_embedding_api_base / BASIC_MEMORY_SEMANTIC_EMBEDDING_API_BASE - add semantic_embedding_api_key / BASIC_MEMORY_SEMANTIC_EMBEDDING_API_KEY - pass configured api_base and api_key through to LiteLLM embedding calls - preserve existing environment credential fallback by omitting api_key when unset - keep LiteLLM routing values out of persisted vector identity - separate process-local provider cache entries for explicit api_base/api_key using digests only - update LiteLLM and semantic-search documentation for local OpenAI-compatible endpoints Why: Basic Memory already supports LiteLLM embeddings, but it did not have a first-class way to point semantic embeddings at a custom OpenAI-compatible endpoint. That blocks local and self-hosted embedding deployments such as TEI, vLLM, Ollama-compatible proxies, LiteLLM proxy deployments, and private OpenAI-compatible embedding services. Some endpoints also require a placeholder or deployment-specific API key, so api_key needs to be configurable alongside api_base. The configured routing values are request-routing inputs, not vector identity fields. Changing only api_base/api_key does not automatically require a Basic Memory reindex, though operators still need to reindex if the endpoint serves semantically different embeddings under the same configured model and dimensions. Fixes basicmachines-co#1005. Signed-off-by: Dmitry Golubev <lastguru@gmail.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 222e99699a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
2 participants
Summary
semantic_embedding_api_base/BASIC_MEMORY_SEMANTIC_EMBEDDING_API_BASEfor LiteLLM embeddings.semantic_embedding_api_key/BASIC_MEMORY_SEMANTIC_EMBEDDING_API_KEYfor LiteLLM embeddings.api_baseandapi_keythrough to LiteLLM embedding calls.api_keywhen it is not configured.Why
Fixes #1005.
Basic Memory can already use LiteLLM for embeddings, but there was no first-class way to point it at a custom OpenAI-compatible embedding endpoint. This blocks local/self-hosted embedding setups that need an
api_base, such as TEI, vLLM, Ollama-compatible proxies, LiteLLM proxy deployments, or private OpenAI-compatible embedding services.Relying only on ambient environment variables is awkward for Basic Memory deployments because the semantic embedding endpoint belongs with the rest of Basic Memory config. Some OpenAI-compatible local endpoints still require a placeholder or deployment-specific API key, so
semantic_embedding_api_keyis needed alongsidesemantic_embedding_api_base. The configured key must not break existing setups that provide credentials throughOPENAI_API_KEYor provider-specific environment variables, so Basic Memory only passesapi_keywhen the config value is explicitly set.Design Notes
api_baseandapi_keyare request-routing inputs, not Basic Memory vector identity fields. They are intentionally not included in the persisted embedding model key. They are also not added to the provider cache key, because equivalent LiteLLM routing can be supplied through environment variables outside Basic Memory config.Changing provider, model, dimensions, dimension forwarding, or LiteLLM document/query role settings still changes vector identity and requires reindexing. Changing only
api_baseis not treated as a Basic Memory vector identity change. Operators remain responsible for reindexing if a different endpoint serves semantically different embeddings under the same configured model/dimensions.Testing
Both repository test and live test has been completed: the new configuration is applied and worked as expected.