Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
15b3ec5
feat(plugins): add per-plugin Python virtual environments using UV
mcosgriff May 1, 2026
8aff72a
fix(plugins): improve admin UI and harden Docker builds
mcosgriff May 1, 2026
6a3e276
fix(plugins): simplify python packages UI and revert --no-build
mcosgriff May 1, 2026
23c377a
feat(admin): add collapsible Python dependency tree to packages tab
mcosgriff May 1, 2026
4ce1c03
feat(packages): add plugin venv selection for Python package installs
mcosgriff May 1, 2026
81a0800
refactor(packages): replace uv tree with uv pip list for package display
mcosgriff May 1, 2026
8093708
Merge remote-tracking branch 'origin/main' into uv-per-plugin-virtual…
mcosgriff May 4, 2026
c94f8b9
refactor(packages): clean up python package display format
mcosgriff May 4, 2026
76a7e78
feat(plugins): add UV migration tools for legacy plugins
mcosgriff May 4, 2026
e244422
Merge remote-tracking branch 'origin/main' into uv-per-plugin-virtual…
mcosgriff May 18, 2026
a31e5d7
Merge remote-tracking branch 'origin/main' into uv-per-plugin-virtual…
mcosgriff May 28, 2026
9ed23bb
test(uv-venv): add unit tests for per-plugin UV virtual environment f…
mcosgriff May 29, 2026
863b532
feat(admin-packages): add system, cached, and shared sections to Pyth…
mcosgriff May 29, 2026
91329bc
feat(admin-packages): seed UV cache from image and show as single sec…
mcosgriff May 29, 2026
4d3e38d
docs(uv-venv): document per-plugin virtual environments across user-f…
mcosgriff May 29, 2026
d894191
docs(uv-venv): add comments to per-plugin venv code for clarity
mcosgriff Jun 1, 2026
758c5df
feat(admin-packages): show uploaded wheels in cache and support per-p…
mcosgriff Jun 1, 2026
0b53344
Merge remote-tracking branch 'origin/main' into uv-per-plugin-virtual…
mcosgriff Jun 1, 2026
5196af8
fix(admin-packages): resolve polynomial regex backtracking in CodeQL
mcosgriff Jun 1, 2026
fdae503
chore(demo-plugin): upgrade urllib3 and document uv lock workflow
mcosgriff Jun 1, 2026
fd45fa1
fix(demo-plugin): upgrade idna to fix CVE bypass vulnerability
mcosgriff Jun 1, 2026
5dd191f
fix(uv-venv): resolve CodeQL and static analysis warnings
mcosgriff Jun 1, 2026
f30a687
fix(uv-venv): resolve remaining SonarCloud warnings
mcosgriff Jun 1, 2026
0eb3462
build(docker): upgrade uv from 0.11.8 to 0.11.17
mcosgriff Jun 1, 2026
3662d6a
test(uv-venv): add coverage for put, migrate_to_uv, and error paths
mcosgriff Jun 1, 2026
e4fa14e
docs(admin): add screenshots for per-plugin Python packages UI
mcosgriff Jun 1, 2026
bc1114a
feat(script-runner): add plugin venv selector for temp Python scripts
mcosgriff Jun 1, 2026
06bb1e8
feat(build): auto-detect core base images for enterprise builds
mcosgriff Jun 2, 2026
91e0d7b
fix(docker): resolve SonarLint warnings in buckets and redis Dockerfiles
mcosgriff Jun 2, 2026
d5ba005
Merge remote-tracking branch 'origin/main' into uv-per-plugin-virtual…
mcosgriff Jun 2, 2026
e96fe68
refactor(build): simplify enterprise core image build strategy
mcosgriff Jun 2, 2026
ebdaabb
docs(script-runner): fix formatting and example target name
mcosgriff Jun 2, 2026
bf4301c
test(uv-plugin-venvs): add coverage for per-plugin virtual environmen…
mcosgriff Jun 2, 2026
9b25d16
fix(docker): revert /data permissions in buckets UBI image
mcosgriff Jun 2, 2026
25bea17
style(test): prefix unused block parameters with underscores
mcosgriff Jun 2, 2026
3f34318
docs(cli): add migratetouv command documentation
mcosgriff Jun 2, 2026
b4d71b7
fix(playwright): use exact match for suite template menu item
mcosgriff Jun 2, 2026
3e69219
refactor(script-runner): rename pluginVenv to pythonVenv and fix builds
mcosgriff Jun 2, 2026
ff0701b
fix(docker): remove pre-created plugin_venvs from base image
mcosgriff Jun 2, 2026
341e30c
fix(docker): copy source before bundle install in API Dockerfiles
mcosgriff Jun 2, 2026
e761a3f
ci: retrigger CI on latest commit
mcosgriff Jun 2, 2026
3da00a6
feat(scripts): add destroy command to remove Docker images selectively
mcosgriff Jun 5, 2026
9b8ac2f
fix(docker): remove OPENC3_PATH build arg from API Dockerfiles
mcosgriff Jun 5, 2026
76c5846
Merge remote-tracking branch 'origin/main' into uv-per-plugin-virtual…
mcosgriff Jun 5, 2026
9145f90
feat(scripts): add list and status commands, expose runtime Python env
mcosgriff Jun 5, 2026
22a096c
fix(script-runner): sanitize python_venv parameter to prevent path tr…
mcosgriff Jun 5, 2026
2dbf8bc
fix(script-runner): use File.basename for python_venv path sanitization
mcosgriff Jun 8, 2026
ef2c275
Merge branch 'main' into uv-per-plugin-virtual-environments
mcosgriff Jun 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion _openc3
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,9 @@ _openc3_cli() {
'localinit:initialize local mode'
'initbuckets:initialize buckets'
'runmigrations:run database migrations'
'setpassword:set initial password from OPENC3_API_PASSWORD'
'migratepassword:migrate password hash to argon2'
'migratetouv:migrate plugin to per-plugin UV virtual environment'
)
_describe 'cli commands' cli_commands
return
Expand Down Expand Up @@ -494,6 +497,20 @@ _openc3_cli() {
fi
fi
;;
migratetouv)
if (( CURRENT == 2 )); then
_message 'PLUGIN_NAME'
elif (( CURRENT == 3 )); then
_message 'SCOPE (default: DEFAULT)'
fi
;;
setpassword|migratepassword)
if (( CURRENT == 2 )); then
local -a options
options=('--help:show help message' '-h:show help message')
_describe 'options' options
fi
;;
removebase|removeenterprise|localinit|initbuckets)
if (( CURRENT == 2 )); then
local -a options
Expand Down Expand Up @@ -523,6 +540,9 @@ _openc3() {
'cliroot:run a cli command as root user'
'start:build and run'
'stop:stop the containers'
'list:list Docker images for this installation'
'status:show container status'
'destroy:remove Docker images for this installation'
'cleanup:REMOVE volumes / data'
'build:build the containers'
'run:run the containers'
Expand All @@ -536,6 +556,10 @@ _openc3() {
;;
args)
case $line[1] in
destroy)
_values 'destroy options' \
'force[skip confirmation prompt]'
;;
cleanup)
_values 'cleanup options' \
'local[also cleanup local plugin files]' \
Expand Down Expand Up @@ -601,7 +625,7 @@ _openc3() {
words=("${save_words[@]}")
CURRENT=$save_current
;;
start|stop|build|run|start-ubi|build-ubi|run-ubi)
start|stop|list|status|build|run|start-ubi|build-ubi|run-ubi)
# These commands don't take arguments
;;
esac
Expand Down
2 changes: 2 additions & 0 deletions compose-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ services:
OPENC3_REGISTRY: ${OPENC3_REGISTRY}
OPENC3_NAMESPACE: ${OPENC3_NAMESPACE}
OPENC3_TAG: ${OPENC3_TAG}
OPENC3_PATH: /openc3
image: "${OPENC3_REGISTRY}/${OPENC3_NAMESPACE}/openc3-cosmos-cmd-tlm-api:${OPENC3_TAG}"
depends_on:
- "openc3-base"
Expand All @@ -97,6 +98,7 @@ services:
OPENC3_REGISTRY: ${OPENC3_REGISTRY}
OPENC3_NAMESPACE: ${OPENC3_NAMESPACE}
OPENC3_TAG: ${OPENC3_TAG}
OPENC3_PATH: /openc3
image: "${OPENC3_REGISTRY}/${OPENC3_NAMESPACE}/openc3-cosmos-script-runner-api:${OPENC3_TAG}"
depends_on:
- "openc3-base"
Expand Down
3 changes: 3 additions & 0 deletions compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ services:
RAILS_ENV: "production"
GEM_HOME: "/gems"
PYTHONUSERBASE: "/gems/python_packages"
OPENC3_USE_UV: "true"
ANYCABLE_REDIS_URL: "redis://${OPENC3_REDIS_USERNAME}:${OPENC3_REDIS_PASSWORD}@${OPENC3_REDIS_HOSTNAME}:${OPENC3_REDIS_PORT}"
env_file:
- .env
Expand Down Expand Up @@ -244,6 +245,7 @@ services:
- CI
- GEM_HOME=/gems
- PYTHONUSERBASE=/gems/python_packages
- OPENC3_USE_UV=true
env_file:
- .env
extra_hosts:
Expand Down Expand Up @@ -318,6 +320,7 @@ services:
- CI
- GEM_HOME=/gems
- PYTHONUSERBASE=/gems/python_packages
- OPENC3_USE_UV=true
env_file:
- .env

Expand Down
11 changes: 10 additions & 1 deletion docs.openc3.com/docs/configuration/_plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,16 @@ COSMOS uploads the plugin gem file to an internal gem server and extracts the ge

### Phase 2: Deploy

Once variables are set, COSMOS registers the plugin model in Redis, installs the Ruby gem, and if the plugin contains a `pyproject.toml` or `requirements.txt`, installs Python dependencies as well. It then parses `plugin.txt` again with [ERB](/docs/configuration/format#erb) variable substitution applied and deploys each component declared in the file: targets, interfaces, routers, microservices, tools, widgets, and script engines.
Once variables are set, COSMOS registers the plugin model in Redis and installs the Ruby gem. If the plugin contains Python dependencies (`pyproject.toml` or `requirements.txt`), COSMOS creates an isolated [UV](https://docs.astral.sh/uv/) virtual environment for the plugin at `/gems/plugin_venvs/<plugin>/.venv` and installs the dependencies into it. This gives each plugin full dependency isolation — different plugins can require different versions of the same package without conflicts.

COSMOS supports two formats for declaring Python dependencies:

- **`pyproject.toml`** (recommended) — When paired with a `uv.lock` file, enables reproducible installs via `uv sync --frozen`. COSMOS uses `uv sync` for `pyproject.toml`-based plugins.
- **`requirements.txt`** — COSMOS uses `uv pip install -r requirements.txt` for requirements-based plugins.

System Python packages (those shipped in the COSMOS Docker image) are pre-seeded into the UV download cache, so plugins that depend on those packages reuse them without re-downloading. If the UV install fails for any reason, COSMOS falls back to a shared pip install and logs a warning.

After installing dependencies, COSMOS parses `plugin.txt` again with [ERB](/docs/configuration/format#erb) variable substitution applied and deploys each component declared in the file: targets, interfaces, routers, microservices, tools, widgets, and script engines.

### Target Deployment

Expand Down
11 changes: 10 additions & 1 deletion docs.openc3.com/docs/configuration/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,16 @@ COSMOS uploads the plugin gem file to an internal gem server and extracts the ge

### Phase 2: Deploy

Once variables are set, COSMOS registers the plugin model in Redis, installs the Ruby gem, and if the plugin contains a `pyproject.toml` or `requirements.txt`, installs Python dependencies as well. It then parses `plugin.txt` again with [ERB](/docs/configuration/format#erb) variable substitution applied and deploys each component declared in the file: targets, interfaces, routers, microservices, tools, widgets, and script engines.
Once variables are set, COSMOS registers the plugin model in Redis and installs the Ruby gem. If the plugin contains Python dependencies (`pyproject.toml` or `requirements.txt`), COSMOS creates an isolated [UV](https://docs.astral.sh/uv/) virtual environment for the plugin at `/gems/plugin_venvs/<plugin>/.venv` and installs the dependencies into it. This gives each plugin full dependency isolation — different plugins can require different versions of the same package without conflicts.

COSMOS supports two formats for declaring Python dependencies:

- **`pyproject.toml`** (recommended) — When paired with a `uv.lock` file, enables reproducible installs via `uv sync --frozen`. COSMOS uses `uv sync` for `pyproject.toml`-based plugins.
- **`requirements.txt`** — COSMOS uses `uv pip install -r requirements.txt` for requirements-based plugins.

System Python packages (those shipped in the COSMOS Docker image) are pre-seeded into the UV download cache, so plugins that depend on those packages reuse them without re-downloading. If the UV install fails for any reason, COSMOS falls back to a shared pip install and logs a warning.

After installing dependencies, COSMOS parses `plugin.txt` again with [ERB](/docs/configuration/format#erb) variable substitution applied and deploys each component declared in the file: targets, interfaces, routers, microservices, tools, widgets, and script engines.

### Target Deployment

Expand Down
17 changes: 17 additions & 0 deletions docs.openc3.com/docs/development/microservices.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,23 @@ OpenC3::BackgroundMicroservice.run if __FILE__ == $0
</TabItem>
</Tabs>

## Python Runtime Environment

When a Python plugin microservice starts, the COSMOS operator automatically configures its environment to use the plugin's isolated UV virtual environment. The following environment variables are set:

| Variable | Value | Purpose |
| --- | --- | --- |
| `VIRTUAL_ENV` | `/gems/plugin_venvs/<plugin>/.venv` | Points to the plugin's isolated virtual environment |
| `PATH` | Prepended with the venv's `bin/` directory | Ensures the venv's Python binary is used |
| `PYTHONUSERBASE` | Same as `VIRTUAL_ENV` | Directs user-level installs to the venv |
| `PYTHONPATH` | `/openc3/python/.venv/lib/python3.X/site-packages` | Makes the core `openc3` package and system dependencies available |

This means plugin-specific dependencies (declared in `pyproject.toml` or `requirements.txt`) are installed in the per-plugin venv, while the core `openc3` library and system dependencies remain available from the base COSMOS Python environment. No manual configuration is needed — the operator handles this automatically when starting the microservice.

:::note
The `add_to_search_path` pattern for importing from a plugin's own `lib/` directory is still needed, as described in the section below. The per-plugin virtual environment handles third-party package isolation, not plugin-internal imports.
:::

## Importing Plugin Helpers from `lib/`

A plugin may have a top-level `lib/` directory holding shared helper files that
Expand Down
17 changes: 17 additions & 0 deletions docs.openc3.com/docs/getting-started/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Usage:
cli xtce_converter # Convert to and from the XTCE format. Run with --help for more info.
cli cstol_converter # Converts CSTOL files (.prc) to COSMOS. Run with --help for more info.
cli setpassword # Set the initial password from OPENC3_API_PASSWORD env var
cli migratetouv PLUGIN_NAME SCOPE # Migrate plugin to per-plugin UV virtual environment
```

:::note[seccomp profile]
Expand Down Expand Up @@ -371,6 +372,22 @@ Password set successfully.
The `OPENC3_API_PASSWORD` environment variable must be set (typically in your `.env` file). The password is read from this variable, not from a command line argument, to avoid exposing it in shell history.
:::

## Migrate to UV

Migrates an installed plugin from the shared Python virtual environment to a per-plugin UV virtual environment. This creates an isolated Python environment for the plugin with its own dependencies, preventing version conflicts between plugins.

The `PLUGIN_NAME` argument is the full installed plugin name (including the timestamp suffix), not a `.gem` file path. Use `cli list` to find the installed plugin name. The `SCOPE` argument is optional and defaults to `DEFAULT`.

```bash
% openc3.sh cli list
openc3-cosmos-demo-7.2.0.gem__20260101120000
openc3-cosmos-my-plugin-1.0.0.gem__20260501150000
% openc3.sh cli migratetouv openc3-cosmos-my-plugin-1.0.0.gem__20260501150000
Successfully migrated plugin 'openc3-cosmos-my-plugin-1.0.0.gem__20260501150000' to per-plugin UV venv
```

If the plugin has no Python dependencies (`pyproject.toml` or `requirements.txt`), the command reports success without creating a venv. If the plugin is already migrated, the command is a no-op.

## CSTOL Converter

Converts from the Colorado System Test and Operations Language (CSTOL) to a COSMOS Script Runner Ruby script. It currently does not support conversion to Python. Simply run it in the same directory as CSTOL files (\*.prc) and it will convert them all.
Expand Down
23 changes: 22 additions & 1 deletion docs.openc3.com/docs/getting-started/generators.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,28 @@ This creates the following files:
| plugin.txt | COSMOS specific file for Plugin creation. Learn more [here](../configuration/plugins). |
| Rakefile | Ruby Rakefile configured to support building the plugin by running "openc3.sh cli rake build VERSION=X.X.X" where X.X.X is the plugin version number |
| README.md | Markdown file used to document the plugin |
| requirements.txt | Python dependencies file (only for Python plugins) |
| requirements.txt | Python dependencies file (only for Python plugins). You can replace this with a `pyproject.toml` (recommended) and optionally a `uv.lock` for reproducible installs. See note below. |

:::note[Python Dependency Management]
Python plugins can declare dependencies using either `pyproject.toml` (recommended) or `requirements.txt`. When a plugin is installed, COSMOS creates an isolated UV virtual environment for it, so each plugin's dependencies are fully isolated from other plugins.

If you use `pyproject.toml`, you can include a `uv.lock` file alongside it to enable reproducible installs via `uv sync --frozen`. This ensures the exact same package versions are installed every time.

**Managing dependencies during development:**

```bash
# Generate or regenerate the lockfile after editing pyproject.toml
uv lock

# Upgrade a single package to its latest compatible version
uv lock --upgrade-package urllib3

# Upgrade all packages to their latest compatible versions
uv lock --upgrade
```

Always commit `uv.lock` alongside `pyproject.toml` so that production installs are reproducible.
:::

While this structure is required, it is not very useful by itself. The plugin generator just creates the framework for other generators to use.

Expand Down
6 changes: 6 additions & 0 deletions docs.openc3.com/docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,12 @@ git checkout vX.Y.Z # <- change to the specific version you want
<p style={{"margin-bottom": 20 + 'px'}}><code>./openc3.sh util load X.Y.Z # &lt;- update to match the save version</code></p>
{/* prettier-ignore */}
<p style={{"margin-bottom": 20 + 'px'}}>Note the version specified in save needs to match the version in load.</p>

#### Python Dependencies in Offline Environments

COSMOS pre-seeds the UV wheel cache with all system Python dependencies during first startup. This means plugins that only require packages already available in the cache can install without any network access.

To see which Python packages are available in the cache, check the **Cached** section under the **Admin > Packages** tab. If your plugin requires additional packages not in the cache, you can upload `.whl` files via the Admin > Packages tab before installing the plugin.
:::

### Certificates
Expand Down
28 changes: 28 additions & 0 deletions docs.openc3.com/docs/tools/admin.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ The Plugins tab is where you install new plugins into the COSMOS system. Plugins

The plugin gem name is listed along with all the targets it contains. You can Download, Edit, Upgrade, or Delete (uninstall) the plugin using the buttons to the right. If a plugin's target has been modified, the target name turns into a link which when clicked will download the changed files. New plugins are installed by clicking the top field.

Plugins that were installed before the UV per-plugin virtual environment feature was added show a **Migrate to UV** button. Clicking this button creates an isolated UV virtual environment for the plugin and reinstalls its Python dependencies, giving it the same dependency isolation as newly installed plugins.

### Targets

The Targets tab shows all the targets installed and what plugin they came from. Clicking the eyeball shows the raw JSON that makes up the target configuration.
Expand Down Expand Up @@ -45,8 +47,34 @@ The Microservices tab shows all the microservices installed, their update time,

The Packages tab shows all the Ruby gems and Python packages installed in the system. You can also install packages from this tab if you're in an offline (air gapped) environment where COSMOS can't pull dependencies from Rubygems or Pypi.

The Python packages section is organized into the following subsections:

- **Cached** — Python wheels available in the UV download cache. This includes system packages (seeded from the COSMOS Docker image at first startup) plus any packages downloaded during plugin installs. This section shows what's available for installation without network access, which is useful for planning plugin installs in airgapped environments. You can upload additional `.whl` files here to make them available to future plugin installs.
- **Plugin venvs** — Lists each installed plugin's isolated virtual environment and the packages installed in it. Each plugin gets its own venv at `/gems/plugin_venvs/<plugin>/.venv`, so different plugins can use different versions of the same package without conflicts.
- **Shared** (legacy) — Packages from the pre-UV shared install path. This section only appears when packages are present from plugins that were installed before the UV per-plugin virtual environment feature was added. Use the **Migrate to UV** button on the Plugins tab to migrate these plugins to isolated virtual environments.

![Packages](/img/admin/packages.png)

Expanding the **Cached** section shows all the Python wheels available in the UV download cache. The number in parentheses indicates the total count of cached packages:

![Cached Packages Expanded](/img/admin/packages_cache.png)

Each plugin has its own isolated virtual environment. Expanding a plugin section shows the packages installed in that plugin's venv. In this example, the demo plugin has only its direct dependency (numpy) installed:

![Plugin with numpy](/img/admin/packages_venv.png)

You can install additional packages into a plugin's venv from the Packages tab. After installing a package like `requests`, the plugin's venv shows the new package along with its transitive dependencies. Each package has a delete button to remove it from the venv:

![Plugin with requests added](/img/admin/packages_venv_added.png)

Expanding the plugin venv further shows the full list of installed packages with their versions:

![Plugin venv expanded](/img/admin/packages_venv_full.png)

When multiple plugins install different versions of the same package, the Cached section lists all versions. This is expected — each plugin's venv is isolated, so different plugins can use different versions without conflicts:

![Multiple versions in cache](/img/admin/packages_cache_versions.png)

### Tools

The Tools tab lists all the tools installed. You can reorder the tools in the Navigation bar by dragging and dropping the left side grab handle.
Expand Down
Loading
Loading