You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ci(container): publish official Docker images to GHCR #379
FreeToken currently has no official container image published by the project.
Users who want to run ft serve in Docker must assemble their own CUDA, Python,
build-toolchain, and FreeToken environment. This makes deployments less
reproducible and leaves users without a project-maintained image tag to use in
automation.
I checked the Roadmap, FAQ, and existing issues before opening this request.
Proposed solution
Add a project-maintained CUDA development container image and a GitHub Actions
workflow that:
builds the image for pull requests and manual workflow runs without publishing;
publishes only from the upstream repository on pushes to main and version tags;
publishes to ghcr.io/flashml-org/freetoken;
provides latest for the default branch plus semantic-version and commit-SHA tags;
uses GitHub Actions cache for Buildx layers;
documents how to mount a model directory and run ft serve with Docker.
The image should retain nvcc, because FreeToken can compile CUDA kernels at
first use.
Alternatives considered
Users maintain local Dockerfiles: flexible, but duplicates setup work and
produces inconsistent CUDA and dependency combinations.
Publish only Python wheels: already useful, but does not provide a
reproducible deployment image.
Use a runtime-only CUDA image: smaller, but unsuitable when JIT compilation
of FreeToken CUDA kernels is needed.
Activity