Repository navigation
Enable Decision Model Support in Lemonade (via mainline llamacpp) #3745
sawansri
started this conversation in
Request for Comment (RFC)
Replies: 1 comment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Proposal size
Minor feature (a single PR)
Updates
No response
User story
As the hype behind decision models like Clef, OpenJev, Laya, etc. grows, Lemonade users will naturally want to try these models. As of ggml-org/llama.cpp#29818, mainline llamacpp has merged initial support for this category of models.
High-level design
The llamacpp PR adds a new /v1/systemone API endpoint as well as support for 5 decision models: laya, julia-1, lev, openjev, and kev. Lemonade will mirror this change by enabling the systemone endpoint and adding the initially supported decision models to the model registry.
Breaking changes
None
Maintenance plan
Correctness, model support, etc are tested upstream by mainline llamacpp. Since Lemonade would only forward /v1/systemone requests to llama-server, maintenance would be limited to the route integration and llamacpp version bump tests.
Risks
No response
All reactions