Archive: RFC: Backend Sandbox Architecture and Permission Policy (nono)

[abn]

Proposal size: Major feature (spans multiple well-scoped PRs).
Scope: process sandboxing for every engine backend, built-in or external. This RFC is standalone and does not depend on any particular backend manifest format.

Every engine process, built-in or external, runs confined to least privilege: no ambient secrets, no outbound network by default, and only the files and devices it was explicitly granted. Each backend declares its grants through its own configuration surface, and this RFC defines what those grants mean and how they are enforced. PR #3212 is the reference implementation; this RFC defines the contract and the deltas that expand it, not the code.

1. Why this exists

An inference engine is a large attack surface, whether it ships in-tree or comes from elsewhere. The built-ins wrap third-party projects (llama.cpp, whisper.cpp, stable-diffusion.cpp, vLLM, vendor NPU runtimes), and external backends wrap other people's binaries and containers outright. Give any of them the server's own privileges and it can read credentials, reach the network, and write anywhere the server can. One compromised engine then takes the whole host.

Manual review of every engine is not the alternative. The answer is a declarative policy that starts from nothing and adds only what a backend needs, enforced by the operating system, with the difference between declared and enforced made visible.

2. User stories

As an operator running any engine, in-tree or third-party, I want every backend process confined to least privilege (no ambient secrets, no egress, only the files and devices it was granted), so a compromised or malicious engine cannot exfiltrate credentials or damage the host.

As a backend developer, I want sandboxing by default from a declarative policy, so security is a baseline property rather than something each backend reimplements.

As a security-conscious user, I want to see the enforcement state and to have a hard mode that refuses to run when confinement is unavailable, so I can run untrusted engines with confidence or not at all.

3. High-level design

The sandbox wraps the outermost untrusted process. When a backend launches an engine directly, that is the engine itself. When a future out-of-process adapter hosts the engine, it is the adapter. Being in-tree buys packaging and support; it does not buy trust. Built-in engines are sandboxed too.

Containerized recipes are the one exception, and only to the wrapping rule. The container runtime already provides the isolation, so Lemonade does not wrap it a second time. The policy still applies: the declared grants are reviewed at consent, and the enforcement state reported comes from the runtime, not as a claim of kernel confinement.

Policy is assembled additively, never subtractively. It starts at deny-all (0 paths, 0 devices, 0 environment variables, loopback only) and adds, in order: the system runtime, the hardware profile, the workload assets, and finally the backend's declared grant delta (a sandbox block for a manifest backend, a code-declared hook for a built-in). Nothing is inherited by default. Engine-specific caches have to be declared; none are ambient.

4. The grant block (normative)

This section describes how a backend declares the engine-specific grants it needs beyond its hardware profile. A backend that ships as a manifest (an external backend) carries one sandbox object in JSON. A built-in has no such file: its grants are the shared presets plus a code-declared sandbox hook on its server class (the reference calls it customize_sandbox_policy), which sits beside the code that builds its launch plan and is reviewed in-tree. Both forms are validated and merged into one policy object by the assembly in section 5. Every field is optional, so an absent field grants nothing.

{
  "sandbox": {
    "read_paths":  ["{model_dir}", "{exe_dir}"],
    "write_paths": ["{scratch_dir}"],
    "devices":     ["/dev/dri", "/dev/kfd"],
    "network":     { "egress": false },
    "env_allowlist": ["HF_HOME"]
  }
}

4.1 read_paths

Absolute directories or token-resolved paths the engine may read. Model read grants are scoped to the model snapshot subtree, not the cache root that contains it. A read grant that resolves outside an allowed root, or into a credential root, is rejected fatally at policy validation; it is not narrowed silently.

4.2 write_paths

Absolute directories the engine may write: scratch space, logs, and declared caches. A read grant does not imply a write grant. A backend that needs to write KV caches or compiled artifacts declares where.

4.3 devices

Device nodes the engine may open, for example /dev/dri for a GPU, /dev/kfd for ROCm, /dev/accel for an NPU. Device grants are selected per hardware profile (section 5), so each recipe does not have to guess them. A recipe may narrow the profile's set; it may not widen it.

4.4 network

network is an object, not a boolean. {"egress": false} is the default and means loopback only. Outbound egress requires {"egress": true, "hosts": ["huggingface.co", "cdn-lfs.huggingface.co"]}. The host list is required when egress is enabled; a bare egress: true is rejected. Egress is a host allowlist, not a bare bool.

Egress is a strong grant, so it is treated as one: a backend that self-manages its model downloads declares the hosts it reaches, and that declaration is not inferred from self-management. A backend with no local model does not earn egress on the strength of needing to fetch one.

4.5 env_allowlist

Names of parent environment variables the child may inherit. The default is empty. PATH and loader paths come from the system runtime profile, not from this list. Cloud and VCS secret names are never inheritable, even if named here.

4.6 What is never grantable

Credential roots are non-grantable and fatal at validation: the Hugging Face and ModelScope token files, ~/.aws, ~/.ssh, and the LEMONADE_* environment namespace. A backend that names one is rejected with an error that says why, instead of being quietly trimmed to nothing. The goal is least privilege an operator can reason about, so the boundary is stated rather than implied.

5. Policy assembly

The effective policy is computed in a fixed order so no step can be skipped by a backend declaration:

1. Deny-all baseline. Zero paths, zero devices, zero environment, loopback only.
2. System runtime. The dynamic loader paths and the base execution environment the engine needs to start at all. Every engine gets the same set.
3. Hardware profile. The device nodes and driver paths for the detected accelerator: ROCm, CUDA, DirectML, XRT, or Vulkan. The profile is keyed by the same accelerator selection the platform matrix uses, so the grant and the launch agree.
4. Workload assets. The backend executable directory and the model snapshot subtree the engine was handed.
5. Backend delta. The backend's declared grants, applied last and able only to narrow what the profile allows, except for network and env_allowlist, which are additive grants. For a manifest backend this is its sandbox object; for a built-in it is the code-declared hook.

5.1 Built-in backends and manifest backends

The assembly is identical for both. They differ only in where the last step's delta comes from.

• A built-in backend has no JSON grant block. A shared builder applies the three presets, and the server class's sandbox hook adds the engine-specific needs: extra read paths (a draft model, a projector, a VAE), write paths for its caches (vLLM's Triton and MIOpen caches, a Mesa shader cache), and allowed environment names. The hook can narrow the profile's device set but cannot widen it, and it is reviewed with the code.
• A manifest backend (an external backend) adds a declarative sandbox object that is validated against the schema and shown at consent. Once assembled it is the same policy object as a built-in's.
• A container backend delegates isolation to the runtime. Its assembled policy is reported as runtime-owned, and consent still gates the declared mounts and devices.

Reporting does not distinguish how a grant was declared, only what is enforced (section 10). A built-in and an external backend that need the same access end up with the same effective policy, and an operator reading the sandbox status sees one shape.

Engine-specific caches (a Triton cache, a compiled-kernel directory) are declared through this block or a per-backend hook; they are not inherited from the environment.

6. Enforcement backends

One policy, several enforcement mechanisms, each reporting what it actually does.

6.1 Linux

Landlock and seccomp are applied in-child with no-new-privileges set. The default-deny outbound rule rests on a seccomp socket-family filter that denies non-Unix sockets. It holds on every kernel regardless of the Landlock ABI level, so egress default-deny does not depend on the kernel version.

Where Landlock network rules exist (ABI v4 and above, kernel 6.7 and above), the socket filter can be lifted to allow the declared outbound TCP. Those rules are port-scoped, not address-scoped, and TCP only, so a permitted port reaches any host and UDP egress stays denied. Address-scoped outbound, which is what a host allowlist requires, is enforced by the supervised proxy in section 9.

On kernels below 6.7 the seccomp filter and filesystem confinement remain, and the absence of address-scoped egress is surfaced, not silent.

6.2 macOS

Apple Seatbelt (SBPL compile and sandbox_init) is applied through the lemonade-sandbox-exec trampoline. Without the trampoline, macOS runs scrubbed-only and reports that state. Seatbelt is a deprecated Apple API: it works today but is unsupported upstream, so this path is revalidated each release and documented to fall back to filesystem-only confinement if the API is removed. Per-port egress granularity is coarser than Linux.

6.3 Windows

AppContainer plus transient NTFS ACEs plus anonymous Job Objects with KILL_ON_JOB_CLOSE. Native Windows egress is all-or-nothing through the AppContainer network capability, with no per-host or per-port filtering, so the reported state reflects what is enforced rather than what is declared. Abnormal termination can leave transient ACEs; they are RAII-managed and the residual risk is documented.

6.4 WSL2 and containers

WSL2 delegates to the nono and Landlock path. Containers are not wrapped again: the runtime is the isolation boundary, and the container recipe's grants are consent-gated at the Lemonade boundary. The sandbox status for a container backend says so, and does not claim kernel confinement Lemonade did not apply.

6.5 Stub builds

A C++ stub build (LEMON_USE_NONO_RUST=OFF) preserves environment scrubbing without a Rust toolchain. Distro builders without a current rustc (1.95 and above) land on the same stub. A stub build ships without kernel confinement, so that absence must be visible in /system-info and in lemonade backends sandbox status, never assumed away.

7. Enforcement modes

The mode governs kernel confinement only. Environment scrubbing is the invariant floor and does not vary with the mode, except in disabled.

• auto (default). Kernel confinement applies where the platform supports it and degrades gracefully otherwise. Scrubbing is always on. Degradation is reported.
• enforced. Kernel confinement is required; a host that cannot enforce it fails the load rather than degrading. Scrubbing is always on. This is the mode for high-security deployments and for untrusted engines.
• scrubbed_only. Kernel confinement is off and scrubbing stays on. Use when confinement breaks an engine but secret isolation is still wanted.
• disabled. Pre-sandbox behavior: no kernel confinement and no scrubbing, for debugging or engines that need the ambient environment. This is the only mode that drops the scrubbing floor, so it is surfaced in /system-info, never silent.

Configuration accepts exactly these four names through config.json, the CLI, or an environment override. Nothing else is a mode.

8. Environment scrubbing

Children start from a clean environment. Only an explicit allowlist passes through: the system runtime paths, and the recipe's env_allowlist. LEMONADE_* keys, cloud and VCS secrets, and any secret-shaped name are never inherited. Scrubbing is the invariant floor because it is the cheapest and most portable protection, and it holds even where kernel confinement does not.

One documented exception is live model fetching from Hugging Face, which needs the proxy and CA variables the scrubber would otherwise drop. That exception is explicit (via the hf_load path), not ambient.

9. Egress and the supervised proxy

The host allowlist is enforced by a supervised proxy: the child's outbound TCP is routed through a Lemonade-managed proxy that permits only the declared hosts, and everything else is denied. This is what makes address-scoped outbound possible where Landlock's port-scoped rules cannot express it. The seccomp socket filter remains underneath as the floor on every kernel.

The limits are stated, not hidden: on Linux, allowed ports reach any host unless the proxy is in front of them; UDP egress stays denied; on native Windows egress is all-or-nothing. A recipe that under-declares fails fast with an error naming the grant to add.

10. Configuration and visibility

Enforcement state and mode are exposed in /system-info and in lemonade backends sandbox status. The report distinguishes declared from enforced, per host, because the two differ: a stub build has no kernel confinement, a container backend relies on the runtime, and native Windows cannot filter egress by host. The consent surface shows the enforced state, not the requested one.

11. Consent and widening

Grants are reviewed in the install-time consent flow, authenticated with the admin API key. Widening any of read_paths, write_paths, devices, network, or env_allowlist re-triggers consent, so a backend cannot quietly acquire a new capability after it was trusted once. Narrowing does not.

12. Worked examples

The first three examples are declarations: the sandbox object a manifest backend carries. The fourth is a built-in, which has no JSON block and is assembled from the shared profiles plus a code-declared hook.

A passthrough engine on a GPU host: read the model snapshot and its own directory, write scratch, open the GPU nodes, loopback only, no inherited environment.

{
  "sandbox": {
    "read_paths":  ["{model_dir}", "{exe_dir}"],
    "write_paths": ["{scratch_dir}"],
    "devices":     ["/dev/dri", "/dev/kfd"],
    "network":     { "egress": false },
    "env_allowlist": []
  }
}

A backend that fetches its own weights: it needs egress to the declared hosts, a cache write grant, and the proxy and CA variables.

{
  "sandbox": {
    "read_paths":  ["{model_dir}", "{exe_dir}"],
    "write_paths": ["{scratch_dir}", "{hf_cache}"],
    "devices":     ["/dev/dri", "/dev/kfd"],
    "network":     { "egress": true, "hosts": ["huggingface.co", "cdn-lfs.huggingface.co"] },
    "env_allowlist": ["HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "SSL_CERT_FILE", "SSL_CERT_DIR"]
  }
}

A containerized recipe: the grant block states that the runtime provides isolation, so Lemonade does not wrap it a second time and the consent surface says so.

{
  "sandbox": { "note": "exempt_container_backend" }
}

A built-in, for contrast. An in-tree engine such as llama.cpp on a ROCm host carries no sandbox file. Its effective policy is assembled: the system runtime loader paths; the ROCm hardware profile device nodes and driver paths; the backend executable directory and the model snapshot subtree as read assets; scratch plus any engine cache the server class declares in code as write assets; the profile's devices; loopback only; and no ambient environment. Assembled, it is the same shape as the JSON above, and the only part a backend author writes is the engine-specific cache, reviewed with the code.

13. Scenarios

A community engine that turns out to be hostile. You installed an out-of-tree llama.cpp fork. It tries to read your cloud credentials and phone home. The read grant does not include the credential root and egress defaults to loopback, so the kernel denies both attempts and the engine fails rather than exfiltrating.

A self-managed engine that streams weights. The engine fetches from Hugging Face on first use. It declares the two hosts it reaches and the proxy and CA variables it needs. The proxy allows those hosts and denies the rest, and scrubbing still withholds every secret that was not named.

A high-security deployment. enforced is set. On a host that cannot apply kernel confinement, the load fails instead of degrading, and /system-info says why. Nothing runs unsandboxed without an explicit, visible opt-in.

14. Reference implementation and required deltas

PR #3212 is the reference implementation: the in-tree nono C FFI integration (Landlock and Seatbelt engines, the C++ stub, the policy structs, environment scrubbing, and the mode enum). It already sandboxes built-ins: a shared builder applies the three presets, each backend overrides a sandbox hook for its caches and environment, and the result is passed to the process spawn. This RFC does not implement anything. It defines the contract, and the deltas below are what expand the reference into it:

• Egress host allowlist. The reference exposes a three-way network enum (deny all, loopback only, full). This RFC requires egress as a host allowlist enforced through the supervised proxy; full is not a mode a backend may request.
• Config surface. The reference still accepts learn, profile, and audit as modes. This RFC accepts only auto, enforced, scrubbed_only, and disabled; the extra names are removed rather than documented.
• Named credential roots. The reference rejects broad roots (the home root, /etc, /root, /boot, C:\Users) and forbidden environment patterns. This RFC requires the specific credential roots (the Hugging Face and ModelScope token files, ~/.aws, ~/.ssh) to be non-grantable and fatal by name.
• Fail closed on a missing policy. The spawn path checks whether a policy was supplied and runs the child unconfined and unscrubbed when it was not, so coverage rests on every backend remembering to build one. This RFC requires the spawn path itself to apply the configured default, and to fail in enforced, so a backend cannot run unconfined by omission.
• Explicit, never silent. The reference was silent in several places (a stub build that did not surface, sub-6.7 degradation that was not logged, an earlier macOS state that reported Seatbelt as enforced, and a Windows ACE lifetime bug). This RFC makes surfacing a requirement: a degraded or absent state is always reported.
• hf_load environment. The scrubber drops the proxy, CA, and Hugging Face cache variables, so live downloads need them allowlisted in addition to the egress and write grants.
• Enforcement reflects reality. The reported state distinguishes declared from enforced, including stub builds, containers, and native Windows.

15. Maintenance plan

CI. Dedicated sandbox suites under the cpp-ci target: policy assembly, engine lifecycle, environment scrubbing and confinement, process egress, wrapped-server confinement, and Windows. Distro and backend CI labels exercise packaging with and without the Rust path, and assert that stub builds surface the absence of kernel confinement.

Human. Real-hardware validation of GPU and NPU device-node grants on AMD, NVIDIA, and Windows targets through the existing self-hosted runners, plus periodic macOS trampoline validation. Watchdog and teardown edge cases (zombie processes, Job Object lifecycle) are covered by integration tests and targeted manual runs.

Upstream. Policy engines are maintained by nono (Landlock and Seatbelt) and in-tree for Windows. nono is pinned (0.73) and vendored for hermetic, air-gapped builds. Version bumps are deliberate, reviewable changes with CI revalidation. The macOS path is revalidated each release against the deprecated Seatbelt API.

16. Risks

• False confidence. Sandboxing is defense in depth, not proof. A loader-level exploit in an engine binary can still run transient code. Confinement bounds the blast radius; it does not replace consent, pinning, or scrubbing.
• Kernel and platform variance. Mode awareness and telemetry mitigate this. Degraded enforcement is explicit, never silent.
• Supply chain. nono is pinned and vendored. Responding to a CVE means a deliberate bump plus revalidation. The stub build must not silently disable scrubbing.
• Egress and consent churn. A recipe that under-declares fails fast with an error that names the grant to add. Grant authoring stays deliberate; a recording mode is not included, because kernel denials are not observable.
• Landlock network limits. Allowed ports reach any host, UDP egress stays denied, and address-scoped outbound needs the supervised proxy. The residual risk is documented, not papered over.
• Performance. Kernel enforcement adds per-spawn ruleset construction, measured negligible against inference time in prototyping.

17. Breaking changes

• Backends that rely on the ambient environment or arbitrary egress change behavior under the default auto on capable hosts. Live model streaming from Hugging Face requires an explicit egress host grant, a cache write grant, and pass-through of the proxy and CA variables, as documented.
• The server binary grows roughly 12 to 20 MB when the in-tree nono library is linked. Release-profile optimizations recover about 8 MB, and the stub build keeps distro builds lean. A stub build ships without kernel confinement, and that absence is visible in /system-info, never silent.
• Platform caveats: kernels below 6.7 keep egress default-deny through the seccomp filter but lose address-scoped outbound; macOS egress is coarser and Seatbelt is deprecated; native Windows egress is all-or-nothing; abnormal Windows termination may leave transient AppContainer ACEs.
• Mitigation posture: auto degrades gracefully, and enforced is opt-in for high-security deployments. Nothing fails closed silently.

18. Non-goals

• Proving an engine is safe. This bounds the blast radius of an untrusted engine; it is not a substitute for consent.
• Sandboxing the user's own shell or unrelated processes. The policy applies to engine subprocesses.
• A general container runtime manager. Container isolation stays with the runtime.
• Recording or replaying kernel denials. They are not observable, so recipes declare grants rather than learning them.
• An in-process plugin sandbox. In-process plugins are out of scope entirely.

19. Deferred items

• Landlock network rules are port-scoped and TCP only; address-scoped outbound depends on the supervised proxy, and UDP egress stays denied.
• macOS per-port egress granularity is coarser than Linux, and the Seatbelt API is deprecated; the path may fall back to filesystem-only confinement.
• Native Windows egress is all-or-nothing, with no per-host or per-port filtering.
• The learn profiling surface is removed from the config in this RFC; a capability-profiling workflow, if wanted, needs its own design.
• Per-backend cache declarations for engines such as vLLM Triton remain a hook to be specified as engines need them.

20. Design decisions

#	Decision	Why
D1	Sandbox every engine, built-in or external.	Being in-tree buys support, not trust.
D2	Policy is additive from deny-all.	Least privilege is the default, not a per-recipe achievement.
D3	Egress is a host allowlist, never a bare bool.	A boolean cannot express where a backend may reach, and self-management is not a reason to grant the whole network.
D4	Environment scrubbing is the invariant floor.	It is cheap, portable, and holds where kernel confinement does not.
D5	enforced fails closed; auto degrades and reports.	High-security deployments need a hard mode; everyone else needs no silent breakage.
D6	Enforcement state is reported as enforced, not declared.	Stub builds, containers, and native Windows differ from the declaration, and the consent surface must say so.
D7	Containers are not double-wrapped.	The runtime is the isolation boundary; consent gates the grants.
D8	Credential roots are non-grantable and fatal.	The boundary is stated so operators can reason about it.

21. References

• PR feat(sandbox): add in-tree nono C FFI backend sandboxing #3212, in-tree nono C FFI sandboxing (reference implementation)
• Related, optional: RFC Backend Plugin Specification, whose external backends declare the same grant block this RFC enforces. This RFC stands on its own and does not depend on it.
• nono upstream (nolabs), Landlock and Seatbelt capability sandbox
• The Lemonade Backend Gateway: Research and design output based on various maintainer feedback, testing and verification

[abn]

Update: This is an archive for anyone who wants to build on it, or parts of it. Other in-flight RFCs have momentum and priority. I'm not maintaining or pushing this beyond this point - fork and take what you want. Happy to answer questions if someone picks it up.