Repository navigation
Conversation
Contributor
[High impact] The changes since the last review appear safe to merge.
|
Mkhoa161
force-pushed
the
feat/ram-mount-capacity
branch
from
October 27, 2026 19:34
d4b14aa to
d776cbc
Compare
Mkhoa161
force-pushed
the
feat/ram-mount-capacity
branch
from
October 27, 2026 19:46
d776cbc to
50791f9
Compare
A `w` file that opens but fails as its bytes land (a full disk, or a quota) printed `sed: couldn't open file F: ...` and exited 0, and under -i too. status() read the failure from write_err, the buffer of those lines, but take_stderr hands that buffer on once the run is over, and both hosts take stderr before the status, so the buffer was always empty by then. A write_failed flag now records the failure; the run exits 4, as GNU sed's failed write does. A w file refused at open was already right (exit 4). The test host gains `/brim/`, whose held writes fail with ENOSPC at their flush. Red before: both the -i and the plain run exited 0. Proved by dropping the flag's assignment: the -i assertion fails. Under -i GNU also abandons the edit when a w write fails; mirage still commits it, a divergence this change leaves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EDQUOT ("Disk quota exceeded") joins FsCondition on both hosts, raised in
Python as QuotaExceededError through _stamped, as efbig builds
FileTooLargeError, and in TypeScript as edquot(), an FsError with code
EDQUOT. Every table keyed on the vocabulary gains its row: POSIX (the
host errno, Linux 122), WASI preview1 19, the classify arms by class and
by number, the per-operand sets (so a command reports it against its
operand and moves on), the node kernel adapter, Pyodide's Emscripten key
(EIO when a build lacks it) and HTTP 507 Insufficient Storage. In
mirage-rs, sed -i's refused edit says `couldn't edit` and exits 4, as GNU
sed's failed write exits 4, and ends the run as any other failed edit
write does. Monty needs no row: CPython raises a plain
OSError for errno 122. The phrase is fixed because macOS's strerror
spells it "Disc quota exceeded". Nothing raises it yet; the capacity
check that does comes later in this series.
QuotaExceededError is Python-only in errors/types and is excused in
symbol_exceptions.json (TypeScript errors are tagged by code), so the
unexcused count stays at 2932.
Proved by mutation, each alone: dropping EDQUOT from OPERAND_CONDITIONS
fails the new render row on both hosts (the constructor test alone
passed, since it reads the same set); dropping the class arm, the errno
arm, CONDITION_CLASS, FS_ERRORS or the 507 row, renumbering Linux 122 or
spelling the phrase "Disc" each fail their table test; dropping the
Pyodide key fails its row; dropping sed's EDQUOT arm fails the -i edit
test (the line loses `couldn't edit` and exits 1).
The node CONDITION_ERRNO row is held by its Record type: an unknown
stamped code already passes through node:os, so only tsc fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
parse_limit read sizes differently on each host and accepted spellings no size should have: "-5" and "+3" as numbers, "1_000" as 1000 in Python and 1 in TypeScript, "1.5MB" as 1 MiB in TypeScript, "5T" and "5abc" as 5 there, "" as NaN, an Arabic-Indic digit as 5 in Python, True as True. Both now take a non-bool integer from 0 to 2**53 - 1 (a whole-number float is its integer, since JavaScript cannot tell 100.0 from 100), or ASCII digits, optionally followed by K, M or G and then an optional B, in any case, inside outer ASCII whitespace, and refuse anything else with one message: `must be a size like 100MB, got <the value>`, printed as JavaScript prints it (JSON.stringify's quoting, non-ASCII kept, no spaces after separators; a number in plain decimals from 1e-6 up to 1e21 and in e-notation outside; Infinity and NaN by name), so the two hosts print the same bytes. The cache's `limit` and a runtime's `max_memory` read sizes through it; max_memory drops its own regex and keeps its `max_memory` prefix on the refusal. xxd -l keeps its own parser. A cache limit of a refused spelling now fails when the workspace is built. 0 stays valid. CacheConfig refuses a boolean limit when it is built, as TypeScript's schema does, rather than reading it as 1; a whole-number float such as 1e6 still reads as its integer. Proved by mutation, each alone: accepting a bool, dropping either bound, dropping re.ASCII (the Kelvin sign and the no-break space rows), dropping ensure_ascii=False, letting a bare B through, dropping Number.isSafeInteger, matching Unicode whitespace in TypeScript, or dropping the max_memory prefix each fails a row. Spelling [0-9] as \d survives as an equivalent mutant under re.ASCII. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StorageUsage (vfs/capacity.py, core/src/vfs/capacity.ts) holds one storage's counted bytes and the caps of the mounts on it: `used` (None until counted, never 0), `pending` (stream chunks admitted but not landed), and the mount -> cap map whose smallest cap is the limit. One rule admits growth: positive growth past used + pending refuses with EDQUOT, reaching the limit exactly passes, and a change that does not grow always passes, so a storage over its cap can always shrink. `report()` is df's QUOTA row, available never below 0; with no caps left it is UNKNOWN and admits everything, so a writer that reached a usage as its last cap went cannot raise. Python carries one threading.Lock per usage: FUSE and NFS change stores from loop threads of their own, and every read and write of the usage happens under it. TypeScript runs every callback on one thread and has none. CapacityLedger (workspace/mount/capacity.py and .ts) is a workspace's storage key -> StorageUsage map: a first finite cap creates the usage uncounted, a storage already counted keeps its count, and the last release drops it, so a later cap starts afresh. countable() says whether mirage can count a backend's bytes: RAM is the only backend that can (CAPACITY_BACKENDS). /dev is not: DevVFS inherits the name `ram`, so it is told apart by class, as read_policy already does. Proved by mutation, 29 each alone and each failing a test: the largest cap as the limit, >= at the boundary, pending ignored, zero growth checked, charge or hold skipping admit, land keeping its hold or adding it instead of the measured change, negative or room-blind reports, a ledger that makes a new usage per bind, changes caps outside the lock, keeps or returns a dropped usage, or never clears, and /dev counted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each RAM mutation (write, its stream chunks and publishes, create, pwrite, truncate, copy, rename, unlink, rm -r, load_state) changes its table in one synchronous block, now a plain helper wrapped in store_change (core/ram/usage.py and .ts). An uncapped store yields no usage: no lock and no count, only the store_change frame, which reads the usage before and after the change (about half a microsecond per mutation in Python). A counted store's block takes its StorageUsage lock, re-checks that the store still holds that usage (a cap released or replaced while it waited is retried, or taken uncapped), counts the table if it is uncounted, and admits the change before it is made: a refused growth leaves the table as it was, and a sparse pwrite or truncate past the cap is refused before its bytes are built. A block that fails past its charge leaves the usage uncounted; after the block, a usage that appeared or changed meanwhile is marked uncounted, so the next counted operation recounts. A stream holds each chunk in pending before it is buffered, so a refused chunk keeps the ones before it, and each publish lands the change it measured, giving a hold taken on an old usage back to it. A replacing stream gives back the old bytes first; a directory move gives back the destination files it overwrites. RAMVFS.capacity() reports the store's QUOTA row; load_state swaps the table and marks it uncounted in one section. The registry binds a usage through Accessor.bind_usage / release_usage, as it binds the workspace clock. RAMAccessor keeps the one owner that counts the store: a second owner's cap on it is refused, and an uncapped mount of it elsewhere accounts against the counting usage. A rename onto its own name changes nothing and returns before the block, as in TypeScript. Python takes one threading.Lock per counted store because FUSE and NFS serve on loop threads of their own; TypeScript runs every callback on one thread, so its storeChange has no lock and no re-read. A publish lands its held bytes in a finally, so a failed publish cannot leave them in pending. Proved by mutation, 60 each alone and each failing a test: the lock not taken (a second writer charges inside the first one's block; a timer publish runs outside it), no re-check under the lock, no re-read after the block, marking on every write or only from no usage, keeping the count after a failure or dropping it after a refusal, counting without the snapshot default, load_state swapping outside the lock or not marking, capacity ignoring the usage, an inline publish inside the chunk's lock, and at every mutator (both hosts) the charge dropped, taken whole instead of as growth, taken after the bytes are built, or a hold never given back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mkhoa161
force-pushed
the
feat/ram-mount-capacity
branch
from
October 27, 2026 21:27
50791f9 to
1f07727
Compare
…hots `capacity` is a mount setting beside `mode`, `read` and `write`: a size (`100MB`, `0`) or a count of bytes, None for no limit. It is accepted as `Mount(vfs, capacity=...)`, `add_mount(..., capacity=)` / addMount, a YAML mount block, a server clone or load override, and a dsh mount block, and every door's cap is checked once, in MountRegistry.mount: coerce_capacity refuses a value that is not a size (`mount '/a/': capacity must be a size like 100MB, got true`) and a backend mirage cannot count. The YAML loaders refuse a bad cap at load with `capacity must be a size like 100MB`, a phrase the shared config corpus pins on both hosts. The cache `limit` is checked the same way at load, on both hosts, so Python no longer reads `limit: true` as a one-byte cache. The registry owns the workspace's CapacityLedger, keyed by vfs_storage_location, which moves from workspace/mount/storage into the registry (storage imports the registry, so the registry could not import it from there). A finite cap binds its storage's usage and hands it to the mount's accessor before anything counts it; an alias of the same RAM store, capped or not, accounts through the accessor it shares. A store another workspace counts is refused as `mount '/x/': this RAM store is counted by another workspace; ...` (a CapacityError keeping its cause), and a placement refused after its bind, by the bind or by a later step such as a walk_limit conflict, releases the cap. TypeScript binds its constructor's mounts once the workspace is built, so a construction that fails earlier binds nothing. Unmounting a mount releases its own cap at once, so the remaining mounts' smallest cap governs, and the last cap's release detaches the usage, so the store pays nothing again. Closing, a failed construction and a failed load release every cap with the clocks; Python's construction covers everything from placing the mounts on, its own /usr/bin and history mounts included. Snapshots save each mount's own cap (MountKey.CAPACITY) and restore it unless the override names its own; an override on a backend that takes no cap (a disk folder handed back for a saved RAM mount) loads uncapped, while a RAM override keeps the saved cap. Copy and clone rebuild it. dsh now refuses a mount block key it does not take (vfs, mode, config, capacity), so a misspelt cap is an error rather than dropped. A server clone with a bad cap answers 400 on both hosts. Proved by mutation, 38 each alone and each failing a test: the size or capability check skipped, the cap kept in the ledger or still attached after its last release, unmount or close or a failed construction keeping caps, a refused bind not rolled back, add_mount, install, normalize, the YAML builder, clone or dsh dropping the cap, the YAML validator dropped, a snapshot dropping the cap, a restore ignoring the override or the saved cap, and dsh accepting a stray key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
df and stat -f already ask mount.vfs.capacity(), so a capped RAM store reports its QUOTA row with no change at either call site: total is the storage's effective cap, used the counted bytes, available the room left, each 1K figure rounded up as GNU rounds (1000 bytes on a 1MB cap reads 1024 1 1024). An uncapped alias of the store shows the cap that governs its writes. The same lines print the same bytes on both hosts. Proved by mutation on each host: RAMVFS.capacity() ignoring its usage turns both rows into dashes and fails the test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen shell cases in integ/unix/capacity/ pin the shell side of a capped mount on both runners: a refused redirect exits 1, creates no file and the line goes on, cp reports its own line, mv across mounts keeps its source and leaves an empty destination, the cap's exact boundary, df and stat -f reporting the cap, a refused stream keeping what landed, sed -i exiting 4 and leaving the file whole, sed exiting 4 on a refused w file, an uncapped mount unchanged, a store over its cap at load keeping its data, a nested mount's cap governing under it, an uncapped child never counted against its parent, and an uncapped alias sharing the cap. A capacity case names `mount_capacity` (per mount, as `mount_read` names policies) and `capacity_door`: `open` caps the mounts as the workspace is built, `load` seeds uncapped, snapshots and loads with capped override mounts. Every capacity case opens and seeds its target fresh, so no golden depends on case order; the shared per-target loop skips them as it skips `read` cases. validate_cases on both hosts refuses a door without caps, an unknown door, and caps on a scenario case, and gate_selftest covers all three. The runtime harness takes a world mount's `capacity`, and integ/runtime/write/capacity.json pins that a monty guest sees `[Errno 122] Disk quota exceeded: '/data/x'` on both hosts. The FUSE probes gain four keys on a mount capped at 10, on both hosts: a 100-byte kernel write plus fsync is refused, with EDQUOT (WinFsp's respelling folded in), the file keeps 0 bytes, and a shell write after a kernel write counts the kernel's bytes, which is the end-to-end check that the per-store lock orders the FUSE loop thread against the shell. The fuse path filter gains the capacity modules and, on both hosts, the RAM mutators, store and accessor that charge them, and integ/README.md's chart names them. topology.json's cases name the topology they test and are excused in target_exceptions.json, so the dropped-target count stays at 27. GNU pins are in each case's note: bash prints `write error: No space left on device` for `echo hi > /dev/full`, then 1 and the next command; mirage keeps its redirect voice. GNU cp says `error writing`, where mirage's cp and mv say `cannot create regular file` for any write failure, a divergence that predates capacity. GNU sed exits 4 when it cannot write an edit; mirage says `couldn't edit`. The TypeScript runner refuses a capacity case on a target that is not RAM, where Python's mount refuses the cap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mkhoa161
force-pushed
the
feat/ram-mount-capacity
branch
from
October 27, 2026 22:12
1f07727 to
65a1fbe
Compare
The class is a map from a storage to its StorageUsage, not a record of entries, and the registry already calls its policy decision store a ledger. MountRegistry.ledger becomes MountRegistry.usages on both hosts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and OPFS Capacity moves from the mount to the store's config, so mounting, unmounting and adding mounts leave it alone and every mount of a store shares it. Each check and change runs as one synchronous step on the event loop, so parallel writers stay inside it without locks. Sizes go through one parser shared with the cache limit and max_memory, and a snapshot keeps the setting in the store's config. Python refuses a capped store over FUSE or FSKit for now; the count is per process.
Member
|
I pushed a follow-up commit, 32a83e7, that reworks how capacity is set. In short, capacity is now a setting on the store, like tmpfs User interfaceWhere you set it
mounts:
/data:
vfs: ram
mode: write
config:
capacity: 100MBws = Workspace({"/data": RAMVFS(capacity="100MB")}, mode=MountMode.WRITE)const ws = new Workspace({ '/data': new RAMVFS({ capacity: '100MB' }) }, { mode: MountMode.WRITE })No What a user sees
Known limitations (documented)
How it works now
Docs
|
… quiet A replacement reserves only what passes the bytes it replaces, so a smaller rewrite at the cap lands. Writes in flight are tracked; two on overlapping paths, or a df or a native write, mark the count for a recount that waits until no write is in flight. A no-create truncate of a missing file reserves nothing. Disk takes capacity through the TS registry, Python refuses a capped store added under a FUSE or FSKit mount, the TS cache config checks its limit at load, and Python size errors print numbers as JavaScript does. mdformat on the integ tables.
…laps it A folder move in flight can take the bytes a replacement was credited with elsewhere, so a write overlapping another reserves every byte it writes from then on.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A mount can cap the bytes its storage holds:
capacity: 100MBin a mount block,Mount(vfs, capacity=...)/new Mount(vfs, { capacity })in code,add_mount(..., capacity=)/addMount, a server clone or load override, or a dsh mount block. RAM only for now; any other backend refuses a cap (mount '/s3/': capacity needs a backend mirage can count; s3 does not).EDQUOT, "Disk quota exceeded": a newFsCondition, raised in Python asQuotaExceededError(anOSError, built asefbigbuildsFileTooLargeError) and in TypeScript as anFsErrorwith codeEDQUOT. The command reports it with its write-error status (1 for most, 2 fortar, 4 forsed, as GNU's), and the line goes on. Output written at once (printf,echo) is all or nothing: a refused>creates no file and leaves an existing one as it was, where GNU truncates at open. A streaming command (seq) keeps the bytes that landed before the refused chunk. Reaching the cap exactly passes.sed -ireports a refused edit ascouldn't editand exits 4, as GNU sed's failed write does.df.vfs_storage_location, moved fromworkspace/mount/storageinto the registry, its caller). Mounts of one store share one count and the smallest cap applies under every prefix, an uncapped alias included. A nested child mount's cap governs under it; an uncapped child is never counted against its parent. A store counted by one workspace cannot be capped by a second; the second's uncapped mounts account against the first's count. Unmounting a mount releases its own cap at once; the last cap's release detaches the usage, so the store pays nothing again. Close, a failed construction and a failed load release every cap.store_changeframe, which reads the usage before and after the change (about half a microsecond in Python).dfandstat -freport a capped storage asQUOTA: total = the cap, used = counted bytes, available = cap − used, never below 0, each 1K figure rounded up as GNU rounds. A kernelstatfskeeps its fixed answer.parse_limitis now strict and identical on both hosts: a non-bool integer from 0 to 2^53 − 1 (a whole-number float is its integer, since JavaScript cannot tell100.0from100), or ASCII digits, optionally followed byK,MorGand then an optionalB, any case, inside outer ASCII whitespace. Anything else is refused with one message,must be a size like 100MB, got <value>(JSON as JavaScript prints it; Infinity and NaN by name). The cachelimit(checked at load on both hosts, ascapacityis) and a runtime'smax_memoryread sizes through it, so spellings both hosts used to read differently ("1_000","1.5MB","-5","5T", an Arabic-Indic digit) are now refused on both.vfs,mode,config,capacity), so a misspelt cap is an error rather than silently dropped.Why a per-store lock instead of one loop per workspace
Python FUSE and NFS serve their callbacks on loop threads of their own (
fuse/fs.py:83-84,workspace/nfs.py:89-93), so a counted RAM store can be changed from two threads at once. This change keeps that, and makes each capacity check and the table change it guards atomic with onethreading.Lockper counted store. Callers take a public lock on the state object, as_Floordoes (utils/ids.py:155-162,:194). The lock is held only inside plain synchronous helpers, never across an await, and a store nobody caps has no lock to take. The alternative is to serve FUSE and NFS on the workspace's own loop, so that no lock is needed. Traced from the code, that deadlocks on the caller's loop in two places. A synchronous file call on the mountpoint from async code (a notebook cell, a FastAPI handler) would block the loop thatMirageFS._callhands each kernel request to (fuse/fs.py:105-108, which waits atbridge/sync.py:72-77). Alocalruntime started with its working directory on the mount would block inPopen's exec handshake on that loop while the child'schdirwaits on FUSE (runtime/sandbox/program.py:237-250). Avoiding both needs a dedicated loop thread per workspace, with every async API hopping onto it: a change of its own. Table operations outside the capacity path (listings, reads, attribute changes) are not taken under the lock and behave as on main.threading.Lockper counted storestore_changeis 28 of them); none in TypeScript. At run time, a cappedpwriteor publish copies the file's bytes while it holds the lock, as it copied them before this changeHow the lock is held:
store_change(core/ram/usage.py) takes the counted store'sStorageUsage.lock, re-checks under it that the store still holds that usage (a cap released or replaced while it waited is retried against the current usage, or the uncapped path), counts the table if it is uncounted, and yields; the block admits and charges before it changes the table. A block that raises past its charge marks the usage uncounted. After the block, a usage that appeared or changed while it ran is marked uncounted under its own lock, never while holding another, so the next counted operation recounts. TypeScript runs every callback on one thread, so itsstoreChangehas no lock and no re-read.Bounded landings
Counted but not admitted, each bounded, with later growth refused once it lands: a stream's pad back up to its position after another op shortens its file mid-stream (at most the stream's own earlier output); chunks a stream buffered before its store gained a usage (at most one publish window); and operations in flight when a store's usage appears, goes or is replaced, which land under the usage they started with and leave the current one to be recounted.
The other way round, one refusal is stricter than the bytes need: a replacing stream reserves each chunk's full length, because the file can change before the chunk lands. If another writer refills that file after the stream emptied it, a chunk that would only overwrite those bytes counts as growth, and at the exact cap it is refused. It takes two writers on one file at once; across 23,503 recorded agent shell lines, the only such shape is a background job relaunched onto the same log file (14 lines).
Decisions
/devis refused a capacity by class:DevVFSinherits the nameram, so a name check alone would let it through. It followsread_policy'sisinstance(vfs, DevVFS); the message still namesdev./usr/binand history mounts included.EINVALbefore anything is charged, asftruncate(2)answers.RAMAccessorkeeps only the owner that counts its store; nothing reads the owners of uncapped mounts.finally, so a publish that fails cannot leave them inpending.mount '/x/': this RAM store is counted by another workspace; ..., aCapacityError) or by a later step such as awalk_limitconflict, releases the cap it bound.vfs_storage_locationlives in the registry, which keys the ledger by it;workspace/mount/storageimports the registry, so it takes the function from there.mount '/x/': ...), as the write-policy refusals do.Tests
Unit tests on both hosts for every error table, the size grammar,
StorageUsage, the ledger and capability, every RAM mutator (charge, refusal leaving the table unchanged, shrinking over the cap, sparsepwrite/truncaterefused before the bytes are built, streams), every door, alias and nested topologies, release on unmount/close/failed construction/failed load, snapshots (an uncapped alias still counts against the cap aftercopy()and a load) anddf/stat -f. Python adds lock tests on real threads: a second writer waits for the first one's locked block, a timer publish takes the lock, an inline publish runs outside it, usage bound or replaced mid-write is recounted, a writer waiting on a released usage takes the uncapped path, a failure past the charge leaves the usage uncounted, andload_stateswaps and marks in one section.Integration: 13 shell cases in
integ/unix/capacity/, each on a workspace opened for that case alone (mount_capacity,capacity_door: open|load), on both runners;integ/runtime/write/capacity.jsonfor a Monty guest; four FUSE probe keys (a kernel write refused with EDQUOT, the file left empty, and a shell write counting the kernel's bytes on the same store).GNU
Pinned in
debian:stable-slim(GNU coreutils 9.7):echo hi > /dev/full; echo $?; echo nextprintsbash: line 1: echo: write error: No space left on device, then1andnext. Mirage keeps its redirect voice (/data2/b: Disk quota exceeded); the exit status and the line going on match. GNUcpreports a failed write aserror writing; mirage'scpsayscannot create regular filefor any write failure, which predates this change.Not in this PR
Disk capacity (a follow-up: per-file windows across loops, a folder walk to count, recounts after native writes). Memory and workspace-wide quotas.
🤖 Generated with Claude Code