Skip to content

Charge directory creation against a mount's write_bytes_limit - #1187

Open
rewitt94 wants to merge 1 commit into
pydantic:mainfrom
rewitt94:charge-mkdir-upstream
Open

rewitt94 wants to merge 1 commit into
pydantic:mainfrom
rewitt94:charge-mkdir-upstream

Conversation

@rewitt94

@rewitt94 rewitt94 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

We can optionally (default off) set write limits on a mount, however we do not currently charge for writing directories.

This prices directories at 4096 bytes charging them towards the same write_bytes_limit.

🤖 Generated with Claude Code


Summary by cubic

Makes mkdir count against a mount's write_bytes_limit so a write budget also bounds how many directories sandboxed code can create.

  • Each directory created, parents included, is charged 4,096 bytes (DIRECTORY_WRITE_BYTES) before anything is created; the charge is committed only after the operation succeeds.
  • Overlay mkdir(parents=True) collects the directories it will record first, so a refused call leaves no partial tree.
  • mkdir on an existing directory is free, and the limit remains optional with unlimited as the default.

Written for commit 56b273b. Summary will update on new commits.

View guided diff

`mkdir` never touched `write_bytes_limit`, so a mount with a write budget
could still be filled with unlimited directories. Direct and overlay mounts
now charge `DIRECTORY_WRITE_BYTES` (4,096) per directory created, parents
included, checked before anything is created and committed after. Overlay
`mkdir(parents=True)` collects the directories it will record first, so a
refused call records none of them; read-only mounts already refuse mkdir.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Branch diff: charge-mkdir-upstream

Commit Message Production Test Docs Other
56b273b Charge directory creation aga… +75 -43 +69 -1 +4 -0 +0 -0
TOTAL +75 -43 +69 -1 +4 -0 +0 -0

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 issues found across 9 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/monty-fs/src/direct.rs">

<violation number="1" location="crates/monty-fs/src/direct.rs:202">
P3: The quota check runs before `host_mkdir` validates the request, so a `mkdir` that would fail for another reason can report `WriteLimitExceeded` instead. Consider running the path checks first, or accept that the quota error takes precedence and document it.</violation>

<violation number="2" location="crates/monty-fs/src/direct.rs:218">
P2: `missing_dirs` treats a dangling symlink as an absent directory and can reject `mkdir` on that existing entry with `WriteLimitExceeded` before `host_mkdir` reports `FileExistsError`. Use a no-follow existence check when determining which entries are actually missing.</violation>
</file>

<file name="crates/monty-fs/src/overlay.rs">

<violation number="1" location="crates/monty-fs/src/overlay.rs:884">
P2: The per-directory `state.insert` loop can leave a partial tree when the overlay memory limit rejects a later entry, while the write charge is not committed. Preflight the complete replacement set and insert it atomically, or roll back earlier entries on failure; otherwise a retry can undercharge directories that were already recorded.</violation>
</file>

<file name="docs/limitations/filesystem.md">

<violation number="1" location="docs/limitations/filesystem.md:128">
P3: The new sentence says the 4,096-byte charge is taken 'before anything is created', but the code only checks the budget first and records the charge after `mkdir` succeeds. Also, 'would pass the limit' is ambiguous; say 'would exceed the limit'.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

.find(|&n| absent(&components[..n].join("/")))
.map_or(0, |n| components.len() + 1 - n)
} else {
usize::from(absent(rel))

@cubic-dev-ai cubic-dev-ai Bot Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: missing_dirs treats a dangling symlink as an absent directory and can reject mkdir on that existing entry with WriteLimitExceeded before host_mkdir reports FileExistsError. Use a no-follow existence check when determining which entries are actually missing.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At crates/monty-fs/src/direct.rs, line 218:

<comment>`missing_dirs` treats a dangling symlink as an absent directory and can reject `mkdir` on that existing entry with `WriteLimitExceeded` before `host_mkdir` reports `FileExistsError`. Use a no-follow existence check when determining which entries are actually missing.</comment>

<file context>
@@ -188,9 +188,35 @@ fn append_bytes(path: &str, data: &[u8], ctx: &mut MountContext<'_>) -> Result<M
+            .find(|&n| absent(&components[..n].join("/")))
+            .map_or(0, |n| components.len() + 1 - n)
+    } else {
+        usize::from(absent(rel))
+    }
 }
</file context>
Suggested change
usize::from(absent(rel))
usize::from(dir.symlink_metadata(rel).is_err())
Fix with cubic

let charge = to_create.len().saturating_mul(DIRECTORY_WRITE_BYTES);
check_write_limit(charge, ctx)?;
for dir in to_create {
state.insert(

@cubic-dev-ai cubic-dev-ai Bot Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The per-directory state.insert loop can leave a partial tree when the overlay memory limit rejects a later entry, while the write charge is not committed. Preflight the complete replacement set and insert it atomically, or roll back earlier entries on failure; otherwise a retry can undercharge directories that were already recorded.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At crates/monty-fs/src/overlay.rs, line 884:

<comment>The per-directory `state.insert` loop can leave a partial tree when the overlay memory limit rejects a later entry, while the write charge is not committed. Preflight the complete replacement set and insert it atomically, or roll back earlier entries on failure; otherwise a retry can undercharge directories that were already recorded.</comment>

<file context>
@@ -870,30 +871,42 @@ fn mkdir(
+    let charge = to_create.len().saturating_mul(DIRECTORY_WRITE_BYTES);
+    check_write_limit(charge, ctx)?;
+    for dir in to_create {
+        state.insert(
+            dir,
+            OverlayEntry::Directory {
</file context>
Fix with cubic

`OverlayMemory`, appending to an existing real file can materialize that
file into the in-memory overlay, so the existing file bytes count against
the limit along with the newly appended bytes.
Each directory `mkdir` creates, parents included, is charged 4,096 bytes against the limit before anything is

@cubic-dev-ai cubic-dev-ai Bot Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The new sentence says the 4,096-byte charge is taken 'before anything is created', but the code only checks the budget first and records the charge after mkdir succeeds. Also, 'would pass the limit' is ambiguous; say 'would exceed the limit'.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/limitations/filesystem.md, line 128:

<comment>The new sentence says the 4,096-byte charge is taken 'before anything is created', but the code only checks the budget first and records the charge after `mkdir` succeeds. Also, 'would pass the limit' is ambiguous; say 'would exceed the limit'.</comment>

<file context>
@@ -125,6 +125,8 @@ Hosts can configure a cumulative `write_bytes_limit` per mount. In
 `OverlayMemory`, appending to an existing real file can materialize that
 file into the in-memory overlay, so the existing file bytes count against
 the limit along with the newly appended bytes.
+Each directory `mkdir` creates, parents included, is charged 4,096 bytes against the limit before anything is
+created, so a `mkdir(parents=True)` that would pass the limit creates nothing.
 
</file context>
Fix with cubic

} else {
0
};
check_write_limit(charge, ctx)?;

@cubic-dev-ai cubic-dev-ai Bot Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The quota check runs before host_mkdir validates the request, so a mkdir that would fail for another reason can report WriteLimitExceeded instead. Consider running the path checks first, or accept that the quota error takes precedence and document it.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At crates/monty-fs/src/direct.rs, line 202:

<comment>The quota check runs before `host_mkdir` validates the request, so a `mkdir` that would fail for another reason can report `WriteLimitExceeded` instead. Consider running the path checks first, or accept that the quota error takes precedence and document it.</comment>

<file context>
@@ -188,9 +188,35 @@ fn append_bytes(path: &str, data: &[u8], ctx: &mut MountContext<'_>) -> Result<M
+    } else {
+        0
+    };
+    check_write_limit(charge, ctx)?;
+    let result = host_mkdir(ctx.mount_dir, rel, parents, exist_ok, path)?;
+    commit_write_bytes(charge, ctx);
</file context>
Fix with cubic

@codecov

codecov Bot commented Oct 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.61905% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/monty-fs/src/overlay.rs 95.83% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will regress 5 benchmarks

1 warning

⚠️ 9 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 9 improved benchmarks
❌ 5 regressed benchmarks
✅ 92 untouched benchmarks
🆕 14 new benchmarks
⏩ 92 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation ext_calls_1000 8.4 ms 10.7 ms -21.25%
❌ CPU Simulation session_checkout_run 74.1 µs 91.9 µs -19.43%
❌ CPU Simulation func_call_kwargs__monty 55.7 µs 63.1 µs -11.78%
❌ CPU Simulation pool_create_session_run 160.9 µs 181.6 µs -11.36%
❌ CPU Simulation add_two__monty 35.5 µs 39.5 µs -10.12%
⚡ CPU Simulation list_append_str__monty 240.5 ms 163.9 ms +46.68%
⚡ CPU Simulation container_repr__monty 53.3 ms 41.2 ms +29.4%
⚡ CPU Simulation fstring_report__monty 37.2 ms 29.8 ms +24.86%
⚡ CPU Simulation str_parse__monty 92 ms 73.8 ms +24.71%
⚡ CPU Simulation end_to_end__monty 132 µs 112.5 µs +17.31%
⚡ CPU Simulation session_load__monty 9.1 ms 7.9 ms +15.79%
⚡ CPU Simulation re_extract__monty 28.4 ms 24.8 ms +14.76%
⚡ CPU Simulation session_dump__monty 2.5 ms 2.2 ms +12.11%
⚡ CPU Simulation repl_feed_after_2k_snippets__monty 158.3 µs 143.4 µs +10.41%
🆕 CPU Simulation reduce__monty N/A 23.2 ms N/A
🆕 CPU Simulation csv_read__monty N/A 212.8 ms N/A
🆕 CPU Simulation csv_write__monty N/A 126 ms N/A
🆕 CPU Simulation pyperformance_barnes_hut__monty N/A 96.1 ms N/A
🆕 CPU Simulation pyperformance_fannkuch__monty N/A 20 ms N/A
🆕 CPU Simulation pyperformance_float__monty N/A 101.4 ms N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rewitt94:charge-mkdir-upstream (56b273b) with main (6e31d92)

Open in CodSpeed

Footnotes

  1. 92 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant