[bytecii]

Summary

This PR does two things on both hosts:

• it moves snapshot tar_io onto mirage-rs's tar engine;
• it lets a gathered Read carry end-of-input with its last bytes.

The git 2.56 work that started here landed separately as #1909; this branch now takes main's version of every git file, so none of that is in this diff.

1. Snapshots on the tar engine

• write_tar/read_tar (Python) and writeSnapshotTar/readSnapshotTar/writeTar/readTar (TypeScript) now drive the tar stream with a small host:
  • Writing: the manifest goes first, then each blob. A disk file is opened only while it is read, so a large snapshot no longer holds every file open at once.
  • Reading: one pass, with the compressor sniffed. The manifest comes first; each disk mount's blob is staged under staging as it arrives, and every other blob is held in memory.
• tarfile is gone from the snapshot path and the server's load route. modern-tar is gone from core's dependencies, the lockfile and the tests:
  • the snapshot tests build their foreign archives by hand;
  • the relay test makes its .tgz with mirage's own tar -cz.
• Compatibility:
  • Snapshots written by tarfile (pax, plain, .gz, .xz) still load.
  • New snapshots are GNU format with ././@LongLink for long names, and read back fine in tarfile.
• The safety checks hold:
  • a captured file replaced by a link is refused before anything is written;
  • a staged blob cannot escape staging;
  • unsafe or missing blob paths raise the same errors as before.

2. A gathered Read ends with its data

• StreamHost (both hosts) now sends end together with the last bytes of a least read that reaches end-of-input. Patch's host already did.

• tar's and zip's readers use those bytes and then stop. Before, each member cost one more round trip to learn the input had ended.

• The Rust test host does the same, so tar's and zip's unit tests exercise it.

• Measured on a disk mount, eight runs each:

  Case	TypeScript CPU	Python CPU
  tar -c of 1,000 × 100 KiB files	1.026 → 1.002 s	0.885 → 0.877 s
  tar -c of one 200 MiB file	unchanged	unchanged
  tar -c of the whole 298 MiB tree	2.53 → 2.50 s	1.76 → 1.75 s

  The gain is small (1–2%) and about the size of the noise at a load average around 6. The extra round trip was cheaper than tar: one Rust engine for both hosts #1893 estimated.

Size

+608 / −316 lines over 21 files: snapshots are most of it, the read change +66 / −30. Driving the engine needs a small host per side, which costs more lines than calling tarfile and modern-tar did; in exchange modern-tar leaves the npm tree.

Verification

• Rust:
  • cargo fmt --check, cargo clippy --workspace --all-targets -D warnings (with PYO3_PYTHON set to the venv), and -p mirage-rs-wasm --target wasm32-unknown-unknown: clean.
  • cargo test --workspace: all pass, 592 in mirage-rs.
• Python:
  • uv run pytest -n 8 before the latest merge: 33,972 passed, 1 failed. The failure, test_cancelling_a_call_kills_what_the_child_started, is a timing-sensitive sandbox test; it passed 3 out of 3 alone.
  • After merging main: the workspace, git, stream-host, generic-command and server suites, 12,614 passed.
• TypeScript:
  • Core vitest before the merge: 24,119 passed. The 47 failures are the QuickJS-backed js and runtime tests, which exit 127 because qjs-wasi isn't installed locally; it's the same set as on main.
  • After merging main: snapshot, relay, stream-host, git, tar and zip tests, 3,229 passed. tsc: clean.
• Integ before the merge: ram, disk, git-ram and git-disk, 18,412 passed per host, 0 failed.
• Gates:
  • check_symbol_parity --strict: baseline lowered from main's 2,913 to 2,910.
  • check_layout_parity --strict, check_barrel_surface, check_case_targets --strict (27), check_spec_parity (101 commands, 12 CLI trees), check_cli_cold_start, check_release_artifacts and check_dependency_licenses --python --npm (917 packages): pass.
  • pre-commit on every changed file: clean.

🤖 Generated with Claude Code

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Moves snapshot reads and writes onto the shared Rust tar engine. Gathered reads can now carry their last bytes alongside end.

• Workspace snapshots now use Mirage’s tar engine on both hosts.
• Gathered reads return their last bytes with the end marker.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Save["Manifest and blobs"] --> Host["Python or TypeScript snapshot host"]
  Host <--> Engine["Rust tar engine"]
  Host <--> Archive["Plain or compressed archive"]
  Host --> Stage["Referenced disk blobs staged"]
  Host --> Held["Other referenced blobs held in memory"]
  Host --> Skip["Unused members discarded"]

Loading

_{Reviews (6) · Last reviewed commit: "snapshot tars: refuse blob paths that na..." · Reviewed by Greptile}