Skip to content

fs: btrfs: fix zstd decompression of compressed inline extents - #38

Closed
munzzyy wants to merge 1 commit into
flipperdevices:rk3576from
munzzyy:btrfs-zstd-sector-padding
Closed

fs: btrfs: fix zstd decompression of compressed inline extents#38
munzzyy wants to merge 1 commit into
flipperdevices:rk3576from
munzzyy:btrfs-zstd-sector-padding

Conversation

@munzzyy

@munzzyy munzzyy commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Fixes #35.

v2. You and Qu Wenruo asked for the same thing from opposite directions, so this drops the bounce buffer and sizes the destination at the caller instead.

Here is what actually breaks. The kernel compresses an inline extent as a whole block: run_delalloc_inline() calls btrfs_compress_bio(inode, 0, blocksize, ...), so the tail past EOF is zero-filled and the zstd frame declares a content size of one block. The extent item stores the unaligned size though. __cow_file_range_inline() passes i_size down to insert_inline_extent(), which writes it as ram_bytes.

btrfs_read_extent_inline() sizes its buffer from ram_bytes, so a 1900-byte file gets a 1900-byte destination for a frame that decodes to 4096. Since 918adf8 ("btrfs: Use U-Boot API for decompression") btrfs goes through the shared helper, which is the one-shot zstd_decompress_dctx(). That API wants the destination to cover the whole frame and returns dstSize_tooSmall, error 70, when it doesn't. The streaming path it replaced stopped once the output was full, so it never hit this. The kernel side doesn't notice either, since fs/btrfs/zstd.c streams into its own buffer and copies out at most destlen.

So the fix is to hand the decompressor a full block and copy ram_bytes back out. An inline extent never spans more than one block, which bounds the allocation.

On the pre-zeroing, it turns out not to be needed. The decompressor writes the whole frame, and the existing if (ret < dsize) memset already covers a short read, so every byte in [0, dsize) is written before the copy-out.

v1 claimed regular extents could carry a frame bigger than ram_bytes. Qu pushed back, since for regular extents ram_bytes is always block aligned, and he was right. Only inline extents have an unaligned ram_bytes. The commit message says that now instead of the sector-padding hand-wave.

Testing: sandbox builds clean, and reading both compressed files back out of a mkfs.btrfs --rootdir --compress zstd image matches the originals byte for byte, regular extent and inline extent.

20700 bytes read in 0 ms
md5 for 01000000 ... 010050db ==> 582f8b2aeca56160bb7c7cb91d274fc0
1900 bytes read in 0 ms
md5 for 02000000 ... 0200076b ==> f2226421650a90b18829eb4069e1daa0

One thing worth knowing if you try to reproduce it: mkfs-built images don't fail. btrfs-progs writes an inline frame whose content size already equals ram_bytes, so only files written at runtime through the kernel trip it. That fits the Armbian reports (armbian/build#9651, #10208), where it turned up on /boot overlays after a package update rewrote them.

Upstream carries the same patch with Reviewed-by: Qu Wenruo <wqu@suse.com>.

@munzzyy
munzzyy requested a review from a team July 28, 2026 05:16
@munzzyy
munzzyy force-pushed the btrfs-zstd-sector-padding branch from 665c6de to 809772e Compare July 31, 2026 19:23
@munzzyy

munzzyy commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Sent this upstream to the U-Boot list, since the bug is in mainline btrfs too, not
just the rk3576 branch — the abuf-based decompress_zstd in fs/btrfs/compression.c
is unchanged on master (v2026.07), so a sector-padded zstd extent fails the same way
there. Figured a generic fs fix belongs upstream rather than carried here.

Patch: https://lore.kernel.org/u-boot/9acf406b5e73fa83001e1951e883143e2afb8b2d.1785528314.git.Munzzyy1@proton.me/

It's the same change as this PR, rebased onto mainline master with a proper
Fixes: 918adf8e0733 tag. checkpatch is clean and it builds on sandbox with btrfs +
zstd. If it lands upstream, this branch can just drop the local commit and pull it in;
until then the PR keeps the fix on rk3576. Happy either way.

The kernel compresses an inline extent as a whole block:
run_delalloc_inline() calls btrfs_compress_bio(inode, 0, blocksize, ...),
so the data is zero-filled past EOF and the resulting zstd frame declares
a content size of one block. The extent item records the unaligned file
size though - __cow_file_range_inline() passes i_size down to
insert_inline_extent(), which stores it as ram_bytes.

btrfs_read_extent_inline() sizes its decompression buffer from ram_bytes,
so for a 1900-byte file the destination is 1900 bytes while the frame
decodes to 4096. Since commit 918adf8 ("btrfs: Use U-Boot API for
decompression") btrfs decompresses through the common U-Boot helper,
which uses the one-shot zstd_decompress_dctx(). That API requires the
destination to cover the whole frame and fails with dstSize_tooSmall,
error 70, otherwise. The streaming ZSTD_decompressStream() path it
replaced stopped once the output buffer was full, so it never hit this.

The kernel side does not notice because fs/btrfs/zstd.c streams into its
own buffer and copies out at most destlen.

Allocate a full block for the decompression buffer and copy only
ram_bytes back to the caller. An inline extent never spans more than one
block, which bounds the allocation.

This shows up on RK3399 and ODROID-N2 as "zstd_decompress: failed to
decompress: 70" (armbian/build#9651, #10208), where it breaks fdt apply
on zstd-compressed overlays. Images built with mkfs.btrfs --rootdir
--compress zstd do not reproduce it, since btrfs-progs writes a frame
whose content size already equals ram_bytes. Only files written at
runtime through the kernel trip it.

Fixes: 918adf8 ("btrfs: Use U-Boot API for decompression")
Signed-off-by: Cole Munz <Munzzyy1@proton.me>
Comment thread fs/btrfs/compression.c
@munzzyy
munzzyy force-pushed the btrfs-zstd-sector-padding branch from 809772e to f6ac2bb Compare August 1, 2026 06:25
@munzzyy munzzyy changed the title fs: btrfs: fix zstd decompression of sector-padded extents fs: btrfs: fix zstd decompression of compressed inline extents Aug 1, 2026
@alchark

alchark commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Applied the version from the upstream mailing list, thanks a lot @munzzyy!

b5b70ee

@alchark alchark closed this Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

btrfs: zstd-compressed files read back corrupt (regular + inline extents)

2 participants