fs: btrfs: fix zstd decompression of compressed inline extents - #38
Closed
munzzyy wants to merge 1 commit into
Closed
fs: btrfs: fix zstd decompression of compressed inline extents#38munzzyy wants to merge 1 commit into
munzzyy wants to merge 1 commit into
Conversation
munzzyy
force-pushed
the
btrfs-zstd-sector-padding
branch
from
July 31, 2026 19:23
665c6de to
809772e
Compare
Contributor
Author
|
Sent this upstream to the U-Boot list, since the bug is in mainline btrfs too, not It's the same change as this PR, rebased onto mainline master with a proper |
The kernel compresses an inline extent as a whole block: run_delalloc_inline() calls btrfs_compress_bio(inode, 0, blocksize, ...), so the data is zero-filled past EOF and the resulting zstd frame declares a content size of one block. The extent item records the unaligned file size though - __cow_file_range_inline() passes i_size down to insert_inline_extent(), which stores it as ram_bytes. btrfs_read_extent_inline() sizes its decompression buffer from ram_bytes, so for a 1900-byte file the destination is 1900 bytes while the frame decodes to 4096. Since commit 918adf8 ("btrfs: Use U-Boot API for decompression") btrfs decompresses through the common U-Boot helper, which uses the one-shot zstd_decompress_dctx(). That API requires the destination to cover the whole frame and fails with dstSize_tooSmall, error 70, otherwise. The streaming ZSTD_decompressStream() path it replaced stopped once the output buffer was full, so it never hit this. The kernel side does not notice because fs/btrfs/zstd.c streams into its own buffer and copies out at most destlen. Allocate a full block for the decompression buffer and copy only ram_bytes back to the caller. An inline extent never spans more than one block, which bounds the allocation. This shows up on RK3399 and ODROID-N2 as "zstd_decompress: failed to decompress: 70" (armbian/build#9651, #10208), where it breaks fdt apply on zstd-compressed overlays. Images built with mkfs.btrfs --rootdir --compress zstd do not reproduce it, since btrfs-progs writes a frame whose content size already equals ram_bytes. Only files written at runtime through the kernel trip it. Fixes: 918adf8 ("btrfs: Use U-Boot API for decompression") Signed-off-by: Cole Munz <Munzzyy1@proton.me>
alchark
reviewed
Aug 1, 2026
munzzyy
force-pushed
the
btrfs-zstd-sector-padding
branch
from
August 1, 2026 06:25
809772e to
f6ac2bb
Compare
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #35.
v2. You and Qu Wenruo asked for the same thing from opposite directions, so this drops the bounce buffer and sizes the destination at the caller instead.
Here is what actually breaks. The kernel compresses an inline extent as a whole block:
run_delalloc_inline()callsbtrfs_compress_bio(inode, 0, blocksize, ...), so the tail past EOF is zero-filled and the zstd frame declares a content size of one block. The extent item stores the unaligned size though.__cow_file_range_inline()passesi_sizedown toinsert_inline_extent(), which writes it asram_bytes.btrfs_read_extent_inline()sizes its buffer fromram_bytes, so a 1900-byte file gets a 1900-byte destination for a frame that decodes to 4096. Since 918adf8 ("btrfs: Use U-Boot API for decompression") btrfs goes through the shared helper, which is the one-shotzstd_decompress_dctx(). That API wants the destination to cover the whole frame and returnsdstSize_tooSmall, error 70, when it doesn't. The streaming path it replaced stopped once the output was full, so it never hit this. The kernel side doesn't notice either, sincefs/btrfs/zstd.cstreams into its own buffer and copies out at mostdestlen.So the fix is to hand the decompressor a full block and copy
ram_bytesback out. An inline extent never spans more than one block, which bounds the allocation.On the pre-zeroing, it turns out not to be needed. The decompressor writes the whole frame, and the existing
if (ret < dsize)memset already covers a short read, so every byte in[0, dsize)is written before the copy-out.v1 claimed regular extents could carry a frame bigger than
ram_bytes. Qu pushed back, since for regular extentsram_bytesis always block aligned, and he was right. Only inline extents have an unalignedram_bytes. The commit message says that now instead of the sector-padding hand-wave.Testing: sandbox builds clean, and reading both compressed files back out of a
mkfs.btrfs --rootdir --compress zstdimage matches the originals byte for byte, regular extent and inline extent.One thing worth knowing if you try to reproduce it: mkfs-built images don't fail. btrfs-progs writes an inline frame whose content size already equals
ram_bytes, so only files written at runtime through the kernel trip it. That fits the Armbian reports (armbian/build#9651, #10208), where it turned up on /boot overlays after a package update rewrote them.Upstream carries the same patch with
Reviewed-by: Qu Wenruo <wqu@suse.com>.