Decompress return UnexpectedEOF if Skippable frames written

[myxo]

What version of Go are you using (go version)?

go version go1.19.3 linux/amd64

What did you do?

Zstd spec provide skippable-frames - frames that ignores by decompressor and allows to write any user metadata into archive (e.g. seekable table).

If I try to decompress archive with skippable frame, I get io.UnexpectedEOF. Here is an example:

import (
	"bytes"
	"encoding/binary"
	"fmt"
	"testing"

	"github.com/DataDog/zstd"
)

func TestDataDog(t *testing.T) {
	// write payload
	payload := make([]byte, 1_000_001)
	out, err := zstd.Compress(nil, payload)
	if err != nil {
		t.Fail()
	}

	// write skip frame
	skipFrame := make([]byte, 12)
	binary.LittleEndian.PutUint32(skipFrame, 0x184D2A50) // skip frame magic
	binary.LittleEndian.PutUint32(skipFrame[4:], 4)      // user data size
	binary.LittleEndian.PutUint32(skipFrame[8:], 12345)  // user data
	out = append(out, skipFrame...)

	decomp, err := zstd.Decompress(nil, out)
	if err != nil {
		fmt.Printf("error: %s\n", err) // got here "error: unexpected EOF"
		t.Fail()
	}
	if !bytes.Equal(payload, decomp) {
		t.Fail()
	}
}

Note, that payload size is not accidental, if it's small and can be processed by ZSTD_decompress (https://github.com/DataDog/zstd/blob/5f14d6af117fa84b37f99d4cde775e6039be6d3b/zstd.go#LL146C16-L146C16), zstd.Decompress will return no error. Problem occur then we fallback to read it with stream reader.

This is related to #112, but in this case I add a valid zstd frame

[Viq111]

Hi @myxo, thanks for the report and the reproducible case!

Indeed it seems like it's directly linked to this other use-case (streaming use-case): 42c5dcb#diff-ff3e40bda515a4af0936733c5415920eb09cc903913a7243bdec1074266784b0R470

Let me think a bit more if we can safely remove the UnexpectedEOF case / make it work for both.
In the meantime, if you want to contribute a fix, feel free to make a PR!