[SUPPORT] How to use ref-version-mismatch

[vitorfloriano]

Hi there,

So, in our repo we run zizmor-action in CI, but it missed a few comment version mismatches caught by pinact.

I tried to run zizmor locally, with different flag combinations, and it still didn't catch the mismatches.

I haven't filed this as a bug because I'm not sure if I'm using zizmor correctly.

🌈 zizmor Version

$ zizmor --version
zizmor 1.23.1

Example Workflow

$ cat .github/workflows/apidiff.yml 
name: APIDiff

on:
  push:
    paths-ignore:
      - '**/*.md'
  pull_request:
    paths-ignore:
      - '**/*.md'

permissions: {}

jobs:
  go-apidiff:
    permissions:
      contents: read
    name: Verify API differences
    runs-on: ubuntu-latest
    # Pull requests from different repository only trigger this checks
    if: (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
    steps:
      - name: Clone the code
        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
        with:
          fetch-depth: 0
          persist-credentials: false
      - name: Setup Go
        uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v9.9.9   <---- This version is wrong
        with:
          go-version-file: go.mod
      - name: Execute go-apidiff
        uses: joelanford/go-apidiff@60c4206be8f84348ebda2a3e0c3ac9cb54b8f685 # v0.8.3
        with:
          compare-imports: true
          print-compatible: true

zizmor Output

$ zizmor .github/workflows/apidiff.yml 
🌈 zizmor v1.23.1
 INFO audit: zizmor: 🌈 completed .github/workflows/apidiff.yml
No findings to report. Good job! (1 suppressed)

[woodruffw]

Hi @vitorfloriano, thanks for opening this. I'll take a look.

[woodruffw]

Huh yeah, that's interesting, you may have found a bug here 🙂

Just copying as a MRE:

name: APIDiff

on:
  push:

permissions: {}

jobs:
  go-apidiff:
    name: Verify API differences
    runs-on: ubuntu-latest
    steps:
      - name: Setup Go
        uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v9.9.9

[woodruffw]

Oh, this is my own doing:

zizmor/crates/zizmor/src/audit/ref_version_mismatch.rs

Lines 107 to 117 in b06d448

	let Some(commit_for_ref) = self
	.client
	.commit_for_ref(uses.owner(), uses.repo(), version_from_comment)
	.await
	.map_err(Self::err)?
	else {
	// TODO(ww): Does it make sense to flag this as well?
	// This indicates a completely bogus version comment,
	// rather than a mismatch.
	return Ok(findings);
	};

TL;DR is that if the comment isn't a git reference at all, we currently silently ignore it. I think the reason I did that originally was because people sometimes put random comments on the uses: line and I didn't want to add more noise, but given that we match on version patterns I see no harm in actually flagging this as well.

[woodruffw]

Opened #1853. Thanks @vitorfloriano!

[vitorfloriano]

Thanks for looking into this @woodruffw !