Skip to content

sovereign-ci: 'Supply chain audit (cargo deny)' fails on the runner when it cannot clone RustSec advisory-db — 6 of 6 red CI runs on paiml-mcp-agent-toolkit today, two on master #57

Description

@noahgift

What fails

sovereign-ci.yml step Supply chain audit (cargo deny) (cargo deny check advisories licenses sources, ~line 546) fails on the self-hosted runners with:

[ERROR] failed to fetch advisory database https://github.com/RustSec/advisory-db with cli: failed to clone: ["clone", "--depth=1", "--branch", "main", "https://github.com/RustSec/advisory-db", "/usr/local/cargo/advisory-dbs/advisory-db-3157b0e258782691"]
Cloning into '/usr/local/cargo/advisory-dbs/advisory-db-3157b0e258782691'...
fatal: could not read Username for 'https://github.com': No such device or address
fatal: expected flush after ref listing
##[error]Process completed with exit code 1.

git is being asked for a username on an anonymous clone of a public repository — i.e. GitHub answered the ref listing with an auth challenge (rate limit / secondary limit for the runner IP, or a credential helper intercepting), and with no tty the clone dies. ci / lint goes red, ci / gate follows, and the PR is blocked.

Evidence — 2026-09-02, paiml/paiml-mcp-agent-toolkit

Every red ci.yml run today whose ci / lint job failed, grepped for the string above:

run branch advisory-db clone failure
33629705363 master yes
33626740377 master yes
33618636676 master yes
33626947080 dependabot/cargo/uuid-1.26.0 yes
33618875436 dependabot/cargo/uuid-1.26.0 yes
33645035902 (in progress) docs/crux-architecture-audit — a docs-only PR yes

The one red run not caused by this (33618987347) was a genuine compile error. The same clone succeeds from this host outside the runner container (git ls-remote https://github.com/RustSec/advisory-db main → reachable), and it succeeds on other runs minutes later, so it is intermittent, not a network outage.

Why it matters

This is the only blocking supply-chain gate the fleet has (per the step's own comment). Failing closed is the right default — but a gate that goes red on network weather trains everyone to rerun --failed until green, which is the same reflex that lets a real advisory through. Two master runs red in one day on a step that measured nothing about the code is the wrong kind of signal.

Proposed fix (in this workflow, not in the ~19 consumers)

Any of these would do; the first is the smallest:

  1. Authenticate the clone. cargo-deny shells out to git, so give it the job token:
    env:
      GIT_CONFIG_COUNT: 1
      GIT_CONFIG_KEY_0: url.https://x-access-token:${{ github.token }}@github.com/.insteadOf
      GIT_CONFIG_VALUE_0: https://github.com/
    on the deny step. Authenticated requests are not subject to the anonymous limits.
  2. Fetch with retry, then check offline. for i in 1 2 3; do cargo deny fetch && break; sleep 20; done followed by cargo deny check --disable-fetch advisories licenses sources, so a transient failure is retried and a persistent one is still red.
  3. Cache the advisory DB between runs (actions/cache on /usr/local/cargo/advisory-dbs), so a failed fetch degrades to a stale-but-present DB with a ::warning::, and only a missing DB is fatal.

Whichever lands, the step should say why it is red: "could not fetch advisory database" is a different failure from "advisory found", and today they share an exit code.


Found while running the pmat CRUX audit's release pipeline. Related: #48 (GHSA blind spot) is about what this gate cannot see; this issue is about the gate not running at all.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions