Skip to content

chore(deps): three of the four manifest bumps; safetensors is blocked upstream - #71

Merged
ciresnave merged 1 commit into
mainfrom
deps-manifest-bumps
Sep 10, 2026
Merged

chore(deps): three of the four manifest bumps; safetensors is blocked upstream#71
ciresnave merged 1 commit into
mainfrom
deps-manifest-bumps

Conversation

@ciresnave-bot

@ciresnave-bot ciresnave-bot commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

⚠️ Stacked on #70 (base: deps-to-latest). Retarget to main once that lands.

bump status
sha2 0.10 → 0.11 (mlmf-conformance)
sysinfo 0.38 → 0.39
tokenizers 0.22 → 0.23
safetensors 0.7 → 0.8 BLOCKED UPSTREAM

⚠️ safetensors 0.8 cannot be taken here, and it is not our call

Measured, not inferred:

error[E0277]: the trait bound `&candlelight::Tensor: View` is not satisfied
  --> src/formats/safetensors_export.rs:81
note: there are multiple different versions of crate `safetensors` in the
      dependency graph

cargo tree -i safetensors:
  safetensors v0.7.0
  +-- candle-core v0.10.2
      +-- candle-nn / candle-transformers
          +-- candlelight (git, rev d70a03f3)
              +-- mlmf

candlelight implements View against safetensors 0.7. Bumping ours puts
two versions in the graph, and a trait implemented against one is not the
trait required by the other. The blocker is candle-core's pin, reached through
candlelight — upstream of this repository.

So the standing rule is unsatisfiable for safetensors until candlelight moves.
Recorded here rather than left as an unexplained lag: a rule that cannot be met
needs its reason attached
, or the next person re-derives this error from scratch.

sha2 0.11 broke one call site, and the fix had to be byte-identical

finalize() returns Array instead of GenericArray, and Array does not
implement LowerHex — so format!("{:x}", h.finalize()) is a compile error. One
site: dump_gguf_meta.rs:65.

⚠️ That digest is a cross-project contract with the lightbulb lane's dumper,
so the replacement had to produce the same bytes, not merely valid hex. Verified
at two levels:

1 — against Python's hashlib, a different language and implementation:

""                      e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
"abc"                   ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad
"gguf-metadata-dump/v1" a485606325d35b784428cd0939b04ca8e1fc3ee06610e307d3d8b370dcc3a6a9

2 — the whole artifact, old code vs new. A detached worktree at the previous
commit (sha2 0.10) ran the dumper over the same 19-file corpus; cmp reports the
two 10,099-byte dumps IDENTICAL, all 7 emitted digests included.

(2) is the one that matters. (1) proves the encoder is correct; only (2) proves
the artifact did not move.

⚠️ I nearly shipped a comment citing a test that does not exist

My first draft said the round-trip was "pinned by
the_digest_is_lowercase_hex_of_the_utf8_body below"
. There is no such test,
and no tests in that file at all.

And writing one there would not have helped: CI runs cargo test -p mlmf-conformance, which executes the lib unit tests, tests/* and doc-tests —
measured, and example targets are not among them. A #[test] in an example
reads as coverage and runs never.

The comment now states what is true: verified by an independent oracle and by
artifact comparison, not by a test, with the reason a test cannot live there
and what it would take to have one.

Verification

cargo clippy --workspace --all-targets --all-features   exit 0
cargo fmt --all --check                                 exit 0
cargo test -p mlmf --lib                                106 passed
cargo test -p mlmf --doc                                29 passed
cargo test -p mlmf-core                                 exit 0
cargo test -p mlmf-conformance                          exit 0
cargo test -p mlmf-hf-layout                            exit 0

🤖 Generated with Claude Code

https://claude.ai/code/session_01MdVuiraXRfDHQ227cjBt51

Summary by Sourcery

Refresh the available dependency versions and preserve conformance artifact compatibility while documenting the blocked safetensors upgrade.

Bug Fixes:

  • Update the GGUF metadata digest generation for sha2 0.11 while preserving byte-identical output.

Enhancements:

  • Record the upstream safetensors 0.8 compatibility blocker through candlelight and document the digest verification rationale.

Build:

  • Bump sysinfo to 0.39, tokenizers to 0.23, and sha2 to 0.11 while refreshing the lockfile.

Tests:

  • Validate the dependency updates and digest compatibility across workspace linting, formatting, unit tests, documentation tests, and conformance checks.

Chores:

  • Leave safetensors at 0.7 until the upstream candlelight dependency moves to a compatible version.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @ciresnave-bot, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 5 days and 11 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@sourcery-ai

sourcery-ai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR updates three dependency manifests and the lockfile, while intentionally deferring safetensors 0.8 due to an upstream candlelight/candle-core pin. It also adapts the sha2 0.11 digest API with byte-identical lowercase hex output and records independent and artifact-level verification of the compatibility-sensitive dumper output.

Sequence diagram for byte-compatible GGUF metadata hashing

sequenceDiagram
    participant Dumper as dump_gguf_meta
    participant Sha2 as sha2::Sha256
    participant Artifact as GGUF metadata dump
    participant Oracle as Python hashlib

    Dumper->>Sha2: update(body.as_bytes())
    Dumper->>Sha2: finalize()
    Sha2-->>Dumper: Array digest bytes
    Dumper->>Dumper: write each byte as {b:02x}
    Dumper->>Artifact: emit lowercase hexadecimal digest
    Oracle-->>Dumper: matching hexdigest vectors
    Dumper-->>Artifact: byte-identical output verified
Loading

File-Level Changes

Change Details Files
Refresh three dependency versions while documenting why the fourth remains pinned.
  • Upgrade sysinfo and optional tokenizers dependencies.
  • Upgrade sha2 for the conformance crate.
  • Leave safetensors at 0.7 because candlelight/candle-core requires that version, making a 0.8 migration fail with incompatible View traits.
  • Regenerate the lockfile for the dependency updates.
Cargo.toml
crates/mlmf-conformance/Cargo.toml
Cargo.lock
Adapt GGUF metadata digest generation to the sha2 0.11 API without changing artifact bytes.
  • Replace LowerHex formatting of the finalized digest with explicit lowercase, zero-padded per-byte encoding.
  • Document verification against Python hashlib and byte-for-byte comparison of old and new 19-file artifacts.
  • Document that tests in the example target are not executed by the existing conformance test command.
crates/mlmf-conformance/examples/dump_gguf_meta.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results
Complexity 0
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@ciresnave-bot
ciresnave-bot changed the base branch from deps-to-latest to main September 10, 2026 17:56
… upstream

Continues the freshness pass. The two range-permitted upgrades landed in the
previous commit; these are the four that needed a manifest change.

  sha2        0.10 -> 0.11   (mlmf-conformance)
  sysinfo     0.38 -> 0.39
  tokenizers  0.22 -> 0.23
  safetensors 0.7  -> 0.8    ✗ BLOCKED, see below

## ⚠️ safetensors 0.8 CANNOT BE TAKEN HERE, AND IT IS NOT OUR CALL

Measured, not inferred:

  error[E0277]: the trait bound `&candlelight::Tensor: View` is not satisfied
    --> src/formats/safetensors_export.rs:81
  note: there are multiple different versions of crate `safetensors` in the
        dependency graph

  cargo tree -i safetensors:
    safetensors v0.7.0
    +-- candle-core v0.10.2
        +-- candle-nn / candle-transformers
            +-- candlelight (git, rev d70a03f3)
                +-- mlmf

`candlelight` implements `View` against safetensors 0.7. Bumping ours puts TWO
versions in the graph, and a trait implemented against one is not the trait
required by the other. THE BLOCKER IS candle-core's PIN, reached through
candlelight -- upstream of this repository.

So CireSnave's standing rule ("all dependencies on their most recent versions at
all times") is UNSATISFIABLE for safetensors until candlelight moves. Recorded
here rather than left as an unexplained lag: a rule that cannot be met needs its
reason attached, or the next person re-derives this error from scratch.

## sha2 0.11 broke one call site, and the fix had to be byte-identical

`finalize()` now returns `Array` instead of `GenericArray`, and `Array` does not
implement `LowerHex` -- so `format!("{:x}", h.finalize())` is a compile error.
One site: `dump_gguf_meta.rs:65`.

⚠️ THAT DIGEST IS A CROSS-PROJECT CONTRACT with the lightbulb lane's dumper,
so the replacement had to produce the SAME BYTES, not merely valid hex.
Verified twice, at two levels:

  1. against Python's hashlib on three vectors -- a different language and a
     different implementation:
       ""                      e3b0c442...b7852b855
       "abc"                   ba7816bf...f20015ad
       "gguf-metadata-dump/v1" a4856063...dcc3a6a9

  2. THE WHOLE ARTIFACT, old code vs new. A detached worktree at the previous
     commit (sha2 0.10) ran the dumper over the same 19-file corpus; `cmp`
     reports the two 10,099-byte dumps IDENTICAL, all 7 emitted digests
     included.

(2) is the one that matters. (1) proves the encoder is correct; only (2) proves
the ARTIFACT did not move.

## ⚠️ AND I NEARLY SHIPPED A COMMENT CITING A TEST THAT DOES NOT EXIST

My first draft of that comment said the round-trip was "pinned by
`the_digest_is_lowercase_hex_of_the_utf8_body` below". THERE IS NO SUCH TEST,
and there are no tests in that file at all.

Worse, writing one there would not have helped: CI runs
`cargo test -p mlmf-conformance`, which executes the lib unit tests, `tests/*`
and doc-tests -- measured, and EXAMPLE TARGETS ARE NOT AMONG THEM. A `#[test]`
in an example reads as coverage and runs never.

The comment now states what is actually true: verified against an independent
oracle and by artifact comparison, NOT by a test, with the reason a test cannot
live there and what it would take to have one.

VERIFIED:
  cargo clippy --workspace --all-targets --all-features   exit 0
  cargo fmt --all --check                                 exit 0
  cargo test -p mlmf --lib                                106 passed
  cargo test -p mlmf --doc                                29 passed
  cargo test -p mlmf-core                                 exit 0
  cargo test -p mlmf-conformance                          exit 0
  cargo test -p mlmf-hf-layout                            exit 0

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdVuiraXRfDHQ227cjBt51
@ciresnave
ciresnave merged commit 76b0fde into main Sep 10, 2026
6 checks passed
@ciresnave
ciresnave deleted the deps-manifest-bumps branch September 10, 2026 21:09
ciresnave pushed a commit that referenced this pull request Sep 10, 2026
Closes the last dependency-freshness item in this workspace.

  candlelight rev  d70a03f -> 3009b2d   (ciresnave/candlelight#1, merged)
  safetensors      0.7     -> 0.8

## Why this could not be done in #71

`candlelight` implemented `View` against safetensors 0.7. Bumping ours put TWO
versions of safetensors in one graph, and a trait implemented against one is not
the trait required by the other:

  error[E0277]: the trait bound `&candlelight::Tensor: View` is not satisfied
  note: there are multiple different versions of crate `safetensors` in the
        dependency graph

candlelight now depends on candle 0.11.0, which is the first release requiring
safetensors ^0.8.0.

## Verified

  cargo build --workspace --all-features                 exit 0
  cargo clippy --workspace --all-targets --all-features  exit 0
  cargo fmt --all --check                                exit 0
  cargo test -p mlmf --lib                               106 passed
  cargo test -p mlmf --doc                               29 passed
  cargo test -p mlmf-core                                exit 0
  cargo test -p mlmf-safetensors                         exit 0
  cargo test -p mlmf-conformance                         exit 0

  safetensors 0.7 edges in the graph                     ZERO
  safetensors 0.8 edges                                  3

⚠️ The `safetensors v0.4.0` row a substring match reports is OUR OWN
`mlmf-safetensors` crate at the workspace version, not a third safetensors.
Checked rather than assumed, because a version count is exactly the figure that
gets quoted without its subject.

## ⚠️ THIS PATH WAS PROVEN BEFORE candlelight#1 WAS OPENED

While scoping the candlelight change I built mlmf against a local path patch of
that branch, with safetensors already at 0.8: workspace build exit 0, 106 lib
tests, no 0.7 edge. So this commit is the SAME measurement re-run against the
published rev rather than a fresh hope -- the only variable that changed is
whether the code came from a path or from a git rev.

That is why the upstream PR was worth the round trip: the consumer-side check is
the one that answers "does this unblock anything", and it was available before
the upstream change existed.


Claude-Session: https://claude.ai/code/session_01MdVuiraXRfDHQ227cjBt51

Co-authored-by: MLMF Agent <ciresnave+mlmf@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants