ci(repo): migrate Cargo-Rail planning to v0.25

[loadingalias]

Which issue does this PR address?

Closes #(N/A)

Rationale

I am the maintainer of Cargo-Rail. I simply wanted to alleviate the extra work for the Iggy team across the latest (v0.25.0 release).

Iggy's pre-merge workflow consumes a retired Cargo-Rail plan format, while post-merge image selection maintains a separate affected-change policy. These paths can drift from Cargo-Rail's dep graph and from each other.

Planning remains an optimization; invalid/unavailable plans must preserve Iggy's existing workspace wide and all-image behavior.

What Changed?

Pre-merge and post-merge planning now use cargo-rail-action v8.2.0, pinned by commit, w/ its validated v8 plan reader. Test selection uses typed Cargo arguments, and integration tests include their exact runtime bin and connector-lib pre-req's.

Edge-image selection now comes from a five-variant Cargo-Rail catalog instead of separate shell and publish-config gates. Action, checkout, plan, reader, and matrix failures fall back to full workspace testing or building all five images.

Local Execution

• Passed
• Pre-commit hooks not run; the applicable checks were run directly

Validation performed locally:

• Cargo-Rail v0.25.0 plan and checkout validation
• scoped, workspace, skipped, planner-failure, reader-failure, selected-image, and all-image paths
• cargo fmt --all -- --check
• cargo clippy --all-targets --all-features -- -D warnings
• cargo check --all --all-features
• cargo build --locked
• cargo test --locked --no-run
• doctests and docs
• repo Taplo, ShellCheck, YAML, JSON, & Actionlint

The full external-service integration suite was NOT completed locally. Its Docker-backed connector fixtures require services and resources beyond this workflow-planning change; all Rust test targets compiled successfully.

AI Usage

1. Tool: Codex
2. Scope: Analysis of the workflows within Iggy that were touched by Cargo-Rail, fallback integration/testing, and some validation command launch points.
3. Verification: I reviewed everything. The applicable Rust, workflow, config, and repo checks listed above passed locally.
4. Can you explain every line if asked? Yes. I am the maintainer of Cargo-Rail.

[github-actions]

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

• /ready - back to S-waiting-on-review after addressing feedback
• /author - flip to S-waiting-on-author while you finish changes
• /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

[loadingalias]

Aside from this PR, I evaluated Cargo-Rail v0.25 against Iggy in an isolated Linux build.

unify produced one concrete follow up: promote server, iggy-gateway-kafka, and iggy_connector_doris_sink into workspace.dependencies, then convert 33 existing declarations across 10 manifests to workspace inheritance. It preserved the optional mimalloc and strum feature edges and found no unused dependencies, dead features, duplicate version opportunities, version mismatches, or transitive pins.

If the dev team are using cargo-udeps, cargo-msrv, cargo-workspace-hack, etc. - you can free the resources on those dev machines now. Cargo-Rail unify will cover it.

I would keep it sparse for now:

[unify]
exact_pin_handling = "preserve"

That protects Iggy's exact compio pin while retaining Cargo-Rail's conservative defaults. Because ordinary analysis currently trips Iggy's -D unused-crate-dependencies policy, I would treat this as a reviewed cleanup PR rather than enable unify in CI yet.

I would like to slowly get the other workflows integrated, but I can't recommend them until I have more testing/benching time.

• Cache needs an isolated benchmark against Iggy's current Swatinem/rust-cache setup. However, Cargo-Rail caching is restoring more locally and remotely that sccache in current testing. I would almost bet a 50% reduction in build times across the entire workspace.

• Surface in v0.25 stops while acquiring facts for the tokio::task_local! declaration in leader_aware.rs; I've patched in v0.25.1, but haven't cut the release yet. No intergration for Iggy under 'surface' just yet.

• I think the reviewed .changes files look useful as a standalone release intent/notes pilot, too. I have a few fixes to make under the full release workflow; Iggy exposed shortcomings in the current. Cargo-Rail cannot currently preserve Iggy's four tag formats or broader Apache and cross-language release process.

If you wanted me to submit the PRs, I think I'd prefer one small dep-inheritance PR followed by the .changes pilot. I will make sure cache gets a measured experiment in the next few days, Surface gets patched, and full release automation stays out for now.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.80%. Comparing base (5916e6f) to head (2c562d6).

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3999      +/-   ##
============================================
- Coverage     84.91%   84.80%   -0.11%     
  Complexity     1405     1405              
============================================
  Files          1224     1224              
  Lines        179301   179301              
  Branches     145615   145615              
============================================
- Hits         152250   152057     -193     
- Misses        23024    23218     +194     
+ Partials       4027     4026       -1     

Components	Coverage Δ
Rust Core	85.80% <ø> (ø)
Java SDK	67.35% <ø> (ø)
C# SDK	75.38% <ø> (-0.02%)	⬇️
Python SDK	90.06% <ø> (ø)
PHP SDK	85.65% <ø> (ø)
Node SDK	94.51% <ø> (-1.63%)	⬇️
Go SDK	69.41% <ø> (+0.15%)	⬆️
see 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hubcio]

hi,
as a apache project we cannot use actions that are not approved in https://github.com/apache/infrastructure-actions due to supply attack risk.

[hubcio]

btw, unify sounds nice. can we use it without action, via something fast, like cargo binstall? (+pin the version)

and thanks for contribution!

[hubcio]

have you already joined our discord?

[loadingalias]

Hey @hubcio.

Thanks for the quick review. I genuinely didn't realize that the ASF projects had the guard in place. It makes sense.

So, I'm going to prepare and submit the cargo-rail-action to the ASF for review. I've got some cleanup and patches to make w/ respect to the compiler fight I'm in, but when it's green - I will submit. I don't see it taking longer than today, or maybe tomorrow.

I'm really glad that I ran cargo-rail across Iggy. I've found a handful of simple bugs and one significant issue w/ the compiler work. These will get patched in v0.26.0.

With respect to 'unify' - yes... it's independent of the GHA. A simple cargo binstall cargo-rail does the trick. You can run cargo rail unify --check to see what's what. unify --check doesn't modify manifests; it exits 1 when it finds a pending dep-coherence change.

For this PR, I can either revise it to avoid the external action for now or return with the exact approved SHA after ASF review... whatever is less disruptive for you guys. TBH, I would prefer to return post ASF review. Just give me a day or two?

Discord - no, I find it to be a time sink and I just don't have that luxury. Cargo-Rail is a side quest because my own storage engine & query engine work needed it. I say this to say... I'm just too busy. Haha. I will join, though. It's easier to communicate there, for sure.

[hubcio]

no worries. you can take your time. we're almost done with cluster-related code and it's even more convenient to put non-production code further away in time, due to incoming release.

thanks for contribution!