test framework: Orchard -> Ironwood migration test scaffolding#172
Draft
dannywillems wants to merge 23 commits into
Draft
test framework: Orchard -> Ironwood migration test scaffolding#172dannywillems wants to merge 23 commits into
dannywillems wants to merge 23 commits into
Conversation
Add wallet_ironwood_migration.py, scaffolding for the Orchard -> Ironwood value-pool migration (NU6.3, ZIP 2005). The migration itself is still being built: zallet will expose the command, and the librustzcash note-split planner crate zcash_ironwood_migration_backend is still evolving. The module therefore registers the end-to-end flow and its fixtures without asserting any final behavior, so it stays independent of the final zallet/librustzcash API. It reuses the deferred-activation pattern from wallet_ironwood_activation.py to mint a real v2 Orchard note as the migration source, then marks TODO points where the migration command and its post-conditions will plug in. The denomination policy is deliberately not pinned, since it is still in flux. Two things keep it from failing the suite while unimplemented: it is listed in DISABLED_SCRIPTS so the runner does not spend a coinbase-maturity window on a test that can only skip today (the framework's equivalent of skip/xfail: registered in NEW_SCRIPTS but excluded from the run lists), and if run directly it self-skips by probing for the migration RPC and returning cleanly when none is found. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n test
zallet is gaining a generic pool-migration JSON-RPC parameterized by
from_pool/to_pool (backed by the librustzcash zcash_ironwood_migration_backend
note-split planner). Point the scaffolding's capability probe at the real
method names and drive it generically as orchard -> ironwood.
The probe now resolves each migration method (start, status, advance,
cancel, list) from a small per-method candidate list, so it stays robust
while zallet settles the exact names; the start method is the capability
gate. The flow grows in stages:
* Stage 0 (method absent): self-skip cleanly (exit 0), as before. The
currently-shipping zallet build lands here.
* Stage 1 (method present, engine not landed): assert the RPC surface
only -- start is callable for orchard -> ironwood and either returns a
plan-summary object or a clear "not implemented yet" response, and an
unsupported pool pair is rejected with an error.
* Stage 2 (engine landed): the fund_orchard_source fixture and the TODO
block mark where the funded end-to-end drive-and-assert goes. The
denomination policy is deliberately not pinned; assert only
conservation-style invariants once it is finalized.
Kept registered in NEW_SCRIPTS and listed in DISABLED_SCRIPTS. Running the
module against the current build self-skips and exits 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Exercise z_previewpoolmigration end to end: fund an Orchard note pre-NU6.3, cross the activation boundary, then assert input validation (an unsupported pool pair and an unknown strategy are rejected), a well-shaped plan, and value conservation for both the default and canonical strategies. The denomination policy is deliberately not pinned, so the test survives strategy tuning. Stage 0 self-skips when the running zallet lacks the method, so it is safe against older builds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The zallet preview RPC was rewired onto the migration engine's planning slice (zcash_pool_migration_backend::engine::plan_migration), which changes the response contract. Update the test to match: - Drop the strategy parameter and its validation: the engine no longer exposes a denomination-strategy choice, so the method takes only account, from_pool, to_pool, and an optional minconf. - Assert the new plan shape: funding_notes carrying each note's crossing value, self-funding output, and transfer schedule (broadcast height and expiry); plus the note_split and preparation summaries. - Replace the old single-transaction conservation identity (which no longer holds now that preparation is a multi-transaction, fee-charging, reconciled plan) with honest invariants: each output funds its crossing plus the fixed transfer fee buffer, the migratable total is the sum of the crossings and never exceeds the balance, funding notes are a subset of the raw split, and every transfer is scheduled at or after the chain tip with an expiry beyond its broadcast height. - The reserved prep fee is now the padded ZIP-317 preparation-transaction fee (PREP_TX_ACTIONS * marginal fee), not the ZIP-317 minimum. The Stage-0 self-skip is unchanged, so the test stays safe against zallet builds that predate the preview RPC. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the fully-wired z_previewpoolmigration test, and update the migration scaffolding entry for the renamed engine crate (zcash_ironwood_migration_backend -> zcash_pool_migration_backend). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add ZIP-318 denomination-structure invariants to the migration preview
test: every crossing in the raw note split is a canonical {1,2,5}*10^k
value in zatoshi (down to the sub-1-ZEC dust floor, so not necessarily a
whole ZEC) and within the denomination cap, the split is non-increasing (a
descending greedy decomposition) summing to its migratable total, and each
scheduled funding note is a canonical denomination. Also assert the shape
is deterministic: a second preview of the same wallet yields the same
multiset of crossings, since only the randomized transfer schedule varies.
Mark the migration preview and generic migration test scripts executable
so the test runner can exec them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the Orchard -> Ironwood preview test with the cases a real caller hits, all asserted against the RPC's actual behavior and verified end-to-end on the zebra backend: - Realistic bad requests are rejected with an RPC error, never a plan: migrating a pool to itself, the reverse (Ironwood -> Orchard) and other unsupported directions, an unsupported source pool (Sapling), and a nonexistent account (in addition to the existing unknown-pool-name case). - A minconf larger than any recent note's confirmations leaves nothing spendable, so a funded account correctly reports nothing to migrate rather than a stale plan (exercises the preview's minconf filter and its empty-balance path). - The preview is side-effect-free: after previewing and every rejected request, the account's spendable Orchard balance is unchanged and no Ironwood notes have been minted. Also note in the source fixture's docstring that the single funded note is split by the engine into the several funding notes/crossings the plan asserts, which is the common real-world case (a wallet consolidates, then migrates). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…inconf More real-world preview cases, all asserted against the RPC's actual behavior and verified end-to-end on the zebra backend: - Previewing BEFORE NU6.3 activates is rejected with a clear "network upgrade required" error (the activation check runs before the balance, so a funded account still gets it) -- the case a user hits by trying to migrate early. - The account resolves by its ZIP-32 index as well as its UUID; both name the same account and yield the same plan. - A modest minconf still includes the well-confirmed source note (the included side of the confirmation filter), complementing the existing unreachable-minconf case on the excluded side. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ming The start/status/list methods are now registered as scaffold stubs, so this lifecycle test runs Stage 1 rather than self-skipping. Two changes: - Cross the NU6.3 boundary before the Stage-1 surface checks. The start method validates the pool pair (which requires the upgrade) before returning its scaffold response, so running the check pre-activation got an "upgrade not active" error instead of the expected scaffold answer. Activate first so the surface checks reach the real response. - Assert the status and list methods' surface too (callable, returning a structured answer or a clear "not implemented yet"), the read-only half of the lifecycle surface assertable before the engine can run a migration. status takes a migration id, so probe it with a placeholder. Also enumerate the funded Stage-2 lifecycle scenarios in the run_test TODO -- the full run, resume-after-restart, status/list reflecting progress, and expiry/reorg recovery -- so the drive-and-assert is specified for when the engine's commit/broadcast slices land in zallet. Verified end-to-end on the zebra backend: the surface checks pass against the current stubs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The zallet pool-migration RPC interface is stable, so the per-method lists of candidate names (a hedge from when the naming was unsettled) are collapsed to a single fixed name each: START_RPC, STATUS_RPC, ADVANCE_RPC, CANCEL_RPC, LIST_RPC, and PREVIEW_RPC. The list-probing resolve_rpc_name helper becomes resolve_method, which probes one name and returns it or None. Behavior is unchanged (the real names were always the first candidate); both tests still pass end-to-end on the zebra backend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the RPC-surface stub flow with the real funded lifecycle now that z_startpoolmigration and z_advancepoolmigration are wired: fund an Orchard source pre-NU6.3, cross the boundary, start the migration, then advance one step per call (mining and syncing between steps) until it completes, asserting that Ironwood notes appear and the Orchard balance drains. The drive gates on the preview's layer_count: commit_preparation currently supports only single-layer preparation, and plan_migration produces a multi-layer (3-layer) preparation for every balance tried (1 to 137 ZEC), so the test skips cleanly with a clear message rather than failing. Once multi-layer preparation is supported the drive runs unchanged. Finding to investigate: the engine's standalone end-to-end test gets a single layer for 78 ZEC, but zallet's plan_migration over the funded wallet gets three layers for the same nominal balance; the wallet's actual Orchard note set (the source note plus any change/dust) likely differs from the one clean note the engine test uses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The engine now commits a multi-layer preparation phase by phase (each later layer built once its predecessor mines), so the test no longer skips balances that fan out across layers. Remove the single-layer preview gate and let the advance loop drive the whole lifecycle. Verified on a zebra regtest node: a 137 ZEC Orchard source produces a 3-layer preparation and 10 funding notes, and the migration completes with 10 Ironwood notes minted and the Orchard balance drained. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dannywillems
commented
Jul 18, 2026
| LIST_RPC = 'z_listpoolmigrations' | ||
|
|
||
|
|
||
| def orchard_notes(wallet, minconf=1): |
Contributor
Author
There was a problem hiding this comment.
This can be moved into utils.py
dannywillems
commented
Jul 18, 2026
|
|
||
| # ---- capability probing ------------------------------------------------ | ||
|
|
||
| @staticmethod |
Contributor
Author
There was a problem hiding this comment.
This can be moved in the TestFramework class directly.
dannywillems
commented
Jul 18, 2026
| return (e.error.get('code') == RPC_METHOD_NOT_FOUND | ||
| or 'method not found' in msg) | ||
|
|
||
| def resolve_method(self, w, name, *probe_args): |
Contributor
Author
There was a problem hiding this comment.
This can be moved in the TestFramework class directly.
dannywillems
commented
Jul 18, 2026
|
|
||
| # Report which of the companion methods this build exposes | ||
| # (informational; only `start` gates the flow). | ||
| status_name = self.resolve_method(w, STATUS_RPC, UNSUPPORTED_POOL) |
Contributor
Author
There was a problem hiding this comment.
Could we not use directly the method name?
dannywillems
commented
Jul 18, 2026
| # boundary so the migration is enabled. | ||
| acct = w.z_listaccounts()[0]['account_uuid'] | ||
| print("Funding an Orchard source note pre-NU6.3...") | ||
| orchard_before = self.fund_orchard_source(node, w, taddr, acct) |
Contributor
Author
There was a problem hiding this comment.
Could be nice to generalize this method, with a number of notes and the total balance.
A catalog of real-world integration-test scenarios for the pool migration: personas (small holder, retail, whale, exchange), note distributions (single, many, dust-heavy, mixed), the multi-layer anchor-bucket dependency invariant, the mobile-wallet next-action/resume surface, and edge cases (reorg, expiry, cancel, stall, concurrent deposit, ceiling, minconf, pre-activation, read-only). One scenario per file once agreed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Record that the migration state machine, state transitions, advance decision, and next-actions/status view live in the librustzcash engine crate (the mobile wallet consumes it directly; zallet is a thin consumer), so scenarios assert shared-engine behavior through the zallet regtest harness. Note the two existing tests are the first harness entries and are activated once the catalog is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The transfers are broadcast on a randomized privacy schedule, so a run can need more advance steps than a tight schedule; raise the step and per-step mine budget so the lifecycle completes for any schedule spread (verified end to end: a 137 ZEC source, 3-layer preparation, completes at step 100 with 10 Ironwood notes). Add ironwood_migration_common.py, the shared scenario harness: funding by wallet shape, the advance/mine drive loop, and asserters for value conservation, the multi-layer anchor-bucket ordering, and the engine's machine-readable next-actions surface. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…enarios Two fast validation scenarios on the shared harness: starting a migration for an account with no spendable source-pool balance is rejected cleanly (D8), and a second start while one is in progress is refused so an in-flight migration's pre-signed transactions are never discarded (D12). Both verified on a regtest zebra node. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Status, advance, and cancel must reject an id that does not name the wallet's migration, both with no migration in progress and while one is running, so a wallet polling or resuming with a stale id gets a clear error rather than a crash or a silent no-op. Verified on a regtest zebra node. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drive a multi-layer migration and, at every advance step, assert the engine's machine-readable next-actions surface: a ready transaction names an action and is unblocked; a waiting one reports its blocker; and the anchor-bucket ordering holds both ways (a later layer is never ready, and is never built, while any transaction of its predecessor layer is unmined). Fold the anchor-bucket check into the per-snapshot consistency assertion (a built layer-N implies a mined layer-(N-1)), which is correct without cross-step timing. Verified end to end: a 3-layer migration completes with the ordering held across all layers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Starting a migration before NU6.3 is active is rejected with a clear error that names the required upgrade, even when the account holds a real Orchard note. Verified on a regtest zebra node. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cancel a migration mid-flight (after some layer-0 transactions are broadcast), then assert it becomes terminal (phase 'cancelled') and a subsequent advance is a no-op that does not resurrect it. This surfaced a real engine bug, now fixed: recompute_status revived a cancelled (Failed) migration to InProgress on the next advance. Verified on a regtest zebra node with the fixed engine. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the balance correctness into each scenario file: the harness provides only data helpers (ironwood_balance_zat, orchard_balance_zat), and each scenario asserts inline the source balance, every individual Ironwood note value (via the valueZat field), that the note count equals the plan's crossing count, value conservation, the fee bound, and the sub-dust residual, so a reviewer is convinced by reading one file. Verified: a 137.5 ZEC source crosses into the canonical 1-2-5 note set summing to the source minus a bounded fee and residual. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
qa/rpc-tests/wallet_ironwood_migration.py: scaffolding for the Orchard -> Ironwood value-pool migration flow (NU6.3). It stands up the regtest fixtures (zebrad + zaino + zallet), mints a real Orchard note before activation, crosses the NU6.3 boundary so the Ironwood pool is live, and marksTODO(migration)/TODO(assert)points where the still-evolving zallet migration RPC and thezcash_ironwood_migration_backendplanner will plug in. It is deliberately independent of the final API: the{1,2,5}*10^kdenomination policy is not pinned, and the test self-skips (exit 0) when no migration RPC is exposed. Registered inDISABLED_SCRIPTSso the suite does not run it until the RPC lands.Draft; part of the Ironwood migration effort. Do not merge yet.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com