Repository navigation
feat(mock-host): a TrUAPI-native test host - #294
Conversation
4a1e425 to
0d90893
Compare
0d90893 to
fcda658
Compare
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
CI Status: 21 required jobs green, 20 passed and 1 skipped by path filter. All job results
Commit |
3d799fd to
9932a60
Compare
iOS simulator previewBuilt from gh run download 35658128216 --name simulator-preview-ab65421f9
unzip polkadot-app-*.app.zip
xcrun simctl install booted polkadot-app.app
xcrun simctl launch booted io.parity.polkadotapp.developOr download it in a browser, which arrives as a zip wrapping An arm64 simulator slice, so it needs an Apple Silicon Mac and does not |
9932a60 to
62c7b49
Compare
5ec83ef to
a4eff89
Compare
…reads and more faults
3e32e18 to
99f3133
Compare
… guards able to fail
# Conflicts: # rust/crates/truapi-server/src/runtime.rs
TarikGul
left a comment
There was a problem hiding this comment.
Ran a few agents over this alongside my own pass. The surface guard is genuinely good, both drift directions and the vacuity floor all fire when I break them, but the two halves still diverge in behaviour in ways it cannot see. Details inline.
Also the description's codec 2 / 0.16.0 framing is stale, codec was already 3 at this branch's merge-base and the packages are 0.17.0 in the PR itself.
# Conflicts: # .github/workflows/ci.yml # js/packages/truapi-host/src/web/create-worker-host-runtime.ts # rust/crates/truapi/src/lib.rs
…op three false greens
… revive two tests that could not fail
# Conflicts: # js/packages/truapi-host/package.json # package-lock.json
The two mocks disagreed on three things a conformance suite would hit.
Permission decisions were keyed on the `Display` prose in Rust and on the
SCALE variant tag in TypeScript, so `grantPermission("Camera")` granted in
one and silently missed in the other. `RemotePermission::Remote` rendered
its domain list into the key, which left it impossible to grant before
knowing what the product would ask for. Both sides key on the tag now.
The default chain set was three chains in TypeScript and empty in Rust.
Both serve the same three, and the placeholder genesis hashes are exported
from Rust as `mock_genesis`, so a fixture naming a chain by hash means the
same chain to either.
`subscribe_chat_rooms` ignored `chat_error` while every other chat call
honoured it, so a failing host handed back a stream that looked healthy and
never carried a room.
Three lifecycle fixes on the TypeScript side: a subscription closed before
its first `next()` had no generator body to unwind and stayed registered,
`dispose` left every subscriber in place, and `simulateDisconnect` ended
nothing on a non-proxied connection because only `connectToChain`
registered a disconnector. The default connection registers its own now, so
a product parked on `responses()` learns the transport dropped.
Tests cover the four fault knobs that were wired but unasserted, the
disconnect teardown, and the permission key space.
… the rest of the parity gaps Closing a subscription unregistered its push before waking the body, so a `return()` queued behind a parked `next()` could never run: both promises hung where they previously settled on the next publish. Release now wakes the body and the body checks for it, so a close ends the stream from either side. `dispose` ends every live subscription and chain stream through the same path rather than dropping registrations underneath consumers, which left them parked forever. Two more places where the mocks disagreed. Core storage keyed on the variant tag alone in TypeScript, so all six parameterized slots collided and one product's manifest read back another's, where Rust renders the payload into the key and misses. Chat rooms and bots came back in insertion order against Rust's key order, and that list is protocol payload, not just an assertion oracle. The surface test now reads the genesis constants and the permission key arms out of `mock.rs` and compares them to the TypeScript side, so the hand-written tags cannot drift: renaming a variant updates the match pattern and leaves the string beside it compiling, which is exactly the drift this mock exists to remove. All fourteen variants are pinned. Tests cover the parked close, the chain set both sides serve, the remote permission key, core storage slot addressing, chat ordering, the second confirmation entry point, and that a refused subscription ends rather than stalls.
A TrUAPI-native test host
A mock host products can be tested against, built in this repo: an in-memory mock platform in Rust, its TypeScript sibling, a browser host page that runs the real core, a Playwright fixture over it, and the server that serves them. Built against
@parity/truapi0.16.0 (wire codec 2).What it provides
A mock host on both sides of the boundary.
MockPlatform(truapi-platform, behind themockfeature) andcreateMockHost(@parity/truapi-host/testing) expose the same control surface: recordings of what the core asked the host to do, per-capability permission answers, chat, theme, chain-connection simulation, fault injection, andreset()for isolation between cases. Confirmation reviews are recorded with their payloads, so a test can assert what was put to the user rather than only that something was.The TypeScript surface uses
@parity/host-api-test-sdk's method names, so a suite adopting it changes its import rather than its assertions.A guard against the two halves drifting.
mock-host-surface.test.tsparses theimpl MockPlatformblock out ofmock.rsand fails when a control method exists on one side and not the other, naming the missing member. A second test pins a member floor, so a regex that matched nothing cannot make the check vacuously green. Intentional naming differences live in one explicit alias map.A Playwright fixture on the production topology. The host page runs the core in a Web Worker, the way a web host does, and embeds the product in an iframe over a
MessagePort. It ships from the existing@parity/truapi-hostpackage rather than a new one, on subpaths this branch adds:./testing/playwrightfor the fixture,./testing/serverfor the node server,./testing/clientfor a no-iframe variant a product's own unit tests can use, and./testing/dev-accountsfor the named accounts. No published version carries them yet.productUrlis the only required option: the fixture starts its own host server when none is supplied, shares it across a file, and unrefs it so it cannot hold a worker open. It also accepts@parity/host-api-test-sdk'snetworksshape and expands it into the proxy, chain set and runtime genesis that have to agree. The two options TrUAPI cannot honour --productAccountsand an account given by derivationuri-- are accepted and rejected at construction with what to do instead, rather than being absent and failing as a type error.Dev accounts that sign. The host runs a signing host and establishes sessions from fixed BIP-39 entropy, so a product exercises real sr25519 signatures. Addresses are derived root-from-entropy then per-product, so they differ from polkadot-js
//Alice; the module says so where someone will read it.Live chain, opt-in.
chainProxiesforwards to a real node when a suite needs real state;liveChain()keeps the proxy, the reported chain set and the runtime config's genesis in agreement, since the product checks the last two. The default is hermetic — nothing reaches the network unless a suite asks.Two WASM bundles.
webis the production browser host.testingaddswasm-signing-host, which is what lets the test host own keys. A test asserts the testing bundle has a signing host and the web one does not.A
host-wasmCI job that builds both bundles and runs the bridge suite withREQUIRE_WASM=1, so a missing artefact fails loudly instead of skipping green.What is verified
Nine of product-sdk's example suites, real browser, real core, real wire:
pallet-reviveaccount mapping, a write22 of 22 across the chain-free suites. Adopting the fixture is an import change: the whole diff for eight of the nine suites is the import line in
e2e/fixtures.tsplus a type import ine2e/helpers.ts.Seven tests out of 31 needed more. Five read the old host's internal storage keys rather than product behaviour, and now read through the control surface. One asserted that switching the host account leaves the product account unchanged -- it does not, because TrUAPI derives the product account from the session root, so that is a real behavioural difference rather than churn. One is proposed as skipped: the core answers a decided permission from its own storage, so a revoked permission never re-reaches the host and no test host can make it pass.
Those rewrites are staged as a patch in
docs/migration/, with a cover note. The patch applies cleanly to a fresh clone of product-sdk main, and product-sdk builds against the published@parity/truapi0.16.0 oncecreateFakeHostgains the two members the newer client requires.Wire codec 2 is why this matters now.
@parity/truapi0.16.0 is the first release on codec 2, and the handshake refuses a mismatched codec. product-sdk main has moved to it; the publishedproduct-sdk-host(0.19.1) has not, pinning^0.13.1, which a caret on a0.xversion cannot resolve past.@parity/host-api-test-sdkreimplements the protocol rather than running the core, and it reimplements codec 1. Against product-sdk main it fails the handshake: running main's nine suites on a clean checkout, unmigrated, gives 1 suite passing and 8 failing — and the one iscloud-storage-demo, which is skipped upstream. The same tree on this test host gives 6 passing and 3 failing, the 3 being the chain-write suites above.So this is not a like-for-like swap of one test SDK for another. It is what lets product-sdk's suites run at all once it releases on 0.16.0. The remaining step is a
@parity/truapi-hostrelease carrying the./testingsubpath, which no published version has yet; everything else is verified locally against pristine main with@parity/truapiresolved from npm.What it deliberately does not do
Chain writes.
tx-demoboots against the real Asset Hub, signs through the host and submits, and the chain rejects the transaction withInvalid.Payment: a product account derives per (session entropy, product id), so the addresses hold no balance for fees. Funding them is an operational decision with a standing cost, not a code change.One run instead stalled before reaching the signer and ended on
submitAndWatch's own 300s timeout. That has not reproduced, and every run since reaches signing. An intermittent failure against a public testnet is worth knowing about before someone reads a red CI job as a regression here.The statement store. The store connects over a proxied people chain and its subscription traffic is observable, but publishing is rejected inside the core before any RPC is emitted, because the submitting account needs a statement-store allowance — so it cannot be recorded on the transport either.
getSubmittedStatements,injectStatementandclearStatementsexist on the fixture and throw with that reason.Granting the allowance is
truapi-host alloc-check --target <hex-32> --submit, which requires an onboarded person: the extrinsic proves LitePeople ring membership. The identity available here does not have it —alloc-checkreportsonboarding pendingacross all 26 rings andidentity-checkreturnsIDENTITY_NONE, so the grant cannot be made from this environment. Slots themselves are free (PeopleandLitePeopleboth reportfree seq=0for the current period), so this is a personhood gap rather than a quota one.Payments. The protocol declares them and no host implements them: every method in
capabilities/payment.rsreturns an error and ignores its arguments. The mock throws with that reason.In each case the mock refuses and explains rather than returning something plausible, so a test reaching for an unserved path learns why instead of passing against a fake.
Funding the derived accounts is what
tx-demoandcontracts-demoare waiting on.statement-store-demoneeds something different: a statement-store allowance for an onboarded person, which no amount of host-side work substitutes for.What this needs to land
In this repo, to merge: nothing outstanding. CI is green, the branch carries its own guards, and the Rust and TypeScript halves are checked against each other.
To make it usable, one release:
@parity/truapi-host0.17.0, carrying the./testingsubpath. Additive only — the four published subpaths are untouched and six are added, so nothing already installed changes behaviour. Per this repo's convention that is arelease: @parity/truapi-host 0.17.0PR title.Then product-sdk adopts, using the patch and cover note in
docs/migration/. It applies cleanly to product-sdkmainas it stands; its catalog is already on@parity/truapi0.16.0, so the only line the release changes is the@parity/truapi-hostversion. Verified by installing pristine main with@parity/truapiresolved from npm and the host linked locally: the suites behave exactly as recorded above.Three things that are decisions, not work:
13B6hYAQ…fortx-demo.dot/0,12h1wPZE…forcontracts-demo.dot/0) bringstx-demoandcontracts-demointo reach. Deterministic per (account, product), and a standing cost.statement-store-demoneeds; no host-side work substitutes for it.productAccounts) rework those assertions to read the derived address back instead. That is a real behavioural difference rather than churn, and it is the suite's call.Notes for review
The mock is not a reimplementation of the protocol — it mocks only the platform seam, and the real Rust core runs behind it. That is what makes a product test here exercise the code that ships, including behaviour a host cannot override: the core persists a decided permission per (product, permission) and answers from its own storage, so a permission revoked mid-run never reaches the host.
signer-demo's permission spec is skipped for that reason, with the reason in the skip.dist/anddist/wasm/are gitignored and have no freshness check, so a suite reading them can pass against a bundle older than its source.make wasmandnpm run buildbefore trusting a result that depends on either.Verification
cargo checkclean on both targets, includingwasm32-unknown-unknownwithwasm-signing-host;clippy -D warningsclean;tsc --noEmitclean;bun test195 passed.cargo test360 passed with one failure that predates this branch and is not in its diff:sso_handler_contracts, a trybuild snapshot intruapi-macrospinned to older rustc wording (no variant, associated function, or constantwhere current rustc saysno variant or associated item). It is left unblessed rather than silenced, since blessing it would hide a real drift signal.Every guard in this branch was mutation-tested, and the mutations are the reason several of them exist in their current shape: one statement-filter test passed with its method check deleted until a frame carrying a string parameter was added, and three source-level assertions for the Chat action path were deleted outright after only one proved able to fail.
Measured against products rather than only unit tests: host-playground runs 38 passed / 15 failed, where all 15 are allowance allocation stopping at personhood ring membership — an on-chain identity a dev account does not hold, not a defect in the host. browse's suite fails on
@parity/host-api-test-sdktoo, so it is not a migration blocker.Closes #257
Refs #260
Three of #260's five criteria are still open: no Playwright suite in this repo runs the fixture,
there is no
mock-e2eCI job, and the harness ships from the published@parity/truapi-hostrather than as a private package.