Skip to content

cas-215: harded keypair handling - #169

Merged
lvboudre merged 6 commits into
mainfrom
cas-215
Sep 3, 2026
Merged

cas-215: harded keypair handling#169
lvboudre merged 6 commits into
mainfrom
cas-215

Conversation

@lvboudre

@lvboudre lvboudre commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Harden how the TPU sender's identity secret is held in memory

Motivation

The TPU sender's identity keypair — and every intermediate copy of it produced while loading, converting, and using it — previously lived in ordinary process memory: swappable to disk, and in some cases never zeroed after use. This PR closes that gap end-to-end, from reading a keypair off disk through to signing the QUIC/TLS handshake, without changing the wire formats or on-disk file format anything downstream depends on.

What changed

New yellowstone_jet_tpu_client::identity module (crates/tpu-client/src/identity.rs), replacing direct use of solana_keypair::Keypair throughout the sender:

  • TpuIdentity — a Pubkey plus its derived QUIC client TLS credentials. Cheap to clone (Arc internally, exposed via an explicit insecure_clone() rather than Clone, to make duplication visible at call sites). Implements quinn::crypto::ClientConfig directly, so it can be handed straight to quinn::ClientConfig::new().
    • Ships its own rustls::sign::SigningKey/Signer (MlockedSigningKey) that signs directly from a dedicated, mlocked buffer — bypassing rustls's standard with_client_auth_certring key-loading path entirely, which would otherwise leave an unlocked, never-zeroized copy of the key parsed into ring's internal representation for the connection's whole lifetime.
    • Ships its own new_dummy_x509_certificate, writing the PKCS#8-encoded key directly into a buffer that's mlocked before any byte is copied into it (the solana-tls-utils version returns an ordinary, unlocked Vec).
    • Generalized over a new TpuEd25519SigningKey trait, so it can build from a plain Keypair, a raw ed25519_dalek::SigningKey, or a HardenedKeypair.
  • HardenedKeypair — a keypair whose secret bytes are mlocked from the moment they exist and zeroized on drop. Reads (read_from_reader/read_from_file) go straight into a growable, always-locked scratch buffer instead of solana_keypair::read_keypair's plain String; TryFrom<&[u8]> verifies the public half actually corresponds to the secret half (rejecting a corrupted/tampered keypair) using the same check ed25519-dalek uses internally.
  • GrowableHardenedBuffer — backs HardenedKeypair's file reads. Grows by allocating a new, larger, locked buffer, copying existing bytes over, and zeroizing the old allocation before dropping it — so a resize never leaves a byte outside locked memory.
  • TpuSenderIdentityUpdater::current_identity() — a new Arc<ArcSwap<Pubkey>> shared between the driver and the updater handle, so callers can cheaply read the driver's current identity pubkey without round-tripping through the command-and-control channel (replaces the old tokio::sync::watch-based observer plumbing).

apps/jet:

  • JetIdentitySyncGroup/JetIdentitySyncMember (the old fan-out-to-multiple-members abstraction) are removed; TpuSenderIdentityUpdater now implements JetIdentityUpdater directly, since it was the only real member.
  • Admin RPC identity endpoints (setIdentity, setIdentityFromBytes, resetIdentity) now build a HardenedKeypair instead of a plain Keypair.
  • Removed the Prometheus push-gateway support (ConfigJet::prometheus, spawn_push_prometheus_metrics) — it depended on the old watch::Receiver<Pubkey> identity-observer plumbing that no longer exists. Flagging this explicitly since it's a user-facing config removal, not an internal refactor — worth a second look if anyone relies on it.
  • The stake-metrics refresh task now polls current_identity() on a fixed interval (15s) instead of being woken by the identity-observer channel.

Known, accepted residual exposure

Both MlockedSigner::sign() and HardenedKeypair's consistency check construct a transient ed25519_dalek::SigningKey on the stack to do the actual crypto operation — ed25519-dalek's API has no way to sign/derive using externally-owned, already-locked memory. That transient value self-zeroizes on drop (ZeroizeOnDrop) and is gone within a handful of instructions; it's a materially smaller exposure than the previous status quo (heap-resident inside ring, for the connection's entire lifetime, never zeroized), but it isn't literally zero. Closing it further would mean hand-rolling Ed25519 point arithmetic ourselves — a deliberate call not to make, for the same "don't roll your own crypto" reasoning that ruled out a custom quinn_proto::crypto::ClientConfig earlier in this effort.

Testing

  • crates/tpu-client/src/identity.rs: 12 new unit tests, including:
    • new_dummy_x509_certificate produces byte-identical output to solana_tls_utils::new_dummy_x509_certificate.
    • The custom MlockedSigningKey produces byte-identical Ed25519 signatures to ring's own EdDSA signer for the same key/message (signing is deterministic per RFC 8032, so this is a strong equivalence check, not just "it doesn't crash").
    • HardenedKeypair::read_from_reader/read_from_file parse the same JSON format as solana_keypair::read_keypair and recover byte-identical keys, including with input larger than the initial buffer guess (forces a grow).
    • HardenedKeypair::try_from accepts valid keypair bytes and rejects a tampered public half.
  • Existing integration suites (test_quic_gateway, test_yellowstone_tpu_sender, test_rpc_admin) updated for the new types and passing, including real QUIC handshake tests that exercise the new signing path end-to-end (not just unit-level).
  • Full workspace: cargo check, cargo clippy --all-targets --all-features -- -D warnings, cargo fmt --check, and all tests green.

Comment on lines +68 to +83
fn leader_lookahead(
&self,
leader_forward_lookahead: usize,
out: &mut [MaybeUninit<Pubkey>],
) -> usize {
let schedule = self.share.read().unwrap();
schedule[..leader_forward_lookahead].to_vec()

let it = schedule[..leader_forward_lookahead]
.iter()
.zip(out.iter_mut());
let mut i = 0;
for (src, dst) in it {
dst.write(*src);
i += 1;
}
i

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know you reutilize the same thing for tests so can you use the same function just make sure we don't change something here or there and forget something?

@lvboudre
lvboudre merged commit be59ac2 into main Sep 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants