Skip to content

describe wire format version four - #131

Open
alii wants to merge 1 commit into
masterfrom
wire-format-v4
Open

alii wants to merge 1 commit into
masterfrom
wire-format-v4

Conversation

@alii

@alii alii commented Sep 27, 2026

Copy link
Copy Markdown
Member

Docs only. docs/wire-format.md described version 3, what the old VM wrote, with two changes noted as "already decided". It now describes version 4, which is what the new VM will write, and says why.

Nothing is built yet. This is the spec the descriptor work is being written against.

What changed in the spec

A closure names its code, not its run. Version 3 stamped every closure with the 128 random bits of the run that made it and refused any other run. So a closure could never cross a network — not unsafely, at all. Version 4 names the code: module, the lambda's own id from source, a fingerprint of its shape, and a hash of the module's source. A peer with the same code decodes it; a peer with different code gets NoSuchCode.

The name is a lookup key, never an index, for three reasons written into the doc: a function index is minted fresh each compile; if Scarlet ever specialises generics, which functions exist depends on how the whole program used them, so two peers built from one source hold different sets; and a C backend has no runtime function table to index at all.

Captures decode against a descriptor. Version 3 checked a capture's form — one tag byte saying "I am an Int" — and nothing checked that the body reading it wanted an Int. The run stamp was what made that safe. Drop the stamp and a peer can hand you captures of the wrong kinds, and the body then does IntAdd on a string.

So each function carries the static type of each capture slot, and captures are read against it like any other value. The tag bytes go away.

A capture whose static type is a type variable gets a new Any node. Sound by parametricity: a body can't do anything type-specific with a value of type a, so any well-formed value is safe there. If Scarlet specialises generics later, or passes type descriptors at runtime, the wildcards disappear and the check becomes exact.

An Int has no size limit, so the format has to say how one past 64 bits is written. Flagged, not yet specified.

What is deliberately unchanged: decoding a closure means running your own code with captures the sender chose. That's a trust decision, not a memory-safety one — the same one BEAM makes and answers with a shared cookie. The doc says so rather than leaving it implicit.

Also

  • The fingerprint's VERSION constant moves to 4, so every fingerprint changes. That's what a version bump is for.
  • Node kind 15 is Any. 11, 12 and 13 stay reserved — they were version 3's capture tags and are never reused.
  • A "Closures" section replaces "The capture format", and the refusal order for a closure is spelled out: every check happens before a single capture is read, because a call reaches its code without re-checking.

All seven CI gates pass locally. hawk ran for the host target, and nothing here is gated on a cfg.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant