Skip to content

feat(targets): install target toolchains with pocket target add - #594

Merged
HalfSweet merged 1 commit into
mainfrom
feat/target-install
Oct 19, 2026
Merged

HalfSweet merged 1 commit into
mainfrom
feat/target-install

Conversation

@HalfSweet

@HalfSweet HalfSweet commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Stacked on #593.

Why

The framework pinned the PSP toolchain in tools/cli/psp-toolchain.json, a copy of which also shipped in the CLI tarball, so pocket doctor checked the CLI's pins instead of the pins of the framework doing the build. tools/bootstrap.ts installed into per-tool cache paths with tool-specific receipts. A PSP build also took whatever clang/llvm-ar it found. With Homebrew's LLVM 23 every QuickJS object fails to link (floating point ABI '-msingle-float' is incompatible with target floating point ABI '-mdouble-float'), and nothing pinned a working version.

What

pocket target add|list|remove: an install engine for the toolchains the framework pins (tools/targets/).

  • Manifests (pocket-toolchain/2) live beside the catalog. The component kinds are rustup (a version or dated channel, never stable), archive (zip/tar, SHA-256, root, markers), file (SHA-256) and cargo-install (git revision).
  • Store: <cache>/store/<component digest>/{receipt.json,tree/}, with install records at <cache>/targets/<id>/<toolchain digest>.json.
    • A component digest covers the pin, not the name or role, so equal pins share one entry.
    • A toolchain digest covers a target's components on one host. A pin change produces a new record.
  • Install: every component stages under <cache>/tmp and publishes by rename.
    • Downloads are checked against their SHA-256 before anything reads them.
    • Archive entries and symbolic links that leave the directory are rejected.
    • A second add downloads nothing.
  • Locks:
    • one lock per component;
    • one per target record: add/remove take it exclusive and a build takes it shared, so remove waits for a running build;
    • one for the store: add holds it shared and prune takes it exclusive, so a prune never deletes what an unfinished add published.
    • withArtifactLock/publishStagedDirectory move to tools/targets/lock.ts.
  • Readiness: pocket build for a target that has a manifest stops before type checking when the record is not installed, outdated (installed for another framework version's digest) or broken (a store entry or rustup toolchain is gone), and names pocket target add <id>.
  • pocket show prints each target's state. pocket target list --check exits 1 unless every target in pocket.config.ts is ready.
  • Targets that need nothing installed: web-app, macos-app, linux-app and macos-widget have empty manifests. Targets whose scripts still install their own tools show no manifest until their own PRs.

PSP: tools/targets/psp.toolchain.json pins four components:

hosts/psp links libquickjs-sys at a5cac5d, and tools/psp.ts sets QUICKJS_PREBUILT_LIB. A PSP build runs no C compiler: no LLVM requirement, no TARGET_CC/TARGET_CFLAGS/AR_mipsel_sony_psp. The PSP toolchain now installs on Linux as well as macOS. PSP_SDK/PSPDEV/POCKETJS_LLVM_BIN no longer select tools; every path comes from the record.

Removed: pocket doctor, pocket setup, tools/bootstrap.ts, bun run bootstrap, and tools/cli/psp-toolchain.json; the CLI carries no pins. In a checkout, bun run pocket target add psp replaces bun run bootstrap.

CI: .github/workflows/psp-build.yml runs on Ubuntu 24.04 and macOS:

  • pocket target add psp into an empty cache;
  • a second add that must not download;
  • pocket +psp build of hero with TARGET_CC/CC_mipsel_sony_psp/AR set to /usr/bin/false.

Validation

PSP build workflow on this PR: green on both runners (run):

Runner target add psp into an empty cache second add pocket +psp build (C compiler /usr/bin/false)
ubuntu-24.04 541 s (rustup nightly, cargo install cargo-psp, SDK 157 MB, QuickJS 1.5 MB); toolchain 616351b5 0 s, no download 26 s, EBOOT written
macos-latest 661 s 0 s, no download 32 s, EBOOT written

The SDK and QuickJS store digests on Linux (ad0880352fff, afca6a7ff64d) equal the ones on macOS; cargo-psp's digest includes the host.

Local, macOS arm64:

  • Real install into an empty cache: POCKET_NEXUS_CACHE_DIR=/tmp/… bun run pocket target add psp finished in 32 s. Store entries: SDK 386 MB, cargo-psp 5.4 MB, QuickJS 1.4 MB. Toolchain 9fc39642. A second add printed every component as installed and downloaded nothing.
  • Real build through the CLI: pocket +psp build --manifest apps/hero/pocket.json --project-root . -- --release with TARGET_CC, TARGET_AR, CC_mipsel_sony_psp and AR_mipsel_sony_psp pointing at a program that exits 97. libquickjs-sys compiled at a5cac5d and linked the prebuilt archive; the EBOOT is 3 752 847 bytes, and the fake tools were never called. Not run on PPSSPP.
  • Diagnostics:
    • with an empty cache, +psp build and bun tools/psp.ts hero stop with target psp is not installed for @pocketjs/framework 0.14.0 (toolchain 9fc39642) / run: pocket target add psp;
    • with the QuickJS store entry deleted, broken names the missing entry;
    • target remove psp deleted the record and pruned both orphaned store entries.
  • tests/targets-install.test.ts (19) uses a local HTTP server, ustar archives the test writes itself (../ entries, links that leave the directory), and a fake rustup. It covers:
    • archive/file/rustup/cargo-install installs;
    • one download for two concurrent adds and none for a second add;
    • SHA-256 mismatch leaving no entry or record;
    • unsafe entries;
    • unsupported host, missing rustup, outdated, broken;
    • remove keeping shared components;
    • remove waiting for a build;
    • prune keeping an unfinished add's components;
    • the moved lock tests.
  • tests/targets-psp.test.ts (5):
    • the pins;
    • the prebuilt tag matches the libquickjs-sys revision in hosts/psp/Cargo.toml and Cargo.lock;
    • rust-psp revisions;
    • the build environment read from a record carries QUICKJS_PREBUILT_LIB and no C toolchain variables.
  • bun run e2e:install: 4 ok. The new pocket target list --check step passes from the read-only installed framework.
  • bun tools/test.ts, stage by stage: every stage passes except unit, which has 1097 passing and 1 failing. The failure is --strict-classes fails on the ones in a class attribute; it also fails locally on main and passes in CI.
  • tsc --noEmit: only the existing errors in tests/runtime-glyphs.test.ts and tests/pocket3d-icon.test.ts.

Not in this PR

  • pocket component add|list|remove: no target has a run or deploy component until the run/deploy backends land.
  • Manifests for Vita, 3DS, Android, BlackBerry, Meizu, Symbian and Apple; git, container, provided and derived components; target add --verify.
  • tests/psp-prx.test.ts's opt-in link test now needs POCKETJS_LLVM_BIN for clang, which assembles its fixture; PSP builds no longer use clang.

@HalfSweet
HalfSweet force-pushed the feat/target-install branch from 10dcf5d to ef7e8ca Compare October 19, 2026 07:08
@HalfSweet
HalfSweet force-pushed the feat/cli-target-selection branch from 84d1e11 to 542d50e Compare October 19, 2026 07:08
@HalfSweet
HalfSweet force-pushed the feat/target-install branch from ef7e8ca to 9b1002f Compare October 19, 2026 07:28
@HalfSweet
HalfSweet force-pushed the feat/cli-target-selection branch from 542d50e to 21332cc Compare October 19, 2026 07:28
@HalfSweet
HalfSweet force-pushed the feat/target-install branch from 9b1002f to 5beef3d Compare October 19, 2026 07:35
@HalfSweet
HalfSweet force-pushed the feat/cli-target-selection branch from 21332cc to 6768188 Compare October 19, 2026 07:35
@HalfSweet
HalfSweet force-pushed the feat/target-install branch from 5beef3d to 159229e Compare October 19, 2026 07:49
@HalfSweet
HalfSweet force-pushed the feat/cli-target-selection branch from 6768188 to 7a87fb1 Compare October 19, 2026 07:49
Base automatically changed from feat/cli-target-selection to main October 19, 2026 07:57
The framework pinned the PSP toolchain in tools/cli/psp-toolchain.json,
which the CLI tarball also carried, so pocket doctor checked the CLI's pins
rather than the pins of the framework that builds. tools/bootstrap.ts
installed into per-tool cache paths, and a PSP build compiled QuickJS with
whatever clang it found; Homebrew's LLVM 23 fails the link.

pocket target add|list|remove installs the toolchains the framework pins in
tools/targets/<id>.toolchain.json (pocket-toolchain/2: rustup, archive,
file and cargo-install components). Each component stages under
<cache>/tmp, is checked against its SHA-256, and publishes by rename to
<cache>/store/<component digest>; the install record
<cache>/targets/<id>/<toolchain digest>.json names what a build reads. A
component digest covers its pin, so equal pins share one store entry; a pin
change gives a new toolchain digest and a new record. Archive entries and
links that leave the directory are rejected. A second add downloads
nothing.

- pocket build for a target with a manifest stops before type checking
  when its toolchain is not installed, outdated or broken, naming
  `pocket target add <id>`, and holds the record shared until the backend
  finishes; remove waits for it. pocket show prints every target's state;
  pocket target list --check gates CI on the project's targets.
- The PSP manifest pins nightly-2026-05-28 with rust-src, cargo-psp, the
  SDK release, and libquickjs-mipsel-sony-psp.a from pocket-nexus/quickjs-rs
  prebuilt-psp-a5cac5d8631b. hosts/psp links libquickjs-sys at a5cac5d and
  tools/psp.ts sets QUICKJS_PREBUILT_LIB: a PSP build runs no C compiler
  and installs on Linux as well as macOS. Every path comes from the record.
- pocket doctor, pocket setup, tools/bootstrap.ts and the CLI's
  psp-toolchain.json are removed; `bun run pocket target add psp` replaces
  `bun run bootstrap` in a checkout.
- .github/workflows/psp-build.yml installs the PSP toolchain into an empty
  cache on Ubuntu and macOS and builds hero with the target C compiler set
  to /usr/bin/false.
@HalfSweet
HalfSweet force-pushed the feat/target-install branch from 159229e to b8f1a92 Compare October 19, 2026 07:57
@HalfSweet
HalfSweet merged commit 3247286 into main Oct 19, 2026
16 checks passed
@HalfSweet
HalfSweet deleted the feat/target-install branch October 19, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant