Skip to content

feat(vcs): layer worktree include manifests by Git precedence - #1

Closed
ironerumi wants to merge 4 commits into
mainfrom
fm/th-189-build
Closed

ironerumi wants to merge 4 commits into
mainfrom
fm/th-189-build

Conversation

@ironerumi

@ironerumi ironerumi commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Intent

go ahead, build and test on fork then as upstream PR

Build the change proposed in upstream issue kunchenguid#189 as corrected in kunchenguid#189 (comment): layer the include manifest the way Git layers ignore rules. Decisions taken: (1a) literal Git precedence, lowest to highest: user layer, then clone layer, then the repository layer, so the committed file wins; (2) the user layer is a treehouse-defined Git config key worktree.includeFile, resolved with normal Git config lookup (--path typing, includes/includeIf) in the main source checkout, with no implicit XDG fallback; (3) the clone layer is $GIT_COMMON_DIR/info/worktreeinclude of the source clone, and creating that file is the opt-in; (4) --include-file keeps replacing every layer, an empty file still meaning no seeds, and lower-layer read errors do not apply when it is given; (5) layers are combined as one ordered pattern stream fed to Git, preserving file boundaries and stripping a leading UTF-8 BOM per file, so ! negation works across layers; (6) persistent-layer read errors surface at seeding time with today's remove-or-quarantine handling; (7) the same behavior for both jj modes. Selection stays a separate predicate intersected with --exclude-standard; an explicitly configured but missing or unreadable file is an error, an absent key or sidecar means no rules, an empty configured value is a configuration error. Copy, inventory, cleanup and pool state are untouched.

The referenced design report's implementation and required regression substance:

Smallest coherent implementation

Limit production work to the shared seeding resolver/matcher and contract text. In selectedSeedPathsEnv, preserve the non-nil override short circuit. Only the nil/default path discovers and reads the configured user file, common-directory sidecar, and destination committed blob. An empty/missing committed file must no longer terminate the search early. Resolve metadata/config using source repoRoot plus existing gitEnv; keep committed revision lookup exactly as now. Read all chosen layers before matching/copying. Join them in the selected precedence order with file-boundary normalization; keep the two-stage selection/ignore intersection and copy implementation untouched. Keep the existing pool and backend override parameter, state schema, HMAC, and cleanup untouched. Both backends already enter this shared code. Update comments so nil means the layered default, not committed-only. Update README seeding text/flag table, get help, and the design seeding note: precise order, locations, opt-in/trust statement, root-relative pattern semantics, directory-negation caveat, errors, and all-layer override. No new config subsystem.

Read new default sources at seeding time, after a slot was created/reset. Existing failure paths remove fresh slots or quarantine reused ones. Do not promise that a bad persistent file leaves the pool untouched. The explicit flag's existing pre-acquisition read/error guarantee stays unchanged.

Required regression work

Extend existing test files, using isolated HOME/XDG/Git/JJ config and deterministic filesystem/process events:

  • Discovery and absence: each personal layer alone works without a committed manifest; all absent preserves existing output. Dirty/untracked source manifest still cannot add rules, with and without personal layers. Different destination revision supplies different committed rules.
  • Order and cross-file negation: all three conflict pairs, negation-only layer, later re-add, and nonconflicting additions. Assert exact selected/copied paths, not just successful exit. Include the excluded-parent counterexample so documentation and behavior agree.
  • Byte/file boundaries: missing final newline, CRLF, each file's BOM, comments, escaped leading #/!, spaces, anchored paths and **. Compare the combined matcher against ordered --exclude-from output. Keep NUL-safe paths.
  • Independent ignore predicate: .gitignore, info/exclude, and core.excludesFile can each supply eligibility. Selected but unignored files, tracked source edits in Git, unselected ignored files, and destination tracked ancestor/descendant/case aliases must not be copied. Personal excludes on the ignore side must not become include rules.
  • Lookup context: invocation from nested directories and existing linked worktrees; common metadata rather than private worktree gitdir; nonstandard Git metadata; source versus destination per-worktree config. Config includes/conditional includes, scope precedence, spaces, ~/, relative path base, missing key, malformed config, empty value, missing file, directory, and unreadable file. Permission-only fixtures must work as privileged users; a directory-as-file is deterministic.
  • Override remains replacement: nonempty and empty explicit overrides bypass all default sources, including broken configured files and invalid committed manifests. Preserve early read, no fallback, cwd-relative filename, source-root-relative patterns, and later restoration of defaults.
  • Lifecycle: new and reused slots; changed/deleted personal files; return/reuse removes old inventory regardless of current rules and preserves unmanaged ignored files. Layer-read errors follow documented failure timing. Keep failure/quarantine/HMAC regressions.
  • Safety: existing no-filter, symlink flattening/root/destination refusal, private permissions, and partial-inventory cases still pass. No broader copy permissions or cleanup authorization.
  • jj: real colocated and non-colocated execution, inherited personal/default layers, destination revision, source-store lookup, ignored status after seeding, and authenticated return. Keep original mutable-manifest and override tests; do not call a skipped jj test a parity pass.
  • Windows: use filepath helpers for filesystem paths, but leave manifest patterns as Git's slash/backslash grammar. No /dev/stdin, shell concatenation, or hardcoded .git directory. Test spaces, drive/UNC paths as applicable, CRLF, and temp-file close-before-Git behavior. Keep raw seed-byte assertions distinct from checkout conversion.

Before shipping: make test, make lint, GOOS=windows go build ./..., real CLI leased/path/JSON and interactive smoke tests for create/reuse/return, and real jj tests with jj installed. Capture an unconfigured before/after path/content comparison and a negative input that must fail.

What Changed

  • Layer default worktree seeding from worktree.includeFile, the source clone's common-directory sidecar, and the destination revision's committed manifest, preserving Git pattern precedence and file-boundary handling.
  • Keep --include-file as a complete replacement that bypasses default-layer discovery and errors, while surfacing persistent-layer read failures through existing acquisition handling.
  • Update CLI and design documentation and add regression coverage for Git and jj layers, lookup contexts, precedence, overrides, lifecycle cleanup, and pool error handling.

Risk Assessment

⚠️ Medium: The change expands seeding discovery across shared Git and jj lifecycle paths, but the resolver preserves override, selection, inventory, and cleanup boundaries with focused regression coverage.

Testing

Validation on pushed head db51f32418b62a3dac1a96ac9a13d650e7909132 completed with all six fork CI checks green, including Linux, macOS, and Windows tests. Local full-suite, lint, Windows cross-build, focused seeding/lifecycle regressions, and the supplemental live checks below passed. The initial pipeline analyzer was inconclusive (4/6 live scenarios); Test approval records an authorized exception supported by completing the missing scenarios, not a clean initial analyzer verdict.

Supplemental live pattern-boundary validation used the real built Treehouse CLI with a UTF-8 BOM in each layer, CRLF, missing final newlines, and cross-layer ! negation. It copied exactly alpha.env, clone.env, and repo.env, with byte-for-byte source parity. The negated veto.env, unselected ignored file, and an explicitly selected but unignored clear.txt were absent.

jj was initially unavailable locally. An official, temporary, worktree-only darwin-arm64 jj 0.46.0 binary was then used; its release archive's published SHA-256 was verified: 43daa37ce0f9716cdf6fb65a29cbcae74a3160aef048cf3cb3d166d41cb4ab58. The full real jj package tests ran without skips, including both colocated and non-colocated layer/revision/store-lookup regressions; Git/jj CLI override tests also passed. Real Treehouse CLI scenarios in both jj modes verified layered copies, an empty jj working-copy commit after snapshot, and authenticated return removing inventoried seeds while preserving an unmanaged ignored file. These supplemental scenarios and real jj package tests were repeated on the pushed head above.

The fork CI workflow does not install jj and its Go output does not prove real jj execution, so CI package passes are not claimed as jj parity evidence; the verified local runs provide that evidence. The unchanged analyzer table and evidence below (and the Pipeline Test block) are the historical initial record, superseded for the two missing scenarios by the supplemental live validation above.

  • Live validation: ⚠️ inconclusive - 4 of 6 scenarios driven live against the product
Scenario Result Live Evidence
Unconfigured acquisition preserves no-seed behavior ✅ pass live th189-live-unconfigured.txt; disposable repo run with the built CLI
Default layers discover user, clone, and committed rules in precedence order ✅ pass live th189-live-cli.txt; go test ./cmd -run 'TestGetIncludeLayersAndOverride|TestGetIncludeFileReplacesCommittedManifest' -count=1
Reuse refreshes seeded inventory and surfaces persistent-layer errors ✅ pass live th189-live-lifecycle.txt; go test ./internal/pool -run 'TestAcquire_(PersonalLayersRefreshAndInventoryCleanup|PersistentLayerReadErrorsRemoveOrQuarantine)' -count=1
Explicit include-file replacement bypasses broken defaults ✅ pass live th189-live-lifecycle.txt; go test ./cmd -run 'TestGetIncludeLayersAndOverride|TestGetIncludeFileReplacesCommittedManifest' -count=1
Pattern-stream boundaries and independent ignore eligibility remain safe ⏸️ untested no The prior payload cited automated backend tests rather than a separately driven live CLI session, so it did not establish a live result for this scenario.
jj colocated and non-colocated seeding parity ⏸️ untested no Live validation was attempted, but jj is not installed (command -v jj failed); the targeted tests explicitly skipped with jj is not installed. No workspace-local jj executable/source was provide…
Evidence: Live layered CLI smoke
Treehouse issue #189 live CLI evidence
isolated HOME=/tmp/treehouse-th189-live/home and pool=/tmp/treehouse-th189-live/pool
layered acquire exit=0
json={"path":"/tmp/treehouse-th189-live/pool/.treehouse/repo-25f364/1/repo","lease_id":"eef4b9ec7dddcd428995a5ca406033f7","lease_holder":"","leased_at":"2026-10-19T02:28:26.179114+09:00","base_branch":"main"}
seeds observed in acquired worktree: user.seed=present clone.seed=present repository.seed=present explicit.seed=absent
return exit=0; stderr reported Worktree returned to pool.
negative missing configured manifest exit=1
negative stdout empty
negative stderr: failed to seed .worktreeinclude ... read worktree.includeFile "/tmp/treehouse-th189-live/home/missing.include": open ... no such file or directory
Evidence: Live lifecycle and override smoke
Treehouse issue #189 live CLI lifecycle/override evidence
lifecycle acquire=0 reuse=0 override=0 returns=0/0/0 missing-config=1
path-same=yes second-present=yes old-personal-removed=yes old-clone-removed=yes unmanaged-preserved=yes
explicit-only=yes broken-config-and-clone-bypassed=yes
negative stderr: failed to seed .worktreeinclude into /tmp/treehouse-th189-live/pool/.treehouse/repo-25f364/1/repo: read worktree.includeFile "/tmp/treehouse-th189-live/home/missing.include": open /tmp/treehouse-th189-live/home/missing.include: no such file or directory
Evidence: Live unconfigured baseline smoke
Treehouse issue #189 unconfigured baseline live CLI evidence
unconfigured acquire=0; ignored source not copied=yes; return=0
- Outcome: ⚠️ 1 warning across 1 run (3m56s)

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed ✅
  • 🚨 internal/vcs/gitvcs/gitvcs.go:570 - defaultWorktreeInclude unconditionally invokes git rev-parse --path-format=absolute --git-common-dir at line 570. On Git versions that do not support --path-format, every default acquisition fails before checking the committed manifest—even when the user key, sidecar, and committed file are all absent—regressing the required no-layer behavior. The existing CommonGitDir helper at lines 78-89 already preserves compatibility with a fallback; use the equivalent fallback while retaining the supplied Git environment.
  • ⚠️ internal/vcs/gitvcs/gitvcs.go:580 - The clone sidecar is treated as absent whenever os.ReadFile returns fs.ErrNotExist at line 580. A dangling symlink at $GIT_COMMON_DIR/info/worktreeinclude is a present, opted-in persistent layer but produces ENOENT while reading its target, so the code silently skips it and may seed only lower-precedence committed rules instead of surfacing the required persistent-layer read error. Distinguish a missing sidecar entry from a read failure (for example with Lstat) before applying the absent-sidecar exception.

🔧 Fix applied.
✅ Re-checked - no issues remain.

⚠️ **Test** - 1 warning
  • ⚠️ live validation verdict: inconclusive (4 of 6 scenarios were driven live against the product); untested: Pattern-stream boundaries and independent ignore eligibility remain safe, jj colocated and non-colocated seeding parity
  • Live validation: ⚠️ inconclusive - 4 of 6 scenarios driven live against the product
Scenario Result Live Evidence
Unconfigured acquisition preserves no-seed behavior ✅ pass live th189-live-unconfigured.txt; disposable repo run with the built CLI
Default layers discover user, clone, and committed rules in precedence order ✅ pass live th189-live-cli.txt; go test ./cmd -run 'TestGetIncludeLayersAndOverride|TestGetIncludeFileReplacesCommittedManifest' -count=1
Reuse refreshes seeded inventory and surfaces persistent-layer errors ✅ pass live th189-live-lifecycle.txt; go test ./internal/pool -run 'TestAcquire_(PersonalLayersRefreshAndInventoryCleanup|PersistentLayerReadErrorsRemoveOrQuarantine)' -count=1
Explicit include-file replacement bypasses broken defaults ✅ pass live th189-live-lifecycle.txt; go test ./cmd -run 'TestGetIncludeLayersAndOverride|TestGetIncludeFileReplacesCommittedManifest' -count=1
Pattern-stream boundaries and independent ignore eligibility remain safe ⏸️ untested no The prior payload cited automated backend tests rather than a separately driven live CLI session, so it did not establish a live result for this scenario.
jj colocated and non-colocated seeding parity ⏸️ untested no Live validation was attempted, but jj is not installed (command -v jj failed); the targeted tests explicitly skipped with jj is not installed. No workspace-local jj executable/source was provide…
  • go test ./internal/vcs/gitvcs -run 'TestSeed' -count=1
  • go test ./internal/pool -run 'TestAcquire_(PersonalLayersRefreshAndInventoryCleanup|PersistentLayerReadErrorsRemoveOrQuarantine|ManifestOverrideRefreshesAndRemovesSeedsOnReuse)' -count=1
  • go test ./cmd -run 'TestGetInclude(LayersAndOverride|FileReplacesCommittedManifest|FileReadFailureLeavesPoolUntouched)' -count=1
  • go test ./internal/vcs/jjvcs -run 'TestSeed' -count=1 (skipped: jj unavailable)
  • Built and drove the target CLI against isolated disposable Git repositories; captured evidence in the three listed logs.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…sition-resolved common Git directory through Git and jj seeding, including colocated jj, while preserving non-colocated discovery. Added executable regressions for older-Git fallback, single config lookup, and dangling/absent sidecar behavior; restored the authorized state comment. Verified make test, make lint, Windows cross-build, focused tests, and real jj tests. External PR Testing-section publication was not performed in this ci phase
@ironerumi

Copy link
Copy Markdown
Owner Author

Superseded by the upstream PR kunchenguid#191 (same change on current upstream main). The local build is made from branch fm/th-189-upstream, which stays.

@ironerumi ironerumi closed this Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant