Repository navigation
Conversation
git fetch ends with `git maintenance run --auto` (older git: `git gc --auto`), which runs detached by default. During an incremental collection the last fetch can leave a background repack running while libgit2 already scans objects/pack/, so a temporary .tmp-<pid>-pack-*.idx disappears under it and Collect Commits fails intermittently with "could not find ... to open". Pass -c gc.auto=0 -c gc.autoDetach=false -c maintenance.auto=false to every git command run by GitcliCloner. The clone is a temporary directory, so automatic housekeeping has no benefit there; the explicit `git repack -d` in deepen() still runs synchronously. Add a regression test that traces an incremental clone with GIT_TRACE and asserts that git starts neither `maintenance run` nor `gc --auto`. Signed-off-by: DoDiODev <DoDiDev@proton.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Incremental git collection fails sporadically in
Collect Commitswith:Root cause: since git 2.29,
git fetchends withgit maintenance run --auto(older versions:
git gc --auto), and that runs detached in the backgroundby default. For an incremental collection
GitcliClonerruns several fetches(shallow fetch,
--shallow-since,--deepenrounds). The last one may leave abackground maintenance or gc process repacking the repository while libgit2 is
already scanning
objects/pack/. libgit2 sees the temporary.tmp-<pid>-pack-*.idxfile, which git renames or removes a moment later, andthe subtask aborts. Because this is a race, the failure is intermittent.
This PR passes
-c gc.auto=0 -c gc.autoDetach=false -c maintenance.auto=falseto every gitcommand run by
GitcliCloner. The clone is a short-lived temporary directorythat is removed after extraction, so automatic housekeeping has no benefit
there. The explicit
git repack -dindeepen()is still executedsynchronously as before.
A regression test runs an incremental clone with
GIT_TRACEand asserts thatgit does not start
maintenance run/gc --auto. Without the fix the testfails with
run_command: git maintenance run --auto --no-quiet --detachafter each fetch.Does this close any open issues?
N/A
Screenshots
N/A
Other Information
maintenance.autois safe on gitversions older than 2.29;
gc.auto=0covers those versions.run failed in
Collect Commitswith the error above; with the change, 47gitextractor tasks across two further pipeline runs completed without it.
lint and e2e on MySQL and PostgreSQL:
run