Skip to content

Add patch to work around slowdowns in 1.26.x - #23

Closed
bartvdbraak wants to merge 1 commit into
cifrom
ci-workaround-slowdowns
Closed

Add patch to work around slowdowns in 1.26.x#23
bartvdbraak wants to merge 1 commit into
cifrom
ci-workaround-slowdowns

Conversation

@bartvdbraak

Copy link
Copy Markdown
Member

Relevant info

  • After updating to 1.26.2
  • We've also updated to 1.26.4, no change

Frontend errors when trying to merge PR

Users are reporting slow downs and issues when creating PRs, even causing frontend errors like:

Hello. I got this error message when merging a PR (but seems the PR was merged even if there is an error message)
I can confirm, the branch was not deleted automatically

image

Gitea logs context canceled errors

2026/06/12 07:10:35 services/pull/pull.go:600:pushToBaseRepoHelper() [E] Unable to push PR head for blender/blender#160003 (<Repository 15:blender/blender>:refs/pull/160003/head) due to Error: push failed: context canceled
2026/06/12 07:10:35 routers/web/repo/pull.go:1503:CompareAndPullRequestPost() [E] Unexpected error of NewPullRequest: *fmt.wrapError Push: llm-test/llm-blender:llm-fix-material-remap-curves-pointcloud blender/blender:refs/pull/160003/head push failed: context canceled
2026/06/12 07:10:35 routers/web/repo/pull.go:1504:CompareAndPullRequestPost() [E] CompareAndPullRequest: Push: llm-test/llm-blender:llm-fix-material-remap-curves-pointcloud blender/blender:refs/pull/160003/head push failed: context canceled
2026/06/12 07:56:05 services/pull/pull.go:600:pushToBaseRepoHelper() [E] Unable to push PR head for blender/blender#160003 (<Repository 15:blender/blender>:refs/pull/160003/head) due to Error: push failed: context canceled
2026/06/12 07:56:05 routers/web/repo/pull.go:1503:CompareAndPullRequestPost() [E] Unexpected error of NewPullRequest: *fmt.wrapError Push: PratikPB2123/blender:99584-outliner blender/blender:refs/pull/160003/head push failed: context canceled
2026/06/12 07:56:05 routers/web/repo/pull.go:1504:CompareAndPullRequestPost() [E] CompareAndPullRequest: Push: PratikPB2123/blender:99584-outliner blender/blender:refs/pull/160003/head push failed: context canceled
2026/06/12 07:57:57 services/pull/pull.go:600:pushToBaseRepoHelper() [E] Unable to push PR head for blender/blender#160003 (<Repository 15:blender/blender>:refs/pull/160003/head) due to Error: push failed: context canceled
2026/06/12 07:57:57 routers/web/repo/pull.go:1503:CompareAndPullRequestPost() [E] Unexpected error of NewPullRequest: *fmt.wrapError Push: Sean-Kim/blender:add-more-texpaint-build-markers blender/blender:refs/pull/160003/head push failed: context canceled
2026/06/12 07:57:57 routers/web/repo/pull.go:1504:CompareAndPullRequestPost() [E] CompareAndPullRequest: Push: Sean-Kim/blender:add-more-texpaint-build-markers blender/blender:refs/pull/160003/head push failed: context canceled
2026/06/12 07:58:36 .../actions/notifier_helper.go:159:notify() [W] Event "pull_request_comment" should only trigger workflows on the default branch, but its ref is "refs/pull/160003/head". Will fall back to the default branch

2026/06/12 08:03:36 services/pull/pull.go:600:pushToBaseRepoHelper() [E] Unable to push PR head for blender/blender#160004 (<Repository 15:blender/blender>:refs/pull/160004/head) due to Error: push failed: context canceled
2026/06/12 08:03:36 routers/web/repo/pull.go:1503:CompareAndPullRequestPost() [E] Unexpected error of NewPullRequest: *fmt.wrapError Push: Sean-Kim/blender:encode-parallel blender/blender:refs/pull/160004/head push failed: context canceled
2026/06/12 08:03:36 routers/web/repo/pull.go:1504:CompareAndPullRequestPost() [E] CompareAndPullRequest: Push: Sean-Kim/blender:encode-parallel blender/blender:refs/pull/160004/head push failed: context canceled
2026/06/12 08:05:10 .../actions/notifier_helper.go:159:notify() [W] Event "pull_request_comment" should only trigger workflows on the default branch, but its ref is "refs/pull/160004/head". Will fall back to the default branch

git commands sometimes stall:

[bart@ws-bart:~/.../blender/blender]$ git push me
Locking support detected on remote "me". Consider enabling it with:
  $ git config lfs.https://git.blender.org/bartvdbraak/blender.git/info/lfs.locksverify true
Enumerating objects: 11, done.
Counting objects: 100% (11/11), done.
Delta compression using up to 32 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 722 bytes | 722.00 KiB/s, done.
Total 6 (delta 4), reused 0 (delta 0), pack-reused 0 (from 0)

< hangs for over 20-30 seconds before actually pushing >

Potential resolution

@brechtvl mentioned:

I looked at relevant changes, best guess I found is this one: go-gitea#36400

A theory is that while this avoids the overhead of creating a temporary repo, it locks the base repo during conflict checking instead. When PR merge conflict checks happens after a push to main there may be a lot of contention and timeouts from many open PRs.

If true, the workaround to fall back to the old code path would be:

diff --git a/modules/git/git.go b/modules/git/git.go
index 0c2deb0281..afb83e298a 100644
--- a/modules/git/git.go
+++ b/modules/git/git.go
@@ -79,7 +79,7 @@ func loadGitVersionFeatures() (*Features, error) {
        }
        features.SupportCheckAttrOnBare = features.CheckVersionAtLeast("2.40")
        features.SupportCatFileBatchCommand = features.CheckVersionAtLeast("2.36")
-       features.SupportGitMergeTree = features.CheckVersionAtLeast("2.40") // we also need "--merge-base"
+       features.SupportGitMergeTree = false // features.CheckVersionAtLeast("2.40") // we also need "--merge-base"
        return features, nil
 }

@lunny mentioned:

SupportGitMergeTree only changes how Gitea checks whether a pull request is mergeable or has conflicts. The errors we are seeing (push failed: context canceled) happen later, when Gitea tries to push the PR head to refs/pull/*/head. That usually means the HTTP request was canceled or timed out, rather than the merge-conflict check failing. The more likely issue is request cancellation or timeout in front of Gitea, or slow repository/storage operations during PR creation.

If needed, we can still test the old path in a staging environment for comparison, but I would treat that as a diagnostic experiment, not as the primary fix.

@brechtvl mentioned:

My guess was that merge conflict checking on the base repo may hold a lock either in Gitea or Git, which would then cause other operations to time out as they wait to acquire that lock.
But it's just a guess, could be something else entirely. It's just a fairly easy thing to check if these errors are happening frequently.

@lunny mentioned:

Yes. FetchRemoteCommit currently holds a repository-level lock, which means all pull requests have to wait for other fetch operations to complete.

I also submitted PR go-gitea#38105 to remove the global lock used during fetching.

@lunny mentioned:

Thanks! Let's hope that is the underlying issue.

@bartvdbraak
bartvdbraak requested a review from brechtvl July 9, 2026 13:03
@bartvdbraak

Copy link
Copy Markdown
Member Author

Unfortunately, this didn't seem to fix the issue:

[bart@ws-bart:~/.../blender/blender]$ git push
Locking support detected on remote "me". Consider enabling it with:
  $ git config lfs.https://git.blender.org/bartvdbraak/blender.git/info/lfs.locksverify true
Enumerating objects: 11, done.
Counting objects: 100% (11/11), done.
Delta compression using up to 32 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 1.36 KiB | 1.36 MiB/s, done.
Total 6 (delta 5), reused 0 (delta 0), pack-reused 0 (from 0)


<hanging for over 60 seconds>

remote: . Processing 1 references
remote: Processed 1 references in total
remote: 
remote: Visit the existing pull request:
remote:   https://projects.blender.org/blender/blender/pulls/161074
remote: 
To git.blender.org:bartvdbraak/blender.git
   5af5f184ce5..74b6f31994e  deps-linux-warn-unattend -> deps-linux-warn-unattend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants