Fail mirroring more gracefully #34002
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GiteaBot
added
the
lgtm/need 2
rremer
force-pushed
the
missing-lfs-fixes
branch
from
39f376f to
c6337ab
Compare
wxiaoguang
reviewed
Mar 25, 2025
rremer
force-pushed
the
missing-lfs-fixes
branch
from
c6337ab to
62a85f3
Compare
* reuse recoverable error checks across mirror_pull * add new cases for 'cannot lock ref/not our ref' (race condition in fetch) and 'Unable to create/lock" * move lfs sync right after commit graph write, and before other maintenance which may fail * try a prune for 'broken reference' as well as 'not our ref' * always sync LFS right after commit graph write, and before other maintenance which may fail
rremer
force-pushed
the
missing-lfs-fixes
branch
from
62a85f3 to
1871276
Compare
wxiaoguang
approved these changes
Mar 26, 2025
GiteaBot
added
lgtm/need 1
wxiaoguang
changed the title
wxiaoguang
force-pushed
the
missing-lfs-fixes
branch
from
e35b603 to
f1e4b64
Compare
lunny
approved these changes
Mar 26, 2025
GiteaBot
added
lgtm/done
lunny
added
the
reviewed/wait-merge
GiteaBot
removed
the
reviewed/wait-merge
zjjhot
added a commit
to zjjhot/gitea
that referenced
this pull request
Mar 27, 2025
* giteaofficial/main: [skip ci] Updated translations via Crowdin Download actions job logs from API (go-gitea#33858) Fail mirroring more gracefully (go-gitea#34002) Fix dropdown module accessing (go-gitea#34026) Polyfill WeakRef (go-gitea#34025) Fix dropdown delegating and some UI problems (go-gitea#34014)
project-mirrors-bot-tu bot
pushed a commit
to project-mirrors/forgejo-as-gitea-fork
that referenced
this pull request
Apr 1, 2025
* reuse recoverable error checks across mirror_pull * add new cases for 'cannot lock ref/not our ref' (race condition in fetch) and 'Unable to create/lock" * move lfs sync right after commit graph write, and before other maintenance which may fail * try a prune for 'broken reference' as well as 'not our ref' * always sync LFS right after commit graph write, and before other maintenance which may fail This handles a few cases where our very large and very active repositories could serve mirrored git refs, but be missing lfs files: ## Case 1 (multiple variants): Race condition in git fetch There was already a check for 'unable to resolve reference' on a failed git fetch, after which a git prune and then subsequent fetch are performed. This is to work around a race condition where the git remote tells Gitea about a ref for some HEAD of a branch, then fails a few seconds later because the remote branch was deleted, or the ref was updated (force push). There are two more variants to the error message you can get, but for the same kind of race condition. These *may* be related to the git binary version Gitea has access to (in my case, it was 2.48.1). ## Case 2: githttp.go can serve updated git refs before it's synced lfs oids There is probably a more aggressive refactor we could do here to have the cat-file loop use FETCH_HEAD instead of relying on the commit graphs to be committed locally (and thus serveable to clients of Gitea), but a simple reduction in the occurrences of this for me was to move the lfs sync block immediately after the commit-graph write and before any other time-consuming (or potentially erroring/exiting) blocks. --------- Co-authored-by: wxiaoguang <[email protected]> (cherry picked from commit e0ad72e)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This handles a few cases where our very large and very active repositories could serve mirrored git refs, but be missing lfs files:
Case 1 (multiple variants): Race condition in git fetch
There was already a check for 'unable to resolve reference' on a failed git fetch, after which a git prune and then subsequent fetch are performed. This is to work around a race condition where the git remote tells Gitea about a ref for some HEAD of a branch, then fails a few seconds later because the remote branch was deleted, or the ref was updated (force push).
There are two more variants to the error message you can get, but for the same kind of race condition. These may be related to the git binary version Gitea has access to (in my case, it was 2.48.1).
Case 2: githttp.go can serve updated git refs before it's synced lfs oids
There is probably a more aggressive refactor we could do here to have the cat-file loop use FETCH_HEAD instead of relying on the commit graphs to be committed locally (and thus serveable to clients of Gitea), but a simple reduction in the occurrences of this for me was to move the lfs sync block immediately after the commit-graph write and before any other time-consuming (or potentially erroring/exiting) blocks.