From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Subject: [PATCH 0/5] fetch: more optimizations for mirror fetches
Date: Wed, 23 Feb 2022 13:35:23 +0100 [thread overview]
Message-ID: <cover.1645619224.git.ps@pks.im> (raw)
[-- Attachment #1: Type: text/plain, Size: 1860 bytes --]
Hi,
this is another patch series with the aim to speed up mirror fetches. It
applies on top of e6ebfd0e8c (The sixth batch, 2022-02-18) with
3824153b23 (Merge branch 'ps/fetch-atomic' into next, 2022-02-18) merged
into it to fix a conflict.
The patch series is structured as follows:
- Patch 1: Another conversion to use the commit-graph, this time to
look up "want" and "want-ref" lines in git-upload-pack(1).
- Patch 2: This skips useless commit lookups when we don't write
FETCH_HEAD. These were only used to compute the correct order in
which we append to FETCH_HEAD.
- Patch 3-5: This optimizes the way we look up symbolic refs in the
files backend. Currently we always search the packed-refs file,
which cannot ever contain symbolic references.
In total, this patch series brings down the time it takes to replicate
one of our biggest repos in numbers of references from 75 seconds to 60
seconds.
Patrick
Patrick Steinhardt (5):
upload-pack: look up "want" lines via commit-graph
fetch: avoid lookup of commits when not appending to FETCH_HEAD
refs: add ability for backends to special-case reading of symbolic
refs
remote: read symbolic refs via `refs_read_symbolic_ref()`
refs/files-backend: optimize reading of symbolic refs
builtin/fetch.c | 42 +++++++++++++++++++++++++++---------------
builtin/remote.c | 8 +++++---
refs.c | 17 +++++++++++++++++
refs.h | 3 +++
refs/debug.c | 1 +
refs/files-backend.c | 33 ++++++++++++++++++++++++++++-----
refs/packed-backend.c | 1 +
refs/refs-internal.h | 16 ++++++++++++++++
remote.c | 14 +++++++-------
upload-pack.c | 20 +++++++++++++++++---
10 files changed, 122 insertions(+), 33 deletions(-)
--
2.35.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next reply other threads:[~2022-02-23 12:35 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-23 12:35 Patrick Steinhardt [this message]
2022-02-23 12:35 ` [PATCH 1/5] upload-pack: look up "want" lines via commit-graph Patrick Steinhardt
2022-02-23 14:13 ` Derrick Stolee
2022-03-01 8:43 ` Patrick Steinhardt
2022-03-01 9:24 ` Patrick Steinhardt
2022-03-02 18:53 ` Derrick Stolee
2022-02-23 12:35 ` [PATCH 2/5] fetch: avoid lookup of commits when not appending to FETCH_HEAD Patrick Steinhardt
2022-02-23 14:18 ` Derrick Stolee
2022-03-01 8:44 ` Patrick Steinhardt
2022-02-23 12:35 ` [PATCH 3/5] refs: add ability for backends to special-case reading of symbolic refs Patrick Steinhardt
2022-02-23 12:35 ` [PATCH 4/5] remote: read symbolic refs via `refs_read_symbolic_ref()` Patrick Steinhardt
2022-02-23 12:35 ` [PATCH 5/5] refs/files-backend: optimize reading of symbolic refs Patrick Steinhardt
2022-03-01 9:33 ` [PATCH v2 0/5] fetch: more optimizations for mirror fetches Patrick Steinhardt
2022-03-01 9:33 ` [PATCH v2 1/5] upload-pack: look up "want" lines via commit-graph Patrick Steinhardt
2022-03-01 9:33 ` [PATCH v2 2/5] fetch: avoid lookup of commits when not appending to FETCH_HEAD Patrick Steinhardt
2022-03-01 9:33 ` [PATCH v2 3/5] refs: add ability for backends to special-case reading of symbolic refs Patrick Steinhardt
2022-03-01 9:33 ` [PATCH v2 4/5] remote: read symbolic refs via `refs_read_symbolic_ref()` Patrick Steinhardt
2022-03-01 9:33 ` [PATCH v2 5/5] refs/files-backend: optimize reading of symbolic refs Patrick Steinhardt
2022-03-01 22:02 ` [PATCH v2 0/5] fetch: more optimizations for mirror fetches Junio C Hamano
2022-03-02 18:54 ` Derrick Stolee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1645619224.git.ps@pks.im \
--to=ps@pks.im \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).