git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Taylor Blau <me@ttaylorr.com>
To: git@vger.kernel.org
Cc: "Derrick Stolee" <derrickstolee@github.com>,
	"Jeff King" <peff@peff.net>,
	"Jonathan Tan" <jonathantanmy@google.com>,
	"Junio C Hamano" <gitster@pobox.com>,
	"Victoria Dye" <vdye@github.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH 0/4] repack: implement `--expire-to` option
Date: Mon, 24 Oct 2022 14:42:58 -0400	[thread overview]
Message-ID: <cover.1666636974.git.me@ttaylorr.com> (raw)

This series polishes off the RFC [1] I sent over the summer teaching
`git repack` how to move pruned objects into a separate cruft pack in a
different repository.

We've been using this series at GitHub for some time now as an
escape-hatch mechanism for quickly recovering pruned objects that run
afoul of the "advertise-then-prune" race described in [2].

The general idea is that we write out a second cruft pack to capture the
pruned objects, with two minor changes:

  - The second cruft pack counts the first cruft pack as "saved",
    meaning that we'll exclude its objects (thus ignoring both reachable
    and unreachable-but-recent objects from the second cruft pack).

  - The second cruft pack does *not* exclude objects based on age (i.e.,
    it acts as if `--cruft-expiration=never` were given) so as to avoid
    excluding objects which are being pruned due to their age.

The first few patches are preparatory, and the substantive change is
implemented in the fourth patch. I would suggest that reviewers start
there, and then read the first three patches.

Since this series basically boils down to adding one new conditional,
like:

    if (delete_redundant && expire_to) {
        ret = write_cruft_pack(...):
        if (ret)
            return ret;
    }

The (second) call to `write_cruft_pack()` comes with a rather extensive
inline comment explaining what is going on ;-).

Thanks in advance for your review.

[1]: https://lore.kernel.org/git/cover.1656528343.git.me@ttaylorr.com/
[2]: https://lore.kernel.org/git/20190319001829.GL29661@sigill.intra.peff.net/

Taylor Blau (4):
  builtin/repack.c: pass "out" to `prepare_pack_objects`
  builtin/repack.c: pass "cruft_expiration" to `write_cruft_pack`
  builtin/repack.c: write cruft packs to arbitrary locations
  builtin/repack.c: implement `--expire-to` for storing pruned objects

 Documentation/git-repack.txt |   6 ++
 builtin/repack.c             |  67 ++++++++++++++++---
 t/t7700-repack.sh            | 121 +++++++++++++++++++++++++++++++++++
 3 files changed, 186 insertions(+), 8 deletions(-)

-- 
2.38.0.16.g393fd4c6db

             reply	other threads:[~2022-10-24 20:35 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-24 18:42 Taylor Blau [this message]
2022-10-24 18:43 ` [PATCH 1/4] builtin/repack.c: pass "out" to `prepare_pack_objects` Taylor Blau
2022-10-24 20:47   ` Junio C Hamano
2022-11-07 19:28     ` Derrick Stolee
2022-10-24 18:43 ` [PATCH 2/4] builtin/repack.c: pass "cruft_expiration" to `write_cruft_pack` Taylor Blau
2022-10-24 20:50   ` Junio C Hamano
2022-10-24 18:43 ` [PATCH 3/4] builtin/repack.c: write cruft packs to arbitrary locations Taylor Blau
2022-10-24 21:30   ` Junio C Hamano
2022-10-28 19:42     ` Taylor Blau
2022-11-07 19:32   ` Derrick Stolee
2022-11-07 19:52     ` Taylor Blau
2022-10-24 18:43 ` [PATCH 4/4] builtin/repack.c: implement `--expire-to` for storing pruned objects Taylor Blau
2022-11-07 19:42   ` Derrick Stolee
2022-11-07 19:52     ` Taylor Blau

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1666636974.git.me@ttaylorr.com \
    --to=me@ttaylorr.com \
    --cc=avarab@gmail.com \
    --cc=derrickstolee@github.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@google.com \
    --cc=peff@peff.net \
    --cc=vdye@github.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).