From: Jeff King <peff@peff.net>
To: Martin Fick <mfick@codeaurora.org>
Cc: "Konstantin Ryabitsev" <konstantin@linuxfoundation.org>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Derrick Stolee" <stolee@gmail.com>,
"Lars Schneider" <larsxschneider@gmail.com>,
git <git@vger.kernel.org>, "Duy Nguyen" <pclouds@gmail.com>
Subject: Re: worktrees vs. alternates
Date: Wed, 16 May 2018 13:06:59 -0700 [thread overview]
Message-ID: <20180516200658.GC4036@sigill.intra.peff.net> (raw)
In-Reply-To: <42435260.5sd4EuToWN@mfick-lnx>
On Wed, May 16, 2018 at 01:40:56PM -0600, Martin Fick wrote:
> > In theory the fetch means that it's safe to actually prune
> > in the mother repo, but in practice there are still
> > races. They don't come up often, but if you have enough
> > repositories, they do eventually. :)
>
> Peff,
>
> I would be very curious to hear what you think of this
> approach to mitigating the effect of those races?
>
> https://git.eclipse.org/r/c/122288/2
The crux of the problem is that we have no way to atomically mark an
object as "I am using this -- do not delete" with respect to the actual
deletion.
So if I'm reading your approach correctly, you put objects into a
purgatory rather than delete them, and let some operations rescue them
from purgatory if we had a race. That's certainly a direction we've
considered, but I think there are some open questions, like:
1. When do you rescue from purgatory? Any time the object is
referenced? Do you then pull in all of its reachable objects too?
2. How do you decide when to drop an object from purgatory? And
specifically, how do you avoid racing with somebody using the
object as you're pruning purgatory?
3. How do you know that an operation has been run that will actually
rescue the object, as opposed to silently having a corrupted state
on disk?
E.g., imagine this sequence:
a. git-prune computes reachability and finds that commit X is
ready to be pruned
b. another process sees that commit X exists and builds a commit
that references it as a parent
c. git-prune drops the object into purgatory
Now we have a corrupt state created by the process in (b), since we
have a reachable object in purgatory. But what if nobody goes back
and tries to read those commits in the meantime?
I think this might be solvable by using the purgatory as a kind of
"lock", where prune does something like:
1. compute reachability
2. move candidate objects into purgatory; nobody can look into
purgatory except us
3. compute reachability _again_, making sure that no purgatory objects
are used (if so, rollback the deletion and try again)
But even that's not quite there, because you need to have some
consistent atomic view of what's "used". Just checking refs isn't
enough, because some other process may be planning to reference a
purgatory object but not yet have updated the ref. So you need some
atomic way of saying "I am interested in using this object".
-Peff
next prev parent reply other threads:[~2018-05-16 20:07 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-16 8:13 worktrees vs. alternates Lars Schneider
2018-05-16 9:29 ` Ævar Arnfjörð Bjarmason
2018-05-16 9:42 ` Robert P. J. Day
2018-05-16 11:07 ` Ævar Arnfjörð Bjarmason
2018-05-16 9:51 ` Lars Schneider
2018-05-16 10:33 ` Ævar Arnfjörð Bjarmason
2018-05-16 13:02 ` Derrick Stolee
2018-05-16 14:58 ` Konstantin Ryabitsev
2018-05-16 15:34 ` Ævar Arnfjörð Bjarmason
2018-05-16 15:49 ` Konstantin Ryabitsev
2018-05-16 17:54 ` Ævar Arnfjörð Bjarmason
2018-05-16 17:14 ` Martin Fick
2018-05-16 17:41 ` Konstantin Ryabitsev
2018-05-16 18:02 ` Ævar Arnfjörð Bjarmason
2018-05-16 18:12 ` Konstantin Ryabitsev
2018-05-16 18:26 ` Martin Fick
2018-05-16 19:01 ` Konstantin Ryabitsev
2018-05-16 19:03 ` Martin Fick
2018-05-16 19:11 ` Konstantin Ryabitsev
2018-05-16 19:18 ` Martin Fick
2018-05-16 19:23 ` Jeff King
2018-05-16 19:29 ` Konstantin Ryabitsev
2018-05-16 19:37 ` Jeff King
2018-05-16 19:40 ` Martin Fick
2018-05-16 20:06 ` Jeff King [this message]
2018-05-16 20:43 ` Martin Fick
2018-05-16 20:02 ` Konstantin Ryabitsev
2018-05-16 20:17 ` Jeff King
2018-05-17 0:43 ` Sitaram Chamarty
2018-05-17 3:31 ` Jeff King
2018-05-19 5:45 ` Duy Nguyen
2018-05-16 19:14 ` Jeff King
2018-05-16 21:18 ` Stefan Beller
2018-05-16 23:45 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180516200658.GC4036@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=konstantin@linuxfoundation.org \
--cc=larsxschneider@gmail.com \
--cc=mfick@codeaurora.org \
--cc=pclouds@gmail.com \
--cc=stolee@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).