git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Michal Suchánek" <msuchanek@suse.de>
To: Phillip Wood <phillip.wood@talktalk.net>
Cc: phillip.wood@dunelm.org.uk,
	Eric Sunshine <sunshine@sunshineco.com>,
	Duy Nguyen <pclouds@gmail.com>,
	Git Mailing List <git@vger.kernel.org>,
	Marketa Calabkova <mcalabkova@suse.cz>,
	Junio C Hamano <gitster@pobox.com>
Subject: Re: [PATCH 2/2] setup: don't fail if commondir reference is deleted.
Date: Mon, 4 Mar 2019 14:30:02 +0100	[thread overview]
Message-ID: <20190304143002.56a6c4ec@kitsune.suse.cz> (raw)
In-Reply-To: <4309841e-2b98-22d4-505e-1b9ea2f5e3bb@talktalk.net>

Hello,

On Thu, 21 Feb 2019 17:27:04 +0000
Phillip Wood <phillip.wood@talktalk.net> wrote:

> Hi Eric
> 
> On 21/02/2019 17:12, Eric Sunshine wrote:
> > On Thu, Feb 21, 2019 at 12:07 PM Phillip Wood <phillip.wood@talktalk.net> wrote:  
> >> On 21/02/2019 13:50, Michal Suchánek wrote:  
> >>>> On Tue, Feb 19, 2019 at 12:05 AM Michal Suchanek <msuchanek@suse.de> wrote:  
> >>> The problem is we don't forbid worktree names ending with ".lock".
> >>> Which means that if we start to forbid them now existing worktrees
> >>> might become inaccessible.  
> >>
> >> I think it is also racy as the renaming breaks the use of mkdir erroring
> >> out if the directory already exists. One solution is to have a lock
> >> entry in $GIT_COMMON_DIR/worktree-locks and make sure the code that
> >> iterates over the entries in $GIT_COMMON_DIR/worktrees skips any that
> >> have a corresponding ignores in $GIT_COMMON_DIR/worktree-locks. If the
> >> worktree-locks/<dir> is created before worktree/<dir> then it should be
> >> race free (you will have to remove the lock if the real entry cannot be
> >> created and then increment the counter and try again). Entries could
> >> also be locked on removal to prevent a race there.  
> > 
> > I wonder, though, how much this helps or hinders the use-case which
> > prompted this patch series in the first place; to wit, creating
> > hundreds or thousands of worktrees. Doing so serially was too slow, so
> > the many "git worktree add" invocations were instead run in parallel
> > (which led to "discovery" of race conditions). Using a global worktree
> > lock would serialize worktree creation, thus slowing it down once
> > again.  
> 
> The idea is that there are per-worktree lock stored under worktree-locks 
> (hence the plural name). Using a separate directory for the locks gets 
> round the problems of name clashes between the lock for a worktree 
> called foo and one called foo.lock and means we can rely on mkdir 
> erroring out if the worktree name already exists as there is no renaming.

I suppose this separate directory would work. When are you supposed to
take the lock, though?

When adding worktree, sure.

When managing worktrees, sure. Otherwise you would see the incomplete
worktrees.

When doing anything in git? Probably. Because otherwise you could
accidentally use the incomplete worktree. Or somebody deleting worktree
would fail removing it because you would keep adding files to it.

Isn't git supposed to allow parallel access to the repository?

As things stand if you wanted to implement worktree locking you would
need to lock the worktree for *every* operation that touches it, and
for many operations you would have to lock/unlock *all* worktrees one by
one to find the worktree you are supposed to work on.

I don't feel like adding locking to all of git to fix this problem.

Sure, adding enough locking to ensure repository consistency at all
times would be nice but it also needs to be granular enough to not harm
performance. I can't say I understand the git repository layout and
usage well enough to design that.

Thanks

Michal

  reply	other threads:[~2019-03-04 13:30 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-18 17:04 [PATCH 0/2] worktree add race fix Michal Suchanek
2019-02-18 17:04 ` [PATCH 1/2] worktree: fix worktree add race Michal Suchanek
2019-02-18 17:04 ` [PATCH 2/2] setup: don't fail if commondir reference is deleted Michal Suchanek
2019-02-18 21:00   ` Eric Sunshine
2019-02-21 10:50   ` Duy Nguyen
2019-02-21 13:50     ` Michal Suchánek
2019-02-21 17:07       ` Phillip Wood
2019-02-21 17:12         ` Eric Sunshine
2019-02-21 17:27           ` Phillip Wood
2019-03-04 13:30             ` Michal Suchánek [this message]
2019-02-21 17:33           ` Michal Suchánek
2019-02-22  9:32         ` Duy Nguyen
2019-02-22 10:20           ` Phillip Wood
2019-02-22  9:26       ` Duy Nguyen
2019-02-20 16:16 ` [PATCH v3 1/2] worktree: fix worktree add race Michal Suchanek
2019-02-20 16:34   ` Eric Sunshine
2019-02-20 17:29     ` Michal Suchánek
2019-03-08  9:20   ` Duy Nguyen
2019-03-08  9:37     ` Eric Sunshine
2019-03-11  1:55     ` Junio C Hamano
2019-02-20 16:16 ` [PATCH v3 2/2] setup: don't fail if commondir reference is deleted Michal Suchanek
2019-02-20 16:55   ` Eric Sunshine
2019-02-20 17:16     ` Michal Suchánek
2019-02-20 18:35       ` Eric Sunshine
2019-02-21  9:27         ` Eric Sunshine
2019-02-21 11:13           ` Michal Suchánek
2019-02-21 11:19         ` Michal Suchánek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190304143002.56a6c4ec@kitsune.suse.cz \
    --to=msuchanek@suse.de \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=mcalabkova@suse.cz \
    --cc=pclouds@gmail.com \
    --cc=phillip.wood@dunelm.org.uk \
    --cc=phillip.wood@talktalk.net \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).