Re: [PATCH v3 2/2] setup: don't fail if commondir reference is deleted.

[Eric Sunshine <sunshine@sunshineco.com>]

On Wed, Feb 20, 2019 at 12:16 PM Michal Suchánek <msuchanek@suse.de> wrote:
> On Wed, 20 Feb 2019 11:55:46 -0500
> Eric Sunshine <sunshine@sunshineco.com> wrote:
> > On Wed, Feb 20, 2019 at 11:17 AM Michal Suchanek <msuchanek@suse.de> wrote:
> > > Apparently it can happen that stat() claims there is a commondir file but when
> > > trying to open the file it is missing.
> >
> > Under what circumstances?
>
> I would like to know that as well. The only command tested was worktree
> add which should not remove the file. Nonetheless running many woktree
> add commands in parallel can cause the file to go away for some of
> them.

You actually encountered this particular error message, correct? Was
that before or after you fixed the race in builtin/worktree.c itself
via patch 1/2? Did the reported 'errno' indicate that the file did not
exist or was it some other error?

> For many commands git calls itself recursively so there is
> probably much more going on than the single function that creates the
> worktree.

"git worktree add" is careful to invoke other Git commands only after
"commondir" exists, so it's not clear how this circumstance arises if
the file is indeed missing by the time the other Git command is run.

> > > Another even rarer issue is that the file might be zero size because another
> > > process initializing a worktree opened the file but has not written is content
> > > yet.
> >
> > Based upon the explanation thus far, I'm having trouble understanding
> > under what circumstances these race conditions can arise. Are you
> > trying to invoke Git commands in a particular worktree even as the
> > worktree itself is being created?
>
> It's explained in the following paragraph. If you have multiple
> worktrees some *other* worktreee may be uninitialized.

I understand that, but setup.c:get_common_dir_noenv() is concerned
only with _this_ worktree -- the one in which the Git command is being
run -- so it's not clear if or how some other partially-initialized
worktree could have any impact. (And, I'm having trouble fathoming how
it could, which is why I'm asking these questions).

Is it possible that when you saw that error message, it actually arose
from some code other than setup.c:get_common_dir_noenv()?

> > > -       if (file_exists(path.buf)) {
> > > -               if (strbuf_read_file(&data, path.buf, 0) <= 0)
> > > +       ret = strbuf_read_file(&data, path.buf, 0);
> > > +       if (ret <= 0) {
> > > +               /*
> > > +                * if file is missing or zero size (just being written)
> > > +                * assume default, bail otherwise
> > > +                */
> > > +               if (ret && errno != ENOENT && errno != ENOTDIR)
> > >                         die_errno(_("failed to read %s"), path.buf);
> >
> > It's not clear from the explanation given in the commit message if the
> > new behavior is indeed sensible. The original intent of the code, as I
> > understand it, is to validate "commondir", to ensure that it is not
> > somehow corrupt (such as the user editing it and making it empty).
>
> How is it validated in the code below when it is non-zero size?

Checking whether the file has content _is_ a form of validation, even
if not extensive validation.

> There is *no* validation whatsoever. Yet zero size is somehow totally
> unacceptable and requires that git working in *any* worktree aborts if
> commondir file in *any* worktree is zero size.

As noted above, it's not clear from the commit message how this case
can arise given that setup.c:get_common_dir_noenv() is presumably
concerned with and only consults _this_ worktree, so I'm having
trouble understanding how the state of other worktrees could impact
it.

> > Following this change, that particular validation no longer takes
> > place. But, more importantly, what does it mean to fall back to
> > "default" for this particular worktree? I'm having trouble
> > understanding how the new behavior can be correct or desirable. (Am I
> > missing something obvious?)
>
> If the file can be missing altogether and it is not an error how it is
> incorrect or undesirable to ignore zero size file?

Because the _presence_ of that file indicates a linked worktree,
whereas it's absence indicates the main worktree. If the file is
present but empty, then that is an abnormal condition, i.e. some form
of corruption.

The difference is significant, and that's why I'm asking if the new
behavior is correct or desirable. If you start interpreting this
abnormal condition as a non-error, then get_common_dir_noenv() will be
reporting that this is the main worktree when in fact it is (a somehow
corrupted) linked worktree. Such false reporting could trigger
undesirable and outright wrong behavior in callers.