git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Elijah Newren <newren@gmail.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: Glen Choo <chooglen@google.com>,
	Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>,
	Git Mailing List <git@vger.kernel.org>, Jeff King <peff@peff.net>
Subject: Re: [PATCH 8/8] dir: avoid removing the current working directory
Date: Tue, 23 Nov 2021 20:35:44 -0800	[thread overview]
Message-ID: <CABPp-BGZPheDqWD1pXbePOYe2BcT1-TqX-zGVqnNLGtFuXab1g@mail.gmail.com> (raw)
In-Reply-To: <211124.86ee76e4fl.gmgdl@evledraar.gmail.com>

On Tue, Nov 23, 2021 at 5:19 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Tue, Nov 23 2021, Elijah Newren wrote:
>
> > On Tue, Nov 23, 2021 at 10:19 AM Glen Choo <chooglen@google.com> wrote:
> >>
> >> Elijah Newren <newren@gmail.com> writes:
> >> [...]
> >> I'm not going to say that we'll *definitely* need remove_path()
> >> in its current form, but mixing concerns like this is an invitation to
> >> unexpected behavior. An (imperfect) example that demonstrates this
> >> principle is https://lore.kernel.org/git/24bffdab139435173712101aaf72f7277298c99d.1632497954.git.gitgitgadget@gmail.com/,
> >> where we made a change to a generic path matching function in order to
> >> speed up unpack_trees(), but accidentally ended up breaking gitignore.
> >
> > There's no mixture of concerns; my patch is correcting this library
> > function to more fully match its documented intent; from dir.h:
> >
> >     /* tries to remove the path with empty directories along it,
> > ignores ENOENT */
> >     int remove_path(const char *path);
> >
> > Since the parent process's current working directory is still likely
> > parked in that directory, there is a good reason to treat it as
> > non-empty.  Thus the cwd should not be one of those directories
> > removed along with the specified path.  No need to die, just stop
> > removing the leading directories once it hits the cwd (much like it'd
> > stop once it hit a directory that had files left in it).
>
> I can buy that we'd pick this new behavior as a worthwhile trade-off,
> but not that anyone intended for this to be the behavior all along.
>
> I don't think "a process is sitting in it" has ever been anyone's idea
> of a "non-empty directory". Rather it's what rmdir() returning EEXIST or
> ENOTEMPTY maps to.

Yeah, Junio commented on my reasoning in that same paragraph of mine.
Bad reasoning on my part, and you were both right to call it out.

But that reasoning wasn't the underlying motivation for Peff to
suggest the behavior behind this series[1] and this patch, nor the
rationale Junio used to say that the overall behavioral change behind
this series "makes sense".[2]

[1] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/
[2] https://lore.kernel.org/git/xmqqo86elyht.fsf@gitster.g/

> Doesn't this series also change the behavior of e.g.:
>
>     cd contrib/subtree
>     git rm -r ../subtree

Yes, of course!

Before:

    $ cd contrib/subtree
    $ git rm -r -q ../subtree/
    $ ls -ld ../subtree/
    ls: cannot access '../subtree/': No such file or directory
    $ git status --porcelain
    fatal: Unable to read current working directory: No such file or directory
    $ git checkout HEAD ../subtree/
    fatal: Unable to read current working directory: No such file or directory

After:

    $ cd contrib/subtree
    $ git rm -r -q ../subtree/
    $ ls -ld ../subtree/
    drwxrwxr-x. 1 newren newren 0 Nov 23 19:18 ../subtree/
    $ git status --porcelain
    D  contrib/subtree/.gitignore
    D  contrib/subtree/COPYING
    D  contrib/subtree/INSTALL
    D  contrib/subtree/Makefile
    D  contrib/subtree/README
    D  contrib/subtree/git-subtree.sh
    D  contrib/subtree/git-subtree.txt
    D  contrib/subtree/t/Makefile
    D  contrib/subtree/t/t7900-subtree.sh
    D  contrib/subtree/todo
    $ git checkout HEAD ../subtree/
    Updated 10 paths from c557be478e

Very nice fix, don't you think?


> If so then re the "spidey sense" comment I had earlier: There's no rm
> codepaths or tests changed by this series,

That's not correct; I explicitly added a new rm test in the first
patch in my series.  Further, that same test was modified to mark it
as passing by this particular patch you are commenting on.

> so the implementation of
> doing it at this lower level might be casting too wide a net.

I'm getting the vibe that you are assuming I'm changing these two
functions without realizing what places might be calling them;
basically, that I'm just flippantly changing them.  Ignoring the
ramifications of such an assumption, if this vibe is correct, then let
me inform you that I've read over each and every caller (as well as
searched for other callers of unlink() and rmdir() throughout the tree
to see if they needed similar changes).  In my opinion, *each* *and*
*every* *single* *one* of the calls to remove_path() and
remove_dir_recursively() should take the behavioral change suggested
in this patch.

It's also not clear to me that you understand the point of the change
behind the series.  Clearly, I'm not doing well explaining it, but
have you read Peff's or Junio's comments on why they thought
protecting the_original_cwd makes sense?  Again, see the links [1] and
[2] above.  I think it'd help me understand how to respond to you
better if you could clarify to me whether you disagree with them, or
whether you agree with them but think I've gone wrong in the
implementation of their high level explanation somehow in this
particular patch.

> e.g. changing callers that use "remove_dir_recursively()" to use a
> "remove_dir_recursively_not_cwd()" (or whatever) be a gentler way of
> introducing this, and make sure that each step of the way we grok what's
> being changed, that there's test coverage etc.

I'll ask you the same two questions I asked Glen when he suggested
basically the same thing; if you can provide an answer to either one
of my questions that is compelling to me, I'd be totally onboard with
your suggested change:

(1) Can you point to any concrete example caller anywhere in the code
tree (or even provide a future hypothetical caller) that would in fact
be harmed by the change in this patch?

...and no, I'm not asking you to do my work for me; I've done that
same work -- in fact looking at all callsites -- and came up
empty-handed.  Since this and your previous email are essentially
claiming that I've probably missed things, I think it's a fair
question.

(2) What benefit would there be to introducing these new functions?

In particular, if I were to introduce these new functions, it would
look like this:

  * add new remove_path_not_cwd() and remove_dir_recursively_not_cwd()
function in one patch
  * convert all relevant callsites to use these new functions in the
subsequent patch(es)
  * delete the existing remove_path() and remove_dir_recursively()
functions for two reasons: (1) they are now unused, and (2) future
potential callers of these old functions would more than likely
reintroduce the bugs this series is trying to fix if they were to use
them and should be discouraged from doing so
  * rename {remove_path,remove_dir_recursively}_not_cwd() to remove
the "_not_cwd" suffix in order to have more memorable and less ugly
names

Once finished, the end result would be identical to this patch.  Well,
identical other than taking more patches to get to the same end result
and using up more reviewer time.  If there's some benefit to taking
this circuitous route, though, I'm more than willing to do so.

  reply	other threads:[~2021-11-24  4:36 UTC|newest]

Thread overview: 163+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-21  0:46 [PATCH 0/8] Avoid removing the current working directory, even if it becomes empty Elijah Newren via GitGitGadget
2021-11-21  0:46 ` [PATCH 1/8] t2501: add various tests for removing the current working directory Elijah Newren via GitGitGadget
2021-11-21 17:57   ` Ævar Arnfjörð Bjarmason
2021-11-23  1:45     ` Elijah Newren
2021-11-23  2:19       ` Ævar Arnfjörð Bjarmason
2021-11-23  3:11         ` Elijah Newren
2021-11-25 10:04           ` Ævar Arnfjörð Bjarmason
2021-11-21  0:46 ` [PATCH 2/8] repository, setup: introduce the_cwd Elijah Newren via GitGitGadget
2021-11-21  8:00   ` Junio C Hamano
2021-11-22 22:38     ` Elijah Newren
2021-11-21  8:56   ` René Scharfe
2021-11-22 23:09     ` Elijah Newren
2021-11-21  0:46 ` [PATCH 3/8] unpack-trees: refuse to remove the current working directory Elijah Newren via GitGitGadget
2021-11-21  0:46 ` [PATCH 4/8] unpack-trees: add special cwd handling Elijah Newren via GitGitGadget
2021-11-21  0:46 ` [PATCH 5/8] symlinks: do not include current working directory in dir removal Elijah Newren via GitGitGadget
2021-11-21  8:56   ` René Scharfe
2021-11-23  0:35     ` Elijah Newren
2021-11-21  0:46 ` [PATCH 6/8] clean: do not attempt to remove current working directory Elijah Newren via GitGitGadget
2021-11-21 17:51   ` Ævar Arnfjörð Bjarmason
2021-11-23  1:28     ` Elijah Newren
2021-11-21  0:46 ` [PATCH 7/8] stash: " Elijah Newren via GitGitGadget
2021-11-21  0:47 ` [PATCH 8/8] dir: avoid removing the " Elijah Newren via GitGitGadget
2021-11-23  0:39   ` Glen Choo
2021-11-23  1:19     ` Elijah Newren
2021-11-23 18:19       ` Glen Choo
2021-11-23 19:56         ` Elijah Newren
2021-11-23 20:32           ` Glen Choo
2021-11-23 21:57             ` Junio C Hamano
2021-11-23 23:23               ` Elijah Newren
2021-11-24  5:46                 ` Junio C Hamano
2021-11-23 23:13             ` Elijah Newren
2021-11-24  0:39               ` Glen Choo
2021-11-24  5:46                 ` Junio C Hamano
2021-11-24  1:10           ` Ævar Arnfjörð Bjarmason
2021-11-24  4:35             ` Elijah Newren [this message]
2021-11-24 11:14               ` Ævar Arnfjörð Bjarmason
2021-11-24 14:11                 ` Ævar Arnfjörð Bjarmason
2021-11-25  2:54                   ` Elijah Newren
2021-11-25 11:12                     ` Ævar Arnfjörð Bjarmason
2021-11-26 21:40                       ` The overhead of bin-wrappers/ (was: [PATCH 8/8] dir: avoid removing the current working directory) Ævar Arnfjörð Bjarmason
2021-11-24 14:33                 ` [PATCH 8/8] dir: avoid removing the current working directory Philip Oakley
2021-11-24 19:46                   ` Junio C Hamano
2021-11-25 12:54                     ` Philip Oakley
2021-11-25 13:51                       ` Ævar Arnfjörð Bjarmason
2021-11-25  2:48                 ` Elijah Newren
2021-11-24 19:43               ` Junio C Hamano
2021-11-21  8:11 ` [PATCH 0/8] Avoid removing the current working directory, even if it becomes empty Junio C Hamano
2021-11-25  8:39 ` [PATCH v2 0/9] " Elijah Newren via GitGitGadget
2021-11-25  8:39   ` [PATCH v2 1/9] t2501: add various tests for removing the current working directory Elijah Newren via GitGitGadget
2021-11-25 10:21     ` Ævar Arnfjörð Bjarmason
2021-11-25  8:39   ` [PATCH v2 2/9] setup: introduce startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-25 10:44     ` Ævar Arnfjörð Bjarmason
2021-11-26 17:55       ` Elijah Newren
2021-11-26  6:52     ` Junio C Hamano
2021-11-26 18:01       ` Elijah Newren
2021-11-29 14:05     ` Derrick Stolee
2021-11-29 17:18       ` Elijah Newren
2021-11-29 17:43         ` Derrick Stolee
2021-11-29 17:42       ` Junio C Hamano
2021-11-25  8:39   ` [PATCH v2 3/9] unpack-trees: refuse to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-25 10:56     ` Ævar Arnfjörð Bjarmason
2021-11-26 18:06       ` Elijah Newren
2021-11-29 14:10     ` Derrick Stolee
2021-11-29 17:26       ` Elijah Newren
2021-11-25  8:39   ` [PATCH v2 4/9] unpack-trees: add special cwd handling Elijah Newren via GitGitGadget
2021-11-29 14:14     ` Derrick Stolee
2021-11-29 17:33       ` Elijah Newren
2021-11-25  8:39   ` [PATCH v2 5/9] symlinks: do not include startup_info->original_cwd in dir removal Elijah Newren via GitGitGadget
2021-11-25  8:39   ` [PATCH v2 6/9] clean: do not attempt to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-25  8:39   ` [PATCH v2 7/9] stash: " Elijah Newren via GitGitGadget
2021-11-25 10:58     ` Ævar Arnfjörð Bjarmason
2021-11-26 18:04       ` Elijah Newren
2021-11-25  8:39   ` [PATCH v2 8/9] dir: avoid incidentally removing the original_cwd in remove_path() Elijah Newren via GitGitGadget
2021-11-25  8:39   ` [PATCH v2 9/9] dir: new flag to remove_dir_recurse() to spare the original_cwd Elijah Newren via GitGitGadget
2021-11-26 22:40   ` [PATCH v3 00/11] Avoid removing the current working directory, even if it becomes empty Elijah Newren via GitGitGadget
2021-11-26 22:40     ` [PATCH v3 01/11] t2501: add various tests for removing the current working directory Elijah Newren via GitGitGadget
2021-11-27 10:32       ` Ævar Arnfjörð Bjarmason
2021-11-27 19:16         ` Elijah Newren
2021-11-26 22:40     ` [PATCH v3 02/11] setup: introduce startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-27 10:35       ` Ævar Arnfjörð Bjarmason
2021-11-27 17:05         ` Elijah Newren
2021-11-27 10:40       ` Ævar Arnfjörð Bjarmason
2021-11-27 18:31         ` Elijah Newren
2021-11-28 18:04           ` Ævar Arnfjörð Bjarmason
2021-11-29 21:58             ` Elijah Newren
2021-11-26 22:40     ` [PATCH v3 03/11] unpack-trees: refuse to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-26 22:40     ` [PATCH v3 04/11] unpack-trees: add special cwd handling Elijah Newren via GitGitGadget
2021-11-26 22:40     ` [PATCH v3 05/11] symlinks: do not include startup_info->original_cwd in dir removal Elijah Newren via GitGitGadget
2021-11-26 22:40     ` [PATCH v3 06/11] clean: do not attempt to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-26 22:40     ` [PATCH v3 07/11] rebase: " Elijah Newren via GitGitGadget
2021-11-29 17:50       ` Derrick Stolee
2021-11-29 19:22         ` Elijah Newren
2021-11-29 19:42           ` Derrick Stolee
2021-11-26 22:40     ` [PATCH v3 08/11] stash: " Elijah Newren via GitGitGadget
2021-11-26 22:41     ` [PATCH v3 09/11] dir: avoid incidentally removing the original_cwd in remove_path() Elijah Newren via GitGitGadget
2021-11-26 22:41     ` [PATCH v3 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd Elijah Newren via GitGitGadget
2021-11-26 22:41     ` [PATCH v3 11/11] t2501: simplify the tests since we can now assume desired behavior Elijah Newren via GitGitGadget
2021-11-29 17:57     ` [PATCH v3 00/11] Avoid removing the current working directory, even if it becomes empty Derrick Stolee
2021-11-29 22:37     ` [PATCH v4 " Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 01/11] t2501: add various tests for removing the current working directory Elijah Newren via GitGitGadget
2021-11-30  6:47         ` Junio C Hamano
2021-11-30  6:53           ` Elijah Newren
2021-11-29 22:37       ` [PATCH v4 02/11] setup: introduce startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 03/11] unpack-trees: refuse to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 04/11] unpack-trees: add special cwd handling Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 05/11] symlinks: do not include startup_info->original_cwd in dir removal Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 06/11] clean: do not attempt to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 07/11] rebase: " Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 08/11] stash: " Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 09/11] dir: avoid incidentally removing the original_cwd in remove_path() Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd Elijah Newren via GitGitGadget
2021-11-29 22:37       ` [PATCH v4 11/11] t2501: simplify the tests since we can now assume desired behavior Elijah Newren via GitGitGadget
2021-11-29 23:38       ` [PATCH v4 00/11] Avoid removing the current working directory, even if it becomes empty Eric Sunshine
2021-11-30  0:16         ` Elijah Newren
2021-12-01  6:40       ` [PATCH v5 " Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 01/11] t2501: add various tests for removing the current working directory Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 02/11] setup: introduce startup_info->original_cwd Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 03/11] unpack-trees: refuse to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 04/11] unpack-trees: add special cwd handling Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 05/11] symlinks: do not include startup_info->original_cwd in dir removal Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 06/11] clean: do not attempt to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 07/11] rebase: " Elijah Newren via GitGitGadget
2022-01-25 20:26           ` [Bug] Rebase from worktree subdir is broken (was Re: [PATCH v5 07/11] rebase: do not attempt to remove startup_info->original_cwd) Glen Choo
2022-01-25 23:59             ` Elijah Newren
2022-01-26  0:30               ` Glen Choo
2022-01-26 19:04                 ` Elijah Newren
2022-01-26  0:32               ` Eric Sunshine
2022-01-26  0:38                 ` Eric Sunshine
2022-01-26  0:51                   ` Elijah Newren
2022-01-26  1:15                     ` Glen Choo
2022-01-26  1:38                       ` Elijah Newren
2022-01-26 11:00               ` Phillip Wood
2022-01-26 17:53                 ` Eric Sunshine
2022-01-27 11:01                   ` Phillip Wood
2022-01-27 20:03                   ` Elijah Newren
2022-02-05 11:23                     ` Eric Sunshine
2022-02-05 11:42                       ` Eric Sunshine
2022-02-05 22:35                         ` Elijah Newren
2021-12-01  6:40         ` [PATCH v5 08/11] stash: do not attempt to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 09/11] dir: avoid incidentally removing the original_cwd in remove_path() Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd Elijah Newren via GitGitGadget
2021-12-01  6:40         ` [PATCH v5 11/11] t2501: simplify the tests since we can now assume desired behavior Elijah Newren via GitGitGadget
2021-12-07 16:09         ` [PATCH v5 00/11] Avoid removing the current working directory, even if it becomes empty Derrick Stolee
2021-12-07 18:30           ` Ævar Arnfjörð Bjarmason
2021-12-07 20:57             ` Derrick Stolee
2021-12-08 10:23               ` Ævar Arnfjörð Bjarmason
2021-12-07 20:43           ` Elijah Newren
2021-12-07 21:00             ` Derrick Stolee
2021-12-09  5:08         ` [PATCH v6 " Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 01/11] t2501: add various tests for removing the current working directory Elijah Newren via GitGitGadget
2022-03-11 11:57             ` Ævar Arnfjörð Bjarmason
2021-12-09  5:08           ` [PATCH v6 02/11] setup: introduce startup_info->original_cwd Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 03/11] unpack-trees: refuse to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 04/11] unpack-trees: add special cwd handling Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 05/11] symlinks: do not include startup_info->original_cwd in dir removal Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 06/11] clean: do not attempt to remove startup_info->original_cwd Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 07/11] rebase: " Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 08/11] stash: " Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 09/11] dir: avoid incidentally removing the original_cwd in remove_path() Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 10/11] dir: new flag to remove_dir_recurse() to spare the original_cwd Elijah Newren via GitGitGadget
2021-12-09  5:08           ` [PATCH v6 11/11] t2501: simplify the tests since we can now assume desired behavior Elijah Newren via GitGitGadget
2021-11-30 11:04     ` [PATCH v3 00/11] Avoid removing the current working directory, even if it becomes empty Phillip Wood
2021-12-01  0:03       ` Elijah Newren

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CABPp-BGZPheDqWD1pXbePOYe2BcT1-TqX-zGVqnNLGtFuXab1g@mail.gmail.com \
    --to=newren@gmail.com \
    --cc=avarab@gmail.com \
    --cc=chooglen@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).