git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To: "SZEDER Gábor" <szeder.dev@gmail.com>
Cc: Thomas Gummerer <t.gummerer@gmail.com>,
	Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>,
	git@vger.kernel.org
Subject: Re: [PATCH v12 18/26] stash: convert push to builtin
Date: Tue, 19 Feb 2019 11:47:51 +0100 (STD)	[thread overview]
Message-ID: <nycvar.QRO.7.76.6.1902191127420.41@tvgsbejvaqbjf.bet> (raw)
In-Reply-To: <20190219002336.GN1622@szeder.dev>

[-- Attachment #1: Type: text/plain, Size: 4622 bytes --]

Hi Gábor & Thomas,

On Tue, 19 Feb 2019, SZEDER Gábor wrote:

> On Tue, Feb 12, 2019 at 11:18:37PM +0000, Thomas Gummerer wrote:
> > Thanks.  I still didn't manage to reproduce it locally, but I was now
> > able to test it on Travis CI.
> > 
> > The diff below fixes the issue, but I still need to spend some time to
> > better understand why it does.
> 
> There is nothing like a fix that works, but you have no idea why :)

I know why. Now. See below for the analysis.

> FWIW, I'm at a couple of thousands of '--stress' repetitions with your
> patch below, and not a single failure yet.

Good, and yes, there is a problem.

> > I'll hopefully be in a position to
> > send a patch with a proper log message why this is the right fix in
> > the next couple of days.
> > 
> > diff --git a/builtin/stash--helper.c b/builtin/stash--helper.c
> > index c77f62c895..3dab488bd6 100644
> > --- a/builtin/stash--helper.c
> > +++ b/builtin/stash--helper.c
> > @@ -231,6 +231,7 @@ static int reset_tree(struct object_id *i_tree, int update, int reset)
> >  	struct tree *tree;
> >  	struct lock_file lock_file = LOCK_INIT;
> >  
> > +	discard_cache();
> >  	read_cache_preload(NULL);
> >  	if (refresh_cache(REFRESH_QUIET))
> >  		return -1;
> > 

So this is working, but it is not the correct spot for that
`discard_cache()`, as it forces unnecessary cycles on code paths calling
`reset_tree()` (which corresponds to `git read-tree`, admittedly a bit
confusing) with a fully up to date index.

The real fix, I believe, is this:

-- snip --
diff --git a/builtin/stash.c b/builtin/stash.c
index 2d6dfce883..516dee0fa4 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -1372,6 +1372,7 @@ static int do_push_stash(struct pathspec ps, const char *stash_msg, int quiet,
 			}
 		} else {
 			struct child_process cp = CHILD_PROCESS_INIT;
+			discard_cache();
 			cp.git_cmd = 1;
 			argv_array_pushl(&cp.args, "reset", "--hard", "-q",
 					 NULL);
-- snap --

And the reason this is needed: we spawn a `git reset --hard` here, which
will change the index, but outside of the current process. So the
in-process copy is stale. And when the index' mtime does not help us
detect that, we run into that test breakage.

Now, I seriously believe that we missed the best time to move
ps/stash-in-c into `next` for cooking. The best time would have been just
after Paul submitted the latest patch series: we know for a fact that he
is too busy to really take care of this patch series, so keeping it in
`pu` puts everybody into that awkward spot where nobody wants to step on
Paul's toes messing with his patch series, but where Paul also lacks the
time to push it further, so everything is stuck in a limbo and is *so very
much* not cooking at all. You might say that it has turned bad because we
failed to stoke the fire appropriately.

Since it is now way too late in the v2.21.0 process, this problem is only
exacerbated, because it won't even enter `next` "better late than never".

To address this unfortunate situation, my current plan is to take over
from Paul (we had been chatting about this privately in the past, and he
is okay with this because of University eating all his time).

I will open the whole bag again, most likely squashing the late fixups
into the patches that introduced the problems, re-review with a much finer
comb than the patch series has enjoyed on the Git mailing list (even just
a quick look at `do_apply_stash()` revealed an unnecessary `reset_tree()`
call that *no* reviewer spotted, even I myself, but then, I am hardly
solely responsible for that review), and most likely I'll even take my
sweet little time changing the code to avoid more spawned Git processes.

It will take a long time, and the `stash` project that has been discussed
recently to be given to GSoC students is no longer available, as I will
take care of it before GSoC even starts, and I won't spend much time
reviewing other people's code in the meantime.

I will start that only after v2.21.0 final is out, obviously.

Once I submit a new iteration, it will look quite a bit different from
before, and reviewers will have to re-review *everything*, wasting
everybody's time even more. It will have to be re-reviewed in its entirety
anyway because it has been *such* a long time since the latest review, and
that's just the price we all have to pay for missing the right moment to
advance this to `next`. Thomas, I will ask you to review, and Gábor, I
will expect you to review that iteration, too, as you are now a bit
familiar with the code, and I will really need your help here.

Anyway, that's my plan for now.

Ciao,
Dscho

  reply	other threads:[~2019-02-19 10:48 UTC|newest]

Thread overview: 106+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <https://public-inbox.org/git/cover.1542925164.git.ungureanupaulsebastian@gmail.com/>
2018-12-20 19:44 ` [PATCH v12 00/26] Convert "git stash" to C builtin Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 01/26] sha1-name.c: add `get_oidf()` which acts like `get_oid()` Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 02/26] strbuf.c: add `strbuf_join_argv()` Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 03/26] strbuf.c: add `strbuf_insertf()` and `strbuf_vinsertf()` Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 04/26] ident: add the ability to provide a "fallback identity" Paul-Sebastian Ungureanu
2018-12-26 21:21     ` Junio C Hamano
2018-12-27 21:24       ` Johannes Schindelin
2018-12-28 19:40         ` Junio C Hamano
2018-12-20 19:44   ` [PATCH v12 05/26] stash: improve option parsing test coverage Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 06/26] t3903: modernize style Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 07/26] stash: rename test cases to be more descriptive Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 08/26] stash: add tests for `git stash show` config Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 09/26] stash: mention options in `show` synopsis Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 10/26] stash: convert apply to builtin Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 11/26] stash: convert drop and clear " Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 12/26] stash: convert branch " Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 13/26] stash: convert pop " Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 14/26] stash: convert list " Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 15/26] stash: convert show " Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 16/26] stash: convert store " Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 17/26] stash: convert create " Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 18/26] stash: convert push " Paul-Sebastian Ungureanu
2019-02-08 11:30     ` SZEDER Gábor
2019-02-10 22:17       ` Thomas Gummerer
2019-02-11  1:13         ` SZEDER Gábor
2019-02-12 23:18           ` Thomas Gummerer
2019-02-19  0:23             ` SZEDER Gábor
2019-02-19 10:47               ` Johannes Schindelin [this message]
2019-02-19 19:59                 ` Junio C Hamano
2019-02-20 21:01                   ` Johannes Schindelin
2019-02-19 23:59                 ` Thomas Gummerer
2019-02-20  4:37                   ` Junio C Hamano
2019-02-20 21:10                     ` Johannes Schindelin
2019-02-20 22:30                     ` Thomas Gummerer
2019-02-25 23:16                 ` [PATCH v13 00/27] Convert "git stash" to C builtin Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 01/27] sha1-name.c: add `get_oidf()` which acts like `get_oid()` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 02/27] strbuf.c: add `strbuf_join_argv()` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 03/27] strbuf.c: add `strbuf_insertf()` and `strbuf_vinsertf()` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 04/27] ident: add the ability to provide a "fallback identity" Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 05/27] stash: improve option parsing test coverage Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 06/27] t3903: modernize style Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 07/27] t3903: add test for --intent-to-add file Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 08/27] stash: rename test cases to be more descriptive Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 09/27] stash: add tests for `git stash show` config Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 10/27] stash: mention options in `show` synopsis Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 11/27] stash: convert apply to builtin Thomas Gummerer
2019-03-14 13:19                     ` regression in new built-in stash + fsmonitor (was Re: [PATCH v13 11/27] stash: convert apply to builtin) Ævar Arnfjörð Bjarmason
2019-03-14 15:20                       ` Johannes Schindelin
2019-03-14 15:40                         ` Ævar Arnfjörð Bjarmason
2019-03-14 22:45                           ` Johannes Schindelin
2019-03-14 23:39                             ` Ævar Arnfjörð Bjarmason
2019-03-15  2:23                               ` Ben Peart
2019-02-25 23:16                   ` [PATCH v13 12/27] stash: convert drop and clear to builtin Thomas Gummerer
2019-03-07 19:15                     ` Jeff King
2019-03-09 18:30                       ` Thomas Gummerer
2019-03-10 23:26                         ` Jeff King
2019-03-11  1:40                         ` Junio C Hamano
2019-03-11 21:40                           ` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 13/27] stash: convert branch " Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 14/27] stash: convert pop " Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 15/27] stash: convert list " Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 16/27] stash: convert show " Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 17/27] stash: convert store " Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 18/27] stash: convert create " Thomas Gummerer
2019-03-07 19:18                     ` Jeff King
2019-03-08 15:30                       ` Johannes Schindelin
2019-03-09 18:26                         ` Thomas Gummerer
2019-03-11  1:47                           ` Junio C Hamano
2019-03-11  7:30                             ` Junio C Hamano
2019-03-11 21:42                               ` Thomas Gummerer
2019-03-11 22:16                                 ` [PATCH v2] stash: pass pathspec as pointer Thomas Gummerer
2019-03-12  6:50                                   ` Junio C Hamano
2019-03-12 22:35                                   ` Johannes Schindelin
2019-03-12 23:40                                     ` Thomas Gummerer
2019-03-13  1:47                                       ` Junio C Hamano
2019-03-13 22:14                                       ` Johannes Schindelin
2019-03-15 22:33                                         ` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 19/27] stash: convert push to builtin Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 20/27] stash: make push -q quiet Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 21/27] stash: convert save to builtin Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 22/27] stash: optimize `get_untracked_files()` and `check_changes()` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 23/27] stash: replace all `write-tree` child processes with API calls Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 24/27] stash: convert `stash--helper.c` into `stash.c` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 25/27] stash: add back the original, scripted `git stash` Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 26/27] stash: optionally use the scripted version again Thomas Gummerer
2019-02-25 23:16                   ` [PATCH v13 27/27] tests: add a special setup where stash.useBuiltin is off Thomas Gummerer
2019-02-26 12:40                   ` [PATCH v13 00/27] Convert "git stash" to C builtin Johannes Schindelin
2019-02-26 20:48                     ` Thomas Gummerer
2019-02-26 21:45                   ` Ævar Arnfjörð Bjarmason
2019-02-26 22:37                     ` Johannes Schindelin
2019-03-03  1:24                   ` Junio C Hamano
2019-03-03  1:25                   ` Junio C Hamano
2019-03-03 10:03                     ` Thomas Gummerer
2018-12-20 19:44   ` [PATCH v12 19/26] stash: make push -q quiet Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 20/26] stash: convert save to builtin Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 21/26] stash: optimize `get_untracked_files()` and `check_changes()` Paul-Sebastian Ungureanu
2019-01-06 22:47     ` Thomas Gummerer
2018-12-20 19:44   ` [PATCH v12 22/26] stash: replace all `write-tree` child processes with API calls Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 23/26] stash: convert `stash--helper.c` into `stash.c` Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 24/26] stash: add back the original, scripted `git stash` Paul-Sebastian Ungureanu
2018-12-20 19:44   ` [PATCH v12 25/26] stash: optionally use the scripted version again Paul-Sebastian Ungureanu
2019-01-06 22:59     ` Thomas Gummerer
2018-12-20 19:44   ` [PATCH v12 26/26] tests: add a special setup where stash.useBuiltin is off Paul-Sebastian Ungureanu
2019-01-03 23:39   ` [PATCH v12 00/26] Convert "git stash" to C builtin Junio C Hamano
2019-01-18 12:06     ` Johannes Schindelin
2019-01-18 17:49       ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=nycvar.QRO.7.76.6.1902191127420.41@tvgsbejvaqbjf.bet \
    --to=johannes.schindelin@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=szeder.dev@gmail.com \
    --cc=t.gummerer@gmail.com \
    --cc=ungureanupaulsebastian@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).