should "git bisect" support "git bisect next?"

should "git bisect" support "git bisect next?"

[Robert P. J. Day]

  the man page for "git bisect" makes no mention of "git bisect next",
but the script git-bisect.sh does:

#!/bin/sh

USAGE='[help|start|bad|good|new|old|terms|skip|next|reset|visualize|replay|log|run]'
                                               ^^^^
LONG_USAGE='git bisect help
        print this long help message.
git bisect start [--term-{old,good}=<term> --term-{new,bad}=<term>]
                 [--no-checkout] [<bad> [<good>...]] [--] [<pathspec>...]
        reset bisect state and start bisection.
git bisect (bad|new) [<rev>]
        mark <rev> a known-bad revision/
                a revision after change in a given property.
git bisect (good|old) [<rev>...]
        mark <rev>... known-good revisions/
                revisions before change in a given property.
git bisect terms [--term-good | --term-bad]
        show the terms used for old and new commits (default: bad, good)
git bisect skip [(<rev>|<range>)...]
        mark <rev>... untestable revisions.
git bisect next
        find next bisection to test and check it out.

  ... snip ...

case "$#" in
0)
        usage ;;
*)
        cmd="$1"
        get_terms
        shift
        case "$cmd" in
        help)
                git bisect -h ;;
        start)
                bisect_start "$@" ;;
        bad|good|new|old|"$TERM_BAD"|"$TERM_GOOD")
                bisect_state "$cmd" "$@" ;;
        skip)
                bisect_skip "$@" ;;
        next)
                # Not sure we want "next" at the UI level anymore.
                bisect_next "$@" ;;

  ... snip ...

so, is it supported or not? should be consistent.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: should "git bisect" support "git bisect next?"

[Christian Couder]

On Sat, Nov 11, 2017 at 12:42 PM, Robert P. J. Day
<rpjday@crashcourse.ca> wrote:
>
>   the man page for "git bisect" makes no mention of "git bisect next",
> but the script git-bisect.sh does:

Yeah the following patch was related:

https://public-inbox.org/git/1460294354-7031-2-git-send-email-s-beyer@gmx.net/

You might want to discuss with Stephan (cc'ed).

Thanks,
Christian.

Re: should "git bisect" support "git bisect next?"

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> On Sat, Nov 11, 2017 at 12:42 PM, Robert P. J. Day
> <rpjday@crashcourse.ca> wrote:
>>
>>   the man page for "git bisect" makes no mention of "git bisect next",
>> but the script git-bisect.sh does:
>
> Yeah the following patch was related:
>
> https://public-inbox.org/git/1460294354-7031-2-git-send-email-s-beyer@gmx.net/
>
> You might want to discuss with Stephan (cc'ed).

Thanks for saving me time to explain why 'next' is still a very
important command but the end users do not actually need to be
strongly aware of it, because most commands automatically invokes it
as their final step due to the importance of what it does ;-)

Re: should "git bisect" support "git bisect next?"

[Theodore Ts'o]

On Sat, Nov 11, 2017 at 11:38:23PM +0900, Junio C Hamano wrote:
> 
> Thanks for saving me time to explain why 'next' is still a very
> important command but the end users do not actually need to be
> strongly aware of it, because most commands automatically invokes it
> as their final step due to the importance of what it does ;-)

This reminds me; is there a way to suppress it because I'm about to
give a large set of good and bit commits (perhaps because I'm
replaying part of a git biset log, minus one or two lines that are
suspected of being bogus thanks to flaky reproduction), and so there's
no point having git bisect figure the "next" commit to try until I'm
done giving it a list of good/bad commits?

	     	       	      	 	  - Ted

Re: should "git bisect" support "git bisect next?"

[Junio C Hamano]

Theodore Ts'o <tytso@mit.edu> writes:

> On Sat, Nov 11, 2017 at 11:38:23PM +0900, Junio C Hamano wrote:
>> 
>> Thanks for saving me time to explain why 'next' is still a very
>> important command but the end users do not actually need to be
>> strongly aware of it, because most commands automatically invokes it
>> as their final step due to the importance of what it does ;-)
>
> This reminds me; is there a way to suppress it because I'm about to
> give a large set of good and bit commits (perhaps because I'm
> replaying part of a git biset log, minus one or two lines that are
> suspected of being bogus thanks to flaky reproduction), and so there's
> no point having git bisect figure the "next" commit to try until I'm
> done giving it a list of good/bad commits?

It is surprising that I've never heard of this idea, but I think
that it is an excellent one.

When the user knows what bad and good commits in what sequence will
be fed to the command (i.e. replaying a saved output of "git bisect
log"), ideally we would want to "plug" the auto-next processing, and
just mark good and bad in refs/bisect/* without doing anything other
than creating these refs, and then run a "next" before giving the
control back to present the final working tree to be tested by the
user.  That would save the cost of intermediate checkouts but also
the cost of extra merge-base computation that is done to catch the
case where the user gave a good commit that is an ancestor of a bad
commit by mistake.

I think that the output of "git bisect log" was designed to be just
an executable shell script that the user can edit (the edit is
mostly designed to make it possible: "I know I screwed up in this
step, so I change its 'bad' to 'good' and remove the remaining
lines") and just execute it.  Which makes the simplest approach that
would first come to my mind not work very well, unfortunately.

	The "simplest approach" would teach the "--no-autonext"
	option to "git bisect good" and "git bisect bad" and skip
	the call to bisect_auto_next in bisect_state when it is
	given.  Then update the output from "git bisect log" to add
	that option to all good/bad commands, and then add an
	explicit "git bisect next" at the end.  This won't work well
	because it is likely that with the "remove the remaining
	lines" step, it is likely that the user would remove the
	final "bisect next".

A workable alternative approach is to teach "git bisect replay" to
be more intelligent.  Right now, I do not think it does anything
more than what happens by an execution of the input file with a
shell, but "replay" should be able to read a single step ahead in
the command sequence while doing its step-by-step execution, and
when it notices that it is about to run "bisect good" or "bisect
bad", and if the command to be run after that is also one of these
two, it can decide to skip the auto-next processing in the current
step.  It shouldn't need any new "--no-auto-next" option (I am not
saying that adding the option is bad--in fact, I think it is a good
addition for expert users; I am just saying that the approach to
make "replay" smarter does not require it).

Christian, what do you think?  Am I missing something?

Re: should "git bisect" support "git bisect next?"

[Christian Couder]

On Sun, Nov 12, 2017 at 2:43 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Theodore Ts'o <tytso@mit.edu> writes:
>
>> On Sat, Nov 11, 2017 at 11:38:23PM +0900, Junio C Hamano wrote:
>>>
>>> Thanks for saving me time to explain why 'next' is still a very
>>> important command but the end users do not actually need to be
>>> strongly aware of it, because most commands automatically invokes it
>>> as their final step due to the importance of what it does ;-)
>>
>> This reminds me; is there a way to suppress it because I'm about to
>> give a large set of good and bit commits (perhaps because I'm
>> replaying part of a git biset log, minus one or two lines that are
>> suspected of being bogus thanks to flaky reproduction), and so there's
>> no point having git bisect figure the "next" commit to try until I'm
>> done giving it a list of good/bad commits?
>
> It is surprising that I've never heard of this idea, but I think
> that it is an excellent one.
>
> When the user knows what bad and good commits in what sequence will
> be fed to the command (i.e. replaying a saved output of "git bisect
> log"), ideally we would want to "plug" the auto-next processing, and
> just mark good and bad in refs/bisect/* without doing anything other
> than creating these refs, and then run a "next" before giving the
> control back to present the final working tree to be tested by the
> user.  That would save the cost of intermediate checkouts but also
> the cost of extra merge-base computation that is done to catch the
> case where the user gave a good commit that is an ancestor of a bad
> commit by mistake.

Yeah I agree that it might be something interesting for the user to do.
But in this case the sequence in which you give the good and the bad
commits is not important.
Only the last bad commit and the set of good commits that were given
are important.

If you can get that and pass it to 'git bisect start' then you will
avoid all the intermediate computation and actually start from the
state you want.

> I think that the output of "git bisect log" was designed to be just
> an executable shell script that the user can edit (the edit is
> mostly designed to make it possible: "I know I screwed up in this
> step, so I change its 'bad' to 'good' and remove the remaining
> lines") and just execute it.  Which makes the simplest approach that
> would first come to my mind not work very well, unfortunately.
>
>         The "simplest approach" would teach the "--no-autonext"
>         option to "git bisect good" and "git bisect bad" and skip
>         the call to bisect_auto_next in bisect_state when it is
>         given.  Then update the output from "git bisect log" to add
>         that option to all good/bad commands, and then add an
>         explicit "git bisect next" at the end.  This won't work well
>         because it is likely that with the "remove the remaining
>         lines" step, it is likely that the user would remove the
>         final "bisect next".
>
> A workable alternative approach is to teach "git bisect replay" to
> be more intelligent.  Right now, I do not think it does anything
> more than what happens by an execution of the input file with a
> shell, but "replay" should be able to read a single step ahead in
> the command sequence while doing its step-by-step execution, and
> when it notices that it is about to run "bisect good" or "bisect
> bad", and if the command to be run after that is also one of these
> two, it can decide to skip the auto-next processing in the current
> step.  It shouldn't need any new "--no-auto-next" option (I am not
> saying that adding the option is bad--in fact, I think it is a good
> addition for expert users; I am just saying that the approach to
> make "replay" smarter does not require it).

To automate that one could start with a dirty oneliner like this:

git bisect start $(git bisect log | perl -ne '$b = $1 if m/# bad:
\[(.*)\]/; push @g, $1 if m/# good: \[(.*)\]/; END { print $b . " ".
join(" ", @g) . "\n"; }')

So yeah we could add an option to "git replay" that would do that.

Re: should "git bisect" support "git bisect next?"

[Theodore Ts'o]

On Sun, Nov 12, 2017 at 03:21:57PM +0100, Christian Couder wrote:
> 
> Yeah I agree that it might be something interesting for the user to do.
> But in this case the sequence in which you give the good and the bad
> commits is not important.
> Only the last bad commit and the set of good commits that were given
> are important.

Is it really true that of the bad commits, only the last one is significant?

Suppose we have a git tree that looks like this:

          *---*---*---*---*---*---M2---*---B1
          |                        |
  G1--*--D1---*---*---*---B2-\     |
          |                   \    /
          *---*---*---B3--*---M1--/

If we know that commits B2 and B3 are bad, if we assume that all
commits before the "bad" commit are good, all commits after the "bad"
commit are bad, can we not deduce that commit D1 should also be "bad"?

       	   	       	   	       	      - Ted

Re: should "git bisect" support "git bisect next?"

[Junio C Hamano]

Theodore Ts'o <tytso@mit.edu> writes:

> On Sun, Nov 12, 2017 at 03:21:57PM +0100, Christian Couder wrote:
>> 
>> Yeah I agree that it might be something interesting for the user to do.
>> But in this case the sequence in which you give the good and the bad
>> commits is not important.
>> Only the last bad commit and the set of good commits that were given
>> are important.
>
> Is it really true that of the bad commits, only the last one is significant?
>
> Suppose we have a git tree that looks like this:
>
>           *---*---*---*---*---*---M2---*---B1
>           |                        |
>   G1--*--D1---*---*---*---B2-\     |
>           |                   \    /
>           *---*---*---B3--*---M1--/
>
> If we know that commits B2 and B3 are bad, if we assume that all
> commits before the "bad" commit are good, all commits after the "bad"
> commit are bad, can we not deduce that commit D1 should also be "bad"?

You are correct.  Christian fell into an understandable and common
confusion.  It is true that we only maintain one significant bad
(i.e. the breakage that is known-ealiest so far), but that oldest
bad is the result of the bisection taking into account all the 'bad'
we have got in sequence so far, not necessarily the same as, and
hopefully way better than, the last bad the user gave from the
command line.  In your topology, "git bisect log" would contain "bad
B1", "bad B2", and "bad B3", and when the earlier session that
produced that log saw these three bad commits, it would have marked
D1 as the known-earliest bad one.

Taking the last-given bad B3 is suboptimal than that.

Re: should "git bisect" support "git bisect next?"

[Robert P. J. Day]

On Sat, 11 Nov 2017, Theodore Ts'o wrote:

> On Sat, Nov 11, 2017 at 11:38:23PM +0900, Junio C Hamano wrote:
> >
> > Thanks for saving me time to explain why 'next' is still a very
> > important command but the end users do not actually need to be
> > strongly aware of it, because most commands automatically invokes
> > it as their final step due to the importance of what it does ;-)
>
> This reminds me; is there a way to suppress it because I'm about to
> give a large set of good and bit commits (perhaps because I'm
                      ^^^^^^^^^^^^^^^^^^^^
> replaying part of a git biset log, minus one or two lines that are
> suspected of being bogus thanks to flaky reproduction), and so
> there's no point having git bisect figure the "next" commit to try
> until I'm done giving it a list of good/bad commits?

  i'm sure i'll regret asking this, but (assuming "bit" should read
"bad") is this suggesting one can hand bisect more than one bad
commit? i thought we just went through that discussion where there
could be only one bad commit but multiple good commits. clarification?

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: should "git bisect" support "git bisect next?"

[Junio C Hamano]

"Robert P. J. Day" <rpjday@crashcourse.ca> writes:

>> This reminds me; is there a way to suppress it because I'm about to
>> give a large set of good and bit commits (perhaps because I'm
>                       ^^^^^^^^^^^^^^^^^^^^
>> replaying part of a git biset log, minus one or two lines that are
>> suspected of being bogus thanks to flaky reproduction), and so
>> there's no point having git bisect figure the "next" commit to try
>> until I'm done giving it a list of good/bad commits?
>
>   i'm sure i'll regret asking this, but (assuming "bit" should read
> "bad") is this suggesting one can hand bisect more than one bad
> commit? i thought we just went through that discussion where there
> could be only one bad commit but multiple good commits. clarification?

The documentation you have been futzing with is about the fact that
the initial set of known to be good/bad commits that "git bisect
start <bad> <good>..." take can have one bad and zero or more good.

What is being discussed in this thread is different (and I tried to
clarify the fact by saying "what bad and good commits in what
sequence").  

Ted is talking about replaying the series of "git bisect (good|bad)
$a_single_commit" that are recorded during a bisection session and
can be read via "git bisect log".  When Ted says "good commits"
and/or "bad commits", he is not talking about giving all of them to
a single invocation of "git bisect" command.  The replay session
will take one commit at a time (which is what "git bisect replay"
does) and feed it to either "git bisect good" or "git bisect bad".

Does it make sense?

By the way, I do not think there is anything to regret in asking
what you do not understand.  Showing how much you know (and you
don't) will allow others who communicate with you to calibrate their
expectations, which eases later discussions; it is a good thing.

Re: should "git bisect" support "git bisect next?"

[Stephan Beyer]

Hi,

On 11/11/2017 03:38 PM, Junio C Hamano wrote:
> Christian Couder <christian.couder@gmail.com> writes:
> 
>> On Sat, Nov 11, 2017 at 12:42 PM, Robert P. J. Day
>> <rpjday@crashcourse.ca> wrote:
>>>
>>>   the man page for "git bisect" makes no mention of "git bisect next",
>>> but the script git-bisect.sh does:
>>
>> Yeah the following patch was related:
>>
>> https://public-inbox.org/git/1460294354-7031-2-git-send-email-s-beyer@gmx.net/
>>
>> You might want to discuss with Stephan (cc'ed).
> 
> Thanks for saving me time to explain why 'next' is still a very
> important command but the end users do not actually need to be
> strongly aware of it, because most commands automatically invokes it
> as their final step due to the importance of what it does ;-)

I will nonetheless re-roll the patch (that Christian linked to)
after Pranit's bisect part II series is in good shape. I think the
documentation change in the patch shows why the user should be aware of
it (although not strongly).

Best
Stephan