git bisect should return 1 when the first bad commit is found

git bisect should return 1 when the first bad commit is found

[Pedro Larroy]

Hi

I'm using git bisect, and for complex reasons I can't use git bisect
run to drive the bisection. I don't think git bisect bad/good is
returning a different status code when the first bad commit is found
and this would be very useful to stop the driver of the bisection.

What do you think?  Would such a change be acceptable?

Thank you!

Pedro.

Re: git bisect should return 1 when the first bad commit is found

[Junio C Hamano]

Pedro Larroy <pedro.larroy.lists@gmail.com> writes:

> I'm using git bisect, and for complex reasons I can't use git bisect
> run to drive the bisection. I don't think git bisect bad/good is
> returning a different status code when the first bad commit is found
> and this would be very useful to stop the driver of the bisection.
>
> What do you think?  Would such a change be acceptable?

Probably not.

A non-zero exit code would indicate that "git bisect" had trouble
finding the next commit to check after being told 'good' (or 'bad'),
which you wouldn't be able to tell from your "first bad one is
found".

The 'first bad one' is not necessarily the HEAD when the bisect
procedure finds it, so you'd have to read and parse "x is the first
such commit" message anyway, no?  And once you start parsing the
message from 'git bisect', you should be able to tell between
"please try this one, you have approximately 7 steps left" and "we
found the first bad one, which is X", without relying on the exit
status, no?

Re: git bisect should return 1 when the first bad commit is found

[Pedro Larroy]

Thanks for your answer.

I was expecting the HEAD to point to the first bad commit.

In mercurial, the exit status tells you information about the
bisection process:  https://www.mercurial-scm.org/repo/hg/help/bisect

Sure one can parse stdout, it's just more tedious than just checking
the return code and having the HEAD left to the original bad commit.

Pedro.

On Thu, Jun 13, 2019 at 11:22 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Pedro Larroy <pedro.larroy.lists@gmail.com> writes:
>
> > I'm using git bisect, and for complex reasons I can't use git bisect
> > run to drive the bisection. I don't think git bisect bad/good is
> > returning a different status code when the first bad commit is found
> > and this would be very useful to stop the driver of the bisection.
> >
> > What do you think?  Would such a change be acceptable?
>
> Probably not.
>
> A non-zero exit code would indicate that "git bisect" had trouble
> finding the next commit to check after being told 'good' (or 'bad'),
> which you wouldn't be able to tell from your "first bad one is
> found".
>
> The 'first bad one' is not necessarily the HEAD when the bisect
> procedure finds it, so you'd have to read and parse "x is the first
> such commit" message anyway, no?  And once you start parsing the
> message from 'git bisect', you should be able to tell between
> "please try this one, you have approximately 7 steps left" and "we
> found the first bad one, which is X", without relying on the exit
> status, no?
>

Re: git bisect should return 1 when the first bad commit is found

[Jeff King]

On Sun, Jun 23, 2019 at 01:32:16PM -0700, Pedro Larroy wrote:

> Thanks for your answer.
> 
> I was expecting the HEAD to point to the first bad commit.
> 
> In mercurial, the exit status tells you information about the
> bisection process:  https://www.mercurial-scm.org/repo/hg/help/bisect
> 
> Sure one can parse stdout, it's just more tedious than just checking
> the return code and having the HEAD left to the original bad commit.

I think it might be nice for Git to write a well-known refname (like
BISECT_RESULT or similar) so that you can refer to that instead of
having to read stdout (whether by machine or by a user
cutting-and-pasting). And I cannot offhand think of a particular reason
why that could not just be HEAD (instead of something bisect-specific)
after the bisect finishes.

We do not promise any particular value in HEAD now. The only downside
would be the minor cost to checkout the working tree of the known-bad
commit if we are not already there.

-Peff

Re: git bisect should return 1 when the first bad commit is found

[Christian Couder]

On Mon, Jun 24, 2019 at 4:51 AM Jeff King <peff@peff.net> wrote:
>
> On Sun, Jun 23, 2019 at 01:32:16PM -0700, Pedro Larroy wrote:
>
> > Thanks for your answer.
> >
> > I was expecting the HEAD to point to the first bad commit.
> >
> > In mercurial, the exit status tells you information about the
> > bisection process:  https://www.mercurial-scm.org/repo/hg/help/bisect

It's not clear from he above URL how that differs from what git bisect
does. I only found "Returns 0 on success" there which is not very
explicit, and we could argue that it's also what git bisect does.

> > Sure one can parse stdout, it's just more tedious than just checking
> > the return code and having the HEAD left to the original bad commit.

The git bisect documentation says:

       Eventually there will be no more revisions left to inspect, and
the command will print out a description of the first bad commit. The
reference
       refs/bisect/bad will be left pointing at that commit.

So you just need to parse stdout to detect that it found the first bad
commit, and then you can use refs/bisect/bad.

If the return code was used, how would you distinguish between a
failure in the command (for example if you give bad information to
`git bisect good` or `git bisect bad`) and the fact that it has not
yet found the first bad commit? Anyway you would need to add some
logic for that.

> I think it might be nice for Git to write a well-known refname (like
> BISECT_RESULT or similar) so that you can refer to that instead of
> having to read stdout (whether by machine or by a user
> cutting-and-pasting). And I cannot offhand think of a particular reason
> why that could not just be HEAD (instead of something bisect-specific)
> after the bisect finishes.

If it would be HEAD, it would mean that git bisect would potentially
have to do one more checkout so that HEAD points to the first bad
commit. This checkout would sometimes be useless, so it's more
efficient to use something like refs/bisect/bad rather than HEAD.

> We do not promise any particular value in HEAD now. The only downside
> would be the minor cost to checkout the working tree of the known-bad
> commit if we are not already there.

Though we might not explicitly promise in the doc that HEAD will stay
at the last commit that was tested, I think that's something people
can expect from the way we describe how bisect work. So I don't think
it would be a good idea to change our behavior.

Re: git bisect should return 1 when the first bad commit is found

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Sun, Jun 23, 2019 at 01:32:16PM -0700, Pedro Larroy wrote:
>
>> Thanks for your answer.
>> 
>> I was expecting the HEAD to point to the first bad commit.
>> 
>> In mercurial, the exit status tells you information about the
>> bisection process:  https://www.mercurial-scm.org/repo/hg/help/bisect
>> 
>> Sure one can parse stdout, it's just more tedious than just checking
>> the return code and having the HEAD left to the original bad commit.
>
> I think it might be nice for Git to write a well-known refname (like
> BISECT_RESULT or similar) so that you can refer to that instead of
> having to read stdout (whether by machine or by a user
> cutting-and-pasting). And I cannot offhand think of a particular reason
> why that could not just be HEAD (instead of something bisect-specific)
> after the bisect finishes.

As Christian downthread reminds us, that is what the bisect/bad ref
is (which I totally forgot when I gave the earlier response).  I do
not think we need a new ref, but I do not think it is so bad to add
an option "git bisect --exit-code ( --good | --bad ) [<commit-ish>]"
that makes the command usually exit with non-zero status.  Unless we
have found the final answer successfully, that is, and in that case
the command would exit with 0 status to signall "all done".

But that should be an option.

Re: git bisect should return 1 when the first bad commit is found

[Jeff King]

On Mon, Jun 24, 2019 at 11:41:53AM -0700, Junio C Hamano wrote:

> > I think it might be nice for Git to write a well-known refname (like
> > BISECT_RESULT or similar) so that you can refer to that instead of
> > having to read stdout (whether by machine or by a user
> > cutting-and-pasting). And I cannot offhand think of a particular reason
> > why that could not just be HEAD (instead of something bisect-specific)
> > after the bisect finishes.
> 
> As Christian downthread reminds us, that is what the bisect/bad ref
> is (which I totally forgot when I gave the earlier response).  I do
> not think we need a new ref, but I do not think it is so bad to add
> an option "git bisect --exit-code ( --good | --bad ) [<commit-ish>]"
> that makes the command usually exit with non-zero status.  Unless we
> have found the final answer successfully, that is, and in that case
> the command would exit with 0 status to signall "all done".
> 
> But that should be an option.

Ah, right, I forgot that we already have such a ref. So that does seem
like a reasonable way to access it (and as Christian notes, it's
documented as well).  And I agree with you that any exit-code magic
should require the user to ask for it explicitly.

I do think that accessing the bisect/bad ref is a little intimate with
the implementation (i.e., it implies knowing that there is only a single
"bad" that we are moving around, unlike "good", where we may mark many
such tips).

So another option is to put the result in a new ref (bisect/result,
perhaps), but with one twist: delete it at the beginning of a bisection,
so its presence can be used as a marker that the bisection is complete.
I.e., you could do "git rev-parse --verify refs/bisect/result" as an
alternative to "--exit-code". That gives you the added flexibility of
asking "is the bisection done" separately from "do this next bisection
step".

-Peff

Re: git bisect should return 1 when the first bad commit is found

[Jeff King]

On Mon, Jun 24, 2019 at 09:16:13AM +0200, Christian Couder wrote:

> So you just need to parse stdout to detect that it found the first bad
> commit, and then you can use refs/bisect/bad.
>
> If the return code was used, how would you distinguish between a
> failure in the command (for example if you give bad information to
> `git bisect good` or `git bisect bad`) and the fact that it has not
> yet found the first bad commit? Anyway you would need to add some
> logic for that.

Right, thanks for reminding us of that ref. I did leave a suggestion
elsewhere in the thread that we might be able to avoid the exit-code
pitfalls by using a specific "result" ref.

> If it would be HEAD, it would mean that git bisect would potentially
> have to do one more checkout so that HEAD points to the first bad
> commit. This checkout would sometimes be useless, so it's more
> efficient to use something like refs/bisect/bad rather than HEAD.

I'm not too concerned with the cost of checking out a tree. After all,
we've just done a bunch of checkouts and probably some operations on the
tree itself. But I think there's a much more important reason my
suggestion may have problems.

Sometimes we _can't_ checkout another commit, because part of testing
the bisection may involve mucking with the working tree. For instance,
earlier today I was bisecting a case where git.git did not show the
breakage itself, but applying a debugging patch on top did. So I was
bisecting along the real history, adding new contents, testing, and then
cleaning up my changes before jumping to the next bisection candidate.

On the other hand, I have to clean them up anyway before running "git
bisect good" or it cannot jump to the next candidate when we are not
finished. Jumping to "bad" when we _are_ finished is no different there.
But I think it does show that there can be surprising implications to
moving HEAD around.

-Peff

Re: git bisect should return 1 when the first bad commit is found

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I do think that accessing the bisect/bad ref is a little intimate with
> the implementation (i.e., it implies knowing that there is only a single
> "bad" that we are moving around, unlike "good", where we may mark many
> such tips).

I actually do not think it is merely an implementation detail, but a
rather fundamental part of the "bisect" design.  After all, we are
trying to find *the* single transition point from "the old world" to
"the new world", and the primary assumption that makes "bisection"
work in the first place is that there is only one such transition
point from old to new.  Otherwise, there may be a stretch of history
where one transition from "old" to "new" is followed by another
transition from "new" back to "old" and then again another one from
"old" to "new"; in such a "stripe snake" part of the history, "find
the midpoint, if it is from the new world, then narrow to the world
before that midpoint", which is the central idea of the "bisection"
would not work, would it?

So, ... under that "single transition point" assumption, how would a
"bisect" that uses more than one "bad" would work?  "bad" is the one
that is known to belong to the "new" world that is topologically the
oldest and it moves backward in topology to find the boundary from
the "old" world.  If there were another "bad" (i.e. one that is
known to belong the the "new" world, that may not be the oldest),
shouldn't that have the other one as its ancestor, making it
redundant (i.e. we know we have inherited the badness from the other
one)?

Having said that ...

> So another option is to put the result in a new ref (bisect/result,
> perhaps), but with one twist: delete it at the beginning of a bisection,
> so its presence can be used as a marker that the bisection is complete.

... I kind of like this one exactly for the reasons you give below.

> I.e., you could do "git rev-parse --verify refs/bisect/result" as an
> alternative to "--exit-code". That gives you the added flexibility of
> asking "is the bisection done" separately from "do this next bisection
> step".
>
> -Peff