When exactly should REBASE_HEAD exist?

When exactly should REBASE_HEAD exist?

[Stefan Haller]

As far as I can tell, REBASE_HEAD always points to the last commit that
was attempted to be applied in a rebase, not matter whether that attempt
was successful or not. In other words, when you break in a rebase with
"edit" or "break", REBASE_HEAD is the same as HEAD (except when you
break before the first commit, in that case REBASE_HEAD is unset).

I wonder whether it would make sense to set REBASE_HEAD only when
applying a patch failed. That seems to be the main use case of it.

The reason why I am asking this is: I'm using lazygit, which, during
interactive rebases, shows a combined view of the real commits that were
already applied, and the remaining commits that are yet to be applied
(it gets these by parsing rebase-merge/git-rebase-todo); something like
this, when I set the 2nd commit to "edit":

  pick   4th commit
  pick   3rd commit
         2nd commit  <-- YOU ARE HERE
         1st commit

This is great, but assuming that the 2nd commit conflicted, currently
the display looks like this:

  pick   4th commit
  pick   3rd commit
         1st commit  <-- YOU ARE HERE

I would like to extend this to also show a "fake entry" for the commit
that conflicted, if there is one. REBASE_HEAD is perfect for this,
except that I need a way to distinguish whether it was applied already
or not.

I currently do that by comparing it against HEAD, and only showing it
when they are different. That doesn't always work though, e.g. when I
amend the current "edit" commit.

Any better suggestions how to work around this?

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Phillip Wood]

Hi Stefan

On 28/02/2023 12:55, Stefan Haller wrote:
> As far as I can tell, REBASE_HEAD always points to the last commit that
> was attempted to be applied in a rebase, not matter whether that attempt
> was successful or not. In other words, when you break in a rebase with
> "edit" or "break", REBASE_HEAD is the same as HEAD 

As you said at the beginning REBASE_HEAD points to the commit that we're 
trying to pick, so it will only be the same as HEAD if the pick was 
successful and it was not rebased onto a new parent. I think it is 
useful to have REBASE_HEAD set when editing a commit as I find it is 
sometimes useful to look at the original commit. I don't think we do (or 
should) set it when rebase stops for a "break" command or a failed 
"exec" command.

>(except when you
> break before the first commit, in that case REBASE_HEAD is unset).
> 
> I wonder whether it would make sense to set REBASE_HEAD only when
> applying a patch failed. That seems to be the main use case of it.

As I mentioned above, I think having it available when editing a commit 
us useful.

> The reason why I am asking this is: I'm using lazygit, which, during
> interactive rebases, shows a combined view of the real commits that were
> already applied, and the remaining commits that are yet to be applied
> (it gets these by parsing rebase-merge/git-rebase-todo); something like
> this, when I set the 2nd commit to "edit":
> 
>    pick   4th commit
>    pick   3rd commit
>           2nd commit  <-- YOU ARE HERE
>           1st commit
> 
> This is great, but assuming that the 2nd commit conflicted, currently
> the display looks like this:
> 
>    pick   4th commit
>    pick   3rd commit
>           1st commit  <-- YOU ARE HERE
> 
> I would like to extend this to also show a "fake entry" for the commit
> that conflicted, if there is one. REBASE_HEAD is perfect for this,
> except that I need a way to distinguish whether it was applied already
> or not.

Can you check the index for conflicts when the rebase stops?

Best Wishes

Phillip

> I currently do that by comparing it against HEAD, and only showing it
> when they are different. That doesn't always work though, e.g. when I
> amend the current "edit" commit.
> 
> Any better suggestions how to work around this?
> 
> -Stefan
> 

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 02.03.23 11:19, Phillip Wood wrote:
> On 28/02/2023 12:55, Stefan Haller wrote:
>> As far as I can tell, REBASE_HEAD always points to the last commit that
>> was attempted to be applied in a rebase, not matter whether that attempt
>> was successful or not. In other words, when you break in a rebase with
>> "edit" or "break", REBASE_HEAD is the same as HEAD 
> 
> As you said at the beginning REBASE_HEAD points to the commit that we're
> trying to pick, so it will only be the same as HEAD if the pick was
> successful and it was not rebased onto a new parent.

Right, I was a bit sloppy in describing the current behavior.

> I think it is
> useful to have REBASE_HEAD set when editing a commit as I find it is
> sometimes useful to look at the original commit.

OK.

>> The reason why I am asking this is: I'm using lazygit, which, during
>> interactive rebases, shows a combined view of the real commits that were
>> already applied, and the remaining commits that are yet to be applied
>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>> this, when I set the 2nd commit to "edit":
>>
>>    pick   4th commit
>>    pick   3rd commit
>>           2nd commit  <-- YOU ARE HERE
>>           1st commit
>>
>> This is great, but assuming that the 2nd commit conflicted, currently
>> the display looks like this:
>>
>>    pick   4th commit
>>    pick   3rd commit
>>           1st commit  <-- YOU ARE HERE
>>
>> I would like to extend this to also show a "fake entry" for the commit
>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>> except that I need a way to distinguish whether it was applied already
>> or not.
> 
> Can you check the index for conflicts when the rebase stops?

I could do that, but then the fake entry would go away as soon as I have
staged all conflict resolutions. I would find it useful for it to stay
visible in that case, until I continue the rebase.

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 02.03.23 21:27, Stefan Haller wrote:
> On 02.03.23 11:19, Phillip Wood wrote:
>> On 28/02/2023 12:55, Stefan Haller wrote:
>>> The reason why I am asking this is: I'm using lazygit, which, during
>>> interactive rebases, shows a combined view of the real commits that were
>>> already applied, and the remaining commits that are yet to be applied
>>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>>> this, when I set the 2nd commit to "edit":
>>>
>>>    pick   4th commit
>>>    pick   3rd commit
>>>           2nd commit  <-- YOU ARE HERE
>>>           1st commit
>>>
>>> This is great, but assuming that the 2nd commit conflicted, currently
>>> the display looks like this:
>>>
>>>    pick   4th commit
>>>    pick   3rd commit
>>>           1st commit  <-- YOU ARE HERE
>>>
>>> I would like to extend this to also show a "fake entry" for the commit
>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>> except that I need a way to distinguish whether it was applied already
>>> or not.
>>
>> Can you check the index for conflicts when the rebase stops?
> 
> I could do that, but then the fake entry would go away as soon as I have
> staged all conflict resolutions. I would find it useful for it to stay
> visible in that case, until I continue the rebase.

It seems that I can get close by checking whether the file
.git/rebase-merge/amend exists. If it does, the current patch was
applied already, and I don't show the fake entry. If it doesn't, we
still need to commit the changes from REBASE-HEAD, so it makes sense to
show it in the list.

Does this sound like a reasonable approach? I must admit that the code
in sequencer.c is too complex for me to tell at a glance whether there
are situations where this heuristic would do the wrong thing.

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Chris Torek]

On Fri, Mar 3, 2023 at 3:04 AM Stefan Haller <lists@haller-berlin.de> wrote:
> It seems that I can get close by checking whether the file
> .git/rebase-merge/amend exists.

The location and existence of this file depends on Git version and
whether you're using added working trees, so that's not generally
the right directionA to go.  It would be best if `git status`
reported all of this information directly: perhaps --porcelain=v2
can be augmented to provide this, or there might even be a
--porcelain=v3 if necessary.

Of course, you'll still have to do some kind of guesswork for
versions of Git that don't produce the right status.

In any case someone would need to add the desired information
to `git status`.

Chris

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 03.03.23 22:25, Chris Torek wrote:
> On Fri, Mar 3, 2023 at 3:04 AM Stefan Haller <lists@haller-berlin.de> wrote:
>> It seems that I can get close by checking whether the file
>> .git/rebase-merge/amend exists.
> 
> The location and existence of this file depends on Git version and
> whether you're using added working trees, so that's not generally
> the right directionA to go.

From what I can see, the current behavior of the .git/rebase-merge/amend
file was done in 2.12, that would be more than good enough for me. Did I
miss any changes to the behavior since then?

I'm surprised to hear that worktrees play a role in this. Do you have
more details about this?

> It would be best if `git status`
> reported all of this information directly: perhaps --porcelain=v2
> can be augmented to provide this, or there might even be a
> --porcelain=v3 if necessary.

Yes, that would be nice, but based on what state would it provide this
information? Are you suggesting that some new file should be added to
.git/rebase-merge/ to keep track of it (more reliably than the amend
file does today), or could the information be derived somehow from what
exists already? And if it's the latter, then why shouldn't I be able to
do that myself in the same way for the time being?

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Phillip Wood]

Hi Stefan

On 02/03/2023 20:27, Stefan Haller wrote:
> On 02.03.23 11:19, Phillip Wood wrote:
>> On 28/02/2023 12:55, Stefan Haller wrote:
>>> The reason why I am asking this is: I'm using lazygit, which, during
>>> interactive rebases, shows a combined view of the real commits that were
>>> already applied, and the remaining commits that are yet to be applied
>>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>>> this, when I set the 2nd commit to "edit":
>>>
>>>     pick   4th commit
>>>     pick   3rd commit
>>>            2nd commit  <-- YOU ARE HERE
>>>            1st commit
>>>
>>> This is great, but assuming that the 2nd commit conflicted, currently
>>> the display looks like this:
>>>
>>>     pick   4th commit
>>>     pick   3rd commit
>>>            1st commit  <-- YOU ARE HERE
>>>
>>> I would like to extend this to also show a "fake entry" for the commit
>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>> except that I need a way to distinguish whether it was applied already
>>> or not.
>>
>> Can you check the index for conflicts when the rebase stops?
> 
> I could do that, but then the fake entry would go away as soon as I have
> staged all conflict resolutions. I would find it useful for it to stay
> visible in that case, until I continue the rebase.

I've not used lazygit but looking at the github page it seems that it is 
a persistent process that runs "git rebase". If that's the case I would 
think that you can check for conflicts when the rebase stops and keep 
that value in memory until the rebase is started again.

Another thing to bear in mind is that if the commit being picked has 
already been applied upstream then the rebase will stop without 
conflicts but you'd want to add the commit being picked to the list in 
lazygit.

I think your best bet might be to read "$(git rev-parse --git-path 
rebase-merge/done)" the last line of which contains the last todo 
command the rebase tried to execute. Not that failed command are 
sometimes rescheduled (for example when a pick would overwrite an 
untracked file) so you'd need to check the first line of "$(git 
rev-parse --git-path rebase-merge/git-rebase-todo)" as well.

Best Wishes

Phillip

> -Stefan

Re: When exactly should REBASE_HEAD exist?

[Phillip Wood]

Hi Stefan

On 03/03/2023 10:57, Stefan Haller wrote:
> On 02.03.23 21:27, Stefan Haller wrote:
>> On 02.03.23 11:19, Phillip Wood wrote:
>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>> The reason why I am asking this is: I'm using lazygit, which, during
>>>> interactive rebases, shows a combined view of the real commits that were
>>>> already applied, and the remaining commits that are yet to be applied
>>>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>>>> this, when I set the 2nd commit to "edit":
>>>>
>>>>     pick   4th commit
>>>>     pick   3rd commit
>>>>            2nd commit  <-- YOU ARE HERE
>>>>            1st commit
>>>>
>>>> This is great, but assuming that the 2nd commit conflicted, currently
>>>> the display looks like this:
>>>>
>>>>     pick   4th commit
>>>>     pick   3rd commit
>>>>            1st commit  <-- YOU ARE HERE
>>>>
>>>> I would like to extend this to also show a "fake entry" for the commit
>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>> except that I need a way to distinguish whether it was applied already
>>>> or not.
>>>
>>> Can you check the index for conflicts when the rebase stops?
>>
>> I could do that, but then the fake entry would go away as soon as I have
>> staged all conflict resolutions. I would find it useful for it to stay
>> visible in that case, until I continue the rebase.
> 
> It seems that I can get close by checking whether the file
> .git/rebase-merge/amend exists. If it does, the current patch was
> applied already, and I don't show the fake entry. If it doesn't, we
> still need to commit the changes from REBASE-HEAD, so it makes sense to
> show it in the list.
> 
> Does this sound like a reasonable approach? I must admit that the code
> in sequencer.c is too complex for me to tell at a glance whether there
> are situations where this heuristic would do the wrong thing.

When a fixup or squash stops for a confict resolution that file also 
exists as the result needs to be squashed into HEAD.

Best Wishes

Phillip

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 05.03.23 15:33, Phillip Wood wrote:
> Hi Stefan
> 
> On 03/03/2023 10:57, Stefan Haller wrote:
>> On 02.03.23 21:27, Stefan Haller wrote:
>>> On 02.03.23 11:19, Phillip Wood wrote:
>>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>>> The reason why I am asking this is: I'm using lazygit, which, during
>>>>> interactive rebases, shows a combined view of the real commits that
>>>>> were
>>>>> already applied, and the remaining commits that are yet to be applied
>>>>> (it gets these by parsing rebase-merge/git-rebase-todo); something
>>>>> like
>>>>> this, when I set the 2nd commit to "edit":
>>>>>
>>>>>     pick   4th commit
>>>>>     pick   3rd commit
>>>>>            2nd commit  <-- YOU ARE HERE
>>>>>            1st commit
>>>>>
>>>>> This is great, but assuming that the 2nd commit conflicted, currently
>>>>> the display looks like this:
>>>>>
>>>>>     pick   4th commit
>>>>>     pick   3rd commit
>>>>>            1st commit  <-- YOU ARE HERE
>>>>>
>>>>> I would like to extend this to also show a "fake entry" for the commit
>>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>>> except that I need a way to distinguish whether it was applied already
>>>>> or not.
>>>>
>>>> Can you check the index for conflicts when the rebase stops?
>>>
>>> I could do that, but then the fake entry would go away as soon as I have
>>> staged all conflict resolutions. I would find it useful for it to stay
>>> visible in that case, until I continue the rebase.
>>
>> It seems that I can get close by checking whether the file
>> .git/rebase-merge/amend exists. If it does, the current patch was
>> applied already, and I don't show the fake entry. If it doesn't, we
>> still need to commit the changes from REBASE-HEAD, so it makes sense to
>> show it in the list.
> 
> When a fixup or squash stops for a confict resolution that file also
> exists as the result needs to be squashed into HEAD.

Ah damn, yes. And that's even a pretty common situation, so I can't just
ignore it as a rare edge case.

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 05.03.23 15:31, Phillip Wood wrote:
> Hi Stefan
> 
> On 02/03/2023 20:27, Stefan Haller wrote:
>> On 02.03.23 11:19, Phillip Wood wrote:
>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>> The reason why I am asking this is: I'm using lazygit, which, during
>>>> interactive rebases, shows a combined view of the real commits that
>>>> were
>>>> already applied, and the remaining commits that are yet to be applied
>>>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>>>> this, when I set the 2nd commit to "edit":
>>>>
>>>>     pick   4th commit
>>>>     pick   3rd commit
>>>>            2nd commit  <-- YOU ARE HERE
>>>>            1st commit
>>>>
>>>> This is great, but assuming that the 2nd commit conflicted, currently
>>>> the display looks like this:
>>>>
>>>>     pick   4th commit
>>>>     pick   3rd commit
>>>>            1st commit  <-- YOU ARE HERE
>>>>
>>>> I would like to extend this to also show a "fake entry" for the commit
>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>> except that I need a way to distinguish whether it was applied already
>>>> or not.
>>>
>>> Can you check the index for conflicts when the rebase stops?
>>
>> I could do that, but then the fake entry would go away as soon as I have
>> staged all conflict resolutions. I would find it useful for it to stay
>> visible in that case, until I continue the rebase.
> 
> I've not used lazygit but looking at the github page it seems that it is
> a persistent process that runs "git rebase". If that's the case I would
> think that you can check for conflicts when the rebase stops and keep
> that value in memory until the rebase is started again.

I had considered that, but it would be preferable if it were possible to
quit lazygit, start it again, and have it show the same state again. Or
even start the rebase outside of lazygit while it isn't running at all,
and then start it and have it display the correct state.

> I think your best bet might be to read "$(git rev-parse --git-path
> rebase-merge/done)" the last line of which contains the last todo
> command the rebase tried to execute.

I'm not sure I understand; you mean in order to distinguish whether it
was a pick or a fixup?

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 05.03.23 17:59, Stefan Haller wrote:
> On 05.03.23 15:31, Phillip Wood wrote:
>> Hi Stefan
>>
>> On 02/03/2023 20:27, Stefan Haller wrote:
>>> On 02.03.23 11:19, Phillip Wood wrote:
>>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>>> The reason why I am asking this is: I'm using lazygit, which, during
>>>>> interactive rebases, shows a combined view of the real commits that
>>>>> were
>>>>> already applied, and the remaining commits that are yet to be applied
>>>>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>>>>> this, when I set the 2nd commit to "edit":
>>>>>
>>>>>     pick   4th commit
>>>>>     pick   3rd commit
>>>>>            2nd commit  <-- YOU ARE HERE
>>>>>            1st commit
>>>>>
>>>>> This is great, but assuming that the 2nd commit conflicted, currently
>>>>> the display looks like this:
>>>>>
>>>>>     pick   4th commit
>>>>>     pick   3rd commit
>>>>>            1st commit  <-- YOU ARE HERE
>>>>>
>>>>> I would like to extend this to also show a "fake entry" for the commit
>>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>>> except that I need a way to distinguish whether it was applied already
>>>>> or not.
>>>>
>>>> Can you check the index for conflicts when the rebase stops?
>>>
>>> I could do that, but then the fake entry would go away as soon as I have
>>> staged all conflict resolutions. I would find it useful for it to stay
>>> visible in that case, until I continue the rebase.
>>
>> I've not used lazygit but looking at the github page it seems that it is
>> a persistent process that runs "git rebase". If that's the case I would
>> think that you can check for conflicts when the rebase stops and keep
>> that value in memory until the rebase is started again.
> 
> I had considered that, but it would be preferable if it were possible to
> quit lazygit, start it again, and have it show the same state again. Or
> even start the rebase outside of lazygit while it isn't running at all,
> and then start it and have it display the correct state.
> 
>> I think your best bet might be to read "$(git rev-parse --git-path
>> rebase-merge/done)" the last line of which contains the last todo
>> command the rebase tried to execute.
> 
> I'm not sure I understand; you mean in order to distinguish whether it
> was a pick or a fixup?

OK, I guess it's something like

show_fake_entry :=
  REBASE_HEAD exists
  && (last command in "done" file was not "edit"
      || "amend" file exists)

Is that what you meant? (Minus the bit about rescheduling failed
commands, which I still need to wrap my head around...)

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Phillip Wood]

Hi Stefan

On 05/03/2023 19:13, Stefan Haller wrote:
> On 05.03.23 17:59, Stefan Haller wrote:
>> On 05.03.23 15:31, Phillip Wood wrote:
>>> Hi Stefan
>>>
>>> On 02/03/2023 20:27, Stefan Haller wrote:
>>>> On 02.03.23 11:19, Phillip Wood wrote:
>>>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>>>> The reason why I am asking this is: I'm using lazygit, which, during
>>>>>> interactive rebases, shows a combined view of the real commits that
>>>>>> were
>>>>>> already applied, and the remaining commits that are yet to be applied
>>>>>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>>>>>> this, when I set the 2nd commit to "edit":
>>>>>>
>>>>>>      pick   4th commit
>>>>>>      pick   3rd commit
>>>>>>             2nd commit  <-- YOU ARE HERE
>>>>>>             1st commit
>>>>>>
>>>>>> This is great, but assuming that the 2nd commit conflicted, currently
>>>>>> the display looks like this:
>>>>>>
>>>>>>      pick   4th commit
>>>>>>      pick   3rd commit
>>>>>>             1st commit  <-- YOU ARE HERE
>>>>>>
>>>>>> I would like to extend this to also show a "fake entry" for the commit
>>>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>>>> except that I need a way to distinguish whether it was applied already
>>>>>> or not.
>>>>>
>>>>> Can you check the index for conflicts when the rebase stops?
>>>>
>>>> I could do that, but then the fake entry would go away as soon as I have
>>>> staged all conflict resolutions. I would find it useful for it to stay
>>>> visible in that case, until I continue the rebase.
>>>
>>> I've not used lazygit but looking at the github page it seems that it is
>>> a persistent process that runs "git rebase". If that's the case I would
>>> think that you can check for conflicts when the rebase stops and keep
>>> that value in memory until the rebase is started again.
>>
>> I had considered that, but it would be preferable if it were possible to
>> quit lazygit, start it again, and have it show the same state again. Or
>> even start the rebase outside of lazygit while it isn't running at all,
>> and then start it and have it display the correct state.
>>
>>> I think your best bet might be to read "$(git rev-parse --git-path
>>> rebase-merge/done)" the last line of which contains the last todo
>>> command the rebase tried to execute.
>>
>> I'm not sure I understand; you mean in order to distinguish whether it
>> was a pick or a fixup?
> 
> OK, I guess it's something like
> 
> show_fake_entry :=
>    REBASE_HEAD exists
>    && (last command in "done" file was not "edit"
>        || "amend" file exists)
> 
> Is that what you meant? (Minus the bit about rescheduling failed
> commands, which I still need to wrap my head around...)

I meant you could just use the done file to get the last command, there 
is no need to look at REBASE_HEAD.

Best Wishes

Phillip

> -Stefan

Re: When exactly should REBASE_HEAD exist?

[Chris Torek]

On Sat, Mar 4, 2023 at 12:36 AM Stefan Haller <lists@haller-berlin.de> wrote:
> From what I can see, the current behavior of the .git/rebase-merge/amend
> file was done in 2.12, that would be more than good enough for me. Did I
> miss any changes to the behavior since then?

Probably not - but `git status` remains the right place to decode the
information, since it will be updated appropriately whenever this changes.

> I'm surprised to hear that worktrees play a role in this. Do you have
> more details about this?

$ git worktree add foo
$ cd ../foo
$ git rebase -i HEAD~2

puts the rebase-todo file into
`.git/worktrees/foo/rebase-merge/git-rebase-todo`;
all the status files will likewise be in the `.git/worktrees/foo/`
directory.

>> It would be best if `git status`
>> reported all of this information directly ...

> Yes, that would be nice, but based on what state would it provide this
> information? Are you suggesting that some new file should be added to
> .git/rebase-merge/ to keep track of it (more reliably than the amend
> file does today), or could the information be derived somehow from what
> exists already?

I don't know which parts are reliably derive-able today, but to
the extent that `git status` *can't* provide the necessary information,
I'd call that a bug in `gitt status`, to be fixed by saving that information
somewhere. Until it exists, there's nothing you can do, but if and when
(a) it does exist and (b) `git status` provides it, it won't matter
whether it's in some added worktree, or moves around or changes
form: you'll get it in a reliable fashion.

In other words, this is not helpful *yet*. :-)

Chris

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 06.03.23 04:31, Chris Torek wrote:
> On Sat, Mar 4, 2023 at 12:36 AM Stefan Haller <lists@haller-berlin.de> wrote:
>> I'm surprised to hear that worktrees play a role in this. Do you have
>> more details about this?
> 
> $ git worktree add foo
> $ cd ../foo
> $ git rebase -i HEAD~2
> 
> puts the rebase-todo file into
> `.git/worktrees/foo/rebase-merge/git-rebase-todo`;
> all the status files will likewise be in the `.git/worktrees/foo/`
> directory.

I see. That should be taken care of by not hard-coding ".git/...", but
using something like "git rev-parse --git-path ...". We are doing that
already, otherwise most of lazygit's other rebase commands wouldn't work
in worktrees.

>>> It would be best if `git status`
>>> reported all of this information directly ...
> 
>> Yes, that would be nice, but based on what state would it provide this
>> information? Are you suggesting that some new file should be added to
>> .git/rebase-merge/ to keep track of it (more reliably than the amend
>> file does today), or could the information be derived somehow from what
>> exists already?
> 
> I don't know which parts are reliably derive-able today, but to
> the extent that `git status` *can't* provide the necessary information,
> I'd call that a bug in `gitt status`, to be fixed by saving that information
> somewhere. Until it exists, there's nothing you can do, but if and when
> (a) it does exist and (b) `git status` provides it, it won't matter
> whether it's in some added worktree, or moves around or changes
> form: you'll get it in a reliable fashion.
> 
> In other words, this is not helpful *yet*. :-)

I see, thanks for the explanation. I agree. Trouble is, I don't have the
capacity to drive such an addition to git status, and I doubt anybody
else would be interested in doing it for me (are you? :-), so...

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Junio C Hamano]

Stefan Haller <lists@haller-berlin.de> writes:

> The reason why I am asking this is: I'm using lazygit, which, during
> interactive rebases, shows a combined view of the real commits that were
> already applied, and the remaining commits that are yet to be applied
> (it gets these by parsing rebase-merge/git-rebase-todo); something like
> this, when I set the 2nd commit to "edit":
>
>   pick   4th commit
>   pick   3rd commit
>          2nd commit  <-- YOU ARE HERE
>          1st commit
>
> This is great, but ...

Stepping a bit, how does our "git status" fare here?  It shows what
step in a sequence "rebase -i" the user who got control back (due to
"break", "exec sh", "edit" or a conflicted "pick") is in.  Or at
least it tries to.  Does it suffer from the same "great, but ..."?

If it works better than how lazygit shows, perhaps how it computes
the current state can be reproduced, or better yet, the current
state it computed can be exposed, and it can be prototyped by
parsing "LC_ALL=C git status -uno" output, perhaps?

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 05.03.23 21:15, Phillip Wood wrote:
> Hi Stefan
> 
> On 05/03/2023 19:13, Stefan Haller wrote:
>> On 05.03.23 17:59, Stefan Haller wrote:
>>> On 05.03.23 15:31, Phillip Wood wrote:
>>>> Hi Stefan
>>>>
>>>> On 02/03/2023 20:27, Stefan Haller wrote:
>>>>> On 02.03.23 11:19, Phillip Wood wrote:
>>>>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>>>>> The reason why I am asking this is: I'm using lazygit, which, during
>>>>>>> interactive rebases, shows a combined view of the real commits that
>>>>>>> were
>>>>>>> already applied, and the remaining commits that are yet to be
>>>>>>> applied
>>>>>>> (it gets these by parsing rebase-merge/git-rebase-todo);
>>>>>>> something like
>>>>>>> this, when I set the 2nd commit to "edit":
>>>>>>>
>>>>>>>      pick   4th commit
>>>>>>>      pick   3rd commit
>>>>>>>             2nd commit  <-- YOU ARE HERE
>>>>>>>             1st commit
>>>>>>>
>>>>>>> This is great, but assuming that the 2nd commit conflicted,
>>>>>>> currently
>>>>>>> the display looks like this:
>>>>>>>
>>>>>>>      pick   4th commit
>>>>>>>      pick   3rd commit
>>>>>>>             1st commit  <-- YOU ARE HERE
>>>>>>>
>>>>>>> I would like to extend this to also show a "fake entry" for the
>>>>>>> commit
>>>>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>>>>> except that I need a way to distinguish whether it was applied
>>>>>>> already
>>>>>>> or not.
>>>>>>
>>>>>> Can you check the index for conflicts when the rebase stops?
>>>>>
>>>>> I could do that, but then the fake entry would go away as soon as I
>>>>> have
>>>>> staged all conflict resolutions. I would find it useful for it to stay
>>>>> visible in that case, until I continue the rebase.
>>>>
>>>> I've not used lazygit but looking at the github page it seems that
>>>> it is
>>>> a persistent process that runs "git rebase". If that's the case I would
>>>> think that you can check for conflicts when the rebase stops and keep
>>>> that value in memory until the rebase is started again.
>>>
>>> I had considered that, but it would be preferable if it were possible to
>>> quit lazygit, start it again, and have it show the same state again. Or
>>> even start the rebase outside of lazygit while it isn't running at all,
>>> and then start it and have it display the correct state.
>>>
>>>> I think your best bet might be to read "$(git rev-parse --git-path
>>>> rebase-merge/done)" the last line of which contains the last todo
>>>> command the rebase tried to execute.
>>>
>>> I'm not sure I understand; you mean in order to distinguish whether it
>>> was a pick or a fixup?
>>
>> OK, I guess it's something like
>>
>> show_fake_entry :=
>>    REBASE_HEAD exists
>>    && (last command in "done" file was not "edit"
>>        || "amend" file exists)
>>
>> Is that what you meant? (Minus the bit about rescheduling failed
>> commands, which I still need to wrap my head around...)
> 
> I meant you could just use the done file to get the last command, there
> is no need to look at REBASE_HEAD.

OK, I see. Sounds like a possible algorithm could be:

func commitNameToShowAsTheCurrentlyConflictingCommit() {
    lastDone := last command of "done" file
    if lastDone.command is "break" or "exec" {
        return nil
    }

    next := first command of "git-rebase-todo" file
    if lastDone == next {
        // Command was rescheduled and shows in remaining todos already
        return nil
    }

    if lastDone.command is "edit" {
        if "amend" file exists {
            // "edit" command was successful
            return nil
        }
    }

    return lastDone.commitName
}

Does this sound reasonable?

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 07.03.23 19:07, Junio C Hamano wrote:
> Stefan Haller <lists@haller-berlin.de> writes:
> 
>> The reason why I am asking this is: I'm using lazygit, which, during
>> interactive rebases, shows a combined view of the real commits that were
>> already applied, and the remaining commits that are yet to be applied
>> (it gets these by parsing rebase-merge/git-rebase-todo); something like
>> this, when I set the 2nd commit to "edit":
>>
>>   pick   4th commit
>>   pick   3rd commit
>>          2nd commit  <-- YOU ARE HERE
>>          1st commit
>>
>> This is great, but ...
> 
> Stepping a bit, how does our "git status" fare here?  It shows what
> step in a sequence "rebase -i" the user who got control back (due to
> "break", "exec sh", "edit" or a conflicted "pick") is in.  Or at
> least it tries to.  Does it suffer from the same "great, but ..."?
> 
> If it works better than how lazygit shows, perhaps how it computes
> the current state can be reproduced, or better yet, the current
> state it computed can be exposed, and it can be prototyped by
> parsing "LC_ALL=C git status -uno" output, perhaps?

It fares a little better, but not much, and it doesn't look like I can
use its information to implement the behavior I want.

The difference to lazygit is that git status shows both the last few
entries of the "done" file and the first few entries of the
"git-rebase-todo" file, so at least I have a complete picture of which
commits are involved. Lazygit doesn't show the "done" file, only the
commits that are on the branch now, so the currently conflicting commit
is missing.

In addition, git status shows the message "(fix conflicts and then run
"git rebase --continue")", but only if there are unmerged files; once I
resolve all conflicts and stage the changes, that message is no longer
shown, and at that point I don't see any way to tell whether the last
commit was applied successfully or not.

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Junio C Hamano]

Stefan Haller <lists@haller-berlin.de> writes:

> On 07.03.23 19:07, Junio C Hamano wrote:
> ...
>> Stepping a bit, how does our "git status" fare here?  It shows what
>> step in a sequence "rebase -i" the user who got control back (due to
>> "break", "exec sh", "edit" or a conflicted "pick") is in.  Or at
>> least it tries to.  Does it suffer from the same "great, but ..."?
>> ...
>
> It fares a little better, but not much, and it doesn't look like I can
> use its information to implement the behavior I want.

Thanks.  That is the kind of information I was trying to find.  It
means that the current "git status" does not give our users enough
clue as to where in their "rebase -i" session they are at, and we
will help more users by teaching "git status" the trick you are
designing.  Instead of peeking into how the implementation details
like REBASE_HEAD currently happen to work, making sure underlying
"git" knows how to present the information you want and letting it
perform the heavy lifting would make sure the solution will stay
supported across versions of future git.

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 08.03.23 20:40, Junio C Hamano wrote:
> Stefan Haller <lists@haller-berlin.de> writes:
> 
>> On 07.03.23 19:07, Junio C Hamano wrote:
>> ...
>>> Stepping a bit, how does our "git status" fare here?  It shows what
>>> step in a sequence "rebase -i" the user who got control back (due to
>>> "break", "exec sh", "edit" or a conflicted "pick") is in.  Or at
>>> least it tries to.  Does it suffer from the same "great, but ..."?
>>> ...
>>
>> It fares a little better, but not much, and it doesn't look like I can
>> use its information to implement the behavior I want.
> 
> Thanks.  That is the kind of information I was trying to find.  It
> means that the current "git status" does not give our users enough
> clue as to where in their "rebase -i" session they are at, and we
> will help more users by teaching "git status" the trick you are
> designing.  Instead of peeking into how the implementation details
> like REBASE_HEAD currently happen to work, making sure underlying
> "git" knows how to present the information you want and letting it
> perform the heavy lifting would make sure the solution will stay
> supported across versions of future git.

Yes; as I said over in the other branch of this thread, I agree that it
would be great to have this in status, but I'm afraid I'm not the one to
drive the work to get it in there. I don't have the capacity to
contribute to yet another open source project.

If somebody else wanted to take this on, I'd be happy to join any design
discussions, or do early testing to see if whatever is being added works
for my use case.

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Phillip Wood]

Hi Stefan

On 08/03/2023 19:02, Stefan Haller wrote:
> On 05.03.23 21:15, Phillip Wood wrote:
>> Hi Stefan
>>
>> On 05/03/2023 19:13, Stefan Haller wrote:
>>> On 05.03.23 17:59, Stefan Haller wrote:
>>>> On 05.03.23 15:31, Phillip Wood wrote:
>>>>> Hi Stefan
>>>>>
>>>>> On 02/03/2023 20:27, Stefan Haller wrote:
>>>>>> On 02.03.23 11:19, Phillip Wood wrote:
>>>>>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>>>>>> The reason why I am asking this is: I'm using lazygit, which, during
>>>>>>>> interactive rebases, shows a combined view of the real commits that
>>>>>>>> were
>>>>>>>> already applied, and the remaining commits that are yet to be
>>>>>>>> applied
>>>>>>>> (it gets these by parsing rebase-merge/git-rebase-todo);
>>>>>>>> something like
>>>>>>>> this, when I set the 2nd commit to "edit":
>>>>>>>>
>>>>>>>>       pick   4th commit
>>>>>>>>       pick   3rd commit
>>>>>>>>              2nd commit  <-- YOU ARE HERE
>>>>>>>>              1st commit
>>>>>>>>
>>>>>>>> This is great, but assuming that the 2nd commit conflicted,
>>>>>>>> currently
>>>>>>>> the display looks like this:
>>>>>>>>
>>>>>>>>       pick   4th commit
>>>>>>>>       pick   3rd commit
>>>>>>>>              1st commit  <-- YOU ARE HERE
>>>>>>>>
>>>>>>>> I would like to extend this to also show a "fake entry" for the
>>>>>>>> commit
>>>>>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>>>>>> except that I need a way to distinguish whether it was applied
>>>>>>>> already
>>>>>>>> or not.
> OK, I see. Sounds like a possible algorithm could be:
> 
> func commitNameToShowAsTheCurrentlyConflictingCommit() {

Going back to your original email, if all you want to do is show "YOU 
ARE HERE" against the correct commit then you can get that from the last 
entry in the "done" file irrespective of whether there were conflicts or 
not. If you also want to show whether it was picked cleanly or not then 
it is more complicated.

>      lastDone := last command of "done" file
>      if lastDone.command is "break" or "exec" {
>          return nil
>      }
> 
>      next := first command of "git-rebase-todo" file
>      if lastDone == next {
>          // Command was rescheduled and shows in remaining todos already
>          return nil
>      }

I don't know what your current implementation looks like but if I was 
starting from scratch I think it would be simpler to always use the last 
command from "done" and then suppress the first command from 
"git-rebase-todo" if it is the same. That way the "YOU ARE HERE" always 
marks the last command from "done"

>      if lastDone.command is "edit" {
>          if "amend" file exists {
>              // "edit" command was successful
>              return nil
>          }
>      }

If you need to know if the pick was successful or not then that should 
tell you, but if all you need to know is which commit the rebase stopped 
on then you don't need this.

Best Wishes

Phillip


>      return lastDone.commitName
> }
> 
> Does this sound reasonable?

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 10.03.23 10:56, Phillip Wood wrote:
> Hi Stefan
> 
> On 08/03/2023 19:02, Stefan Haller wrote:
>> On 05.03.23 21:15, Phillip Wood wrote:
>>> Hi Stefan
>>>
>>> On 05/03/2023 19:13, Stefan Haller wrote:
>>>> On 05.03.23 17:59, Stefan Haller wrote:
>>>>> On 05.03.23 15:31, Phillip Wood wrote:
>>>>>> Hi Stefan
>>>>>>
>>>>>> On 02/03/2023 20:27, Stefan Haller wrote:
>>>>>>> On 02.03.23 11:19, Phillip Wood wrote:
>>>>>>>> On 28/02/2023 12:55, Stefan Haller wrote:
>>>>>>>>> The reason why I am asking this is: I'm using lazygit, which,
>>>>>>>>> during
>>>>>>>>> interactive rebases, shows a combined view of the real commits
>>>>>>>>> that
>>>>>>>>> were
>>>>>>>>> already applied, and the remaining commits that are yet to be
>>>>>>>>> applied
>>>>>>>>> (it gets these by parsing rebase-merge/git-rebase-todo);
>>>>>>>>> something like
>>>>>>>>> this, when I set the 2nd commit to "edit":
>>>>>>>>>
>>>>>>>>>       pick   4th commit
>>>>>>>>>       pick   3rd commit
>>>>>>>>>              2nd commit  <-- YOU ARE HERE
>>>>>>>>>              1st commit
>>>>>>>>>
>>>>>>>>> This is great, but assuming that the 2nd commit conflicted,
>>>>>>>>> currently
>>>>>>>>> the display looks like this:
>>>>>>>>>
>>>>>>>>>       pick   4th commit
>>>>>>>>>       pick   3rd commit
>>>>>>>>>              1st commit  <-- YOU ARE HERE
>>>>>>>>>
>>>>>>>>> I would like to extend this to also show a "fake entry" for the
>>>>>>>>> commit
>>>>>>>>> that conflicted, if there is one. REBASE_HEAD is perfect for this,
>>>>>>>>> except that I need a way to distinguish whether it was applied
>>>>>>>>> already
>>>>>>>>> or not.
>> OK, I see. Sounds like a possible algorithm could be:
>>
>> func commitNameToShowAsTheCurrentlyConflictingCommit() {
> 
> Going back to your original email, if all you want to do is show "YOU
> ARE HERE" against the correct commit then you can get that from the last
> entry in the "done" file irrespective of whether there were conflicts or
> not. If you also want to show whether it was picked cleanly or not then
> it is more complicated.

In my first example above, I'm editing the 2nd commit and it applied
cleanly. What I see in that case is

pick   4th commit
pick   3rd commit
       2nd commit  <-- YOU ARE HERE
       1st commit

2nd commit is the current HEAD, which corresponds to the last entry in
the "done" file, so there's no reason to display the "done" file entry
too in this case.

Only if it wasn't applied cleanly (and wasn't rescheduled) do I want to
show the "done" entry. This way I always see four entries in my list, no
matter what happened (unless I drop commits, of course), which is what I
want.

>>      lastDone := last command of "done" file
>>      if lastDone.command is "break" or "exec" {
>>          return nil
>>      }
>>
>>      next := first command of "git-rebase-todo" file
>>      if lastDone == next {
>>          // Command was rescheduled and shows in remaining todos already
>>          return nil
>>      }
> 
> I don't know what your current implementation looks like but if I was
> starting from scratch I think it would be simpler to always use the last
> command from "done" and then suppress the first command from
> "git-rebase-todo" if it is the same.

That would also be possible, but in lazygit it makes things harder,
because items from "git-rebase-todo" can be interacted with (e.g. moved
up or down, or dropped), whereas the "done" entry is only for display
purposes and can't be interacted with. That's why I called it a "fake"
entry above.

I tried the algorithm now and seems to do exactly what I want, so thanks
for the help with this, I'm happy now.

There's a slight correction though: if a command is rescheduled, then
git appears to re-append the last successful command to the end of the
"done" file (it's in there twice now). So I have to check the
second-to-last command of "done" against the first command of
"git-rebase-todo" to find out if a command was rescheduled.

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Phillip Wood]

Hi Stefan

On 10/03/2023 17:42, Stefan Haller wrote:
> I tried the algorithm now and seems to do exactly what I want, so thanks
> for the help with this, I'm happy now.

I'm glad you've got something working

> There's a slight correction though: if a command is rescheduled, then
> git appears to re-append the last successful command to the end of the
> "done" file (it's in there twice now). So I have to check the > second-to-last command of "done" against the first command of
> "git-rebase-todo" to find out if a command was rescheduled.

Oh, sorry I'd misremembered how it works. I think it is actually a bug 
that we write the last successful command to "done" again when 
rescheduling but it sounds like you have a reliable work-around.

Best Wishes

Phillip

> -Stefan

Re: When exactly should REBASE_HEAD exist?

[Phillip Wood]

Hi Stefan

On 16/03/2023 17:46, Phillip Wood wrote:
> Hi Stefan
> 
> On 10/03/2023 17:42, Stefan Haller wrote:
>> I tried the algorithm now and seems to do exactly what I want, so thanks
>> for the help with this, I'm happy now.
> 
> I'm glad you've got something working
> 
>> There's a slight correction though: if a command is rescheduled, then
>> git appears to re-append the last successful command to the end of the
>> "done" file (it's in there twice now). So I have to check the > 
>> second-to-last command of "done" against the first command of
>> "git-rebase-todo" to find out if a command was rescheduled.
> 
> Oh, sorry I'd misremembered how it works. I think it is actually a bug 
> that we write the last successful command to "done" again when 
> rescheduling but it sounds like you have a reliable work-around.

Hmm, I think it is a bit complicated to do this reliably as if "pick B" 
has conflicts and git-rebase-todo contains

	exec make test

and done contains

	pick A
	exec make test
	pick B

it looks like "exec make test" has been rescheduled when in fact "pick 
B" failed with conflicts. If you look at the last three lines of "done" 
you can distinguish the two cases but it is a bit of a faff. I've 
submitted a bugfix at 
https://lore.kernel.org/git/pull.1492.git.1679237337683.gitgitgadget@gmail.com

Best Wishes

Phillip

> Best Wishes
> 
> Phillip
> 
>> -Stefan

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 19.03.23 15:50, Phillip Wood wrote:
> Hmm, I think it is a bit complicated to do this reliably as if "pick B"
> has conflicts and git-rebase-todo contains
> 
>     exec make test
> 
> and done contains
> 
>     pick A
>     exec make test
>     pick B
> 
> it looks like "exec make test" has been rescheduled when in fact "pick
> B" failed with conflicts. If you look at the last three lines of "done"
> you can distinguish the two cases but it is a bit of a faff.

Ah, thanks for coming up with this tricky case. I'm leaning towards
simply ignoring it, since (at least for my usage of git) it's very
uncommon; if I use "-x make test" to check all commits, I almost always
do that in place, so I won't get conflicts.

But it's one more argument for adding the information to git status, I
suppose.

-Stefan

Re: When exactly should REBASE_HEAD exist?

[Stefan Haller]

On 20.03.23 08:29, Stefan Haller wrote:
> Ah, thanks for coming up with this tricky case. I'm leaning towards
> simply ignoring it, since (at least for my usage of git) it's very
> uncommon; if I use "-x make test" to check all commits, I almost always
> do that in place, so I won't get conflicts.

On the other hand, it seems easy enough to do it properly, so why not.
Something like this:

  num_done := len(done)
  is_rescheduled :=
     # for new versions of git:
     done[num_done - 1] == todo[0]
     # for older versions of git:
     || (num_done >= 3
         && done[num_done - 2] == todo[0]
         && done[num_done - 1] == done[num_done - 3])

Could I be missing any other cases with this logic?

-Stefan