Git Evolve

Git Evolve

[Stefan Xenos]

Hello, List!

I'm interested in porting something like Mercurial's evolve command to
Git. I'll be following up with a formal proposal shortly, but before I
do I thought I'd introduce myself to the list and find out if anyone
else is interested in this topic.

What is the evolve command?

Imagine you have three dependent changes up for review and you receive
feedback that requires editing all three changes. While you're editing
one, more feedback arrives on one of the others. What do you do?

The evolve command is a convenient way to work with chains of commits
that are under review. Whenever you rebase or amend a commit, the
repository remembers that the old commit is obsolete and has been
replaced by the new one. Then, at some point in the future, you can
run "git evolve" and the correct sequence of rebases will occur in the
correct order such that no commit has an obsolete parent.

Part of making the "evolve" command work involves tracking the edits
to a commit over time, which could provide a lot of other benefits:

- Systems like gerrit would no longer need to rely on "change-id" tags
in commit comments to associate commits with the change that they
edit, since git itself would have that information.
- You could directly view the history of a commit over time (ie: the
sequence of amends and rebases that occurred with that commit,
orthogonal to the history of the branch it is on). If you've used
mercurial, this would be a git equivalent to "hg obslog". If you've
used gerrit, this would be like the gerrit "change log" but it would
work for all commits and work offline.
- You can easily list all the changes that you have as works-in
progress. If you've used gerrit, this would be an offline equivalent
to the gerrit dashboard.
- You could choose to share the history of a commit with others, or
collaborate on and merge different variants of the same change.

Some information about Mercurial's evolve command can be found here:
https://www.mercurial-scm.org/doc/evolution/

Other similar technologies:

rebase -i can be used to solve the same problem, but you can't easily
switch tasks midway through an interactive rebase or have more than
one interactive rebase going on at the same time. It also can't handle
the case where you have multiple changes sharing the same parent that
needs to be rebased and won't let you collaborate with others on
resolving a complicated interactive rebase.

patch queues (topgit, stgit, quilt) address a very similar problem,
however since they're built on top of git rather than integrated with
it, most of them end up managing extra state that can get easily
damaged whenever you run a native git command that doesn't know about
the patch queue. Most of them also have various workflow problems that
aren't present in hg evolve.

Is anyone else interested in this? Please email me directly or on this
list. Let's chat: I want to make sure that whatever we come up with is
at least as good as any similar technology that has come before.

Re: Git Evolve

[Junio C Hamano]

Stefan Xenos <sxenos@google.com> writes:

> What is the evolve command?
> ...
> - Systems like gerrit would no longer need to rely on "change-id" tags
> in commit comments to associate commits with the change that they
> edit, since git itself would have that information.
> ...
> Is anyone else interested in this? Please email me directly or on this
> list. Let's chat: I want to make sure that whatever we come up with is
> at least as good as any similar technology that has come before.

As you listed in the related technologies section, I think the
underlying machinery that supports "rebase -i", especially with the
recent addition of redoing the existing merges (i.e. "rebase -i
-r"), may be enough to rewrite the histories that were built on top
of a commit that has been obsoleted by amending.

I would imagine that the main design effort you would need to make
is to figure out a good way to

 (1) keep track of which commits are obsoleted by which other ones
     [*1*], and

 (2) to figure out what histories are still to be rebuilt in what
     order on top of what commit efficiently.

Once these are done, you should be able to write out the sequence of
instructions to feed the same sequencer machinery used by the
"rebase -i" command.

[Side note]

*1* It is very desirable to keep track of the evolution of a change
    without polluting the commit object with things like Change-Id:
    and other cruft, either in the body or in the header.  If we
    lose the Change-Id: footer without adding any new cruft in the
    commit object header, that would be a great success.  It would
    be a failure if we end up touching the object header.

Re: Git Evolve

[Stefan Xenos]

> If we lose the Change-Id: footer without adding any new cruft in the
> commit object header, that would be a great success.  It would
> be a failure if we end up touching the object header.

Yes, I was thinking along the same lines.

If obsolescence graph is stored as part of the commit header, it would
get unconditionally shared whenever that commit is pushed, and I would
consider it a desirable property if users could choose whether to push
just a commit or a the obsolescence graph that contains the commit.
Sometimes the reason you're amending a commit may be because the
original contained content that shouldn't be pushed.

Putting it in the header would also either cause it to get retained
forever by git gc or create a situation where we permit dangling
references - and neither seem desirable to me. It would be nice if
users could retain the obsolescence graph for stuff a user is actively
working on, but we could discard it (with 0 cost) for commits they're
done with or were never interested in editing.

On Sat, Sep 29, 2018 at 5:55 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Xenos <sxenos@google.com> writes:
>
>> What is the evolve command?
>> ...
>> - Systems like gerrit would no longer need to rely on "change-id" tags
>> in commit comments to associate commits with the change that they
>> edit, since git itself would have that information.
>> ...
>> Is anyone else interested in this? Please email me directly or on this
>> list. Let's chat: I want to make sure that whatever we come up with is
>> at least as good as any similar technology that has come before.
>
> As you listed in the related technologies section, I think the
> underlying machinery that supports "rebase -i", especially with the
> recent addition of redoing the existing merges (i.e. "rebase -i
> -r"), may be enough to rewrite the histories that were built on top
> of a commit that has been obsoleted by amending.
>
> I would imagine that the main design effort you would need to make
> is to figure out a good way to
>
>  (1) keep track of which commits are obsoleted by which other ones
>      [*1*], and
>
>  (2) to figure out what histories are still to be rebuilt in what
>      order on top of what commit efficiently.
>
> Once these are done, you should be able to write out the sequence of
> instructions to feed the same sequencer machinery used by the
> "rebase -i" command.
>
> [Side note]
>
> *1* It is very desirable to keep track of the evolution of a change
>     without polluting the commit object with things like Change-Id:
>     and other cruft, either in the body or in the header.  If we
>     lose the Change-Id: footer without adding any new cruft in the
>     commit object header, that would be a great success.  It would
>     be a failure if we end up touching the object header.
>
>
>
>

Re: Git Evolve

[Jakub Narebski]

Junio C Hamano <gitster@pobox.com> writes:
> Stefan Xenos <sxenos@google.com> writes:
>
>> What is the evolve command?
>> ...
>> - Systems like gerrit would no longer need to rely on "change-id" tags
>> in commit comments to associate commits with the change that they
>> edit, since git itself would have that information.
>> ...
>> Is anyone else interested in this? Please email me directly or on this
>> list. Let's chat: I want to make sure that whatever we come up with is
>> at least as good as any similar technology that has come before.
>
> As you listed in the related technologies section, I think the
> underlying machinery that supports "rebase -i", especially with the
> recent addition of redoing the existing merges (i.e. "rebase -i
> -r"), may be enough to rewrite the histories that were built on top
> of a commit that has been obsoleted by amending.
>
> I would imagine that the main design effort you would need to make
> is to figure out a good way to
>
>  (1) keep track of which commits are obsoleted by which other ones
>      [*1*], and
>
>  (2) to figure out what histories are still to be rebuilt in what
>      order on top of what commit efficiently.
>
> Once these are done, you should be able to write out the sequence of
> instructions to feed the same sequencer machinery used by the
> "rebase -i" command.

Well, that assumes that "rebase -i" can correctly recreate merges, if
needed.

> [Side note]
>
> *1* It is very desirable to keep track of the evolution of a change
>     without polluting the commit object with things like Change-Id:
>     and other cruft, either in the body or in the header.  If we
>     lose the Change-Id: footer without adding any new cruft in the
>     commit object header, that would be a great success.  It would
>     be a failure if we end up touching the object header.

Doesn't Gerrit use git-notes instead of 'Change-Id:' trailer nowadays?
Notes transport is quite easily controlled; the problem with notes merge
does not matter for this use.

Best,
--
Jakub Narębski

Re: Git Evolve

[Stefan Xenos]

Gerrit uses notes and branches of meta-commits internally for its
database, but it still uses the change-id footers to associate an
uploaded commit with a change within its database.

On Thu, Oct 4, 2018 at 9:05 AM, Jakub Narebski <jnareb@gmail.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>> Stefan Xenos <sxenos@google.com> writes:
>>
>>> What is the evolve command?
>>> ...
>>> - Systems like gerrit would no longer need to rely on "change-id" tags
>>> in commit comments to associate commits with the change that they
>>> edit, since git itself would have that information.
>>> ...
>>> Is anyone else interested in this? Please email me directly or on this
>>> list. Let's chat: I want to make sure that whatever we come up with is
>>> at least as good as any similar technology that has come before.
>>
>> As you listed in the related technologies section, I think the
>> underlying machinery that supports "rebase -i", especially with the
>> recent addition of redoing the existing merges (i.e. "rebase -i
>> -r"), may be enough to rewrite the histories that were built on top
>> of a commit that has been obsoleted by amending.
>>
>> I would imagine that the main design effort you would need to make
>> is to figure out a good way to
>>
>>  (1) keep track of which commits are obsoleted by which other ones
>>      [*1*], and
>>
>>  (2) to figure out what histories are still to be rebuilt in what
>>      order on top of what commit efficiently.
>>
>> Once these are done, you should be able to write out the sequence of
>> instructions to feed the same sequencer machinery used by the
>> "rebase -i" command.
>
> Well, that assumes that "rebase -i" can correctly recreate merges, if
> needed.
>
>> [Side note]
>>
>> *1* It is very desirable to keep track of the evolution of a change
>>     without polluting the commit object with things like Change-Id:
>>     and other cruft, either in the body or in the header.  If we
>>     lose the Change-Id: footer without adding any new cruft in the
>>     commit object header, that would be a great success.  It would
>>     be a failure if we end up touching the object header.
>
> Doesn't Gerrit use git-notes instead of 'Change-Id:' trailer nowadays?
> Notes transport is quite easily controlled; the problem with notes merge
> does not matter for this use.
>
> Best,
> --
> Jakub Narębski

Re: Git Evolve

[Derrick Stolee]

On 9/29/2018 7:00 PM, Stefan Xenos wrote:
> Hello, List!

Hello! Welcome.

> I'm interested in porting something like Mercurial's evolve command to
> Git. I'll be following up with a formal proposal shortly, but before I
> do I thought I'd introduce myself to the list and find out if anyone
> else is interested in this topic.

I'm CC'ing some contributors who have also expressed interest in this topic.

> What is the evolve command?

I'm snipping the rest of your thread because I'm vaguely familiar with 
how this is used in hg, but I haven't used it myself. Instead, I'm going 
to ask you the same questions I asked the last time I had a conversation 
about this with someone. In my opinion, these questions should have good 
answers before we start working on the solution, or else we could paint 
ourselves into a corner as we build the first pieces.

---

What would the command-line experience look like for this workflow? Be 
specific, including examples!

How does one create a commit that obsoletes another? Are we in the 
middle of an interactive rebase, or are we simply checking out the 
commit? How does a use keep track of their progress in a topic?

How do I view which commits in my topic are obsolete, and to what commits?

If I want to obsolete commits on one machine and then finish the work on 
another machine, how do I do that? Similarly: how can I share obsolete 
commits with other users so they can apply them (or not)?

Do obsolescence markers live in the ref space? (This is one way to help 
answer the question above.)

Can I make multiple commits obsolete into one commit (merge patches)? 
Can I make my commit obsolete in multiple ways (split a patch)? How is 
this communicated to the user?

---

In my opinion, a good way to move forward is to create a patch that adds 
a design document to Documentation/technical that answers these 
questions. Then we can dig in more to make the vision clearer.

I'm not a UX expert, but this seems like a place where we could use 
someone with expertise in the area. If we are not careful, we could end 
up with something even harder to use than interactive rebase. The main 
goal here is to make it easy to rewrite a topic (plus, the ability to do 
so in stages).

I look forward to see what goes on in this space. Count me in to review 
the technical details.

Thanks,
-Stolee

Re: Git Evolve

[Stefan Xenos]

Sorry it's taken awhile for me to get back to this thread. I've been
keeping my open source contributions timeboxed, and had to work
through a bit of a backlog before this email thread got back to the
front of the queue.

> What would the command-line experience look like for this workflow? Be specific, including examples!

Yes, I agree that's critical. I'm not going to try to improvise an
answer over email, but I will be sure to include such examples when I
submit the patch to Documentation/technical, after some careful
thought.

> How does one create a commit that obsoletes another?

I was planning to classify all git commands that create commits as
either "modifications" or "clones". Modification commands (such as
rebase or commit --amend) would automatically obsolete their
predecessors. Clone commands (such as cherry-pick) would create copies
without obsoleting the original.

> Are we in the middle of an interactive rebase, or are we simply checking out the commit?

To obsolete a commit, you'd just check it out and amend. You really
wouldn't want to have to do anything extra or put yourself in a
special mode to mark things as obsolete or they wouldn't be much of a
convenience. However, if you were to run the "evolve" command (the
thing that reads the obsolescence markers) and it encountered errors,
you'd be in the middle of something very much like an interactive
rebase.

> How does a use keep track of their progress in a topic?
> How do I view which commits in my topic are obsolete, and to what commits?
> Do obsolescence markers live in the ref space? (This is one way to help answer the question above.)

The answers to these questions are more complicated. I'll be sure to
address them in the technical proposal, but I wouldn't want create a
misunderstanding with an incomplete explanation.

> Can I make multiple commits obsolete into one commit (merge patches)? Can I make my commit obsolete in multiple ways (split a patch)? How is this communicated to the user?

Yes and yes. When a single commit is obsoleted by one or more
non-obsolete commits, this is called divergence and it needs to be
resolved by the user (the evolve command will stop when it encounters
divergence, much like a rebase will stop when it encounters a merge
conflict). Possible resolutions are to discard one of the two
non-obsolete commits by marking them as obsolete, merge the two, or
keep both by cloning one of them and marking it as no longer
obsoleting its predecessor.

Unnecessary divergence isn't a good thing since it requires user
intervention to resolve. Splitting a patch doesn't need to create
divergence (and probably shouldn't). If you split a commit, the
earlier one would be a completely new commit and the latter one would
obsolete the original. Doing it the other way around creates a merge
conflict between the later commit and anything that followed the
original. If both commits obsolete the original, it creates divergence
- and either divergence or a merge would get the user involved
unnecessarily.

A more likely scenario where divergence would occur is of two
different users try to amend the same commit and then share their
results -- or if a user amends a commit, resets back to the original,
and then amends it again.

> Count me in to review the technical details.

Will do!

On Mon, Oct 1, 2018 at 5:37 AM, Derrick Stolee <stolee@gmail.com> wrote:
> On 9/29/2018 7:00 PM, Stefan Xenos wrote:
>>
>> Hello, List!
>
>
> Hello! Welcome.
>
>> I'm interested in porting something like Mercurial's evolve command to
>> Git. I'll be following up with a formal proposal shortly, but before I
>> do I thought I'd introduce myself to the list and find out if anyone
>> else is interested in this topic.
>
>
> I'm CC'ing some contributors who have also expressed interest in this topic.
>
>> What is the evolve command?
>
>
> I'm snipping the rest of your thread because I'm vaguely familiar with how
> this is used in hg, but I haven't used it myself. Instead, I'm going to ask
> you the same questions I asked the last time I had a conversation about this
> with someone. In my opinion, these questions should have good answers before
> we start working on the solution, or else we could paint ourselves into a
> corner as we build the first pieces.
>
> ---
>
> What would the command-line experience look like for this workflow? Be
> specific, including examples!
>
> How does one create a commit that obsoletes another? Are we in the middle of
> an interactive rebase, or are we simply checking out the commit? How does a
> use keep track of their progress in a topic?
>
> How do I view which commits in my topic are obsolete, and to what commits?
>
> If I want to obsolete commits on one machine and then finish the work on
> another machine, how do I do that? Similarly: how can I share obsolete
> commits with other users so they can apply them (or not)?
>
> Do obsolescence markers live in the ref space? (This is one way to help
> answer the question above.)
>
> Can I make multiple commits obsolete into one commit (merge patches)? Can I
> make my commit obsolete in multiple ways (split a patch)? How is this
> communicated to the user?
>
> ---
>
> In my opinion, a good way to move forward is to create a patch that adds a
> design document to Documentation/technical that answers these questions.
> Then we can dig in more to make the vision clearer.
>
> I'm not a UX expert, but this seems like a place where we could use someone
> with expertise in the area. If we are not careful, we could end up with
> something even harder to use than interactive rebase. The main goal here is
> to make it easy to rewrite a topic (plus, the ability to do so in stages).
>
> I look forward to see what goes on in this space. Count me in to review the
> technical details.
>
> Thanks,
> -Stolee

Re: Git Evolve

[Taylor Blau]

Hi Stefan,

On Sat, Sep 29, 2018 at 04:00:04PM -0700, Stefan Xenos wrote:
> Hello, List!
>
> I'm interested in porting something like Mercurial's evolve command to
> Git.

Welcome to Git :-). I think that the discussion in this thread is good,
but it's not why I'm replying. I have also wanted a Mercurial feature in
Git, but a different one than yours.

Specifically, I've wanted the 'hg absorb' command. My understanding of
the commands functionality is that it builds a sort of flamegraph-esque
view of the blame, and then cascades downwards parts of a change. I am
sure that I'm not doing the command justice, so I'll defer to [1] where
it is explained in more detail.

The benefit of this command is that it gives you a way to--without
ambiguity--absorb changes into earlier commits, and in fact, the
earliest commit that they make sense to belong to.

This would simplify my workflow greatly when re-rolling patches, as I
often want to rewrite a part of an earlier commit. This is certainly
possible by a number of different `git rebase` invocations (e.g., (1)
create fixup commits, and then re-order them, or (2) mark points in your
history as 'edit', and rewrite them in a detached state, and I'm sure
many more).

I'm curious if you or anyone else has thought about how this might work
in Git.

Thanks,
Taylor

[1]: http://files.lihdd.net/hgabsorb-note.pdf

Re: Git Evolve

[Ævar Arnfjörð Bjarmason]

On Tue, Oct 02 2018, Taylor Blau wrote:

> Hi Stefan,
>
> On Sat, Sep 29, 2018 at 04:00:04PM -0700, Stefan Xenos wrote:
>> Hello, List!
>>
>> I'm interested in porting something like Mercurial's evolve command to
>> Git.
>
> Welcome to Git :-). I think that the discussion in this thread is good,
> but it's not why I'm replying. I have also wanted a Mercurial feature in
> Git, but a different one than yours.
>
> Specifically, I've wanted the 'hg absorb' command. My understanding of
> the commands functionality is that it builds a sort of flamegraph-esque
> view of the blame, and then cascades downwards parts of a change. I am
> sure that I'm not doing the command justice, so I'll defer to [1] where
> it is explained in more detail.
>
> The benefit of this command is that it gives you a way to--without
> ambiguity--absorb changes into earlier commits, and in fact, the
> earliest commit that they make sense to belong to.
>
> This would simplify my workflow greatly when re-rolling patches, as I
> often want to rewrite a part of an earlier commit. This is certainly
> possible by a number of different `git rebase` invocations (e.g., (1)
> create fixup commits, and then re-order them, or (2) mark points in your
> history as 'edit', and rewrite them in a detached state, and I'm sure
> many more).
>
> I'm curious if you or anyone else has thought about how this might work
> in Git.

I've wanted a "git absorb" for a while, but have done no actual work on
it, I just found out about it.

I think a combination of these two heuristics would probably do the
trick:

 1. If a change in your "git diff" output has a hunk whose lines overlap
    with an earlier commit in the @{u}.. range, we do the equivalent of
    "git add -p", select that hunk, and "git commit --fixup <that
    commit>". We fixup the most recent commit that matches (otherwise
    commit>we'd conflict).

 2. Have some mode where we fall back from #1 and consider changes to
    entire files, if that's unambiguous.

The neat thing about this would be that you could tweak how promiscuous
#1 would be via the -U option to git-diff, and #2 would just be a
special case of -U9999999999999 (we should really add a -Uinf...).

Then once you ran this you could run "git rebase -i --autosquash" to see
how the TODO list would look, and optionally have some "git absorb
--now" or whatever to do the "git add -p", "git commit --fixup" and "git
rebase --autosquash" all in one go.

> [1]: http://files.lihdd.net/hgabsorb-note.pdf

Re: Git Evolve

[Stefan Xenos]

If you're curious how the Mercurial absorb command works, here's the code:

https://www.mercurial-scm.org/repo/hg/file/tip/hgext/absorb.py

It's GPL 2:

https://www.mercurial-scm.org/repo/hg/file/tip/COPYING



On Tue, Oct 2, 2018 at 2:11 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Tue, Oct 02 2018, Taylor Blau wrote:
>
>> Hi Stefan,
>>
>> On Sat, Sep 29, 2018 at 04:00:04PM -0700, Stefan Xenos wrote:
>>> Hello, List!
>>>
>>> I'm interested in porting something like Mercurial's evolve command to
>>> Git.
>>
>> Welcome to Git :-). I think that the discussion in this thread is good,
>> but it's not why I'm replying. I have also wanted a Mercurial feature in
>> Git, but a different one than yours.
>>
>> Specifically, I've wanted the 'hg absorb' command. My understanding of
>> the commands functionality is that it builds a sort of flamegraph-esque
>> view of the blame, and then cascades downwards parts of a change. I am
>> sure that I'm not doing the command justice, so I'll defer to [1] where
>> it is explained in more detail.
>>
>> The benefit of this command is that it gives you a way to--without
>> ambiguity--absorb changes into earlier commits, and in fact, the
>> earliest commit that they make sense to belong to.
>>
>> This would simplify my workflow greatly when re-rolling patches, as I
>> often want to rewrite a part of an earlier commit. This is certainly
>> possible by a number of different `git rebase` invocations (e.g., (1)
>> create fixup commits, and then re-order them, or (2) mark points in your
>> history as 'edit', and rewrite them in a detached state, and I'm sure
>> many more).
>>
>> I'm curious if you or anyone else has thought about how this might work
>> in Git.
>
> I've wanted a "git absorb" for a while, but have done no actual work on
> it, I just found out about it.
>
> I think a combination of these two heuristics would probably do the
> trick:
>
>  1. If a change in your "git diff" output has a hunk whose lines overlap
>     with an earlier commit in the @{u}.. range, we do the equivalent of
>     "git add -p", select that hunk, and "git commit --fixup <that
>     commit>". We fixup the most recent commit that matches (otherwise
>     commit>we'd conflict).
>
>  2. Have some mode where we fall back from #1 and consider changes to
>     entire files, if that's unambiguous.
>
> The neat thing about this would be that you could tweak how promiscuous
> #1 would be via the -U option to git-diff, and #2 would just be a
> special case of -U9999999999999 (we should really add a -Uinf...).
>
> Then once you ran this you could run "git rebase -i --autosquash" to see
> how the TODO list would look, and optionally have some "git absorb
> --now" or whatever to do the "git add -p", "git commit --fixup" and "git
> rebase --autosquash" all in one go.
>
>> [1]: http://files.lihdd.net/hgabsorb-note.pdf

Re: Git Evolve

[Kyle Meyer]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Tue, Oct 02 2018, Taylor Blau wrote:

[...]

>> Specifically, I've wanted the 'hg absorb' command. My understanding of
>> the commands functionality is that it builds a sort of flamegraph-esque
>> view of the blame, and then cascades downwards parts of a change. I am
>> sure that I'm not doing the command justice, so I'll defer to [1] where
>> it is explained in more detail.
>>
>> The benefit of this command is that it gives you a way to--without
>> ambiguity--absorb changes into earlier commits, and in fact, the
>> earliest commit that they make sense to belong to.
>>
>> This would simplify my workflow greatly when re-rolling patches, as I
>> often want to rewrite a part of an earlier commit. This is certainly
>> possible by a number of different `git rebase` invocations (e.g., (1)
>> create fixup commits, and then re-order them, or (2) mark points in your
>> history as 'edit', and rewrite them in a detached state, and I'm sure
>> many more).
>>
>> I'm curious if you or anyone else has thought about how this might work
>> in Git.
>
> I've wanted a "git absorb" for a while, but have done no actual work on
> it, I just found out about it.

It may be worth looking into git-autofixup [0] (author cc'd).  I learned
about it when Magit used it in its magit-commit-absorb command, but I
haven't used it yet myself.

[0] https://github.com/torbiak/git-autofixup

-- 
Kyle

Re: Git Evolve

[Taylor Blau]

On Tue, Oct 02, 2018 at 11:11:11AM +0200, Ævar Arnfjörð Bjarmason wrote:

You timed this email quite well ;-).

> On Tue, Oct 02 2018, Taylor Blau wrote:
>
> > Hi Stefan,
> >
> > On Sat, Sep 29, 2018 at 04:00:04PM -0700, Stefan Xenos wrote:
> >> Hello, List!
> >>
> >> I'm interested in porting something like Mercurial's evolve command to
> >> Git.
> >
> > Welcome to Git :-). I think that the discussion in this thread is good,
> > but it's not why I'm replying. I have also wanted a Mercurial feature in
> > Git, but a different one than yours.
> >
> > Specifically, I've wanted the 'hg absorb' command. My understanding of
> > the commands functionality is that it builds a sort of flamegraph-esque
> > view of the blame, and then cascades downwards parts of a change. I am
> > sure that I'm not doing the command justice, so I'll defer to [1] where
> > it is explained in more detail.
> >
> > The benefit of this command is that it gives you a way to--without
> > ambiguity--absorb changes into earlier commits, and in fact, the
> > earliest commit that they make sense to belong to.
> >
> > This would simplify my workflow greatly when re-rolling patches, as I
> > often want to rewrite a part of an earlier commit. This is certainly
> > possible by a number of different `git rebase` invocations (e.g., (1)
> > create fixup commits, and then re-order them, or (2) mark points in your
> > history as 'edit', and rewrite them in a detached state, and I'm sure
> > many more).
> >
> > I'm curious if you or anyone else has thought about how this might work
> > in Git.
>
> I've wanted a "git absorb" for a while, but have done no actual work on
> it, I just found out about it.
>
> I think a combination of these two heuristics would probably do the
> trick:
>
>  1. If a change in your "git diff" output has a hunk whose lines overlap
>     with an earlier commit in the @{u}.. range, we do the equivalent of
>     "git add -p", select that hunk, and "git commit --fixup <that
>     commit>". We fixup the most recent commit that matches (otherwise
>     commit>we'd conflict).

I had imagined this working slightly differently. I think about it in
terms of a flamegraph-shape, where each line is affected by gravity.
Consider this:

       [---------------]
  L0   L1   L2   L3   L4   L5   L6   L7

Here's a line in a diff that affects L1-L4. Were we to create a
``fixup'' to L3-L4, it would look like this:

                 [-----] --|
       [---------|-----] <-|
  L0   L1   L2   L3   L4   L5   L6   L7

The commit owning the adjacent edit hunk is the one that gets the
changes applied to it.

Consider instead the case where we have two overlapping parts of a
change:

                 [-----|----]
       [---------|-----]
  L0   L1   L2   L3   L4   L5   L6   L7

The left-hand side of the top-most hunk belongs to the hunk below it,
but the right-hand side is ``affected by gravity'' down to the base:

                 [-----|
       [---------|-----]|---]
  L0   L1   L2   L3   L4   L5   L6   L7

And thus could be reapplied anywhere. I have not spent time proving this,
but I believe that this resolves appropriate bases without ambiguity
(or, at least can detect when finding a base introduces ambiguity).

>  2. Have some mode where we fall back from #1 and consider changes to
>     entire files, if that's unambiguous.
>
> The neat thing about this would be that you could tweak how promiscuous
> #1 would be via the -U option to git-diff, and #2 would just be a
> special case of -U9999999999999 (we should really add a -Uinf...).
>
> Then once you ran this you could run "git rebase -i --autosquash" to see
> how the TODO list would look, and optionally have some "git absorb
> --now" or whatever to do the "git add -p", "git commit --fixup" and "git
> rebase --autosquash" all in one go.

That's cool. I hadn't known about '--autosquash' before, but it now
makes sense to me why I have often seen patches that begin with
"fixup!".

Another thought that I am reminded of was an off-list discussion with
Peff, where we thought that a particularly Git-like way to implement
something like this would be to generate a set of commits that would be
immediately '--autosquash'-able.

> > [1]: http://files.lihdd.net/hgabsorb-note.pdf

Thanks,
Taylor

Re: Git Evolve

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 4217 bytes --]

Hi,

On Tue, 2 Oct 2018, Ævar Arnfjörð Bjarmason wrote:

> On Tue, Oct 02 2018, Taylor Blau wrote:
> 
> > Hi Stefan,
> >
> > On Sat, Sep 29, 2018 at 04:00:04PM -0700, Stefan Xenos wrote:
> >> Hello, List!
> >>
> >> I'm interested in porting something like Mercurial's evolve command to
> >> Git.
> >
> > Welcome to Git :-). I think that the discussion in this thread is good,
> > but it's not why I'm replying. I have also wanted a Mercurial feature in
> > Git, but a different one than yours.
> >
> > Specifically, I've wanted the 'hg absorb' command. My understanding of
> > the commands functionality is that it builds a sort of flamegraph-esque
> > view of the blame, and then cascades downwards parts of a change. I am
> > sure that I'm not doing the command justice, so I'll defer to [1] where
> > it is explained in more detail.
> >
> > The benefit of this command is that it gives you a way to--without
> > ambiguity--absorb changes into earlier commits, and in fact, the
> > earliest commit that they make sense to belong to.
> >
> > This would simplify my workflow greatly when re-rolling patches, as I
> > often want to rewrite a part of an earlier commit. This is certainly
> > possible by a number of different `git rebase` invocations (e.g., (1)
> > create fixup commits, and then re-order them, or (2) mark points in your
> > history as 'edit', and rewrite them in a detached state, and I'm sure
> > many more).
> >
> > I'm curious if you or anyone else has thought about how this might work
> > in Git.
> 
> I've wanted a "git absorb" for a while, but have done no actual work on
> it, I just found out about it.
> 
> I think a combination of these two heuristics would probably do the
> trick:
> 
>  1. If a change in your "git diff" output has a hunk whose lines overlap
>     with an earlier commit in the @{u}.. range, we do the equivalent of
>     "git add -p", select that hunk, and "git commit --fixup <that
>     commit>". We fixup the most recent commit that matches (otherwise
>     commit>we'd conflict).
> 
>  2. Have some mode where we fall back from #1 and consider changes to
>     entire files, if that's unambiguous.
> 
> The neat thing about this would be that you could tweak how promiscuous
> #1 would be via the -U option to git-diff, and #2 would just be a
> special case of -U9999999999999 (we should really add a -Uinf...).
> 
> Then once you ran this you could run "git rebase -i --autosquash" to see
> how the TODO list would look, and optionally have some "git absorb
> --now" or whatever to do the "git add -p", "git commit --fixup" and "git
> rebase --autosquash" all in one go.

This is essentially what the script does that I sent to this here mailing
list back in May:
https://public-inbox.org/git/nycvar.QRO.7.76.6.1805052220360.77@tvgsbejvaqbjf.bet/

I used this quite a lot, but I still find it slow (especially on Windows),
so I am still in search for a better solution. And yes, I was intrigued by
`hg absorb` when it was presented at GitMerge last year, and wanted to
have the same.

In the meantime, what I often do is to call `git log -L <range>:<file>`
where <range> and <file> are taken from the hunk(s) of the current diff.
Actually, I have a script to do that, hidden behind a Git alias. Then I
inspect the diffs in that log and call `git commit --fixup` with the one I
deem most appropriate.

Note that it sometimes fails because of semantic dependencies. That is,
even if my current change overlaps with an earlier change, it may be too
early to be squashed in.

As an example: imagine that I moved a git_config() call from some function
into the cmd_<command>() function. Only: I intended to move it, but in
fact, I just added the call to the latter function. Eventually, I figure
it out! So I want to make a fixup! commit. My script, as well as Ævar's
suggestion, as well as literally all the other attempts to solve this that
I am aware of, would try to squash this change into whichever commit
introduced the function that originally called git_config(). But that is
wrong. It should be squashed into the commit that added the git_config()
call to the cmd_<command>() function.

Ciao,
Dscho

> 
> > [1]: http://files.lihdd.net/hgabsorb-note.pdf
>