Propose promoting 'contrib/rerere-train.sh' to command

Propose promoting 'contrib/rerere-train.sh' to command

[Tom Miller]

I would like to propose promoting 'contrib/rerere-train.sh' to one of the
following:

    1. A builtin c command 'builtin/rerere-train.c'
    2. To the top level directory as a built in script 'git-rerere-train.sh'

I have recently found myself writing scripts using 'contrib/rerere-train.sh'
and I wish it was built into the command. This would make it easier to use
rather than having to find it on different platforms. I think it could also
benefit from some documentation.

I am trying to gauge the interest in this change before spending some time on
working on it. I would also appreciate feedback or alternative approaches to
what I have suggested. Thank you for your time.

Thanks,
Tom Miller

Re: Propose promoting 'contrib/rerere-train.sh' to command

[Philip Oakley]

Hi Tom,

On 21/12/2019 02:17, Tom Miller wrote:
> I would like to propose promoting 'contrib/rerere-train.sh' to one of the
> following:
>
>     1. A builtin c command 'builtin/rerere-train.c'
>     2. To the top level directory as a built in script 'git-rerere-train.sh'
>
> I have recently found myself writing scripts using 'contrib/rerere-train.sh'
> and I wish it was built into the command. This would make it easier to use
> rather than having to find it on different platforms. I think it could also
> benefit from some documentation.
>
> I am trying to gauge the interest in this change before spending some time on
> working on it. I would also appreciate feedback or alternative approaches to
> what I have suggested. Thank you for your time.
>
> Thanks,
> Tom Miller
This sounds like a useful capability (especially the documentation!).

I also had aspirations that there should also be a way of exchanging the
essence of the rerere data when conflicts (e.g. feature vs master branch
merges) are known locally and ought to be communicable with the patch sets.

see messages:
<37ccaad0-40b4-ca63-e057-791119d7fa69@talktalk.net>
<nycvar.QRO.7.76.6.1909261253400.15067@tvgsbejvaqbjf.bet>

Philip

Re: Propose promoting 'contrib/rerere-train.sh' to command

[Philip Oakley]

On 21/12/2019 17:00, Philip Oakley wrote:
> Hi Tom,
>
> On 21/12/2019 02:17, Tom Miller wrote:
>> I would like to propose promoting 'contrib/rerere-train.sh' to one of the
>> following:
>>
>>     1. A builtin c command 'builtin/rerere-train.c'
>>     2. To the top level directory as a built in script 'git-rerere-train.sh'
>>
>> I have recently found myself writing scripts using 'contrib/rerere-train.sh'
>> and I wish it was built into the command. This would make it easier to use
>> rather than having to find it on different platforms. I think it could also
>> benefit from some documentation.
>>
>> I am trying to gauge the interest in this change before spending some time on
>> working on it. I would also appreciate feedback or alternative approaches to
>> what I have suggested. Thank you for your time.
>>
>> Thanks,
>> Tom Miller
> This sounds like a useful capability (especially the documentation!).
>
> I also had aspirations that there should also be a way of exchanging the
> essence of the rerere data when conflicts (e.g. feature vs master branch
> merges) are known locally and ought to be communicable with the patch sets.
>
> see messages:
> <37ccaad0-40b4-ca63-e057-791119d7fa69@talktalk.net>
> <nycvar.QRO.7.76.6.1909261253400.15067@tvgsbejvaqbjf.bet>
>
> Philip
There is also audio of the section of the Virtual Conf in a personal
email <20190914165929.GA22300@sigill.intra.peff.net> (~30MB) if needed.

Re: Propose promoting 'contrib/rerere-train.sh' to command

[Jeff King]

On Fri, Dec 20, 2019 at 08:17:57PM -0600, Tom Miller wrote:

> I would like to propose promoting 'contrib/rerere-train.sh' to one of the
> following:
> 
>     1. A builtin c command 'builtin/rerere-train.c'
>     2. To the top level directory as a built in script 'git-rerere-train.sh'
> 
> I have recently found myself writing scripts using 'contrib/rerere-train.sh'
> and I wish it was built into the command. This would make it easier to use
> rather than having to find it on different platforms. I think it could also
> benefit from some documentation.
> 
> I am trying to gauge the interest in this change before spending some time on
> working on it. I would also appreciate feedback or alternative approaches to
> what I have suggested. Thank you for your time.

The situations where I need rerere-train don't come up often, but when
they do, it has always worked easily and without hiccups for me. So
perhaps there are lurking gotchas that Junio might know about, but AFAIK
the quality is high enough for it to be part of normal Git.

I'd suggest converting it to C and making it a sub-command of rerere
(i.e., "git rerere train a..b") rather than a separate command.

It would be a nice bonus if it could do its work internally by looking
at the commits, rather than munging the working tree through checkout
and merge. I'm not sure how easy that would be, though. Historically the
merge-recursive machinery has not worked well without having a working
tree to dump the conflicted paths into, but I think that has gotten
somewhat better recently. So even a straight C conversion of the shell
script would be an OK starting point, I think.

-Peff

Re: Propose promoting 'contrib/rerere-train.sh' to command

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> The situations where I need rerere-train don't come up often, but when
> they do, it has always worked easily and without hiccups for me. So
> perhaps there are lurking gotchas that Junio might know about, but AFAIK
> the quality is high enough for it to be part of normal Git.

I actually suspect that rewriting has a high chance of initially
degrading the quality, so we should take a two step approach if we
really want it as part of the core distribution.  As to the UI, I
think "git rerere train a..b" would be a good one, but if the
scripted version is of high quality (I haven't looked at it for a
long time---even though I used it for a couple of times a year in
recent years), perhaps we can add it as "git-rerere--train"
subcommand that is spawned from "builtin/rerere.c" for the first
cut?

> I'd suggest converting it to C and making it a sub-command of rerere
> (i.e., "git rerere train a..b") rather than a separate command.

Re: Propose promoting 'contrib/rerere-train.sh' to command

[Jeff King]

On Sat, Dec 21, 2019 at 03:52:53PM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > The situations where I need rerere-train don't come up often, but when
> > they do, it has always worked easily and without hiccups for me. So
> > perhaps there are lurking gotchas that Junio might know about, but AFAIK
> > the quality is high enough for it to be part of normal Git.
> 
> I actually suspect that rewriting has a high chance of initially
> degrading the quality, so we should take a two step approach if we
> really want it as part of the core distribution.  As to the UI, I
> think "git rerere train a..b" would be a good one, but if the
> scripted version is of high quality (I haven't looked at it for a
> long time---even though I used it for a couple of times a year in
> recent years), perhaps we can add it as "git-rerere--train"
> subcommand that is spawned from "builtin/rerere.c" for the first
> cut?

Yeah, I'd be pretty happy with that, too.

I just suspect its ultimate fate is conversion to C, given the general
trend. And converting it to C that just calls out to other git commands
via run_command would presumably behave just like the original, leaving
the more challenging and error-prone conversion for later. Hopefully any
upgrade to "real Git command" would include some tests, though. :)

-Peff

Re: Propose promoting 'contrib/rerere-train.sh' to command

[Tom Miller]

On Sun Dec 22, 2019 at 2:58 AM, Jeff King wrote:
> On Sat, Dec 21, 2019 at 03:52:53PM -0800, Junio C Hamano wrote:
>
> 
> > Jeff King <peff@peff.net> writes:
> > 
> > > The situations where I need rerere-train don't come up often, but when
> > > they do, it has always worked easily and without hiccups for me. So
> > > perhaps there are lurking gotchas that Junio might know about, but AFAIK
> > > the quality is high enough for it to be part of normal Git.
> > 
> > I actually suspect that rewriting has a high chance of initially
> > degrading the quality, so we should take a two step approach if we
> > really want it as part of the core distribution.  As to the UI, I
> > think "git rerere train a..b" would be a good one, but if the
> > scripted version is of high quality (I haven't looked at it for a
> > long time---even though I used it for a couple of times a year in
> > recent years), perhaps we can add it as "git-rerere--train"
> > subcommand that is spawned from "builtin/rerere.c" for the first
> > cut?
>
> 
> Yeah, I'd be pretty happy with that, too.
>
> 
> I just suspect its ultimate fate is conversion to C, given the general
> trend. And converting it to C that just calls out to other git commands
> via run_command would presumably behave just like the original, leaving
> the more challenging and error-prone conversion for later. Hopefully any
> upgrade to "real Git command" would include some tests, though. :)
>
> 
> -Peff

Thanks for the feedback everyone! This is roughly the feedback I
expected. I think moving the shell version first will give more time to
focus on getting a strong test harness in place and a some decent
documentation. I happy with doing this in a multi phase approach as it
introduces less risk. When I get to the point of writing it in C I will
try to do the work internally by looking at commits as Jeff has
suggested, and fall back to run commands if I have to.

Re: Propose promoting 'contrib/rerere-train.sh' to command

[Johannes Schindelin]

Hi,

On Sun, 22 Dec 2019, Tom Miller wrote:

> On Sun Dec 22, 2019 at 2:58 AM, Jeff King wrote:
> > On Sat, Dec 21, 2019 at 03:52:53PM -0800, Junio C Hamano wrote:
> >
> >
> > > Jeff King <peff@peff.net> writes:
> > >
> > > > The situations where I need rerere-train don't come up often, but when
> > > > they do, it has always worked easily and without hiccups for me. So
> > > > perhaps there are lurking gotchas that Junio might know about, but AFAIK
> > > > the quality is high enough for it to be part of normal Git.
> > >
> > > I actually suspect that rewriting has a high chance of initially
> > > degrading the quality, so we should take a two step approach if we
> > > really want it as part of the core distribution.  As to the UI, I
> > > think "git rerere train a..b" would be a good one, but if the
> > > scripted version is of high quality (I haven't looked at it for a
> > > long time---even though I used it for a couple of times a year in
> > > recent years), perhaps we can add it as "git-rerere--train"
> > > subcommand that is spawned from "builtin/rerere.c" for the first
> > > cut?
> >
> >
> > Yeah, I'd be pretty happy with that, too.
> >
> >
> > I just suspect its ultimate fate is conversion to C, given the general
> > trend. And converting it to C that just calls out to other git commands
> > via run_command would presumably behave just like the original, leaving
> > the more challenging and error-prone conversion for later. Hopefully any
> > upgrade to "real Git command" would include some tests, though. :)
> >
> >
> > -Peff
>
> Thanks for the feedback everyone! This is roughly the feedback I
> expected. I think moving the shell version first will give more time to
> focus on getting a strong test harness in place and a some decent
> documentation. I happy with doing this in a multi phase approach as it
> introduces less risk. When I get to the point of writing it in C I will
> try to do the work internally by looking at commits as Jeff has
> suggested, and fall back to run commands if I have to.

As the person who traditionally ends up being stuck with such conversions,
I would like to point out that the rerere-train script has not changed _in
years_ and that there is little point in trying to let it "cook" some more
in the scripted form, _in particular_ because this would prevent the
command from having access to the more powerful C API for future
improvements.

One very obvious enhancement would be to perform the training in memory,
without even clobbering the worktree. That's simply not a thing that a
shell script can perform.

Food for thought,
Dscho