[RFC] new subcommand: git sync

[RFC] new subcommand: git sync

[Jean-Marie Lemetayer]

Hi folks,

I created a new "git sync" sub-command a few months ago to deal with the pull
request workflow.

Its goals are to:
 - keep all configured branches synchronized with the remotes (--set-upstream)
 - do not touch your wip feature branches (which has diverged from upstream)
 - prune the remotes

As I use it on a daily basis, to synchronize the remotes and then be able to
quickly rebase my pull requests. I think it's worth sharing. What do you think?

For now it is a simple shell script available here:
https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync

If you think it's a good idea, I'll propose a series of patches with the new
sub-command, the manual page and the associated tests.

Best regards,
Jean-Marie Lemetayer

Re: [RFC] new subcommand: git sync

[Ævar Arnfjörð Bjarmason]

On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:

> Hi folks,
>
> I created a new "git sync" sub-command a few months ago to deal with the pull
> request workflow.
>
> Its goals are to:
>  - keep all configured branches synchronized with the remotes (--set-upstream)
>  - do not touch your wip feature branches (which has diverged from upstream)
>  - prune the remotes
>
> As I use it on a daily basis, to synchronize the remotes and then be able to
> quickly rebase my pull requests. I think it's worth sharing. What do you think?
>
> For now it is a simple shell script available here:
> https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
>
> If you think it's a good idea, I'll propose a series of patches with the new
> sub-command, the manual page and the associated tests.

Have you seen 'git branch -v' and 'git branch -v --format=*'? There
seems to be a high amount of overlap between this wrapper you've written
and it.

I suspect most of what you have here could be turned into an %(if:*)
directive where you emit the pull/push command as appropriate.

If you search the internet for "git-sync" there's dozens of such command
(and I've personally observed at least two of them being written by
co-workers in real time, not sure if either of those is in the Google
results).

So I think there's probably a worthwhile problem to be solved here that
could be turned into patches to git.git, something between "git
[clone|push] --mirror" and "git branch -v".

I don't think there's any interest in getting new shellscript built-ins
in the future. We've been actively migrating away from those.

But most of the logic in your script is just calling the ref-filter.c
API behind the scenes.

B.t.w. you can probably speed up & simplify your script a lot by making
use of IFS="" in the shell and not calling N for-each-ref commands when
it seems to me that one invocation would do. Just dump the N fields you
need split on some token, and split on that token in your loop.

RE: [RFC] new subcommand: git sync

[Randall S. Becker]

On February 26, 2021 10:25 AM, : Ævar Arnfjörð Bjarmason wrote:
> To: Jean-Marie Lemetayer <jeanmarie.lemetayer@gmail.com>
> Cc: git@vger.kernel.org
> Subject: Re: [RFC] new subcommand: git sync
> 
> On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:
> 
> > Hi folks,
> >
> > I created a new "git sync" sub-command a few months ago to deal with
> > the pull request workflow.
> >
> > Its goals are to:
> >  - keep all configured branches synchronized with the remotes
> > (--set-upstream)
> >  - do not touch your wip feature branches (which has diverged from
> > upstream)
> >  - prune the remotes
> >
> > As I use it on a daily basis, to synchronize the remotes and then be
> > able to quickly rebase my pull requests. I think it's worth sharing. What do
> you think?
> >
> > For now it is a simple shell script available here:
> > https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
> >
> > If you think it's a good idea, I'll propose a series of patches with
> > the new sub-command, the manual page and the associated tests.
> 
> Have you seen 'git branch -v' and 'git branch -v --format=*'? There seems to
> be a high amount of overlap between this wrapper you've written and it.
> 
> I suspect most of what you have here could be turned into an %(if:*)
> directive where you emit the pull/push command as appropriate.
> 
> If you search the internet for "git-sync" there's dozens of such command
> (and I've personally observed at least two of them being written by co-
> workers in real time, not sure if either of those is in the Google results).
> 
> So I think there's probably a worthwhile problem to be solved here that
> could be turned into patches to git.git, something between "git [clone|push]
> --mirror" and "git branch -v".
> 
> I don't think there's any interest in getting new shellscript built-ins in the
> future. We've been actively migrating away from those.
> 
> But most of the logic in your script is just calling the ref-filter.c API behind the
> scenes.
> 
> B.t.w. you can probably speed up & simplify your script a lot by making use of
> IFS="" in the shell and not calling N for-each-ref commands when it seems to
> me that one invocation would do. Just dump the N fields you need split on
> some token, and split on that token in your loop.

Yeah, I'm one of those that has made extensive use of the name "sync" when using git to synchronize between (POSIX) OSS/USS and (Non-POSIX) NonStop/MVS respectively. If you're going somewhere with it, could I suggest something like "reconcile" or of it is specific to the pull workflow, maybe "pull-sync"? I agree that scripts are not desirable long-term.

My $0.02.
Randall

-- Brief whoami:
 NonStop developer since approximately 211288444200000000
 UNIX developer since approximately 421664400
 MVS developer a long time ago in a galaxy far far away
-- In my real life, I talk too much.

Re: [RFC] new subcommand: git sync

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> B.t.w. you can probably speed up & simplify your script a lot by making
> use of IFS="" in the shell and not calling N for-each-ref commands when
> it seems to me that one invocation would do. Just dump the N fields you
> need split on some token, and split on that token in your loop.

A hidden gem in for-each-ref is its ability to quote the placeholder
values in a language specific way, and that is to allow the --format
to generate a script that can be eval'ed.  E.g.

  $ git for-each-ref --shell \
    --format='doit %(authorname) %(subject)' refs/heads/ab/\* |
    head -n 3
  doit 'Ævar Arnfjörð Bjarmason' 'branch: show "HEAD detached" first ...'
  doit 'Ævar Arnfjörð Bjarmason' 'CoC: update to version 2.0 + local ...'
  doit 'Ævar Arnfjörð Bjarmason' 'config.mak.uname: remove unused NEE...'

so that you can prepare

	doit () {
		person=$1 subject=$2
		... do things on the branch data ...
	}

beforehand and then eval the output from the script you wrote with
"git for-each-ref".

Re: [RFC] new subcommand: git sync

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> so that you can prepare
> ...
> beforehand and then eval the output from the script you wrote with
> "git for-each-ref".

Ehhh, not "eval the output from the script", but "eval the script".
In essense, you use "for-each-ref --format" as a tool to write a
script, and then execute it.

Re: [RFC] new subcommand: git sync

[Jean-Marie Lemetayer]

On Fri, Feb 26, 2021 at 5:42 PM Randall S. Becker
<rsbecker@nexbridge.com> wrote:
>
> On February 26, 2021 10:25 AM, : Ævar Arnfjörð Bjarmason wrote:
> > To: Jean-Marie Lemetayer <jeanmarie.lemetayer@gmail.com>
> > Cc: git@vger.kernel.org
> > Subject: Re: [RFC] new subcommand: git sync
> >
> > On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:
> >
> > > Hi folks,
> > >
> > > I created a new "git sync" sub-command a few months ago to deal with
> > > the pull request workflow.
> > >
> > > Its goals are to:
> > >  - keep all configured branches synchronized with the remotes
> > > (--set-upstream)
> > >  - do not touch your wip feature branches (which has diverged from
> > > upstream)
> > >  - prune the remotes
> > >
> > > As I use it on a daily basis, to synchronize the remotes and then be
> > > able to quickly rebase my pull requests. I think it's worth sharing. What do
> > you think?
> > >
> > > For now it is a simple shell script available here:
> > > https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
> > >
> > > If you think it's a good idea, I'll propose a series of patches with
> > > the new sub-command, the manual page and the associated tests.
> >
> > Have you seen 'git branch -v' and 'git branch -v --format=*'? There seems to
> > be a high amount of overlap between this wrapper you've written and it.
> >
> > I suspect most of what you have here could be turned into an %(if:*)
> > directive where you emit the pull/push command as appropriate.
> >
> > If you search the internet for "git-sync" there's dozens of such command
> > (and I've personally observed at least two of them being written by co-
> > workers in real time, not sure if either of those is in the Google results).
> >
> > So I think there's probably a worthwhile problem to be solved here that
> > could be turned into patches to git.git, something between "git [clone|push]
> > --mirror" and "git branch -v".
> >
> > I don't think there's any interest in getting new shellscript built-ins in the
> > future. We've been actively migrating away from those.
> >
> > But most of the logic in your script is just calling the ref-filter.c API behind the
> > scenes.
> >
> > B.t.w. you can probably speed up & simplify your script a lot by making use of
> > IFS="" in the shell and not calling N for-each-ref commands when it seems to
> > me that one invocation would do. Just dump the N fields you need split on
> > some token, and split on that token in your loop.
>
> Yeah, I'm one of those that has made extensive use of the name "sync" when using git to synchronize between (POSIX) OSS/USS and (Non-POSIX) NonStop/MVS respectively. If you're going somewhere with it, could I suggest something like "reconcile" or of it is specific to the pull workflow, maybe "pull-sync"? I agree that scripts are not desirable long-term.

Thanks for the feedback !

To sum up -- if I have the time to do it -- I must do it in native C
and not use a generic name like "sync".

As it is a way to keep my branches up to date with the remotes, maybe
create an option for the "git branch" submodule instead of a new
subcommand? Maybe "git branch --synchronize" or something better ...

As using "Pull request" is not the only way to use this command and
also because there is a big trolling issue with the termes"Pull
request" and "Merge request" I don't think using "pull-sync" is a good
idea either.

I know that the first golden rule of coding is "Naming", but damn,
sometimes it's hard to find a good name.

Best regards,
Jean-Marie

RE: [RFC] new subcommand: git sync

[Randall S. Becker]

On March 1, 2021 4:48 AM, Jean-Marie Lemetayer wrote:
> To: Randall S. Becker <rsbecker@nexbridge.com>
> Cc: Ævar Arnfjörð Bjarmason <avarab@gmail.com>; git@vger.kernel.org
> Subject: Re: [RFC] new subcommand: git sync
> 
> On Fri, Feb 26, 2021 at 5:42 PM Randall S. Becker <rsbecker@nexbridge.com>
> wrote:
> >
> > On February 26, 2021 10:25 AM, : Ævar Arnfjörð Bjarmason wrote:
> > > To: Jean-Marie Lemetayer <jeanmarie.lemetayer@gmail.com>
> > > Cc: git@vger.kernel.org
> > > Subject: Re: [RFC] new subcommand: git sync
> > >
> > > On Fri, Feb 26 2021, Jean-Marie Lemetayer wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I created a new "git sync" sub-command a few months ago to deal
> > > > with the pull request workflow.
> > > >
> > > > Its goals are to:
> > > >  - keep all configured branches synchronized with the remotes
> > > > (--set-upstream)
> > > >  - do not touch your wip feature branches (which has diverged from
> > > > upstream)
> > > >  - prune the remotes
> > > >
> > > > As I use it on a daily basis, to synchronize the remotes and then
> > > > be able to quickly rebase my pull requests. I think it's worth
> > > > sharing. What do
> > > you think?
> > > >
> > > > For now it is a simple shell script available here:
> > > > https://github.com/jmlemetayer/one-time-setup/blob/main/git-sync
> > > >
> > > > If you think it's a good idea, I'll propose a series of patches
> > > > with the new sub-command, the manual page and the associated tests.
> > >
> > > Have you seen 'git branch -v' and 'git branch -v --format=*'? There
> > > seems to be a high amount of overlap between this wrapper you've
> written and it.
> > >
> > > I suspect most of what you have here could be turned into an %(if:*)
> > > directive where you emit the pull/push command as appropriate.
> > >
> > > If you search the internet for "git-sync" there's dozens of such
> > > command (and I've personally observed at least two of them being
> > > written by co- workers in real time, not sure if either of those is in the
> Google results).
> > >
> > > So I think there's probably a worthwhile problem to be solved here
> > > that could be turned into patches to git.git, something between "git
> > > [clone|push] --mirror" and "git branch -v".
> > >
> > > I don't think there's any interest in getting new shellscript
> > > built-ins in the future. We've been actively migrating away from those.
> > >
> > > But most of the logic in your script is just calling the
> > > ref-filter.c API behind the scenes.
> > >
> > > B.t.w. you can probably speed up & simplify your script a lot by
> > > making use of IFS="" in the shell and not calling N for-each-ref
> > > commands when it seems to me that one invocation would do. Just
> dump
> > > the N fields you need split on some token, and split on that token in your
> loop.
> >
> > Yeah, I'm one of those that has made extensive use of the name "sync"
> when using git to synchronize between (POSIX) OSS/USS and (Non-POSIX)
> NonStop/MVS respectively. If you're going somewhere with it, could I
> suggest something like "reconcile" or of it is specific to the pull workflow,
> maybe "pull-sync"? I agree that scripts are not desirable long-term.
> 
> Thanks for the feedback !
> 
> To sum up -- if I have the time to do it -- I must do it in native C and not use a
> generic name like "sync".
> 
> As it is a way to keep my branches up to date with the remotes, maybe
> create an option for the "git branch" submodule instead of a new
> subcommand? Maybe "git branch --synchronize" or something better ...
> 
> As using "Pull request" is not the only way to use this command and also
> because there is a big trolling issue with the termes"Pull request" and
> "Merge request" I don't think using "pull-sync" is a good idea either.
> 
> I know that the first golden rule of coding is "Naming", but damn, sometimes
> it's hard to find a good name.

It's almost feels like you are thinking along the lines of a variant of a mirror rather than pulls and merges. I wonder whether that analogy would hold up? I would love to have a stable mechanism of mirroring repositories when doing development on 2-3 platforms at the same time. Yes, pull/merge can be a bit heavy handed in this situation. I have considered something like rsync to do the job (which it can), but within a git context it sounds more like a mirror - although it would be nice to mirror work in progress. So perhaps this should go through the stash mechanism.