Git multiple remotes push stop at first failed connection

Git multiple remotes push stop at first failed connection

[John Siu]

Hi,

git version 2.26.2
OS: MacOS 10.15.5
Install via brew

Let say my project has following remotes:

$ git remote -v
git.all "server A git url" (fetch)
git.all "server A git url" (push)
git.all "server B git url" (push)
git.all "server C git ur" (push)

When all serverA/B/C are online, "git push" works.

However "git push" will stop at the first server it failed to connect.
So if git cannot connect to server A, it will not continue with server
B/C.

In the past I have server C turn off from time to time, so failing the
last push is expected. However recently server A went offline
completely and we notice git is not pushing to the remaining 2
remotes.

Not sure if this is intended behavior or can be improved.

John Siu

Re: Git multiple remotes push stop at first failed connection

[Jeff King]

On Sun, May 31, 2020 at 08:28:38PM -0400, John Siu wrote:

> Let say my project has following remotes:
> 
> $ git remote -v
> git.all "server A git url" (fetch)
> git.all "server A git url" (push)
> git.all "server B git url" (push)
> git.all "server C git ur" (push)
> 
> When all serverA/B/C are online, "git push" works.

A slight nomenclature nit, but that's _one_ remote that has several
push urls.

> However "git push" will stop at the first server it failed to connect.
> So if git cannot connect to server A, it will not continue with server
> B/C.
> 
> In the past I have server C turn off from time to time, so failing the
> last push is expected. However recently server A went offline
> completely and we notice git is not pushing to the remaining 2
> remotes.
> 
> Not sure if this is intended behavior or can be improved.

I don't think we've ever documented the error-handling semantics.
Looking at the relevant code in builtin/push.c:do_push():

          url_nr = push_url_of_remote(remote, &url);
          if (url_nr) {
                  for (i = 0; i < url_nr; i++) {
                          struct transport *transport =
                                  transport_get(remote, url[i]);
                          if (flags & TRANSPORT_PUSH_OPTIONS)
                                  transport->push_options = push_options;
                          if (push_with_options(transport, push_refspec, flags))
                                  errs++;
                  }
          } else {
                  struct transport *transport =
                          transport_get(remote, NULL);
                  if (flags & TRANSPORT_PUSH_OPTIONS)
                          transport->push_options = push_options;
                  if (push_with_options(transport, push_refspec, flags))
                          errs++;
          }
          return !!errs;

it does seem to try each one and collect the errors. But the underlying
transport code is so ready to die() on errors, taking down the whole
process, that I suspect it rarely manages to do so. You're probably much
better off defining a separate remote for each push destination, then
running your own shell loop:

  err=0
  for dst in serverA serverB serverC; do
    git push $dst || err=1
  done
  exit $err

There's really no benefit to doing it all in a single Git process, as
we'd connect to each independently, run a separate independent
pack-objects for each, etc.

I'd even suggest that Git implement such a loop itself, as we did for
"git fetch --all", but sadly "push --all" is already taken for a
different meaning (but it might still be worth doing under a different
option name).

-Peff

Re: Git multiple remotes push stop at first failed connection

[John Siu]

Thank Jeff for replying.

Yes, one route we are looking into is use fetch -all,  push -all and
wrap push tag with a loop, or may do what you suggest and wrap all 3
in a loop for consistency.

On Mon, Jun 1, 2020 at 5:40 PM Jeff King <peff@peff.net> wrote:
>
> On Sun, May 31, 2020 at 08:28:38PM -0400, John Siu wrote:
>
> > Let say my project has following remotes:
> >
> > $ git remote -v
> > git.all "server A git url" (fetch)
> > git.all "server A git url" (push)
> > git.all "server B git url" (push)
> > git.all "server C git ur" (push)
> >
> > When all serverA/B/C are online, "git push" works.
>
> A slight nomenclature nit, but that's _one_ remote that has several
> push urls.
>
> > However "git push" will stop at the first server it failed to connect.
> > So if git cannot connect to server A, it will not continue with server
> > B/C.
> >
> > In the past I have server C turn off from time to time, so failing the
> > last push is expected. However recently server A went offline
> > completely and we notice git is not pushing to the remaining 2
> > remotes.
> >
> > Not sure if this is intended behavior or can be improved.
>
> I don't think we've ever documented the error-handling semantics.
> Looking at the relevant code in builtin/push.c:do_push():
>
>           url_nr = push_url_of_remote(remote, &url);
>           if (url_nr) {
>                   for (i = 0; i < url_nr; i++) {
>                           struct transport *transport =
>                                   transport_get(remote, url[i]);
>                           if (flags & TRANSPORT_PUSH_OPTIONS)
>                                   transport->push_options = push_options;
>                           if (push_with_options(transport, push_refspec, flags))
>                                   errs++;
>                   }
>           } else {
>                   struct transport *transport =
>                           transport_get(remote, NULL);
>                   if (flags & TRANSPORT_PUSH_OPTIONS)
>                           transport->push_options = push_options;
>                   if (push_with_options(transport, push_refspec, flags))
>                           errs++;
>           }
>           return !!errs;
>
> it does seem to try each one and collect the errors. But the underlying
> transport code is so ready to die() on errors, taking down the whole
> process, that I suspect it rarely manages to do so. You're probably much
> better off defining a separate remote for each push destination, then
> running your own shell loop:
>
>   err=0
>   for dst in serverA serverB serverC; do
>     git push $dst || err=1
>   done
>   exit $err
>
> There's really no benefit to doing it all in a single Git process, as
> we'd connect to each independently, run a separate independent
> pack-objects for each, etc.
>
> I'd even suggest that Git implement such a loop itself, as we did for
> "git fetch --all", but sadly "push --all" is already taken for a
> different meaning (but it might still be worth doing under a different
> option name).
>
> -Peff

Re: Git multiple remotes push stop at first failed connection

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> There's really no benefit to doing it all in a single Git process, as
> we'd connect to each independently, run a separate independent
> pack-objects for each, etc.
>
> I'd even suggest that Git implement such a loop itself, as we did for
> "git fetch --all", but sadly "push --all" is already taken for a
> different meaning (but it might still be worth doing under a different
> option name).

I wonder if it is possible to update the implementation to do so
without changing the UI at all, though.

The presence of the "--all" option in "fetch" command is tied
closely to the fact that it makes no sense to have multiple URLs
that are used to download from at the same time under a single
remote name (e.g. what should "remotes/origin/master" point at if
two URLs say different things if such an arrangement were allowed?).

On the other hand, the pushURL for a single remote can be multiple
places for redundancy (a possible #leftoverbits here is that we
should probably disable the "pretend that we immediately turned
around and fetched from them after pushing" optimization when
pushing to a remote that has multiple pushURLs defined) does not
need an extra option.  If the way we currently push is suboptimal
and it is better to spawn a separate "git push" instance via the
run_command() API, that can safely be done as a bugfix without
affecting any UI elements, no?

Re: Git multiple remotes push stop at first failed connection

[John Siu]

On Tue, Jun 2, 2020 at 12:26 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Jeff King <peff@peff.net> writes:
>
> > There's really no benefit to doing it all in a single Git process, as
> > we'd connect to each independently, run a separate independent
> > pack-objects for each, etc.
> >
> > I'd even suggest that Git implement such a loop itself, as we did for
> > "git fetch --all", but sadly "push --all" is already taken for a
> > different meaning (but it might still be worth doing under a different
> > option name).
>

Yes. We notice the fetch/push --all is for branches.

> I wonder if it is possible to update the implementation to do so
> without changing the UI at all, though.
>
> The presence of the "--all" option in "fetch" command is tied
> closely to the fact that it makes no sense to have multiple URLs
> that are used to download from at the same time under a single
> remote name (e.g. what should "remotes/origin/master" point at if
> two URLs say different things if such an arrangement were allowed?).
>
> On the other hand, the pushURL for a single remote can be multiple
> places for redundancy (a possible #leftoverbits here is that we
> should probably disable the "pretend that we immediately turned
> around and fetched from them after pushing" optimization when
> pushing to a remote that has multiple pushURLs defined) does not
> need an extra option.  If the way we currently push is suboptimal
> and it is better to spawn a separate "git push" instance via the
> run_command() API, that can safely be done as a bugfix without
> affecting any UI elements, no?
>

I agree a "bugfix" for push only is good enough and safe. As the
current behavior is already pushing to all pushURLs of a single
remote. We are not trying to change behavior or do anything extra.