Proposal: Output should push to different servers in parallel

Proposal: Output should push to different servers in parallel

[Victor Porton]

I experienced a slowdown in Git pushing when I push to more than one server.

I propose:

Run push to several servers in parallel.

Not to mix the output, nevertheless serialize the output, that is for 
example cache the output of the second server push and start to output 
it immediately after the first server push is finished.

This approach combines the advantages of the current way (I suppose it 
is so) to serialize pushes: first push to the first server, then to the 
second, etc. and of my idea to push in parallel.

I think the best way would be use multithreading, but multiprocessing 
would be a good quick solution.

Re: Proposal: Output should push to different servers in parallel

[Ævar Arnfjörð Bjarmason]

On Wed, Feb 06 2019, Victor Porton wrote:

> I experienced a slowdown in Git pushing when I push to more than one server.
>
> I propose:
>
> Run push to several servers in parallel.
>
> Not to mix the output, nevertheless serialize the output, that is for
> example cache the output of the second server push and start to output
> it immediately after the first server push is finished.
>
> This approach combines the advantages of the current way (I suppose it
> is so) to serialize pushes: first push to the first server, then to
> the second, etc. and of my idea to push in parallel.
>
> I think the best way would be use multithreading, but multiprocessing
> would be a good quick solution.

This seems like a reasonable idea, until such time as someone submits
patches to implement this in git you can do this with some invocation of
GNU parallel -k, i.e. operate on N remotes in parallel, and use the -k
option to buffer up all their output and present it in sequence.

Re: Proposal: Output should push to different servers in parallel

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> This seems like a reasonable idea, until such time as someone submits
> patches to implement this in git you can do this with some invocation of
> GNU parallel -k, i.e. operate on N remotes in parallel, and use the -k
> option to buffer up all their output and present it in sequence.

Stopping the message there makes it sound like a polite way to say
"a generic tool to allow you doing it on anything, not limited to
Git, is already available, and a solution specific to Git is
unwanted."

I wanted to follow up with something that says "The 'parallel' tool
works in the meantime, but here are examples of very useful things
that we would not be able to live without that 'parallel' wouldn't
let us do, and we need a Git specific solution to obtain that", but
I am coming up with empty, so perhaps indeed we do not want a Git
specific solution ;-)