Re: Parallel checkout (Was Re: 0 bot for Git)

[Duy Nguyen <pclouds@gmail.com>]

On Fri, Apr 15, 2016 at 4:51 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> Numbers are encouraging though. On linux-2.6 repo running on linux and
> ext4 filesystem, checkout_paths() would dominate "git checkout :/".
> Unmodified git takes about 31s.
>
>
> 16:26:00.114029 builtin/checkout.c:1299 performance: 31.184973659 s: checkout_paths
> 16:26:00.114225 trace.c:420             performance: 31.256412935 s: git command: 'git' 'checkout' '.'
>
> When doing write_entry() on 8 processes, it takes 22s (shortened by ~30%)

I continued to develop it into a series. This same laptop now reduces
checkout time closer to 50% on linux-2.6. However my other laptop
gives me the opposite result, parallel checkout takes longer time to
complete. I suspect that only with fast enough disks that CPU may
become temporary bottleneck. This is where parallel checkout shines
because it spreads the load out and quickly moves the bottleneck back
to I/O (after a while I/O queues should be fully populated again). On
systems with slower disks like mine, I/O is always the bottleneck and
spreading I/O over many processes just makes it worse (probably
confuse I/O scheduler more).

Since it's not doing anything for _me_, I'm dropping this. Anybody
interested can check it out and maybe try it from parallel-checkout
branch [1]. It probably can build on windows (epoll is gone). And it
probably help improve performance when smudge filter is used (because
that can potentially add more load to cpu). More notes in commit
8fe9b5c (entry.c: parallel checkout support - 2016-04-18)

[1] https://github.com/pclouds/git/commits/parallel-checkout
-- 
Duy