Git status takes too long- How to improve the performance of git

Git status takes too long- How to improve the performance of git

[ravalika]

Hi All,

We are using git-1.8.2 version for version control.
It is an centralized server and git status takes too long

How to improve the performance of git status

Git repo details:

Size of the .git folder is 8.9MB
Number of commits approx 53838  (git rev-list HEAD --count)
Number of branches -  330  
Number of files - 63883
Working tree clone size is 4.3GB

time git status shows
real	0m23.673s
user	0m9.432s
sys	0m3.793s

then after 5 mins
real    0m4.864s
user    0m1.417s
sys     0m4.710s

And I have experimented the following ways 
-          -  Setting core.ignorestat to true
-          -  Git gc &git clean
-          -  Shallow clone – Reducing number of commits
-          -  Clone only one branch 
      - Git repacking - git repack -ad && git prune
-     - Cold/warm cache 

Could you please let me know, what are the ways to improve the git
performance ?
I have gone through the mailing lists.


Thank you,
Renuka



--
View this message in context: http://git.661346.n2.nabble.com/Git-status-takes-too-long-How-to-improve-the-performance-of-git-tp7657456.html
Sent from the git mailing list archive at Nabble.com.

Re: Git status takes too long- How to improve the performance of git

[Fredrik Gustafsson]

Hi,

On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
> We are using git-1.8.2 version for version control.

That's a three (almost four) year old version of git. Your first test
should be to see if an upgrade to a recent version will improve things.

> It is an centralized server and git status takes too long

A centralized server? How? git is designed to be runned locally. If
you're running git on a network file system, the performance will
suffer. Could you elaborate on how your environment is setup?

> 
> How to improve the performance of git status
> 
> Git repo details:
> 
> Size of the .git folder is 8.9MB
> Number of commits approx 53838  (git rev-list HEAD --count)
> Number of branches -  330  
> Number of files - 63883
> Working tree clone size is 4.3GB

.git folder of 8.9 MEGABYTE and working tree of 4.3 GIGABYTE? Is this a
typo?

> 
> time git status shows
> real	0m23.673s
> user	0m9.432s
> sys	0m3.793s
> 
> then after 5 mins
> real    0m4.864s
> user    0m1.417s
> sys     0m4.710s

A slow disc and empty caches are slow. Two ways of improving this is to
have faster discs or make sure your cache is up to date. When I'd a
really slow disc, I'd my shell to run a git status in the background to
load the cache everytime I started working on a project. This is however
an ugly hack that wasn't approved to be a part of git.

> 
> And I have experimented the following ways 
> -          -  Setting core.ignorestat to true
> -          -  Git gc &git clean
> -          -  Shallow clone – Reducing number of commits
> -          -  Clone only one branch 
>       - Git repacking - git repack -ad && git prune
> -     - Cold/warm cache 
> 
> Could you please let me know, what are the ways to improve the git
> performance ?
> I have gone through the mailing lists.

You could always check the --assume-unchanged bit, see the manual page
for git update-index. However this is quite extreme and demanding for
the user.

-- 
Fredrik Gustafsson

phone: +46 733-608274
e-mail: iveqy@iveqy.com
website: http://www.iveqy.com

Re: Git status takes too long- How to improve the performance of git

[Christian Couder]

On Tue, Nov 15, 2016 at 11:24 AM, Fredrik Gustafsson <iveqy@iveqy.com> wrote:
> On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:

[...]

>> And I have experimented the following ways
>> -          -  Setting core.ignorestat to true
>> -          -  Git gc &git clean
>> -          -  Shallow clone – Reducing number of commits
>> -          -  Clone only one branch
>>       - Git repacking - git repack -ad && git prune
>> -     - Cold/warm cache
>>
>> Could you please let me know, what are the ways to improve the git
>> performance ?
>> I have gone through the mailing lists.
>
> You could always check the --assume-unchanged bit, see the manual page
> for git update-index. However this is quite extreme and demanding for
> the user.

If you install a recent version version, you may be able to use the
untracked cache feature.
(See "core.untrackedCache" in the git config documentation and
--untracked-cache in the git update-index documentation.)

Re: Git status takes too long- How to improve the performance of git

[Fredrik Gustafsson]

On Wed, Nov 16, 2016 at 05:13:57PM +0530, Renuka Pampana wrote:
> > On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
> > > It is an centralized server and git status takes too long
> >
> > A centralized server? How? git is designed to be runned locally. If
> > you're running git on a network file system, the performance will
> > suffer. Could you elaborate on how your environment is setup?
> >
> >
> We have setup main git repository in remote location on Linux server
> And created a git repository in local Linux server, as a reference for the
> remote git repository,
> And update the local git repository for every 15 min in local server
> 
> Users will be able to access the  local git repository through NFS

And each user will have their own copy of the repository locally on
their machine? That is having done a git clone?
> 
> All users will clone the git repository from remote project url  by using
> local git repo as reference
> 
>  For example : git clone --reference ~/gitcaches/reference user@drupal
> :/home/project/drupal.git
> 
> All the users have ssh credentials for the remote server

Why are you using --reference for a 8.9MB big clone?
> 
> 
> What is the best way to implement remote git repo and able to access the
> git repo from other location, without any performance glitches?
> Users should be able to access git repo from different servers and from
> different locations.

The best way is to have it locally cloned. Yes the initial clone will be
expensive but operations after that will be fairly smooth. You do not(!)
want to execute git on one machine and having the repository beeing on
an other machine (for example via a network file system, except git
clone, git fetch, git push, etc.).

> >
> > > How to improve the performance of git status
> > >
> > > Git repo details:
> > >
> > > Size of the .git folder is 8.9MB
> > > Number of commits approx 53838  (git rev-list HEAD --count)
> > > Number of branches -  330
> > > Number of files - 63883
> > > Working tree clone size is 4.3GB
> >
> > .git folder of 8.9 MEGABYTE and working tree of 4.3 GIGABYTE? Is this a
> > typo?
> >
> > All git related information is stored in .git directory of the working
> directory
>   It is 8.9M
> And size of the local workspace is 4.3G

Can you please elaborate on this? How can you store 8.9 MB of data that
will result in a 4.3 G workspace?

-- 
Fredrik Gustafsson

phone: +46 733-608274
e-mail: iveqy@iveqy.com
website: http://www.iveqy.com

Re: Git status takes too long- How to improve the performance of git

[Heiko Voigt]

On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
> Number of files - 63883

Since you also posted this to the "Git for Windows" mailinglist I assume
that you are using Windows. Reduce the number of files. For example
split the repository into two one for documentation and one for source.
Thats what I did with a converted repository that had to many files.

Windows is unfortunately very slow when it comes to handling many files
and if I recall correctly ~30000 files was in a nicely handleable range
for a Git repository on Windows, but that might have changed...

Cheers Heiko

Re: Git status takes too long- How to improve the performance of git

[MaryTurner]

Nice



--
View this message in context: http://git.661346.n2.nabble.com/Git-status-takes-too-long-How-to-improve-the-performance-of-git-tp7657456p7657457.html
Sent from the git mailing list archive at Nabble.com.