From: "Bevan Watkiss" <bevan.watkiss@cloakware.com>
To: "'Linus Torvalds'" <torvalds@linux-foundation.org>
Cc: "'Alex Riesen'" <raa.lkml@gmail.com>, <git@vger.kernel.org>
Subject: RE:
Date: Thu, 7 May 2009 15:37:59 -0400 [thread overview]
Message-ID: <A07C3E66E84D46ACB37EDC7D396CCA62@caottdt504> (raw)
In-Reply-To: <alpine.LFD.2.01.0905071148500.4983@localhost.localdomain>
Looking at the trace it does appear that most of this is the lstat. It's
the problem of having many tiny files on a network drive, and trying to use
git for something it's not meant.
The log has 265430 lines of lstat and 10887 other lines. If you still want
the log file I'll strip out the directory names and send it off.
It would be nice to have an option that you can pull only the files that
changed in the changesets you are updating and ignore the state of the other
files.
Bevan
-----Original Message-----
From: Linus Torvalds [mailto:torvalds@linux-foundation.org]
Sent: May 7, 2009 2:56 PM
To: Bevan Watkiss
Cc: 'Alex Riesen'; git@vger.kernel.org
Subject: RE:
On Thu, 7 May 2009, Bevan Watkiss wrote:
>
> Basically I have a copy of my tree where only git can write to it, so I
know
> the files are right. The NAS box I have the tree on is slow, so reading
the
> tree adds about 10 minutes to the process when I only want to update a few
> files.
Ouch.
You could try doing
[core]
preloadindex = true
and see if that helps some of your loads. It does limit even the parallel
tree stat to 20 or so, but if most of your cost is in just doing the
lstat() over the files to see that they haven't changed, you might be
getting a factor-of-20 speedup for at least _some_ of what you do.
If you can, it might also be interesting to see system call trace patterns
(with times!) to see if there is something obviously horribly bad going
on. If you're running under Linux, and don't think the data contains
anything very private, send me the output of "strace -f -T" of the most
problematic operations, and maybe I can see if I can come up with anything
interesting.
I have long refused to use networked filesystems because I used to find
them -so- painful when working with CVS, so none of my performance work
has ever really directly concentrated on long-latency filesystems. Even
the index preload was all done "blind" with other people reporting issues
(and happily I could see some of the effects with local filesystems and
multiple CPU's ;).
Linus
next prev parent reply other threads:[~2009-05-07 19:41 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-05-07 17:01 (unknown), Bevan Watkiss
2009-05-07 17:13 ` Alex Riesen
2009-05-07 17:26 ` Bevan Watkiss
2009-05-07 18:18 ` Alex Riesen
2009-05-07 18:48 ` Bevan Watkiss
2009-05-07 19:56 ` Björn Steinbrink
2009-05-07 18:56 ` Linus Torvalds
2009-05-07 19:37 ` Bevan Watkiss [this message]
2009-05-07 20:07 ` RE: Linus Torvalds
2009-05-07 20:20 ` RE: Linus Torvalds
2009-05-07 20:43 ` Junio C Hamano
2009-05-07 21:33 ` Re: Linus Torvalds
2009-05-07 21:55 ` Linus Torvalds
2009-05-07 22:27 ` RE: david
2009-05-07 22:36 ` RE: Linus Torvalds
2009-05-07 22:43 ` RE: david
2009-05-07 23:00 ` RE: Linus Torvalds
2009-05-07 23:07 ` RE: david
2009-05-07 23:18 ` RE: Linus Torvalds
2009-05-07 23:31 ` RE: david
2009-05-07 23:57 ` Johan Herland
2009-05-08 16:14 ` Bevan Watkiss
2009-05-08 8:17 ` Alex Riesen
2009-05-08 14:39 ` Re: Linus Torvalds
2009-05-08 15:51 ` Re: Brandon Casey
2009-05-08 16:15 ` Re: Linus Torvalds
2009-05-08 17:27 ` Re: Brandon Casey
2009-05-08 17:43 ` Re: Brandon Casey
2009-05-08 21:49 ` Re: Linus Torvalds
2009-05-08 23:04 ` Re: Brandon Casey
2009-05-09 16:44 ` Re: Linus Torvalds
2009-05-08 17:44 ` Re: Linus Torvalds
2009-05-08 16:47 ` 'git checkout' and unlink() calls (was: Re: ) Kjetil Barvik
2009-05-08 17:57 ` Linus Torvalds
[not found] <TXJgqLzlM6oCfTXKSqrSBk@txt.att.net>
2023-08-09 5:12 ` Luna Jernberg
[not found] <20220301070226.2477769-1-jaydeepjd.8914>
2022-03-06 11:10 ` Jaydeep P Das
2022-03-06 11:22 ` Jaydeep Das
-- strict thread matches above, loose matches on Subject: below --
2021-08-21 14:40 TECOB270_Ganesh Pawar
2021-08-21 23:52 ` Jeff King
2019-11-15 16:03 Martin Nicolay
2019-11-15 16:29 ` Martin Ågren
2019-11-15 16:37 ` Re: Martin Ågren
2019-08-20 17:23 William Baker
2019-08-20 17:27 ` Yagnatinsky, Mark
2019-03-05 14:57 [GSoC][PATCH v2 3/3] t3600: use helpers to replace test -d/f/e/s <path> Eric Sunshine
2019-03-05 23:38 ` Rohit Ashiwal
2019-01-23 10:50 Christopher Hagler
2019-01-23 14:16 ` Cody Kratzer
2019-01-23 14:25 ` Re: Thomas Braun
2019-01-23 16:00 ` Re: Christopher Hagler
2019-01-23 16:35 ` Randall S. Becker
2019-01-24 17:11 ` Johannes Schindelin
2018-10-08 13:33 Netravnen
2018-10-08 13:34 ` Inderpreet Saini
2018-04-27 0:54 [PATCH v3 2/3] merge: Add merge.renames config setting Ben Peart
2018-04-27 18:19 ` Elijah Newren
2018-04-30 13:11 ` Ben Peart
2018-04-30 16:12 ` Re: Elijah Newren
2018-05-02 14:33 ` Re: Ben Peart
2018-02-27 1:18 Alan Gage
2018-02-27 10:26 ` René Scharfe
2017-11-20 15:10 Viet Nguyen
2017-11-20 20:07 ` Stefan Beller
2017-11-12 2:21 hsed
2017-11-13 18:56 ` Stefan Beller
2017-01-25 0:11 [PATCH 7/7] completion: recognize more long-options Cornelius Weig
2017-01-25 0:21 ` Stefan Beller
2017-01-25 0:43 ` Cornelius Weig
2017-01-25 0:52 ` Re: Stefan Beller
2017-01-25 0:54 ` Re: Linus Torvalds
2017-01-25 1:32 ` Re: Eric Wong
2016-04-11 19:04 (unknown), miwilliams
2016-04-12 4:33 ` Stefan Beller
2015-08-19 19:41 Re: christain147
2015-08-19 11:09 Re: christain147
2015-08-05 12:47 (unknown) Ivan Chernyavsky
2015-08-15 9:19 ` Duy Nguyen
2015-08-17 17:49 ` Re: Junio C Hamano
2015-04-08 20:44 (unknown), Mamta Upadhyay
2015-04-08 21:58 ` Thomas Braun
2015-04-09 11:27 ` Re: Konstantin Khomoutov
[not found] <CANSxx61FaNp5SBXJ8Y+pWn0eDcunmibKR5g8rttnWGdGwEMHCA@mail.gmail.com>
2015-03-18 20:45 ` Re: Junio C Hamano
2015-03-18 21:06 ` Re: Stefan Beller
2015-03-18 21:17 ` Re: Jeff King
2015-03-18 21:28 ` Re: Jeff King
2015-03-18 21:33 ` Re: Junio C Hamano
2015-03-18 21:45 ` Re: Stefan Beller
2015-03-13 1:34 (unknown) cody.taylor
2015-03-13 2:00 ` Duy Nguyen
2014-09-08 11:36 (unknown), R. Klomp
[not found] ` <CAOqJoqGSRUw_UT4LhqpYX-WX6AEd2ReAWjgNS76Cra-SMKw3NQ@mail.gmail.com>
2014-09-08 14:36 ` R. Klomp
2014-09-10 0:00 ` Re: David Aguilar
2014-09-15 15:10 ` Re: R. Klomp
2014-02-06 11:54 "Sparse checkout leaves no entry on working directory" all the time on Windows 7 on Git 1.8.5.2.msysgit.0 konstunn
2014-02-06 13:20 ` Johannes Sixt
2014-02-06 19:56 ` Constantine Gorbunov
2012-06-12 21:12 (unknown), rohit sood
2012-06-12 23:51 ` Erik Faye-Lund
2009-11-18 5:03 Re: Anna
2009-05-11 18:57 (unknown) Don Slutz
2009-05-11 20:48 ` Johannes Schindelin
2009-05-12 12:45 ` Re: Don Slutz
2009-03-30 5:03 (unknown), David Aguilar
2009-03-30 7:02 ` Markus Heidelberg
2009-03-30 8:46 ` Re: Junio C Hamano
2007-11-01 20:44 (unknown), Francesco Pretto
2007-11-01 20:48 ` Francesco Pretto
2006-02-02 0:39 [RFC & PATCH] Solaris 8: ENOSYS when mkdir applied to automount., Jason Riedy
2006-02-02 4:18 ` H. Peter Anvin
2005-04-22 22:19 (unknown), atani
2005-04-22 23:16 ` Martin Schlemmer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=A07C3E66E84D46ACB37EDC7D396CCA62@caottdt504 \
--to=bevan.watkiss@cloakware.com \
--cc=git@vger.kernel.org \
--cc=raa.lkml@gmail.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).