git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Marius Storm-Olsen <marius@trolltech.com>
To: Git Mailing List <git@vger.kernel.org>
Subject: Stats in Git
Date: Sun, 02 Sep 2007 16:49:55 +0200	[thread overview]
Message-ID: <46DACD93.9000509@trolltech.com> (raw)

I was checking out the performance situation with Git on Windows, and
found out that the Posix stat functions on Windows are just obscenely
slow. We really can't use them, at least on in Git. So, I made a patch
for the MinGW version, which I'll post right after this mail.

However, while look at that whole stat'ing situation in git, I saw
that doing 'git status' actually stats all the files _thrice_!
Yup, that's not 1 time, or 2 times, but actually 3(!) times before
'git status' is content!
I know that git-status is a script, so I think this clearly indicates
that git-status is a prime candidate for a built-in ;-)

I haven't looked into details as to why it stats the files so many
times. I guess someone more experienced in Git core could give an
opinion, if by writing git-status as a builtin it would be possible to
only stat the files once. It would have a huge impact on Windows where
stats are inheritly much slower than on Linux.

By applying the diff below, you can see for yourself what happens when
you stat the repo created with Moe's script:
    mkdir bummer
    cd bummer
    for ((i=0;i<100;i++)); do
    mkdir $i && pushd $i;
    for ((j=0;j<1000;j++)); do
    echo "$j" >$j; done; popd;
    done

$ git status 2>&1 | wc -l
300137

Fast on Linux now, but still quite slow on Windows..

--
.marius


diff --git a/git-compat-util.h b/git-compat-util.h
index ca0a597..6b6405c 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -369,4 +369,23 @@ static inline int strtoul_ui(char const *s, int base, unsigned int *result)
 	return 0;
 }

+static inline int git_lstat(const char *file_name, struct stat *buf)
+{
+	fprintf(stderr, "lstat: %s\n", file_name);
+	return lstat(file_name, buf);
+}
+static inline int git_fstat(int fd, struct stat *buf)
+{
+	fprintf(stderr, "fstat: %d\n", fd);
+	return fstat(fd, buf);
+}
+static inline int git_stat(const char *file_name, struct stat *buf)
+{
+	fprintf(stderr, "stat: %s\n", file_name);
+	return stat(file_name, buf);
+}
+#define lstat(x,y) git_lstat(x,y)
+#define fstat(x,y) git_fstat(x,y)
+#define stat(x,y) git_stat(x,y)
+
 #endif

             reply	other threads:[~2007-09-02 14:50 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-09-02 14:49 Marius Storm-Olsen [this message]
2007-09-02 14:51 ` [PATCH] Add a new lstat implementation based on Win32 API, and make stat use that implementation too Marius Storm-Olsen
2007-09-02 14:57   ` Marius Storm-Olsen
2007-09-02 15:32   ` Reece Dunn
2007-09-02 16:09     ` Marius Storm-Olsen
2007-09-02 16:33       ` Reece Dunn
2007-09-02 16:47         ` Brian Gernhardt
2007-09-02 16:53           ` Reece Dunn
2007-09-02 17:05             ` Marius Storm-Olsen
2007-09-02 17:44               ` Johannes Schindelin
2007-09-02 17:58                 ` David Kastrup
2007-09-02 18:18                 ` Marius Storm-Olsen
2007-09-02 18:16   ` Johannes Sixt
2007-09-02 18:44     ` Marius Storm-Olsen
2007-09-02 19:07       ` Johannes Sixt
2007-09-02 19:31       ` Marius Storm-Olsen
2007-09-02 20:27         ` Robin Rosenberg
2007-09-02 21:26           ` Johannes Schindelin
2007-09-02 21:42             ` Robin Rosenberg
2007-09-02 23:02               ` Johannes Schindelin
2007-09-03  7:07               ` Johannes Sixt
2007-09-03 11:21                 ` Miklos Vajna
2007-09-03 11:32                   ` David Kastrup
2007-09-05 16:02                     ` Miklos Vajna
2007-09-05 19:01                       ` David Kastrup
2007-09-06 16:26                         ` Miklos Vajna
2007-09-06 16:33                           ` David Kastrup
2007-09-06 23:31                             ` Douglas Stockwell
2007-09-07  6:36                               ` David Kastrup
2007-09-02 21:38           ` Alex Riesen
2007-09-02 22:04             ` Robin Rosenberg
2007-09-03  6:15           ` Marius Storm-Olsen
2007-09-03 11:39             ` Johannes Schindelin
2007-09-03 11:51               ` David Kastrup
2007-09-03 11:53               ` Marius Storm-Olsen
2007-09-03 12:33                 ` Johannes Schindelin
2007-09-02 21:41         ` Alex Riesen
2007-09-03  6:12           ` Marius Storm-Olsen
2007-09-03  7:47   ` Johannes Sixt
2007-09-03  7:55     ` Marius Storm-Olsen
     [not found]     ` <46DBFA2A.7050003@trolltech.com>
2007-09-03 12:38       ` [PATCH] Add a new lstat and fstat implementation based on Win32 API Marius Storm-Olsen
2007-09-03 13:33       ` Johannes Schindelin
2007-09-03 13:52         ` Marius Storm-Olsen
2007-09-03 14:39           ` Johannes Schindelin
2007-09-03 16:22             ` Marius Storm-Olsen
2007-09-03 16:56               ` Johannes Schindelin
2007-09-03 13:53         ` Johannes Sixt
2007-09-03 14:35           ` Johannes Schindelin
2007-09-03 19:21         ` Marius Storm-Olsen
2007-09-04  2:21           ` Johannes Schindelin
2007-09-04  7:41           ` Johannes Sixt
2007-09-04  8:53             ` David Kastrup
2007-09-04 10:20             ` Marius Storm-Olsen
2007-09-04 10:53               ` Johannes Sixt
2007-09-04 11:21                 ` Marius Storm-Olsen
2007-09-04 11:28                   ` Johannes Sixt
2007-09-04 21:31                 ` David Kastrup
2007-09-04 10:48             ` Johannes Schindelin
2007-09-04 11:36               ` Johannes Sixt
2007-09-04 11:53                 ` Marius Storm-Olsen
2007-09-04 13:56                   ` Marius Storm-Olsen
2007-09-04 14:07                     ` Johannes Sixt
2007-09-04 14:32                       ` Johannes Schindelin
2007-09-04 14:52                         ` Johannes Sixt
2007-09-04 14:16                     ` Johannes Schindelin
2007-09-04 14:30                     ` Johannes Schindelin
2007-09-04 14:43                       ` Marius Storm-Olsen
2007-09-04 14:48                         ` Johannes Schindelin
2007-09-04 15:05                           ` David Kastrup
2007-09-04 16:32                           ` Marius Storm-Olsen
2007-09-04 12:46                 ` Johannes Schindelin
2007-09-04 12:57                   ` Johannes Schindelin
2007-09-04 21:02                     ` Rutger Nijlunsing
2007-09-04 21:54                       ` Reece Dunn
2007-09-05  6:22                       ` Marius Storm-Olsen
2007-09-05 10:15                         ` Johannes Schindelin
2007-09-04 13:03                   ` Johannes Sixt
2007-09-06 16:18             ` Johannes Sixt
2007-09-06 16:34               ` Marius Storm-Olsen
2007-09-03 13:49       ` Johannes Sixt
2007-09-03 14:38         ` Johannes Schindelin
2007-09-03 16:15           ` Marius Storm-Olsen
2007-09-03 16:55             ` Johannes Schindelin
2007-09-02 20:02 ` Stats in Git Alex Riesen
2007-09-02 20:09   ` Marius Storm-Olsen
2007-09-03  8:19 ` Matthieu Moy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=46DACD93.9000509@trolltech.com \
    --to=marius@trolltech.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).