git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Marius Storm-Olsen <marius@trolltech.com>
To: Git Mailing List <git@vger.kernel.org>
Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>,
	Johannes Sixt <johannes.sixt@telecom.at>
Subject: [PATCH] Add a new lstat implementation based on Win32 API, and make stat use that implementation too.
Date: Sun, 02 Sep 2007 16:51:57 +0200	[thread overview]
Message-ID: <46DACE0D.5070501@trolltech.com> (raw)
In-Reply-To: <46DACD93.9000509@trolltech.com>

This gives us a significant speedup when adding, committing and stat'ing files.
(Also, since Windows doesn't really handle symlinks, it's fine that stat just uses lstat)

Signed-off-by: Marius Storm-Olsen <mstormo_git@storm-olsen.com>
---
The following patch will override the normal Posix implementation of stat and
lstat on Windows, and use normal Windows API to ensure we're stat'ing as fast
as possible. With this patch I get the performance increase to the far right.

Initially, I only replaced lstat, since that's what the MinGW port of Git had
implemented from before. But, since we don't really care about symlinks on
Windows, I decided to simply use the same implementation for stat as well. The
performance benefit is clearly indicated in the report below.


 With normal lstat & stat   lstat, based on Win32      lstat & stat, as Win32
 -------------------------  -------------------------  -------------------------
 Command: git init          Command: git init          Command: git init
 -------------------------  -------------------------  -------------------------

 real    0m0.047s           real      0m0.047s         real       0m0.078s
 user    0m0.031s           user      0m0.031s         user       0m0.031s
 sys     0m0.000s           sys       0m0.000s         sys        0m0.000s

 -------------------------  -------------------------  -------------------------
 Command: git add .         Command: git add .         Command: git add .
 -------------------------  -------------------------  -------------------------

 real    0m19.390s          real      0m19.390s        real       0m12.187s
 user    0m0.015s           user      0m0.015s         user       0m0.015s
 sys     0m0.030s           sys       0m0.030s         sys        0m0.015s

 -------------------------  -------------------------  -------------------------
 Command: git commit -a...  Command: git commit -a...  Command: git commit -a...
 -------------------------  -------------------------  -------------------------

 real    0m30.812s          real      0m22.547s        real       0m17.297s
 user    0m0.015s           user      0m0.031s         user       0m0.015s
 sys     0m0.000s           sys       0m0.000s         sys        0m0.015s

 -------------------------  -------------------------  -------------------------
 3x Command: git-status     3x Command: git-status     3x Command: git-status
 -------------------------  -------------------------  -------------------------

 real    0m11.860s          real      0m5.360s         real       0m5.344s
 user    0m0.015s           user      0m0.015s         user       0m0.015s
 sys     0m0.015s           sys       0m0.015s         sys        0m0.031s

 real    0m11.703s          real      0m5.312s         real       0m5.390s
 user    0m0.015s           user      0m0.015s         user       0m0.031s
 sys     0m0.000s           sys       0m0.000s         sys        0m0.000s

 real    0m11.672s          real      0m5.359s         real       0m5.344s
 user    0m0.031s           user      0m0.015s         user       0m0.015s
 sys     0m0.000s           sys       0m0.015s         sys        0m0.016s

 -------------------------  -------------------------  -------------------------
 Command: git commit...     Command: git commit...     Command: git commit...
 (single file)              (single file)              (single file)
 -------------------------  -------------------------  -------------------------

 real    0m14.234s          real      0m7.969s         real       0m7.875s
 user    0m0.015s           user      0m0.015s         user       0m0.015s
 sys     0m0.000s           sys       0m0.016s         sys        0m0.000s

 compat/mingw.c    |   52 ++++++++++++++++++++++++++++++++++++++++++++--------
 git-compat-util.h |    5 +++++
 2 files changed, 49 insertions(+), 8 deletions(-)

diff --git a/compat/mingw.c b/compat/mingw.c
index 7711a3f..207378c 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -23,19 +23,52 @@ int fchmod(int fildes, mode_t mode)
 	return -1;
 }

-int lstat(const char *file_name, struct stat *buf)
+static inline time_t filetime_to_time_t(const FILETIME *ft)
+{
+	long long winTime = ((long long)ft->dwHighDateTime << 32) + ft->dwLowDateTime;
+	winTime -= 116444736000000000LL; /* Windows to Unix Epoch conversion */
+	winTime /= 10000000;		 /* Nano to seconds resolution */
+	return (time_t)winTime;
+}
+
+extern int _getdrive( void );
+int git_lstat(const char *file_name, struct stat *buf)
 {
 	int namelen;
 	static char alt_name[PATH_MAX];
-
-	if (!stat(file_name, buf))
+	char* ext;
+	WIN32_FILE_ATTRIBUTE_DATA fdata;
+
+	if (GetFileAttributesExA(file_name, GetFileExInfoStandard, &fdata)) {
+		int fMode = S_IREAD;
+		if (fdata.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
+			fMode |= S_IFDIR;
+		else {
+			fMode |= S_IFREG;
+		ext = strrchr(file_name, '.');
+		if (ext && (!_stricmp(ext, ".exe") ||
+			    !_stricmp(ext, ".com") ||
+			    !_stricmp(ext, ".bat") ||
+			    !_stricmp(ext, ".cmd")))
+			fMode |= S_IEXEC;
+		}
+		if (!(fdata.dwFileAttributes & FILE_ATTRIBUTE_READONLY))
+			fMode |= S_IWRITE;
+
+		buf->st_ino = 0;
+		buf->st_gid = 0;
+		buf->st_uid = 0;
+		buf->st_nlink = 1;
+		buf->st_mode = fMode;
+		buf->st_size = fdata.nFileSizeLow; /* Can't use nFileSizeHigh, since it's not a stat64 */
+		buf->st_dev = buf->st_rdev = (_getdrive() - 1);
+		buf->st_atime = filetime_to_time_t(&(fdata.ftLastAccessTime));
+		buf->st_mtime = filetime_to_time_t(&(fdata.ftLastWriteTime));
+		buf->st_ctime = filetime_to_time_t(&(fdata.ftCreationTime));
 		return 0;
+	}
+	errno = ENOENT;

-	/* if file_name ended in a '/', Windows returned ENOENT;
-	 * try again without trailing slashes
-	 */
-	if (errno != ENOENT)
-		return -1;
 	namelen = strlen(file_name);
 	if (namelen && file_name[namelen-1] != '/')
 		return -1;
@@ -47,6 +80,9 @@ int lstat(const char *file_name, struct stat *buf)
 	alt_name[namelen] = 0;
 	return stat(alt_name, buf);
 }
+int git_stat(const char *file_name, struct stat *buf) {
+    return git_lstat(file_name, buf);
+}

 /* missing: link, mkstemp, fchmod, getuid (?), gettimeofday */
 int socketpair(int d, int type, int protocol, int sv[2])
diff --git a/git-compat-util.h b/git-compat-util.h
index 1ba499f..de1f062 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -488,6 +488,11 @@ int mingw_rename(const char*, const char*);
 extern void quote_argv(const char **dst, const char **src);
 extern const char *parse_interpreter(const char *cmd);

+/* Make git on Windows use git_lstat and git_stat instead of lstat and stat */
+int git_lstat(const char *file_name, struct stat *buf);
+int git_stat(const char *file_name, struct stat *buf);
+#define lstat(x,y) git_lstat(x,y)
+#define stat(x,y) git_stat(x,y)
 #endif /* __MINGW32__ */

 #endif
--
mingw.v1.5.2.4.1311.g376df-dirty

  reply	other threads:[~2007-09-02 14:52 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-09-02 14:49 Stats in Git Marius Storm-Olsen
2007-09-02 14:51 ` Marius Storm-Olsen [this message]
2007-09-02 14:57   ` [PATCH] Add a new lstat implementation based on Win32 API, and make stat use that implementation too Marius Storm-Olsen
2007-09-02 15:32   ` Reece Dunn
2007-09-02 16:09     ` Marius Storm-Olsen
2007-09-02 16:33       ` Reece Dunn
2007-09-02 16:47         ` Brian Gernhardt
2007-09-02 16:53           ` Reece Dunn
2007-09-02 17:05             ` Marius Storm-Olsen
2007-09-02 17:44               ` Johannes Schindelin
2007-09-02 17:58                 ` David Kastrup
2007-09-02 18:18                 ` Marius Storm-Olsen
2007-09-02 18:16   ` Johannes Sixt
2007-09-02 18:44     ` Marius Storm-Olsen
2007-09-02 19:07       ` Johannes Sixt
2007-09-02 19:31       ` Marius Storm-Olsen
2007-09-02 20:27         ` Robin Rosenberg
2007-09-02 21:26           ` Johannes Schindelin
2007-09-02 21:42             ` Robin Rosenberg
2007-09-02 23:02               ` Johannes Schindelin
2007-09-03  7:07               ` Johannes Sixt
2007-09-03 11:21                 ` Miklos Vajna
2007-09-03 11:32                   ` David Kastrup
2007-09-05 16:02                     ` Miklos Vajna
2007-09-05 19:01                       ` David Kastrup
2007-09-06 16:26                         ` Miklos Vajna
2007-09-06 16:33                           ` David Kastrup
2007-09-06 23:31                             ` Douglas Stockwell
2007-09-07  6:36                               ` David Kastrup
2007-09-02 21:38           ` Alex Riesen
2007-09-02 22:04             ` Robin Rosenberg
2007-09-03  6:15           ` Marius Storm-Olsen
2007-09-03 11:39             ` Johannes Schindelin
2007-09-03 11:51               ` David Kastrup
2007-09-03 11:53               ` Marius Storm-Olsen
2007-09-03 12:33                 ` Johannes Schindelin
2007-09-02 21:41         ` Alex Riesen
2007-09-03  6:12           ` Marius Storm-Olsen
2007-09-03  7:47   ` Johannes Sixt
2007-09-03  7:55     ` Marius Storm-Olsen
     [not found]     ` <46DBFA2A.7050003@trolltech.com>
2007-09-03 12:38       ` [PATCH] Add a new lstat and fstat implementation based on Win32 API Marius Storm-Olsen
2007-09-03 13:33       ` Johannes Schindelin
2007-09-03 13:52         ` Marius Storm-Olsen
2007-09-03 14:39           ` Johannes Schindelin
2007-09-03 16:22             ` Marius Storm-Olsen
2007-09-03 16:56               ` Johannes Schindelin
2007-09-03 13:53         ` Johannes Sixt
2007-09-03 14:35           ` Johannes Schindelin
2007-09-03 19:21         ` Marius Storm-Olsen
2007-09-04  2:21           ` Johannes Schindelin
2007-09-04  7:41           ` Johannes Sixt
2007-09-04  8:53             ` David Kastrup
2007-09-04 10:20             ` Marius Storm-Olsen
2007-09-04 10:53               ` Johannes Sixt
2007-09-04 11:21                 ` Marius Storm-Olsen
2007-09-04 11:28                   ` Johannes Sixt
2007-09-04 21:31                 ` David Kastrup
2007-09-04 10:48             ` Johannes Schindelin
2007-09-04 11:36               ` Johannes Sixt
2007-09-04 11:53                 ` Marius Storm-Olsen
2007-09-04 13:56                   ` Marius Storm-Olsen
2007-09-04 14:07                     ` Johannes Sixt
2007-09-04 14:32                       ` Johannes Schindelin
2007-09-04 14:52                         ` Johannes Sixt
2007-09-04 14:16                     ` Johannes Schindelin
2007-09-04 14:30                     ` Johannes Schindelin
2007-09-04 14:43                       ` Marius Storm-Olsen
2007-09-04 14:48                         ` Johannes Schindelin
2007-09-04 15:05                           ` David Kastrup
2007-09-04 16:32                           ` Marius Storm-Olsen
2007-09-04 12:46                 ` Johannes Schindelin
2007-09-04 12:57                   ` Johannes Schindelin
2007-09-04 21:02                     ` Rutger Nijlunsing
2007-09-04 21:54                       ` Reece Dunn
2007-09-05  6:22                       ` Marius Storm-Olsen
2007-09-05 10:15                         ` Johannes Schindelin
2007-09-04 13:03                   ` Johannes Sixt
2007-09-06 16:18             ` Johannes Sixt
2007-09-06 16:34               ` Marius Storm-Olsen
2007-09-03 13:49       ` Johannes Sixt
2007-09-03 14:38         ` Johannes Schindelin
2007-09-03 16:15           ` Marius Storm-Olsen
2007-09-03 16:55             ` Johannes Schindelin
2007-09-02 20:02 ` Stats in Git Alex Riesen
2007-09-02 20:09   ` Marius Storm-Olsen
2007-09-03  8:19 ` Matthieu Moy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=46DACE0D.5070501@trolltech.com \
    --to=marius@trolltech.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=johannes.sixt@telecom.at \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).