[PATCH] preload-index: avoid lstat for skip-worktree items

[PATCH] preload-index: avoid lstat for skip-worktree items

[Johannes Schindelin]

From: Jeff Hostetler <jeffhost@microsoft.com>

Teach preload-index to avoid lstat() calls for index-entries
with skip-worktree bit set.  This is a performance optimization.

During a sparse-checkout, the skip-worktree bit is set on items
that were not populated and therefore are not present in the
worktree.  The per-thread preload-index loop performs a series
of tests on each index-entry as it attempts to compare the
worktree version with the index and mark them up-to-date.
This patch short-cuts that work.

On a Windows 10 system with a very large repo (450MB index)
and various levels of sparseness, performance was improved
in the {preloadindex=true, fscache=false} case by 80% and
in the {preloadindex=true, fscache=true} case by 20% for various
commands.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Published-As: https://github.com/dscho/git/releases/tag/preload-index-sparse-v1
Fetch-It-Via: git fetch https://github.com/dscho/git preload-index-sparse-v1

 preload-index.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/preload-index.c b/preload-index.c
index c1fe3a3ef9c..70a4c808783 100644
--- a/preload-index.c
+++ b/preload-index.c
@@ -53,6 +53,8 @@ static void *preload_thread(void *_data)
 			continue;
 		if (ce_uptodate(ce))
 			continue;
+		if (ce_skip_worktree(ce))
+			continue;
 		if (!ce_path_match(ce, &p->pathspec, NULL))
 			continue;
 		if (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))

base-commit: 6e3a7b3398559305c7a239a42e447c21a8f39ff8
-- 
2.11.1.windows.1

Re: [PATCH] preload-index: avoid lstat for skip-worktree items

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Teach preload-index to avoid lstat() calls for index-entries
> with skip-worktree bit set.  This is a performance optimization.
> ...
> diff --git a/preload-index.c b/preload-index.c
> index c1fe3a3ef9c..70a4c808783 100644
> --- a/preload-index.c
> +++ b/preload-index.c
> @@ -53,6 +53,8 @@ static void *preload_thread(void *_data)
>  			continue;
>  		if (ce_uptodate(ce))
>  			continue;
> +		if (ce_skip_worktree(ce))
> +			continue;
>  		if (!ce_path_match(ce, &p->pathspec, NULL))
>  			continue;
>  		if (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))

Because we are only interested in marking the ones that match
between the index and the working tree as "up-to-date", and we are
not doing the opposite (i.e. toggle "up-to-date" bit off by noticing
that things are now different) in this codepath, this change does
make sense.  The ones marked as "skip", even if there were an
unrelated file or directory at the path where the index expects a
regular file, can be safely ignored.

Thanks.