[RFC 00/14] convert dir.c to take an index parameter

[RFC 00/14] convert dir.c to take an index parameter

[Brandon Williams]

One of the things brought up on the list in the past few days has been
migrating away from using the index compatibility macros.  One of the issues
brought up in that thread was how simply doing that conversion doesn't
eliminate the reliance on global state (specifically the_index).  If one day we
want to have a 'repository object' passed around then we first need to convert
different subsystems to be prepared to handle that.  This series provides a
first step, converting the code in dir.c to take a 'struct index_state' and
using that instead of implicitly using 'the_index'.

Brandon Williams (14):
  dir: stop using the index compatibility macros
  dir: convert read_skip_worktree_file_from_index to take an index
  dir: convert directory_exists_in_index to take index
  dir: convert get_dtype to take index
  dir: convert dir_add* to take an index
  dir: convert last_exclude_matching_from_list to take an index
  dir: convert is_excluded_from_list to take an index
  dir: convert add_excludes to take an index
  dir: convert prep_exclude to take an index
  dir: convert is_excluded to take an index
  dir: convert open_cached_dir to take an index
  dir: convert read_directory_recursive to take an index
  dir: convert read_directory to take an index
  dir: convert fill_directory to take an index

 builtin/add.c          |   7 +-
 builtin/check-ignore.c |   3 +-
 builtin/clean.c        |   4 +-
 builtin/grep.c         |   2 +-
 builtin/ls-files.c     |   4 +-
 dir.c                  | 200 ++++++++++++++++++++++++++++---------------------
 dir.h                  |  27 +++++--
 unpack-trees.c         |  10 +--
 wt-status.c            |   2 +-
 9 files changed, 151 insertions(+), 108 deletions(-)

-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 01/14] dir: stop using the index compatibility macros

[Brandon Williams]

In order to make it clearer where the_index is being referenced, stop
using the index compatibility macros in dir.c.  This is to make it
easier to identify the functions which need to be convert to taking in a
'struct index_state' as a parameter.

The end goal would be to eliminate the need to reference global index
state in dir.c.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 41 +++++++++++++++++++++--------------------
 1 file changed, 21 insertions(+), 20 deletions(-)

diff --git a/dir.c b/dir.c
index f451bfa48..97eadd45e 100644
--- a/dir.c
+++ b/dir.c
@@ -7,6 +7,7 @@
  * Copyright (C) Linus Torvalds, 2005-2006
  *		 Junio Hamano, 2005-2006
  */
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "dir.h"
 #include "attr.h"
@@ -596,12 +597,12 @@ static void *read_skip_worktree_file_from_index(const char *path, size_t *size,
 	void *data;
 
 	len = strlen(path);
-	pos = cache_name_pos(path, len);
+	pos = index_name_pos(&the_index, path, len);
 	if (pos < 0)
 		return NULL;
-	if (!ce_skip_worktree(active_cache[pos]))
+	if (!ce_skip_worktree(the_index.cache[pos]))
 		return NULL;
-	data = read_sha1_file(active_cache[pos]->oid.hash, &type, &sz);
+	data = read_sha1_file(the_index.cache[pos]->oid.hash, &type, &sz);
 	if (!data || type != OBJ_BLOB) {
 		free(data);
 		return NULL;
@@ -609,7 +610,7 @@ static void *read_skip_worktree_file_from_index(const char *path, size_t *size,
 	*size = xsize_t(sz);
 	if (sha1_stat) {
 		memset(&sha1_stat->stat, 0, sizeof(sha1_stat->stat));
-		hashcpy(sha1_stat->sha1, active_cache[pos]->oid.hash);
+		hashcpy(sha1_stat->sha1, the_index.cache[pos]->oid.hash);
 	}
 	return data;
 }
@@ -785,12 +786,12 @@ static int add_excludes(const char *fname, const char *base, int baselen,
 			    !match_stat_data_racy(&the_index, &sha1_stat->stat, &st))
 				; /* no content change, ss->sha1 still good */
 			else if (check_index &&
-				 (pos = cache_name_pos(fname, strlen(fname))) >= 0 &&
-				 !ce_stage(active_cache[pos]) &&
-				 ce_uptodate(active_cache[pos]) &&
+				 (pos = index_name_pos(&the_index, fname, strlen(fname))) >= 0 &&
+				 !ce_stage(the_index.cache[pos]) &&
+				 ce_uptodate(the_index.cache[pos]) &&
 				 !would_convert_to_git(fname))
 				hashcpy(sha1_stat->sha1,
-					active_cache[pos]->oid.hash);
+					the_index.cache[pos]->oid.hash);
 			else
 				hash_sha1_file(buf, size, "blob", sha1_stat->sha1);
 			fill_stat_data(&sha1_stat->stat, &st);
@@ -1235,7 +1236,7 @@ static struct dir_entry *dir_entry_new(const char *pathname, int len)
 
 static struct dir_entry *dir_add_name(struct dir_struct *dir, const char *pathname, int len)
 {
-	if (cache_file_exists(pathname, len, ignore_case))
+	if (index_file_exists(&the_index, pathname, len, ignore_case))
 		return NULL;
 
 	ALLOC_GROW(dir->entries, dir->nr+1, dir->alloc);
@@ -1244,7 +1245,7 @@ static struct dir_entry *dir_add_name(struct dir_struct *dir, const char *pathna
 
 struct dir_entry *dir_add_ignored(struct dir_struct *dir, const char *pathname, int len)
 {
-	if (!cache_name_is_other(pathname, len))
+	if (!index_name_is_other(&the_index, pathname, len))
 		return NULL;
 
 	ALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->ignored_alloc);
@@ -1266,10 +1267,10 @@ static enum exist_status directory_exists_in_index_icase(const char *dirname, in
 {
 	struct cache_entry *ce;
 
-	if (cache_dir_exists(dirname, len))
+	if (index_dir_exists(&the_index, dirname, len))
 		return index_directory;
 
-	ce = cache_file_exists(dirname, len, ignore_case);
+	ce = index_file_exists(&the_index, dirname, len, ignore_case);
 	if (ce && S_ISGITLINK(ce->ce_mode))
 		return index_gitdir;
 
@@ -1290,11 +1291,11 @@ static enum exist_status directory_exists_in_index(const char *dirname, int len)
 	if (ignore_case)
 		return directory_exists_in_index_icase(dirname, len);
 
-	pos = cache_name_pos(dirname, len);
+	pos = index_name_pos(&the_index, dirname, len);
 	if (pos < 0)
 		pos = -pos-1;
-	while (pos < active_nr) {
-		const struct cache_entry *ce = active_cache[pos++];
+	while (pos < the_index.cache_nr) {
+		const struct cache_entry *ce = the_index.cache[pos++];
 		unsigned char endchar;
 
 		if (strncmp(ce->name, dirname, len))
@@ -1460,7 +1461,7 @@ static int get_index_dtype(const char *path, int len)
 	int pos;
 	const struct cache_entry *ce;
 
-	ce = cache_file_exists(path, len, 0);
+	ce = index_file_exists(&the_index, path, len, 0);
 	if (ce) {
 		if (!ce_uptodate(ce))
 			return DT_UNKNOWN;
@@ -1474,12 +1475,12 @@ static int get_index_dtype(const char *path, int len)
 	}
 
 	/* Try to look it up as a directory */
-	pos = cache_name_pos(path, len);
+	pos = index_name_pos(&the_index, path, len);
 	if (pos >= 0)
 		return DT_UNKNOWN;
 	pos = -pos-1;
-	while (pos < active_nr) {
-		ce = active_cache[pos++];
+	while (pos < the_index.cache_nr) {
+		ce = the_index.cache[pos++];
 		if (strncmp(ce->name, path, len))
 			break;
 		if (ce->name[len] > '/')
@@ -1522,7 +1523,7 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 					  int dtype, struct dirent *de)
 {
 	int exclude;
-	int has_path_in_index = !!cache_file_exists(path->buf, path->len, ignore_case);
+	int has_path_in_index = !!index_file_exists(&the_index, path->buf, path->len, ignore_case);
 
 	if (dtype == DT_UNKNOWN)
 		dtype = get_dtype(de, path->buf, path->len);
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 02/14] dir: convert read_skip_worktree_file_from_index to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/dir.c b/dir.c
index 97eadd45e..44307b963 100644
--- a/dir.c
+++ b/dir.c
@@ -588,7 +588,8 @@ void add_exclude(const char *string, const char *base,
 	x->el = el;
 }
 
-static void *read_skip_worktree_file_from_index(const char *path, size_t *size,
+static void *read_skip_worktree_file_from_index(const struct index_state *istate,
+						const char *path, size_t *size,
 						struct sha1_stat *sha1_stat)
 {
 	int pos, len;
@@ -597,12 +598,12 @@ static void *read_skip_worktree_file_from_index(const char *path, size_t *size,
 	void *data;
 
 	len = strlen(path);
-	pos = index_name_pos(&the_index, path, len);
+	pos = index_name_pos(istate, path, len);
 	if (pos < 0)
 		return NULL;
-	if (!ce_skip_worktree(the_index.cache[pos]))
+	if (!ce_skip_worktree(istate->cache[pos]))
 		return NULL;
-	data = read_sha1_file(the_index.cache[pos]->oid.hash, &type, &sz);
+	data = read_sha1_file(istate->cache[pos]->oid.hash, &type, &sz);
 	if (!data || type != OBJ_BLOB) {
 		free(data);
 		return NULL;
@@ -610,7 +611,7 @@ static void *read_skip_worktree_file_from_index(const char *path, size_t *size,
 	*size = xsize_t(sz);
 	if (sha1_stat) {
 		memset(&sha1_stat->stat, 0, sizeof(sha1_stat->stat));
-		hashcpy(sha1_stat->sha1, the_index.cache[pos]->oid.hash);
+		hashcpy(sha1_stat->sha1, istate->cache[pos]->oid.hash);
 	}
 	return data;
 }
@@ -751,7 +752,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,
 		if (0 <= fd)
 			close(fd);
 		if (!check_index ||
-		    (buf = read_skip_worktree_file_from_index(fname, &size, sha1_stat)) == NULL)
+		    (buf = read_skip_worktree_file_from_index(&the_index, fname, &size, sha1_stat)) == NULL)
 			return -1;
 		if (size == 0) {
 			free(buf);
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 03/14] dir: convert directory_exists_in_index to take index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/dir.c b/dir.c
index 44307b963..5c635198d 100644
--- a/dir.c
+++ b/dir.c
@@ -1264,14 +1264,15 @@ enum exist_status {
  * the directory name; instead, use the case insensitive
  * directory hash.
  */
-static enum exist_status directory_exists_in_index_icase(const char *dirname, int len)
+static enum exist_status directory_exists_in_index_icase(struct index_state *istate,
+							 const char *dirname, int len)
 {
 	struct cache_entry *ce;
 
-	if (index_dir_exists(&the_index, dirname, len))
+	if (index_dir_exists(istate, dirname, len))
 		return index_directory;
 
-	ce = index_file_exists(&the_index, dirname, len, ignore_case);
+	ce = index_file_exists(istate, dirname, len, ignore_case);
 	if (ce && S_ISGITLINK(ce->ce_mode))
 		return index_gitdir;
 
@@ -1285,18 +1286,19 @@ static enum exist_status directory_exists_in_index_icase(const char *dirname, in
  * the files it contains) will sort with the '/' at the
  * end.
  */
-static enum exist_status directory_exists_in_index(const char *dirname, int len)
+static enum exist_status directory_exists_in_index(struct index_state *istate,
+						   const char *dirname, int len)
 {
 	int pos;
 
 	if (ignore_case)
-		return directory_exists_in_index_icase(dirname, len);
+		return directory_exists_in_index_icase(istate, dirname, len);
 
-	pos = index_name_pos(&the_index, dirname, len);
+	pos = index_name_pos(istate, dirname, len);
 	if (pos < 0)
 		pos = -pos-1;
-	while (pos < the_index.cache_nr) {
-		const struct cache_entry *ce = the_index.cache[pos++];
+	while (pos < istate->cache_nr) {
+		const struct cache_entry *ce = istate->cache[pos++];
 		unsigned char endchar;
 
 		if (strncmp(ce->name, dirname, len))
@@ -1351,7 +1353,7 @@ static enum path_treatment treat_directory(struct dir_struct *dir,
 	const struct pathspec *pathspec)
 {
 	/* The "len-1" is to strip the final '/' */
-	switch (directory_exists_in_index(dirname, len-1)) {
+	switch (directory_exists_in_index(&the_index, dirname, len-1)) {
 	case index_directory:
 		return path_recurse;
 
@@ -1554,7 +1556,7 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	if ((dir->flags & DIR_COLLECT_KILLED_ONLY) &&
 	    (dtype == DT_DIR) &&
 	    !has_path_in_index &&
-	    (directory_exists_in_index(path->buf, path->len) == index_nonexistent))
+	    (directory_exists_in_index(&the_index, path->buf, path->len) == index_nonexistent))
 		return path_none;
 
 	exclude = is_excluded(dir, path->buf, &dtype);
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 04/14] dir: convert get_dtype to take index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/dir.c b/dir.c
index 5c635198d..4515f0083 100644
--- a/dir.c
+++ b/dir.c
@@ -48,7 +48,8 @@ struct cached_dir {
 static enum path_treatment read_directory_recursive(struct dir_struct *dir,
 	const char *path, int len, struct untracked_cache_dir *untracked,
 	int check_only, const struct pathspec *pathspec);
-static int get_dtype(struct dirent *de, const char *path, int len);
+static int get_dtype(struct dirent *de, struct index_state *istate,
+		     const char *path, int len);
 
 int fspathcmp(const char *a, const char *b)
 {
@@ -975,7 +976,7 @@ static struct exclude *last_exclude_matching_from_list(const char *pathname,
 
 		if (x->flags & EXC_FLAG_MUSTBEDIR) {
 			if (*dtype == DT_UNKNOWN)
-				*dtype = get_dtype(NULL, pathname, pathlen);
+				*dtype = get_dtype(NULL, &the_index, pathname, pathlen);
 			if (*dtype != DT_DIR)
 				continue;
 		}
@@ -1459,12 +1460,13 @@ static int exclude_matches_pathspec(const char *path, int pathlen,
 	return 0;
 }
 
-static int get_index_dtype(const char *path, int len)
+static int get_index_dtype(struct index_state *istate,
+			   const char *path, int len)
 {
 	int pos;
 	const struct cache_entry *ce;
 
-	ce = index_file_exists(&the_index, path, len, 0);
+	ce = index_file_exists(istate, path, len, 0);
 	if (ce) {
 		if (!ce_uptodate(ce))
 			return DT_UNKNOWN;
@@ -1478,12 +1480,12 @@ static int get_index_dtype(const char *path, int len)
 	}
 
 	/* Try to look it up as a directory */
-	pos = index_name_pos(&the_index, path, len);
+	pos = index_name_pos(istate, path, len);
 	if (pos >= 0)
 		return DT_UNKNOWN;
 	pos = -pos-1;
-	while (pos < the_index.cache_nr) {
-		ce = the_index.cache[pos++];
+	while (pos < istate->cache_nr) {
+		ce = istate->cache[pos++];
 		if (strncmp(ce->name, path, len))
 			break;
 		if (ce->name[len] > '/')
@@ -1497,14 +1499,15 @@ static int get_index_dtype(const char *path, int len)
 	return DT_UNKNOWN;
 }
 
-static int get_dtype(struct dirent *de, const char *path, int len)
+static int get_dtype(struct dirent *de, struct index_state *istate,
+		     const char *path, int len)
 {
 	int dtype = de ? DTYPE(de) : DT_UNKNOWN;
 	struct stat st;
 
 	if (dtype != DT_UNKNOWN)
 		return dtype;
-	dtype = get_index_dtype(path, len);
+	dtype = get_index_dtype(istate, path, len);
 	if (dtype != DT_UNKNOWN)
 		return dtype;
 	if (lstat(path, &st))
@@ -1529,7 +1532,7 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	int has_path_in_index = !!index_file_exists(&the_index, path->buf, path->len, ignore_case);
 
 	if (dtype == DT_UNKNOWN)
-		dtype = get_dtype(de, path->buf, path->len);
+		dtype = get_dtype(de, &the_index, path->buf, path->len);
 
 	/* Always exclude indexed files */
 	if (dtype != DT_DIR && has_path_in_index)
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 05/14] dir: convert dir_add* to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/add.c |  3 ++-
 dir.c         | 18 +++++++++++-------
 dir.h         |  4 +++-
 3 files changed, 16 insertions(+), 9 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index 9f53f020d..bf5e676e4 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -437,7 +437,8 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 				if (ignore_missing) {
 					int dtype = DT_UNKNOWN;
 					if (is_excluded(&dir, path, &dtype))
-						dir_add_ignored(&dir, path, pathspec.items[i].len);
+						dir_add_ignored(&dir, &the_index,
+								path, pathspec.items[i].len);
 				} else
 					die(_("pathspec '%s' did not match any files"),
 					    pathspec.items[i].original);
diff --git a/dir.c b/dir.c
index 4515f0083..a508e8076 100644
--- a/dir.c
+++ b/dir.c
@@ -1236,18 +1236,22 @@ static struct dir_entry *dir_entry_new(const char *pathname, int len)
 	return ent;
 }
 
-static struct dir_entry *dir_add_name(struct dir_struct *dir, const char *pathname, int len)
+static struct dir_entry *dir_add_name(struct dir_struct *dir,
+				      struct index_state *istate,
+				      const char *pathname, int len)
 {
-	if (index_file_exists(&the_index, pathname, len, ignore_case))
+	if (index_file_exists(istate, pathname, len, ignore_case))
 		return NULL;
 
 	ALLOC_GROW(dir->entries, dir->nr+1, dir->alloc);
 	return dir->entries[dir->nr++] = dir_entry_new(pathname, len);
 }
 
-struct dir_entry *dir_add_ignored(struct dir_struct *dir, const char *pathname, int len)
+struct dir_entry *dir_add_ignored(struct dir_struct *dir,
+				  struct index_state *istate,
+				  const char *pathname, int len)
 {
-	if (!index_name_is_other(&the_index, pathname, len))
+	if (!index_name_is_other(istate, pathname, len))
 		return NULL;
 
 	ALLOC_GROW(dir->ignored, dir->ignored_nr+1, dir->ignored_alloc);
@@ -1819,18 +1823,18 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,
 		switch (state) {
 		case path_excluded:
 			if (dir->flags & DIR_SHOW_IGNORED)
-				dir_add_name(dir, path.buf, path.len);
+				dir_add_name(dir, &the_index, path.buf, path.len);
 			else if ((dir->flags & DIR_SHOW_IGNORED_TOO) ||
 				((dir->flags & DIR_COLLECT_IGNORED) &&
 				exclude_matches_pathspec(path.buf, path.len,
 							 pathspec)))
-				dir_add_ignored(dir, path.buf, path.len);
+				dir_add_ignored(dir, &the_index, path.buf, path.len);
 			break;
 
 		case path_untracked:
 			if (dir->flags & DIR_SHOW_IGNORED)
 				break;
-			dir_add_name(dir, path.buf, path.len);
+			dir_add_name(dir, &the_index, path.buf, path.len);
 			if (cdir.fdir)
 				add_untracked(untracked, path.buf + baselen);
 			break;
diff --git a/dir.h b/dir.h
index bf23a470a..a9f809983 100644
--- a/dir.h
+++ b/dir.h
@@ -219,7 +219,9 @@ extern int read_directory(struct dir_struct *, const char *path, int len, const
 
 extern int is_excluded_from_list(const char *pathname, int pathlen, const char *basename,
 				 int *dtype, struct exclude_list *el);
-struct dir_entry *dir_add_ignored(struct dir_struct *dir, const char *pathname, int len);
+struct dir_entry *dir_add_ignored(struct dir_struct *dir,
+				  struct index_state *istate,
+				  const char *pathname, int len);
 
 /*
  * these implement the matching logic for dir.c:excluded_from_list and
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 06/14] dir: convert last_exclude_matching_from_list to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/dir.c b/dir.c
index a508e8076..077b756c2 100644
--- a/dir.c
+++ b/dir.c
@@ -961,7 +961,8 @@ static struct exclude *last_exclude_matching_from_list(const char *pathname,
 						       int pathlen,
 						       const char *basename,
 						       int *dtype,
-						       struct exclude_list *el)
+						       struct exclude_list *el,
+						       struct index_state *istate)
 {
 	struct exclude *exc = NULL; /* undecided */
 	int i;
@@ -976,7 +977,7 @@ static struct exclude *last_exclude_matching_from_list(const char *pathname,
 
 		if (x->flags & EXC_FLAG_MUSTBEDIR) {
 			if (*dtype == DT_UNKNOWN)
-				*dtype = get_dtype(NULL, &the_index, pathname, pathlen);
+				*dtype = get_dtype(NULL, istate, pathname, pathlen);
 			if (*dtype != DT_DIR)
 				continue;
 		}
@@ -1012,13 +1013,14 @@ int is_excluded_from_list(const char *pathname,
 			  struct exclude_list *el)
 {
 	struct exclude *exclude;
-	exclude = last_exclude_matching_from_list(pathname, pathlen, basename, dtype, el);
+	exclude = last_exclude_matching_from_list(pathname, pathlen, basename, dtype, el, &the_index);
 	if (exclude)
 		return exclude->flags & EXC_FLAG_NEGATIVE ? 0 : 1;
 	return -1; /* undecided */
 }
 
 static struct exclude *last_exclude_matching_from_lists(struct dir_struct *dir,
+							struct index_state *istate,
 		const char *pathname, int pathlen, const char *basename,
 		int *dtype_p)
 {
@@ -1030,7 +1032,7 @@ static struct exclude *last_exclude_matching_from_lists(struct dir_struct *dir,
 		for (j = group->nr - 1; j >= 0; j--) {
 			exclude = last_exclude_matching_from_list(
 				pathname, pathlen, basename, dtype_p,
-				&group->el[j]);
+				&group->el[j], istate);
 			if (exclude)
 				return exclude;
 		}
@@ -1121,6 +1123,7 @@ static void prep_exclude(struct dir_struct *dir, const char *base, int baselen)
 			int dt = DT_DIR;
 			dir->basebuf.buf[stk->baselen - 1] = 0;
 			dir->exclude = last_exclude_matching_from_lists(dir,
+									&the_index,
 				dir->basebuf.buf, stk->baselen - 1,
 				dir->basebuf.buf + current, &dt);
 			dir->basebuf.buf[stk->baselen - 1] = '/';
@@ -1209,7 +1212,7 @@ struct exclude *last_exclude_matching(struct dir_struct *dir,
 	if (dir->exclude)
 		return dir->exclude;
 
-	return last_exclude_matching_from_lists(dir, pathname, pathlen,
+	return last_exclude_matching_from_lists(dir, &the_index, pathname, pathlen,
 			basename, dtype_p);
 }
 
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 07/14] dir: convert is_excluded_from_list to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c          | 5 +++--
 dir.h          | 6 ++++--
 unpack-trees.c | 4 ++--
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/dir.c b/dir.c
index 077b756c2..497a2db85 100644
--- a/dir.c
+++ b/dir.c
@@ -1010,10 +1010,11 @@ static struct exclude *last_exclude_matching_from_list(const char *pathname,
  */
 int is_excluded_from_list(const char *pathname,
 			  int pathlen, const char *basename, int *dtype,
-			  struct exclude_list *el)
+			  struct exclude_list *el, struct index_state *istate)
 {
 	struct exclude *exclude;
-	exclude = last_exclude_matching_from_list(pathname, pathlen, basename, dtype, el, &the_index);
+	exclude = last_exclude_matching_from_list(pathname, pathlen, basename,
+						  dtype, el, istate);
 	if (exclude)
 		return exclude->flags & EXC_FLAG_NEGATIVE ? 0 : 1;
 	return -1; /* undecided */
diff --git a/dir.h b/dir.h
index a9f809983..64254c7e7 100644
--- a/dir.h
+++ b/dir.h
@@ -217,8 +217,10 @@ extern int within_depth(const char *name, int namelen, int depth, int max_depth)
 extern int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec);
 extern int read_directory(struct dir_struct *, const char *path, int len, const struct pathspec *pathspec);
 
-extern int is_excluded_from_list(const char *pathname, int pathlen, const char *basename,
-				 int *dtype, struct exclude_list *el);
+extern int is_excluded_from_list(const char *pathname, int pathlen,
+				 const char *basename, int *dtype,
+				 struct exclude_list *el,
+				 struct index_state *istate);
 struct dir_entry *dir_add_ignored(struct dir_struct *dir,
 				  struct index_state *istate,
 				  const char *pathname, int len);
diff --git a/unpack-trees.c b/unpack-trees.c
index 6b7356dab..9bcb13e4f 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1040,7 +1040,7 @@ static int clear_ce_flags_dir(struct cache_entry **cache, int nr,
 	struct cache_entry **cache_end;
 	int dtype = DT_DIR;
 	int ret = is_excluded_from_list(prefix->buf, prefix->len,
-					basename, &dtype, el);
+					basename, &dtype, el, &the_index);
 	int rc;
 
 	strbuf_addch(prefix, '/');
@@ -1143,7 +1143,7 @@ static int clear_ce_flags_1(struct cache_entry **cache, int nr,
 		/* Non-directory */
 		dtype = ce_to_dtype(ce);
 		ret = is_excluded_from_list(ce->name, ce_namelen(ce),
-					    name, &dtype, el);
+					    name, &dtype, el, &the_index);
 		if (ret < 0)
 			ret = defval;
 		if (ret > 0)
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 08/14] dir: convert add_excludes to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c          | 29 +++++++++++++++--------------
 dir.h          |  2 +-
 unpack-trees.c |  2 +-
 3 files changed, 17 insertions(+), 16 deletions(-)

diff --git a/dir.c b/dir.c
index 497a2db85..b86d02ff9 100644
--- a/dir.c
+++ b/dir.c
@@ -730,7 +730,7 @@ static void invalidate_directory(struct untracked_cache *uc,
 
 /*
  * Given a file with name "fname", read it (either from disk, or from
- * the index if "check_index" is non-zero), parse it and store the
+ * an index if 'istate' is non-null), parse it and store the
  * exclude rules in "el".
  *
  * If "ss" is not NULL, compute SHA-1 of the exclude file and fill
@@ -738,7 +738,8 @@ static void invalidate_directory(struct untracked_cache *uc,
  * ss_valid is non-zero, "ss" must contain good value as input.
  */
 static int add_excludes(const char *fname, const char *base, int baselen,
-			struct exclude_list *el, int check_index,
+			struct exclude_list *el,
+			struct index_state *istate,
 			struct sha1_stat *sha1_stat)
 {
 	struct stat st;
@@ -752,8 +753,8 @@ static int add_excludes(const char *fname, const char *base, int baselen,
 			warn_on_inaccessible(fname);
 		if (0 <= fd)
 			close(fd);
-		if (!check_index ||
-		    (buf = read_skip_worktree_file_from_index(&the_index, fname, &size, sha1_stat)) == NULL)
+		if (!istate ||
+		    (buf = read_skip_worktree_file_from_index(istate, fname, &size, sha1_stat)) == NULL)
 			return -1;
 		if (size == 0) {
 			free(buf);
@@ -785,15 +786,15 @@ static int add_excludes(const char *fname, const char *base, int baselen,
 		if (sha1_stat) {
 			int pos;
 			if (sha1_stat->valid &&
-			    !match_stat_data_racy(&the_index, &sha1_stat->stat, &st))
+			    !match_stat_data_racy(istate, &sha1_stat->stat, &st))
 				; /* no content change, ss->sha1 still good */
-			else if (check_index &&
-				 (pos = index_name_pos(&the_index, fname, strlen(fname))) >= 0 &&
-				 !ce_stage(the_index.cache[pos]) &&
-				 ce_uptodate(the_index.cache[pos]) &&
+			else if (istate &&
+				 (pos = index_name_pos(istate, fname, strlen(fname))) >= 0 &&
+				 !ce_stage(istate->cache[pos]) &&
+				 ce_uptodate(istate->cache[pos]) &&
 				 !would_convert_to_git(fname))
 				hashcpy(sha1_stat->sha1,
-					the_index.cache[pos]->oid.hash);
+					istate->cache[pos]->oid.hash);
 			else
 				hash_sha1_file(buf, size, "blob", sha1_stat->sha1);
 			fill_stat_data(&sha1_stat->stat, &st);
@@ -824,9 +825,9 @@ static int add_excludes(const char *fname, const char *base, int baselen,
 
 int add_excludes_from_file_to_list(const char *fname, const char *base,
 				   int baselen, struct exclude_list *el,
-				   int check_index)
+				   struct index_state *istate)
 {
-	return add_excludes(fname, base, baselen, el, check_index, NULL);
+	return add_excludes(fname, base, baselen, el, istate, NULL);
 }
 
 struct exclude_list *add_exclude_list(struct dir_struct *dir,
@@ -858,7 +859,7 @@ static void add_excludes_from_file_1(struct dir_struct *dir, const char *fname,
 	if (!dir->untracked)
 		dir->unmanaged_exclude_files++;
 	el = add_exclude_list(dir, EXC_FILE, fname);
-	if (add_excludes(fname, "", 0, el, 0, sha1_stat) < 0)
+	if (add_excludes(fname, "", 0, el, NULL, sha1_stat) < 0)
 		die("cannot use %s as an exclude file", fname);
 }
 
@@ -1166,7 +1167,7 @@ static void prep_exclude(struct dir_struct *dir, const char *base, int baselen)
 			strbuf_addbuf(&sb, &dir->basebuf);
 			strbuf_addstr(&sb, dir->exclude_per_dir);
 			el->src = strbuf_detach(&sb, NULL);
-			add_excludes(el->src, el->src, stk->baselen, el, 1,
+			add_excludes(el->src, el->src, stk->baselen, el, &the_index,
 				     untracked ? &sha1_stat : NULL);
 		}
 		/*
diff --git a/dir.h b/dir.h
index 64254c7e7..1bcda0d23 100644
--- a/dir.h
+++ b/dir.h
@@ -243,7 +243,7 @@ extern int is_excluded(struct dir_struct *dir, const char *name, int *dtype);
 extern struct exclude_list *add_exclude_list(struct dir_struct *dir,
 					     int group_type, const char *src);
 extern int add_excludes_from_file_to_list(const char *fname, const char *base, int baselen,
-					  struct exclude_list *el, int check_index);
+					  struct exclude_list *el, struct  index_state *istate);
 extern void add_excludes_from_file(struct dir_struct *, const char *fname);
 extern void parse_exclude_pattern(const char **string, int *patternlen, unsigned *flags, int *nowildcardlen);
 extern void add_exclude(const char *string, const char *base,
diff --git a/unpack-trees.c b/unpack-trees.c
index 9bcb13e4f..5b19849e4 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1223,7 +1223,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
 		o->skip_sparse_checkout = 1;
 	if (!o->skip_sparse_checkout) {
 		char *sparse = git_pathdup("info/sparse-checkout");
-		if (add_excludes_from_file_to_list(sparse, "", 0, &el, 0) < 0)
+		if (add_excludes_from_file_to_list(sparse, "", 0, &el, NULL) < 0)
 			o->skip_sparse_checkout = 1;
 		else
 			o->el = &el;
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 09/14] dir: convert prep_exclude to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/dir.c b/dir.c
index b86d02ff9..50b5e720e 100644
--- a/dir.c
+++ b/dir.c
@@ -1046,7 +1046,9 @@ static struct exclude *last_exclude_matching_from_lists(struct dir_struct *dir,
  * Loads the per-directory exclude list for the substring of base
  * which has a char length of baselen.
  */
-static void prep_exclude(struct dir_struct *dir, const char *base, int baselen)
+static void prep_exclude(struct dir_struct *dir,
+			 struct index_state *istate,
+			 const char *base, int baselen)
 {
 	struct exclude_list_group *group;
 	struct exclude_list *el;
@@ -1125,7 +1127,7 @@ static void prep_exclude(struct dir_struct *dir, const char *base, int baselen)
 			int dt = DT_DIR;
 			dir->basebuf.buf[stk->baselen - 1] = 0;
 			dir->exclude = last_exclude_matching_from_lists(dir,
-									&the_index,
+									istate,
 				dir->basebuf.buf, stk->baselen - 1,
 				dir->basebuf.buf + current, &dt);
 			dir->basebuf.buf[stk->baselen - 1] = '/';
@@ -1167,7 +1169,7 @@ static void prep_exclude(struct dir_struct *dir, const char *base, int baselen)
 			strbuf_addbuf(&sb, &dir->basebuf);
 			strbuf_addstr(&sb, dir->exclude_per_dir);
 			el->src = strbuf_detach(&sb, NULL);
-			add_excludes(el->src, el->src, stk->baselen, el, &the_index,
+			add_excludes(el->src, el->src, stk->baselen, el, istate,
 				     untracked ? &sha1_stat : NULL);
 		}
 		/*
@@ -1209,7 +1211,7 @@ struct exclude *last_exclude_matching(struct dir_struct *dir,
 	const char *basename = strrchr(pathname, '/');
 	basename = (basename) ? basename+1 : pathname;
 
-	prep_exclude(dir, pathname, basename-pathname);
+	prep_exclude(dir, &the_index, pathname, basename-pathname);
 
 	if (dir->exclude)
 		return dir->exclude;
@@ -1695,10 +1697,10 @@ static int valid_cached_dir(struct dir_struct *dir,
 	 */
 	if (path->len && path->buf[path->len - 1] != '/') {
 		strbuf_addch(path, '/');
-		prep_exclude(dir, path->buf, path->len);
+		prep_exclude(dir, &the_index, path->buf, path->len);
 		strbuf_setlen(path, path->len - 1);
 	} else
-		prep_exclude(dir, path->buf, path->len);
+		prep_exclude(dir, &the_index, path->buf, path->len);
 
 	/* hopefully prep_exclude() haven't invalidated this entry... */
 	return untracked->valid;
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 10/14] dir: convert is_excluded to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/add.c          |  2 +-
 builtin/check-ignore.c |  3 ++-
 builtin/clean.c        |  2 +-
 builtin/ls-files.c     |  2 +-
 dir.c                  | 16 +++++++++-------
 dir.h                  |  5 ++++-
 unpack-trees.c         |  2 +-
 7 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index bf5e676e4..0b52aeced 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -436,7 +436,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 			     !file_exists(path))) {
 				if (ignore_missing) {
 					int dtype = DT_UNKNOWN;
-					if (is_excluded(&dir, path, &dtype))
+					if (is_excluded(&dir, &the_index, path, &dtype))
 						dir_add_ignored(&dir, &the_index,
 								path, pathspec.items[i].len);
 				} else
diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index 1d73d3ca3..d2293b22e 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -101,7 +101,8 @@ static int check_ignore(struct dir_struct *dir,
 		full_path = pathspec.items[i].match;
 		exclude = NULL;
 		if (!seen[i]) {
-			exclude = last_exclude_matching(dir, full_path, &dtype);
+			exclude = last_exclude_matching(dir, &the_index,
+							full_path, &dtype);
 		}
 		if (!quiet && (exclude || show_non_matching))
 			output_exclude(pathspec.items[i].original, exclude);
diff --git a/builtin/clean.c b/builtin/clean.c
index d861f836a..39866afab 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -683,7 +683,7 @@ static int filter_by_patterns_cmd(void)
 		for_each_string_list_item(item, &del_list) {
 			int dtype = DT_UNKNOWN;
 
-			if (is_excluded(&dir, item->string, &dtype)) {
+			if (is_excluded(&dir, &the_index, item->string, &dtype)) {
 				*item->string = '\0';
 				changed++;
 			}
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d449e46db..ccba227e2 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -315,7 +315,7 @@ static void show_ru_info(void)
 static int ce_excluded(struct dir_struct *dir, const struct cache_entry *ce)
 {
 	int dtype = ce_to_dtype(ce);
-	return is_excluded(dir, ce->name, &dtype);
+	return is_excluded(dir, &the_index, ce->name, &dtype);
 }
 
 static void show_files(struct dir_struct *dir)
diff --git a/dir.c b/dir.c
index 50b5e720e..a15da672c 100644
--- a/dir.c
+++ b/dir.c
@@ -1204,19 +1204,20 @@ static void prep_exclude(struct dir_struct *dir,
  * undecided.
  */
 struct exclude *last_exclude_matching(struct dir_struct *dir,
-					     const char *pathname,
-					     int *dtype_p)
+				      struct index_state *istate,
+				      const char *pathname,
+				      int *dtype_p)
 {
 	int pathlen = strlen(pathname);
 	const char *basename = strrchr(pathname, '/');
 	basename = (basename) ? basename+1 : pathname;
 
-	prep_exclude(dir, &the_index, pathname, basename-pathname);
+	prep_exclude(dir, istate, pathname, basename-pathname);
 
 	if (dir->exclude)
 		return dir->exclude;
 
-	return last_exclude_matching_from_lists(dir, &the_index, pathname, pathlen,
+	return last_exclude_matching_from_lists(dir, istate, pathname, pathlen,
 			basename, dtype_p);
 }
 
@@ -1225,10 +1226,11 @@ struct exclude *last_exclude_matching(struct dir_struct *dir,
  * scans all exclude lists to determine whether pathname is excluded.
  * Returns 1 if true, otherwise 0.
  */
-int is_excluded(struct dir_struct *dir, const char *pathname, int *dtype_p)
+int is_excluded(struct dir_struct *dir, struct index_state *istate,
+		const char *pathname, int *dtype_p)
 {
 	struct exclude *exclude =
-		last_exclude_matching(dir, pathname, dtype_p);
+		last_exclude_matching(dir, istate, pathname, dtype_p);
 	if (exclude)
 		return exclude->flags & EXC_FLAG_NEGATIVE ? 0 : 1;
 	return 0;
@@ -1573,7 +1575,7 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	    (directory_exists_in_index(&the_index, path->buf, path->len) == index_nonexistent))
 		return path_none;
 
-	exclude = is_excluded(dir, path->buf, &dtype);
+	exclude = is_excluded(dir, &the_index, path->buf, &dtype);
 
 	/*
 	 * Excluded? If we don't explicitly want to show
diff --git a/dir.h b/dir.h
index 1bcda0d23..b745f1e5d 100644
--- a/dir.h
+++ b/dir.h
@@ -236,9 +236,12 @@ extern int match_pathname(const char *, int,
 			  const char *, int, int, unsigned);
 
 extern struct exclude *last_exclude_matching(struct dir_struct *dir,
+					     struct index_state *istate,
 					     const char *name, int *dtype);
 
-extern int is_excluded(struct dir_struct *dir, const char *name, int *dtype);
+extern int is_excluded(struct dir_struct *dir,
+		       struct index_state *istate,
+		       const char *name, int *dtype);
 
 extern struct exclude_list *add_exclude_list(struct dir_struct *dir,
 					     int group_type, const char *src);
diff --git a/unpack-trees.c b/unpack-trees.c
index 5b19849e4..abdd2922b 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1606,7 +1606,7 @@ static int check_ok_to_remove(const char *name, int len, int dtype,
 		return 0;
 
 	if (o->dir &&
-	    is_excluded(o->dir, name, &dtype))
+	    is_excluded(o->dir, &the_index, name, &dtype))
 		/*
 		 * ce->name is explicitly excluded, so it is Ok to
 		 * overwrite it.
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 11/14] dir: convert open_cached_dir to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/dir.c b/dir.c
index a15da672c..f08d30ee4 100644
--- a/dir.c
+++ b/dir.c
@@ -1664,6 +1664,7 @@ static void add_untracked(struct untracked_cache_dir *dir, const char *name)
 
 static int valid_cached_dir(struct dir_struct *dir,
 			    struct untracked_cache_dir *untracked,
+			    struct index_state *istate,
 			    struct strbuf *path,
 			    int check_only)
 {
@@ -1678,7 +1679,7 @@ static int valid_cached_dir(struct dir_struct *dir,
 		return 0;
 	}
 	if (!untracked->valid ||
-	    match_stat_data_racy(&the_index, &untracked->stat_data, &st)) {
+	    match_stat_data_racy(istate, &untracked->stat_data, &st)) {
 		if (untracked->valid)
 			invalidate_directory(dir->untracked, untracked);
 		fill_stat_data(&untracked->stat_data, &st);
@@ -1699,10 +1700,10 @@ static int valid_cached_dir(struct dir_struct *dir,
 	 */
 	if (path->len && path->buf[path->len - 1] != '/') {
 		strbuf_addch(path, '/');
-		prep_exclude(dir, &the_index, path->buf, path->len);
+		prep_exclude(dir, istate, path->buf, path->len);
 		strbuf_setlen(path, path->len - 1);
 	} else
-		prep_exclude(dir, &the_index, path->buf, path->len);
+		prep_exclude(dir, istate, path->buf, path->len);
 
 	/* hopefully prep_exclude() haven't invalidated this entry... */
 	return untracked->valid;
@@ -1711,12 +1712,13 @@ static int valid_cached_dir(struct dir_struct *dir,
 static int open_cached_dir(struct cached_dir *cdir,
 			   struct dir_struct *dir,
 			   struct untracked_cache_dir *untracked,
+			   struct index_state *istate,
 			   struct strbuf *path,
 			   int check_only)
 {
 	memset(cdir, 0, sizeof(*cdir));
 	cdir->untracked = untracked;
-	if (valid_cached_dir(dir, untracked, path, check_only))
+	if (valid_cached_dir(dir, untracked, istate, path, check_only))
 		return 0;
 	cdir->fdir = opendir(path->len ? path->buf : ".");
 	if (dir->untracked)
@@ -1789,7 +1791,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,
 
 	strbuf_add(&path, base, baselen);
 
-	if (open_cached_dir(&cdir, dir, untracked, &path, check_only))
+	if (open_cached_dir(&cdir, dir, untracked, &the_index, &path, check_only))
 		goto out;
 
 	if (untracked)
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 12/14] dir: convert read_directory_recursive to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c | 52 +++++++++++++++++++++++++++++-----------------------
 1 file changed, 29 insertions(+), 23 deletions(-)

diff --git a/dir.c b/dir.c
index f08d30ee4..3318ebbcb 100644
--- a/dir.c
+++ b/dir.c
@@ -46,7 +46,8 @@ struct cached_dir {
 };
 
 static enum path_treatment read_directory_recursive(struct dir_struct *dir,
-	const char *path, int len, struct untracked_cache_dir *untracked,
+	struct index_state *istate, const char *path, int len,
+	struct untracked_cache_dir *untracked,
 	int check_only, const struct pathspec *pathspec);
 static int get_dtype(struct dirent *de, struct index_state *istate,
 		     const char *path, int len);
@@ -1362,12 +1363,13 @@ static enum exist_status directory_exists_in_index(struct index_state *istate,
  *  (c) otherwise, we recurse into it.
  */
 static enum path_treatment treat_directory(struct dir_struct *dir,
+	struct index_state *istate,
 	struct untracked_cache_dir *untracked,
 	const char *dirname, int len, int baselen, int exclude,
 	const struct pathspec *pathspec)
 {
 	/* The "len-1" is to strip the final '/' */
-	switch (directory_exists_in_index(&the_index, dirname, len-1)) {
+	switch (directory_exists_in_index(istate, dirname, len-1)) {
 	case index_directory:
 		return path_recurse;
 
@@ -1392,7 +1394,7 @@ static enum path_treatment treat_directory(struct dir_struct *dir,
 
 	untracked = lookup_untracked(dir->untracked, untracked,
 				     dirname + baselen, len - baselen);
-	return read_directory_recursive(dir, dirname, len,
+	return read_directory_recursive(dir, istate, dirname, len,
 					untracked, 1, pathspec);
 }
 
@@ -1536,16 +1538,17 @@ static int get_dtype(struct dirent *de, struct index_state *istate,
 
 static enum path_treatment treat_one_path(struct dir_struct *dir,
 					  struct untracked_cache_dir *untracked,
+					  struct index_state *istate,
 					  struct strbuf *path,
 					  int baselen,
 					  const struct pathspec *pathspec,
 					  int dtype, struct dirent *de)
 {
 	int exclude;
-	int has_path_in_index = !!index_file_exists(&the_index, path->buf, path->len, ignore_case);
+	int has_path_in_index = !!index_file_exists(istate, path->buf, path->len, ignore_case);
 
 	if (dtype == DT_UNKNOWN)
-		dtype = get_dtype(de, &the_index, path->buf, path->len);
+		dtype = get_dtype(de, istate, path->buf, path->len);
 
 	/* Always exclude indexed files */
 	if (dtype != DT_DIR && has_path_in_index)
@@ -1572,10 +1575,10 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	if ((dir->flags & DIR_COLLECT_KILLED_ONLY) &&
 	    (dtype == DT_DIR) &&
 	    !has_path_in_index &&
-	    (directory_exists_in_index(&the_index, path->buf, path->len) == index_nonexistent))
+	    (directory_exists_in_index(istate, path->buf, path->len) == index_nonexistent))
 		return path_none;
 
-	exclude = is_excluded(dir, &the_index, path->buf, &dtype);
+	exclude = is_excluded(dir, istate, path->buf, &dtype);
 
 	/*
 	 * Excluded? If we don't explicitly want to show
@@ -1589,7 +1592,7 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 		return path_none;
 	case DT_DIR:
 		strbuf_addch(path, '/');
-		return treat_directory(dir, untracked, path->buf, path->len,
+		return treat_directory(dir, istate, untracked, path->buf, path->len,
 				       baselen, exclude, pathspec);
 	case DT_REG:
 	case DT_LNK:
@@ -1600,6 +1603,7 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 static enum path_treatment treat_path_fast(struct dir_struct *dir,
 					   struct untracked_cache_dir *untracked,
 					   struct cached_dir *cdir,
+					   struct index_state *istate,
 					   struct strbuf *path,
 					   int baselen,
 					   const struct pathspec *pathspec)
@@ -1618,7 +1622,7 @@ static enum path_treatment treat_path_fast(struct dir_struct *dir,
 		 * to its bottom. Verify again the same set of directories
 		 * with check_only set.
 		 */
-		return read_directory_recursive(dir, path->buf, path->len,
+		return read_directory_recursive(dir, istate, path->buf, path->len,
 						cdir->ucd, 1, pathspec);
 	/*
 	 * We get path_recurse in the first run when
@@ -1632,6 +1636,7 @@ static enum path_treatment treat_path_fast(struct dir_struct *dir,
 static enum path_treatment treat_path(struct dir_struct *dir,
 				      struct untracked_cache_dir *untracked,
 				      struct cached_dir *cdir,
+				      struct index_state *istate,
 				      struct strbuf *path,
 				      int baselen,
 				      const struct pathspec *pathspec)
@@ -1640,7 +1645,7 @@ static enum path_treatment treat_path(struct dir_struct *dir,
 	struct dirent *de = cdir->de;
 
 	if (!de)
-		return treat_path_fast(dir, untracked, cdir, path,
+		return treat_path_fast(dir, untracked, cdir, istate, path,
 				       baselen, pathspec);
 	if (is_dot_or_dotdot(de->d_name) || !strcmp(de->d_name, ".git"))
 		return path_none;
@@ -1650,7 +1655,7 @@ static enum path_treatment treat_path(struct dir_struct *dir,
 		return path_none;
 
 	dtype = DTYPE(de);
-	return treat_one_path(dir, untracked, path, baselen, pathspec, dtype, de);
+	return treat_one_path(dir, untracked, istate, path, baselen, pathspec, dtype, de);
 }
 
 static void add_untracked(struct untracked_cache_dir *dir, const char *name)
@@ -1781,9 +1786,9 @@ static void close_cached_dir(struct cached_dir *cdir)
  * Returns the most significant path_treatment value encountered in the scan.
  */
 static enum path_treatment read_directory_recursive(struct dir_struct *dir,
-				    const char *base, int baselen,
-				    struct untracked_cache_dir *untracked, int check_only,
-				    const struct pathspec *pathspec)
+	struct index_state *istate, const char *base, int baselen,
+	struct untracked_cache_dir *untracked, int check_only,
+	const struct pathspec *pathspec)
 {
 	struct cached_dir cdir;
 	enum path_treatment state, subdir_state, dir_state = path_none;
@@ -1791,7 +1796,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,
 
 	strbuf_add(&path, base, baselen);
 
-	if (open_cached_dir(&cdir, dir, untracked, &the_index, &path, check_only))
+	if (open_cached_dir(&cdir, dir, untracked, istate, &path, check_only))
 		goto out;
 
 	if (untracked)
@@ -1799,7 +1804,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,
 
 	while (!read_cached_dir(&cdir)) {
 		/* check how the file or directory should be treated */
-		state = treat_path(dir, untracked, &cdir, &path,
+		state = treat_path(dir, untracked, &cdir, istate, &path,
 				   baselen, pathspec);
 
 		if (state > dir_state)
@@ -1812,7 +1817,7 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,
 					      path.buf + baselen,
 					      path.len - baselen);
 			subdir_state =
-				read_directory_recursive(dir, path.buf,
+				read_directory_recursive(dir, istate, path.buf,
 							 path.len, ud,
 							 check_only, pathspec);
 			if (subdir_state > dir_state)
@@ -1834,18 +1839,18 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir,
 		switch (state) {
 		case path_excluded:
 			if (dir->flags & DIR_SHOW_IGNORED)
-				dir_add_name(dir, &the_index, path.buf, path.len);
+				dir_add_name(dir, istate, path.buf, path.len);
 			else if ((dir->flags & DIR_SHOW_IGNORED_TOO) ||
 				((dir->flags & DIR_COLLECT_IGNORED) &&
 				exclude_matches_pathspec(path.buf, path.len,
 							 pathspec)))
-				dir_add_ignored(dir, &the_index, path.buf, path.len);
+				dir_add_ignored(dir, istate, path.buf, path.len);
 			break;
 
 		case path_untracked:
 			if (dir->flags & DIR_SHOW_IGNORED)
 				break;
-			dir_add_name(dir, &the_index, path.buf, path.len);
+			dir_add_name(dir, istate, path.buf, path.len);
 			if (cdir.fdir)
 				add_untracked(untracked, path.buf + baselen);
 			break;
@@ -1870,6 +1875,7 @@ static int cmp_name(const void *p1, const void *p2)
 }
 
 static int treat_leading_path(struct dir_struct *dir,
+			      struct index_state *istate,
 			      const char *path, int len,
 			      const struct pathspec *pathspec)
 {
@@ -1897,7 +1903,7 @@ static int treat_leading_path(struct dir_struct *dir,
 			break;
 		if (simplify_away(sb.buf, sb.len, pathspec))
 			break;
-		if (treat_one_path(dir, NULL, &sb, baselen, pathspec,
+		if (treat_one_path(dir, NULL, istate, &sb, baselen, pathspec,
 				   DT_DIR, NULL) == path_none)
 			break; /* do not recurse into it */
 		if (len <= baselen) {
@@ -2080,8 +2086,8 @@ int read_directory(struct dir_struct *dir, const char *path,
 		 * e.g. prep_exclude()
 		 */
 		dir->untracked = NULL;
-	if (!len || treat_leading_path(dir, path, len, pathspec))
-		read_directory_recursive(dir, path, len, untracked, 0, pathspec);
+	if (!len || treat_leading_path(dir, &the_index, path, len, pathspec))
+		read_directory_recursive(dir, &the_index, path, len, untracked, 0, pathspec);
 	QSORT(dir->entries, dir->nr, cmp_name);
 	QSORT(dir->ignored, dir->ignored_nr, cmp_name);
 	if (dir->untracked) {
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 13/14] dir: convert read_directory to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 dir.c          | 16 ++++++++--------
 dir.h          |  4 +++-
 unpack-trees.c |  2 +-
 3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/dir.c b/dir.c
index 3318ebbcb..4eb8cb6a2 100644
--- a/dir.c
+++ b/dir.c
@@ -190,7 +190,7 @@ int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec)
 	prefix = prefix_len ? pathspec->items[0].match : "";
 
 	/* Read the directory and prune it */
-	read_directory(dir, prefix, prefix_len, pathspec);
+	read_directory(dir, &the_index, prefix, prefix_len, pathspec);
 
 	return prefix_len;
 }
@@ -2071,8 +2071,8 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d
 	return root;
 }
 
-int read_directory(struct dir_struct *dir, const char *path,
-		   int len, const struct pathspec *pathspec)
+int read_directory(struct dir_struct *dir, struct index_state *istate,
+		   const char *path, int len, const struct pathspec *pathspec)
 {
 	struct untracked_cache_dir *untracked;
 
@@ -2086,8 +2086,8 @@ int read_directory(struct dir_struct *dir, const char *path,
 		 * e.g. prep_exclude()
 		 */
 		dir->untracked = NULL;
-	if (!len || treat_leading_path(dir, &the_index, path, len, pathspec))
-		read_directory_recursive(dir, &the_index, path, len, untracked, 0, pathspec);
+	if (!len || treat_leading_path(dir, istate, path, len, pathspec))
+		read_directory_recursive(dir, istate, path, len, untracked, 0, pathspec);
 	QSORT(dir->entries, dir->nr, cmp_name);
 	QSORT(dir->ignored, dir->ignored_nr, cmp_name);
 	if (dir->untracked) {
@@ -2101,12 +2101,12 @@ int read_directory(struct dir_struct *dir, const char *path,
 				 dir->untracked->gitignore_invalidated,
 				 dir->untracked->dir_invalidated,
 				 dir->untracked->dir_opened);
-		if (dir->untracked == the_index.untracked &&
+		if (dir->untracked == istate->untracked &&
 		    (dir->untracked->dir_opened ||
 		     dir->untracked->gitignore_invalidated ||
 		     dir->untracked->dir_invalidated))
-			the_index.cache_changed |= UNTRACKED_CHANGED;
-		if (dir->untracked != the_index.untracked) {
+			istate->cache_changed |= UNTRACKED_CHANGED;
+		if (dir->untracked != istate->untracked) {
 			free(dir->untracked);
 			dir->untracked = NULL;
 		}
diff --git a/dir.h b/dir.h
index b745f1e5d..a23bcd005 100644
--- a/dir.h
+++ b/dir.h
@@ -215,7 +215,9 @@ extern int report_path_error(const char *ps_matched, const struct pathspec *path
 extern int within_depth(const char *name, int namelen, int depth, int max_depth);
 
 extern int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec);
-extern int read_directory(struct dir_struct *, const char *path, int len, const struct pathspec *pathspec);
+extern int read_directory(struct dir_struct *, struct index_state *istate,
+			  const char *path, int len,
+			  const struct pathspec *pathspec);
 
 extern int is_excluded_from_list(const char *pathname, int pathlen,
 				 const char *basename, int *dtype,
diff --git a/unpack-trees.c b/unpack-trees.c
index abdd2922b..48d9cf921 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1564,7 +1564,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 	memset(&d, 0, sizeof(d));
 	if (o->dir)
 		d.exclude_per_dir = o->dir->exclude_per_dir;
-	i = read_directory(&d, pathbuf, namelen+1, NULL);
+	i = read_directory(&d, &the_index, pathbuf, namelen+1, NULL);
 	if (i)
 		return o->gently ? -1 :
 			add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
-- 
2.13.0.rc1.294.g07d810a77f-goog

[RFC 14/14] dir: convert fill_directory to take an index

[Brandon Williams]

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/add.c      | 2 +-
 builtin/clean.c    | 2 +-
 builtin/grep.c     | 2 +-
 builtin/ls-files.c | 2 +-
 dir.c              | 6 ++++--
 dir.h              | 4 +++-
 wt-status.c        | 2 +-
 7 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index 0b52aeced..36cad00ae 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -400,7 +400,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		}
 
 		/* This picks up the paths that are not tracked */
-		baselen = fill_directory(&dir, &pathspec);
+		baselen = fill_directory(&dir, &the_index, &pathspec);
 		if (pathspec.nr)
 			seen = prune_directory(&dir, &pathspec, baselen);
 	}
diff --git a/builtin/clean.c b/builtin/clean.c
index 39866afab..329b68c40 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -930,7 +930,7 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 		       PATHSPEC_PREFER_CWD,
 		       prefix, argv);
 
-	fill_directory(&dir, &pathspec);
+	fill_directory(&dir, &the_index, &pathspec);
 
 	for (i = 0; i < dir.nr; i++) {
 		struct dir_entry *ent = dir.entries[i];
diff --git a/builtin/grep.c b/builtin/grep.c
index 65070c52f..ca1344133 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -866,7 +866,7 @@ static int grep_directory(struct grep_opt *opt, const struct pathspec *pathspec,
 	if (exc_std)
 		setup_standard_excludes(&dir);
 
-	fill_directory(&dir, pathspec);
+	fill_directory(&dir, &the_index, pathspec);
 	for (i = 0; i < dir.nr; i++) {
 		if (!dir_path_match(dir.entries[i], pathspec, 0, NULL))
 			continue;
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index ccba227e2..239c18a1f 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -326,7 +326,7 @@ static void show_files(struct dir_struct *dir)
 	if (show_others || show_killed) {
 		if (!show_others)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
-		fill_directory(dir, &pathspec);
+		fill_directory(dir, &the_index, &pathspec);
 		if (show_others)
 			show_other_files(dir);
 		if (show_killed)
diff --git a/dir.c b/dir.c
index 4eb8cb6a2..3f3167e55 100644
--- a/dir.c
+++ b/dir.c
@@ -177,7 +177,9 @@ char *common_prefix(const struct pathspec *pathspec)
 	return len ? xmemdupz(pathspec->items[0].match, len) : NULL;
 }
 
-int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec)
+int fill_directory(struct dir_struct *dir,
+		   struct index_state *istate,
+		   const struct pathspec *pathspec)
 {
 	const char *prefix;
 	size_t prefix_len;
@@ -190,7 +192,7 @@ int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec)
 	prefix = prefix_len ? pathspec->items[0].match : "";
 
 	/* Read the directory and prune it */
-	read_directory(dir, &the_index, prefix, prefix_len, pathspec);
+	read_directory(dir, istate, prefix, prefix_len, pathspec);
 
 	return prefix_len;
 }
diff --git a/dir.h b/dir.h
index a23bcd005..17d110693 100644
--- a/dir.h
+++ b/dir.h
@@ -214,7 +214,9 @@ extern int match_pathspec(const struct pathspec *pathspec,
 extern int report_path_error(const char *ps_matched, const struct pathspec *pathspec, const char *prefix);
 extern int within_depth(const char *name, int namelen, int depth, int max_depth);
 
-extern int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec);
+extern int fill_directory(struct dir_struct *dir,
+			  struct index_state *istate,
+			  const struct pathspec *pathspec);
 extern int read_directory(struct dir_struct *, struct index_state *istate,
 			  const char *path, int len,
 			  const struct pathspec *pathspec);
diff --git a/wt-status.c b/wt-status.c
index 308cf3779..9e2d03cd9 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -652,7 +652,7 @@ static void wt_status_collect_untracked(struct wt_status *s)
 		dir.untracked = the_index.untracked;
 	setup_standard_excludes(&dir);
 
-	fill_directory(&dir, &s->pathspec);
+	fill_directory(&dir, &the_index, &s->pathspec);
 
 	for (i = 0; i < dir.nr; i++) {
 		struct dir_entry *ent = dir.entries[i];
-- 
2.13.0.rc1.294.g07d810a77f-goog

Re: [RFC 00/14] convert dir.c to take an index parameter

[Junio C Hamano]

Brandon Williams <bmwill@google.com> writes:

> One of the things brought up on the list in the past few days has been
> migrating away from using the index compatibility macros.  One of the issues
> brought up in that thread was how simply doing that conversion doesn't
> eliminate the reliance on global state (specifically the_index).  If one day we
> want to have a 'repository object' passed around then we first need to convert
> different subsystems to be prepared to handle that.  This series provides a
> first step, converting the code in dir.c to take a 'struct index_state' and
> using that instead of implicitly using 'the_index'.

Very nicely done (I only skimmed "dir.c" in the end result and didn't
go through the changes with fine toothed comb, though).

I would have done this without the first step and then instead had a
final patch that only inserts a single

    #define NO_THE_INDEX_COMPATIBILITY_MACROS

at the beginning of dir.c once everybody in dir.c loses the
reference to all "cache" macros at the end, if I were doing this
series, but it is a personal taste.  

The resulting dir.c does not even refer to the_index, which is very
nice.

Thanks.

> Brandon Williams (14):
>   dir: stop using the index compatibility macros
>   dir: convert read_skip_worktree_file_from_index to take an index
>   dir: convert directory_exists_in_index to take index
>   dir: convert get_dtype to take index
>   dir: convert dir_add* to take an index
>   dir: convert last_exclude_matching_from_list to take an index
>   dir: convert is_excluded_from_list to take an index
>   dir: convert add_excludes to take an index
>   dir: convert prep_exclude to take an index
>   dir: convert is_excluded to take an index
>   dir: convert open_cached_dir to take an index
>   dir: convert read_directory_recursive to take an index
>   dir: convert read_directory to take an index
>   dir: convert fill_directory to take an index
>
>  builtin/add.c          |   7 +-
>  builtin/check-ignore.c |   3 +-
>  builtin/clean.c        |   4 +-
>  builtin/grep.c         |   2 +-
>  builtin/ls-files.c     |   4 +-
>  dir.c                  | 200 ++++++++++++++++++++++++++++---------------------
>  dir.h                  |  27 +++++--
>  unpack-trees.c         |  10 +--
>  wt-status.c            |   2 +-
>  9 files changed, 151 insertions(+), 108 deletions(-)

Re: [RFC 00/14] convert dir.c to take an index parameter

[Brandon Williams]

On 05/06, Junio C Hamano wrote:
> Brandon Williams <bmwill@google.com> writes:
> 
> > One of the things brought up on the list in the past few days has been
> > migrating away from using the index compatibility macros.  One of the issues
> > brought up in that thread was how simply doing that conversion doesn't
> > eliminate the reliance on global state (specifically the_index).  If one day we
> > want to have a 'repository object' passed around then we first need to convert
> > different subsystems to be prepared to handle that.  This series provides a
> > first step, converting the code in dir.c to take a 'struct index_state' and
> > using that instead of implicitly using 'the_index'.
> 
> Very nicely done (I only skimmed "dir.c" in the end result and didn't
> go through the changes with fine toothed comb, though).
> 
> I would have done this without the first step and then instead had a
> final patch that only inserts a single
> 
>     #define NO_THE_INDEX_COMPATIBILITY_MACROS
> 
> at the beginning of dir.c once everybody in dir.c loses the
> reference to all "cache" macros at the end, if I were doing this
> series, but it is a personal taste.  
> 
> The resulting dir.c does not even refer to the_index, which is very
> nice.

Thanks! I'm glad there's a few people who see this as a positive change.

> 
> Thanks.
> 
> > Brandon Williams (14):
> >   dir: stop using the index compatibility macros
> >   dir: convert read_skip_worktree_file_from_index to take an index
> >   dir: convert directory_exists_in_index to take index
> >   dir: convert get_dtype to take index
> >   dir: convert dir_add* to take an index
> >   dir: convert last_exclude_matching_from_list to take an index
> >   dir: convert is_excluded_from_list to take an index
> >   dir: convert add_excludes to take an index
> >   dir: convert prep_exclude to take an index
> >   dir: convert is_excluded to take an index
> >   dir: convert open_cached_dir to take an index
> >   dir: convert read_directory_recursive to take an index
> >   dir: convert read_directory to take an index
> >   dir: convert fill_directory to take an index
> >
> >  builtin/add.c          |   7 +-
> >  builtin/check-ignore.c |   3 +-
> >  builtin/clean.c        |   4 +-
> >  builtin/grep.c         |   2 +-
> >  builtin/ls-files.c     |   4 +-
> >  dir.c                  | 200 ++++++++++++++++++++++++++++---------------------
> >  dir.h                  |  27 +++++--
> >  unpack-trees.c         |  10 +--
> >  wt-status.c            |   2 +-
> >  9 files changed, 151 insertions(+), 108 deletions(-)

-- 
Brandon Williams

Re: [RFC 00/14] convert dir.c to take an index parameter

[Jeff Hostetler]

On 5/8/2017 1:12 PM, Brandon Williams wrote:
> On 05/06, Junio C Hamano wrote:
>> Brandon Williams <bmwill@google.com> writes:
>>
>>> One of the things brought up on the list in the past few days has been
>>> migrating away from using the index compatibility macros.  One of the issues
>>> brought up in that thread was how simply doing that conversion doesn't
>>> eliminate the reliance on global state (specifically the_index).  If one day we
>>> want to have a 'repository object' passed around then we first need to convert
>>> different subsystems to be prepared to handle that.  This series provides a
>>> first step, converting the code in dir.c to take a 'struct index_state' and
>>> using that instead of implicitly using 'the_index'.
>>
>> Very nicely done (I only skimmed "dir.c" in the end result and didn't
>> go through the changes with fine toothed comb, though).
>>
>> I would have done this without the first step and then instead had a
>> final patch that only inserts a single
>>
>>     #define NO_THE_INDEX_COMPATIBILITY_MACROS
>>
>> at the beginning of dir.c once everybody in dir.c loses the
>> reference to all "cache" macros at the end, if I were doing this
>> series, but it is a personal taste.
>>
>> The resulting dir.c does not even refer to the_index, which is very
>> nice.
>
> Thanks! I'm glad there's a few people who see this as a positive change.

Agreed.  This looks like a nice start.
Jeff