git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
* [PATCH 00/31] repository object
@ 2017-05-31 21:43 Brandon Williams
  2017-05-31 21:43 ` [PATCH 01/31] config: create config.h Brandon Williams
                   ` (33 more replies)
  0 siblings, 34 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Given the vast interest expressed when I sent out my RFC series I decided it
would be worth it to invest more time to making a repository object a reality.

This series is an extension of the last series I sent out (in that ls-files is
converted to working on submodules in-process using repository objects instead
of spawning a child process to do the work).  The big difference from the RFC
series is that I went through and did the work to migrate key repository state
from global variables in 'environment.c' to being stored in a repository object
itself.  I migrated the bits of state that seemed reasonable for this series,
there is still a lot of global state which could be migrated in the future.

I do think that we need to be slightly cautious about moving global state into
the repository object though, I don't want 'struct repo' to simply become a
kitchen sink where everything gets dumped.  But this is just a warning for the
future.

Since this is a v1 I'm fairly certain that it still has a lot of rough edges
(like I think I need to write better commit messages, and we should probably
have more comments documenting object fields/contract) but I want to get the
review process started sooner rather than later since I'm sure people will have
opinions (e.g. should it be called 'struct repo' or 'struct repository'?!).

So here it is!  Thank you so much for taking the time review this, any and all
comments would be appreciated.

- Brandon

Patches [01-14] -> Introducing the Repository object and migrating some state
		   to be stored in 'the_repository'.

Patches [15-31] -> Converting ls-files to use 'struct repo' in order to recurse
		   submodules in-process.

Brandon Williams (31):
  config: create config.h
  config: don't include config.h by default
  config: don't implicitly use gitdir
  setup: don't perform lazy initialization of repository state
  environment: remove namespace_len variable
  repo: introduce the repository object
  environment: place key repository state in the_repository
  environment: store worktree in the_repository
  setup: add comment indicating a hack
  config: migrate the_configset to the_repository
  repo: add index_state to struct repo
  submodule-config: store the_submodule_cache in the_repository
  repo: add repo_read_gitmodules
  submodule: convert is_submodule_initialized to work on a repository
  convert: convert get_cached_convert_stats_ascii to take an index
  convert: convert crlf_to_git to take an index
  convert: convert convert_to_git_filter_fd to take an index
  convert: convert convert_to_git to take an index
  convert: convert renormalize_buffer to take an index
  tree: convert read_tree to take an index parameter
  ls-files: convert overlay_tree_on_cache to take an index
  ls-files: convert write_eolinfo to take an index
  ls-files: convert show_killed_files to take an index
  ls-files: convert show_other_files to take an index
  ls-files: convert show_ru_info to take an index
  ls-files: convert ce_excluded to take an index
  ls-files: convert prune_cache to take an index
  ls-files: convert show_files to take an index
  ls-files: factor out debug info into a function
  ls-files: factor out tag calculation
  ls-files: use repository object

 Makefile                               |   1 +
 advice.c                               |   1 +
 alias.c                                |   1 +
 apply.c                                |   3 +-
 archive-tar.c                          |   1 +
 archive-zip.c                          |   1 +
 archive.c                              |   1 +
 attr.c                                 |   1 +
 bisect.c                               |   1 +
 branch.c                               |   1 +
 builtin/add.c                          |   1 +
 builtin/am.c                           |   1 +
 builtin/blame.c                        |   3 +-
 builtin/branch.c                       |   1 +
 builtin/cat-file.c                     |   1 +
 builtin/check-attr.c                   |   1 +
 builtin/check-ignore.c                 |   1 +
 builtin/check-mailmap.c                |   1 +
 builtin/checkout-index.c               |   1 +
 builtin/checkout.c                     |   1 +
 builtin/clean.c                        |   1 +
 builtin/clone.c                        |   1 +
 builtin/column.c                       |   1 +
 builtin/commit-tree.c                  |   1 +
 builtin/commit.c                       |   4 +-
 builtin/config.c                       |   3 +
 builtin/count-objects.c                |   1 +
 builtin/describe.c                     |   1 +
 builtin/diff-files.c                   |   1 +
 builtin/diff-index.c                   |   1 +
 builtin/diff-tree.c                    |   1 +
 builtin/diff.c                         |   1 +
 builtin/difftool.c                     |   1 +
 builtin/fast-export.c                  |   1 +
 builtin/fetch.c                        |   1 +
 builtin/fmt-merge-msg.c                |   1 +
 builtin/for-each-ref.c                 |   1 +
 builtin/fsck.c                         |   1 +
 builtin/gc.c                           |   1 +
 builtin/grep.c                         |   4 +-
 builtin/hash-object.c                  |   1 +
 builtin/help.c                         |   1 +
 builtin/index-pack.c                   |   1 +
 builtin/init-db.c                      |   1 +
 builtin/log.c                          |   1 +
 builtin/ls-files.c                     | 343 ++++++++++++++++-----------------
 builtin/ls-tree.c                      |   1 +
 builtin/merge-base.c                   |   1 +
 builtin/merge-file.c                   |   1 +
 builtin/merge.c                        |   1 +
 builtin/mv.c                           |   1 +
 builtin/name-rev.c                     |   1 +
 builtin/notes.c                        |   1 +
 builtin/pack-objects.c                 |   1 +
 builtin/patch-id.c                     |   1 +
 builtin/pull.c                         |   1 +
 builtin/push.c                         |   1 +
 builtin/read-tree.c                    |   1 +
 builtin/rebase--helper.c               |   1 +
 builtin/receive-pack.c                 |   1 +
 builtin/reflog.c                       |   1 +
 builtin/remote.c                       |   1 +
 builtin/repack.c                       |   1 +
 builtin/replace.c                      |   1 +
 builtin/rerere.c                       |   1 +
 builtin/reset.c                        |   1 +
 builtin/rev-list.c                     |   1 +
 builtin/rev-parse.c                    |   1 +
 builtin/revert.c                       |   1 +
 builtin/rm.c                           |   1 +
 builtin/send-pack.c                    |   1 +
 builtin/shortlog.c                     |   1 +
 builtin/show-branch.c                  |   1 +
 builtin/stripspace.c                   |   1 +
 builtin/submodule--helper.c            |  10 +-
 builtin/symbolic-ref.c                 |   1 +
 builtin/tag.c                          |   1 +
 builtin/unpack-file.c                  |   1 +
 builtin/unpack-objects.c               |   1 +
 builtin/update-index.c                 |   1 +
 builtin/update-ref.c                   |   1 +
 builtin/update-server-info.c           |   1 +
 builtin/var.c                          |   1 +
 builtin/verify-commit.c                |   1 +
 builtin/verify-pack.c                  |   1 +
 builtin/verify-tag.c                   |   1 +
 builtin/worktree.c                     |   1 +
 builtin/write-tree.c                   |   1 +
 cache.h                                | 196 +------------------
 color.c                                |   1 +
 column.c                               |   1 +
 combine-diff.c                         |   2 +-
 config.c                               | 186 +++++++++++-------
 config.h                               | 204 ++++++++++++++++++++
 connect.c                              |   1 +
 convert.c                              |  32 +--
 convert.h                              |  19 +-
 credential-cache--daemon.c             |   1 +
 credential.c                           |   1 +
 daemon.c                               |   1 +
 diff.c                                 |   7 +-
 dir.c                                  |   3 +-
 environment.c                          | 101 +++-------
 fast-import.c                          |   1 +
 fetch-pack.c                           |   1 +
 git.c                                  |   3 +-
 gpg-interface.c                        |   1 +
 graph.c                                |   1 +
 grep.c                                 |   1 +
 help.c                                 |   1 +
 http-backend.c                         |   1 +
 http-fetch.c                           |   1 +
 http.c                                 |   1 +
 ident.c                                |   1 +
 imap-send.c                            |   1 +
 ll-merge.c                             |   3 +-
 log-tree.c                             |   1 +
 mailinfo.c                             |   1 +
 merge-recursive.c                      |   5 +-
 notes-utils.c                          |   1 +
 notes.c                                |   1 +
 pager.c                                |   1 +
 parse-options.c                        |   1 +
 path.c                                 |  11 +-
 pathspec.c                             |   1 +
 pretty.c                               |   1 +
 prompt.c                               |   1 +
 read-cache.c                           |   1 +
 refs.c                                 |   1 +
 refs/files-backend.c                   |   1 +
 remote-curl.c                          |   1 +
 remote.c                               |   1 +
 repo.c                                 | 192 ++++++++++++++++++
 repo.h                                 |  45 +++++
 rerere.c                               |   1 +
 send-pack.c                            |   1 +
 sequencer.c                            |   1 +
 setup.c                                |  34 ++++
 sha1_file.c                            |   7 +-
 sha1_name.c                            |   1 +
 submodule-config.c                     |  72 +++++--
 submodule-config.h                     |  10 +
 submodule.c                            |  22 +--
 submodule.h                            |   3 +-
 t/helper/test-config.c                 |   1 +
 t/helper/test-submodule-config.c       |   1 +
 t/t3007-ls-files-recurse-submodules.sh |  39 ++++
 trailer.c                              |   1 +
 transport.c                            |   1 +
 tree.c                                 |  28 ++-
 tree.h                                 |   3 +-
 unpack-trees.c                         |   1 +
 upload-pack.c                          |   1 +
 userdiff.c                             |   1 +
 versioncmp.c                           |   1 +
 wrapper.c                              |   1 +
 xdiff-interface.c                      |   1 +
 157 files changed, 1130 insertions(+), 593 deletions(-)
 create mode 100644 config.h
 create mode 100644 repo.c
 create mode 100644 repo.h

-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH 01/31] config: create config.h
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 02/31] config: don't include config.h by default Brandon Williams
                   ` (32 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Move all config related declarations from cache.h to a new config.h
header file.  This makes cache.h smaller and allows for the opportunity
in a following patch to only include config.h when needed.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h  | 190 +------------------------------------------------------------
 config.h | 194 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 195 insertions(+), 189 deletions(-)
 create mode 100644 config.h

diff --git a/cache.h b/cache.h
index ae4c45d37..a27fee458 100644
--- a/cache.h
+++ b/cache.h
@@ -11,6 +11,7 @@
 #include "string-list.h"
 #include "pack-revindex.h"
 #include "hash.h"
+#include "config.h"
 
 #ifndef platform_SHA_CTX
 /*
@@ -1866,188 +1867,9 @@ extern int packed_object_info(struct packed_git *pack, off_t offset, struct obje
 /* Dumb servers support */
 extern int update_server_info(int);
 
-/* git_config_parse_key() returns these negated: */
-#define CONFIG_INVALID_KEY 1
-#define CONFIG_NO_SECTION_OR_NAME 2
-/* git_config_set_gently(), git_config_set_multivar_gently() return the above or these: */
-#define CONFIG_NO_LOCK -1
-#define CONFIG_INVALID_FILE 3
-#define CONFIG_NO_WRITE 4
-#define CONFIG_NOTHING_SET 5
-#define CONFIG_INVALID_PATTERN 6
-#define CONFIG_GENERIC_ERROR 7
-
-#define CONFIG_REGEX_NONE ((void *)1)
-
-struct git_config_source {
-	unsigned int use_stdin:1;
-	const char *file;
-	const char *blob;
-};
-
-enum config_origin_type {
-	CONFIG_ORIGIN_BLOB,
-	CONFIG_ORIGIN_FILE,
-	CONFIG_ORIGIN_STDIN,
-	CONFIG_ORIGIN_SUBMODULE_BLOB,
-	CONFIG_ORIGIN_CMDLINE
-};
-
-struct config_options {
-	unsigned int respect_includes : 1;
-	const char *git_dir;
-};
-
-typedef int (*config_fn_t)(const char *, const char *, void *);
-extern int git_default_config(const char *, const char *, void *);
-extern int git_config_from_file(config_fn_t fn, const char *, void *);
-extern int git_config_from_mem(config_fn_t fn, const enum config_origin_type,
-					const char *name, const char *buf, size_t len, void *data);
-extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
-				     const unsigned char *sha1, void *data);
-extern void git_config_push_parameter(const char *text);
-extern int git_config_from_parameters(config_fn_t fn, void *data);
-extern void read_early_config(config_fn_t cb, void *data);
-extern void git_config(config_fn_t fn, void *);
-extern int git_config_with_options(config_fn_t fn, void *,
-				   struct git_config_source *config_source,
-				   const struct config_options *opts);
-extern int git_parse_ulong(const char *, unsigned long *);
-extern int git_parse_maybe_bool(const char *);
-extern int git_config_int(const char *, const char *);
-extern int64_t git_config_int64(const char *, const char *);
-extern unsigned long git_config_ulong(const char *, const char *);
-extern ssize_t git_config_ssize_t(const char *, const char *);
-extern int git_config_bool_or_int(const char *, const char *, int *);
-extern int git_config_bool(const char *, const char *);
-extern int git_config_maybe_bool(const char *, const char *);
-extern int git_config_string(const char **, const char *, const char *);
-extern int git_config_pathname(const char **, const char *, const char *);
-extern int git_config_set_in_file_gently(const char *, const char *, const char *);
-extern void git_config_set_in_file(const char *, const char *, const char *);
-extern int git_config_set_gently(const char *, const char *);
-extern void git_config_set(const char *, const char *);
-extern int git_config_parse_key(const char *, char **, int *);
-extern int git_config_key_is_valid(const char *key);
-extern int git_config_set_multivar_gently(const char *, const char *, const char *, int);
-extern void git_config_set_multivar(const char *, const char *, const char *, int);
-extern int git_config_set_multivar_in_file_gently(const char *, const char *, const char *, const char *, int);
-extern void git_config_set_multivar_in_file(const char *, const char *, const char *, const char *, int);
-extern int git_config_rename_section(const char *, const char *);
-extern int git_config_rename_section_in_file(const char *, const char *, const char *);
-extern const char *git_etc_gitconfig(void);
-extern int git_env_bool(const char *, int);
-extern unsigned long git_env_ulong(const char *, unsigned long);
-extern int git_config_system(void);
-extern int config_error_nonbool(const char *);
-#if defined(__GNUC__)
-#define config_error_nonbool(s) (config_error_nonbool(s), const_error())
-#endif
 extern const char *get_log_output_encoding(void);
 extern const char *get_commit_output_encoding(void);
 
-extern int git_config_parse_parameter(const char *, config_fn_t fn, void *data);
-
-enum config_scope {
-	CONFIG_SCOPE_UNKNOWN = 0,
-	CONFIG_SCOPE_SYSTEM,
-	CONFIG_SCOPE_GLOBAL,
-	CONFIG_SCOPE_REPO,
-	CONFIG_SCOPE_CMDLINE,
-};
-
-extern enum config_scope current_config_scope(void);
-extern const char *current_config_origin_type(void);
-extern const char *current_config_name(void);
-
-struct config_include_data {
-	int depth;
-	config_fn_t fn;
-	void *data;
-	const struct config_options *opts;
-};
-#define CONFIG_INCLUDE_INIT { 0 }
-extern int git_config_include(const char *name, const char *value, void *data);
-
-/*
- * Match and parse a config key of the form:
- *
- *   section.(subsection.)?key
- *
- * (i.e., what gets handed to a config_fn_t). The caller provides the section;
- * we return -1 if it does not match, 0 otherwise. The subsection and key
- * out-parameters are filled by the function (and *subsection is NULL if it is
- * missing).
- *
- * If the subsection pointer-to-pointer passed in is NULL, returns 0 only if
- * there is no subsection at all.
- */
-extern int parse_config_key(const char *var,
-			    const char *section,
-			    const char **subsection, int *subsection_len,
-			    const char **key);
-
-struct config_set_element {
-	struct hashmap_entry ent;
-	char *key;
-	struct string_list value_list;
-};
-
-struct configset_list_item {
-	struct config_set_element *e;
-	int value_index;
-};
-
-/*
- * the contents of the list are ordered according to their
- * position in the config files and order of parsing the files.
- * (i.e. key-value pair at the last position of .git/config will
- * be at the last item of the list)
- */
-struct configset_list {
-	struct configset_list_item *items;
-	unsigned int nr, alloc;
-};
-
-struct config_set {
-	struct hashmap config_hash;
-	int hash_initialized;
-	struct configset_list list;
-};
-
-extern void git_configset_init(struct config_set *cs);
-extern int git_configset_add_file(struct config_set *cs, const char *filename);
-extern int git_configset_get_value(struct config_set *cs, const char *key, const char **value);
-extern const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key);
-extern void git_configset_clear(struct config_set *cs);
-extern int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest);
-extern int git_configset_get_string(struct config_set *cs, const char *key, char **dest);
-extern int git_configset_get_int(struct config_set *cs, const char *key, int *dest);
-extern int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest);
-extern int git_configset_get_bool(struct config_set *cs, const char *key, int *dest);
-extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest);
-extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
-extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
-
-extern int git_config_get_value(const char *key, const char **value);
-extern const struct string_list *git_config_get_value_multi(const char *key);
-extern void git_config_clear(void);
-extern void git_config_iter(config_fn_t fn, void *data);
-extern int git_config_get_string_const(const char *key, const char **dest);
-extern int git_config_get_string(const char *key, char **dest);
-extern int git_config_get_int(const char *key, int *dest);
-extern int git_config_get_ulong(const char *key, unsigned long *dest);
-extern int git_config_get_bool(const char *key, int *dest);
-extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
-extern int git_config_get_maybe_bool(const char *key, int *dest);
-extern int git_config_get_pathname(const char *key, const char **dest);
-extern int git_config_get_untracked_cache(void);
-extern int git_config_get_split_index(void);
-extern int git_config_get_max_percent_split_change(void);
-
-/* This dies if the configured or default date is in the future */
-extern int git_config_get_expiry(const char *key, const char **output);
-
 /*
  * This is a hack for test programs like test-dump-untracked-cache to
  * ensure that they do not modify the untracked cache when reading it.
@@ -2055,16 +1877,6 @@ extern int git_config_get_expiry(const char *key, const char **output);
  */
 extern int ignore_untracked_cache_config;
 
-struct key_value_info {
-	const char *filename;
-	int linenr;
-	enum config_origin_type origin_type;
-	enum config_scope scope;
-};
-
-extern NORETURN void git_die_config(const char *key, const char *err, ...) __attribute__((format(printf, 2, 3)));
-extern NORETURN void git_die_config_linenr(const char *key, const char *filename, int linenr);
-
 extern int committer_ident_sufficiently_given(void);
 extern int author_ident_sufficiently_given(void);
 
diff --git a/config.h b/config.h
new file mode 100644
index 000000000..f7f8b66c5
--- /dev/null
+++ b/config.h
@@ -0,0 +1,194 @@
+#ifndef CONFIG_H
+#define CONFIG_H
+
+/* git_config_parse_key() returns these negated: */
+#define CONFIG_INVALID_KEY 1
+#define CONFIG_NO_SECTION_OR_NAME 2
+/* git_config_set_gently(), git_config_set_multivar_gently() return the above or these: */
+#define CONFIG_NO_LOCK -1
+#define CONFIG_INVALID_FILE 3
+#define CONFIG_NO_WRITE 4
+#define CONFIG_NOTHING_SET 5
+#define CONFIG_INVALID_PATTERN 6
+#define CONFIG_GENERIC_ERROR 7
+
+#define CONFIG_REGEX_NONE ((void *)1)
+
+struct git_config_source {
+	unsigned int use_stdin:1;
+	const char *file;
+	const char *blob;
+};
+
+enum config_origin_type {
+	CONFIG_ORIGIN_BLOB,
+	CONFIG_ORIGIN_FILE,
+	CONFIG_ORIGIN_STDIN,
+	CONFIG_ORIGIN_SUBMODULE_BLOB,
+	CONFIG_ORIGIN_CMDLINE
+};
+
+struct config_options {
+	unsigned int respect_includes : 1;
+	const char *git_dir;
+};
+
+typedef int (*config_fn_t)(const char *, const char *, void *);
+extern int git_default_config(const char *, const char *, void *);
+extern int git_config_from_file(config_fn_t fn, const char *, void *);
+extern int git_config_from_mem(config_fn_t fn, const enum config_origin_type,
+					const char *name, const char *buf, size_t len, void *data);
+extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
+				     const unsigned char *sha1, void *data);
+extern void git_config_push_parameter(const char *text);
+extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
+extern void git_config(config_fn_t fn, void *);
+extern int git_config_with_options(config_fn_t fn, void *,
+				   struct git_config_source *config_source,
+				   const struct config_options *opts);
+extern int git_parse_ulong(const char *, unsigned long *);
+extern int git_parse_maybe_bool(const char *);
+extern int git_config_int(const char *, const char *);
+extern int64_t git_config_int64(const char *, const char *);
+extern unsigned long git_config_ulong(const char *, const char *);
+extern ssize_t git_config_ssize_t(const char *, const char *);
+extern int git_config_bool_or_int(const char *, const char *, int *);
+extern int git_config_bool(const char *, const char *);
+extern int git_config_maybe_bool(const char *, const char *);
+extern int git_config_string(const char **, const char *, const char *);
+extern int git_config_pathname(const char **, const char *, const char *);
+extern int git_config_set_in_file_gently(const char *, const char *, const char *);
+extern void git_config_set_in_file(const char *, const char *, const char *);
+extern int git_config_set_gently(const char *, const char *);
+extern void git_config_set(const char *, const char *);
+extern int git_config_parse_key(const char *, char **, int *);
+extern int git_config_key_is_valid(const char *key);
+extern int git_config_set_multivar_gently(const char *, const char *, const char *, int);
+extern void git_config_set_multivar(const char *, const char *, const char *, int);
+extern int git_config_set_multivar_in_file_gently(const char *, const char *, const char *, const char *, int);
+extern void git_config_set_multivar_in_file(const char *, const char *, const char *, const char *, int);
+extern int git_config_rename_section(const char *, const char *);
+extern int git_config_rename_section_in_file(const char *, const char *, const char *);
+extern const char *git_etc_gitconfig(void);
+extern int git_env_bool(const char *, int);
+extern unsigned long git_env_ulong(const char *, unsigned long);
+extern int git_config_system(void);
+extern int config_error_nonbool(const char *);
+#if defined(__GNUC__)
+#define config_error_nonbool(s) (config_error_nonbool(s), const_error())
+#endif
+
+extern int git_config_parse_parameter(const char *, config_fn_t fn, void *data);
+
+enum config_scope {
+	CONFIG_SCOPE_UNKNOWN = 0,
+	CONFIG_SCOPE_SYSTEM,
+	CONFIG_SCOPE_GLOBAL,
+	CONFIG_SCOPE_REPO,
+	CONFIG_SCOPE_CMDLINE,
+};
+
+extern enum config_scope current_config_scope(void);
+extern const char *current_config_origin_type(void);
+extern const char *current_config_name(void);
+
+struct config_include_data {
+	int depth;
+	config_fn_t fn;
+	void *data;
+	const struct config_options *opts;
+};
+#define CONFIG_INCLUDE_INIT { 0 }
+extern int git_config_include(const char *name, const char *value, void *data);
+
+/*
+ * Match and parse a config key of the form:
+ *
+ *   section.(subsection.)?key
+ *
+ * (i.e., what gets handed to a config_fn_t). The caller provides the section;
+ * we return -1 if it does not match, 0 otherwise. The subsection and key
+ * out-parameters are filled by the function (and *subsection is NULL if it is
+ * missing).
+ *
+ * If the subsection pointer-to-pointer passed in is NULL, returns 0 only if
+ * there is no subsection at all.
+ */
+extern int parse_config_key(const char *var,
+			    const char *section,
+			    const char **subsection, int *subsection_len,
+			    const char **key);
+
+struct config_set_element {
+	struct hashmap_entry ent;
+	char *key;
+	struct string_list value_list;
+};
+
+struct configset_list_item {
+	struct config_set_element *e;
+	int value_index;
+};
+
+/*
+ * the contents of the list are ordered according to their
+ * position in the config files and order of parsing the files.
+ * (i.e. key-value pair at the last position of .git/config will
+ * be at the last item of the list)
+ */
+struct configset_list {
+	struct configset_list_item *items;
+	unsigned int nr, alloc;
+};
+
+struct config_set {
+	struct hashmap config_hash;
+	int hash_initialized;
+	struct configset_list list;
+};
+
+extern void git_configset_init(struct config_set *cs);
+extern int git_configset_add_file(struct config_set *cs, const char *filename);
+extern int git_configset_get_value(struct config_set *cs, const char *key, const char **value);
+extern const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key);
+extern void git_configset_clear(struct config_set *cs);
+extern int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest);
+extern int git_configset_get_string(struct config_set *cs, const char *key, char **dest);
+extern int git_configset_get_int(struct config_set *cs, const char *key, int *dest);
+extern int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest);
+extern int git_configset_get_bool(struct config_set *cs, const char *key, int *dest);
+extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest);
+extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
+extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
+
+extern int git_config_get_value(const char *key, const char **value);
+extern const struct string_list *git_config_get_value_multi(const char *key);
+extern void git_config_clear(void);
+extern void git_config_iter(config_fn_t fn, void *data);
+extern int git_config_get_string_const(const char *key, const char **dest);
+extern int git_config_get_string(const char *key, char **dest);
+extern int git_config_get_int(const char *key, int *dest);
+extern int git_config_get_ulong(const char *key, unsigned long *dest);
+extern int git_config_get_bool(const char *key, int *dest);
+extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
+extern int git_config_get_maybe_bool(const char *key, int *dest);
+extern int git_config_get_pathname(const char *key, const char **dest);
+extern int git_config_get_untracked_cache(void);
+extern int git_config_get_split_index(void);
+extern int git_config_get_max_percent_split_change(void);
+
+/* This dies if the configured or default date is in the future */
+extern int git_config_get_expiry(const char *key, const char **output);
+
+struct key_value_info {
+	const char *filename;
+	int linenr;
+	enum config_origin_type origin_type;
+	enum config_scope scope;
+};
+
+extern NORETURN void git_die_config(const char *key, const char *err, ...) __attribute__((format(printf, 2, 3)));
+extern NORETURN void git_die_config_linenr(const char *key, const char *filename, int linenr);
+
+#endif /* CONFIG_H */
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 02/31] config: don't include config.h by default
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
  2017-05-31 21:43 ` [PATCH 01/31] config: create config.h Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 03/31] config: don't implicitly use gitdir Brandon Williams
                   ` (31 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Stop including config.h by default in cache.h.  Instead only include
config.h in those files which require use of the config system.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 advice.c                         | 1 +
 alias.c                          | 1 +
 apply.c                          | 1 +
 archive-tar.c                    | 1 +
 archive-zip.c                    | 1 +
 archive.c                        | 1 +
 attr.c                           | 1 +
 bisect.c                         | 1 +
 branch.c                         | 1 +
 builtin/add.c                    | 1 +
 builtin/am.c                     | 1 +
 builtin/blame.c                  | 1 +
 builtin/branch.c                 | 1 +
 builtin/cat-file.c               | 1 +
 builtin/check-attr.c             | 1 +
 builtin/check-ignore.c           | 1 +
 builtin/check-mailmap.c          | 1 +
 builtin/checkout-index.c         | 1 +
 builtin/checkout.c               | 1 +
 builtin/clean.c                  | 1 +
 builtin/clone.c                  | 1 +
 builtin/column.c                 | 1 +
 builtin/commit-tree.c            | 1 +
 builtin/commit.c                 | 1 +
 builtin/config.c                 | 1 +
 builtin/count-objects.c          | 1 +
 builtin/describe.c               | 1 +
 builtin/diff-files.c             | 1 +
 builtin/diff-index.c             | 1 +
 builtin/diff-tree.c              | 1 +
 builtin/diff.c                   | 1 +
 builtin/difftool.c               | 1 +
 builtin/fast-export.c            | 1 +
 builtin/fetch.c                  | 1 +
 builtin/fmt-merge-msg.c          | 1 +
 builtin/for-each-ref.c           | 1 +
 builtin/fsck.c                   | 1 +
 builtin/gc.c                     | 1 +
 builtin/grep.c                   | 1 +
 builtin/hash-object.c            | 1 +
 builtin/help.c                   | 1 +
 builtin/index-pack.c             | 1 +
 builtin/init-db.c                | 1 +
 builtin/log.c                    | 1 +
 builtin/ls-files.c               | 1 +
 builtin/ls-tree.c                | 1 +
 builtin/merge-base.c             | 1 +
 builtin/merge-file.c             | 1 +
 builtin/merge.c                  | 1 +
 builtin/mv.c                     | 1 +
 builtin/name-rev.c               | 1 +
 builtin/notes.c                  | 1 +
 builtin/pack-objects.c           | 1 +
 builtin/patch-id.c               | 1 +
 builtin/pull.c                   | 1 +
 builtin/push.c                   | 1 +
 builtin/read-tree.c              | 1 +
 builtin/rebase--helper.c         | 1 +
 builtin/receive-pack.c           | 1 +
 builtin/reflog.c                 | 1 +
 builtin/remote.c                 | 1 +
 builtin/repack.c                 | 1 +
 builtin/replace.c                | 1 +
 builtin/rerere.c                 | 1 +
 builtin/reset.c                  | 1 +
 builtin/rev-list.c               | 1 +
 builtin/rev-parse.c              | 1 +
 builtin/revert.c                 | 1 +
 builtin/rm.c                     | 1 +
 builtin/send-pack.c              | 1 +
 builtin/shortlog.c               | 1 +
 builtin/show-branch.c            | 1 +
 builtin/stripspace.c             | 1 +
 builtin/submodule--helper.c      | 1 +
 builtin/symbolic-ref.c           | 1 +
 builtin/tag.c                    | 1 +
 builtin/unpack-file.c            | 1 +
 builtin/unpack-objects.c         | 1 +
 builtin/update-index.c           | 1 +
 builtin/update-ref.c             | 1 +
 builtin/update-server-info.c     | 1 +
 builtin/var.c                    | 1 +
 builtin/verify-commit.c          | 1 +
 builtin/verify-pack.c            | 1 +
 builtin/verify-tag.c             | 1 +
 builtin/worktree.c               | 1 +
 builtin/write-tree.c             | 1 +
 cache.h                          | 1 -
 color.c                          | 1 +
 column.c                         | 1 +
 config.c                         | 1 +
 connect.c                        | 1 +
 convert.c                        | 1 +
 credential-cache--daemon.c       | 1 +
 credential.c                     | 1 +
 daemon.c                         | 1 +
 diff.c                           | 1 +
 dir.c                            | 1 +
 environment.c                    | 1 +
 fast-import.c                    | 1 +
 fetch-pack.c                     | 1 +
 git.c                            | 1 +
 gpg-interface.c                  | 1 +
 graph.c                          | 1 +
 grep.c                           | 1 +
 help.c                           | 1 +
 http-backend.c                   | 1 +
 http-fetch.c                     | 1 +
 http.c                           | 1 +
 ident.c                          | 1 +
 imap-send.c                      | 1 +
 ll-merge.c                       | 1 +
 log-tree.c                       | 1 +
 mailinfo.c                       | 1 +
 merge-recursive.c                | 1 +
 notes-utils.c                    | 1 +
 notes.c                          | 1 +
 pager.c                          | 1 +
 parse-options.c                  | 1 +
 pathspec.c                       | 1 +
 pretty.c                         | 1 +
 prompt.c                         | 1 +
 read-cache.c                     | 1 +
 refs.c                           | 1 +
 refs/files-backend.c             | 1 +
 remote-curl.c                    | 1 +
 remote.c                         | 1 +
 rerere.c                         | 1 +
 send-pack.c                      | 1 +
 sequencer.c                      | 1 +
 setup.c                          | 1 +
 sha1_file.c                      | 1 +
 sha1_name.c                      | 1 +
 submodule-config.c               | 1 +
 submodule.c                      | 1 +
 t/helper/test-config.c           | 1 +
 t/helper/test-submodule-config.c | 1 +
 trailer.c                        | 1 +
 transport.c                      | 1 +
 unpack-trees.c                   | 1 +
 upload-pack.c                    | 1 +
 userdiff.c                       | 1 +
 versioncmp.c                     | 1 +
 wrapper.c                        | 1 +
 xdiff-interface.c                | 1 +
 145 files changed, 144 insertions(+), 1 deletion(-)

diff --git a/advice.c b/advice.c
index b84ae4960..3fa04fca0 100644
--- a/advice.c
+++ b/advice.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 
 int advice_push_update_rejected = 1;
 int advice_push_non_ff_current = 1;
diff --git a/alias.c b/alias.c
index 3b90397a9..052f34136 100644
--- a/alias.c
+++ b/alias.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 
 char *alias_lookup(const char *alias)
 {
diff --git a/apply.c b/apply.c
index c49cef063..87db9618d 100644
--- a/apply.c
+++ b/apply.c
@@ -8,6 +8,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "blob.h"
 #include "delta.h"
 #include "diff.h"
diff --git a/archive-tar.c b/archive-tar.c
index 073e60ebd..c6ed96ee7 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -2,6 +2,7 @@
  * Copyright (c) 2005, 2006 Rene Scharfe
  */
 #include "cache.h"
+#include "config.h"
 #include "tar.h"
 #include "archive.h"
 #include "streaming.h"
diff --git a/archive-zip.c b/archive-zip.c
index 27563e9e2..e8913e5a2 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -2,6 +2,7 @@
  * Copyright (c) 2006 Rene Scharfe
  */
 #include "cache.h"
+#include "config.h"
 #include "archive.h"
 #include "streaming.h"
 #include "utf8.h"
diff --git a/archive.c b/archive.c
index b15a922da..60b3035a7 100644
--- a/archive.c
+++ b/archive.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "tree-walk.h"
diff --git a/attr.c b/attr.c
index 7e2134471..9f8b02936 100644
--- a/attr.c
+++ b/attr.c
@@ -9,6 +9,7 @@
 
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "exec_cmd.h"
 #include "attr.h"
 #include "dir.h"
diff --git a/bisect.c b/bisect.c
index d88f9bc3d..c2acb9455 100644
--- a/bisect.c
+++ b/bisect.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/branch.c b/branch.c
index 985316eb7..a8a548ccf 100644
--- a/branch.c
+++ b/branch.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 #include "cache.h"
+#include "config.h"
 #include "branch.h"
 #include "refs.h"
 #include "remote.h"
diff --git a/builtin/add.c b/builtin/add.c
index d9a2491e4..f2415e99f 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -4,6 +4,7 @@
  * Copyright (C) 2006 Linus Torvalds
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "lockfile.h"
 #include "dir.h"
diff --git a/builtin/am.c b/builtin/am.c
index 0f63dcab1..0b6b1265d 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -4,6 +4,7 @@
  * Based on git-am.sh by Junio C Hamano.
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "exec_cmd.h"
 #include "parse-options.h"
diff --git a/builtin/blame.c b/builtin/blame.c
index 1043e5376..c0ae49298 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -6,6 +6,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "builtin.h"
 #include "blob.h"
diff --git a/builtin/branch.c b/builtin/branch.c
index 83fcda43d..c958e9325 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -6,6 +6,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "color.h"
 #include "refs.h"
 #include "commit.h"
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 9af863e79..7bc45f600 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "userdiff.h"
diff --git a/builtin/check-attr.c b/builtin/check-attr.c
index 4d01ca0c8..91444dc04 100644
--- a/builtin/check-attr.c
+++ b/builtin/check-attr.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index c7b8c0889..3e280b9c7 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "quote.h"
 #include "pathspec.h"
diff --git a/builtin/check-mailmap.c b/builtin/check-mailmap.c
index cf0f54f6b..cdce144f3 100644
--- a/builtin/check-mailmap.c
+++ b/builtin/check-mailmap.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "mailmap.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c
index 07631d0c9..39c8be05d 100644
--- a/builtin/checkout-index.c
+++ b/builtin/checkout-index.c
@@ -5,6 +5,7 @@
  *
  */
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "quote.h"
 #include "cache-tree.h"
diff --git a/builtin/checkout.c b/builtin/checkout.c
index a6b2af39d..45fab5a4d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "parse-options.h"
 #include "refs.h"
diff --git a/builtin/clean.c b/builtin/clean.c
index 329b68c40..09adf0551 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -8,6 +8,7 @@
 
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/clone.c b/builtin/clone.c
index 743f16ae2..4922a5496 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -9,6 +9,7 @@
  */
 
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "parse-options.h"
 #include "fetch-pack.h"
diff --git a/builtin/column.c b/builtin/column.c
index 33314b4d7..0c3223d64 100644
--- a/builtin/column.c
+++ b/builtin/column.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "strbuf.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index f39c2b273..a4a923d7c 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "tree.h"
 #include "builtin.h"
diff --git a/builtin/commit.c b/builtin/commit.c
index da1ba4c86..805da4915 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -6,6 +6,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "cache-tree.h"
 #include "color.h"
diff --git a/builtin/config.c b/builtin/config.c
index 7f6c25d4d..753c40a5c 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "color.h"
 #include "parse-options.h"
 #include "urlmatch.h"
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index acb05940f..1d82e61f2 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -5,6 +5,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "builtin.h"
 #include "parse-options.h"
diff --git a/builtin/describe.c b/builtin/describe.c
index 893c8789f..70eb14460 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "commit.h"
 #include "tag.h"
diff --git a/builtin/diff-files.c b/builtin/diff-files.c
index 15c61fd8d..29d3e4f41 100644
--- a/builtin/diff-files.c
+++ b/builtin/diff-files.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "revision.h"
diff --git a/builtin/diff-index.c b/builtin/diff-index.c
index 1af373d00..e3d418361 100644
--- a/builtin/diff-index.c
+++ b/builtin/diff-index.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "revision.h"
diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index 5ea1c1431..9af942b6b 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "log-tree.h"
diff --git a/builtin/diff.c b/builtin/diff.c
index 8c03ddaf5..b85af112a 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "color.h"
 #include "commit.h"
diff --git a/builtin/difftool.c b/builtin/difftool.c
index b9a892f26..9199227f6 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -12,6 +12,7 @@
  * Copyright (C) 2016 Johannes Schindelin
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 24e29ad7e..0dfd980dd 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -5,6 +5,7 @@
  */
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "object.h"
diff --git a/builtin/fetch.c b/builtin/fetch.c
index d4d573b98..284595911 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -2,6 +2,7 @@
  * "git fetch"
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "builtin.h"
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index 70137b0b7..10cbb4341 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c
index eca365bf8..52be99cba 100644
--- a/builtin/for-each-ref.c
+++ b/builtin/for-each-ref.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "object.h"
 #include "parse-options.h"
diff --git a/builtin/fsck.c b/builtin/fsck.c
index cb2ba6cd1..90593491e 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
diff --git a/builtin/gc.c b/builtin/gc.c
index f484eda43..bd91f136f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
 #include "parse-options.h"
diff --git a/builtin/grep.c b/builtin/grep.c
index c6c26e9b9..0f4a1e5a3 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index bbeaf20bc..d04baf999 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -5,6 +5,7 @@
  * Copyright (C) Junio C Hamano, 2005
  */
 #include "builtin.h"
+#include "config.h"
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/help.c b/builtin/help.c
index 49f7a07f8..334a8494a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -2,6 +2,7 @@
  * Builtin help command
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "exec_cmd.h"
 #include "parse-options.h"
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 04b9dcaf0..edc1a91d8 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "delta.h"
 #include "pack.h"
 #include "csum-file.h"
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 8a6acb0ec..47823f9aa 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "builtin.h"
 #include "exec_cmd.h"
diff --git a/builtin/log.c b/builtin/log.c
index a440601ef..72a9fd968 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -5,6 +5,7 @@
  *		 2006 Junio Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "color.h"
 #include "commit.h"
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index b376afc31..c4357dc30 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -6,6 +6,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "quote.h"
 #include "dir.h"
 #include "builtin.h"
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index ee7b293b1..ef965408e 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/builtin/merge-base.c b/builtin/merge-base.c
index 0c36a70ad..6dbd167d3 100644
--- a/builtin/merge-base.c
+++ b/builtin/merge-base.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "diff.h"
diff --git a/builtin/merge-file.c b/builtin/merge-file.c
index 47dde7c39..b08803e61 100644
--- a/builtin/merge-file.c
+++ b/builtin/merge-file.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "xdiff/xdiff.h"
 #include "xdiff-interface.h"
 #include "parse-options.h"
diff --git a/builtin/merge.c b/builtin/merge.c
index a4a098f40..2512370c1 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -7,6 +7,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "builtin.h"
 #include "lockfile.h"
diff --git a/builtin/mv.c b/builtin/mv.c
index 61d20037a..dcf6736b5 100644
--- a/builtin/mv.c
+++ b/builtin/mv.c
@@ -4,6 +4,7 @@
  * Copyright (C) 2006 Johannes Schindelin
  */
 #include "builtin.h"
+#include "config.h"
 #include "pathspec.h"
 #include "lockfile.h"
 #include "dir.h"
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 7fc7e66e8..e21715f1d 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "tag.h"
 #include "refs.h"
diff --git a/builtin/notes.c b/builtin/notes.c
index f2847c41e..231ad1f01 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -8,6 +8,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "notes.h"
 #include "blob.h"
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 80439047a..9fe1fa8e1 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "object.h"
 #include "blob.h"
diff --git a/builtin/patch-id.c b/builtin/patch-id.c
index 81552e02e..970d0d30b 100644
--- a/builtin/patch-id.c
+++ b/builtin/patch-id.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 
 static void flush_current_id(int patchlen, struct object_id *id, struct object_id *result)
 {
diff --git a/builtin/pull.c b/builtin/pull.c
index 318c273eb..af325d9fc 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -6,6 +6,7 @@
  * Fetch one or more remote refs and merge it/them into the current HEAD.
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "exec_cmd.h"
diff --git a/builtin/push.c b/builtin/push.c
index a597759d8..76aa713d2 100644
--- a/builtin/push.c
+++ b/builtin/push.c
@@ -2,6 +2,7 @@
  * "git push"
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "run-command.h"
 #include "builtin.h"
diff --git a/builtin/read-tree.c b/builtin/read-tree.c
index 78d319365..6eb2a0e57 100644
--- a/builtin/read-tree.c
+++ b/builtin/read-tree.c
@@ -5,6 +5,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "object.h"
 #include "tree.h"
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ca1ebb2fa..c82b4dce6 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "sequencer.h"
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b1706a573..71c0c768d 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "pack.h"
 #include "refs.h"
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 920c16dac..44cdc2dbd 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "commit.h"
 #include "refs.h"
diff --git a/builtin/remote.c b/builtin/remote.c
index addf97ad2..a470ed7c6 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "parse-options.h"
 #include "transport.h"
 #include "remote.h"
diff --git a/builtin/repack.c b/builtin/repack.c
index 38ba4ef82..f17a68a17 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "parse-options.h"
 #include "run-command.h"
diff --git a/builtin/replace.c b/builtin/replace.c
index c921bc976..80a15cf35 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -9,6 +9,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "refs.h"
 #include "parse-options.h"
diff --git a/builtin/rerere.c b/builtin/rerere.c
index 1bf72423b..ffb66e290 100644
--- a/builtin/rerere.c
+++ b/builtin/rerere.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/reset.c b/builtin/reset.c
index 430602d10..d0b604103 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -8,6 +8,7 @@
  * Copyright (c) 2005, 2006 Linus Torvalds and Junio C Hamano
  */
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "tag.h"
 #include "object.h"
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 718c6059c..f8e2b7bf5 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index efdc14473..c78b7b33d 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "quote.h"
diff --git a/builtin/revert.c b/builtin/revert.c
index 345d9586a..16028b9ea 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "diff.h"
diff --git a/builtin/rm.c b/builtin/rm.c
index 7c323d012..824d1de9e 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds 2006
  */
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "dir.h"
 #include "cache-tree.h"
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index b8e2e74fe..633e0c3cd 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "pkt-line.h"
diff --git a/builtin/shortlog.c b/builtin/shortlog.c
index 7cff1839f..43c4799ea 100644
--- a/builtin/shortlog.c
+++ b/builtin/shortlog.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "diff.h"
 #include "string-list.h"
diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index 4a6cc6f49..e4cf1b5bb 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "builtin.h"
diff --git a/builtin/stripspace.c b/builtin/stripspace.c
index 1e62a008c..bdf032886 100644
--- a/builtin/stripspace.c
+++ b/builtin/stripspace.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "strbuf.h"
 
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 8cc648d85..4dcbfb952 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "quote.h"
 #include "pathspec.h"
diff --git a/builtin/symbolic-ref.c b/builtin/symbolic-ref.c
index 70addef15..df75cb9d4 100644
--- a/builtin/symbolic-ref.c
+++ b/builtin/symbolic-ref.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "cache.h"
 #include "refs.h"
 #include "parse-options.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 1f74a56db..01154ea8d 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -7,6 +7,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "refs.h"
 #include "tag.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index 6fc6bcdf7..73f133419 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 
 static char *create_temp_file(unsigned char *sha1)
 {
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 8bc999776..193f8b9d5 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "object.h"
 #include "delta.h"
 #include "pack.h"
diff --git a/builtin/update-index.c b/builtin/update-index.c
index ebfc09faa..0a4c23648 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "quote.h"
 #include "cache-tree.h"
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 0b2ecf41a..40ccfc193 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "builtin.h"
 #include "parse-options.h"
diff --git a/builtin/update-server-info.c b/builtin/update-server-info.c
index 6c8cc3edc..873070e51 100644
--- a/builtin/update-server-info.c
+++ b/builtin/update-server-info.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 
diff --git a/builtin/var.c b/builtin/var.c
index aedbb53a2..6c6f46b4a 100644
--- a/builtin/var.c
+++ b/builtin/var.c
@@ -4,6 +4,7 @@
  * Copyright (C) Eric Biederman, 2005
  */
 #include "builtin.h"
+#include "config.h"
 
 static const char var_usage[] = "git var (-l | <variable>)";
 
diff --git a/builtin/verify-commit.c b/builtin/verify-commit.c
index 05b734e6d..ba38ac9b1 100644
--- a/builtin/verify-commit.c
+++ b/builtin/verify-commit.c
@@ -6,6 +6,7 @@
  * Based on git-verify-tag
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "commit.h"
 #include "run-command.h"
diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index c94e15693..c2a1a5c50 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "parse-options.h"
 
diff --git a/builtin/verify-tag.c b/builtin/verify-tag.c
index 5199553d9..f9a5f7535 100644
--- a/builtin/verify-tag.c
+++ b/builtin/verify-tag.c
@@ -6,6 +6,7 @@
  * Based on git-verify-tag.sh
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "tag.h"
 #include "run-command.h"
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 793306ea5..0c5476ee9 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "dir.h"
 #include "parse-options.h"
diff --git a/builtin/write-tree.c b/builtin/write-tree.c
index 084c0df78..bd0a78aa3 100644
--- a/builtin/write-tree.c
+++ b/builtin/write-tree.c
@@ -5,6 +5,7 @@
  */
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "tree.h"
 #include "cache-tree.h"
 #include "parse-options.h"
diff --git a/cache.h b/cache.h
index a27fee458..272f8e021 100644
--- a/cache.h
+++ b/cache.h
@@ -11,7 +11,6 @@
 #include "string-list.h"
 #include "pack-revindex.h"
 #include "hash.h"
-#include "config.h"
 
 #ifndef platform_SHA_CTX
 /*
diff --git a/color.c b/color.c
index dee61557e..31b6207a0 100644
--- a/color.c
+++ b/color.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "color.h"
 
 static int git_use_color_default = GIT_COLOR_AUTO;
diff --git a/column.c b/column.c
index d55ead18e..ff7bdab1a 100644
--- a/column.c
+++ b/column.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "column.h"
 #include "string-list.h"
 #include "parse-options.h"
diff --git a/config.c b/config.c
index 146cb3452..2390f98e3 100644
--- a/config.c
+++ b/config.c
@@ -6,6 +6,7 @@
  *
  */
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "exec_cmd.h"
 #include "strbuf.h"
diff --git a/connect.c b/connect.c
index cd21a1b6f..efddb30ea 100644
--- a/connect.c
+++ b/connect.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 #include "cache.h"
+#include "config.h"
 #include "pkt-line.h"
 #include "quote.h"
 #include "refs.h"
diff --git a/convert.c b/convert.c
index f1e168bc3..5f4a4b1f5 100644
--- a/convert.c
+++ b/convert.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "run-command.h"
 #include "quote.h"
diff --git a/credential-cache--daemon.c b/credential-cache--daemon.c
index f3814cc47..0d5c62509 100644
--- a/credential-cache--daemon.c
+++ b/credential-cache--daemon.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "tempfile.h"
 #include "credential.h"
 #include "unix-socket.h"
diff --git a/credential.c b/credential.c
index aa996669f..67a523353 100644
--- a/credential.c
+++ b/credential.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "credential.h"
 #include "string-list.h"
 #include "run-command.h"
diff --git a/daemon.c b/daemon.c
index ac7181a48..30747075f 100644
--- a/daemon.c
+++ b/daemon.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "pkt-line.h"
 #include "run-command.h"
 #include "strbuf.h"
diff --git a/diff.c b/diff.c
index f3546536b..2f2467c6d 100644
--- a/diff.c
+++ b/diff.c
@@ -2,6 +2,7 @@
  * Copyright (C) 2005 Junio C Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "tempfile.h"
 #include "quote.h"
 #include "diff.h"
diff --git a/dir.c b/dir.c
index 3f3167e55..0c26a53d2 100644
--- a/dir.c
+++ b/dir.c
@@ -9,6 +9,7 @@
  */
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "attr.h"
 #include "refs.h"
diff --git a/environment.c b/environment.c
index aa478e71d..d40b21fb7 100644
--- a/environment.c
+++ b/environment.c
@@ -8,6 +8,7 @@
  * are.
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
diff --git a/fast-import.c b/fast-import.c
index e69d21968..2881d4898 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -154,6 +154,7 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "object.h"
 #include "blob.h"
diff --git a/fetch-pack.c b/fetch-pack.c
index cd86865be..fbbc99c88 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "refs.h"
 #include "pkt-line.h"
diff --git a/git.c b/git.c
index 8ff44f081..594436e43 100644
--- a/git.c
+++ b/git.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "exec_cmd.h"
 #include "help.h"
 #include "run-command.h"
diff --git a/gpg-interface.c b/gpg-interface.c
index e44cc27da..8ab32df45 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "gpg-interface.h"
diff --git a/graph.c b/graph.c
index 8b9049dd2..e7e20650d 100644
--- a/graph.c
+++ b/graph.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "color.h"
 #include "graph.h"
diff --git a/grep.c b/grep.c
index 47cee4506..d5211fc5a 100644
--- a/grep.c
+++ b/grep.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "grep.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
diff --git a/help.c b/help.c
index db7f3d79a..5d44f811a 100644
--- a/help.c
+++ b/help.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "exec_cmd.h"
 #include "run-command.h"
diff --git a/http-backend.c b/http-backend.c
index ba5ff1aa2..519025d2c 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "object.h"
diff --git a/http-fetch.c b/http-fetch.c
index 3b556d661..8af380050 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "exec_cmd.h"
 #include "http.h"
 #include "walker.h"
diff --git a/http.c b/http.c
index d2e11ec6f..013bb0cc6 100644
--- a/http.c
+++ b/http.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 #include "http.h"
+#include "config.h"
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/ident.c b/ident.c
index bea871c8e..d41fc9119 100644
--- a/ident.c
+++ b/ident.c
@@ -6,6 +6,7 @@
  * Copyright (C) 2005 Linus Torvalds
  */
 #include "cache.h"
+#include "config.h"
 
 static struct strbuf git_default_name = STRBUF_INIT;
 static struct strbuf git_default_email = STRBUF_INIT;
diff --git a/imap-send.c b/imap-send.c
index 857591660..59e9b12d2 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -23,6 +23,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "credential.h"
 #include "exec_cmd.h"
 #include "run-command.h"
diff --git a/ll-merge.c b/ll-merge.c
index ac0d4a5d7..24ff94e1d 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -5,6 +5,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "xdiff-interface.h"
 #include "run-command.h"
diff --git a/log-tree.c b/log-tree.c
index a4ec11c2b..7e1dfd749 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "tag.h"
diff --git a/mailinfo.c b/mailinfo.c
index f92cb9f72..f59162453 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "utf8.h"
 #include "strbuf.h"
 #include "mailinfo.h"
diff --git a/merge-recursive.c b/merge-recursive.c
index ae5238d82..c2494f34f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -4,6 +4,7 @@
  * The thieves were Alex Riesen and Johannes Schindelin, in June/July 2006
  */
 #include "cache.h"
+#include "config.h"
 #include "advice.h"
 #include "lockfile.h"
 #include "cache-tree.h"
diff --git a/notes-utils.c b/notes-utils.c
index 325ff3daa..ebde2273c 100644
--- a/notes-utils.c
+++ b/notes-utils.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "notes-utils.h"
diff --git a/notes.c b/notes.c
index 542563b28..dbcfef4d7 100644
--- a/notes.c
+++ b/notes.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "notes.h"
 #include "blob.h"
 #include "tree.h"
diff --git a/pager.c b/pager.c
index c113d898a..4dd9e1b26 100644
--- a/pager.c
+++ b/pager.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "sigchain.h"
 
diff --git a/parse-options.c b/parse-options.c
index a23a1e67f..cbf84a604 100644
--- a/parse-options.c
+++ b/parse-options.c
@@ -1,6 +1,7 @@
 #include "git-compat-util.h"
 #include "parse-options.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "color.h"
 #include "utf8.h"
diff --git a/pathspec.c b/pathspec.c
index 828405021..ecc5331c2 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -1,5 +1,6 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "pathspec.h"
 #include "attr.h"
diff --git a/pretty.c b/pretty.c
index 09701bd2f..9c9f81b5b 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "utf8.h"
 #include "diff.h"
diff --git a/prompt.c b/prompt.c
index 75406390c..6d5885d00 100644
--- a/prompt.c
+++ b/prompt.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "prompt.h"
diff --git a/read-cache.c b/read-cache.c
index 92eb15c00..2650ad98a 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -5,6 +5,7 @@
  */
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
 #include "cache-tree.h"
diff --git a/refs.c b/refs.c
index 8af9641aa..7dfdc08bf 100644
--- a/refs.c
+++ b/refs.c
@@ -3,6 +3,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "hashmap.h"
 #include "lockfile.h"
 #include "iterator.h"
diff --git a/refs/files-backend.c b/refs/files-backend.c
index cb1f528cb..f88b94484 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1,4 +1,5 @@
 #include "../cache.h"
+#include "../config.h"
 #include "../refs.h"
 #include "refs-internal.h"
 #include "ref-cache.h"
diff --git a/remote-curl.c b/remote-curl.c
index ece45993d..0053b0954 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "remote.h"
 #include "strbuf.h"
 #include "walker.h"
diff --git a/remote.c b/remote.c
index fdc52d802..a84e68767 100644
--- a/remote.c
+++ b/remote.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "remote.h"
 #include "refs.h"
 #include "commit.h"
diff --git a/rerere.c b/rerere.c
index 3bd55caf3..344d6aa81 100644
--- a/rerere.c
+++ b/rerere.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "string-list.h"
 #include "rerere.h"
diff --git a/send-pack.c b/send-pack.c
index 78bb34ebe..ed3cee321 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "pkt-line.h"
diff --git a/sequencer.c b/sequencer.c
index a23b948ac..b9962dc89 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "sequencer.h"
 #include "dir.h"
diff --git a/setup.c b/setup.c
index e3f7699a9..e99a82cbe 100644
--- a/setup.c
+++ b/setup.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "string-list.h"
 
diff --git a/sha1_file.c b/sha1_file.c
index 59a4ed2ed..44561e0b9 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -7,6 +7,7 @@
  * creation etc.
  */
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 #include "lockfile.h"
 #include "delta.h"
diff --git a/sha1_name.c b/sha1_name.c
index 389276e9d..0ad85ec54 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "tag.h"
 #include "commit.h"
 #include "tree.h"
diff --git a/submodule-config.c b/submodule-config.c
index 4f58491dd..d8f8d5ea3 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
 #include "strbuf.h"
diff --git a/submodule.c b/submodule.c
index bf5a93d16..95328de61 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
 #include "dir.h"
diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 8e3ed6a76..1a7b8bd3d 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 
 /*
diff --git a/t/helper/test-submodule-config.c b/t/helper/test-submodule-config.c
index 2f144d539..c6c57bba0 100644
--- a/t/helper/test-submodule-config.c
+++ b/t/helper/test-submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
 
diff --git a/trailer.c b/trailer.c
index 11f0b9fb4..751b56c00 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
diff --git a/transport.c b/transport.c
index 9bfcf870f..b9995306f 100644
--- a/transport.c
+++ b/transport.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "transport.h"
 #include "run-command.h"
 #include "pkt-line.h"
diff --git a/unpack-trees.c b/unpack-trees.c
index d38c37e38..dd535bc84 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1,5 +1,6 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/upload-pack.c b/upload-pack.c
index 5330c02c1..7efff2fbf 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "sideband.h"
diff --git a/userdiff.c b/userdiff.c
index 8b732e40b..2c1502f71 100644
--- a/userdiff.c
+++ b/userdiff.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "userdiff.h"
 #include "attr.h"
 
diff --git a/versioncmp.c b/versioncmp.c
index 9f81dc106..069ee94a4 100644
--- a/versioncmp.c
+++ b/versioncmp.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 
 /*
diff --git a/wrapper.c b/wrapper.c
index d83741770..487a9f753 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -2,6 +2,7 @@
  * Various trivial helper wrappers around standard functions
  */
 #include "cache.h"
+#include "config.h"
 
 static void do_nothing(size_t size)
 {
diff --git a/xdiff-interface.c b/xdiff-interface.c
index 060038c2d..5ac07d734 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "xdiff-interface.h"
 #include "xdiff/xtypes.h"
 #include "xdiff/xdiffi.h"
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 03/31] config: don't implicitly use gitdir
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
  2017-05-31 21:43 ` [PATCH 01/31] config: create config.h Brandon Williams
  2017-05-31 21:43 ` [PATCH 02/31] config: don't include config.h by default Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-06-08 10:41   ` Johannes Schindelin
  2017-05-31 21:43 ` [PATCH 04/31] setup: don't perform lazy initialization of repository state Brandon Williams
                   ` (30 subsequent siblings)
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Commit 2185fde56 (config: handle conditional include when $GIT_DIR is
not set up) added a 'git_dir' field to the config_options struct.  Let's
use this option field explicitly all the time instead of occasionally
falling back to calling 'git_pathdup("config")' to get the path to the
local repository configuration.  This allows 'go_git_config_sequence()'
to not implicitly rely on global repository state.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/config.c | 2 ++
 config.c         | 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/builtin/config.c b/builtin/config.c
index 753c40a5c..90f49a6ee 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -539,6 +539,8 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		config_options.respect_includes = !given_config_source.file;
 	else
 		config_options.respect_includes = respect_includes_opt;
+	if (have_git_dir())
+		config_options.git_dir = get_git_common_dir();
 
 	if (end_null) {
 		term = '\0';
diff --git a/config.c b/config.c
index 2390f98e3..ff09b27b8 100644
--- a/config.c
+++ b/config.c
@@ -1548,8 +1548,6 @@ static int do_git_config_sequence(const struct config_options *opts,
 
 	if (opts->git_dir)
 		repo_config = mkpathdup("%s/config", opts->git_dir);
-	else if (have_git_dir())
-		repo_config = git_pathdup("config");
 	else
 		repo_config = NULL;
 
@@ -1613,6 +1611,8 @@ static void git_config_raw(config_fn_t fn, void *data)
 	struct config_options opts = {0};
 
 	opts.respect_includes = 1;
+	if (have_git_dir())
+		opts.git_dir = get_git_common_dir();
 	if (git_config_with_options(fn, data, NULL, &opts) < 0)
 		/*
 		 * git_config_with_options() normally returns only
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 04/31] setup: don't perform lazy initialization of repository state
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (2 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 03/31] config: don't implicitly use gitdir Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-06-01 19:23   ` Stefan Beller
  2017-05-31 21:43 ` [PATCH 05/31] environment: remove namespace_len variable Brandon Williams
                   ` (29 subsequent siblings)
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Under some circumstances (bogus GIT_DIR value or the discovered gitdir
is '.git') 'setup_git_directory()' won't initialize key repository
state.  This leads to inconsistent state after running the setup code.
To account for this inconsistent state, lazy initialization is done once
a caller asks for the repository's gitdir or some other piece of
repository state.  This is confusing and can be error prone.

Instead let's tighten the expected outcome of 'setup_git_directory()'
and ensure that it initializes repository state in all cases that would
have been handled by lazy initialization.

This also lets us drop the requirement to have 'have_git_dir()' check if
the environment variable GIT_DIR was set as that will be handled by the
end of the setup code.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  2 ++
 environment.c | 17 ++++++++---------
 setup.c       | 14 ++++++++++++++
 3 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/cache.h b/cache.h
index 272f8e021..d41aab82f 100644
--- a/cache.h
+++ b/cache.h
@@ -462,6 +462,8 @@ static inline enum object_type object_type(unsigned int mode)
  */
 extern const char * const local_repo_env[];
 
+extern void setup_git_env(void);
+
 /*
  * Returns true iff we have a configured git repository (either via
  * setup_git_directory, or in the environment via $GIT_DIR).
diff --git a/environment.c b/environment.c
index d40b21fb7..a73b08f5d 100644
--- a/environment.c
+++ b/environment.c
@@ -160,7 +160,7 @@ static char *git_path_from_env(const char *envvar, const char *git_dir,
 	return xstrdup(value);
 }
 
-static void setup_git_env(void)
+void setup_git_env(void)
 {
 	struct strbuf sb = STRBUF_INIT;
 	const char *gitfile;
@@ -205,28 +205,27 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir
-		|| getenv(GIT_DIR_ENVIRONMENT);
+		|| git_dir;
 }
 
 const char *get_git_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_dir;
 }
 
 const char *get_git_common_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_common_dir;
 }
 
 const char *get_git_namespace(void)
 {
 	if (!namespace)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return namespace;
 }
 
@@ -276,7 +275,7 @@ const char *get_git_work_tree(void)
 char *get_object_directory(void)
 {
 	if (!git_object_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_object_dir;
 }
 
@@ -316,14 +315,14 @@ int odb_pack_keep(const char *name)
 char *get_index_file(void)
 {
 	if (!git_index_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_index_file;
 }
 
 char *get_graft_file(void)
 {
 	if (!git_graft_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_graft_file;
 }
 
diff --git a/setup.c b/setup.c
index e99a82cbe..b2e05145c 100644
--- a/setup.c
+++ b/setup.c
@@ -1063,6 +1063,20 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	startup_info->have_repository = !nongit_ok || !*nongit_ok;
 	startup_info->prefix = prefix;
 
+	/*
+	 * Not all paths through the setup code will call 'set_git_dir()' (which
+	 * directly sets up the environment) so in order to guarantee that the
+	 * environment is in a consistent state after setup, explicitly setup
+	 * the environment if we have a repository.
+	 *
+	 * NEEDSWORK: currently we allow bogus GIT_DIR values to be set in some
+	 * code paths so we also need to explicitly setup the environment if
+	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
+	 * GIT_DIR values at some point in the future.
+	 */
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
+		setup_git_env();
+
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 05/31] environment: remove namespace_len variable
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (3 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 04/31] setup: don't perform lazy initialization of repository state Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-06-01 19:28   ` Stefan Beller
  2017-05-31 21:43 ` [PATCH 06/31] repo: introduce the repository object Brandon Williams
                   ` (28 subsequent siblings)
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Use 'skip_prefix' instead of 'starts_with' so that we can drop the need
to keep around 'namespace_len'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index a73b08f5d..e035f6372 100644
--- a/environment.c
+++ b/environment.c
@@ -98,7 +98,6 @@ char *git_work_tree_cfg;
 static char *work_tree;
 
 static const char *namespace;
-static size_t namespace_len;
 
 static const char *super_prefix;
 
@@ -190,7 +189,6 @@ void setup_git_env(void)
 	git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
 							  : "refs/replace/");
 	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
-	namespace_len = strlen(namespace);
 	shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
 	if (shallow_file)
 		set_alternate_shallow_file(shallow_file, 0);
@@ -231,9 +229,10 @@ const char *get_git_namespace(void)
 
 const char *strip_namespace(const char *namespaced_ref)
 {
-	if (!starts_with(namespaced_ref, get_git_namespace()))
-		return NULL;
-	return namespaced_ref + namespace_len;
+	const char *out;
+	if (skip_prefix(namespaced_ref, get_git_namespace(), &out))
+		return out;
+	return NULL;
 }
 
 const char *get_super_prefix(void)
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 06/31] repo: introduce the repository object
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (4 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 05/31] environment: remove namespace_len variable Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-06-01 19:53   ` Stefan Beller
  2017-05-31 21:43 ` [PATCH 07/31] environment: place key repository state in the_repository Brandon Williams
                   ` (27 subsequent siblings)
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce the repository object 'struct repo' which can be used hold all
state pertaining to a git repository.

The aim of object-ifying the repository is to (1) make the code base
more readable and easier to reason about and (2) allow for working on
multiple repositories, specifically submodules, within the same process.

TODO: Add more motivating points for adding a repository object?

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 Makefile      |   1 +
 cache.h       |   1 +
 environment.c |   2 +-
 repo.c        | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 repo.h        |  25 ++++++++++++
 5 files changed, 152 insertions(+), 1 deletion(-)
 create mode 100644 repo.c
 create mode 100644 repo.h

diff --git a/Makefile b/Makefile
index 2ed6db728..3d3d556ef 100644
--- a/Makefile
+++ b/Makefile
@@ -821,6 +821,7 @@ LIB_OBJS += refs/ref-cache.o
 LIB_OBJS += ref-filter.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace_object.o
+LIB_OBJS += repo.o
 LIB_OBJS += rerere.o
 LIB_OBJS += resolve-undo.o
 LIB_OBJS += revision.o
diff --git a/cache.h b/cache.h
index d41aab82f..c0b4c8d83 100644
--- a/cache.h
+++ b/cache.h
@@ -483,6 +483,7 @@ extern char *get_graft_file(void);
 extern int set_git_dir(const char *path);
 extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);
 extern int get_common_dir(struct strbuf *sb, const char *gitdir);
+extern char *expand_namespace(const char *raw_namespace);
 extern const char *get_git_namespace(void);
 extern const char *strip_namespace(const char *namespaced_ref);
 extern const char *get_super_prefix(void);
diff --git a/environment.c b/environment.c
index e035f6372..77900b31a 100644
--- a/environment.c
+++ b/environment.c
@@ -127,7 +127,7 @@ const char * const local_repo_env[] = {
 	NULL
 };
 
-static char *expand_namespace(const char *raw_namespace)
+char *expand_namespace(const char *raw_namespace)
 {
 	struct strbuf buf = STRBUF_INIT;
 	struct strbuf **components, **c;
diff --git a/repo.c b/repo.c
new file mode 100644
index 000000000..f7c3617a9
--- /dev/null
+++ b/repo.c
@@ -0,0 +1,124 @@
+#include "cache.h"
+#include "repo.h"
+
+/*
+ * This may be the wrong place for this.
+ * It may be better to go in env.c or setup for the time being?
+ */
+struct repo the_repository;
+
+static char *git_path_from_env(const char *envvar, const char *git_dir,
+			       const char *path, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(envvar);
+		if (value)
+			return xstrdup(value);
+	}
+
+	return xstrfmt("%s/%s", git_dir, path);
+}
+
+static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
+		if (value) {
+			strbuf_addstr(sb, value);
+			return 1;
+		}
+	}
+
+	return get_common_dir_noenv(sb, gitdir);
+}
+
+/* called after setting gitdir */
+static void repo_setup_env(struct repo *repo)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	if (!repo->gitdir)
+		BUG("gitdir wasn't set before setting up the environment");
+
+	repo->different_commondir = find_common_dir(&sb, repo->gitdir,
+						    !repo->ignore_env);
+	repo->commondir = strbuf_detach(&sb, NULL);
+	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+					    "objects", !repo->ignore_env);
+	repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
+					     "index", !repo->ignore_env);
+	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
+					     "info/grafts", !repo->ignore_env);
+	repo->namespace = expand_namespace(repo->ignore_env ? NULL :
+					   getenv(GIT_NAMESPACE_ENVIRONMENT));
+}
+
+static void repo_clear_env(struct repo *repo)
+{
+	free(repo->gitdir);
+	repo->gitdir = NULL;
+	free(repo->commondir);
+	repo->commondir = NULL;
+	free(repo->objectdir);
+	repo->objectdir = NULL;
+	free(repo->index_file);
+	repo->index_file = NULL;
+	free(repo->graft_file);
+	repo->graft_file = NULL;
+	free(repo->namespace);
+	repo->namespace = NULL;
+}
+
+void repo_set_gitdir(struct repo *repo, const char *path)
+{
+	const char *gitfile = read_gitfile(path);
+
+	/*
+	 * NEEDSWORK: Eventually we want to be able to free gitdir and the rest
+	 * of the environment before reinitializing it again, but we have some
+	 * crazy code paths where we try to set gitdir with the current gitdir
+	 * and we don't want to free gitdir before copying the passed in value.
+	 */
+	repo->gitdir = xstrdup(gitfile ? gitfile : path);
+
+	repo_setup_env(repo);
+}
+
+int repo_init(struct repo *repo, const char *gitdir)
+{
+	int error = 0;
+	char *abspath = real_pathdup(gitdir, 1);
+	char *suspect = xstrfmt("%s/.git", abspath);
+	struct strbuf sb = STRBUF_INIT;
+	const char *resolved_gitdir;
+
+	memset(repo, 0, sizeof(struct repo));
+
+	repo->ignore_env = 1;
+
+	/* First assume 'gitdir' references the gitdir directly */
+	resolved_gitdir = resolve_gitdir_gently(abspath, &error);
+	/* otherwise; try 'gitdir'.git */
+	if (!resolved_gitdir) {
+		resolved_gitdir = resolve_gitdir_gently(suspect, &error);
+		if (!resolved_gitdir) {
+			die("'%s' is not a repository\n", abspath);
+			return error;
+		}
+	}
+
+	repo_set_gitdir(repo, resolved_gitdir);
+
+	/* NEEDSWORK: Verify repository format version */
+
+	free(abspath);
+	free(suspect);
+	strbuf_release(&sb);
+
+	return error;
+}
+
+void repo_clear(struct repo *repo)
+{
+	repo_clear_env(repo);
+}
diff --git a/repo.h b/repo.h
new file mode 100644
index 000000000..453049e67
--- /dev/null
+++ b/repo.h
@@ -0,0 +1,25 @@
+#ifndef REPO_H
+#define REPO_H
+
+struct repo {
+	/* Environment */
+	char *gitdir;
+	char *commondir;
+	char *objectdir;
+	char *index_file;
+	char *graft_file;
+	char *namespace;
+
+	/* Configurations */
+	unsigned ignore_env:1;
+	/* Indicates if a repository has a different 'commondir' from 'gitdir' */
+	unsigned different_commondir:1;
+};
+
+extern struct repo the_repository;
+
+extern void repo_set_gitdir(struct repo *repo, const char *path);
+extern int repo_init(struct repo *repo, const char *path);
+extern void repo_clear(struct repo *repo);
+
+#endif /* REPO_H */
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 07/31] environment: place key repository state in the_repository
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (5 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 06/31] repo: introduce the repository object Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 08/31] environment: store worktree " Brandon Williams
                   ` (26 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'git_dir', 'git_common_dir', 'git_object_dir', 'git_index_file',
'git_graft_file', and 'namespace' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  1 -
 environment.c | 65 ++++++++++++++---------------------------------------------
 path.c        | 11 +++++-----
 setup.c       | 17 ++++++++++++++--
 4 files changed, 36 insertions(+), 58 deletions(-)

diff --git a/cache.h b/cache.h
index c0b4c8d83..02ab5f801 100644
--- a/cache.h
+++ b/cache.h
@@ -769,7 +769,6 @@ extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
 extern int protect_hfs;
 extern int protect_ntfs;
-extern int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
 
 /*
  * Include broken refs in all ref iterations, which will
diff --git a/environment.c b/environment.c
index 77900b31a..a0519f4f3 100644
--- a/environment.c
+++ b/environment.c
@@ -8,6 +8,7 @@
  * are.
  */
 #include "cache.h"
+#include "repo.h"
 #include "config.h"
 #include "refs.h"
 #include "fmt-merge-msg.h"
@@ -97,14 +98,8 @@ int ignore_untracked_cache_config;
 char *git_work_tree_cfg;
 static char *work_tree;
 
-static const char *namespace;
-
 static const char *super_prefix;
 
-static const char *git_dir, *git_common_dir;
-static char *git_object_dir, *git_index_file, *git_graft_file;
-int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
-
 /*
  * Repository-local GIT_* environment variables; see cache.h for details.
  */
@@ -148,47 +143,16 @@ char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-static char *git_path_from_env(const char *envvar, const char *git_dir,
-			       const char *path, int *fromenv)
-{
-	const char *value = getenv(envvar);
-	if (!value)
-		return xstrfmt("%s/%s", git_dir, path);
-	if (fromenv)
-		*fromenv = 1;
-	return xstrdup(value);
-}
-
 void setup_git_env(void)
 {
-	struct strbuf sb = STRBUF_INIT;
-	const char *gitfile;
 	const char *shallow_file;
 	const char *replace_ref_base;
 
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
-	if (!git_dir) {
-		if (!startup_info->have_repository)
-			BUG("setup_git_env called without repository");
-		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
-	}
-	gitfile = read_gitfile(git_dir);
-	git_dir = xstrdup(gitfile ? gitfile : git_dir);
-	if (get_common_dir(&sb, git_dir))
-		git_common_dir_env = 1;
-	git_common_dir = strbuf_detach(&sb, NULL);
-	git_object_dir = git_path_from_env(DB_ENVIRONMENT, git_common_dir,
-					   "objects", &git_db_env);
-	git_index_file = git_path_from_env(INDEX_ENVIRONMENT, git_dir,
-					   "index", &git_index_env);
-	git_graft_file = git_path_from_env(GRAFT_ENVIRONMENT, git_common_dir,
-					   "info/grafts", &git_graft_env);
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		check_replace_refs = 0;
 	replace_ref_base = getenv(GIT_REPLACE_REF_BASE_ENVIRONMENT);
 	git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
 							  : "refs/replace/");
-	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
 	shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
 	if (shallow_file)
 		set_alternate_shallow_file(shallow_file, 0);
@@ -203,28 +167,28 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir;
+		|| the_repository.gitdir;
 }
 
 const char *get_git_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository.gitdir)
 		BUG("git environment hasn't been setup");
-	return git_dir;
+	return the_repository.gitdir;
 }
 
 const char *get_git_common_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository.commondir)
 		BUG("git environment hasn't been setup");
-	return git_common_dir;
+	return the_repository.commondir;
 }
 
 const char *get_git_namespace(void)
 {
-	if (!namespace)
+	if (!the_repository.namespace)
 		BUG("git environment hasn't been setup");
-	return namespace;
+	return the_repository.namespace;
 }
 
 const char *strip_namespace(const char *namespaced_ref)
@@ -273,9 +237,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!git_object_dir)
+	if (!the_repository.objectdir)
 		BUG("git environment hasn't been setup");
-	return git_object_dir;
+	return the_repository.objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
@@ -313,22 +277,23 @@ int odb_pack_keep(const char *name)
 
 char *get_index_file(void)
 {
-	if (!git_index_file)
+	if (!the_repository.index_file)
 		BUG("git environment hasn't been setup");
-	return git_index_file;
+	return the_repository.index_file;
 }
 
 char *get_graft_file(void)
 {
-	if (!git_graft_file)
+	if (!the_repository.graft_file)
 		BUG("git environment hasn't been setup");
-	return git_graft_file;
+	return the_repository.graft_file;
 }
 
 int set_git_dir(const char *path)
 {
 	if (setenv(GIT_DIR_ENVIRONMENT, path, 1))
 		return error("Could not set GIT_DIR to '%s'", path);
+	repo_set_gitdir(&the_repository, path);
 	setup_git_env();
 	return 0;
 }
diff --git a/path.c b/path.c
index c1cb1cf62..a1a7c2653 100644
--- a/path.c
+++ b/path.c
@@ -2,6 +2,7 @@
  * Utilities for paths and pathnames
  */
 #include "cache.h"
+#include "repo.h"
 #include "strbuf.h"
 #include "string-list.h"
 #include "dir.h"
@@ -355,7 +356,7 @@ void report_linked_checkout_garbage(void)
 	const struct common_dir *p;
 	int len;
 
-	if (!git_common_dir_env)
+	if (!the_repository.different_commondir)
 		return;
 	strbuf_addf(&sb, "%s/", get_git_dir());
 	len = sb.len;
@@ -374,17 +375,17 @@ void report_linked_checkout_garbage(void)
 static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
-	if (git_graft_env && is_dir_file(base, "info", "grafts"))
+	if (is_dir_file(base, "info", "grafts"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_graft_file(), strlen(get_graft_file()));
-	else if (git_index_env && !strcmp(base, "index"))
+	else if (!strcmp(base, "index"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_index_file(), strlen(get_index_file()));
-	else if (git_db_env && dir_prefix(base, "objects"))
+	else if (dir_prefix(base, "objects"))
 		replace_dir(buf, git_dir_len + 7, get_object_directory());
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
-	else if (git_common_dir_env)
+	else if (the_repository.different_commondir)
 		update_common_dir(buf, git_dir_len, NULL);
 }
 
diff --git a/setup.c b/setup.c
index b2e05145c..dba3abd00 100644
--- a/setup.c
+++ b/setup.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repo.h"
 #include "config.h"
 #include "dir.h"
 #include "string-list.h"
@@ -376,6 +377,11 @@ void setup_work_tree(void)
 	if (getenv(GIT_WORK_TREE_ENVIRONMENT))
 		setenv(GIT_WORK_TREE_ENVIRONMENT, ".", 1);
 
+	/*
+	 * NEEDSWORK: this call can essentially be set_git_dir(get_git_dir())
+	 * which can cause some problems when trying to free the old value of
+	 * gitdir.
+	 */
 	set_git_dir(remove_leading_path(git_dir, work_tree));
 	initialized = 1;
 }
@@ -1074,8 +1080,15 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
 	 * GIT_DIR values at some point in the future.
 	 */
-	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
-		setup_git_env();
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT)) {
+		if (!the_repository.gitdir) {
+			const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);
+			if (!gitdir)
+				gitdir = DEFAULT_GIT_DIR_ENVIRONMENT;
+			repo_set_gitdir(&the_repository, gitdir);
+			setup_git_env();
+		}
+	}
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 08/31] environment: store worktree in the_repository
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (6 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 07/31] environment: place key repository state in the_repository Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 09/31] setup: add comment indicating a hack Brandon Williams
                   ` (25 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'work_tree' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c | 9 ++++-----
 repo.c        | 7 +++++++
 repo.h        | 2 ++
 3 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index a0519f4f3..2722ebb9f 100644
--- a/environment.c
+++ b/environment.c
@@ -96,7 +96,6 @@ int ignore_untracked_cache_config;
 
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
 char *git_work_tree_cfg;
-static char *work_tree;
 
 static const char *super_prefix;
 
@@ -220,19 +219,19 @@ void set_git_work_tree(const char *new_work_tree)
 {
 	if (git_work_tree_initialized) {
 		new_work_tree = real_path(new_work_tree);
-		if (strcmp(new_work_tree, work_tree))
+		if (strcmp(new_work_tree, the_repository.worktree))
 			die("internal error: work tree has already been set\n"
 			    "Current worktree: %s\nNew worktree: %s",
-			    work_tree, new_work_tree);
+			    the_repository.worktree, new_work_tree);
 		return;
 	}
 	git_work_tree_initialized = 1;
-	work_tree = real_pathdup(new_work_tree, 1);
+	repo_set_worktree(&the_repository, new_work_tree);
 }
 
 const char *get_git_work_tree(void)
 {
-	return work_tree;
+	return the_repository.worktree;
 }
 
 char *get_object_directory(void)
diff --git a/repo.c b/repo.c
index f7c3617a9..789ffdd78 100644
--- a/repo.c
+++ b/repo.c
@@ -84,6 +84,11 @@ void repo_set_gitdir(struct repo *repo, const char *path)
 	repo_setup_env(repo);
 }
 
+void repo_set_worktree(struct repo *repo, const char *path)
+{
+	repo->worktree = real_pathdup(path, 1);
+}
+
 int repo_init(struct repo *repo, const char *gitdir)
 {
 	int error = 0;
@@ -121,4 +126,6 @@ int repo_init(struct repo *repo, const char *gitdir)
 void repo_clear(struct repo *repo)
 {
 	repo_clear_env(repo);
+	free(repo->worktree);
+	repo->worktree = NULL;
 }
diff --git a/repo.h b/repo.h
index 453049e67..95d021049 100644
--- a/repo.h
+++ b/repo.h
@@ -9,6 +9,7 @@ struct repo {
 	char *index_file;
 	char *graft_file;
 	char *namespace;
+	char *worktree;
 
 	/* Configurations */
 	unsigned ignore_env:1;
@@ -19,6 +20,7 @@ struct repo {
 extern struct repo the_repository;
 
 extern void repo_set_gitdir(struct repo *repo, const char *path);
+extern void repo_set_worktree(struct repo *repo, const char *path);
 extern int repo_init(struct repo *repo, const char *path);
 extern void repo_clear(struct repo *repo);
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 09/31] setup: add comment indicating a hack
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (7 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 08/31] environment: store worktree " Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 10/31] config: migrate the_configset to the_repository Brandon Williams
                   ` (24 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

'GIT_TOPLEVEL_PREFIX_ENVIRONMENT' was added in (b58a68c1c setup: allow
for prefix to be passed to git commands) to aid in fixing a bug where
'ls-files' and 'grep' were not able to properly recurse when called from
within a subdirectory.  Add a 'NEEDSWORK' comment indicating that this
envvar should be removed once 'ls-files' and 'grep' can recurse
in-process.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 setup.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/setup.c b/setup.c
index dba3abd00..c3d21345d 100644
--- a/setup.c
+++ b/setup.c
@@ -1057,6 +1057,12 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die("BUG: unhandled setup_git_directory_1() result");
 	}
 
+	/*
+	 * NEEDSWORK: This was a hack in order to get ls-files and grep to have
+	 * properly formated output when recursing submodules.  Once ls-files
+	 * and grep have been changed to perform this recursing in-process this
+	 * needs to be removed.
+	 */
 	env_prefix = getenv(GIT_TOPLEVEL_PREFIX_ENVIRONMENT);
 	if (env_prefix)
 		prefix = env_prefix;
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 10/31] config: migrate the_configset to the_repository
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (8 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 09/31] setup: add comment indicating a hack Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 11/31] repo: add index_state to struct repo Brandon Williams
                   ` (23 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate the default config to be stored within 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 config.c | 185 ++++++++++++++++++++++++++++++++++++++-------------------------
 config.h |   1 +
 repo.c   |  21 ++++++++
 repo.h   |  11 ++++
 4 files changed, 146 insertions(+), 72 deletions(-)

diff --git a/config.c b/config.c
index ff09b27b8..930333e89 100644
--- a/config.c
+++ b/config.c
@@ -7,6 +7,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "repo.h"
 #include "lockfile.h"
 #include "exec_cmd.h"
 #include "strbuf.h"
@@ -72,13 +73,6 @@ static int core_compression_seen;
 static int pack_compression_seen;
 static int zlib_compression_seen;
 
-/*
- * Default config_set that contains key-value pairs from the usual set of config
- * config files (i.e repo specific .git/config, user wide ~/.gitconfig, XDG
- * config file and the global /etc/gitconfig)
- */
-static struct config_set the_config_set;
-
 static int config_file_fgetc(struct config_source *conf)
 {
 	return getc_unlocked(conf->u.file);
@@ -1606,28 +1600,6 @@ int git_config_with_options(config_fn_t fn, void *data,
 	return do_git_config_sequence(opts, fn, data);
 }
 
-static void git_config_raw(config_fn_t fn, void *data)
-{
-	struct config_options opts = {0};
-
-	opts.respect_includes = 1;
-	if (have_git_dir())
-		opts.git_dir = get_git_common_dir();
-	if (git_config_with_options(fn, data, NULL, &opts) < 0)
-		/*
-		 * git_config_with_options() normally returns only
-		 * zero, as most errors are fatal, and
-		 * non-fatal potential errors are guarded by "if"
-		 * statements that are entered only when no error is
-		 * possible.
-		 *
-		 * If we ever encounter a non-fatal error, it means
-		 * something went really wrong and we should stop
-		 * immediately.
-		 */
-		die(_("unknown error occurred while reading the configuration files"));
-}
-
 static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 {
 	int i, value_index;
@@ -1676,14 +1648,6 @@ void read_early_config(config_fn_t cb, void *data)
 	strbuf_release(&buf);
 }
 
-static void git_config_check_init(void);
-
-void git_config(config_fn_t fn, void *data)
-{
-	git_config_check_init();
-	configset_iter(&the_config_set, fn, data);
-}
-
 static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
 {
 	struct config_set_element k;
@@ -1782,7 +1746,7 @@ void git_configset_clear(struct config_set *cs)
 	cs->list.items = NULL;
 }
 
-static int config_set_callback(const char *key, const char *value, void *cb)
+int config_set_callback(const char *key, const char *value, void *cb)
 {
 	struct config_set *cs = cb;
 	configset_add_value(cs, key, value);
@@ -1893,87 +1857,164 @@ int git_configset_get_pathname(struct config_set *cs, const char *key, const cha
 		return 1;
 }
 
-static void git_config_check_init(void)
+/* Functions use to read configuration from a repository */
+static void git_config_check_init(struct repo *repository)
 {
-	if (the_config_set.hash_initialized)
+	if (repository->config && repository->config->hash_initialized)
 		return;
-	git_configset_init(&the_config_set);
-	git_config_raw(config_set_callback, &the_config_set);
+	repo_read_config(repository);
 }
 
-void git_config_clear(void)
+static void repo_config_clear(struct repo *repository)
 {
-	if (!the_config_set.hash_initialized)
+	if (!repository->config || !repository->config->hash_initialized)
 		return;
-	git_configset_clear(&the_config_set);
+	git_configset_clear(repository->config);
 }
 
-int git_config_get_value(const char *key, const char **value)
+static void repo_config(struct repo *repository, config_fn_t fn, void *data)
 {
-	git_config_check_init();
-	return git_configset_get_value(&the_config_set, key, value);
+	git_config_check_init(repository);
+	configset_iter(repository->config, fn, data);
 }
 
-const struct string_list *git_config_get_value_multi(const char *key)
+static int repo_config_get_value(struct repo *repository,
+				 const char *key, const char **value)
 {
-	git_config_check_init();
-	return git_configset_get_value_multi(&the_config_set, key);
+	git_config_check_init(repository);
+	return git_configset_get_value(repository->config, key, value);
 }
 
-int git_config_get_string_const(const char *key, const char **dest)
+static const struct string_list *repo_config_get_value_multi(struct repo *repository,
+							     const char *key)
+{
+	git_config_check_init(repository);
+	return git_configset_get_value_multi(repository->config, key);
+}
+
+static int repo_config_get_string_const(struct repo *repository,
+					const char *key, const char **dest)
 {
 	int ret;
-	git_config_check_init();
-	ret = git_configset_get_string_const(&the_config_set, key, dest);
+	git_config_check_init(repository);
+	ret = git_configset_get_string_const(repository->config, key, dest);
 	if (ret < 0)
 		git_die_config(key, NULL);
 	return ret;
 }
 
+static int repo_config_get_string(struct repo *repository,
+				  const char *key, char **dest)
+{
+	git_config_check_init(repository);
+	return repo_config_get_string_const(repository, key, (const char **)dest);
+}
+
+static int repo_config_get_int(struct repo *repository,
+			       const char *key, int *dest)
+{
+	git_config_check_init(repository);
+	return git_configset_get_int(repository->config, key, dest);
+}
+
+static int repo_config_get_ulong(struct repo *repository,
+				const char *key, unsigned long *dest)
+{
+	git_config_check_init(repository);
+	return git_configset_get_ulong(repository->config, key, dest);
+}
+
+static int repo_config_get_bool(struct repo *repository,
+				const char *key, int *dest)
+{
+	git_config_check_init(repository);
+	return git_configset_get_bool(repository->config, key, dest);
+}
+
+static int repo_config_get_bool_or_int(struct repo *repository,
+				       const char *key, int *is_bool, int *dest)
+{
+	git_config_check_init(repository);
+	return git_configset_get_bool_or_int(repository->config, key, is_bool, dest);
+}
+
+static int repo_config_get_maybe_bool(struct repo *repository,
+				      const char *key, int *dest)
+{
+	git_config_check_init(repository);
+	return git_configset_get_maybe_bool(repository->config, key, dest);
+}
+
+static int repo_config_get_pathname(struct repo *repository,
+				    const char *key, const char **dest)
+{
+	int ret;
+	git_config_check_init(repository);
+	ret = git_configset_get_pathname(repository->config, key, dest);
+	if (ret < 0)
+		git_die_config(key, NULL);
+	return ret;
+}
+
+/* Functions used historically to read configuration from 'the_repository' */
+void git_config(config_fn_t fn, void *data)
+{
+	repo_config(&the_repository, fn, data);
+}
+
+void git_config_clear(void)
+{
+	repo_config_clear(&the_repository);
+}
+
+int git_config_get_value(const char *key, const char **value)
+{
+	return repo_config_get_value(&the_repository, key, value);
+}
+
+const struct string_list *git_config_get_value_multi(const char *key)
+{
+	return repo_config_get_value_multi(&the_repository, key);
+}
+
+int git_config_get_string_const(const char *key, const char **dest)
+{
+	return repo_config_get_string_const(&the_repository, key, dest);
+}
+
 int git_config_get_string(const char *key, char **dest)
 {
-	git_config_check_init();
-	return git_config_get_string_const(key, (const char **)dest);
+	return repo_config_get_string(&the_repository, key, dest);
 }
 
 int git_config_get_int(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_int(&the_config_set, key, dest);
+	return repo_config_get_int(&the_repository, key, dest);
 }
 
 int git_config_get_ulong(const char *key, unsigned long *dest)
 {
-	git_config_check_init();
-	return git_configset_get_ulong(&the_config_set, key, dest);
+	return repo_config_get_ulong(&the_repository, key, dest);
 }
 
 int git_config_get_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool(&the_config_set, key, dest);
+	return repo_config_get_bool(&the_repository, key, dest);
 }
 
 int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool_or_int(&the_config_set, key, is_bool, dest);
+	return repo_config_get_bool_or_int(&the_repository, key, is_bool, dest);
 }
 
 int git_config_get_maybe_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_maybe_bool(&the_config_set, key, dest);
+	return repo_config_get_maybe_bool(&the_repository, key, dest);
 }
 
 int git_config_get_pathname(const char *key, const char **dest)
 {
-	int ret;
-	git_config_check_init();
-	ret = git_configset_get_pathname(&the_config_set, key, dest);
-	if (ret < 0)
-		git_die_config(key, NULL);
-	return ret;
+	return repo_config_get_pathname(&the_repository, key, dest);
 }
 
 int git_config_get_expiry(const char *key, const char **output)
diff --git a/config.h b/config.h
index f7f8b66c5..4cd9e2915 100644
--- a/config.h
+++ b/config.h
@@ -149,6 +149,7 @@ struct config_set {
 };
 
 extern void git_configset_init(struct config_set *cs);
+extern int config_set_callback(const char *key, const char *value, void *cb);
 extern int git_configset_add_file(struct config_set *cs, const char *filename);
 extern int git_configset_get_value(struct config_set *cs, const char *key, const char **value);
 extern const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key);
diff --git a/repo.c b/repo.c
index 789ffdd78..c67cad5a2 100644
--- a/repo.c
+++ b/repo.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "repo.h"
+#include "config.h"
 
 /*
  * This may be the wrong place for this.
@@ -89,6 +90,20 @@ void repo_set_worktree(struct repo *repo, const char *path)
 	repo->worktree = real_pathdup(path, 1);
 }
 
+void repo_read_config(struct repo *repo)
+{
+	struct config_options opts = { 1, repo->commondir };
+
+	if (!repo->config)
+		repo->config = xcalloc(1, sizeof(struct config_set));
+	else
+		git_configset_clear(repo->config);
+
+	git_configset_init(repo->config);
+
+	git_config_with_options(config_set_callback, repo->config, NULL, &opts);
+}
+
 int repo_init(struct repo *repo, const char *gitdir)
 {
 	int error = 0;
@@ -128,4 +143,10 @@ void repo_clear(struct repo *repo)
 	repo_clear_env(repo);
 	free(repo->worktree);
 	repo->worktree = NULL;
+
+	if (repo->config) {
+		git_configset_clear(repo->config);
+		free(repo->config);
+		repo->config = NULL;
+	}
 }
diff --git a/repo.h b/repo.h
index 95d021049..284452832 100644
--- a/repo.h
+++ b/repo.h
@@ -1,6 +1,8 @@
 #ifndef REPO_H
 #define REPO_H
 
+struct config_set;
+
 struct repo {
 	/* Environment */
 	char *gitdir;
@@ -11,6 +13,14 @@ struct repo {
 	char *namespace;
 	char *worktree;
 
+	/* Subsystems */
+	/*
+	 * Repository's config which contains key-value pairs from the usual
+	 * set of config config files (i.e repo specific .git/config, user wide
+	 * ~/.gitconfig, XDG config file and the global /etc/gitconfig)
+	 */
+	struct config_set *config;
+
 	/* Configurations */
 	unsigned ignore_env:1;
 	/* Indicates if a repository has a different 'commondir' from 'gitdir' */
@@ -21,6 +31,7 @@ extern struct repo the_repository;
 
 extern void repo_set_gitdir(struct repo *repo, const char *path);
 extern void repo_set_worktree(struct repo *repo, const char *path);
+extern void repo_read_config(struct repo *repo);
 extern int repo_init(struct repo *repo, const char *path);
 extern void repo_clear(struct repo *repo);
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 11/31] repo: add index_state to struct repo
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (9 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 10/31] config: migrate the_configset to the_repository Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 12/31] submodule-config: store the_submodule_cache in the_repository Brandon Williams
                   ` (22 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repo.c | 17 +++++++++++++++++
 repo.h |  3 +++
 2 files changed, 20 insertions(+)

diff --git a/repo.c b/repo.c
index c67cad5a2..c79d29534 100644
--- a/repo.c
+++ b/repo.c
@@ -104,6 +104,17 @@ void repo_read_config(struct repo *repo)
 	git_config_with_options(config_set_callback, repo->config, NULL, &opts);
 }
 
+void repo_read_index(struct repo *repo)
+{
+	if (!repo->index)
+		repo->index = xcalloc(1, sizeof(struct index_state));
+	else
+		discard_index(repo->index);
+
+	if (read_index_from(repo->index, repo->index_file) < 0)
+		die(_("failure reading index"));
+}
+
 int repo_init(struct repo *repo, const char *gitdir)
 {
 	int error = 0;
@@ -149,4 +160,10 @@ void repo_clear(struct repo *repo)
 		free(repo->config);
 		repo->config = NULL;
 	}
+
+	if (repo->index) {
+		discard_index(repo->index);
+		free(repo->index);
+		repo->index = NULL;
+	}
 }
diff --git a/repo.h b/repo.h
index 284452832..756cda9e1 100644
--- a/repo.h
+++ b/repo.h
@@ -2,6 +2,7 @@
 #define REPO_H
 
 struct config_set;
+struct index_state;
 
 struct repo {
 	/* Environment */
@@ -20,6 +21,7 @@ struct repo {
 	 * ~/.gitconfig, XDG config file and the global /etc/gitconfig)
 	 */
 	struct config_set *config;
+	struct index_state *index;
 
 	/* Configurations */
 	unsigned ignore_env:1;
@@ -32,6 +34,7 @@ extern struct repo the_repository;
 extern void repo_set_gitdir(struct repo *repo, const char *path);
 extern void repo_set_worktree(struct repo *repo, const char *path);
 extern void repo_read_config(struct repo *repo);
+extern void repo_read_index(struct repo *repo);
 extern int repo_init(struct repo *repo, const char *path);
 extern void repo_clear(struct repo *repo);
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 12/31] submodule-config: store the_submodule_cache in the_repository
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (10 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 11/31] repo: add index_state to struct repo Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:43 ` [PATCH 13/31] repo: add repo_read_gitmodules Brandon Williams
                   ` (21 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Refactor how 'the_submodule_cache' is handled so that it can be stored
inside of a repository object.  Also migrate 'the_submodule_cache' to be
stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repo.c             |  6 +++++
 repo.h             |  2 ++
 submodule-config.c | 71 ++++++++++++++++++++++++++++++++++++++++--------------
 submodule-config.h | 10 ++++++++
 4 files changed, 71 insertions(+), 18 deletions(-)

diff --git a/repo.c b/repo.c
index c79d29534..13b7d244f 100644
--- a/repo.c
+++ b/repo.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "repo.h"
 #include "config.h"
+#include "submodule-config.h"
 
 /*
  * This may be the wrong place for this.
@@ -166,4 +167,9 @@ void repo_clear(struct repo *repo)
 		free(repo->index);
 		repo->index = NULL;
 	}
+
+	if (repo->submodule_cache) {
+		submodule_cache_free(repo->submodule_cache);
+		repo->submodule_cache = NULL;
+	}
 }
diff --git a/repo.h b/repo.h
index 756cda9e1..ebce2a408 100644
--- a/repo.h
+++ b/repo.h
@@ -3,6 +3,7 @@
 
 struct config_set;
 struct index_state;
+struct submodule_cache;
 
 struct repo {
 	/* Environment */
@@ -22,6 +23,7 @@ struct repo {
 	 */
 	struct config_set *config;
 	struct index_state *index;
+	struct submodule_cache *submodule_cache;
 
 	/* Configurations */
 	unsigned ignore_env:1;
diff --git a/submodule-config.c b/submodule-config.c
index d8f8d5ea3..cd356fec0 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repo.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -15,6 +16,7 @@
 struct submodule_cache {
 	struct hashmap for_path;
 	struct hashmap for_name;
+	unsigned initialized:1;
 };
 
 /*
@@ -31,9 +33,6 @@ enum lookup_type {
 	lookup_path
 };
 
-static struct submodule_cache the_submodule_cache;
-static int is_cache_init;
-
 static int config_path_cmp(const struct submodule_entry *a,
 			   const struct submodule_entry *b,
 			   const void *unused)
@@ -50,10 +49,16 @@ static int config_name_cmp(const struct submodule_entry *a,
 	       hashcmp(a->config->gitmodules_sha1, b->config->gitmodules_sha1);
 }
 
-static void cache_init(struct submodule_cache *cache)
+static struct submodule_cache *submodule_cache_alloc(void)
+{
+	return xcalloc(1, sizeof(struct submodule_cache));
+}
+
+static void submodule_cache_init(struct submodule_cache *cache)
 {
 	hashmap_init(&cache->for_path, (hashmap_cmp_fn) config_path_cmp, 0);
 	hashmap_init(&cache->for_name, (hashmap_cmp_fn) config_name_cmp, 0);
+	cache->initialized = 1;
 }
 
 static void free_one_config(struct submodule_entry *entry)
@@ -65,11 +70,14 @@ static void free_one_config(struct submodule_entry *entry)
 	free(entry->config);
 }
 
-static void cache_free(struct submodule_cache *cache)
+static void submodule_cache_clear(struct submodule_cache *cache)
 {
 	struct hashmap_iter iter;
 	struct submodule_entry *entry;
 
+	if (!cache->initialized)
+		return;
+
 	/*
 	 * We iterate over the name hash here to be symmetric with the
 	 * allocation of struct submodule entries. Each is allocated by
@@ -81,6 +89,13 @@ static void cache_free(struct submodule_cache *cache)
 
 	hashmap_free(&cache->for_path, 1);
 	hashmap_free(&cache->for_name, 1);
+	cache->initialized = 0;
+}
+
+void submodule_cache_free(struct submodule_cache *cache)
+{
+	submodule_cache_clear(cache);
+	free(cache);
 }
 
 static unsigned int hash_sha1_string(const unsigned char *sha1,
@@ -494,43 +509,63 @@ static const struct submodule *config_from(struct submodule_cache *cache,
 	return submodule;
 }
 
-static void ensure_cache_init(void)
+static void submodule_cache_check_init(struct repo *repository)
 {
-	if (is_cache_init)
+	if (repository->submodule_cache && repository->submodule_cache->initialized)
 		return;
 
-	cache_init(&the_submodule_cache);
-	is_cache_init = 1;
+	if (!repository->submodule_cache)
+		repository->submodule_cache = submodule_cache_alloc();
+
+	submodule_cache_init(repository->submodule_cache);
 }
 
-int parse_submodule_config_option(const char *var, const char *value)
+int submodule_config_option(struct repo *repository,
+			    const char *var, const char *value)
 {
 	struct parse_config_parameter parameter;
-	parameter.cache = &the_submodule_cache;
+
+	submodule_cache_check_init(repository);
+
+	parameter.cache = repository->submodule_cache;
 	parameter.treeish_name = NULL;
 	parameter.gitmodules_sha1 = null_sha1;
 	parameter.overwrite = 1;
 
-	ensure_cache_init();
 	return parse_config(var, value, &parameter);
 }
 
+int parse_submodule_config_option(const char *var, const char *value)
+{
+	return submodule_config_option(&the_repository, var, value);
+}
+
 const struct submodule *submodule_from_name(const unsigned char *treeish_name,
 		const char *name)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, name, lookup_name);
+	submodule_cache_check_init(&the_repository);
+	return config_from(the_repository.submodule_cache, treeish_name, name, lookup_name);
 }
 
 const struct submodule *submodule_from_path(const unsigned char *treeish_name,
 		const char *path)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, path, lookup_path);
+	submodule_cache_check_init(&the_repository);
+	return config_from(the_repository.submodule_cache, treeish_name, path, lookup_path);
 }
 
+const struct submodule *submodule_from_cache(struct repo *repository,
+					     const unsigned char *treeish_name,
+					     const char *key)
+{
+	submodule_cache_check_init(repository);
+	return config_from(repository->submodule_cache, treeish_name,
+			   key, lookup_path);
+}
+
+
 void submodule_free(void)
 {
-	cache_free(&the_submodule_cache);
-	is_cache_init = 0;
+	if (the_repository.submodule_cache)
+		submodule_cache_clear(the_repository.submodule_cache);
 }
diff --git a/submodule-config.h b/submodule-config.h
index d434ecdb4..2bfe78d89 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -22,14 +22,24 @@ struct submodule {
 	int recommend_shallow;
 };
 
+struct submodule_cache;
+struct repo;
+
+extern void submodule_cache_free(struct submodule_cache *cache);
+
 extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_submodule_config_option(const char *var, const char *value);
+extern int submodule_config_option(struct repo *repository,
+				   const char *var, const char *value);
 extern const struct submodule *submodule_from_name(
 		const unsigned char *commit_or_tree, const char *name);
 extern const struct submodule *submodule_from_path(
 		const unsigned char *commit_or_tree, const char *path);
+extern const struct submodule *submodule_from_cache(struct repo *repository,
+						    const unsigned char *treeish_name,
+						    const char *key);
 extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,
 				      unsigned char *gitmodules_sha1,
 				      struct strbuf *rev);
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 13/31] repo: add repo_read_gitmodules
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (11 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 12/31] submodule-config: store the_submodule_cache in the_repository Brandon Williams
@ 2017-05-31 21:43 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 14/31] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
                   ` (20 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Teach the repo object to be able to populate the submodule_cache by
reading the repository's gitmodules file.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repo.c | 14 ++++++++++++++
 repo.h |  1 +
 2 files changed, 15 insertions(+)

diff --git a/repo.c b/repo.c
index 13b7d244f..6f7b2015f 100644
--- a/repo.c
+++ b/repo.c
@@ -116,6 +116,20 @@ void repo_read_index(struct repo *repo)
 		die(_("failure reading index"));
 }
 
+static int gitmodules_cb(const char *var, const char *value, void *data)
+{
+	struct repo *repo = data;
+	return submodule_config_option(repo, var, value);
+}
+
+void repo_read_gitmodules(struct repo *repo)
+{
+	char *gitmodules_path = xstrfmt("%s/.gitmodules", repo->worktree);
+
+	git_config_from_file(gitmodules_cb, gitmodules_path, repo);
+	free(gitmodules_path);
+}
+
 int repo_init(struct repo *repo, const char *gitdir)
 {
 	int error = 0;
diff --git a/repo.h b/repo.h
index ebce2a408..ad0184eaa 100644
--- a/repo.h
+++ b/repo.h
@@ -37,6 +37,7 @@ extern void repo_set_gitdir(struct repo *repo, const char *path);
 extern void repo_set_worktree(struct repo *repo, const char *path);
 extern void repo_read_config(struct repo *repo);
 extern void repo_read_index(struct repo *repo);
+extern void repo_read_gitmodules(struct repo *repo);
 extern int repo_init(struct repo *repo, const char *path);
 extern void repo_clear(struct repo *repo);
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 14/31] submodule: convert is_submodule_initialized to work on a repository
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (12 preceding siblings ...)
  2017-05-31 21:43 ` [PATCH 13/31] repo: add repo_read_gitmodules Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 15/31] convert: convert get_cached_convert_stats_ascii to take an index Brandon Williams
                   ` (19 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert 'is_submodule_initialized()' to take a repository object and
while we're at it, lets rename the function to 'is_submodule_active()'
and remove the NEEDSWORK comment.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/grep.c              |  3 ++-
 builtin/submodule--helper.c |  9 +++++----
 config.c                    | 12 ++++++------
 config.h                    |  9 +++++++++
 submodule.c                 | 21 +++++++++------------
 submodule.h                 |  3 ++-
 6 files changed, 33 insertions(+), 24 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index 0f4a1e5a3..c3086bc0b 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "repo.h"
 #include "config.h"
 #include "blob.h"
 #include "tree.h"
@@ -626,7 +627,7 @@ static int grep_submodule_launch(struct grep_opt *opt,
 static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,
 			  const char *filename, const char *path)
 {
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(&the_repository, path))
 		return 0;
 	if (!is_submodule_populated_gently(path, NULL)) {
 		/*
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 4dcbfb952..d2d6fa914 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "repo.h"
 #include "cache.h"
 #include "config.h"
 #include "parse-options.h"
@@ -280,7 +281,7 @@ static void module_list_active(struct module_list *list)
 	for (i = 0; i < list->nr; i++) {
 		const struct cache_entry *ce = list->entries[i];
 
-		if (!is_submodule_initialized(ce->name))
+		if (!is_submodule_active(&the_repository, ce->name))
 			continue;
 
 		ALLOC_GROW(active_modules.entries,
@@ -362,7 +363,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_initialized(path)) {
+	if (!is_submodule_active(&the_repository, path)) {
 		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
@@ -817,7 +818,7 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 	}
 
 	/* Check if the submodule has been initialized. */
-	if (!is_submodule_initialized(ce->name)) {
+	if (!is_submodule_active(&the_repository, ce->name)) {
 		next_submodule_warn_missing(suc, out, displaypath);
 		goto cleanup;
 	}
@@ -1193,7 +1194,7 @@ static int is_active(int argc, const char **argv, const char *prefix)
 
 	gitmodules_config();
 
-	return !is_submodule_initialized(argv[1]);
+	return !is_submodule_active(&the_repository, argv[1]);
 }
 
 #define SUPPORT_SUPER_PREFIX (1<<0)
diff --git a/config.c b/config.c
index 930333e89..977bba233 100644
--- a/config.c
+++ b/config.c
@@ -1885,8 +1885,8 @@ static int repo_config_get_value(struct repo *repository,
 	return git_configset_get_value(repository->config, key, value);
 }
 
-static const struct string_list *repo_config_get_value_multi(struct repo *repository,
-							     const char *key)
+const struct string_list *repo_config_get_value_multi(struct repo *repository,
+						      const char *key)
 {
 	git_config_check_init(repository);
 	return git_configset_get_value_multi(repository->config, key);
@@ -1903,8 +1903,8 @@ static int repo_config_get_string_const(struct repo *repository,
 	return ret;
 }
 
-static int repo_config_get_string(struct repo *repository,
-				  const char *key, char **dest)
+int repo_config_get_string(struct repo *repository,
+			   const char *key, char **dest)
 {
 	git_config_check_init(repository);
 	return repo_config_get_string_const(repository, key, (const char **)dest);
@@ -1924,8 +1924,8 @@ static int repo_config_get_ulong(struct repo *repository,
 	return git_configset_get_ulong(repository->config, key, dest);
 }
 
-static int repo_config_get_bool(struct repo *repository,
-				const char *key, int *dest)
+int repo_config_get_bool(struct repo *repository,
+			 const char *key, int *dest)
 {
 	git_config_check_init(repository);
 	return git_configset_get_bool(repository->config, key, dest);
diff --git a/config.h b/config.h
index 4cd9e2915..c4406b728 100644
--- a/config.h
+++ b/config.h
@@ -163,6 +163,15 @@ extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key,
 extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
 extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
 
+struct repo;
+
+extern const struct string_list *repo_config_get_value_multi(struct repo *repository,
+							     const char *key);
+extern int repo_config_get_string(struct repo *repository,
+				  const char *key, char **dest);
+extern int repo_config_get_bool(struct repo *repository,
+				const char *key, int *dest);
+
 extern int git_config_get_value(const char *key, const char **value);
 extern const struct string_list *git_config_get_value_multi(const char *key);
 extern void git_config_clear(void);
diff --git a/submodule.c b/submodule.c
index 95328de61..84600bde7 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repo.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -215,21 +216,17 @@ void gitmodules_config_sha1(const unsigned char *commit_sha1)
 }
 
 /*
- * NEEDSWORK: With the addition of different configuration options to determine
- * if a submodule is of interests, the validity of this function's name comes
- * into question.  Once the dust has settled and more concrete terminology is
- * decided upon, come up with a more proper name for this function.  One
- * potential candidate could be 'is_submodule_active()'.
- *
  * Determine if a submodule has been initialized at a given 'path'
  */
-int is_submodule_initialized(const char *path)
+int is_submodule_active(struct repo *repo, const char *path)
 {
 	int ret = 0;
 	char *key = NULL;
 	char *value = NULL;
 	const struct string_list *sl;
-	const struct submodule *module = submodule_from_path(null_sha1, path);
+	const struct submodule *module;
+	
+	module = submodule_from_cache(repo, null_sha1, path);
 
 	/* early return if there isn't a path->module mapping */
 	if (!module)
@@ -237,14 +234,14 @@ int is_submodule_initialized(const char *path)
 
 	/* submodule.<name>.active is set */
 	key = xstrfmt("submodule.%s.active", module->name);
-	if (!git_config_get_bool(key, &ret)) {
+	if (!repo_config_get_bool(repo, key, &ret)) {
 		free(key);
 		return ret;
 	}
 	free(key);
 
 	/* submodule.active is set */
-	sl = git_config_get_value_multi("submodule.active");
+	sl = repo_config_get_value_multi(repo, "submodule.active");
 	if (sl) {
 		struct pathspec ps;
 		struct argv_array args = ARGV_ARRAY_INIT;
@@ -264,7 +261,7 @@ int is_submodule_initialized(const char *path)
 
 	/* fallback to checking if the URL is set */
 	key = xstrfmt("submodule.%s.url", module->name);
-	ret = !git_config_get_string(key, &value);
+	ret = !repo_config_get_string(repo, key, &value);
 
 	free(value);
 	free(key);
@@ -1469,7 +1466,7 @@ int submodule_move_head(const char *path,
 	const struct submodule *sub;
 	int *error_code_ptr, error_code;
 
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(&the_repository, path))
 		return 0;
 
 	if (flags & SUBMODULE_MOVE_HEAD_FORCE)
diff --git a/submodule.h b/submodule.h
index 8fb0f2549..5c30eea55 100644
--- a/submodule.h
+++ b/submodule.h
@@ -1,6 +1,7 @@
 #ifndef SUBMODULE_H
 #define SUBMODULE_H
 
+struct repo;
 struct diff_options;
 struct argv_array;
 struct oid_array;
@@ -41,7 +42,7 @@ extern void set_diffopt_flags_from_submodule_config(struct diff_options *,
 extern int submodule_config(const char *var, const char *value, void *cb);
 extern void gitmodules_config(void);
 extern void gitmodules_config_sha1(const unsigned char *commit_sha1);
-extern int is_submodule_initialized(const char *path);
+extern int is_submodule_active(struct repo *repo, const char *path);
 /*
  * Determine if a submodule has been populated at a given 'path' by checking if
  * the <path>/.git resolves to a valid git repository.
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 15/31] convert: convert get_cached_convert_stats_ascii to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (13 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 14/31] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 16/31] convert: convert crlf_to_git " Brandon Williams
                   ` (18 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 3 ++-
 convert.c          | 5 +++--
 convert.h          | 5 ++++-
 3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index c4357dc30..f16ce0053 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -64,7 +64,8 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)
 		const char *w_txt = "";
 		const char *a_txt = get_convert_attr_ascii(path);
 		if (ce && S_ISREG(ce->ce_mode))
-			i_txt = get_cached_convert_stats_ascii(ce->name);
+			i_txt = get_cached_convert_stats_ascii(&the_index,
+							       ce->name);
 		if (!lstat(path, &st) && S_ISREG(st.st_mode))
 			w_txt = get_wt_convert_stats_ascii(path);
 		printf("i/%-5s w/%-5s attr/%-17s\t", i_txt, w_txt, a_txt);
diff --git a/convert.c b/convert.c
index 5f4a4b1f5..574003023 100644
--- a/convert.c
+++ b/convert.c
@@ -135,11 +135,12 @@ static const char *gather_convert_stats_ascii(const char *data, unsigned long si
 	}
 }
 
-const char *get_cached_convert_stats_ascii(const char *path)
+const char *get_cached_convert_stats_ascii(const struct index_state *istate,
+					   const char *path)
 {
 	const char *ret;
 	unsigned long sz;
-	void *data = read_blob_data_from_cache(path, &sz);
+	void *data = read_blob_data_from_index(istate, path, &sz);
 	ret = gather_convert_stats_ascii(data, sz);
 	free(data);
 	return ret;
diff --git a/convert.h b/convert.h
index 82871a11d..667b7dfe0 100644
--- a/convert.h
+++ b/convert.h
@@ -4,6 +4,8 @@
 #ifndef CONVERT_H
 #define CONVERT_H
 
+struct index_state;
+
 enum safe_crlf {
 	SAFE_CRLF_FALSE = 0,
 	SAFE_CRLF_FAIL = 1,
@@ -33,7 +35,8 @@ enum eol {
 };
 
 extern enum eol core_eol;
-extern const char *get_cached_convert_stats_ascii(const char *path);
+extern const char *get_cached_convert_stats_ascii(const struct index_state *istate,
+						  const char *path);
 extern const char *get_wt_convert_stats_ascii(const char *path);
 extern const char *get_convert_attr_ascii(const char *path);
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 16/31] convert: convert crlf_to_git to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (14 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 15/31] convert: convert get_cached_convert_stats_ascii to take an index Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 17/31] convert: convert convert_to_git_filter_fd " Brandon Williams
                   ` (17 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 convert.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/convert.c b/convert.c
index 574003023..ff3e72657 100644
--- a/convert.c
+++ b/convert.c
@@ -219,13 +219,13 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,
 	}
 }
 
-static int has_cr_in_index(const char *path)
+static int has_cr_in_index(const struct index_state *istate, const char *path)
 {
 	unsigned long sz;
 	void *data;
 	int has_cr;
 
-	data = read_blob_data_from_cache(path, &sz);
+	data = read_blob_data_from_index(istate, path, &sz);
 	if (!data)
 		return 0;
 	has_cr = memchr(data, '\r', sz) != NULL;
@@ -255,7 +255,8 @@ static int will_convert_lf_to_crlf(size_t len, struct text_stat *stats,
 
 }
 
-static int crlf_to_git(const char *path, const char *src, size_t len,
+static int crlf_to_git(const struct index_state *istate,
+		       const char *path, const char *src, size_t len,
 		       struct strbuf *buf,
 		       enum crlf_action crlf_action, enum safe_crlf checksafe)
 {
@@ -287,7 +288,8 @@ static int crlf_to_git(const char *path, const char *src, size_t len,
 		 * unless we want to renormalize in a merge or
 		 * cherry-pick.
 		 */
-		if ((checksafe != SAFE_CRLF_RENORMALIZE) && has_cr_in_index(path))
+		if ((checksafe != SAFE_CRLF_RENORMALIZE) &&
+		    has_cr_in_index(istate, path))
 			convert_crlf_into_lf = 0;
 	}
 	if ((checksafe == SAFE_CRLF_WARN ||
@@ -1099,7 +1101,7 @@ int convert_to_git(const char *path, const char *src, size_t len,
 		src = dst->buf;
 		len = dst->len;
 	}
-	ret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);
+	ret |= crlf_to_git(&the_index, path, src, len, dst, ca.crlf_action, checksafe);
 	if (ret && dst) {
 		src = dst->buf;
 		len = dst->len;
@@ -1119,7 +1121,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN))
 		die("%s: clean filter '%s' failed", path, ca.drv->name);
 
-	crlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
+	crlf_to_git(&the_index, path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
 	ident_to_git(path, dst->buf, dst->len, dst, ca.ident);
 }
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 17/31] convert: convert convert_to_git_filter_fd to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (15 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 16/31] convert: convert crlf_to_git " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 18/31] convert: convert convert_to_git " Brandon Williams
                   ` (16 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 convert.c   | 5 +++--
 convert.h   | 3 ++-
 sha1_file.c | 2 +-
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/convert.c b/convert.c
index ff3e72657..824b606fa 100644
--- a/convert.c
+++ b/convert.c
@@ -1109,7 +1109,8 @@ int convert_to_git(const char *path, const char *src, size_t len,
 	return ret | ident_to_git(path, src, len, dst, ca.ident);
 }
 
-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
+void convert_to_git_filter_fd(const struct index_state *istate,
+			      const char *path, int fd, struct strbuf *dst,
 			      enum safe_crlf checksafe)
 {
 	struct conv_attrs ca;
@@ -1121,7 +1122,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN))
 		die("%s: clean filter '%s' failed", path, ca.drv->name);
 
-	crlf_to_git(&the_index, path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
+	crlf_to_git(istate, path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
 	ident_to_git(path, dst->buf, dst->len, dst, ca.ident);
 }
 
diff --git a/convert.h b/convert.h
index 667b7dfe0..3a813a797 100644
--- a/convert.h
+++ b/convert.h
@@ -52,7 +52,8 @@ static inline int would_convert_to_git(const char *path)
 	return convert_to_git(path, NULL, 0, NULL, 0);
 }
 /* Precondition: would_convert_to_git_filter_fd(path) == true */
-extern void convert_to_git_filter_fd(const char *path, int fd,
+extern void convert_to_git_filter_fd(const struct index_state *istate,
+				     const char *path, int fd,
 				     struct strbuf *dst,
 				     enum safe_crlf checksafe);
 extern int would_convert_to_git_filter_fd(const char *path);
diff --git a/sha1_file.c b/sha1_file.c
index 44561e0b9..80e9ef3bb 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3581,7 +3581,7 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,
 	assert(path);
 	assert(would_convert_to_git_filter_fd(path));
 
-	convert_to_git_filter_fd(path, fd, &sbuf,
+	convert_to_git_filter_fd(&the_index, path, fd, &sbuf,
 				 write_object ? safe_crlf : SAFE_CRLF_FALSE);
 
 	if (write_object)
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 18/31] convert: convert convert_to_git to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (16 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 17/31] convert: convert convert_to_git_filter_fd " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 19/31] convert: convert renormalize_buffer " Brandon Williams
                   ` (15 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 apply.c         | 2 +-
 builtin/blame.c | 2 +-
 combine-diff.c  | 2 +-
 convert.c       | 7 ++++---
 convert.h       | 8 +++++---
 diff.c          | 6 +++---
 dir.c           | 2 +-
 sha1_file.c     | 4 ++--
 8 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/apply.c b/apply.c
index 87db9618d..8eca54325 100644
--- a/apply.c
+++ b/apply.c
@@ -2268,7 +2268,7 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)
 	case S_IFREG:
 		if (strbuf_read_file(buf, path, st->st_size) != st->st_size)
 			return error(_("unable to open or read %s"), path);
-		convert_to_git(path, buf->buf, buf->len, buf, 0);
+		convert_to_git(&the_index, path, buf->buf, buf->len, buf, 0);
 		return 0;
 	default:
 		return -1;
diff --git a/builtin/blame.c b/builtin/blame.c
index c0ae49298..317d2ec37 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -2384,7 +2384,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 		if (strbuf_read(&buf, 0, 0) < 0)
 			die_errno("failed to read from stdin");
 	}
-	convert_to_git(path, buf.buf, buf.len, &buf, 0);
+	convert_to_git(&the_index, path, buf.buf, buf.len, &buf, 0);
 	origin->file.ptr = buf.buf;
 	origin->file.size = buf.len;
 	pretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_oid.hash);
diff --git a/combine-diff.c b/combine-diff.c
index 2848034fe..74f723af3 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1053,7 +1053,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
 			if (is_file) {
 				struct strbuf buf = STRBUF_INIT;
 
-				if (convert_to_git(elem->path, result, len, &buf, safe_crlf)) {
+				if (convert_to_git(&the_index, elem->path, result, len, &buf, safe_crlf)) {
 					free(result);
 					result = strbuf_detach(&buf, &len);
 					result_size = len;
diff --git a/convert.c b/convert.c
index 824b606fa..5af6fdf3f 100644
--- a/convert.c
+++ b/convert.c
@@ -1085,7 +1085,8 @@ const char *get_convert_attr_ascii(const char *path)
 	return "";
 }
 
-int convert_to_git(const char *path, const char *src, size_t len,
+int convert_to_git(const struct index_state *istate,
+		   const char *path, const char *src, size_t len,
                    struct strbuf *dst, enum safe_crlf checksafe)
 {
 	int ret = 0;
@@ -1101,7 +1102,7 @@ int convert_to_git(const char *path, const char *src, size_t len,
 		src = dst->buf;
 		len = dst->len;
 	}
-	ret |= crlf_to_git(&the_index, path, src, len, dst, ca.crlf_action, checksafe);
+	ret |= crlf_to_git(istate, path, src, len, dst, ca.crlf_action, checksafe);
 	if (ret && dst) {
 		src = dst->buf;
 		len = dst->len;
@@ -1172,7 +1173,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str
 		src = dst->buf;
 		len = dst->len;
 	}
-	return ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE);
+	return ret | convert_to_git(&the_index, path, src, len, dst, SAFE_CRLF_RENORMALIZE);
 }
 
 /*****************************************************************
diff --git a/convert.h b/convert.h
index 3a813a797..60cb41d6a 100644
--- a/convert.h
+++ b/convert.h
@@ -41,15 +41,17 @@ extern const char *get_wt_convert_stats_ascii(const char *path);
 extern const char *get_convert_attr_ascii(const char *path);
 
 /* returns 1 if *dst was used */
-extern int convert_to_git(const char *path, const char *src, size_t len,
+extern int convert_to_git(const struct index_state *istate,
+			  const char *path, const char *src, size_t len,
 			  struct strbuf *dst, enum safe_crlf checksafe);
 extern int convert_to_working_tree(const char *path, const char *src,
 				   size_t len, struct strbuf *dst);
 extern int renormalize_buffer(const char *path, const char *src, size_t len,
 			      struct strbuf *dst);
-static inline int would_convert_to_git(const char *path)
+static inline int would_convert_to_git(const struct index_state *istate,
+				       const char *path)
 {
-	return convert_to_git(path, NULL, 0, NULL, 0);
+	return convert_to_git(istate, path, NULL, 0, NULL, 0);
 }
 /* Precondition: would_convert_to_git_filter_fd(path) == true */
 extern void convert_to_git_filter_fd(const struct index_state *istate,
diff --git a/diff.c b/diff.c
index 2f2467c6d..87ed6d6d3 100644
--- a/diff.c
+++ b/diff.c
@@ -2756,7 +2756,7 @@ static int reuse_worktree_file(const char *name, const unsigned char *sha1, int
 	 * Similarly, if we'd have to convert the file contents anyway, that
 	 * makes the optimization not worthwhile.
 	 */
-	if (!want_file && would_convert_to_git(name))
+	if (!want_file && would_convert_to_git(&the_index, name))
 		return 0;
 
 	len = strlen(name);
@@ -2878,7 +2878,7 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)
 		 * point if the path requires us to run the content
 		 * conversion.
 		 */
-		if (size_only && !would_convert_to_git(s->path))
+		if (size_only && !would_convert_to_git(&the_index, s->path))
 			return 0;
 
 		/*
@@ -2905,7 +2905,7 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)
 		/*
 		 * Convert from working tree format to canonical git format
 		 */
-		if (convert_to_git(s->path, s->data, s->size, &buf, crlf_warn)) {
+		if (convert_to_git(&the_index, s->path, s->data, s->size, &buf, crlf_warn)) {
 			size_t size = 0;
 			munmap(s->data, s->size);
 			s->should_munmap = 0;
diff --git a/dir.c b/dir.c
index 0c26a53d2..8dc74c5d2 100644
--- a/dir.c
+++ b/dir.c
@@ -796,7 +796,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,
 				 (pos = index_name_pos(istate, fname, strlen(fname))) >= 0 &&
 				 !ce_stage(istate->cache[pos]) &&
 				 ce_uptodate(istate->cache[pos]) &&
-				 !would_convert_to_git(fname))
+				 !would_convert_to_git(istate, fname))
 				hashcpy(sha1_stat->sha1,
 					istate->cache[pos]->oid.hash);
 			else
diff --git a/sha1_file.c b/sha1_file.c
index 80e9ef3bb..a900b2804 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3547,7 +3547,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,
 	 */
 	if ((type == OBJ_BLOB) && path) {
 		struct strbuf nbuf = STRBUF_INIT;
-		if (convert_to_git(path, buf, size, &nbuf,
+		if (convert_to_git(&the_index, path, buf, size, &nbuf,
 				   write_object ? safe_crlf : SAFE_CRLF_FALSE)) {
 			buf = strbuf_detach(&nbuf, &size);
 			re_allocated = 1;
@@ -3669,7 +3669,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,
 	else if (!S_ISREG(st->st_mode))
 		ret = index_pipe(sha1, fd, type, path, flags);
 	else if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||
-		 (path && would_convert_to_git(path)))
+		 (path && would_convert_to_git(&the_index, path)))
 		ret = index_core(sha1, fd, xsize_t(st->st_size), type, path,
 				 flags);
 	else
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 19/31] convert: convert renormalize_buffer to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (17 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 18/31] convert: convert convert_to_git " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 20/31] tree: convert read_tree to take an index parameter Brandon Williams
                   ` (14 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 convert.c         | 6 ++++--
 convert.h         | 3 ++-
 ll-merge.c        | 2 +-
 merge-recursive.c | 4 ++--
 4 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/convert.c b/convert.c
index 5af6fdf3f..7d2a519da 100644
--- a/convert.c
+++ b/convert.c
@@ -1,3 +1,4 @@
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "config.h"
 #include "attr.h"
@@ -1166,14 +1167,15 @@ int convert_to_working_tree(const char *path, const char *src, size_t len, struc
 	return convert_to_working_tree_internal(path, src, len, dst, 0);
 }
 
-int renormalize_buffer(const char *path, const char *src, size_t len, struct strbuf *dst)
+int renormalize_buffer(const struct index_state *istate, const char *path,
+		       const char *src, size_t len, struct strbuf *dst)
 {
 	int ret = convert_to_working_tree_internal(path, src, len, dst, 1);
 	if (ret) {
 		src = dst->buf;
 		len = dst->len;
 	}
-	return ret | convert_to_git(&the_index, path, src, len, dst, SAFE_CRLF_RENORMALIZE);
+	return ret | convert_to_git(istate, path, src, len, dst, SAFE_CRLF_RENORMALIZE);
 }
 
 /*****************************************************************
diff --git a/convert.h b/convert.h
index 60cb41d6a..cecf59d1a 100644
--- a/convert.h
+++ b/convert.h
@@ -46,7 +46,8 @@ extern int convert_to_git(const struct index_state *istate,
 			  struct strbuf *dst, enum safe_crlf checksafe);
 extern int convert_to_working_tree(const char *path, const char *src,
 				   size_t len, struct strbuf *dst);
-extern int renormalize_buffer(const char *path, const char *src, size_t len,
+extern int renormalize_buffer(const struct index_state *istate,
+			      const char *path, const char *src, size_t len,
 			      struct strbuf *dst);
 static inline int would_convert_to_git(const struct index_state *istate,
 				       const char *path)
diff --git a/ll-merge.c b/ll-merge.c
index 24ff94e1d..b9576efe9 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -340,7 +340,7 @@ static const struct ll_merge_driver *find_ll_merge_driver(const char *merge_attr
 static void normalize_file(mmfile_t *mm, const char *path)
 {
 	struct strbuf strbuf = STRBUF_INIT;
-	if (renormalize_buffer(path, mm->ptr, mm->size, &strbuf)) {
+	if (renormalize_buffer(&the_index, path, mm->ptr, mm->size, &strbuf)) {
 		free(mm->ptr);
 		mm->size = strbuf.len;
 		mm->ptr = strbuf_detach(&strbuf, NULL);
diff --git a/merge-recursive.c b/merge-recursive.c
index c2494f34f..a4b3858d0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1640,8 +1640,8 @@ static int blob_unchanged(struct merge_options *opt,
 	 * performed.  Comparison can be skipped if both files are
 	 * unchanged since their sha1s have already been compared.
 	 */
-	if (renormalize_buffer(path, o.buf, o.len, &o) |
-	    renormalize_buffer(path, a.buf, a.len, &a))
+	if (renormalize_buffer(&the_index, path, o.buf, o.len, &o) |
+	    renormalize_buffer(&the_index, path, a.buf, a.len, &a))
 		ret = (o.len == a.len && !memcmp(o.buf, a.buf, o.len));
 
 error_return:
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 20/31] tree: convert read_tree to take an index parameter
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (18 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 19/31] convert: convert renormalize_buffer " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 21/31] ls-files: convert overlay_tree_on_cache to take an index Brandon Williams
                   ` (13 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c |  2 +-
 tree.c             | 28 ++++++++++++++++++----------
 tree.h             |  3 ++-
 3 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f16ce0053..620487a77 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -461,7 +461,7 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 			       PATHSPEC_PREFER_CWD, prefix, matchbuf);
 	} else
 		memset(&pathspec, 0, sizeof(pathspec));
-	if (read_tree(tree, 1, &pathspec))
+	if (read_tree(tree, 1, &pathspec, &the_index))
 		die("unable to read tree entries %s", tree_name);
 
 	for (i = 0; i < active_nr; i++) {
diff --git a/tree.c b/tree.c
index 603b29ee8..dd69423d9 100644
--- a/tree.c
+++ b/tree.c
@@ -1,3 +1,4 @@
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "cache-tree.h"
 #include "tree.h"
@@ -8,7 +9,11 @@
 
 const char *tree_type = "tree";
 
-static int read_one_entry_opt(const unsigned char *sha1, const char *base, int baselen, const char *pathname, unsigned mode, int stage, int opt)
+static int read_one_entry_opt(struct index_state *istate,
+			      const unsigned char *sha1,
+			      const char *base, int baselen,
+			      const char *pathname,
+			      unsigned mode, int stage, int opt)
 {
 	int len;
 	unsigned int size;
@@ -27,14 +32,15 @@ static int read_one_entry_opt(const unsigned char *sha1, const char *base, int b
 	memcpy(ce->name, base, baselen);
 	memcpy(ce->name + baselen, pathname, len+1);
 	hashcpy(ce->oid.hash, sha1);
-	return add_cache_entry(ce, opt);
+	return add_index_entry(istate, ce, opt);
 }
 
 static int read_one_entry(const unsigned char *sha1, struct strbuf *base,
 			  const char *pathname, unsigned mode, int stage,
 			  void *context)
 {
-	return read_one_entry_opt(sha1, base->buf, base->len, pathname,
+	struct index_state *istate = context;
+	return read_one_entry_opt(istate, sha1, base->buf, base->len, pathname,
 				  mode, stage,
 				  ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK);
 }
@@ -47,7 +53,8 @@ static int read_one_entry_quick(const unsigned char *sha1, struct strbuf *base,
 				const char *pathname, unsigned mode, int stage,
 				void *context)
 {
-	return read_one_entry_opt(sha1, base->buf, base->len, pathname,
+	struct index_state *istate = context;
+	return read_one_entry_opt(istate, sha1, base->buf, base->len, pathname,
 				  mode, stage,
 				  ADD_CACHE_JUST_APPEND);
 }
@@ -144,7 +151,8 @@ static int cmp_cache_name_compare(const void *a_, const void *b_)
 				  ce2->name, ce2->ce_namelen, ce_stage(ce2));
 }
 
-int read_tree(struct tree *tree, int stage, struct pathspec *match)
+int read_tree(struct tree *tree, int stage, struct pathspec *match,
+	      struct index_state *istate)
 {
 	read_tree_fn_t fn = NULL;
 	int i, err;
@@ -164,23 +172,23 @@ int read_tree(struct tree *tree, int stage, struct pathspec *match)
 	 * do it the original slow way, otherwise, append and then
 	 * sort at the end.
 	 */
-	for (i = 0; !fn && i < active_nr; i++) {
-		const struct cache_entry *ce = active_cache[i];
+	for (i = 0; !fn && i < istate->cache_nr; i++) {
+		const struct cache_entry *ce = istate->cache[i];
 		if (ce_stage(ce) == stage)
 			fn = read_one_entry;
 	}
 
 	if (!fn)
 		fn = read_one_entry_quick;
-	err = read_tree_recursive(tree, "", 0, stage, match, fn, NULL);
+	err = read_tree_recursive(tree, "", 0, stage, match, fn, istate);
 	if (fn == read_one_entry || err)
 		return err;
 
 	/*
 	 * Sort the cache entry -- we need to nuke the cache tree, though.
 	 */
-	cache_tree_free(&active_cache_tree);
-	QSORT(active_cache, active_nr, cmp_cache_name_compare);
+	cache_tree_free(&istate->cache_tree);
+	QSORT(istate->cache, istate->cache_nr, cmp_cache_name_compare);
 	return 0;
 }
 
diff --git a/tree.h b/tree.h
index 0d4734b94..744e6dc2a 100644
--- a/tree.h
+++ b/tree.h
@@ -34,6 +34,7 @@ extern int read_tree_recursive(struct tree *tree,
 			       int stage, const struct pathspec *pathspec,
 			       read_tree_fn_t fn, void *context);
 
-extern int read_tree(struct tree *tree, int stage, struct pathspec *pathspec);
+extern int read_tree(struct tree *tree, int stage, struct pathspec *pathspec,
+		     struct index_state *istate);
 
 #endif /* TREE_H */
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 21/31] ls-files: convert overlay_tree_on_cache to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (19 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 20/31] tree: convert read_tree to take an index parameter Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 22/31] ls-files: convert write_eolinfo " Brandon Williams
                   ` (12 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/commit.c   |  3 ++-
 builtin/ls-files.c | 15 ++++++++-------
 cache.h            |  3 ++-
 3 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 805da4915..3d98084fb 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -254,7 +254,8 @@ static int list_paths(struct string_list *list, const char *with_tree,
 
 	if (with_tree) {
 		char *max_prefix = common_prefix(pattern);
-		overlay_tree_on_cache(with_tree, max_prefix ? max_prefix : prefix);
+		overlay_tree_on_index(&the_index, with_tree,
+				      max_prefix ? max_prefix : prefix);
 		free(max_prefix);
 	}
 
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 620487a77..a5ceeb052 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -432,7 +432,8 @@ static int get_common_prefix_len(const char *common_prefix)
  * that were given from the command line.  We are not
  * going to write this index out.
  */
-void overlay_tree_on_cache(const char *tree_name, const char *prefix)
+void overlay_tree_on_index(struct index_state *istate,
+			   const char *tree_name, const char *prefix)
 {
 	struct tree *tree;
 	struct object_id oid;
@@ -447,8 +448,8 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 		die("bad tree-ish %s", tree_name);
 
 	/* Hoist the unmerged entries up to stage #3 to make room */
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = active_cache[i];
+	for (i = 0; i < istate->cache_nr; i++) {
+		struct cache_entry *ce = istate->cache[i];
 		if (!ce_stage(ce))
 			continue;
 		ce->ce_flags |= CE_STAGEMASK;
@@ -461,11 +462,11 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 			       PATHSPEC_PREFER_CWD, prefix, matchbuf);
 	} else
 		memset(&pathspec, 0, sizeof(pathspec));
-	if (read_tree(tree, 1, &pathspec, &the_index))
+	if (read_tree(tree, 1, &pathspec, istate))
 		die("unable to read tree entries %s", tree_name);
 
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = active_cache[i];
+	for (i = 0; i < istate->cache_nr; i++) {
+		struct cache_entry *ce = istate->cache[i];
 		switch (ce_stage(ce)) {
 		case 0:
 			last_stage0 = ce;
@@ -680,7 +681,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		 */
 		if (show_stage || show_unmerged)
 			die("ls-files --with-tree is incompatible with -s or -u");
-		overlay_tree_on_cache(with_tree, max_prefix);
+		overlay_tree_on_index(&the_index, with_tree, max_prefix);
 	}
 	show_files(&dir);
 	if (show_resolve_undo)
diff --git a/cache.h b/cache.h
index 02ab5f801..73724a3ad 100644
--- a/cache.h
+++ b/cache.h
@@ -1993,7 +1993,8 @@ extern int ws_blank_line(const char *line, int len, unsigned ws_rule);
 #define ws_tab_width(rule)     ((rule) & WS_TAB_WIDTH_MASK)
 
 /* ls-files */
-void overlay_tree_on_cache(const char *tree_name, const char *prefix);
+void overlay_tree_on_index(struct index_state *istate,
+			   const char *tree_name, const char *prefix);
 
 char *alias_lookup(const char *alias);
 int split_cmdline(char *cmdline, const char ***argv);
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 22/31] ls-files: convert write_eolinfo to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (20 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 21/31] ls-files: convert overlay_tree_on_cache to take an index Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 23/31] ls-files: convert show_killed_files " Brandon Williams
                   ` (11 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index a5ceeb052..c37e9de11 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -54,17 +54,16 @@ static const char *tag_modified = "";
 static const char *tag_skip_worktree = "";
 static const char *tag_resolve_undo = "";
 
-static void write_eolinfo(const struct cache_entry *ce, const char *path)
+static void write_eolinfo(const struct index_state *istate,
+			  const struct cache_entry *ce, const char *path)
 {
-	if (!show_eol)
-		return;
-	else {
+	if (show_eol) {
 		struct stat st;
 		const char *i_txt = "";
 		const char *w_txt = "";
 		const char *a_txt = get_convert_attr_ascii(path);
 		if (ce && S_ISREG(ce->ce_mode))
-			i_txt = get_cached_convert_stats_ascii(&the_index,
+			i_txt = get_cached_convert_stats_ascii(istate,
 							       ce->name);
 		if (!lstat(path, &st) && S_ISREG(st.st_mode))
 			w_txt = get_wt_convert_stats_ascii(path);
@@ -106,7 +105,7 @@ static void show_dir_entry(const char *tag, struct dir_entry *ent)
 		return;
 
 	fputs(tag, stdout);
-	write_eolinfo(NULL, ent->name);
+	write_eolinfo(NULL, NULL, ent->name);
 	write_name(ent->name);
 }
 
@@ -276,7 +275,7 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 			       find_unique_abbrev(ce->oid.hash, abbrev),
 			       ce_stage(ce));
 		}
-		write_eolinfo(ce, ce->name);
+		write_eolinfo(&the_index, ce, ce->name);
 		write_name(ce->name);
 		if (debug_mode) {
 			const struct stat_data *sd = &ce->ce_stat_data;
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 23/31] ls-files: convert show_killed_files to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (21 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 22/31] ls-files: convert write_eolinfo " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 24/31] ls-files: convert show_other_files " Brandon Williams
                   ` (10 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index c37e9de11..f9578cf9f 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -121,7 +121,8 @@ static void show_other_files(struct dir_struct *dir)
 	}
 }
 
-static void show_killed_files(struct dir_struct *dir)
+static void show_killed_files(const struct index_state *istate,
+			      const struct dir_struct *dir)
 {
 	int i;
 	for (i = 0; i < dir->nr; i++) {
@@ -135,29 +136,29 @@ static void show_killed_files(struct dir_struct *dir)
 				/* If ent->name is prefix of an entry in the
 				 * cache, it will be killed.
 				 */
-				pos = cache_name_pos(ent->name, ent->len);
+				pos = index_name_pos(istate, ent->name, ent->len);
 				if (0 <= pos)
 					die("BUG: killed-file %.*s not found",
 						ent->len, ent->name);
 				pos = -pos - 1;
-				while (pos < active_nr &&
-				       ce_stage(active_cache[pos]))
+				while (pos < istate->cache_nr &&
+				       ce_stage(istate->cache[pos]))
 					pos++; /* skip unmerged */
-				if (active_nr <= pos)
+				if (istate->cache_nr <= pos)
 					break;
 				/* pos points at a name immediately after
 				 * ent->name in the cache.  Does it expect
 				 * ent->name to be a directory?
 				 */
-				len = ce_namelen(active_cache[pos]);
+				len = ce_namelen(istate->cache[pos]);
 				if ((ent->len < len) &&
-				    !strncmp(active_cache[pos]->name,
+				    !strncmp(istate->cache[pos]->name,
 					     ent->name, ent->len) &&
-				    active_cache[pos]->name[ent->len] == '/')
+				    istate->cache[pos]->name[ent->len] == '/')
 					killed = 1;
 				break;
 			}
-			if (0 <= cache_name_pos(ent->name, sp - ent->name)) {
+			if (0 <= index_name_pos(istate, ent->name, sp - ent->name)) {
 				/* If any of the leading directories in
 				 * ent->name is registered in the cache,
 				 * ent->name will be killed.
@@ -338,7 +339,7 @@ static void show_files(struct dir_struct *dir)
 		if (show_others)
 			show_other_files(dir);
 		if (show_killed)
-			show_killed_files(dir);
+			show_killed_files(&the_index, dir);
 	}
 	if (show_cached || show_stage) {
 		for (i = 0; i < active_nr; i++) {
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 24/31] ls-files: convert show_other_files to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (22 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 23/31] ls-files: convert show_killed_files " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 25/31] ls-files: convert show_ru_info " Brandon Williams
                   ` (9 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f9578cf9f..d00ca7810 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -109,13 +109,14 @@ static void show_dir_entry(const char *tag, struct dir_entry *ent)
 	write_name(ent->name);
 }
 
-static void show_other_files(struct dir_struct *dir)
+static void show_other_files(const struct index_state *istate,
+			     const struct dir_struct *dir)
 {
 	int i;
 
 	for (i = 0; i < dir->nr; i++) {
 		struct dir_entry *ent = dir->entries[i];
-		if (!cache_name_is_other(ent->name, ent->len))
+		if (!index_name_is_other(istate, ent->name, ent->len))
 			continue;
 		show_dir_entry(tag_other, ent);
 	}
@@ -337,7 +338,7 @@ static void show_files(struct dir_struct *dir)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
 		fill_directory(dir, &the_index, &pathspec);
 		if (show_others)
-			show_other_files(dir);
+			show_other_files(&the_index, dir);
 		if (show_killed)
 			show_killed_files(&the_index, dir);
 	}
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 25/31] ls-files: convert show_ru_info to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (23 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 24/31] ls-files: convert show_other_files " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 26/31] ls-files: convert ce_excluded " Brandon Williams
                   ` (8 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d00ca7810..2838e2f75 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -293,14 +293,14 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 	strbuf_release(&name);
 }
 
-static void show_ru_info(void)
+static void show_ru_info(const struct index_state *istate)
 {
 	struct string_list_item *item;
 
-	if (!the_index.resolve_undo)
+	if (!istate->resolve_undo)
 		return;
 
-	for_each_string_list_item(item, the_index.resolve_undo) {
+	for_each_string_list_item(item, istate->resolve_undo) {
 		const char *path = item->string;
 		struct resolve_undo_info *ui = item->util;
 		int i, len;
@@ -686,7 +686,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	}
 	show_files(&dir);
 	if (show_resolve_undo)
-		show_ru_info();
+		show_ru_info(&the_index);
 
 	if (ps_matched) {
 		int bad;
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 26/31] ls-files: convert ce_excluded to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (24 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 25/31] ls-files: convert show_ru_info " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 27/31] ls-files: convert prune_cache " Brandon Williams
                   ` (7 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 2838e2f75..289c6b2a5 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -322,10 +322,11 @@ static void show_ru_info(const struct index_state *istate)
 	}
 }
 
-static int ce_excluded(struct dir_struct *dir, const struct cache_entry *ce)
+static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
+		       const struct cache_entry *ce)
 {
 	int dtype = ce_to_dtype(ce);
-	return is_excluded(dir, &the_index, ce->name, &dtype);
+	return is_excluded(dir, istate, ce->name, &dtype);
 }
 
 static void show_files(struct dir_struct *dir)
@@ -346,7 +347,7 @@ static void show_files(struct dir_struct *dir)
 		for (i = 0; i < active_nr; i++) {
 			const struct cache_entry *ce = active_cache[i];
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, ce))
+			    !ce_excluded(dir, &the_index, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
@@ -362,7 +363,7 @@ static void show_files(struct dir_struct *dir)
 			struct stat st;
 			int err;
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, ce))
+			    !ce_excluded(dir, &the_index, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 27/31] ls-files: convert prune_cache to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (25 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 26/31] ls-files: convert ce_excluded " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 28/31] ls-files: convert show_files " Brandon Williams
                   ` (6 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 289c6b2a5..e2d8fb7f6 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -381,30 +381,31 @@ static void show_files(struct dir_struct *dir)
 /*
  * Prune the index to only contain stuff starting with "prefix"
  */
-static void prune_cache(const char *prefix, size_t prefixlen)
+static void prune_index(struct index_state *istate,
+			const char *prefix, size_t prefixlen)
 {
 	int pos;
 	unsigned int first, last;
 
 	if (!prefix)
 		return;
-	pos = cache_name_pos(prefix, prefixlen);
+	pos = index_name_pos(istate, prefix, prefixlen);
 	if (pos < 0)
 		pos = -pos-1;
 	first = pos;
-	last = active_nr;
+	last = istate->cache_nr;
 	while (last > first) {
 		int next = (last + first) >> 1;
-		const struct cache_entry *ce = active_cache[next];
+		const struct cache_entry *ce = istate->cache[next];
 		if (!strncmp(ce->name, prefix, prefixlen)) {
 			first = next+1;
 			continue;
 		}
 		last = next;
 	}
-	memmove(active_cache, active_cache + pos,
+	memmove(istate->cache, istate->cache + pos,
 		(last - pos) * sizeof(struct cache_entry *));
-	active_nr = last - pos;
+	istate->cache_nr = last - pos;
 }
 
 static int get_common_prefix_len(const char *common_prefix)
@@ -662,7 +663,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		max_prefix = common_prefix(&pathspec);
 	max_prefix_len = get_common_prefix_len(max_prefix);
 
-	prune_cache(max_prefix, max_prefix_len);
+	prune_index(&the_index, max_prefix, max_prefix_len);
 
 	/* Treat unmatching pathspec elements as errors */
 	if (pathspec.nr && error_unmatch)
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 28/31] ls-files: convert show_files to take an index
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (26 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 27/31] ls-files: convert prune_cache " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 29/31] ls-files: factor out debug info into a function Brandon Williams
                   ` (5 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index e2d8fb7f6..3061af2c5 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -329,7 +329,7 @@ static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
 	return is_excluded(dir, istate, ce->name, &dtype);
 }
 
-static void show_files(struct dir_struct *dir)
+static void show_files(struct index_state *istate, struct dir_struct *dir)
 {
 	int i;
 
@@ -337,17 +337,17 @@ static void show_files(struct dir_struct *dir)
 	if (show_others || show_killed) {
 		if (!show_others)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
-		fill_directory(dir, &the_index, &pathspec);
+		fill_directory(dir, istate, &pathspec);
 		if (show_others)
-			show_other_files(&the_index, dir);
+			show_other_files(istate, dir);
 		if (show_killed)
-			show_killed_files(&the_index, dir);
+			show_killed_files(istate, dir);
 	}
 	if (show_cached || show_stage) {
-		for (i = 0; i < active_nr; i++) {
-			const struct cache_entry *ce = active_cache[i];
+		for (i = 0; i < istate->cache_nr; i++) {
+			const struct cache_entry *ce = istate->cache[i];
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, &the_index, ce))
+			    !ce_excluded(dir, istate, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
@@ -358,12 +358,12 @@ static void show_files(struct dir_struct *dir)
 		}
 	}
 	if (show_deleted || show_modified) {
-		for (i = 0; i < active_nr; i++) {
-			const struct cache_entry *ce = active_cache[i];
+		for (i = 0; i < istate->cache_nr; i++) {
+			const struct cache_entry *ce = istate->cache[i];
 			struct stat st;
 			int err;
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, &the_index, ce))
+			    !ce_excluded(dir, istate, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
@@ -372,7 +372,7 @@ static void show_files(struct dir_struct *dir)
 			err = lstat(ce->name, &st);
 			if (show_deleted && err)
 				show_ce_entry(tag_removed, ce);
-			if (show_modified && ce_modified(ce, &st, 0))
+			if (show_modified && ie_modified(istate, ce, &st, 0))
 				show_ce_entry(tag_modified, ce);
 		}
 	}
@@ -686,7 +686,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 			die("ls-files --with-tree is incompatible with -s or -u");
 		overlay_tree_on_index(&the_index, with_tree, max_prefix);
 	}
-	show_files(&dir);
+	show_files(&the_index, &dir);
 	if (show_resolve_undo)
 		show_ru_info(&the_index);
 
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 29/31] ls-files: factor out debug info into a function
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (27 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 28/31] ls-files: convert show_files " Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 30/31] ls-files: factor out tag calculation Brandon Williams
                   ` (4 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 3061af2c5..2849c9a65 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -94,6 +94,19 @@ static void write_name(const char *name)
 	strbuf_release(&full_name);
 }
 
+static void print_debug(const struct cache_entry *ce)
+{
+	if (debug_mode) {
+		const struct stat_data *sd = &ce->ce_stat_data;
+
+		printf("  ctime: %d:%d\n", sd->sd_ctime.sec, sd->sd_ctime.nsec);
+		printf("  mtime: %d:%d\n", sd->sd_mtime.sec, sd->sd_mtime.nsec);
+		printf("  dev: %d\tino: %d\n", sd->sd_dev, sd->sd_ino);
+		printf("  uid: %d\tgid: %d\n", sd->sd_uid, sd->sd_gid);
+		printf("  size: %d\tflags: %x\n", sd->sd_size, ce->ce_flags);
+	}
+}
+
 static void show_dir_entry(const char *tag, struct dir_entry *ent)
 {
 	int len = max_prefix_len;
@@ -279,15 +292,7 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 		}
 		write_eolinfo(&the_index, ce, ce->name);
 		write_name(ce->name);
-		if (debug_mode) {
-			const struct stat_data *sd = &ce->ce_stat_data;
-
-			printf("  ctime: %d:%d\n", sd->sd_ctime.sec, sd->sd_ctime.nsec);
-			printf("  mtime: %d:%d\n", sd->sd_mtime.sec, sd->sd_mtime.nsec);
-			printf("  dev: %d\tino: %d\n", sd->sd_dev, sd->sd_ino);
-			printf("  uid: %d\tgid: %d\n", sd->sd_uid, sd->sd_gid);
-			printf("  size: %d\tflags: %x\n", sd->sd_size, ce->ce_flags);
-		}
+		print_debug(ce);
 	}
 
 	strbuf_release(&name);
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 30/31] ls-files: factor out tag calculation
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (28 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 29/31] ls-files: factor out debug info into a function Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-05-31 21:44 ` [PATCH 31/31] ls-files: use repository object Brandon Williams
                   ` (3 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 41 +++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 2849c9a65..6a0302a28 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -94,6 +94,30 @@ static void write_name(const char *name)
 	strbuf_release(&full_name);
 }
 
+static const char *get_tag(const struct cache_entry *ce, const char *tag)
+{
+	static char alttag[4];
+
+	if (tag && *tag && show_valid_bit && (ce->ce_flags & CE_VALID)) {
+		memcpy(alttag, tag, 3);
+
+		if (isalpha(tag[0])) {
+			alttag[0] = tolower(tag[0]);
+		} else if (tag[0] == '?') {
+			alttag[0] = '!';
+		} else {
+			alttag[0] = 'v';
+			alttag[1] = tag[0];
+			alttag[2] = ' ';
+			alttag[3] = 0;
+		}
+
+		tag = alttag;
+	}
+
+	return tag;
+}
+
 static void print_debug(const struct cache_entry *ce)
 {
 	if (debug_mode) {
@@ -264,22 +288,7 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 				  len, ps_matched,
 				  S_ISDIR(ce->ce_mode) ||
 				  S_ISGITLINK(ce->ce_mode))) {
-		if (tag && *tag && show_valid_bit &&
-		    (ce->ce_flags & CE_VALID)) {
-			static char alttag[4];
-			memcpy(alttag, tag, 3);
-			if (isalpha(tag[0]))
-				alttag[0] = tolower(tag[0]);
-			else if (tag[0] == '?')
-				alttag[0] = '!';
-			else {
-				alttag[0] = 'v';
-				alttag[1] = tag[0];
-				alttag[2] = ' ';
-				alttag[3] = 0;
-			}
-			tag = alttag;
-		}
+		tag = get_tag(ce, tag);
 
 		if (!show_stage) {
 			fputs(tag, stdout);
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 31/31] ls-files: use repository object
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (29 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 30/31] ls-files: factor out tag calculation Brandon Williams
@ 2017-05-31 21:44 ` Brandon Williams
  2017-06-01 20:36   ` Stefan Beller
  2017-05-31 22:56 ` [PATCH 00/31] " Stefan Beller
                   ` (2 subsequent siblings)
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 21:44 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert ls-files to use a repository struct and recurse submodules
inprocess.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c                     | 211 ++++++++++++++-------------------
 git.c                                  |   2 +-
 repo.c                                 |   3 +
 repo.h                                 |   1 +
 t/t3007-ls-files-recurse-submodules.sh |  39 ++++++
 5 files changed, 135 insertions(+), 121 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 6a0302a28..46962815f 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -5,6 +5,7 @@
  *
  * Copyright (C) Linus Torvalds, 2005
  */
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "config.h"
 #include "quote.h"
@@ -17,6 +18,7 @@
 #include "pathspec.h"
 #include "run-command.h"
 #include "submodule.h"
+#include "repo.h"
 
 static int abbrev;
 static int show_deleted;
@@ -32,10 +34,8 @@ static int line_terminator = '\n';
 static int debug_mode;
 static int show_eol;
 static int recurse_submodules;
-static struct argv_array submodule_options = ARGV_ARRAY_INIT;
 
 static const char *prefix;
-static const char *super_prefix;
 static int max_prefix_len;
 static int prefix_len;
 static struct pathspec pathspec;
@@ -73,25 +73,12 @@ static void write_eolinfo(const struct index_state *istate,
 
 static void write_name(const char *name)
 {
-	/*
-	 * Prepend the super_prefix to name to construct the full_name to be
-	 * written.
-	 */
-	struct strbuf full_name = STRBUF_INIT;
-	if (super_prefix) {
-		strbuf_addstr(&full_name, super_prefix);
-		strbuf_addstr(&full_name, name);
-		name = full_name.buf;
-	}
-
 	/*
 	 * With "--full-name", prefix_len=0; this caller needs to pass
 	 * an empty string in that case (a NULL is good for "").
 	 */
 	write_name_quoted_relative(name, prefix_len ? prefix : NULL,
 				   stdout, line_terminator);
-
-	strbuf_release(&full_name);
 }
 
 static const char *get_tag(const struct cache_entry *ce, const char *tag)
@@ -210,101 +197,59 @@ static void show_killed_files(const struct index_state *istate,
 	}
 }
 
-/*
- * Compile an argv_array with all of the options supported by --recurse_submodules
- */
-static void compile_submodule_options(const char **argv,
-				      const struct dir_struct *dir,
-				      int show_tag)
-{
-	if (line_terminator == '\0')
-		argv_array_push(&submodule_options, "-z");
-	if (show_tag)
-		argv_array_push(&submodule_options, "-t");
-	if (show_valid_bit)
-		argv_array_push(&submodule_options, "-v");
-	if (show_cached)
-		argv_array_push(&submodule_options, "--cached");
-	if (show_eol)
-		argv_array_push(&submodule_options, "--eol");
-	if (debug_mode)
-		argv_array_push(&submodule_options, "--debug");
-
-	/* Add Pathspecs */
-	argv_array_push(&submodule_options, "--");
-	for (; *argv; argv++)
-		argv_array_push(&submodule_options, *argv);
-}
+static void show_files(struct repo *repo, struct dir_struct *dir);
 
-/**
- * Recursively call ls-files on a submodule
- */
-static void show_gitlink(const struct cache_entry *ce)
+static void show_submodule(const struct repo *superproject,
+			   struct dir_struct *dir, const char *path)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-	char *dir;
-
-	prepare_submodule_repo_env(&cp.env_array);
-	argv_array_push(&cp.env_array, GIT_DIR_ENVIRONMENT);
-
-	if (prefix_len)
-		argv_array_pushf(&cp.env_array, "%s=%s",
-				 GIT_TOPLEVEL_PREFIX_ENVIRONMENT,
-				 prefix);
-	argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
-			 super_prefix ? super_prefix : "",
-			 ce->name);
-	argv_array_push(&cp.args, "ls-files");
-	argv_array_push(&cp.args, "--recurse-submodules");
-
-	/* add supported options */
-	argv_array_pushv(&cp.args, submodule_options.argv);
-
-	cp.git_cmd = 1;
-	dir = mkpathdup("%s/%s", get_git_work_tree(), ce->name);
-	cp.dir = dir;
-	status = run_command(&cp);
-	free(dir);
-	if (status)
-		exit(status);
+	struct repo submodule;
+	char *gitdir = mkpathdup("%s/%s", superproject->worktree, path);
+	repo_init(&submodule, gitdir);
+
+	repo_read_index(&submodule);
+	repo_read_gitmodules(&submodule);
+
+	if (superproject->submodule_prefix)
+		submodule.submodule_prefix = xstrfmt("%s%s/", superproject->submodule_prefix, path);
+	else
+		submodule.submodule_prefix = xstrfmt("%s/", path);
+	show_files(&submodule, dir);
+
+	repo_clear(&submodule);
+	free(gitdir);
 }
 
-static void show_ce_entry(const char *tag, const struct cache_entry *ce)
+static void show_ce(struct repo *repo, struct dir_struct *dir,
+		    const struct cache_entry *ce, const char *fullname,
+		    const char *tag)
 {
-	struct strbuf name = STRBUF_INIT;
-	int len = max_prefix_len;
-	if (super_prefix)
-		strbuf_addstr(&name, super_prefix);
-	strbuf_addstr(&name, ce->name);
-
-	if (len > ce_namelen(ce))
+	if (max_prefix_len > strlen(fullname))
 		die("git ls-files: internal error - cache entry not superset of prefix");
 
 	if (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&
-	    submodule_path_match(&pathspec, name.buf, ps_matched)) {
-		show_gitlink(ce);
-	} else if (match_pathspec(&pathspec, name.buf, name.len,
-				  len, ps_matched,
+	    submodule_path_match(&pathspec, fullname, ps_matched) &&
+	    is_submodule_active(repo, ce->name)) {
+		show_submodule(repo, dir, ce->name);
+	} else if (match_pathspec(&pathspec, fullname, strlen(fullname),
+				  max_prefix_len, ps_matched,
 				  S_ISDIR(ce->ce_mode) ||
 				  S_ISGITLINK(ce->ce_mode))) {
 		tag = get_tag(ce, tag);
 
-		if (!show_stage) {
-			fputs(tag, stdout);
-		} else {
+		if (show_stage) {
 			printf("%s%06o %s %d\t",
 			       tag,
 			       ce->ce_mode,
 			       find_unique_abbrev(ce->oid.hash, abbrev),
 			       ce_stage(ce));
+		} else {
+			fputs(tag, stdout);
 		}
-		write_eolinfo(&the_index, ce, ce->name);
-		write_name(ce->name);
+
+		write_eolinfo(repo->index, ce, fullname);
+		write_name(fullname);
 		print_debug(ce);
 	}
-
-	strbuf_release(&name);
 }
 
 static void show_ru_info(const struct index_state *istate)
@@ -337,59 +282,80 @@ static void show_ru_info(const struct index_state *istate)
 }
 
 static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
-		       const struct cache_entry *ce)
+		       const char *fullname, const struct cache_entry *ce)
 {
 	int dtype = ce_to_dtype(ce);
-	return is_excluded(dir, istate, ce->name, &dtype);
+	return is_excluded(dir, istate, fullname, &dtype);
 }
 
-static void show_files(struct index_state *istate, struct dir_struct *dir)
+static void construct_fullname(struct strbuf *out, const struct repo *repo,
+			       const struct cache_entry *ce)
+{
+	strbuf_reset(out);
+	if (repo->submodule_prefix)
+		strbuf_addstr(out, repo->submodule_prefix);
+	strbuf_addstr(out, ce->name);
+
+}
+
+static void show_files(struct repo *repo, struct dir_struct *dir)
 {
 	int i;
+	struct strbuf fullname = STRBUF_INIT;
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
 		if (!show_others)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
-		fill_directory(dir, istate, &pathspec);
+		fill_directory(dir, repo->index, &pathspec);
 		if (show_others)
-			show_other_files(istate, dir);
+			show_other_files(repo->index, dir);
 		if (show_killed)
-			show_killed_files(istate, dir);
+			show_killed_files(repo->index, dir);
 	}
 	if (show_cached || show_stage) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
-			show_ce_entry(ce_stage(ce) ? tag_unmerged :
-				(ce_skip_worktree(ce) ? tag_skip_worktree : tag_cached), ce);
+			show_ce(repo, dir, ce, fullname.buf,
+				ce_stage(ce) ? tag_unmerged :
+				(ce_skip_worktree(ce) ? tag_skip_worktree :
+				 tag_cached));
 		}
 	}
 	if (show_deleted || show_modified) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
 			struct stat st;
 			int err;
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
 			if (ce_skip_worktree(ce))
 				continue;
-			err = lstat(ce->name, &st);
+			err = lstat(fullname.buf, &st);
 			if (show_deleted && err)
-				show_ce_entry(tag_removed, ce);
-			if (show_modified && ie_modified(istate, ce, &st, 0))
-				show_ce_entry(tag_modified, ce);
+				show_ce(repo, dir, ce, fullname.buf, tag_removed);
+			if (show_modified && ie_modified(repo->index, ce, &st, 0))
+				show_ce(repo, dir, ce, fullname.buf, tag_modified);
 		}
 	}
+
+	strbuf_release(&fullname);
 }
 
 /*
@@ -544,7 +510,7 @@ static int option_parse_exclude_standard(const struct option *opt,
 int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 {
 	int require_work_tree = 0, show_tag = 0, i;
-	const char *max_prefix;
+	char *max_prefix;
 	struct dir_struct dir;
 	struct exclude_list *el;
 	struct string_list exclude_list = STRING_LIST_INIT_NODUP;
@@ -614,11 +580,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	prefix = cmd_prefix;
 	if (prefix)
 		prefix_len = strlen(prefix);
-	super_prefix = get_super_prefix();
 	git_config(git_default_config, NULL);
 
-	if (read_cache() < 0)
-		die("index file corrupt");
+	repo_read_index(&the_repository);
 
 	argc = parse_options(argc, argv, prefix, builtin_ls_files_options,
 			ls_files_usage, 0);
@@ -650,12 +614,14 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	if (require_work_tree && !is_inside_work_tree())
 		setup_work_tree();
 
-	if (recurse_submodules)
-		compile_submodule_options(argv, &dir, show_tag);
+	if (recurse_submodules) {
+		repo_read_gitmodules(&the_repository);
+	}
 
 	if (recurse_submodules &&
 	    (show_stage || show_deleted || show_others || show_unmerged ||
-	     show_killed || show_modified || show_resolve_undo || with_tree))
+	     show_killed || show_modified || show_resolve_undo || with_tree ||
+	     show_eol))
 		die("ls-files --recurse-submodules unsupported mode");
 
 	if (recurse_submodules && error_unmatch)
@@ -669,7 +635,10 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	/*
 	 * Find common prefix for all pathspec's
 	 * This is used as a performance optimization which unfortunately cannot
-	 * be done when recursing into submodules
+	 * be done when recursing into submodules because when a pathspec is
+	 * given which spans repository boundaries you can't simply remove the
+	 * submodule entry because the pathspec may match something inside the
+	 * submodule.
 	 */
 	if (recurse_submodules)
 		max_prefix = NULL;
@@ -677,7 +646,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		max_prefix = common_prefix(&pathspec);
 	max_prefix_len = get_common_prefix_len(max_prefix);
 
-	prune_index(&the_index, max_prefix, max_prefix_len);
+	prune_index(the_repository.index, max_prefix, max_prefix_len);
 
 	/* Treat unmatching pathspec elements as errors */
 	if (pathspec.nr && error_unmatch)
@@ -698,11 +667,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		 */
 		if (show_stage || show_unmerged)
 			die("ls-files --with-tree is incompatible with -s or -u");
-		overlay_tree_on_index(&the_index, with_tree, max_prefix);
+		overlay_tree_on_index(the_repository.index, with_tree, max_prefix);
 	}
-	show_files(&the_index, &dir);
+
+	show_files(&the_repository, &dir);
+
 	if (show_resolve_undo)
-		show_ru_info(&the_index);
+		show_ru_info(the_repository.index);
 
 	if (ps_matched) {
 		int bad;
diff --git a/git.c b/git.c
index 594436e43..16c6e325d 100644
--- a/git.c
+++ b/git.c
@@ -442,7 +442,7 @@ static struct cmd_struct commands[] = {
 	{ "init-db", cmd_init_db },
 	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY },
 	{ "log", cmd_log, RUN_SETUP },
-	{ "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },
+	{ "ls-files", cmd_ls_files, RUN_SETUP },
 	{ "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY },
 	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
 	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY },
diff --git a/repo.c b/repo.c
index 6f7b2015f..01001e577 100644
--- a/repo.c
+++ b/repo.c
@@ -154,6 +154,7 @@ int repo_init(struct repo *repo, const char *gitdir)
 	}
 
 	repo_set_gitdir(repo, resolved_gitdir);
+	repo_set_worktree(repo, gitdir);
 
 	/* NEEDSWORK: Verify repository format version */
 
@@ -169,6 +170,8 @@ void repo_clear(struct repo *repo)
 	repo_clear_env(repo);
 	free(repo->worktree);
 	repo->worktree = NULL;
+	free(repo->submodule_prefix);
+	repo->submodule_prefix = NULL;
 
 	if (repo->config) {
 		git_configset_clear(repo->config);
diff --git a/repo.h b/repo.h
index ad0184eaa..cb83f765b 100644
--- a/repo.h
+++ b/repo.h
@@ -14,6 +14,7 @@ struct repo {
 	char *graft_file;
 	char *namespace;
 	char *worktree;
+	char *submodule_prefix;
 
 	/* Subsystems */
 	/*
diff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh
index ebb956fd1..318b5bce7 100755
--- a/t/t3007-ls-files-recurse-submodules.sh
+++ b/t/t3007-ls-files-recurse-submodules.sh
@@ -135,6 +135,45 @@ test_expect_success '--recurse-submodules and pathspecs setup' '
 	test_cmp expect actual
 '
 
+test_expect_success 'inactive submodule' '
+	test_when_finished "git config --bool submodule.submodule.active true" &&
+	test_when_finished "git -C submodule config --bool submodule.subsub.active true" &&
+	git config --bool submodule.submodule.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual &&
+
+	git config --bool submodule.submodule.active "true" &&
+	git -C submodule config --bool submodule.subsub.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule/.gitmodules
+	submodule/c
+	submodule/f.TXT
+	submodule/g.txt
+	submodule/subsub
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success '--recurse-submodules and pathspecs' '
 	cat >expect <<-\EOF &&
 	h.txt
-- 
2.13.0.506.g27d5fe0cd-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH 00/31] repository object
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (30 preceding siblings ...)
  2017-05-31 21:44 ` [PATCH 31/31] ls-files: use repository object Brandon Williams
@ 2017-05-31 22:56 ` Stefan Beller
  2017-05-31 23:01   ` Brandon Williams
  2017-06-01 18:10 ` Brandon Williams
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
  33 siblings, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-05-31 22:56 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> Given the vast interest expressed when I sent out my RFC series I decided it
> would be worth it to invest more time to making a repository object a reality.
>
> This series is an extension of the last series I sent out (in that ls-files is
> converted to working on submodules in-process using repository objects instead
> of spawning a child process to do the work).  The big difference from the RFC
> series is that I went through and did the work to migrate key repository state
> from global variables in 'environment.c' to being stored in a repository object
> itself.  I migrated the bits of state that seemed reasonable for this series,
> there is still a lot of global state which could be migrated in the future.
>
> I do think that we need to be slightly cautious about moving global state into
> the repository object though, I don't want 'struct repo' to simply become a
> kitchen sink where everything gets dumped.  But this is just a warning for the
> future.

Or in other words:
You want to have another struct e.g. 'the_command_line_arguments',
which would carry the verbosity/color options for example as they are
not related to a repo object, but to the current command being run?

> Since this is a v1 I'm fairly certain that it still has a lot of rough edges
> (like I think I need to write better commit messages, and we should probably
> have more comments documenting object fields/contract) but I want to get the
> review process started sooner rather than later since I'm sure people will have
> opinions (e.g. should it be called 'struct repo' or 'struct repository'?!).

IMHO this is the most obvious, but bikesheddable part of the series. ;)
Keep it short as everyone knows what a 'repo' is.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 00/31] repository object
  2017-05-31 22:56 ` [PATCH 00/31] " Stefan Beller
@ 2017-05-31 23:01   ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-05-31 23:01 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On 05/31, Stefan Beller wrote:
> On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> > Given the vast interest expressed when I sent out my RFC series I decided it
> > would be worth it to invest more time to making a repository object a reality.
> >
> > This series is an extension of the last series I sent out (in that ls-files is
> > converted to working on submodules in-process using repository objects instead
> > of spawning a child process to do the work).  The big difference from the RFC
> > series is that I went through and did the work to migrate key repository state
> > from global variables in 'environment.c' to being stored in a repository object
> > itself.  I migrated the bits of state that seemed reasonable for this series,
> > there is still a lot of global state which could be migrated in the future.
> >
> > I do think that we need to be slightly cautious about moving global state into
> > the repository object though, I don't want 'struct repo' to simply become a
> > kitchen sink where everything gets dumped.  But this is just a warning for the
> > future.
> 
> Or in other words:
> You want to have another struct e.g. 'the_command_line_arguments',
> which would carry the verbosity/color options for example as they are
> not related to a repo object, but to the current command being run?

Yes exactly.  Library code that needs to operate on a repository would
then be able to take arguments like:

  some_library_function(struct repo *repo, struct lib_opts *ops)

Much like how the grep machinery takes a grep_opts struct.

> 
> > Since this is a v1 I'm fairly certain that it still has a lot of rough edges
> > (like I think I need to write better commit messages, and we should probably
> > have more comments documenting object fields/contract) but I want to get the
> > review process started sooner rather than later since I'm sure people will have
> > opinions (e.g. should it be called 'struct repo' or 'struct repository'?!).
> 
> IMHO this is the most obvious, but bikesheddable part of the series. ;)

I know, that's why I mentioned it ;)

> Keep it short as everyone knows what a 'repo' is.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 00/31] repository object
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (31 preceding siblings ...)
  2017-05-31 22:56 ` [PATCH 00/31] " Stefan Beller
@ 2017-06-01 18:10 ` Brandon Williams
  2017-06-01 18:28   ` Stefan Beller
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-01 18:10 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

On 05/31, Brandon Williams wrote:
> Given the vast interest expressed when I sent out my RFC series I decided it
> would be worth it to invest more time to making a repository object a reality.
> 
> This series is an extension of the last series I sent out (in that ls-files is
> converted to working on submodules in-process using repository objects instead
> of spawning a child process to do the work).  The big difference from the RFC
> series is that I went through and did the work to migrate key repository state
> from global variables in 'environment.c' to being stored in a repository object
> itself.  I migrated the bits of state that seemed reasonable for this series,
> there is still a lot of global state which could be migrated in the future.
> 
> I do think that we need to be slightly cautious about moving global state into
> the repository object though, I don't want 'struct repo' to simply become a
> kitchen sink where everything gets dumped.  But this is just a warning for the
> future.
> 
> Since this is a v1 I'm fairly certain that it still has a lot of rough edges
> (like I think I need to write better commit messages, and we should probably
> have more comments documenting object fields/contract) but I want to get the
> review process started sooner rather than later since I'm sure people will have
> opinions (e.g. should it be called 'struct repo' or 'struct repository'?!).
> 
> So here it is!  Thank you so much for taking the time review this, any and all
> comments would be appreciated.
> 
> - Brandon
> 
> Patches [01-14] -> Introducing the Repository object and migrating some state
> 		   to be stored in 'the_repository'.
> 
> Patches [15-31] -> Converting ls-files to use 'struct repo' in order to recurse
> 		   submodules in-process.

For those who don't like reviewing patches in email form, you can find
this series at:
https://github.com/bmwill/git/tree/repository-object 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 00/31] repository object
  2017-06-01 18:10 ` Brandon Williams
@ 2017-06-01 18:28   ` Stefan Beller
  2017-06-01 20:03     ` Jacob Keller
  0 siblings, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-06-01 18:28 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Thu, Jun 1, 2017 at 11:10 AM, Brandon Williams <bmwill@google.com> wrote:
>
> For those who don't like reviewing patches in email form, you can find
> this series at:
> https://github.com/bmwill/git/tree/repository-object

I was just asking Brandon for a remote to pull these patches from.

As I was looking at the very first patch, I'd like to lobby for trying out
origin/sb/diff-color-moved, as it will tell you about code movements. :)

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 04/31] setup: don't perform lazy initialization of repository state
  2017-05-31 21:43 ` [PATCH 04/31] setup: don't perform lazy initialization of repository state Brandon Williams
@ 2017-06-01 19:23   ` Stefan Beller
  2017-06-02 18:39     ` Jeff King
  0 siblings, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-06-01 19:23 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> Under some circumstances (bogus GIT_DIR value or the discovered gitdir
> is '.git') 'setup_git_directory()' won't initialize key repository
> state.  This leads to inconsistent state after running the setup code.
> To account for this inconsistent state, lazy initialization is done once
> a caller asks for the repository's gitdir or some other piece of
> repository state.  This is confusing and can be error prone.
>
> Instead let's tighten the expected outcome of 'setup_git_directory()'
> and ensure that it initializes repository state in all cases that would
> have been handled by lazy initialization.

Lazy init is usually there for a reason. (As in: it took too long to perform
it at all times, so it has been optimized to only perform the init when needed).

Reading the code of setup_git_env, this is the additional cost incurred to us:
* reading a file ('git_dir' to determine if it is a gitlink or actual directory)
* reading another file to determine the commondir
* some environment variable reading (<10).
  Reading the environment should be fast, though in the implementation
  of libc that I just looked up, it is still a linear search, so O(n) with n the
  number of environment variables. (No actual syscalls though.)

I would think that this is ok as it incurs not much cost. Maybe the
lazyloading implementation was just a habit of the contributors at the time.


>  const char *get_git_dir(void)
>  {
>         if (!git_dir)
> -               setup_git_env();

NEEDSWORK: eventually (say after release 2.16)
we can remove all these

    if (!git_dir)
        BUG(...);

calls. I will read on, as maybe this series touches
this further

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 05/31] environment: remove namespace_len variable
  2017-05-31 21:43 ` [PATCH 05/31] environment: remove namespace_len variable Brandon Williams
@ 2017-06-01 19:28   ` Stefan Beller
  2017-06-01 21:09     ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-06-01 19:28 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> Use 'skip_prefix' instead of 'starts_with' so that we can drop the need
> to keep around 'namespace_len'.

Looks correct.

Performance wise:
The strip_namespace function is only called from {receive/upload}-pack
and http-backend, so all network operations, whose actual operations
should far outweight the tiny CPU saving that this reverts.
We should be fine?

> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  environment.c | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/environment.c b/environment.c
> index a73b08f5d..e035f6372 100644
> --- a/environment.c
> +++ b/environment.c
> @@ -98,7 +98,6 @@ char *git_work_tree_cfg;
>  static char *work_tree;
>
>  static const char *namespace;
> -static size_t namespace_len;
>
>  static const char *super_prefix;
>
> @@ -190,7 +189,6 @@ void setup_git_env(void)
>         git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
>                                                           : "refs/replace/");
>         namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
> -       namespace_len = strlen(namespace);
>         shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
>         if (shallow_file)
>                 set_alternate_shallow_file(shallow_file, 0);
> @@ -231,9 +229,10 @@ const char *get_git_namespace(void)
>
>  const char *strip_namespace(const char *namespaced_ref)
>  {
> -       if (!starts_with(namespaced_ref, get_git_namespace()))
> -               return NULL;
> -       return namespaced_ref + namespace_len;
> +       const char *out;
> +       if (skip_prefix(namespaced_ref, get_git_namespace(), &out))
> +               return out;
> +       return NULL;
>  }
>
>  const char *get_super_prefix(void)
> --
> 2.13.0.506.g27d5fe0cd-goog
>

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 06/31] repo: introduce the repository object
  2017-05-31 21:43 ` [PATCH 06/31] repo: introduce the repository object Brandon Williams
@ 2017-06-01 19:53   ` Stefan Beller
  2017-06-05 17:53     ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-06-01 19:53 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> Introduce the repository object 'struct repo' which can be used hold all
> state pertaining to a git repository.
>
> The aim of object-ifying the repository is to (1) make the code base
> more readable and easier to reason about and (2) allow for working on
> multiple repositories, specifically submodules, within the same process.
>
> TODO: Add more motivating points for adding a repository object?

Yes please (or delete this line).
https://public-inbox.org/git/alpine.DEB.2.21.1.1705221501540.3610@virtualbox/

> +++ b/repo.c
> @@ -0,0 +1,124 @@
> +#include "cache.h"
> +#include "repo.h"
> +
> +/*
> + * This may be the wrong place for this.
> + * It may be better to go in env.c or setup for the time being?

In env.c we say:
/*
 * We put all the git config variables in this same object
 * file, so that programs can link against the config parser
 * without having to link against all the rest of git.
 *
 * In particular, no need to bring in libz etc unless needed,
 * even if you might want to know where the git directory etc
 * are.
 */

And setup.c only has a few variables that matter there locally.
So I would think having 'the_repository' in repo.c is acceptable.

> + */
> +struct repo the_repository;
> +
> +static char *git_path_from_env(const char *envvar, const char *git_dir,
> +                              const char *path, int fromenv)
> +{
> +       if (fromenv) {
> +               const char *value = getenv(envvar);
> +               if (value)
> +                       return xstrdup(value);
> +       }
> +
> +       return xstrfmt("%s/%s", git_dir, path);
> +}
> +
> +static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
> +{
> +       if (fromenv) {
> +               const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
> +               if (value) {
> +                       strbuf_addstr(sb, value);
> +                       return 1;
> +               }
> +       }
> +
> +       return get_common_dir_noenv(sb, gitdir);
> +}
> +
> +/* called after setting gitdir */
> +static void repo_setup_env(struct repo *repo)
> +{
> +       struct strbuf sb = STRBUF_INIT;
> +
> +       if (!repo->gitdir)
> +               BUG("gitdir wasn't set before setting up the environment");
> +
> +       repo->different_commondir = find_common_dir(&sb, repo->gitdir,
> +                                                   !repo->ignore_env);
> +       repo->commondir = strbuf_detach(&sb, NULL);
> +       repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
> +                                           "objects", !repo->ignore_env);
> +       repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
> +                                            "index", !repo->ignore_env);
> +       repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
> +                                            "info/grafts", !repo->ignore_env);
> +       repo->namespace = expand_namespace(repo->ignore_env ? NULL :
> +                                          getenv(GIT_NAMESPACE_ENVIRONMENT));
> +}
> +
> +static void repo_clear_env(struct repo *repo)
> +{
> +       free(repo->gitdir);
> +       repo->gitdir = NULL;
> +       free(repo->commondir);
> +       repo->commondir = NULL;
> +       free(repo->objectdir);
> +       repo->objectdir = NULL;
> +       free(repo->index_file);
> +       repo->index_file = NULL;
> +       free(repo->graft_file);
> +       repo->graft_file = NULL;
> +       free(repo->namespace);
> +       repo->namespace = NULL;

I wonder if we can defer the NULL assignments to
repo_clear, where we would just do a
memset(repo, 0, sizeof(struct repo));

> +
> +       repo_set_gitdir(repo, resolved_gitdir);
> +
> +       /* NEEDSWORK: Verify repository format version */

Care to elaborate on this? I do not understand why we would want
to check the format version here?

> +
> +extern void repo_set_gitdir(struct repo *repo, const char *path);
> +extern int repo_init(struct repo *repo, const char *path);
> +extern void repo_clear(struct repo *repo);

The init and clear method seem obvious to me, but what do we need the
repo_set_gitdir for externally? I would assume the repo auto-discovers its
gitdir on its own?

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 00/31] repository object
  2017-06-01 18:28   ` Stefan Beller
@ 2017-06-01 20:03     ` Jacob Keller
  0 siblings, 0 replies; 214+ messages in thread
From: Jacob Keller @ 2017-06-01 20:03 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, git@vger.kernel.org, Jonathan Nieder,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Thu, Jun 1, 2017 at 11:28 AM, Stefan Beller <sbeller@google.com> wrote:
> On Thu, Jun 1, 2017 at 11:10 AM, Brandon Williams <bmwill@google.com> wrote:
>>
>> For those who don't like reviewing patches in email form, you can find
>> this series at:
>> https://github.com/bmwill/git/tree/repository-object
>
> I was just asking Brandon for a remote to pull these patches from.
>
> As I was looking at the very first patch, I'd like to lobby for trying out
> origin/sb/diff-color-moved, as it will tell you about code movements. :)
>
> Thanks,
> Stefan

Yep, I'll be doing so.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 31/31] ls-files: use repository object
  2017-05-31 21:44 ` [PATCH 31/31] ls-files: use repository object Brandon Williams
@ 2017-06-01 20:36   ` Stefan Beller
  2017-06-05 17:46     ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-06-01 20:36 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Wed, May 31, 2017 at 2:44 PM, Brandon Williams <bmwill@google.com> wrote:
> Convert ls-files to use a repository struct and recurse submodules
> inprocess.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>


> +static void show_submodule(const struct repo *superproject,
> +                          struct dir_struct *dir, const char *path)
>  {
> +       struct repo submodule;
> +       char *gitdir = mkpathdup("%s/%s", superproject->worktree, path);
> +       repo_init(&submodule, gitdir);
> +
> +       repo_read_index(&submodule);
> +       repo_read_gitmodules(&submodule);
> +
> +       if (superproject->submodule_prefix)
> +               submodule.submodule_prefix = xstrfmt("%s%s/", superproject->submodule_prefix, path);
> +       else
> +               submodule.submodule_prefix = xstrfmt("%s/", path);
> +       show_files(&submodule, dir);
> +
> +       repo_clear(&submodule);
> +       free(gitdir);
>  }

I like how it seems easy now to do work in another repository. :)

> -       { "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },
> +       { "ls-files", cmd_ls_files, RUN_SETUP },

With this step, we can get rid of SUPPORT_SUPER_PREFIX eventually.

I do not have comments on the patches in the middle, but they
cleared up some of the questions that I asked in the early patches.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 05/31] environment: remove namespace_len variable
  2017-06-01 19:28   ` Stefan Beller
@ 2017-06-01 21:09     ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-01 21:09 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On 06/01, Stefan Beller wrote:
> On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> > Use 'skip_prefix' instead of 'starts_with' so that we can drop the need
> > to keep around 'namespace_len'.
> 
> Looks correct.
> 
> Performance wise:
> The strip_namespace function is only called from {receive/upload}-pack
> and http-backend, so all network operations, whose actual operations
> should far outweight the tiny CPU saving that this reverts.
> We should be fine?

Actually there should be zero performance degradation with this change
(maybe a very small improvement) because both functions need to do a
char by char comparison, just skip_prefix returns a pointer into the
string with the prefix already stripped.  So no need to do a strlen()
calculation on 'namespace' as well as no additon operation adding
'namespace_len' to 'namespaced_ref'.

> 
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> > ---
> >  environment.c | 9 ++++-----
> >  1 file changed, 4 insertions(+), 5 deletions(-)
> >
> > diff --git a/environment.c b/environment.c
> > index a73b08f5d..e035f6372 100644
> > --- a/environment.c
> > +++ b/environment.c
> > @@ -98,7 +98,6 @@ char *git_work_tree_cfg;
> >  static char *work_tree;
> >
> >  static const char *namespace;
> > -static size_t namespace_len;
> >
> >  static const char *super_prefix;
> >
> > @@ -190,7 +189,6 @@ void setup_git_env(void)
> >         git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
> >                                                           : "refs/replace/");
> >         namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
> > -       namespace_len = strlen(namespace);
> >         shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
> >         if (shallow_file)
> >                 set_alternate_shallow_file(shallow_file, 0);
> > @@ -231,9 +229,10 @@ const char *get_git_namespace(void)
> >
> >  const char *strip_namespace(const char *namespaced_ref)
> >  {
> > -       if (!starts_with(namespaced_ref, get_git_namespace()))
> > -               return NULL;
> > -       return namespaced_ref + namespace_len;
> > +       const char *out;
> > +       if (skip_prefix(namespaced_ref, get_git_namespace(), &out))
> > +               return out;
> > +       return NULL;
> >  }
> >
> >  const char *get_super_prefix(void)
> > --
> > 2.13.0.506.g27d5fe0cd-goog
> >

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 04/31] setup: don't perform lazy initialization of repository state
  2017-06-01 19:23   ` Stefan Beller
@ 2017-06-02 18:39     ` Jeff King
  2017-06-05 17:43       ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Jeff King @ 2017-06-02 18:39 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, git@vger.kernel.org, Jonathan Nieder,
	Jacob Keller, Johannes Schindelin, brian m. carlson, Ben Peart,
	Duy Nguyen, Junio C Hamano, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Thu, Jun 01, 2017 at 12:23:25PM -0700, Stefan Beller wrote:

> On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> > Under some circumstances (bogus GIT_DIR value or the discovered gitdir
> > is '.git') 'setup_git_directory()' won't initialize key repository
> > state.  This leads to inconsistent state after running the setup code.
> > To account for this inconsistent state, lazy initialization is done once
> > a caller asks for the repository's gitdir or some other piece of
> > repository state.  This is confusing and can be error prone.
> >
> > Instead let's tighten the expected outcome of 'setup_git_directory()'
> > and ensure that it initializes repository state in all cases that would
> > have been handled by lazy initialization.
> 
> Lazy init is usually there for a reason. (As in: it took too long to perform
> it at all times, so it has been optimized to only perform the init when needed).

In the case of setup_git_env(), I think it was less about doing work and
more that we didn't want to have to do explicit setup in each program.
But over the years we've moved away from that, and in fact if you hit
the lazy initialization these days you'll generally BUG() anyway.

_But_ I suspect there are still some cases you'd need to handle. For
instance, it's still OK to skip calling setup_git_directory() if you've
got $GIT_DIR in the environment (which is why we have have_git_dir()
instead of checking startup_info->have_repository).

I think it would be nice to do away with that, too, but we're not quite
there yet (and if I am reading this patch correctly, we'd probably hit
these BUGs in such cases).

-Peff

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 04/31] setup: don't perform lazy initialization of repository state
  2017-06-02 18:39     ` Jeff King
@ 2017-06-05 17:43       ` Brandon Williams
  2017-06-05 18:20         ` Jeff King
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-05 17:43 UTC (permalink / raw)
  To: Jeff King
  Cc: Stefan Beller, git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On 06/02, Jeff King wrote:
> On Thu, Jun 01, 2017 at 12:23:25PM -0700, Stefan Beller wrote:
> 
> > On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> > > Under some circumstances (bogus GIT_DIR value or the discovered gitdir
> > > is '.git') 'setup_git_directory()' won't initialize key repository
> > > state.  This leads to inconsistent state after running the setup code.
> > > To account for this inconsistent state, lazy initialization is done once
> > > a caller asks for the repository's gitdir or some other piece of
> > > repository state.  This is confusing and can be error prone.
> > >
> > > Instead let's tighten the expected outcome of 'setup_git_directory()'
> > > and ensure that it initializes repository state in all cases that would
> > > have been handled by lazy initialization.
> > 
> > Lazy init is usually there for a reason. (As in: it took too long to perform
> > it at all times, so it has been optimized to only perform the init when needed).
> 
> In the case of setup_git_env(), I think it was less about doing work and
> more that we didn't want to have to do explicit setup in each program.
> But over the years we've moved away from that, and in fact if you hit
> the lazy initialization these days you'll generally BUG() anyway.
> 
> _But_ I suspect there are still some cases you'd need to handle. For
> instance, it's still OK to skip calling setup_git_directory() if you've
> got $GIT_DIR in the environment (which is why we have have_git_dir()
> instead of checking startup_info->have_repository).

Yes there are a couple places that rely on the lazy initialization but
that's not due to setup not being run.  Rather it has to do with GIT_DIR
being set to a bogus directory so when setup is run gently it does
nothing.  Then at a later point in time the command tries to access
files in the gitdir (which triggers lazy init of the git environment).

So I think that explicitly doing the 'lazy init' portion (which ensures
that the env gets setup even if GIT_DIR is bogus) at the end of setup
should be sufficient, least it seems to be so though perhaps we can't
rely on our tests to tell us that.

> 
> I think it would be nice to do away with that, too, but we're not quite
> there yet (and if I am reading this patch correctly, we'd probably hit
> these BUGs in such cases).
> 
> -Peff

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 31/31] ls-files: use repository object
  2017-06-01 20:36   ` Stefan Beller
@ 2017-06-05 17:46     ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-05 17:46 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On 06/01, Stefan Beller wrote:
> On Wed, May 31, 2017 at 2:44 PM, Brandon Williams <bmwill@google.com> wrote:
> > Convert ls-files to use a repository struct and recurse submodules
> > inprocess.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> 
> 
> > +static void show_submodule(const struct repo *superproject,
> > +                          struct dir_struct *dir, const char *path)
> >  {
> > +       struct repo submodule;
> > +       char *gitdir = mkpathdup("%s/%s", superproject->worktree, path);
> > +       repo_init(&submodule, gitdir);
> > +
> > +       repo_read_index(&submodule);
> > +       repo_read_gitmodules(&submodule);
> > +
> > +       if (superproject->submodule_prefix)
> > +               submodule.submodule_prefix = xstrfmt("%s%s/", superproject->submodule_prefix, path);
> > +       else
> > +               submodule.submodule_prefix = xstrfmt("%s/", path);
> > +       show_files(&submodule, dir);
> > +
> > +       repo_clear(&submodule);
> > +       free(gitdir);
> >  }
> 
> I like how it seems easy now to do work in another repository. :)

It really does make working with another repo easy!  No more compiling
argv options and spawning child processes :D

> 
> > -       { "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },
> > +       { "ls-files", cmd_ls_files, RUN_SETUP },
> 
> With this step, we can get rid of SUPPORT_SUPER_PREFIX eventually.

Yes Ideally most of the little hacks I introduced when originally
teaching ls-files and grep to recurse could be removed.

> 
> I do not have comments on the patches in the middle, but they
> cleared up some of the questions that I asked in the early patches.
> 
> Thanks,
> Stefan

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 06/31] repo: introduce the repository object
  2017-06-01 19:53   ` Stefan Beller
@ 2017-06-05 17:53     ` Brandon Williams
  2017-06-05 18:31       ` Stefan Beller
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-05 17:53 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On 06/01, Stefan Beller wrote:
> On Wed, May 31, 2017 at 2:43 PM, Brandon Williams <bmwill@google.com> wrote:
> > Introduce the repository object 'struct repo' which can be used hold all
> > state pertaining to a git repository.
> >
> > The aim of object-ifying the repository is to (1) make the code base
> > more readable and easier to reason about and (2) allow for working on
> > multiple repositories, specifically submodules, within the same process.
> >
> > TODO: Add more motivating points for adding a repository object?
> 
> Yes please (or delete this line).
> https://public-inbox.org/git/alpine.DEB.2.21.1.1705221501540.3610@virtualbox/
> 
> > +++ b/repo.c
> > @@ -0,0 +1,124 @@
> > +#include "cache.h"
> > +#include "repo.h"
> > +
> > +/*
> > + * This may be the wrong place for this.
> > + * It may be better to go in env.c or setup for the time being?
> 
> In env.c we say:
> /*
>  * We put all the git config variables in this same object
>  * file, so that programs can link against the config parser
>  * without having to link against all the rest of git.
>  *
>  * In particular, no need to bring in libz etc unless needed,
>  * even if you might want to know where the git directory etc
>  * are.
>  */
> 
> And setup.c only has a few variables that matter there locally.
> So I would think having 'the_repository' in repo.c is acceptable.

And perhaps (far down the road) 'the_repoository' could be removed such
that builtin commands take a pointer to a repo object as a parameter.

> 
> > + */
> > +struct repo the_repository;
> > +
> > +static char *git_path_from_env(const char *envvar, const char *git_dir,
> > +                              const char *path, int fromenv)
> > +{
> > +       if (fromenv) {
> > +               const char *value = getenv(envvar);
> > +               if (value)
> > +                       return xstrdup(value);
> > +       }
> > +
> > +       return xstrfmt("%s/%s", git_dir, path);
> > +}
> > +
> > +static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
> > +{
> > +       if (fromenv) {
> > +               const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
> > +               if (value) {
> > +                       strbuf_addstr(sb, value);
> > +                       return 1;
> > +               }
> > +       }
> > +
> > +       return get_common_dir_noenv(sb, gitdir);
> > +}
> > +
> > +/* called after setting gitdir */
> > +static void repo_setup_env(struct repo *repo)
> > +{
> > +       struct strbuf sb = STRBUF_INIT;
> > +
> > +       if (!repo->gitdir)
> > +               BUG("gitdir wasn't set before setting up the environment");
> > +
> > +       repo->different_commondir = find_common_dir(&sb, repo->gitdir,
> > +                                                   !repo->ignore_env);
> > +       repo->commondir = strbuf_detach(&sb, NULL);
> > +       repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
> > +                                           "objects", !repo->ignore_env);
> > +       repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
> > +                                            "index", !repo->ignore_env);
> > +       repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
> > +                                            "info/grafts", !repo->ignore_env);
> > +       repo->namespace = expand_namespace(repo->ignore_env ? NULL :
> > +                                          getenv(GIT_NAMESPACE_ENVIRONMENT));
> > +}
> > +
> > +static void repo_clear_env(struct repo *repo)
> > +{
> > +       free(repo->gitdir);
> > +       repo->gitdir = NULL;
> > +       free(repo->commondir);
> > +       repo->commondir = NULL;
> > +       free(repo->objectdir);
> > +       repo->objectdir = NULL;
> > +       free(repo->index_file);
> > +       repo->index_file = NULL;
> > +       free(repo->graft_file);
> > +       repo->graft_file = NULL;
> > +       free(repo->namespace);
> > +       repo->namespace = NULL;
> 
> I wonder if we can defer the NULL assignments to
> repo_clear, where we would just do a
> memset(repo, 0, sizeof(struct repo));
> 

Yeah perhaps, clearing the env should either be done when setting gitdir
again (so setting up the env will happen again and we don't need to
clear the fields) or clearing the struct as a whole so using memset
would work.

> > +
> > +       repo_set_gitdir(repo, resolved_gitdir);
> > +
> > +       /* NEEDSWORK: Verify repository format version */
> 
> Care to elaborate on this? I do not understand why we would want
> to check the format version here?

When opening up a repository git needs to check if it understands the
repository format and all extensions.  If it doesn't git needs to bail
out and not operate on the repository.  So a part of initializing a repo
object would be to verify that it understands the repository format
version.

> 
> > +
> > +extern void repo_set_gitdir(struct repo *repo, const char *path);
> > +extern int repo_init(struct repo *repo, const char *path);
> > +extern void repo_clear(struct repo *repo);
> 
> The init and clear method seem obvious to me, but what do we need the
> repo_set_gitdir for externally? I would assume the repo auto-discovers its
> gitdir on its own?

Well I didn't completely overhaul the setup code in this series so I
really just hooked into where the setup code already sets the gitdir,
hence why this function is exposed atm.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 04/31] setup: don't perform lazy initialization of repository state
  2017-06-05 17:43       ` Brandon Williams
@ 2017-06-05 18:20         ` Jeff King
  2017-06-05 18:44           ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Jeff King @ 2017-06-05 18:20 UTC (permalink / raw)
  To: Brandon Williams
  Cc: Stefan Beller, git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Mon, Jun 05, 2017 at 10:43:52AM -0700, Brandon Williams wrote:

> > In the case of setup_git_env(), I think it was less about doing work and
> > more that we didn't want to have to do explicit setup in each program.
> > But over the years we've moved away from that, and in fact if you hit
> > the lazy initialization these days you'll generally BUG() anyway.
> > 
> > _But_ I suspect there are still some cases you'd need to handle. For
> > instance, it's still OK to skip calling setup_git_directory() if you've
> > got $GIT_DIR in the environment (which is why we have have_git_dir()
> > instead of checking startup_info->have_repository).
> 
> Yes there are a couple places that rely on the lazy initialization but
> that's not due to setup not being run.  Rather it has to do with GIT_DIR
> being set to a bogus directory so when setup is run gently it does
> nothing.  Then at a later point in time the command tries to access
> files in the gitdir (which triggers lazy init of the git environment).
> 
> So I think that explicitly doing the 'lazy init' portion (which ensures
> that the env gets setup even if GIT_DIR is bogus) at the end of setup
> should be sufficient, least it seems to be so though perhaps we can't
> rely on our tests to tell us that.

In have_git_dir() we do:

  int have_git_dir(void)
  {
	  return startup_info->have_repository
		  || git_dir
		  || getenv(GIT_DIR_ENVIRONMENT);
  }

and generally call that right before calling a function that might do
setup_git_env(). We can assume that if have_repository is set that we
have an actual git_dir (we can check all of the assignment spots and
verify that they always set it in tandem with the global git_dir).
And obviously if git_dir is set, we're good.

But if that third condition ever triggers, it's because we're relying on
the lazy setup to copy it into git_dir. And if I understand correctly,
that would turn into a BUG with your patch.

I guess my question is: does that third condition actually trigger in
practice? I added those conditions in b9605bc4f (config: only read
.git/config from configured repos, 2016-09-12). I remember there being
some reason for needing those back then, but the commit message doesn't
say. But if I remove them and just check have_repository (either on top
of that commit or on top of the current master) the tests all still
pass.

So I'm not sure at this point what the case was that motivated it, or if
it really was just an abundance of caution. But if there is such a case,
I suspect it's broken by your patch series.

I'm not sure we have a good way to find out, though. Like the current
BUG() in setup_git_env, I think we'll just have to cook the patches for
a long time, see if anybody triggers it, and deal with it on a case by
case basis.

I do kind of wonder if we should simplify have_git_dir(), if those other
conditions aren't actually triggering.

-Peff

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 06/31] repo: introduce the repository object
  2017-06-05 17:53     ` Brandon Williams
@ 2017-06-05 18:31       ` Stefan Beller
  0 siblings, 0 replies; 214+ messages in thread
From: Stefan Beller @ 2017-06-05 18:31 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

>> > +
>> > +       repo_set_gitdir(repo, resolved_gitdir);
>> > +
>> > +       /* NEEDSWORK: Verify repository format version */
>>
>> Care to elaborate on this? I do not understand why we would want
>> to check the format version here?
>
> When opening up a repository git needs to check if it understands the
> repository format and all extensions.  If it doesn't git needs to bail
> out and not operate on the repository.  So a part of initializing a repo
> object would be to verify that it understands the repository format
> version.

Ah yes. Conceptually I understand why and what the repo format
version check is. At the time of asking the question I was confused
whether this is the right place, but after some thought this is the
best place to put it.

And this NEEDSWORK is really just telling the reader that it needs to
be moved here, not copied. (That was another brain fart I had upon
reading this comment).

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 04/31] setup: don't perform lazy initialization of repository state
  2017-06-05 18:20         ` Jeff King
@ 2017-06-05 18:44           ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-05 18:44 UTC (permalink / raw)
  To: Jeff King
  Cc: Stefan Beller, git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On 06/05, Jeff King wrote:
> On Mon, Jun 05, 2017 at 10:43:52AM -0700, Brandon Williams wrote:
> 
> > > In the case of setup_git_env(), I think it was less about doing work and
> > > more that we didn't want to have to do explicit setup in each program.
> > > But over the years we've moved away from that, and in fact if you hit
> > > the lazy initialization these days you'll generally BUG() anyway.
> > > 
> > > _But_ I suspect there are still some cases you'd need to handle. For
> > > instance, it's still OK to skip calling setup_git_directory() if you've
> > > got $GIT_DIR in the environment (which is why we have have_git_dir()
> > > instead of checking startup_info->have_repository).
> > 
> > Yes there are a couple places that rely on the lazy initialization but
> > that's not due to setup not being run.  Rather it has to do with GIT_DIR
> > being set to a bogus directory so when setup is run gently it does
> > nothing.  Then at a later point in time the command tries to access
> > files in the gitdir (which triggers lazy init of the git environment).
> > 
> > So I think that explicitly doing the 'lazy init' portion (which ensures
> > that the env gets setup even if GIT_DIR is bogus) at the end of setup
> > should be sufficient, least it seems to be so though perhaps we can't
> > rely on our tests to tell us that.
> 
> In have_git_dir() we do:
> 
>   int have_git_dir(void)
>   {
> 	  return startup_info->have_repository
> 		  || git_dir
> 		  || getenv(GIT_DIR_ENVIRONMENT);
>   }
> 
> and generally call that right before calling a function that might do
> setup_git_env(). We can assume that if have_repository is set that we
> have an actual git_dir (we can check all of the assignment spots and
> verify that they always set it in tandem with the global git_dir).
> And obviously if git_dir is set, we're good.
> 
> But if that third condition ever triggers, it's because we're relying on
> the lazy setup to copy it into git_dir. And if I understand correctly,
> that would turn into a BUG with your patch.
> 
> I guess my question is: does that third condition actually trigger in
> practice? I added those conditions in b9605bc4f (config: only read
> .git/config from configured repos, 2016-09-12). I remember there being
> some reason for needing those back then, but the commit message doesn't
> say. But if I remove them and just check have_repository (either on top
> of that commit or on top of the current master) the tests all still
> pass.

The only instance I saw the third condition trigger was when GIT_DIR was
set to a bogus gitdir so when reading the early config a bogus
gitdir/config file was attempted to be read resulting in a noop.  If
GIT_DIR is set to a valid gitdir then the other condition in
read_early_config (which does repository discovery itself) would
correctly catch and read from the repository's config.

> 
> So I'm not sure at this point what the case was that motivated it, or if
> it really was just an abundance of caution. But if there is such a case,
> I suspect it's broken by your patch series.
> 
> I'm not sure we have a good way to find out, though. Like the current
> BUG() in setup_git_env, I think we'll just have to cook the patches for
> a long time, see if anybody triggers it, and deal with it on a case by
> case basis.
> 
> I do kind of wonder if we should simplify have_git_dir(), if those other
> conditions aren't actually triggering.

If this ends up being that big of an issue I'm sure that it would be
easy to adapt this and revert to doing lazy-init.  I just saw how
clunky it is and was trying to work to start the process of cleaning it
up so that the code becomes easier to reason about.  It seemed to make
things easier for me to package them in a struct at least (and reason
about how the config system would work once stored in a repository
struct).

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 03/31] config: don't implicitly use gitdir
  2017-05-31 21:43 ` [PATCH 03/31] config: don't implicitly use gitdir Brandon Williams
@ 2017-06-08 10:41   ` Johannes Schindelin
  2017-06-08 16:37     ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Johannes Schindelin @ 2017-06-08 10:41 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, sandals, peartben, pclouds,
	gitster, peff, git, avarab, jonathantanmy

Hi Brandon,

On Wed, 31 May 2017, Brandon Williams wrote:

> Commit 2185fde56 (config: handle conditional include when $GIT_DIR is
> not set up) added a 'git_dir' field to the config_options struct.  Let's
> use this option field explicitly all the time instead of occasionally
> falling back to calling 'git_pathdup("config")' to get the path to the
> local repository configuration.  This allows 'go_git_config_sequence()'
> to not implicitly rely on global repository state.

Oh wow. `git_pathdup()`...

Sadly, I lack the time to review the entire patch series, but this here
change definitely looks correct to me.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 03/31] config: don't implicitly use gitdir
  2017-06-08 10:41   ` Johannes Schindelin
@ 2017-06-08 16:37     ` Brandon Williams
  2017-06-08 18:58       ` Johannes Schindelin
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 16:37 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, sbeller, jrnieder, jacob.keller, sandals, peartben, pclouds,
	gitster, peff, git, avarab, jonathantanmy

On 06/08, Johannes Schindelin wrote:
> Hi Brandon,
> 
> On Wed, 31 May 2017, Brandon Williams wrote:
> 
> > Commit 2185fde56 (config: handle conditional include when $GIT_DIR is
> > not set up) added a 'git_dir' field to the config_options struct.  Let's
> > use this option field explicitly all the time instead of occasionally
> > falling back to calling 'git_pathdup("config")' to get the path to the
> > local repository configuration.  This allows 'go_git_config_sequence()'
> > to not implicitly rely on global repository state.
> 
> Oh wow. `git_pathdup()`...
> 
> Sadly, I lack the time to review the entire patch series, but this here
> change definitely looks correct to me.

Thanks for taking a look.  I understand you don't have time to review
the series in whole but I should also mention that patch 04/31 may be
of interest to you as it deals with setup being slightly more robust and
since you've spent a great deal of time in that bit of code any input
would be nice.  Last prod I promise ;)

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 03/31] config: don't implicitly use gitdir
  2017-06-08 16:37     ` Brandon Williams
@ 2017-06-08 18:58       ` Johannes Schindelin
  2017-06-08 19:19         ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Johannes Schindelin @ 2017-06-08 18:58 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, sandals, peartben, pclouds,
	gitster, peff, git, avarab, jonathantanmy

Hi Brandon,

On Thu, 8 Jun 2017, Brandon Williams wrote:

> On 06/08, Johannes Schindelin wrote:
> > 
> > On Wed, 31 May 2017, Brandon Williams wrote:
> > 
> > > Commit 2185fde56 (config: handle conditional include when $GIT_DIR is
> > > not set up) added a 'git_dir' field to the config_options struct.  Let's
> > > use this option field explicitly all the time instead of occasionally
> > > falling back to calling 'git_pathdup("config")' to get the path to the
> > > local repository configuration.  This allows 'go_git_config_sequence()'
> > > to not implicitly rely on global repository state.
> > 
> > Oh wow. `git_pathdup()`...
> > 
> > Sadly, I lack the time to review the entire patch series, but this here
> > change definitely looks correct to me.
> 
> Thanks for taking a look.  I understand you don't have time to review
> the series in whole but I should also mention that patch 04/31 may be
> of interest to you as it deals with setup being slightly more robust and
> since you've spent a great deal of time in that bit of code any input
> would be nice.  Last prod I promise ;)

I had a quick look over the patch and I like the direction.

I also briefly scanned the discussion and have to admit that I am not
knee-deep in those exceptional code paths that do strange^Wuncommon
things, so I feel a bit unqualified to comment about those.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 03/31] config: don't implicitly use gitdir
  2017-06-08 18:58       ` Johannes Schindelin
@ 2017-06-08 19:19         ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 19:19 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, sbeller, jrnieder, jacob.keller, sandals, peartben, pclouds,
	gitster, peff, git, avarab, jonathantanmy

On 06/08, Johannes Schindelin wrote:
> Hi Brandon,
> 
> On Thu, 8 Jun 2017, Brandon Williams wrote:
> 
> > On 06/08, Johannes Schindelin wrote:
> > > 
> > > On Wed, 31 May 2017, Brandon Williams wrote:
> > > 
> > > > Commit 2185fde56 (config: handle conditional include when $GIT_DIR is
> > > > not set up) added a 'git_dir' field to the config_options struct.  Let's
> > > > use this option field explicitly all the time instead of occasionally
> > > > falling back to calling 'git_pathdup("config")' to get the path to the
> > > > local repository configuration.  This allows 'go_git_config_sequence()'
> > > > to not implicitly rely on global repository state.
> > > 
> > > Oh wow. `git_pathdup()`...
> > > 
> > > Sadly, I lack the time to review the entire patch series, but this here
> > > change definitely looks correct to me.
> > 
> > Thanks for taking a look.  I understand you don't have time to review
> > the series in whole but I should also mention that patch 04/31 may be
> > of interest to you as it deals with setup being slightly more robust and
> > since you've spent a great deal of time in that bit of code any input
> > would be nice.  Last prod I promise ;)
> 
> I had a quick look over the patch and I like the direction.
> 
> I also briefly scanned the discussion and have to admit that I am not
> knee-deep in those exceptional code paths that do strange^Wuncommon
> things, so I feel a bit unqualified to comment about those.

All good, thanks for looking!

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v2 00/32] repository object
  2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
                   ` (32 preceding siblings ...)
  2017-06-01 18:10 ` Brandon Williams
@ 2017-06-08 23:40 ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 01/32] config: create config.h Brandon Williams
                     ` (33 more replies)
  33 siblings, 34 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

When I sent out my RFC series there seemed to be a lot of interest but I
haven't seen many people jump to review this series.  Despite lack of review I
wanted to get out another version which includes some changes to fix things
that were bugging me about the series.  Hopfully this v2 will prod some more
people to take a look.

The meat of the series is really from 04-15 and patch 32 which converts
ls-files to recurse using a repository object.  So if you're pressed for time
you can focus on those patches.

You can find this series at: https://github.com/bmwill/git/tree/repository-object

What's different in v2:
* 'struct repo' has been renamed 'struct repository'.  Really just an aesthetic
  change and it looks more "official" without having a shortened name.
* Better commit messages on a few of the patches which were soarly lacking.
* I found that 'git_config_iter' was being exported without an implementation
  so I added a patch to remove it.
* repo_init is a bit cleaner and is written in such a way to not call 'die()',
  instead it returns a non-zero return value upon failure.

Thanks for your help!

Brandon Williams (32):
  config: create config.h
  config: remove git_config_iter
  config: don't include config.h by default
  config: don't implicitly use gitdir
  setup: don't perform lazy initialization of repository state
  environment: remove namespace_len variable
  repository: introduce the repository object
  environment: place key repository state in the_repository
  environment: store worktree in the_repository
  setup: add comment indicating a hack
  config: read config from a repository object
  repository: add index_state to struct repo
  submodule-config: store the_submodule_cache in the_repository
  submodule: add repo_read_gitmodules
  submodule: convert is_submodule_initialized to work on a repository
  convert: convert get_cached_convert_stats_ascii to take an index
  convert: convert crlf_to_git to take an index
  convert: convert convert_to_git_filter_fd to take an index
  convert: convert convert_to_git to take an index
  convert: convert renormalize_buffer to take an index
  tree: convert read_tree to take an index parameter
  ls-files: convert overlay_tree_on_cache to take an index
  ls-files: convert write_eolinfo to take an index
  ls-files: convert show_killed_files to take an index
  ls-files: convert show_other_files to take an index
  ls-files: convert show_ru_info to take an index
  ls-files: convert ce_excluded to take an index
  ls-files: convert prune_cache to take an index
  ls-files: convert show_files to take an index
  ls-files: factor out debug info into a function
  ls-files: factor out tag calculation
  ls-files: use repository object

 Makefile                               |   1 +
 advice.c                               |   1 +
 alias.c                                |   1 +
 apply.c                                |   3 +-
 archive-tar.c                          |   1 +
 archive-zip.c                          |   1 +
 archive.c                              |   1 +
 attr.c                                 |   1 +
 bisect.c                               |   1 +
 branch.c                               |   1 +
 builtin/add.c                          |   1 +
 builtin/am.c                           |   1 +
 builtin/blame.c                        |   3 +-
 builtin/branch.c                       |   1 +
 builtin/cat-file.c                     |   1 +
 builtin/check-attr.c                   |   1 +
 builtin/check-ignore.c                 |   1 +
 builtin/check-mailmap.c                |   1 +
 builtin/checkout-index.c               |   1 +
 builtin/checkout.c                     |   1 +
 builtin/clean.c                        |   1 +
 builtin/clone.c                        |   1 +
 builtin/column.c                       |   1 +
 builtin/commit-tree.c                  |   1 +
 builtin/commit.c                       |   4 +-
 builtin/config.c                       |   3 +
 builtin/count-objects.c                |   1 +
 builtin/describe.c                     |   1 +
 builtin/diff-files.c                   |   1 +
 builtin/diff-index.c                   |   1 +
 builtin/diff-tree.c                    |   1 +
 builtin/diff.c                         |   1 +
 builtin/difftool.c                     |   1 +
 builtin/fast-export.c                  |   1 +
 builtin/fetch.c                        |   1 +
 builtin/fmt-merge-msg.c                |   1 +
 builtin/for-each-ref.c                 |   1 +
 builtin/fsck.c                         |   1 +
 builtin/gc.c                           |   1 +
 builtin/grep.c                         |   4 +-
 builtin/hash-object.c                  |   1 +
 builtin/help.c                         |   1 +
 builtin/index-pack.c                   |   1 +
 builtin/init-db.c                      |   1 +
 builtin/log.c                          |   1 +
 builtin/ls-files.c                     | 330 ++++++++++++++++-----------------
 builtin/ls-tree.c                      |   1 +
 builtin/merge-base.c                   |   1 +
 builtin/merge-file.c                   |   1 +
 builtin/merge.c                        |   1 +
 builtin/mv.c                           |   1 +
 builtin/name-rev.c                     |   1 +
 builtin/notes.c                        |   1 +
 builtin/pack-objects.c                 |   1 +
 builtin/patch-id.c                     |   1 +
 builtin/pull.c                         |   1 +
 builtin/push.c                         |   1 +
 builtin/read-tree.c                    |   1 +
 builtin/rebase--helper.c               |   1 +
 builtin/receive-pack.c                 |   1 +
 builtin/reflog.c                       |   1 +
 builtin/remote.c                       |   1 +
 builtin/repack.c                       |   1 +
 builtin/replace.c                      |   1 +
 builtin/rerere.c                       |   1 +
 builtin/reset.c                        |   1 +
 builtin/rev-list.c                     |   1 +
 builtin/rev-parse.c                    |   1 +
 builtin/revert.c                       |   1 +
 builtin/rm.c                           |   1 +
 builtin/send-pack.c                    |   1 +
 builtin/shortlog.c                     |   1 +
 builtin/show-branch.c                  |   1 +
 builtin/stripspace.c                   |   1 +
 builtin/submodule--helper.c            |  10 +-
 builtin/symbolic-ref.c                 |   1 +
 builtin/tag.c                          |   1 +
 builtin/unpack-file.c                  |   1 +
 builtin/unpack-objects.c               |   1 +
 builtin/update-index.c                 |   1 +
 builtin/update-ref.c                   |   1 +
 builtin/update-server-info.c           |   1 +
 builtin/var.c                          |   1 +
 builtin/verify-commit.c                |   1 +
 builtin/verify-pack.c                  |   1 +
 builtin/verify-tag.c                   |   1 +
 builtin/worktree.c                     |   1 +
 builtin/write-tree.c                   |   1 +
 cache.h                                | 196 +-------------------
 color.c                                |   1 +
 column.c                               |   1 +
 combine-diff.c                         |   2 +-
 config.c                               | 210 ++++++++++++++-------
 config.h                               | 217 ++++++++++++++++++++++
 connect.c                              |   1 +
 convert.c                              |  32 ++--
 convert.h                              |  19 +-
 credential-cache--daemon.c             |   1 +
 credential.c                           |   1 +
 daemon.c                               |   1 +
 diff.c                                 |   7 +-
 dir.c                                  |   3 +-
 environment.c                          | 101 ++++------
 fast-import.c                          |   1 +
 fetch-pack.c                           |   1 +
 git.c                                  |   3 +-
 gpg-interface.c                        |   1 +
 graph.c                                |   1 +
 grep.c                                 |   1 +
 help.c                                 |   1 +
 http-backend.c                         |   1 +
 http-fetch.c                           |   1 +
 http.c                                 |   1 +
 ident.c                                |   1 +
 imap-send.c                            |   1 +
 ll-merge.c                             |   3 +-
 log-tree.c                             |   1 +
 mailinfo.c                             |   1 +
 merge-recursive.c                      |   5 +-
 notes-utils.c                          |   1 +
 notes.c                                |   1 +
 pager.c                                |   1 +
 parse-options.c                        |   1 +
 path.c                                 |  11 +-
 pathspec.c                             |   1 +
 pretty.c                               |   1 +
 prompt.c                               |   1 +
 read-cache.c                           |   1 +
 refs.c                                 |   1 +
 refs/files-backend.c                   |   1 +
 remote-curl.c                          |   1 +
 remote.c                               |   1 +
 repository.c                           | 216 +++++++++++++++++++++
 repository.h                           |  45 +++++
 rerere.c                               |   1 +
 send-pack.c                            |   1 +
 sequencer.c                            |   1 +
 setup.c                                |  34 ++++
 sha1_file.c                            |   7 +-
 sha1_name.c                            |   1 +
 submodule-config.c                     |  71 +++++--
 submodule-config.h                     |  10 +
 submodule.c                            |  36 ++--
 submodule.h                            |   4 +-
 t/helper/test-config.c                 |   1 +
 t/helper/test-submodule-config.c       |   1 +
 t/t3007-ls-files-recurse-submodules.sh |  39 ++++
 trailer.c                              |   1 +
 transport.c                            |   1 +
 tree.c                                 |  28 ++-
 tree.h                                 |   3 +-
 unpack-trees.c                         |   1 +
 upload-pack.c                          |   1 +
 userdiff.c                             |   1 +
 versioncmp.c                           |   1 +
 wrapper.c                              |   1 +
 xdiff-interface.c                      |   1 +
 157 files changed, 1199 insertions(+), 586 deletions(-)
 create mode 100644 config.h
 create mode 100644 repository.c
 create mode 100644 repository.h

-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v2 01/32] config: create config.h
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 02/32] config: remove git_config_iter Brandon Williams
                     ` (32 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Move all config related declarations from cache.h to a new config.h
header file.  This makes cache.h smaller and allows for the opportunity
in a following patch to only include config.h when needed.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h  | 190 +------------------------------------------------------------
 config.h | 194 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 195 insertions(+), 189 deletions(-)
 create mode 100644 config.h

diff --git a/cache.h b/cache.h
index ae4c45d37..a27fee458 100644
--- a/cache.h
+++ b/cache.h
@@ -11,6 +11,7 @@
 #include "string-list.h"
 #include "pack-revindex.h"
 #include "hash.h"
+#include "config.h"
 
 #ifndef platform_SHA_CTX
 /*
@@ -1866,188 +1867,9 @@ extern int packed_object_info(struct packed_git *pack, off_t offset, struct obje
 /* Dumb servers support */
 extern int update_server_info(int);
 
-/* git_config_parse_key() returns these negated: */
-#define CONFIG_INVALID_KEY 1
-#define CONFIG_NO_SECTION_OR_NAME 2
-/* git_config_set_gently(), git_config_set_multivar_gently() return the above or these: */
-#define CONFIG_NO_LOCK -1
-#define CONFIG_INVALID_FILE 3
-#define CONFIG_NO_WRITE 4
-#define CONFIG_NOTHING_SET 5
-#define CONFIG_INVALID_PATTERN 6
-#define CONFIG_GENERIC_ERROR 7
-
-#define CONFIG_REGEX_NONE ((void *)1)
-
-struct git_config_source {
-	unsigned int use_stdin:1;
-	const char *file;
-	const char *blob;
-};
-
-enum config_origin_type {
-	CONFIG_ORIGIN_BLOB,
-	CONFIG_ORIGIN_FILE,
-	CONFIG_ORIGIN_STDIN,
-	CONFIG_ORIGIN_SUBMODULE_BLOB,
-	CONFIG_ORIGIN_CMDLINE
-};
-
-struct config_options {
-	unsigned int respect_includes : 1;
-	const char *git_dir;
-};
-
-typedef int (*config_fn_t)(const char *, const char *, void *);
-extern int git_default_config(const char *, const char *, void *);
-extern int git_config_from_file(config_fn_t fn, const char *, void *);
-extern int git_config_from_mem(config_fn_t fn, const enum config_origin_type,
-					const char *name, const char *buf, size_t len, void *data);
-extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
-				     const unsigned char *sha1, void *data);
-extern void git_config_push_parameter(const char *text);
-extern int git_config_from_parameters(config_fn_t fn, void *data);
-extern void read_early_config(config_fn_t cb, void *data);
-extern void git_config(config_fn_t fn, void *);
-extern int git_config_with_options(config_fn_t fn, void *,
-				   struct git_config_source *config_source,
-				   const struct config_options *opts);
-extern int git_parse_ulong(const char *, unsigned long *);
-extern int git_parse_maybe_bool(const char *);
-extern int git_config_int(const char *, const char *);
-extern int64_t git_config_int64(const char *, const char *);
-extern unsigned long git_config_ulong(const char *, const char *);
-extern ssize_t git_config_ssize_t(const char *, const char *);
-extern int git_config_bool_or_int(const char *, const char *, int *);
-extern int git_config_bool(const char *, const char *);
-extern int git_config_maybe_bool(const char *, const char *);
-extern int git_config_string(const char **, const char *, const char *);
-extern int git_config_pathname(const char **, const char *, const char *);
-extern int git_config_set_in_file_gently(const char *, const char *, const char *);
-extern void git_config_set_in_file(const char *, const char *, const char *);
-extern int git_config_set_gently(const char *, const char *);
-extern void git_config_set(const char *, const char *);
-extern int git_config_parse_key(const char *, char **, int *);
-extern int git_config_key_is_valid(const char *key);
-extern int git_config_set_multivar_gently(const char *, const char *, const char *, int);
-extern void git_config_set_multivar(const char *, const char *, const char *, int);
-extern int git_config_set_multivar_in_file_gently(const char *, const char *, const char *, const char *, int);
-extern void git_config_set_multivar_in_file(const char *, const char *, const char *, const char *, int);
-extern int git_config_rename_section(const char *, const char *);
-extern int git_config_rename_section_in_file(const char *, const char *, const char *);
-extern const char *git_etc_gitconfig(void);
-extern int git_env_bool(const char *, int);
-extern unsigned long git_env_ulong(const char *, unsigned long);
-extern int git_config_system(void);
-extern int config_error_nonbool(const char *);
-#if defined(__GNUC__)
-#define config_error_nonbool(s) (config_error_nonbool(s), const_error())
-#endif
 extern const char *get_log_output_encoding(void);
 extern const char *get_commit_output_encoding(void);
 
-extern int git_config_parse_parameter(const char *, config_fn_t fn, void *data);
-
-enum config_scope {
-	CONFIG_SCOPE_UNKNOWN = 0,
-	CONFIG_SCOPE_SYSTEM,
-	CONFIG_SCOPE_GLOBAL,
-	CONFIG_SCOPE_REPO,
-	CONFIG_SCOPE_CMDLINE,
-};
-
-extern enum config_scope current_config_scope(void);
-extern const char *current_config_origin_type(void);
-extern const char *current_config_name(void);
-
-struct config_include_data {
-	int depth;
-	config_fn_t fn;
-	void *data;
-	const struct config_options *opts;
-};
-#define CONFIG_INCLUDE_INIT { 0 }
-extern int git_config_include(const char *name, const char *value, void *data);
-
-/*
- * Match and parse a config key of the form:
- *
- *   section.(subsection.)?key
- *
- * (i.e., what gets handed to a config_fn_t). The caller provides the section;
- * we return -1 if it does not match, 0 otherwise. The subsection and key
- * out-parameters are filled by the function (and *subsection is NULL if it is
- * missing).
- *
- * If the subsection pointer-to-pointer passed in is NULL, returns 0 only if
- * there is no subsection at all.
- */
-extern int parse_config_key(const char *var,
-			    const char *section,
-			    const char **subsection, int *subsection_len,
-			    const char **key);
-
-struct config_set_element {
-	struct hashmap_entry ent;
-	char *key;
-	struct string_list value_list;
-};
-
-struct configset_list_item {
-	struct config_set_element *e;
-	int value_index;
-};
-
-/*
- * the contents of the list are ordered according to their
- * position in the config files and order of parsing the files.
- * (i.e. key-value pair at the last position of .git/config will
- * be at the last item of the list)
- */
-struct configset_list {
-	struct configset_list_item *items;
-	unsigned int nr, alloc;
-};
-
-struct config_set {
-	struct hashmap config_hash;
-	int hash_initialized;
-	struct configset_list list;
-};
-
-extern void git_configset_init(struct config_set *cs);
-extern int git_configset_add_file(struct config_set *cs, const char *filename);
-extern int git_configset_get_value(struct config_set *cs, const char *key, const char **value);
-extern const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key);
-extern void git_configset_clear(struct config_set *cs);
-extern int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest);
-extern int git_configset_get_string(struct config_set *cs, const char *key, char **dest);
-extern int git_configset_get_int(struct config_set *cs, const char *key, int *dest);
-extern int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest);
-extern int git_configset_get_bool(struct config_set *cs, const char *key, int *dest);
-extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest);
-extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
-extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
-
-extern int git_config_get_value(const char *key, const char **value);
-extern const struct string_list *git_config_get_value_multi(const char *key);
-extern void git_config_clear(void);
-extern void git_config_iter(config_fn_t fn, void *data);
-extern int git_config_get_string_const(const char *key, const char **dest);
-extern int git_config_get_string(const char *key, char **dest);
-extern int git_config_get_int(const char *key, int *dest);
-extern int git_config_get_ulong(const char *key, unsigned long *dest);
-extern int git_config_get_bool(const char *key, int *dest);
-extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
-extern int git_config_get_maybe_bool(const char *key, int *dest);
-extern int git_config_get_pathname(const char *key, const char **dest);
-extern int git_config_get_untracked_cache(void);
-extern int git_config_get_split_index(void);
-extern int git_config_get_max_percent_split_change(void);
-
-/* This dies if the configured or default date is in the future */
-extern int git_config_get_expiry(const char *key, const char **output);
-
 /*
  * This is a hack for test programs like test-dump-untracked-cache to
  * ensure that they do not modify the untracked cache when reading it.
@@ -2055,16 +1877,6 @@ extern int git_config_get_expiry(const char *key, const char **output);
  */
 extern int ignore_untracked_cache_config;
 
-struct key_value_info {
-	const char *filename;
-	int linenr;
-	enum config_origin_type origin_type;
-	enum config_scope scope;
-};
-
-extern NORETURN void git_die_config(const char *key, const char *err, ...) __attribute__((format(printf, 2, 3)));
-extern NORETURN void git_die_config_linenr(const char *key, const char *filename, int linenr);
-
 extern int committer_ident_sufficiently_given(void);
 extern int author_ident_sufficiently_given(void);
 
diff --git a/config.h b/config.h
new file mode 100644
index 000000000..f7f8b66c5
--- /dev/null
+++ b/config.h
@@ -0,0 +1,194 @@
+#ifndef CONFIG_H
+#define CONFIG_H
+
+/* git_config_parse_key() returns these negated: */
+#define CONFIG_INVALID_KEY 1
+#define CONFIG_NO_SECTION_OR_NAME 2
+/* git_config_set_gently(), git_config_set_multivar_gently() return the above or these: */
+#define CONFIG_NO_LOCK -1
+#define CONFIG_INVALID_FILE 3
+#define CONFIG_NO_WRITE 4
+#define CONFIG_NOTHING_SET 5
+#define CONFIG_INVALID_PATTERN 6
+#define CONFIG_GENERIC_ERROR 7
+
+#define CONFIG_REGEX_NONE ((void *)1)
+
+struct git_config_source {
+	unsigned int use_stdin:1;
+	const char *file;
+	const char *blob;
+};
+
+enum config_origin_type {
+	CONFIG_ORIGIN_BLOB,
+	CONFIG_ORIGIN_FILE,
+	CONFIG_ORIGIN_STDIN,
+	CONFIG_ORIGIN_SUBMODULE_BLOB,
+	CONFIG_ORIGIN_CMDLINE
+};
+
+struct config_options {
+	unsigned int respect_includes : 1;
+	const char *git_dir;
+};
+
+typedef int (*config_fn_t)(const char *, const char *, void *);
+extern int git_default_config(const char *, const char *, void *);
+extern int git_config_from_file(config_fn_t fn, const char *, void *);
+extern int git_config_from_mem(config_fn_t fn, const enum config_origin_type,
+					const char *name, const char *buf, size_t len, void *data);
+extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
+				     const unsigned char *sha1, void *data);
+extern void git_config_push_parameter(const char *text);
+extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
+extern void git_config(config_fn_t fn, void *);
+extern int git_config_with_options(config_fn_t fn, void *,
+				   struct git_config_source *config_source,
+				   const struct config_options *opts);
+extern int git_parse_ulong(const char *, unsigned long *);
+extern int git_parse_maybe_bool(const char *);
+extern int git_config_int(const char *, const char *);
+extern int64_t git_config_int64(const char *, const char *);
+extern unsigned long git_config_ulong(const char *, const char *);
+extern ssize_t git_config_ssize_t(const char *, const char *);
+extern int git_config_bool_or_int(const char *, const char *, int *);
+extern int git_config_bool(const char *, const char *);
+extern int git_config_maybe_bool(const char *, const char *);
+extern int git_config_string(const char **, const char *, const char *);
+extern int git_config_pathname(const char **, const char *, const char *);
+extern int git_config_set_in_file_gently(const char *, const char *, const char *);
+extern void git_config_set_in_file(const char *, const char *, const char *);
+extern int git_config_set_gently(const char *, const char *);
+extern void git_config_set(const char *, const char *);
+extern int git_config_parse_key(const char *, char **, int *);
+extern int git_config_key_is_valid(const char *key);
+extern int git_config_set_multivar_gently(const char *, const char *, const char *, int);
+extern void git_config_set_multivar(const char *, const char *, const char *, int);
+extern int git_config_set_multivar_in_file_gently(const char *, const char *, const char *, const char *, int);
+extern void git_config_set_multivar_in_file(const char *, const char *, const char *, const char *, int);
+extern int git_config_rename_section(const char *, const char *);
+extern int git_config_rename_section_in_file(const char *, const char *, const char *);
+extern const char *git_etc_gitconfig(void);
+extern int git_env_bool(const char *, int);
+extern unsigned long git_env_ulong(const char *, unsigned long);
+extern int git_config_system(void);
+extern int config_error_nonbool(const char *);
+#if defined(__GNUC__)
+#define config_error_nonbool(s) (config_error_nonbool(s), const_error())
+#endif
+
+extern int git_config_parse_parameter(const char *, config_fn_t fn, void *data);
+
+enum config_scope {
+	CONFIG_SCOPE_UNKNOWN = 0,
+	CONFIG_SCOPE_SYSTEM,
+	CONFIG_SCOPE_GLOBAL,
+	CONFIG_SCOPE_REPO,
+	CONFIG_SCOPE_CMDLINE,
+};
+
+extern enum config_scope current_config_scope(void);
+extern const char *current_config_origin_type(void);
+extern const char *current_config_name(void);
+
+struct config_include_data {
+	int depth;
+	config_fn_t fn;
+	void *data;
+	const struct config_options *opts;
+};
+#define CONFIG_INCLUDE_INIT { 0 }
+extern int git_config_include(const char *name, const char *value, void *data);
+
+/*
+ * Match and parse a config key of the form:
+ *
+ *   section.(subsection.)?key
+ *
+ * (i.e., what gets handed to a config_fn_t). The caller provides the section;
+ * we return -1 if it does not match, 0 otherwise. The subsection and key
+ * out-parameters are filled by the function (and *subsection is NULL if it is
+ * missing).
+ *
+ * If the subsection pointer-to-pointer passed in is NULL, returns 0 only if
+ * there is no subsection at all.
+ */
+extern int parse_config_key(const char *var,
+			    const char *section,
+			    const char **subsection, int *subsection_len,
+			    const char **key);
+
+struct config_set_element {
+	struct hashmap_entry ent;
+	char *key;
+	struct string_list value_list;
+};
+
+struct configset_list_item {
+	struct config_set_element *e;
+	int value_index;
+};
+
+/*
+ * the contents of the list are ordered according to their
+ * position in the config files and order of parsing the files.
+ * (i.e. key-value pair at the last position of .git/config will
+ * be at the last item of the list)
+ */
+struct configset_list {
+	struct configset_list_item *items;
+	unsigned int nr, alloc;
+};
+
+struct config_set {
+	struct hashmap config_hash;
+	int hash_initialized;
+	struct configset_list list;
+};
+
+extern void git_configset_init(struct config_set *cs);
+extern int git_configset_add_file(struct config_set *cs, const char *filename);
+extern int git_configset_get_value(struct config_set *cs, const char *key, const char **value);
+extern const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key);
+extern void git_configset_clear(struct config_set *cs);
+extern int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest);
+extern int git_configset_get_string(struct config_set *cs, const char *key, char **dest);
+extern int git_configset_get_int(struct config_set *cs, const char *key, int *dest);
+extern int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest);
+extern int git_configset_get_bool(struct config_set *cs, const char *key, int *dest);
+extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest);
+extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
+extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
+
+extern int git_config_get_value(const char *key, const char **value);
+extern const struct string_list *git_config_get_value_multi(const char *key);
+extern void git_config_clear(void);
+extern void git_config_iter(config_fn_t fn, void *data);
+extern int git_config_get_string_const(const char *key, const char **dest);
+extern int git_config_get_string(const char *key, char **dest);
+extern int git_config_get_int(const char *key, int *dest);
+extern int git_config_get_ulong(const char *key, unsigned long *dest);
+extern int git_config_get_bool(const char *key, int *dest);
+extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
+extern int git_config_get_maybe_bool(const char *key, int *dest);
+extern int git_config_get_pathname(const char *key, const char **dest);
+extern int git_config_get_untracked_cache(void);
+extern int git_config_get_split_index(void);
+extern int git_config_get_max_percent_split_change(void);
+
+/* This dies if the configured or default date is in the future */
+extern int git_config_get_expiry(const char *key, const char **output);
+
+struct key_value_info {
+	const char *filename;
+	int linenr;
+	enum config_origin_type origin_type;
+	enum config_scope scope;
+};
+
+extern NORETURN void git_die_config(const char *key, const char *err, ...) __attribute__((format(printf, 2, 3)));
+extern NORETURN void git_die_config_linenr(const char *key, const char *filename, int linenr);
+
+#endif /* CONFIG_H */
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 02/32] config: remove git_config_iter
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 01/32] config: create config.h Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 03/32] config: don't include config.h by default Brandon Williams
                     ` (31 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Since there is no implementation of the function 'git_config_iter' lets
stop exporting it and remove the prototype from config.h.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 config.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/config.h b/config.h
index f7f8b66c5..c70599bd5 100644
--- a/config.h
+++ b/config.h
@@ -165,7 +165,6 @@ extern int git_configset_get_pathname(struct config_set *cs, const char *key, co
 extern int git_config_get_value(const char *key, const char **value);
 extern const struct string_list *git_config_get_value_multi(const char *key);
 extern void git_config_clear(void);
-extern void git_config_iter(config_fn_t fn, void *data);
 extern int git_config_get_string_const(const char *key, const char **dest);
 extern int git_config_get_string(const char *key, char **dest);
 extern int git_config_get_int(const char *key, int *dest);
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 03/32] config: don't include config.h by default
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 01/32] config: create config.h Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 02/32] config: remove git_config_iter Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 04/32] config: don't implicitly use gitdir Brandon Williams
                     ` (30 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Stop including config.h by default in cache.h.  Instead only include
config.h in those files which require use of the config system.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 advice.c                         | 1 +
 alias.c                          | 1 +
 apply.c                          | 1 +
 archive-tar.c                    | 1 +
 archive-zip.c                    | 1 +
 archive.c                        | 1 +
 attr.c                           | 1 +
 bisect.c                         | 1 +
 branch.c                         | 1 +
 builtin/add.c                    | 1 +
 builtin/am.c                     | 1 +
 builtin/blame.c                  | 1 +
 builtin/branch.c                 | 1 +
 builtin/cat-file.c               | 1 +
 builtin/check-attr.c             | 1 +
 builtin/check-ignore.c           | 1 +
 builtin/check-mailmap.c          | 1 +
 builtin/checkout-index.c         | 1 +
 builtin/checkout.c               | 1 +
 builtin/clean.c                  | 1 +
 builtin/clone.c                  | 1 +
 builtin/column.c                 | 1 +
 builtin/commit-tree.c            | 1 +
 builtin/commit.c                 | 1 +
 builtin/config.c                 | 1 +
 builtin/count-objects.c          | 1 +
 builtin/describe.c               | 1 +
 builtin/diff-files.c             | 1 +
 builtin/diff-index.c             | 1 +
 builtin/diff-tree.c              | 1 +
 builtin/diff.c                   | 1 +
 builtin/difftool.c               | 1 +
 builtin/fast-export.c            | 1 +
 builtin/fetch.c                  | 1 +
 builtin/fmt-merge-msg.c          | 1 +
 builtin/for-each-ref.c           | 1 +
 builtin/fsck.c                   | 1 +
 builtin/gc.c                     | 1 +
 builtin/grep.c                   | 1 +
 builtin/hash-object.c            | 1 +
 builtin/help.c                   | 1 +
 builtin/index-pack.c             | 1 +
 builtin/init-db.c                | 1 +
 builtin/log.c                    | 1 +
 builtin/ls-files.c               | 1 +
 builtin/ls-tree.c                | 1 +
 builtin/merge-base.c             | 1 +
 builtin/merge-file.c             | 1 +
 builtin/merge.c                  | 1 +
 builtin/mv.c                     | 1 +
 builtin/name-rev.c               | 1 +
 builtin/notes.c                  | 1 +
 builtin/pack-objects.c           | 1 +
 builtin/patch-id.c               | 1 +
 builtin/pull.c                   | 1 +
 builtin/push.c                   | 1 +
 builtin/read-tree.c              | 1 +
 builtin/rebase--helper.c         | 1 +
 builtin/receive-pack.c           | 1 +
 builtin/reflog.c                 | 1 +
 builtin/remote.c                 | 1 +
 builtin/repack.c                 | 1 +
 builtin/replace.c                | 1 +
 builtin/rerere.c                 | 1 +
 builtin/reset.c                  | 1 +
 builtin/rev-list.c               | 1 +
 builtin/rev-parse.c              | 1 +
 builtin/revert.c                 | 1 +
 builtin/rm.c                     | 1 +
 builtin/send-pack.c              | 1 +
 builtin/shortlog.c               | 1 +
 builtin/show-branch.c            | 1 +
 builtin/stripspace.c             | 1 +
 builtin/submodule--helper.c      | 1 +
 builtin/symbolic-ref.c           | 1 +
 builtin/tag.c                    | 1 +
 builtin/unpack-file.c            | 1 +
 builtin/unpack-objects.c         | 1 +
 builtin/update-index.c           | 1 +
 builtin/update-ref.c             | 1 +
 builtin/update-server-info.c     | 1 +
 builtin/var.c                    | 1 +
 builtin/verify-commit.c          | 1 +
 builtin/verify-pack.c            | 1 +
 builtin/verify-tag.c             | 1 +
 builtin/worktree.c               | 1 +
 builtin/write-tree.c             | 1 +
 cache.h                          | 1 -
 color.c                          | 1 +
 column.c                         | 1 +
 config.c                         | 1 +
 connect.c                        | 1 +
 convert.c                        | 1 +
 credential-cache--daemon.c       | 1 +
 credential.c                     | 1 +
 daemon.c                         | 1 +
 diff.c                           | 1 +
 dir.c                            | 1 +
 environment.c                    | 1 +
 fast-import.c                    | 1 +
 fetch-pack.c                     | 1 +
 git.c                            | 1 +
 gpg-interface.c                  | 1 +
 graph.c                          | 1 +
 grep.c                           | 1 +
 help.c                           | 1 +
 http-backend.c                   | 1 +
 http-fetch.c                     | 1 +
 http.c                           | 1 +
 ident.c                          | 1 +
 imap-send.c                      | 1 +
 ll-merge.c                       | 1 +
 log-tree.c                       | 1 +
 mailinfo.c                       | 1 +
 merge-recursive.c                | 1 +
 notes-utils.c                    | 1 +
 notes.c                          | 1 +
 pager.c                          | 1 +
 parse-options.c                  | 1 +
 pathspec.c                       | 1 +
 pretty.c                         | 1 +
 prompt.c                         | 1 +
 read-cache.c                     | 1 +
 refs.c                           | 1 +
 refs/files-backend.c             | 1 +
 remote-curl.c                    | 1 +
 remote.c                         | 1 +
 rerere.c                         | 1 +
 send-pack.c                      | 1 +
 sequencer.c                      | 1 +
 setup.c                          | 1 +
 sha1_file.c                      | 1 +
 sha1_name.c                      | 1 +
 submodule-config.c               | 1 +
 submodule.c                      | 1 +
 t/helper/test-config.c           | 1 +
 t/helper/test-submodule-config.c | 1 +
 trailer.c                        | 1 +
 transport.c                      | 1 +
 unpack-trees.c                   | 1 +
 upload-pack.c                    | 1 +
 userdiff.c                       | 1 +
 versioncmp.c                     | 1 +
 wrapper.c                        | 1 +
 xdiff-interface.c                | 1 +
 145 files changed, 144 insertions(+), 1 deletion(-)

diff --git a/advice.c b/advice.c
index b84ae4960..3fa04fca0 100644
--- a/advice.c
+++ b/advice.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 
 int advice_push_update_rejected = 1;
 int advice_push_non_ff_current = 1;
diff --git a/alias.c b/alias.c
index 3b90397a9..052f34136 100644
--- a/alias.c
+++ b/alias.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 
 char *alias_lookup(const char *alias)
 {
diff --git a/apply.c b/apply.c
index c49cef063..87db9618d 100644
--- a/apply.c
+++ b/apply.c
@@ -8,6 +8,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "blob.h"
 #include "delta.h"
 #include "diff.h"
diff --git a/archive-tar.c b/archive-tar.c
index 073e60ebd..c6ed96ee7 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -2,6 +2,7 @@
  * Copyright (c) 2005, 2006 Rene Scharfe
  */
 #include "cache.h"
+#include "config.h"
 #include "tar.h"
 #include "archive.h"
 #include "streaming.h"
diff --git a/archive-zip.c b/archive-zip.c
index 27563e9e2..e8913e5a2 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -2,6 +2,7 @@
  * Copyright (c) 2006 Rene Scharfe
  */
 #include "cache.h"
+#include "config.h"
 #include "archive.h"
 #include "streaming.h"
 #include "utf8.h"
diff --git a/archive.c b/archive.c
index b15a922da..60b3035a7 100644
--- a/archive.c
+++ b/archive.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "tree-walk.h"
diff --git a/attr.c b/attr.c
index 7e2134471..9f8b02936 100644
--- a/attr.c
+++ b/attr.c
@@ -9,6 +9,7 @@
 
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "exec_cmd.h"
 #include "attr.h"
 #include "dir.h"
diff --git a/bisect.c b/bisect.c
index d88f9bc3d..c2acb9455 100644
--- a/bisect.c
+++ b/bisect.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/branch.c b/branch.c
index 985316eb7..a8a548ccf 100644
--- a/branch.c
+++ b/branch.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 #include "cache.h"
+#include "config.h"
 #include "branch.h"
 #include "refs.h"
 #include "remote.h"
diff --git a/builtin/add.c b/builtin/add.c
index d9a2491e4..f2415e99f 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -4,6 +4,7 @@
  * Copyright (C) 2006 Linus Torvalds
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "lockfile.h"
 #include "dir.h"
diff --git a/builtin/am.c b/builtin/am.c
index 0f63dcab1..0b6b1265d 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -4,6 +4,7 @@
  * Based on git-am.sh by Junio C Hamano.
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "exec_cmd.h"
 #include "parse-options.h"
diff --git a/builtin/blame.c b/builtin/blame.c
index 1043e5376..c0ae49298 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -6,6 +6,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "builtin.h"
 #include "blob.h"
diff --git a/builtin/branch.c b/builtin/branch.c
index 83fcda43d..c958e9325 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -6,6 +6,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "color.h"
 #include "refs.h"
 #include "commit.h"
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 9af863e79..7bc45f600 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "userdiff.h"
diff --git a/builtin/check-attr.c b/builtin/check-attr.c
index 4d01ca0c8..91444dc04 100644
--- a/builtin/check-attr.c
+++ b/builtin/check-attr.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index c7b8c0889..3e280b9c7 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "quote.h"
 #include "pathspec.h"
diff --git a/builtin/check-mailmap.c b/builtin/check-mailmap.c
index cf0f54f6b..cdce144f3 100644
--- a/builtin/check-mailmap.c
+++ b/builtin/check-mailmap.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "mailmap.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c
index 07631d0c9..39c8be05d 100644
--- a/builtin/checkout-index.c
+++ b/builtin/checkout-index.c
@@ -5,6 +5,7 @@
  *
  */
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "quote.h"
 #include "cache-tree.h"
diff --git a/builtin/checkout.c b/builtin/checkout.c
index a6b2af39d..45fab5a4d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "parse-options.h"
 #include "refs.h"
diff --git a/builtin/clean.c b/builtin/clean.c
index 329b68c40..09adf0551 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -8,6 +8,7 @@
 
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/clone.c b/builtin/clone.c
index 743f16ae2..4922a5496 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -9,6 +9,7 @@
  */
 
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "parse-options.h"
 #include "fetch-pack.h"
diff --git a/builtin/column.c b/builtin/column.c
index 33314b4d7..0c3223d64 100644
--- a/builtin/column.c
+++ b/builtin/column.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "strbuf.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index f39c2b273..a4a923d7c 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "tree.h"
 #include "builtin.h"
diff --git a/builtin/commit.c b/builtin/commit.c
index da1ba4c86..805da4915 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -6,6 +6,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "cache-tree.h"
 #include "color.h"
diff --git a/builtin/config.c b/builtin/config.c
index 7f6c25d4d..753c40a5c 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "color.h"
 #include "parse-options.h"
 #include "urlmatch.h"
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index acb05940f..1d82e61f2 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -5,6 +5,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "builtin.h"
 #include "parse-options.h"
diff --git a/builtin/describe.c b/builtin/describe.c
index 893c8789f..70eb14460 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "commit.h"
 #include "tag.h"
diff --git a/builtin/diff-files.c b/builtin/diff-files.c
index 15c61fd8d..29d3e4f41 100644
--- a/builtin/diff-files.c
+++ b/builtin/diff-files.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "revision.h"
diff --git a/builtin/diff-index.c b/builtin/diff-index.c
index 1af373d00..e3d418361 100644
--- a/builtin/diff-index.c
+++ b/builtin/diff-index.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "revision.h"
diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index 5ea1c1431..9af942b6b 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "log-tree.h"
diff --git a/builtin/diff.c b/builtin/diff.c
index 8c03ddaf5..b85af112a 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "color.h"
 #include "commit.h"
diff --git a/builtin/difftool.c b/builtin/difftool.c
index b9a892f26..9199227f6 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -12,6 +12,7 @@
  * Copyright (C) 2016 Johannes Schindelin
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "run-command.h"
 #include "exec_cmd.h"
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 24e29ad7e..0dfd980dd 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -5,6 +5,7 @@
  */
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "object.h"
diff --git a/builtin/fetch.c b/builtin/fetch.c
index d4d573b98..284595911 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -2,6 +2,7 @@
  * "git fetch"
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "builtin.h"
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index 70137b0b7..10cbb4341 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c
index eca365bf8..52be99cba 100644
--- a/builtin/for-each-ref.c
+++ b/builtin/for-each-ref.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "object.h"
 #include "parse-options.h"
diff --git a/builtin/fsck.c b/builtin/fsck.c
index cb2ba6cd1..90593491e 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
diff --git a/builtin/gc.c b/builtin/gc.c
index f484eda43..bd91f136f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
 #include "parse-options.h"
diff --git a/builtin/grep.c b/builtin/grep.c
index c6c26e9b9..0f4a1e5a3 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index bbeaf20bc..d04baf999 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -5,6 +5,7 @@
  * Copyright (C) Junio C Hamano, 2005
  */
 #include "builtin.h"
+#include "config.h"
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/help.c b/builtin/help.c
index 49f7a07f8..334a8494a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -2,6 +2,7 @@
  * Builtin help command
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "exec_cmd.h"
 #include "parse-options.h"
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 04b9dcaf0..edc1a91d8 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "delta.h"
 #include "pack.h"
 #include "csum-file.h"
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 8a6acb0ec..47823f9aa 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "builtin.h"
 #include "exec_cmd.h"
diff --git a/builtin/log.c b/builtin/log.c
index a440601ef..72a9fd968 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -5,6 +5,7 @@
  *		 2006 Junio Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "color.h"
 #include "commit.h"
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index b376afc31..c4357dc30 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -6,6 +6,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "quote.h"
 #include "dir.h"
 #include "builtin.h"
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index ee7b293b1..ef965408e 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/builtin/merge-base.c b/builtin/merge-base.c
index 0c36a70ad..6dbd167d3 100644
--- a/builtin/merge-base.c
+++ b/builtin/merge-base.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "diff.h"
diff --git a/builtin/merge-file.c b/builtin/merge-file.c
index 47dde7c39..b08803e61 100644
--- a/builtin/merge-file.c
+++ b/builtin/merge-file.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "xdiff/xdiff.h"
 #include "xdiff-interface.h"
 #include "parse-options.h"
diff --git a/builtin/merge.c b/builtin/merge.c
index a4a098f40..2512370c1 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -7,6 +7,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "builtin.h"
 #include "lockfile.h"
diff --git a/builtin/mv.c b/builtin/mv.c
index 61d20037a..dcf6736b5 100644
--- a/builtin/mv.c
+++ b/builtin/mv.c
@@ -4,6 +4,7 @@
  * Copyright (C) 2006 Johannes Schindelin
  */
 #include "builtin.h"
+#include "config.h"
 #include "pathspec.h"
 #include "lockfile.h"
 #include "dir.h"
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 7fc7e66e8..e21715f1d 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "tag.h"
 #include "refs.h"
diff --git a/builtin/notes.c b/builtin/notes.c
index f2847c41e..231ad1f01 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -8,6 +8,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "notes.h"
 #include "blob.h"
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 80439047a..9fe1fa8e1 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "object.h"
 #include "blob.h"
diff --git a/builtin/patch-id.c b/builtin/patch-id.c
index 81552e02e..970d0d30b 100644
--- a/builtin/patch-id.c
+++ b/builtin/patch-id.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 
 static void flush_current_id(int patchlen, struct object_id *id, struct object_id *result)
 {
diff --git a/builtin/pull.c b/builtin/pull.c
index 318c273eb..af325d9fc 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -6,6 +6,7 @@
  * Fetch one or more remote refs and merge it/them into the current HEAD.
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "exec_cmd.h"
diff --git a/builtin/push.c b/builtin/push.c
index a597759d8..76aa713d2 100644
--- a/builtin/push.c
+++ b/builtin/push.c
@@ -2,6 +2,7 @@
  * "git push"
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "run-command.h"
 #include "builtin.h"
diff --git a/builtin/read-tree.c b/builtin/read-tree.c
index 78d319365..6eb2a0e57 100644
--- a/builtin/read-tree.c
+++ b/builtin/read-tree.c
@@ -5,6 +5,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "object.h"
 #include "tree.h"
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ca1ebb2fa..c82b4dce6 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "sequencer.h"
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b1706a573..71c0c768d 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "pack.h"
 #include "refs.h"
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 920c16dac..44cdc2dbd 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "commit.h"
 #include "refs.h"
diff --git a/builtin/remote.c b/builtin/remote.c
index addf97ad2..a470ed7c6 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "parse-options.h"
 #include "transport.h"
 #include "remote.h"
diff --git a/builtin/repack.c b/builtin/repack.c
index 38ba4ef82..f17a68a17 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "parse-options.h"
 #include "run-command.h"
diff --git a/builtin/replace.c b/builtin/replace.c
index c921bc976..80a15cf35 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -9,6 +9,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "refs.h"
 #include "parse-options.h"
diff --git a/builtin/rerere.c b/builtin/rerere.c
index 1bf72423b..ffb66e290 100644
--- a/builtin/rerere.c
+++ b/builtin/rerere.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "parse-options.h"
 #include "string-list.h"
diff --git a/builtin/reset.c b/builtin/reset.c
index 430602d10..d0b604103 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -8,6 +8,7 @@
  * Copyright (c) 2005, 2006 Linus Torvalds and Junio C Hamano
  */
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "tag.h"
 #include "object.h"
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 718c6059c..f8e2b7bf5 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index efdc14473..c78b7b33d 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "quote.h"
diff --git a/builtin/revert.c b/builtin/revert.c
index 345d9586a..16028b9ea 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "diff.h"
diff --git a/builtin/rm.c b/builtin/rm.c
index 7c323d012..824d1de9e 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds 2006
  */
 #include "builtin.h"
+#include "config.h"
 #include "lockfile.h"
 #include "dir.h"
 #include "cache-tree.h"
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index b8e2e74fe..633e0c3cd 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "pkt-line.h"
diff --git a/builtin/shortlog.c b/builtin/shortlog.c
index 7cff1839f..43c4799ea 100644
--- a/builtin/shortlog.c
+++ b/builtin/shortlog.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "diff.h"
 #include "string-list.h"
diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index 4a6cc6f49..e4cf1b5bb 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "builtin.h"
diff --git a/builtin/stripspace.c b/builtin/stripspace.c
index 1e62a008c..bdf032886 100644
--- a/builtin/stripspace.c
+++ b/builtin/stripspace.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "strbuf.h"
 
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 8cc648d85..4dcbfb952 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "parse-options.h"
 #include "quote.h"
 #include "pathspec.h"
diff --git a/builtin/symbolic-ref.c b/builtin/symbolic-ref.c
index 70addef15..df75cb9d4 100644
--- a/builtin/symbolic-ref.c
+++ b/builtin/symbolic-ref.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "cache.h"
 #include "refs.h"
 #include "parse-options.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 1f74a56db..01154ea8d 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -7,6 +7,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "refs.h"
 #include "tag.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index 6fc6bcdf7..73f133419 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 
 static char *create_temp_file(unsigned char *sha1)
 {
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 8bc999776..193f8b9d5 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "object.h"
 #include "delta.h"
 #include "pack.h"
diff --git a/builtin/update-index.c b/builtin/update-index.c
index ebfc09faa..0a4c23648 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "quote.h"
 #include "cache-tree.h"
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 0b2ecf41a..40ccfc193 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "builtin.h"
 #include "parse-options.h"
diff --git a/builtin/update-server-info.c b/builtin/update-server-info.c
index 6c8cc3edc..873070e51 100644
--- a/builtin/update-server-info.c
+++ b/builtin/update-server-info.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "parse-options.h"
 
diff --git a/builtin/var.c b/builtin/var.c
index aedbb53a2..6c6f46b4a 100644
--- a/builtin/var.c
+++ b/builtin/var.c
@@ -4,6 +4,7 @@
  * Copyright (C) Eric Biederman, 2005
  */
 #include "builtin.h"
+#include "config.h"
 
 static const char var_usage[] = "git var (-l | <variable>)";
 
diff --git a/builtin/verify-commit.c b/builtin/verify-commit.c
index 05b734e6d..ba38ac9b1 100644
--- a/builtin/verify-commit.c
+++ b/builtin/verify-commit.c
@@ -6,6 +6,7 @@
  * Based on git-verify-tag
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "commit.h"
 #include "run-command.h"
diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index c94e15693..c2a1a5c50 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "parse-options.h"
 
diff --git a/builtin/verify-tag.c b/builtin/verify-tag.c
index 5199553d9..f9a5f7535 100644
--- a/builtin/verify-tag.c
+++ b/builtin/verify-tag.c
@@ -6,6 +6,7 @@
  * Based on git-verify-tag.sh
  */
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "tag.h"
 #include "run-command.h"
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 793306ea5..0c5476ee9 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "dir.h"
 #include "parse-options.h"
diff --git a/builtin/write-tree.c b/builtin/write-tree.c
index 084c0df78..bd0a78aa3 100644
--- a/builtin/write-tree.c
+++ b/builtin/write-tree.c
@@ -5,6 +5,7 @@
  */
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "tree.h"
 #include "cache-tree.h"
 #include "parse-options.h"
diff --git a/cache.h b/cache.h
index a27fee458..272f8e021 100644
--- a/cache.h
+++ b/cache.h
@@ -11,7 +11,6 @@
 #include "string-list.h"
 #include "pack-revindex.h"
 #include "hash.h"
-#include "config.h"
 
 #ifndef platform_SHA_CTX
 /*
diff --git a/color.c b/color.c
index dee61557e..31b6207a0 100644
--- a/color.c
+++ b/color.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "color.h"
 
 static int git_use_color_default = GIT_COLOR_AUTO;
diff --git a/column.c b/column.c
index d55ead18e..ff7bdab1a 100644
--- a/column.c
+++ b/column.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "column.h"
 #include "string-list.h"
 #include "parse-options.h"
diff --git a/config.c b/config.c
index 146cb3452..2390f98e3 100644
--- a/config.c
+++ b/config.c
@@ -6,6 +6,7 @@
  *
  */
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "exec_cmd.h"
 #include "strbuf.h"
diff --git a/connect.c b/connect.c
index cd21a1b6f..efddb30ea 100644
--- a/connect.c
+++ b/connect.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 #include "cache.h"
+#include "config.h"
 #include "pkt-line.h"
 #include "quote.h"
 #include "refs.h"
diff --git a/convert.c b/convert.c
index f1e168bc3..5f4a4b1f5 100644
--- a/convert.c
+++ b/convert.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "run-command.h"
 #include "quote.h"
diff --git a/credential-cache--daemon.c b/credential-cache--daemon.c
index f3814cc47..0d5c62509 100644
--- a/credential-cache--daemon.c
+++ b/credential-cache--daemon.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "tempfile.h"
 #include "credential.h"
 #include "unix-socket.h"
diff --git a/credential.c b/credential.c
index aa996669f..67a523353 100644
--- a/credential.c
+++ b/credential.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "credential.h"
 #include "string-list.h"
 #include "run-command.h"
diff --git a/daemon.c b/daemon.c
index ac7181a48..30747075f 100644
--- a/daemon.c
+++ b/daemon.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "pkt-line.h"
 #include "run-command.h"
 #include "strbuf.h"
diff --git a/diff.c b/diff.c
index f3546536b..2f2467c6d 100644
--- a/diff.c
+++ b/diff.c
@@ -2,6 +2,7 @@
  * Copyright (C) 2005 Junio C Hamano
  */
 #include "cache.h"
+#include "config.h"
 #include "tempfile.h"
 #include "quote.h"
 #include "diff.h"
diff --git a/dir.c b/dir.c
index 3f3167e55..0c26a53d2 100644
--- a/dir.c
+++ b/dir.c
@@ -9,6 +9,7 @@
  */
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "attr.h"
 #include "refs.h"
diff --git a/environment.c b/environment.c
index aa478e71d..d40b21fb7 100644
--- a/environment.c
+++ b/environment.c
@@ -8,6 +8,7 @@
  * are.
  */
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
diff --git a/fast-import.c b/fast-import.c
index e69d21968..2881d4898 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -154,6 +154,7 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "object.h"
 #include "blob.h"
diff --git a/fetch-pack.c b/fetch-pack.c
index cd86865be..fbbc99c88 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "refs.h"
 #include "pkt-line.h"
diff --git a/git.c b/git.c
index 8ff44f081..594436e43 100644
--- a/git.c
+++ b/git.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "exec_cmd.h"
 #include "help.h"
 #include "run-command.h"
diff --git a/gpg-interface.c b/gpg-interface.c
index e44cc27da..8ab32df45 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "gpg-interface.h"
diff --git a/graph.c b/graph.c
index 8b9049dd2..e7e20650d 100644
--- a/graph.c
+++ b/graph.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "color.h"
 #include "graph.h"
diff --git a/grep.c b/grep.c
index 47cee4506..d5211fc5a 100644
--- a/grep.c
+++ b/grep.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "grep.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
diff --git a/help.c b/help.c
index db7f3d79a..5d44f811a 100644
--- a/help.c
+++ b/help.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "builtin.h"
 #include "exec_cmd.h"
 #include "run-command.h"
diff --git a/http-backend.c b/http-backend.c
index ba5ff1aa2..519025d2c 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "object.h"
diff --git a/http-fetch.c b/http-fetch.c
index 3b556d661..8af380050 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "exec_cmd.h"
 #include "http.h"
 #include "walker.h"
diff --git a/http.c b/http.c
index d2e11ec6f..013bb0cc6 100644
--- a/http.c
+++ b/http.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 #include "http.h"
+#include "config.h"
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/ident.c b/ident.c
index bea871c8e..d41fc9119 100644
--- a/ident.c
+++ b/ident.c
@@ -6,6 +6,7 @@
  * Copyright (C) 2005 Linus Torvalds
  */
 #include "cache.h"
+#include "config.h"
 
 static struct strbuf git_default_name = STRBUF_INIT;
 static struct strbuf git_default_email = STRBUF_INIT;
diff --git a/imap-send.c b/imap-send.c
index 857591660..59e9b12d2 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -23,6 +23,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "credential.h"
 #include "exec_cmd.h"
 #include "run-command.h"
diff --git a/ll-merge.c b/ll-merge.c
index ac0d4a5d7..24ff94e1d 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -5,6 +5,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "attr.h"
 #include "xdiff-interface.h"
 #include "run-command.h"
diff --git a/log-tree.c b/log-tree.c
index a4ec11c2b..7e1dfd749 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "diff.h"
 #include "commit.h"
 #include "tag.h"
diff --git a/mailinfo.c b/mailinfo.c
index f92cb9f72..f59162453 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "utf8.h"
 #include "strbuf.h"
 #include "mailinfo.h"
diff --git a/merge-recursive.c b/merge-recursive.c
index ae5238d82..c2494f34f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -4,6 +4,7 @@
  * The thieves were Alex Riesen and Johannes Schindelin, in June/July 2006
  */
 #include "cache.h"
+#include "config.h"
 #include "advice.h"
 #include "lockfile.h"
 #include "cache-tree.h"
diff --git a/notes-utils.c b/notes-utils.c
index 325ff3daa..ebde2273c 100644
--- a/notes-utils.c
+++ b/notes-utils.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "notes-utils.h"
diff --git a/notes.c b/notes.c
index 542563b28..dbcfef4d7 100644
--- a/notes.c
+++ b/notes.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "notes.h"
 #include "blob.h"
 #include "tree.h"
diff --git a/pager.c b/pager.c
index c113d898a..4dd9e1b26 100644
--- a/pager.c
+++ b/pager.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "sigchain.h"
 
diff --git a/parse-options.c b/parse-options.c
index a23a1e67f..cbf84a604 100644
--- a/parse-options.c
+++ b/parse-options.c
@@ -1,6 +1,7 @@
 #include "git-compat-util.h"
 #include "parse-options.h"
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "color.h"
 #include "utf8.h"
diff --git a/pathspec.c b/pathspec.c
index 828405021..ecc5331c2 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -1,5 +1,6 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "pathspec.h"
 #include "attr.h"
diff --git a/pretty.c b/pretty.c
index 09701bd2f..9c9f81b5b 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "commit.h"
 #include "utf8.h"
 #include "diff.h"
diff --git a/prompt.c b/prompt.c
index 75406390c..6d5885d00 100644
--- a/prompt.c
+++ b/prompt.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "prompt.h"
diff --git a/read-cache.c b/read-cache.c
index 92eb15c00..2650ad98a 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -5,6 +5,7 @@
  */
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
 #include "cache-tree.h"
diff --git a/refs.c b/refs.c
index 8af9641aa..7dfdc08bf 100644
--- a/refs.c
+++ b/refs.c
@@ -3,6 +3,7 @@
  */
 
 #include "cache.h"
+#include "config.h"
 #include "hashmap.h"
 #include "lockfile.h"
 #include "iterator.h"
diff --git a/refs/files-backend.c b/refs/files-backend.c
index cb1f528cb..f88b94484 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1,4 +1,5 @@
 #include "../cache.h"
+#include "../config.h"
 #include "../refs.h"
 #include "refs-internal.h"
 #include "ref-cache.h"
diff --git a/remote-curl.c b/remote-curl.c
index ece45993d..0053b0954 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "remote.h"
 #include "strbuf.h"
 #include "walker.h"
diff --git a/remote.c b/remote.c
index fdc52d802..a84e68767 100644
--- a/remote.c
+++ b/remote.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "remote.h"
 #include "refs.h"
 #include "commit.h"
diff --git a/rerere.c b/rerere.c
index 3bd55caf3..344d6aa81 100644
--- a/rerere.c
+++ b/rerere.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "string-list.h"
 #include "rerere.h"
diff --git a/send-pack.c b/send-pack.c
index 78bb34ebe..ed3cee321 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "config.h"
 #include "commit.h"
 #include "refs.h"
 #include "pkt-line.h"
diff --git a/sequencer.c b/sequencer.c
index a23b948ac..b9962dc89 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "lockfile.h"
 #include "sequencer.h"
 #include "dir.h"
diff --git a/setup.c b/setup.c
index e3f7699a9..e99a82cbe 100644
--- a/setup.c
+++ b/setup.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "string-list.h"
 
diff --git a/sha1_file.c b/sha1_file.c
index 59a4ed2ed..44561e0b9 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -7,6 +7,7 @@
  * creation etc.
  */
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 #include "lockfile.h"
 #include "delta.h"
diff --git a/sha1_name.c b/sha1_name.c
index 389276e9d..0ad85ec54 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "tag.h"
 #include "commit.h"
 #include "tree.h"
diff --git a/submodule-config.c b/submodule-config.c
index 4f58491dd..d8f8d5ea3 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
 #include "strbuf.h"
diff --git a/submodule.c b/submodule.c
index bf5a93d16..95328de61 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
 #include "dir.h"
diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 8e3ed6a76..1a7b8bd3d 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 
 /*
diff --git a/t/helper/test-submodule-config.c b/t/helper/test-submodule-config.c
index 2f144d539..c6c57bba0 100644
--- a/t/helper/test-submodule-config.c
+++ b/t/helper/test-submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
 
diff --git a/trailer.c b/trailer.c
index 11f0b9fb4..751b56c00 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
diff --git a/transport.c b/transport.c
index 9bfcf870f..b9995306f 100644
--- a/transport.c
+++ b/transport.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "transport.h"
 #include "run-command.h"
 #include "pkt-line.h"
diff --git a/unpack-trees.c b/unpack-trees.c
index d38c37e38..dd535bc84 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1,5 +1,6 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "config.h"
 #include "dir.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/upload-pack.c b/upload-pack.c
index 5330c02c1..7efff2fbf 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "refs.h"
 #include "pkt-line.h"
 #include "sideband.h"
diff --git a/userdiff.c b/userdiff.c
index 8b732e40b..2c1502f71 100644
--- a/userdiff.c
+++ b/userdiff.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "userdiff.h"
 #include "attr.h"
 
diff --git a/versioncmp.c b/versioncmp.c
index 9f81dc106..069ee94a4 100644
--- a/versioncmp.c
+++ b/versioncmp.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "string-list.h"
 
 /*
diff --git a/wrapper.c b/wrapper.c
index d83741770..487a9f753 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -2,6 +2,7 @@
  * Various trivial helper wrappers around standard functions
  */
 #include "cache.h"
+#include "config.h"
 
 static void do_nothing(size_t size)
 {
diff --git a/xdiff-interface.c b/xdiff-interface.c
index 060038c2d..5ac07d734 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "config.h"
 #include "xdiff-interface.h"
 #include "xdiff/xtypes.h"
 #include "xdiff/xdiffi.h"
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 04/32] config: don't implicitly use gitdir
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (2 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 03/32] config: don't include config.h by default Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-12 19:57     ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 05/32] setup: don't perform lazy initialization of repository state Brandon Williams
                     ` (29 subsequent siblings)
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Commit 2185fde56 (config: handle conditional include when $GIT_DIR is
not set up) added a 'git_dir' field to the config_options struct.  Let's
use this option field explicitly all the time instead of occasionally
falling back to calling 'git_pathdup("config")' to get the path to the
local repository configuration.  This allows 'do_git_config_sequence()'
to not implicitly rely on global repository state.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/config.c | 2 ++
 config.c         | 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/builtin/config.c b/builtin/config.c
index 753c40a5c..90f49a6ee 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -539,6 +539,8 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		config_options.respect_includes = !given_config_source.file;
 	else
 		config_options.respect_includes = respect_includes_opt;
+	if (have_git_dir())
+		config_options.git_dir = get_git_common_dir();
 
 	if (end_null) {
 		term = '\0';
diff --git a/config.c b/config.c
index 2390f98e3..ff09b27b8 100644
--- a/config.c
+++ b/config.c
@@ -1548,8 +1548,6 @@ static int do_git_config_sequence(const struct config_options *opts,
 
 	if (opts->git_dir)
 		repo_config = mkpathdup("%s/config", opts->git_dir);
-	else if (have_git_dir())
-		repo_config = git_pathdup("config");
 	else
 		repo_config = NULL;
 
@@ -1613,6 +1611,8 @@ static void git_config_raw(config_fn_t fn, void *data)
 	struct config_options opts = {0};
 
 	opts.respect_includes = 1;
+	if (have_git_dir())
+		opts.git_dir = get_git_common_dir();
 	if (git_config_with_options(fn, data, NULL, &opts) < 0)
 		/*
 		 * git_config_with_options() normally returns only
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 05/32] setup: don't perform lazy initialization of repository state
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (3 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 04/32] config: don't implicitly use gitdir Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 06/32] environment: remove namespace_len variable Brandon Williams
                     ` (28 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Under some circumstances (bogus GIT_DIR value or the discovered gitdir
is '.git') 'setup_git_directory()' won't initialize key repository
state.  This leads to inconsistent state after running the setup code.
To account for this inconsistent state, lazy initialization is done once
a caller asks for the repository's gitdir or some other piece of
repository state.  This is confusing and can be error prone.

Instead let's tighten the expected outcome of 'setup_git_directory()'
and ensure that it initializes repository state in all cases that would
have been handled by lazy initialization.

This also lets us drop the requirement to have 'have_git_dir()' check if
the environment variable GIT_DIR was set as that will be handled by the
end of the setup code.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  2 ++
 environment.c | 17 ++++++++---------
 setup.c       | 14 ++++++++++++++
 3 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/cache.h b/cache.h
index 272f8e021..d41aab82f 100644
--- a/cache.h
+++ b/cache.h
@@ -462,6 +462,8 @@ static inline enum object_type object_type(unsigned int mode)
  */
 extern const char * const local_repo_env[];
 
+extern void setup_git_env(void);
+
 /*
  * Returns true iff we have a configured git repository (either via
  * setup_git_directory, or in the environment via $GIT_DIR).
diff --git a/environment.c b/environment.c
index d40b21fb7..a73b08f5d 100644
--- a/environment.c
+++ b/environment.c
@@ -160,7 +160,7 @@ static char *git_path_from_env(const char *envvar, const char *git_dir,
 	return xstrdup(value);
 }
 
-static void setup_git_env(void)
+void setup_git_env(void)
 {
 	struct strbuf sb = STRBUF_INIT;
 	const char *gitfile;
@@ -205,28 +205,27 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir
-		|| getenv(GIT_DIR_ENVIRONMENT);
+		|| git_dir;
 }
 
 const char *get_git_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_dir;
 }
 
 const char *get_git_common_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_common_dir;
 }
 
 const char *get_git_namespace(void)
 {
 	if (!namespace)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return namespace;
 }
 
@@ -276,7 +275,7 @@ const char *get_git_work_tree(void)
 char *get_object_directory(void)
 {
 	if (!git_object_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_object_dir;
 }
 
@@ -316,14 +315,14 @@ int odb_pack_keep(const char *name)
 char *get_index_file(void)
 {
 	if (!git_index_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_index_file;
 }
 
 char *get_graft_file(void)
 {
 	if (!git_graft_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_graft_file;
 }
 
diff --git a/setup.c b/setup.c
index e99a82cbe..b2e05145c 100644
--- a/setup.c
+++ b/setup.c
@@ -1063,6 +1063,20 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	startup_info->have_repository = !nongit_ok || !*nongit_ok;
 	startup_info->prefix = prefix;
 
+	/*
+	 * Not all paths through the setup code will call 'set_git_dir()' (which
+	 * directly sets up the environment) so in order to guarantee that the
+	 * environment is in a consistent state after setup, explicitly setup
+	 * the environment if we have a repository.
+	 *
+	 * NEEDSWORK: currently we allow bogus GIT_DIR values to be set in some
+	 * code paths so we also need to explicitly setup the environment if
+	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
+	 * GIT_DIR values at some point in the future.
+	 */
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
+		setup_git_env();
+
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
 
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 06/32] environment: remove namespace_len variable
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (4 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 05/32] setup: don't perform lazy initialization of repository state Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 07/32] repository: introduce the repository object Brandon Williams
                     ` (27 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Use 'skip_prefix' instead of 'starts_with' so that we can drop the need
to keep around 'namespace_len'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index a73b08f5d..e035f6372 100644
--- a/environment.c
+++ b/environment.c
@@ -98,7 +98,6 @@ char *git_work_tree_cfg;
 static char *work_tree;
 
 static const char *namespace;
-static size_t namespace_len;
 
 static const char *super_prefix;
 
@@ -190,7 +189,6 @@ void setup_git_env(void)
 	git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
 							  : "refs/replace/");
 	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
-	namespace_len = strlen(namespace);
 	shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
 	if (shallow_file)
 		set_alternate_shallow_file(shallow_file, 0);
@@ -231,9 +229,10 @@ const char *get_git_namespace(void)
 
 const char *strip_namespace(const char *namespaced_ref)
 {
-	if (!starts_with(namespaced_ref, get_git_namespace()))
-		return NULL;
-	return namespaced_ref + namespace_len;
+	const char *out;
+	if (skip_prefix(namespaced_ref, get_git_namespace(), &out))
+		return out;
+	return NULL;
 }
 
 const char *get_super_prefix(void)
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 07/32] repository: introduce the repository object
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (5 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 06/32] environment: remove namespace_len variable Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
                       ` (2 more replies)
  2017-06-08 23:40   ` [PATCH v2 08/32] environment: place key repository state in the_repository Brandon Williams
                     ` (26 subsequent siblings)
  33 siblings, 3 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce the repository object 'struct repository' which can be used to
hold all state pertaining to a git repository.

Some of the benefits of object-ifying a repository are:

  1. Make the code base more readable and easier to reason about.

  2. Allow for working on multiple repositories, specifically
     submodules, within the same process.  Currently the process for
     working on a submodule involves setting up an argv_array of options
     for a particular command and then launching a child process to
     execute the command in the context of the submodule.  This is
     clunky and can require lots of little hacks in order to ensure
     correctness.  Ideally it would be nice to simply pass a repository
     and an options struct to a command.

  3. Eliminating reliance on global state will make it easier to
     enable the use of threading to improve performance.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 Makefile      |   1 +
 cache.h       |   1 +
 environment.c |   2 +-
 repository.c  | 171 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 repository.h  |  25 +++++++++
 5 files changed, 199 insertions(+), 1 deletion(-)
 create mode 100644 repository.c
 create mode 100644 repository.h

diff --git a/Makefile b/Makefile
index 2ed6db728..ac82d93f6 100644
--- a/Makefile
+++ b/Makefile
@@ -821,6 +821,7 @@ LIB_OBJS += refs/ref-cache.o
 LIB_OBJS += ref-filter.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace_object.o
+LIB_OBJS += repository.o
 LIB_OBJS += rerere.o
 LIB_OBJS += resolve-undo.o
 LIB_OBJS += revision.o
diff --git a/cache.h b/cache.h
index d41aab82f..c0b4c8d83 100644
--- a/cache.h
+++ b/cache.h
@@ -483,6 +483,7 @@ extern char *get_graft_file(void);
 extern int set_git_dir(const char *path);
 extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);
 extern int get_common_dir(struct strbuf *sb, const char *gitdir);
+extern char *expand_namespace(const char *raw_namespace);
 extern const char *get_git_namespace(void);
 extern const char *strip_namespace(const char *namespaced_ref);
 extern const char *get_super_prefix(void);
diff --git a/environment.c b/environment.c
index e035f6372..77900b31a 100644
--- a/environment.c
+++ b/environment.c
@@ -127,7 +127,7 @@ const char * const local_repo_env[] = {
 	NULL
 };
 
-static char *expand_namespace(const char *raw_namespace)
+char *expand_namespace(const char *raw_namespace)
 {
 	struct strbuf buf = STRBUF_INIT;
 	struct strbuf **components, **c;
diff --git a/repository.c b/repository.c
new file mode 100644
index 000000000..e952238d1
--- /dev/null
+++ b/repository.c
@@ -0,0 +1,171 @@
+#include "cache.h"
+#include "repository.h"
+
+/* The main repository */
+static struct repository the_repo;
+struct repository *the_repository = &the_repo;
+
+static char *git_path_from_env(const char *envvar, const char *git_dir,
+			       const char *path, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(envvar);
+		if (value)
+			return xstrdup(value);
+	}
+
+	return xstrfmt("%s/%s", git_dir, path);
+}
+
+static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
+		if (value) {
+			strbuf_addstr(sb, value);
+			return 1;
+		}
+	}
+
+	return get_common_dir_noenv(sb, gitdir);
+}
+
+/* called after setting gitdir */
+static void repo_setup_env(struct repository *repo)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	if (!repo->gitdir)
+		BUG("gitdir wasn't set before setting up the environment");
+
+	repo->different_commondir = find_common_dir(&sb, repo->gitdir,
+						    !repo->ignore_env);
+	repo->commondir = strbuf_detach(&sb, NULL);
+	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+					    "objects", !repo->ignore_env);
+	repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
+					     "index", !repo->ignore_env);
+	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
+					     "info/grafts", !repo->ignore_env);
+	repo->namespace = expand_namespace(repo->ignore_env ? NULL :
+					   getenv(GIT_NAMESPACE_ENVIRONMENT));
+}
+
+void repo_set_gitdir(struct repository *repo, const char *path)
+{
+	const char *gitfile = read_gitfile(path);
+
+	/*
+	 * NEEDSWORK: Eventually we want to be able to free gitdir and the rest
+	 * of the environment before reinitializing it again, but we have some
+	 * crazy code paths where we try to set gitdir with the current gitdir
+	 * and we don't want to free gitdir before copying the passed in value.
+	 */
+	repo->gitdir = xstrdup(gitfile ? gitfile : path);
+
+	repo_setup_env(repo);
+}
+
+/*
+ * Attempt to resolve and set the provided 'gitdir' for repository 'repo'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+static int repo_init_gitdir(struct repository *repo, const char *gitdir)
+{
+	int ret = 0;
+	int error = 0;
+	char *abspath = NULL;
+	char *suspect = NULL;
+	const char *resolved_gitdir;
+
+	abspath = real_pathdup(gitdir, 0);
+	if (!abspath) {
+		ret = -1;
+		goto out;
+	}
+
+	/* First assume 'gitdir' references the gitdir directly */
+	resolved_gitdir = resolve_gitdir_gently(abspath, &error);
+	/* otherwise; try 'gitdir'.git */
+	if (!resolved_gitdir) {
+		suspect = xstrfmt("%s/.git", abspath);
+		resolved_gitdir = resolve_gitdir_gently(suspect, &error);
+		if (!resolved_gitdir) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+	repo_set_gitdir(repo, resolved_gitdir);
+
+out:
+	free(abspath);
+	free(suspect);
+	return ret;
+}
+
+static int verify_repo_format(struct repository_format *format,
+			      const char *commondir)
+{
+	int ret = 0;
+	struct strbuf sb = STRBUF_INIT;
+
+	strbuf_addf(&sb, "%s/config", commondir);
+	read_repository_format(format, sb.buf);
+	strbuf_reset(&sb);
+
+	if (verify_repository_format(format, &sb) < 0) {
+		warning("%s", sb.buf);
+		ret = -1;
+	}
+
+	strbuf_release(&sb);
+	return ret;
+}
+
+/*
+ * Initialize 'repo' based on the provided 'gitdir'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+int repo_init(struct repository *repo, const char *gitdir)
+{
+	struct repository_format format;
+	memset(repo, 0, sizeof(*repo));
+
+	repo->ignore_env = 1;
+
+	if (repo_init_gitdir(repo, gitdir))
+		goto error;
+
+	if (verify_repo_format(&format, repo->commondir))
+		goto error;
+
+	return 0;
+
+error:
+	repo_clear(repo);
+	return -1;
+}
+
+static void repo_clear_env(struct repository *repo)
+{
+	free(repo->gitdir);
+	repo->gitdir = NULL;
+	free(repo->commondir);
+	repo->commondir = NULL;
+	free(repo->objectdir);
+	repo->objectdir = NULL;
+	free(repo->index_file);
+	repo->index_file = NULL;
+	free(repo->graft_file);
+	repo->graft_file = NULL;
+	free(repo->namespace);
+	repo->namespace = NULL;
+}
+
+void repo_clear(struct repository *repo)
+{
+	repo_clear_env(repo);
+
+	memset(repo, 0, sizeof(*repo));
+}
diff --git a/repository.h b/repository.h
new file mode 100644
index 000000000..174ab0f2d
--- /dev/null
+++ b/repository.h
@@ -0,0 +1,25 @@
+#ifndef REPOSITORY_H
+#define REPOSITORY_H
+
+struct repository {
+	/* Environment */
+	char *gitdir;
+	char *commondir;
+	char *objectdir;
+	char *index_file;
+	char *graft_file;
+	char *namespace;
+
+	/* Configurations */
+	unsigned ignore_env:1;
+	/* Indicates if a repository has a different 'commondir' from 'gitdir' */
+	unsigned different_commondir:1;
+};
+
+extern struct repository *the_repository;
+
+extern void repo_set_gitdir(struct repository *repo, const char *path);
+extern int repo_init(struct repository *repo, const char *path);
+extern void repo_clear(struct repository *repo);
+
+#endif /* REPOSITORY_H */
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 08/32] environment: place key repository state in the_repository
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (6 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 07/32] repository: introduce the repository object Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 09/32] environment: store worktree " Brandon Williams
                     ` (25 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'git_dir', 'git_common_dir', 'git_object_dir', 'git_index_file',
'git_graft_file', and 'namespace' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  1 -
 environment.c | 65 ++++++++++++++---------------------------------------------
 path.c        | 11 +++++-----
 setup.c       | 17 ++++++++++++++--
 4 files changed, 36 insertions(+), 58 deletions(-)

diff --git a/cache.h b/cache.h
index c0b4c8d83..02ab5f801 100644
--- a/cache.h
+++ b/cache.h
@@ -769,7 +769,6 @@ extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
 extern int protect_hfs;
 extern int protect_ntfs;
-extern int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
 
 /*
  * Include broken refs in all ref iterations, which will
diff --git a/environment.c b/environment.c
index 77900b31a..fb52c0daf 100644
--- a/environment.c
+++ b/environment.c
@@ -8,6 +8,7 @@
  * are.
  */
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "refs.h"
 #include "fmt-merge-msg.h"
@@ -97,14 +98,8 @@ int ignore_untracked_cache_config;
 char *git_work_tree_cfg;
 static char *work_tree;
 
-static const char *namespace;
-
 static const char *super_prefix;
 
-static const char *git_dir, *git_common_dir;
-static char *git_object_dir, *git_index_file, *git_graft_file;
-int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
-
 /*
  * Repository-local GIT_* environment variables; see cache.h for details.
  */
@@ -148,47 +143,16 @@ char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-static char *git_path_from_env(const char *envvar, const char *git_dir,
-			       const char *path, int *fromenv)
-{
-	const char *value = getenv(envvar);
-	if (!value)
-		return xstrfmt("%s/%s", git_dir, path);
-	if (fromenv)
-		*fromenv = 1;
-	return xstrdup(value);
-}
-
 void setup_git_env(void)
 {
-	struct strbuf sb = STRBUF_INIT;
-	const char *gitfile;
 	const char *shallow_file;
 	const char *replace_ref_base;
 
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
-	if (!git_dir) {
-		if (!startup_info->have_repository)
-			BUG("setup_git_env called without repository");
-		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
-	}
-	gitfile = read_gitfile(git_dir);
-	git_dir = xstrdup(gitfile ? gitfile : git_dir);
-	if (get_common_dir(&sb, git_dir))
-		git_common_dir_env = 1;
-	git_common_dir = strbuf_detach(&sb, NULL);
-	git_object_dir = git_path_from_env(DB_ENVIRONMENT, git_common_dir,
-					   "objects", &git_db_env);
-	git_index_file = git_path_from_env(INDEX_ENVIRONMENT, git_dir,
-					   "index", &git_index_env);
-	git_graft_file = git_path_from_env(GRAFT_ENVIRONMENT, git_common_dir,
-					   "info/grafts", &git_graft_env);
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		check_replace_refs = 0;
 	replace_ref_base = getenv(GIT_REPLACE_REF_BASE_ENVIRONMENT);
 	git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
 							  : "refs/replace/");
-	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
 	shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
 	if (shallow_file)
 		set_alternate_shallow_file(shallow_file, 0);
@@ -203,28 +167,28 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir;
+		|| the_repository->gitdir;
 }
 
 const char *get_git_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository->gitdir)
 		BUG("git environment hasn't been setup");
-	return git_dir;
+	return the_repository->gitdir;
 }
 
 const char *get_git_common_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository->commondir)
 		BUG("git environment hasn't been setup");
-	return git_common_dir;
+	return the_repository->commondir;
 }
 
 const char *get_git_namespace(void)
 {
-	if (!namespace)
+	if (!the_repository->namespace)
 		BUG("git environment hasn't been setup");
-	return namespace;
+	return the_repository->namespace;
 }
 
 const char *strip_namespace(const char *namespaced_ref)
@@ -273,9 +237,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!git_object_dir)
+	if (!the_repository->objectdir)
 		BUG("git environment hasn't been setup");
-	return git_object_dir;
+	return the_repository->objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
@@ -313,22 +277,23 @@ int odb_pack_keep(const char *name)
 
 char *get_index_file(void)
 {
-	if (!git_index_file)
+	if (!the_repository->index_file)
 		BUG("git environment hasn't been setup");
-	return git_index_file;
+	return the_repository->index_file;
 }
 
 char *get_graft_file(void)
 {
-	if (!git_graft_file)
+	if (!the_repository->graft_file)
 		BUG("git environment hasn't been setup");
-	return git_graft_file;
+	return the_repository->graft_file;
 }
 
 int set_git_dir(const char *path)
 {
 	if (setenv(GIT_DIR_ENVIRONMENT, path, 1))
 		return error("Could not set GIT_DIR to '%s'", path);
+	repo_set_gitdir(the_repository, path);
 	setup_git_env();
 	return 0;
 }
diff --git a/path.c b/path.c
index c1cb1cf62..e4abea083 100644
--- a/path.c
+++ b/path.c
@@ -2,6 +2,7 @@
  * Utilities for paths and pathnames
  */
 #include "cache.h"
+#include "repository.h"
 #include "strbuf.h"
 #include "string-list.h"
 #include "dir.h"
@@ -355,7 +356,7 @@ void report_linked_checkout_garbage(void)
 	const struct common_dir *p;
 	int len;
 
-	if (!git_common_dir_env)
+	if (!the_repository->different_commondir)
 		return;
 	strbuf_addf(&sb, "%s/", get_git_dir());
 	len = sb.len;
@@ -374,17 +375,17 @@ void report_linked_checkout_garbage(void)
 static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
-	if (git_graft_env && is_dir_file(base, "info", "grafts"))
+	if (is_dir_file(base, "info", "grafts"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_graft_file(), strlen(get_graft_file()));
-	else if (git_index_env && !strcmp(base, "index"))
+	else if (!strcmp(base, "index"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_index_file(), strlen(get_index_file()));
-	else if (git_db_env && dir_prefix(base, "objects"))
+	else if (dir_prefix(base, "objects"))
 		replace_dir(buf, git_dir_len + 7, get_object_directory());
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
-	else if (git_common_dir_env)
+	else if (the_repository->different_commondir)
 		update_common_dir(buf, git_dir_len, NULL);
 }
 
diff --git a/setup.c b/setup.c
index b2e05145c..884a74a54 100644
--- a/setup.c
+++ b/setup.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "dir.h"
 #include "string-list.h"
@@ -376,6 +377,11 @@ void setup_work_tree(void)
 	if (getenv(GIT_WORK_TREE_ENVIRONMENT))
 		setenv(GIT_WORK_TREE_ENVIRONMENT, ".", 1);
 
+	/*
+	 * NEEDSWORK: this call can essentially be set_git_dir(get_git_dir())
+	 * which can cause some problems when trying to free the old value of
+	 * gitdir.
+	 */
 	set_git_dir(remove_leading_path(git_dir, work_tree));
 	initialized = 1;
 }
@@ -1074,8 +1080,15 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
 	 * GIT_DIR values at some point in the future.
 	 */
-	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
-		setup_git_env();
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT)) {
+		if (!the_repository->gitdir) {
+			const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);
+			if (!gitdir)
+				gitdir = DEFAULT_GIT_DIR_ENVIRONMENT;
+			repo_set_gitdir(the_repository, gitdir);
+			setup_git_env();
+		}
+	}
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 09/32] environment: store worktree in the_repository
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (7 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 08/32] environment: place key repository state in the_repository Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 10/32] setup: add comment indicating a hack Brandon Williams
                     ` (24 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'work_tree' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c |  9 ++++-----
 repository.c  | 12 ++++++++++++
 repository.h  |  3 +++
 3 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index fb52c0daf..e390a6627 100644
--- a/environment.c
+++ b/environment.c
@@ -96,7 +96,6 @@ int ignore_untracked_cache_config;
 
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
 char *git_work_tree_cfg;
-static char *work_tree;
 
 static const char *super_prefix;
 
@@ -220,19 +219,19 @@ void set_git_work_tree(const char *new_work_tree)
 {
 	if (git_work_tree_initialized) {
 		new_work_tree = real_path(new_work_tree);
-		if (strcmp(new_work_tree, work_tree))
+		if (strcmp(new_work_tree, the_repository->worktree))
 			die("internal error: work tree has already been set\n"
 			    "Current worktree: %s\nNew worktree: %s",
-			    work_tree, new_work_tree);
+			    the_repository->worktree, new_work_tree);
 		return;
 	}
 	git_work_tree_initialized = 1;
-	work_tree = real_pathdup(new_work_tree, 1);
+	repo_set_worktree(the_repository, new_work_tree);
 }
 
 const char *get_git_work_tree(void)
 {
-	return work_tree;
+	return the_repository->worktree;
 }
 
 char *get_object_directory(void)
diff --git a/repository.c b/repository.c
index e952238d1..1b48cc816 100644
--- a/repository.c
+++ b/repository.c
@@ -104,6 +104,16 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)
 	return ret;
 }
 
+void repo_set_worktree(struct repository *repo, const char *path)
+{
+	repo->worktree = real_pathdup(path, 1);
+}
+
+char *repo_worktree_path(struct repository *repo, const char *path)
+{
+	return xstrfmt("%s/%s", repo->worktree, path);
+}
+
 static int verify_repo_format(struct repository_format *format,
 			      const char *commondir)
 {
@@ -166,6 +176,8 @@ static void repo_clear_env(struct repository *repo)
 void repo_clear(struct repository *repo)
 {
 	repo_clear_env(repo);
+	free(repo->worktree);
+	repo->worktree = NULL;
 
 	memset(repo, 0, sizeof(*repo));
 }
diff --git a/repository.h b/repository.h
index 174ab0f2d..a1163ae91 100644
--- a/repository.h
+++ b/repository.h
@@ -9,6 +9,7 @@ struct repository {
 	char *index_file;
 	char *graft_file;
 	char *namespace;
+	char *worktree;
 
 	/* Configurations */
 	unsigned ignore_env:1;
@@ -19,6 +20,8 @@ struct repository {
 extern struct repository *the_repository;
 
 extern void repo_set_gitdir(struct repository *repo, const char *path);
+extern void repo_set_worktree(struct repository *repo, const char *path);
+extern char *repo_worktree_path(struct repository *repo, const char *path);
 extern int repo_init(struct repository *repo, const char *path);
 extern void repo_clear(struct repository *repo);
 
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 10/32] setup: add comment indicating a hack
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (8 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 09/32] environment: store worktree " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 11/32] config: read config from a repository object Brandon Williams
                     ` (23 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

'GIT_TOPLEVEL_PREFIX_ENVIRONMENT' was added in (b58a68c1c setup: allow
for prefix to be passed to git commands) to aid in fixing a bug where
'ls-files' and 'grep' were not able to properly recurse when called from
within a subdirectory.  Add a 'NEEDSWORK' comment indicating that this
envvar should be removed once 'ls-files' and 'grep' can recurse
in-process.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 setup.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/setup.c b/setup.c
index 884a74a54..aca1cf5d3 100644
--- a/setup.c
+++ b/setup.c
@@ -1057,6 +1057,12 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die("BUG: unhandled setup_git_directory_1() result");
 	}
 
+	/*
+	 * NEEDSWORK: This was a hack in order to get ls-files and grep to have
+	 * properly formated output when recursing submodules.  Once ls-files
+	 * and grep have been changed to perform this recursing in-process this
+	 * needs to be removed.
+	 */
 	env_prefix = getenv(GIT_TOPLEVEL_PREFIX_ENVIRONMENT);
 	if (env_prefix)
 		prefix = env_prefix;
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 11/32] config: read config from a repository object
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (9 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 10/32] setup: add comment indicating a hack Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 12/32] repository: add index_state to struct repo Brandon Williams
                     ` (22 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Teach the config machinery to read config information from a repository
object.  This involves storing a 'struct config_set' inside the
repository object and adding a number of functions (repo_config*) to be
able to query a repository's config.

The current config API enables lazy-loading of the config.  This means
that when 'git_config_get_int()' is called, if the_config_set hasn't
been populated yet, then it will be populated and properly initialized by
reading the necessary config files (system wide .gitconfig, user's home
.gitconfig, and the repository's config).  To maintain this paradigm,
the new API to read from a repository object's config will also perform
this lazy-initialization.

Since both APIs (git_config_get* and repo_config_get*) have the same
semantics we can migrate the default config to be stored within
'the_repository' and just have the 'git_config_get*' family of functions
redirect to the 'repo_config_get*' functions.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 config.c     | 209 +++++++++++++++++++++++++++++++++++++++--------------------
 config.h     |  24 +++++++
 repository.c |   7 ++
 repository.h |  10 +++
 4 files changed, 179 insertions(+), 71 deletions(-)

diff --git a/config.c b/config.c
index ff09b27b8..3bfe2766f 100644
--- a/config.c
+++ b/config.c
@@ -7,6 +7,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "lockfile.h"
 #include "exec_cmd.h"
 #include "strbuf.h"
@@ -72,13 +73,6 @@ static int core_compression_seen;
 static int pack_compression_seen;
 static int zlib_compression_seen;
 
-/*
- * Default config_set that contains key-value pairs from the usual set of config
- * config files (i.e repo specific .git/config, user wide ~/.gitconfig, XDG
- * config file and the global /etc/gitconfig)
- */
-static struct config_set the_config_set;
-
 static int config_file_fgetc(struct config_source *conf)
 {
 	return getc_unlocked(conf->u.file);
@@ -1606,28 +1600,6 @@ int git_config_with_options(config_fn_t fn, void *data,
 	return do_git_config_sequence(opts, fn, data);
 }
 
-static void git_config_raw(config_fn_t fn, void *data)
-{
-	struct config_options opts = {0};
-
-	opts.respect_includes = 1;
-	if (have_git_dir())
-		opts.git_dir = get_git_common_dir();
-	if (git_config_with_options(fn, data, NULL, &opts) < 0)
-		/*
-		 * git_config_with_options() normally returns only
-		 * zero, as most errors are fatal, and
-		 * non-fatal potential errors are guarded by "if"
-		 * statements that are entered only when no error is
-		 * possible.
-		 *
-		 * If we ever encounter a non-fatal error, it means
-		 * something went really wrong and we should stop
-		 * immediately.
-		 */
-		die(_("unknown error occurred while reading the configuration files"));
-}
-
 static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 {
 	int i, value_index;
@@ -1676,14 +1648,6 @@ void read_early_config(config_fn_t cb, void *data)
 	strbuf_release(&buf);
 }
 
-static void git_config_check_init(void);
-
-void git_config(config_fn_t fn, void *data)
-{
-	git_config_check_init();
-	configset_iter(&the_config_set, fn, data);
-}
-
 static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
 {
 	struct config_set_element k;
@@ -1893,87 +1857,190 @@ int git_configset_get_pathname(struct config_set *cs, const char *key, const cha
 		return 1;
 }
 
-static void git_config_check_init(void)
+/* Functions use to read configuration from a repository */
+static void repo_read_config(struct repository *repo)
 {
-	if (the_config_set.hash_initialized)
+	struct config_options opts = { 1, repo->commondir };
+
+	if (!repo->config)
+		repo->config = xcalloc(1, sizeof(struct config_set));
+	else
+		git_configset_clear(repo->config);
+
+	git_configset_init(repo->config);
+
+	if (git_config_with_options(config_set_callback, repo->config, NULL, &opts) < 0)
+		/*
+		 * git_config_with_options() normally returns only
+		 * zero, as most errors are fatal, and
+		 * non-fatal potential errors are guarded by "if"
+		 * statements that are entered only when no error is
+		 * possible.
+		 *
+		 * If we ever encounter a non-fatal error, it means
+		 * something went really wrong and we should stop
+		 * immediately.
+		 */
+		die(_("unknown error occurred while reading the configuration files"));
+}
+
+static void git_config_check_init(struct repository *repo)
+{
+	if (repo->config && repo->config->hash_initialized)
 		return;
-	git_configset_init(&the_config_set);
-	git_config_raw(config_set_callback, &the_config_set);
+	repo_read_config(repo);
 }
 
-void git_config_clear(void)
+static void repo_config_clear(struct repository *repo)
 {
-	if (!the_config_set.hash_initialized)
+	if (!repo->config || !repo->config->hash_initialized)
 		return;
-	git_configset_clear(&the_config_set);
+	git_configset_clear(repo->config);
 }
 
-int git_config_get_value(const char *key, const char **value)
+void repo_config(struct repository *repo, config_fn_t fn, void *data)
 {
-	git_config_check_init();
-	return git_configset_get_value(&the_config_set, key, value);
+	git_config_check_init(repo);
+	configset_iter(repo->config, fn, data);
 }
 
-const struct string_list *git_config_get_value_multi(const char *key)
+int repo_config_get_value(struct repository *repo,
+			  const char *key, const char **value)
 {
-	git_config_check_init();
-	return git_configset_get_value_multi(&the_config_set, key);
+	git_config_check_init(repo);
+	return git_configset_get_value(repo->config, key, value);
 }
 
-int git_config_get_string_const(const char *key, const char **dest)
+const struct string_list *repo_config_get_value_multi(struct repository *repo,
+						      const char *key)
+{
+	git_config_check_init(repo);
+	return git_configset_get_value_multi(repo->config, key);
+}
+
+int repo_config_get_string_const(struct repository *repo,
+				 const char *key, const char **dest)
 {
 	int ret;
-	git_config_check_init();
-	ret = git_configset_get_string_const(&the_config_set, key, dest);
+	git_config_check_init(repo);
+	ret = git_configset_get_string_const(repo->config, key, dest);
 	if (ret < 0)
 		git_die_config(key, NULL);
 	return ret;
 }
 
+int repo_config_get_string(struct repository *repo,
+			   const char *key, char **dest)
+{
+	git_config_check_init(repo);
+	return repo_config_get_string_const(repo, key, (const char **)dest);
+}
+
+int repo_config_get_int(struct repository *repo,
+			const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_int(repo->config, key, dest);
+}
+
+int repo_config_get_ulong(struct repository *repo,
+			  const char *key, unsigned long *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_ulong(repo->config, key, dest);
+}
+
+int repo_config_get_bool(struct repository *repo,
+			 const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_bool(repo->config, key, dest);
+}
+
+int repo_config_get_bool_or_int(struct repository *repo,
+				const char *key, int *is_bool, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_bool_or_int(repo->config, key, is_bool, dest);
+}
+
+int repo_config_get_maybe_bool(struct repository *repo,
+			       const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_maybe_bool(repo->config, key, dest);
+}
+
+int repo_config_get_pathname(struct repository *repo,
+			     const char *key, const char **dest)
+{
+	int ret;
+	git_config_check_init(repo);
+	ret = git_configset_get_pathname(repo->config, key, dest);
+	if (ret < 0)
+		git_die_config(key, NULL);
+	return ret;
+}
+
+/* Functions used historically to read configuration from 'the_repository' */
+void git_config(config_fn_t fn, void *data)
+{
+	repo_config(the_repository, fn, data);
+}
+
+void git_config_clear(void)
+{
+	repo_config_clear(the_repository);
+}
+
+int git_config_get_value(const char *key, const char **value)
+{
+	return repo_config_get_value(the_repository, key, value);
+}
+
+const struct string_list *git_config_get_value_multi(const char *key)
+{
+	return repo_config_get_value_multi(the_repository, key);
+}
+
+int git_config_get_string_const(const char *key, const char **dest)
+{
+	return repo_config_get_string_const(the_repository, key, dest);
+}
+
 int git_config_get_string(const char *key, char **dest)
 {
-	git_config_check_init();
-	return git_config_get_string_const(key, (const char **)dest);
+	return repo_config_get_string(the_repository, key, dest);
 }
 
 int git_config_get_int(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_int(&the_config_set, key, dest);
+	return repo_config_get_int(the_repository, key, dest);
 }
 
 int git_config_get_ulong(const char *key, unsigned long *dest)
 {
-	git_config_check_init();
-	return git_configset_get_ulong(&the_config_set, key, dest);
+	return repo_config_get_ulong(the_repository, key, dest);
 }
 
 int git_config_get_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool(&the_config_set, key, dest);
+	return repo_config_get_bool(the_repository, key, dest);
 }
 
 int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool_or_int(&the_config_set, key, is_bool, dest);
+	return repo_config_get_bool_or_int(the_repository, key, is_bool, dest);
 }
 
 int git_config_get_maybe_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_maybe_bool(&the_config_set, key, dest);
+	return repo_config_get_maybe_bool(the_repository, key, dest);
 }
 
 int git_config_get_pathname(const char *key, const char **dest)
 {
-	int ret;
-	git_config_check_init();
-	ret = git_configset_get_pathname(&the_config_set, key, dest);
-	if (ret < 0)
-		git_die_config(key, NULL);
-	return ret;
+	return repo_config_get_pathname(the_repository, key, dest);
 }
 
 int git_config_get_expiry(const char *key, const char **output)
diff --git a/config.h b/config.h
index c70599bd5..a4db8d0f4 100644
--- a/config.h
+++ b/config.h
@@ -162,6 +162,30 @@ extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key,
 extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
 extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
 
+/* Functions for reading a repository's config */
+struct repository;
+extern void repo_config(struct repository *repo, config_fn_t fn, void *data);
+extern int repo_config_get_value(struct repository *repo,
+				 const char *key, const char **value);
+extern const struct string_list *repo_config_get_value_multi(struct repository *repo,
+							     const char *key);
+extern int repo_config_get_string_const(struct repository *repo,
+					const char *key, const char **dest);
+extern int repo_config_get_string(struct repository *repo,
+				  const char *key, char **dest);
+extern int repo_config_get_int(struct repository *repo,
+			       const char *key, int *dest);
+extern int repo_config_get_ulong(struct repository *repo,
+				 const char *key, unsigned long *dest);
+extern int repo_config_get_bool(struct repository *repo,
+				const char *key, int *dest);
+extern int repo_config_get_bool_or_int(struct repository *repo,
+				       const char *key, int *is_bool, int *dest);
+extern int repo_config_get_maybe_bool(struct repository *repo,
+				      const char *key, int *dest);
+extern int repo_config_get_pathname(struct repository *repo,
+				    const char *key, const char **dest);
+
 extern int git_config_get_value(const char *key, const char **value);
 extern const struct string_list *git_config_get_value_multi(const char *key);
 extern void git_config_clear(void);
diff --git a/repository.c b/repository.c
index 1b48cc816..fd53d6bf7 100644
--- a/repository.c
+++ b/repository.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "repository.h"
+#include "config.h"
 
 /* The main repository */
 static struct repository the_repo;
@@ -179,5 +180,11 @@ void repo_clear(struct repository *repo)
 	free(repo->worktree);
 	repo->worktree = NULL;
 
+	if (repo->config) {
+		git_configset_clear(repo->config);
+		free(repo->config);
+		repo->config = NULL;
+	}
+
 	memset(repo, 0, sizeof(*repo));
 }
diff --git a/repository.h b/repository.h
index a1163ae91..2b757df25 100644
--- a/repository.h
+++ b/repository.h
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+struct config_set;
+
 struct repository {
 	/* Environment */
 	char *gitdir;
@@ -11,6 +13,14 @@ struct repository {
 	char *namespace;
 	char *worktree;
 
+	/* Subsystems */
+	/*
+	 * Repository's config which contains key-value pairs from the usual
+	 * set of config files (i.e. repo specific .git/config, user wide
+	 * ~/.gitconfig, XDG config file and the global /etc/gitconfig)
+	 */
+	struct config_set *config;
+
 	/* Configurations */
 	unsigned ignore_env:1;
 	/* Indicates if a repository has a different 'commondir' from 'gitdir' */
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 12/32] repository: add index_state to struct repo
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (10 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 11/32] config: read config from a repository object Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 13/32] submodule-config: store the_submodule_cache in the_repository Brandon Williams
                     ` (21 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c | 16 ++++++++++++++++
 repository.h |  4 ++++
 2 files changed, 20 insertions(+)

diff --git a/repository.c b/repository.c
index fd53d6bf7..21719d0a6 100644
--- a/repository.c
+++ b/repository.c
@@ -186,5 +186,21 @@ void repo_clear(struct repository *repo)
 		repo->config = NULL;
 	}
 
+	if (repo->index) {
+		discard_index(repo->index);
+		free(repo->index);
+		repo->index = NULL;
+	}
+
 	memset(repo, 0, sizeof(*repo));
 }
+
+int repo_read_index(struct repository *repo)
+{
+	if (!repo->index)
+		repo->index = xcalloc(1, sizeof(struct index_state));
+	else
+		discard_index(repo->index);
+
+	return read_index_from(repo->index, repo->index_file);
+}
diff --git a/repository.h b/repository.h
index 2b757df25..9515cc631 100644
--- a/repository.h
+++ b/repository.h
@@ -2,6 +2,7 @@
 #define REPOSITORY_H
 
 struct config_set;
+struct index_state;
 
 struct repository {
 	/* Environment */
@@ -20,6 +21,7 @@ struct repository {
 	 * ~/.gitconfig, XDG config file and the global /etc/gitconfig)
 	 */
 	struct config_set *config;
+	struct index_state *index;
 
 	/* Configurations */
 	unsigned ignore_env:1;
@@ -35,4 +37,6 @@ extern char *repo_worktree_path(struct repository *repo, const char *path);
 extern int repo_init(struct repository *repo, const char *path);
 extern void repo_clear(struct repository *repo);
 
+extern int repo_read_index(struct repository *repo);
+
 #endif /* REPOSITORY_H */
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 13/32] submodule-config: store the_submodule_cache in the_repository
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (11 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 12/32] repository: add index_state to struct repo Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 14/32] submodule: add repo_read_gitmodules Brandon Williams
                     ` (20 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Refactor how 'the_submodule_cache' is handled so that it can be stored
inside of a repository object.  Also migrate 'the_submodule_cache' to be
stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c       |  6 +++++
 repository.h       |  2 ++
 submodule-config.c | 70 ++++++++++++++++++++++++++++++++++++++++--------------
 submodule-config.h | 10 ++++++++
 4 files changed, 70 insertions(+), 18 deletions(-)

diff --git a/repository.c b/repository.c
index 21719d0a6..98a851d20 100644
--- a/repository.c
+++ b/repository.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "submodule-config.h"
 
 /* The main repository */
 static struct repository the_repo;
@@ -192,6 +193,11 @@ void repo_clear(struct repository *repo)
 		repo->index = NULL;
 	}
 
+	if (repo->submodule_cache) {
+		submodule_cache_free(repo->submodule_cache);
+		repo->submodule_cache = NULL;
+	}
+
 	memset(repo, 0, sizeof(*repo));
 }
 
diff --git a/repository.h b/repository.h
index 9515cc631..d9f1f721b 100644
--- a/repository.h
+++ b/repository.h
@@ -3,6 +3,7 @@
 
 struct config_set;
 struct index_state;
+struct submodule_cache;
 
 struct repository {
 	/* Environment */
@@ -22,6 +23,7 @@ struct repository {
 	 */
 	struct config_set *config;
 	struct index_state *index;
+	struct submodule_cache *submodule_cache;
 
 	/* Configurations */
 	unsigned ignore_env:1;
diff --git a/submodule-config.c b/submodule-config.c
index d8f8d5ea3..37cfcceb9 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -15,6 +16,7 @@
 struct submodule_cache {
 	struct hashmap for_path;
 	struct hashmap for_name;
+	unsigned initialized:1;
 };
 
 /*
@@ -31,9 +33,6 @@ enum lookup_type {
 	lookup_path
 };
 
-static struct submodule_cache the_submodule_cache;
-static int is_cache_init;
-
 static int config_path_cmp(const struct submodule_entry *a,
 			   const struct submodule_entry *b,
 			   const void *unused)
@@ -50,10 +49,16 @@ static int config_name_cmp(const struct submodule_entry *a,
 	       hashcmp(a->config->gitmodules_sha1, b->config->gitmodules_sha1);
 }
 
-static void cache_init(struct submodule_cache *cache)
+static struct submodule_cache *submodule_cache_alloc(void)
+{
+	return xcalloc(1, sizeof(struct submodule_cache));
+}
+
+static void submodule_cache_init(struct submodule_cache *cache)
 {
 	hashmap_init(&cache->for_path, (hashmap_cmp_fn) config_path_cmp, 0);
 	hashmap_init(&cache->for_name, (hashmap_cmp_fn) config_name_cmp, 0);
+	cache->initialized = 1;
 }
 
 static void free_one_config(struct submodule_entry *entry)
@@ -65,11 +70,14 @@ static void free_one_config(struct submodule_entry *entry)
 	free(entry->config);
 }
 
-static void cache_free(struct submodule_cache *cache)
+static void submodule_cache_clear(struct submodule_cache *cache)
 {
 	struct hashmap_iter iter;
 	struct submodule_entry *entry;
 
+	if (!cache->initialized)
+		return;
+
 	/*
 	 * We iterate over the name hash here to be symmetric with the
 	 * allocation of struct submodule entries. Each is allocated by
@@ -81,6 +89,13 @@ static void cache_free(struct submodule_cache *cache)
 
 	hashmap_free(&cache->for_path, 1);
 	hashmap_free(&cache->for_name, 1);
+	cache->initialized = 0;
+}
+
+void submodule_cache_free(struct submodule_cache *cache)
+{
+	submodule_cache_clear(cache);
+	free(cache);
 }
 
 static unsigned int hash_sha1_string(const unsigned char *sha1,
@@ -494,43 +509,62 @@ static const struct submodule *config_from(struct submodule_cache *cache,
 	return submodule;
 }
 
-static void ensure_cache_init(void)
+static void submodule_cache_check_init(struct repository *repo)
 {
-	if (is_cache_init)
+	if (repo->submodule_cache && repo->submodule_cache->initialized)
 		return;
 
-	cache_init(&the_submodule_cache);
-	is_cache_init = 1;
+	if (!repo->submodule_cache)
+		repo->submodule_cache = submodule_cache_alloc();
+
+	submodule_cache_init(repo->submodule_cache);
 }
 
-int parse_submodule_config_option(const char *var, const char *value)
+int submodule_config_option(struct repository *repo,
+			    const char *var, const char *value)
 {
 	struct parse_config_parameter parameter;
-	parameter.cache = &the_submodule_cache;
+
+	submodule_cache_check_init(repo);
+
+	parameter.cache = repo->submodule_cache;
 	parameter.treeish_name = NULL;
 	parameter.gitmodules_sha1 = null_sha1;
 	parameter.overwrite = 1;
 
-	ensure_cache_init();
 	return parse_config(var, value, &parameter);
 }
 
+int parse_submodule_config_option(const char *var, const char *value)
+{
+	return submodule_config_option(the_repository, var, value);
+}
+
 const struct submodule *submodule_from_name(const unsigned char *treeish_name,
 		const char *name)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, name, lookup_name);
+	submodule_cache_check_init(the_repository);
+	return config_from(the_repository->submodule_cache, treeish_name, name, lookup_name);
 }
 
 const struct submodule *submodule_from_path(const unsigned char *treeish_name,
 		const char *path)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, path, lookup_path);
+	submodule_cache_check_init(the_repository);
+	return config_from(the_repository->submodule_cache, treeish_name, path, lookup_path);
+}
+
+const struct submodule *submodule_from_cache(struct repository *repo,
+					     const unsigned char *treeish_name,
+					     const char *key)
+{
+	submodule_cache_check_init(repo);
+	return config_from(repo->submodule_cache, treeish_name,
+			   key, lookup_path);
 }
 
 void submodule_free(void)
 {
-	cache_free(&the_submodule_cache);
-	is_cache_init = 0;
+	if (the_repository->submodule_cache)
+		submodule_cache_clear(the_repository->submodule_cache);
 }
diff --git a/submodule-config.h b/submodule-config.h
index d434ecdb4..bc45a25e8 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -22,14 +22,24 @@ struct submodule {
 	int recommend_shallow;
 };
 
+struct submodule_cache;
+struct repository;
+
+extern void submodule_cache_free(struct submodule_cache *cache);
+
 extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_submodule_config_option(const char *var, const char *value);
+extern int submodule_config_option(struct repository *repo,
+				   const char *var, const char *value);
 extern const struct submodule *submodule_from_name(
 		const unsigned char *commit_or_tree, const char *name);
 extern const struct submodule *submodule_from_path(
 		const unsigned char *commit_or_tree, const char *path);
+extern const struct submodule *submodule_from_cache(struct repository *repo,
+						    const unsigned char *treeish_name,
+						    const char *key);
 extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,
 				      unsigned char *gitmodules_sha1,
 				      struct strbuf *rev);
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 14/32] submodule: add repo_read_gitmodules
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (12 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 13/32] submodule-config: store the_submodule_cache in the_repository Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 15/32] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
                     ` (19 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Teach the repo object to be able to populate the submodule_cache by
reading the repository's gitmodules file.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 submodule.c | 15 +++++++++++++++
 submodule.h |  2 ++
 2 files changed, 17 insertions(+)

diff --git a/submodule.c b/submodule.c
index 95328de61..65f60ab57 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -202,6 +203,20 @@ void gitmodules_config(void)
 	}
 }
 
+static int gitmodules_cb(const char *var, const char *value, void *data)
+{
+	struct repository *repo = data;
+	return submodule_config_option(repo, var, value);
+}
+
+void repo_read_gitmodules(struct repository *repo)
+{
+	char *gitmodules_path = repo_worktree_path(repo, ".gitmodules");
+
+	git_config_from_file(gitmodules_cb, gitmodules_path, repo);
+	free(gitmodules_path);
+}
+
 void gitmodules_config_sha1(const unsigned char *commit_sha1)
 {
 	struct strbuf rev = STRBUF_INIT;
diff --git a/submodule.h b/submodule.h
index 8fb0f2549..ec6ea8dfd 100644
--- a/submodule.h
+++ b/submodule.h
@@ -1,6 +1,7 @@
 #ifndef SUBMODULE_H
 #define SUBMODULE_H
 
+struct repository;
 struct diff_options;
 struct argv_array;
 struct oid_array;
@@ -40,6 +41,7 @@ extern void set_diffopt_flags_from_submodule_config(struct diff_options *,
 		const char *path);
 extern int submodule_config(const char *var, const char *value, void *cb);
 extern void gitmodules_config(void);
+extern void repo_read_gitmodules(struct repository *repo);
 extern void gitmodules_config_sha1(const unsigned char *commit_sha1);
 extern int is_submodule_initialized(const char *path);
 /*
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 15/32] submodule: convert is_submodule_initialized to work on a repository
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (13 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 14/32] submodule: add repo_read_gitmodules Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 16/32] convert: convert get_cached_convert_stats_ascii to take an index Brandon Williams
                     ` (18 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert 'is_submodule_initialized()' to take a repository object and
while we're at it, lets rename the function to 'is_submodule_active()'
and remove the NEEDSWORK comment.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/grep.c              |  3 ++-
 builtin/submodule--helper.c |  9 +++++----
 submodule.c                 | 20 ++++++++------------
 submodule.h                 |  2 +-
 4 files changed, 16 insertions(+), 18 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index 0f4a1e5a3..c2473c281 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "blob.h"
 #include "tree.h"
@@ -626,7 +627,7 @@ static int grep_submodule_launch(struct grep_opt *opt,
 static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,
 			  const char *filename, const char *path)
 {
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(the_repository, path))
 		return 0;
 	if (!is_submodule_populated_gently(path, NULL)) {
 		/*
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 4dcbfb952..2cd6047ef 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "repository.h"
 #include "cache.h"
 #include "config.h"
 #include "parse-options.h"
@@ -280,7 +281,7 @@ static void module_list_active(struct module_list *list)
 	for (i = 0; i < list->nr; i++) {
 		const struct cache_entry *ce = list->entries[i];
 
-		if (!is_submodule_initialized(ce->name))
+		if (!is_submodule_active(the_repository, ce->name))
 			continue;
 
 		ALLOC_GROW(active_modules.entries,
@@ -362,7 +363,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_initialized(path)) {
+	if (!is_submodule_active(the_repository, path)) {
 		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
@@ -817,7 +818,7 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 	}
 
 	/* Check if the submodule has been initialized. */
-	if (!is_submodule_initialized(ce->name)) {
+	if (!is_submodule_active(the_repository, ce->name)) {
 		next_submodule_warn_missing(suc, out, displaypath);
 		goto cleanup;
 	}
@@ -1193,7 +1194,7 @@ static int is_active(int argc, const char **argv, const char *prefix)
 
 	gitmodules_config();
 
-	return !is_submodule_initialized(argv[1]);
+	return !is_submodule_active(the_repository, argv[1]);
 }
 
 #define SUPPORT_SUPER_PREFIX (1<<0)
diff --git a/submodule.c b/submodule.c
index 65f60ab57..7ec13253e 100644
--- a/submodule.c
+++ b/submodule.c
@@ -230,21 +230,17 @@ void gitmodules_config_sha1(const unsigned char *commit_sha1)
 }
 
 /*
- * NEEDSWORK: With the addition of different configuration options to determine
- * if a submodule is of interests, the validity of this function's name comes
- * into question.  Once the dust has settled and more concrete terminology is
- * decided upon, come up with a more proper name for this function.  One
- * potential candidate could be 'is_submodule_active()'.
- *
  * Determine if a submodule has been initialized at a given 'path'
  */
-int is_submodule_initialized(const char *path)
+int is_submodule_active(struct repository *repo, const char *path)
 {
 	int ret = 0;
 	char *key = NULL;
 	char *value = NULL;
 	const struct string_list *sl;
-	const struct submodule *module = submodule_from_path(null_sha1, path);
+	const struct submodule *module;
+
+	module = submodule_from_cache(repo, null_sha1, path);
 
 	/* early return if there isn't a path->module mapping */
 	if (!module)
@@ -252,14 +248,14 @@ int is_submodule_initialized(const char *path)
 
 	/* submodule.<name>.active is set */
 	key = xstrfmt("submodule.%s.active", module->name);
-	if (!git_config_get_bool(key, &ret)) {
+	if (!repo_config_get_bool(repo, key, &ret)) {
 		free(key);
 		return ret;
 	}
 	free(key);
 
 	/* submodule.active is set */
-	sl = git_config_get_value_multi("submodule.active");
+	sl = repo_config_get_value_multi(repo, "submodule.active");
 	if (sl) {
 		struct pathspec ps;
 		struct argv_array args = ARGV_ARRAY_INIT;
@@ -279,7 +275,7 @@ int is_submodule_initialized(const char *path)
 
 	/* fallback to checking if the URL is set */
 	key = xstrfmt("submodule.%s.url", module->name);
-	ret = !git_config_get_string(key, &value);
+	ret = !repo_config_get_string(repo, key, &value);
 
 	free(value);
 	free(key);
@@ -1484,7 +1480,7 @@ int submodule_move_head(const char *path,
 	const struct submodule *sub;
 	int *error_code_ptr, error_code;
 
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(the_repository, path))
 		return 0;
 
 	if (flags & SUBMODULE_MOVE_HEAD_FORCE)
diff --git a/submodule.h b/submodule.h
index ec6ea8dfd..af8eada23 100644
--- a/submodule.h
+++ b/submodule.h
@@ -43,7 +43,7 @@ extern int submodule_config(const char *var, const char *value, void *cb);
 extern void gitmodules_config(void);
 extern void repo_read_gitmodules(struct repository *repo);
 extern void gitmodules_config_sha1(const unsigned char *commit_sha1);
-extern int is_submodule_initialized(const char *path);
+extern int is_submodule_active(struct repository *repo, const char *path);
 /*
  * Determine if a submodule has been populated at a given 'path' by checking if
  * the <path>/.git resolves to a valid git repository.
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 16/32] convert: convert get_cached_convert_stats_ascii to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (14 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 15/32] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 17/32] convert: convert crlf_to_git " Brandon Williams
                     ` (17 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 3 ++-
 convert.c          | 5 +++--
 convert.h          | 5 ++++-
 3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index c4357dc30..f16ce0053 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -64,7 +64,8 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)
 		const char *w_txt = "";
 		const char *a_txt = get_convert_attr_ascii(path);
 		if (ce && S_ISREG(ce->ce_mode))
-			i_txt = get_cached_convert_stats_ascii(ce->name);
+			i_txt = get_cached_convert_stats_ascii(&the_index,
+							       ce->name);
 		if (!lstat(path, &st) && S_ISREG(st.st_mode))
 			w_txt = get_wt_convert_stats_ascii(path);
 		printf("i/%-5s w/%-5s attr/%-17s\t", i_txt, w_txt, a_txt);
diff --git a/convert.c b/convert.c
index 5f4a4b1f5..574003023 100644
--- a/convert.c
+++ b/convert.c
@@ -135,11 +135,12 @@ static const char *gather_convert_stats_ascii(const char *data, unsigned long si
 	}
 }
 
-const char *get_cached_convert_stats_ascii(const char *path)
+const char *get_cached_convert_stats_ascii(const struct index_state *istate,
+					   const char *path)
 {
 	const char *ret;
 	unsigned long sz;
-	void *data = read_blob_data_from_cache(path, &sz);
+	void *data = read_blob_data_from_index(istate, path, &sz);
 	ret = gather_convert_stats_ascii(data, sz);
 	free(data);
 	return ret;
diff --git a/convert.h b/convert.h
index 82871a11d..667b7dfe0 100644
--- a/convert.h
+++ b/convert.h
@@ -4,6 +4,8 @@
 #ifndef CONVERT_H
 #define CONVERT_H
 
+struct index_state;
+
 enum safe_crlf {
 	SAFE_CRLF_FALSE = 0,
 	SAFE_CRLF_FAIL = 1,
@@ -33,7 +35,8 @@ enum eol {
 };
 
 extern enum eol core_eol;
-extern const char *get_cached_convert_stats_ascii(const char *path);
+extern const char *get_cached_convert_stats_ascii(const struct index_state *istate,
+						  const char *path);
 extern const char *get_wt_convert_stats_ascii(const char *path);
 extern const char *get_convert_attr_ascii(const char *path);
 
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 17/32] convert: convert crlf_to_git to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (15 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 16/32] convert: convert get_cached_convert_stats_ascii to take an index Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 18/32] convert: convert convert_to_git_filter_fd " Brandon Williams
                     ` (16 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 convert.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/convert.c b/convert.c
index 574003023..ff3e72657 100644
--- a/convert.c
+++ b/convert.c
@@ -219,13 +219,13 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,
 	}
 }
 
-static int has_cr_in_index(const char *path)
+static int has_cr_in_index(const struct index_state *istate, const char *path)
 {
 	unsigned long sz;
 	void *data;
 	int has_cr;
 
-	data = read_blob_data_from_cache(path, &sz);
+	data = read_blob_data_from_index(istate, path, &sz);
 	if (!data)
 		return 0;
 	has_cr = memchr(data, '\r', sz) != NULL;
@@ -255,7 +255,8 @@ static int will_convert_lf_to_crlf(size_t len, struct text_stat *stats,
 
 }
 
-static int crlf_to_git(const char *path, const char *src, size_t len,
+static int crlf_to_git(const struct index_state *istate,
+		       const char *path, const char *src, size_t len,
 		       struct strbuf *buf,
 		       enum crlf_action crlf_action, enum safe_crlf checksafe)
 {
@@ -287,7 +288,8 @@ static int crlf_to_git(const char *path, const char *src, size_t len,
 		 * unless we want to renormalize in a merge or
 		 * cherry-pick.
 		 */
-		if ((checksafe != SAFE_CRLF_RENORMALIZE) && has_cr_in_index(path))
+		if ((checksafe != SAFE_CRLF_RENORMALIZE) &&
+		    has_cr_in_index(istate, path))
 			convert_crlf_into_lf = 0;
 	}
 	if ((checksafe == SAFE_CRLF_WARN ||
@@ -1099,7 +1101,7 @@ int convert_to_git(const char *path, const char *src, size_t len,
 		src = dst->buf;
 		len = dst->len;
 	}
-	ret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);
+	ret |= crlf_to_git(&the_index, path, src, len, dst, ca.crlf_action, checksafe);
 	if (ret && dst) {
 		src = dst->buf;
 		len = dst->len;
@@ -1119,7 +1121,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN))
 		die("%s: clean filter '%s' failed", path, ca.drv->name);
 
-	crlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
+	crlf_to_git(&the_index, path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
 	ident_to_git(path, dst->buf, dst->len, dst, ca.ident);
 }
 
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 18/32] convert: convert convert_to_git_filter_fd to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (16 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 17/32] convert: convert crlf_to_git " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 19/32] convert: convert convert_to_git " Brandon Williams
                     ` (15 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 convert.c   | 5 +++--
 convert.h   | 3 ++-
 sha1_file.c | 2 +-
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/convert.c b/convert.c
index ff3e72657..824b606fa 100644
--- a/convert.c
+++ b/convert.c
@@ -1109,7 +1109,8 @@ int convert_to_git(const char *path, const char *src, size_t len,
 	return ret | ident_to_git(path, src, len, dst, ca.ident);
 }
 
-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
+void convert_to_git_filter_fd(const struct index_state *istate,
+			      const char *path, int fd, struct strbuf *dst,
 			      enum safe_crlf checksafe)
 {
 	struct conv_attrs ca;
@@ -1121,7 +1122,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN))
 		die("%s: clean filter '%s' failed", path, ca.drv->name);
 
-	crlf_to_git(&the_index, path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
+	crlf_to_git(istate, path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
 	ident_to_git(path, dst->buf, dst->len, dst, ca.ident);
 }
 
diff --git a/convert.h b/convert.h
index 667b7dfe0..3a813a797 100644
--- a/convert.h
+++ b/convert.h
@@ -52,7 +52,8 @@ static inline int would_convert_to_git(const char *path)
 	return convert_to_git(path, NULL, 0, NULL, 0);
 }
 /* Precondition: would_convert_to_git_filter_fd(path) == true */
-extern void convert_to_git_filter_fd(const char *path, int fd,
+extern void convert_to_git_filter_fd(const struct index_state *istate,
+				     const char *path, int fd,
 				     struct strbuf *dst,
 				     enum safe_crlf checksafe);
 extern int would_convert_to_git_filter_fd(const char *path);
diff --git a/sha1_file.c b/sha1_file.c
index 44561e0b9..80e9ef3bb 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3581,7 +3581,7 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,
 	assert(path);
 	assert(would_convert_to_git_filter_fd(path));
 
-	convert_to_git_filter_fd(path, fd, &sbuf,
+	convert_to_git_filter_fd(&the_index, path, fd, &sbuf,
 				 write_object ? safe_crlf : SAFE_CRLF_FALSE);
 
 	if (write_object)
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 19/32] convert: convert convert_to_git to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (17 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 18/32] convert: convert convert_to_git_filter_fd " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 20/32] convert: convert renormalize_buffer " Brandon Williams
                     ` (14 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 apply.c         | 2 +-
 builtin/blame.c | 2 +-
 combine-diff.c  | 2 +-
 convert.c       | 7 ++++---
 convert.h       | 8 +++++---
 diff.c          | 6 +++---
 dir.c           | 2 +-
 sha1_file.c     | 4 ++--
 8 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/apply.c b/apply.c
index 87db9618d..8eca54325 100644
--- a/apply.c
+++ b/apply.c
@@ -2268,7 +2268,7 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)
 	case S_IFREG:
 		if (strbuf_read_file(buf, path, st->st_size) != st->st_size)
 			return error(_("unable to open or read %s"), path);
-		convert_to_git(path, buf->buf, buf->len, buf, 0);
+		convert_to_git(&the_index, path, buf->buf, buf->len, buf, 0);
 		return 0;
 	default:
 		return -1;
diff --git a/builtin/blame.c b/builtin/blame.c
index c0ae49298..317d2ec37 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -2384,7 +2384,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 		if (strbuf_read(&buf, 0, 0) < 0)
 			die_errno("failed to read from stdin");
 	}
-	convert_to_git(path, buf.buf, buf.len, &buf, 0);
+	convert_to_git(&the_index, path, buf.buf, buf.len, &buf, 0);
 	origin->file.ptr = buf.buf;
 	origin->file.size = buf.len;
 	pretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_oid.hash);
diff --git a/combine-diff.c b/combine-diff.c
index 2848034fe..74f723af3 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1053,7 +1053,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
 			if (is_file) {
 				struct strbuf buf = STRBUF_INIT;
 
-				if (convert_to_git(elem->path, result, len, &buf, safe_crlf)) {
+				if (convert_to_git(&the_index, elem->path, result, len, &buf, safe_crlf)) {
 					free(result);
 					result = strbuf_detach(&buf, &len);
 					result_size = len;
diff --git a/convert.c b/convert.c
index 824b606fa..5af6fdf3f 100644
--- a/convert.c
+++ b/convert.c
@@ -1085,7 +1085,8 @@ const char *get_convert_attr_ascii(const char *path)
 	return "";
 }
 
-int convert_to_git(const char *path, const char *src, size_t len,
+int convert_to_git(const struct index_state *istate,
+		   const char *path, const char *src, size_t len,
                    struct strbuf *dst, enum safe_crlf checksafe)
 {
 	int ret = 0;
@@ -1101,7 +1102,7 @@ int convert_to_git(const char *path, const char *src, size_t len,
 		src = dst->buf;
 		len = dst->len;
 	}
-	ret |= crlf_to_git(&the_index, path, src, len, dst, ca.crlf_action, checksafe);
+	ret |= crlf_to_git(istate, path, src, len, dst, ca.crlf_action, checksafe);
 	if (ret && dst) {
 		src = dst->buf;
 		len = dst->len;
@@ -1172,7 +1173,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str
 		src = dst->buf;
 		len = dst->len;
 	}
-	return ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE);
+	return ret | convert_to_git(&the_index, path, src, len, dst, SAFE_CRLF_RENORMALIZE);
 }
 
 /*****************************************************************
diff --git a/convert.h b/convert.h
index 3a813a797..60cb41d6a 100644
--- a/convert.h
+++ b/convert.h
@@ -41,15 +41,17 @@ extern const char *get_wt_convert_stats_ascii(const char *path);
 extern const char *get_convert_attr_ascii(const char *path);
 
 /* returns 1 if *dst was used */
-extern int convert_to_git(const char *path, const char *src, size_t len,
+extern int convert_to_git(const struct index_state *istate,
+			  const char *path, const char *src, size_t len,
 			  struct strbuf *dst, enum safe_crlf checksafe);
 extern int convert_to_working_tree(const char *path, const char *src,
 				   size_t len, struct strbuf *dst);
 extern int renormalize_buffer(const char *path, const char *src, size_t len,
 			      struct strbuf *dst);
-static inline int would_convert_to_git(const char *path)
+static inline int would_convert_to_git(const struct index_state *istate,
+				       const char *path)
 {
-	return convert_to_git(path, NULL, 0, NULL, 0);
+	return convert_to_git(istate, path, NULL, 0, NULL, 0);
 }
 /* Precondition: would_convert_to_git_filter_fd(path) == true */
 extern void convert_to_git_filter_fd(const struct index_state *istate,
diff --git a/diff.c b/diff.c
index 2f2467c6d..87ed6d6d3 100644
--- a/diff.c
+++ b/diff.c
@@ -2756,7 +2756,7 @@ static int reuse_worktree_file(const char *name, const unsigned char *sha1, int
 	 * Similarly, if we'd have to convert the file contents anyway, that
 	 * makes the optimization not worthwhile.
 	 */
-	if (!want_file && would_convert_to_git(name))
+	if (!want_file && would_convert_to_git(&the_index, name))
 		return 0;
 
 	len = strlen(name);
@@ -2878,7 +2878,7 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)
 		 * point if the path requires us to run the content
 		 * conversion.
 		 */
-		if (size_only && !would_convert_to_git(s->path))
+		if (size_only && !would_convert_to_git(&the_index, s->path))
 			return 0;
 
 		/*
@@ -2905,7 +2905,7 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)
 		/*
 		 * Convert from working tree format to canonical git format
 		 */
-		if (convert_to_git(s->path, s->data, s->size, &buf, crlf_warn)) {
+		if (convert_to_git(&the_index, s->path, s->data, s->size, &buf, crlf_warn)) {
 			size_t size = 0;
 			munmap(s->data, s->size);
 			s->should_munmap = 0;
diff --git a/dir.c b/dir.c
index 0c26a53d2..8dc74c5d2 100644
--- a/dir.c
+++ b/dir.c
@@ -796,7 +796,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,
 				 (pos = index_name_pos(istate, fname, strlen(fname))) >= 0 &&
 				 !ce_stage(istate->cache[pos]) &&
 				 ce_uptodate(istate->cache[pos]) &&
-				 !would_convert_to_git(fname))
+				 !would_convert_to_git(istate, fname))
 				hashcpy(sha1_stat->sha1,
 					istate->cache[pos]->oid.hash);
 			else
diff --git a/sha1_file.c b/sha1_file.c
index 80e9ef3bb..a900b2804 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3547,7 +3547,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,
 	 */
 	if ((type == OBJ_BLOB) && path) {
 		struct strbuf nbuf = STRBUF_INIT;
-		if (convert_to_git(path, buf, size, &nbuf,
+		if (convert_to_git(&the_index, path, buf, size, &nbuf,
 				   write_object ? safe_crlf : SAFE_CRLF_FALSE)) {
 			buf = strbuf_detach(&nbuf, &size);
 			re_allocated = 1;
@@ -3669,7 +3669,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,
 	else if (!S_ISREG(st->st_mode))
 		ret = index_pipe(sha1, fd, type, path, flags);
 	else if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||
-		 (path && would_convert_to_git(path)))
+		 (path && would_convert_to_git(&the_index, path)))
 		ret = index_core(sha1, fd, xsize_t(st->st_size), type, path,
 				 flags);
 	else
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 20/32] convert: convert renormalize_buffer to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (18 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 19/32] convert: convert convert_to_git " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 21/32] tree: convert read_tree to take an index parameter Brandon Williams
                     ` (13 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 convert.c         | 6 ++++--
 convert.h         | 3 ++-
 ll-merge.c        | 2 +-
 merge-recursive.c | 4 ++--
 4 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/convert.c b/convert.c
index 5af6fdf3f..7d2a519da 100644
--- a/convert.c
+++ b/convert.c
@@ -1,3 +1,4 @@
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "config.h"
 #include "attr.h"
@@ -1166,14 +1167,15 @@ int convert_to_working_tree(const char *path, const char *src, size_t len, struc
 	return convert_to_working_tree_internal(path, src, len, dst, 0);
 }
 
-int renormalize_buffer(const char *path, const char *src, size_t len, struct strbuf *dst)
+int renormalize_buffer(const struct index_state *istate, const char *path,
+		       const char *src, size_t len, struct strbuf *dst)
 {
 	int ret = convert_to_working_tree_internal(path, src, len, dst, 1);
 	if (ret) {
 		src = dst->buf;
 		len = dst->len;
 	}
-	return ret | convert_to_git(&the_index, path, src, len, dst, SAFE_CRLF_RENORMALIZE);
+	return ret | convert_to_git(istate, path, src, len, dst, SAFE_CRLF_RENORMALIZE);
 }
 
 /*****************************************************************
diff --git a/convert.h b/convert.h
index 60cb41d6a..cecf59d1a 100644
--- a/convert.h
+++ b/convert.h
@@ -46,7 +46,8 @@ extern int convert_to_git(const struct index_state *istate,
 			  struct strbuf *dst, enum safe_crlf checksafe);
 extern int convert_to_working_tree(const char *path, const char *src,
 				   size_t len, struct strbuf *dst);
-extern int renormalize_buffer(const char *path, const char *src, size_t len,
+extern int renormalize_buffer(const struct index_state *istate,
+			      const char *path, const char *src, size_t len,
 			      struct strbuf *dst);
 static inline int would_convert_to_git(const struct index_state *istate,
 				       const char *path)
diff --git a/ll-merge.c b/ll-merge.c
index 24ff94e1d..b9576efe9 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -340,7 +340,7 @@ static const struct ll_merge_driver *find_ll_merge_driver(const char *merge_attr
 static void normalize_file(mmfile_t *mm, const char *path)
 {
 	struct strbuf strbuf = STRBUF_INIT;
-	if (renormalize_buffer(path, mm->ptr, mm->size, &strbuf)) {
+	if (renormalize_buffer(&the_index, path, mm->ptr, mm->size, &strbuf)) {
 		free(mm->ptr);
 		mm->size = strbuf.len;
 		mm->ptr = strbuf_detach(&strbuf, NULL);
diff --git a/merge-recursive.c b/merge-recursive.c
index c2494f34f..a4b3858d0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1640,8 +1640,8 @@ static int blob_unchanged(struct merge_options *opt,
 	 * performed.  Comparison can be skipped if both files are
 	 * unchanged since their sha1s have already been compared.
 	 */
-	if (renormalize_buffer(path, o.buf, o.len, &o) |
-	    renormalize_buffer(path, a.buf, a.len, &a))
+	if (renormalize_buffer(&the_index, path, o.buf, o.len, &o) |
+	    renormalize_buffer(&the_index, path, a.buf, a.len, &a))
 		ret = (o.len == a.len && !memcmp(o.buf, a.buf, o.len));
 
 error_return:
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 21/32] tree: convert read_tree to take an index parameter
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (19 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 20/32] convert: convert renormalize_buffer " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 22/32] ls-files: convert overlay_tree_on_cache to take an index Brandon Williams
                     ` (12 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c |  2 +-
 tree.c             | 28 ++++++++++++++++++----------
 tree.h             |  3 ++-
 3 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f16ce0053..620487a77 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -461,7 +461,7 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 			       PATHSPEC_PREFER_CWD, prefix, matchbuf);
 	} else
 		memset(&pathspec, 0, sizeof(pathspec));
-	if (read_tree(tree, 1, &pathspec))
+	if (read_tree(tree, 1, &pathspec, &the_index))
 		die("unable to read tree entries %s", tree_name);
 
 	for (i = 0; i < active_nr; i++) {
diff --git a/tree.c b/tree.c
index 603b29ee8..dd69423d9 100644
--- a/tree.c
+++ b/tree.c
@@ -1,3 +1,4 @@
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "cache-tree.h"
 #include "tree.h"
@@ -8,7 +9,11 @@
 
 const char *tree_type = "tree";
 
-static int read_one_entry_opt(const unsigned char *sha1, const char *base, int baselen, const char *pathname, unsigned mode, int stage, int opt)
+static int read_one_entry_opt(struct index_state *istate,
+			      const unsigned char *sha1,
+			      const char *base, int baselen,
+			      const char *pathname,
+			      unsigned mode, int stage, int opt)
 {
 	int len;
 	unsigned int size;
@@ -27,14 +32,15 @@ static int read_one_entry_opt(const unsigned char *sha1, const char *base, int b
 	memcpy(ce->name, base, baselen);
 	memcpy(ce->name + baselen, pathname, len+1);
 	hashcpy(ce->oid.hash, sha1);
-	return add_cache_entry(ce, opt);
+	return add_index_entry(istate, ce, opt);
 }
 
 static int read_one_entry(const unsigned char *sha1, struct strbuf *base,
 			  const char *pathname, unsigned mode, int stage,
 			  void *context)
 {
-	return read_one_entry_opt(sha1, base->buf, base->len, pathname,
+	struct index_state *istate = context;
+	return read_one_entry_opt(istate, sha1, base->buf, base->len, pathname,
 				  mode, stage,
 				  ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK);
 }
@@ -47,7 +53,8 @@ static int read_one_entry_quick(const unsigned char *sha1, struct strbuf *base,
 				const char *pathname, unsigned mode, int stage,
 				void *context)
 {
-	return read_one_entry_opt(sha1, base->buf, base->len, pathname,
+	struct index_state *istate = context;
+	return read_one_entry_opt(istate, sha1, base->buf, base->len, pathname,
 				  mode, stage,
 				  ADD_CACHE_JUST_APPEND);
 }
@@ -144,7 +151,8 @@ static int cmp_cache_name_compare(const void *a_, const void *b_)
 				  ce2->name, ce2->ce_namelen, ce_stage(ce2));
 }
 
-int read_tree(struct tree *tree, int stage, struct pathspec *match)
+int read_tree(struct tree *tree, int stage, struct pathspec *match,
+	      struct index_state *istate)
 {
 	read_tree_fn_t fn = NULL;
 	int i, err;
@@ -164,23 +172,23 @@ int read_tree(struct tree *tree, int stage, struct pathspec *match)
 	 * do it the original slow way, otherwise, append and then
 	 * sort at the end.
 	 */
-	for (i = 0; !fn && i < active_nr; i++) {
-		const struct cache_entry *ce = active_cache[i];
+	for (i = 0; !fn && i < istate->cache_nr; i++) {
+		const struct cache_entry *ce = istate->cache[i];
 		if (ce_stage(ce) == stage)
 			fn = read_one_entry;
 	}
 
 	if (!fn)
 		fn = read_one_entry_quick;
-	err = read_tree_recursive(tree, "", 0, stage, match, fn, NULL);
+	err = read_tree_recursive(tree, "", 0, stage, match, fn, istate);
 	if (fn == read_one_entry || err)
 		return err;
 
 	/*
 	 * Sort the cache entry -- we need to nuke the cache tree, though.
 	 */
-	cache_tree_free(&active_cache_tree);
-	QSORT(active_cache, active_nr, cmp_cache_name_compare);
+	cache_tree_free(&istate->cache_tree);
+	QSORT(istate->cache, istate->cache_nr, cmp_cache_name_compare);
 	return 0;
 }
 
diff --git a/tree.h b/tree.h
index 0d4734b94..744e6dc2a 100644
--- a/tree.h
+++ b/tree.h
@@ -34,6 +34,7 @@ extern int read_tree_recursive(struct tree *tree,
 			       int stage, const struct pathspec *pathspec,
 			       read_tree_fn_t fn, void *context);
 
-extern int read_tree(struct tree *tree, int stage, struct pathspec *pathspec);
+extern int read_tree(struct tree *tree, int stage, struct pathspec *pathspec,
+		     struct index_state *istate);
 
 #endif /* TREE_H */
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 22/32] ls-files: convert overlay_tree_on_cache to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (20 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 21/32] tree: convert read_tree to take an index parameter Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 23/32] ls-files: convert write_eolinfo " Brandon Williams
                     ` (11 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/commit.c   |  3 ++-
 builtin/ls-files.c | 15 ++++++++-------
 cache.h            |  3 ++-
 3 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 805da4915..3d98084fb 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -254,7 +254,8 @@ static int list_paths(struct string_list *list, const char *with_tree,
 
 	if (with_tree) {
 		char *max_prefix = common_prefix(pattern);
-		overlay_tree_on_cache(with_tree, max_prefix ? max_prefix : prefix);
+		overlay_tree_on_index(&the_index, with_tree,
+				      max_prefix ? max_prefix : prefix);
 		free(max_prefix);
 	}
 
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 620487a77..a5ceeb052 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -432,7 +432,8 @@ static int get_common_prefix_len(const char *common_prefix)
  * that were given from the command line.  We are not
  * going to write this index out.
  */
-void overlay_tree_on_cache(const char *tree_name, const char *prefix)
+void overlay_tree_on_index(struct index_state *istate,
+			   const char *tree_name, const char *prefix)
 {
 	struct tree *tree;
 	struct object_id oid;
@@ -447,8 +448,8 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 		die("bad tree-ish %s", tree_name);
 
 	/* Hoist the unmerged entries up to stage #3 to make room */
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = active_cache[i];
+	for (i = 0; i < istate->cache_nr; i++) {
+		struct cache_entry *ce = istate->cache[i];
 		if (!ce_stage(ce))
 			continue;
 		ce->ce_flags |= CE_STAGEMASK;
@@ -461,11 +462,11 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 			       PATHSPEC_PREFER_CWD, prefix, matchbuf);
 	} else
 		memset(&pathspec, 0, sizeof(pathspec));
-	if (read_tree(tree, 1, &pathspec, &the_index))
+	if (read_tree(tree, 1, &pathspec, istate))
 		die("unable to read tree entries %s", tree_name);
 
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = active_cache[i];
+	for (i = 0; i < istate->cache_nr; i++) {
+		struct cache_entry *ce = istate->cache[i];
 		switch (ce_stage(ce)) {
 		case 0:
 			last_stage0 = ce;
@@ -680,7 +681,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		 */
 		if (show_stage || show_unmerged)
 			die("ls-files --with-tree is incompatible with -s or -u");
-		overlay_tree_on_cache(with_tree, max_prefix);
+		overlay_tree_on_index(&the_index, with_tree, max_prefix);
 	}
 	show_files(&dir);
 	if (show_resolve_undo)
diff --git a/cache.h b/cache.h
index 02ab5f801..73724a3ad 100644
--- a/cache.h
+++ b/cache.h
@@ -1993,7 +1993,8 @@ extern int ws_blank_line(const char *line, int len, unsigned ws_rule);
 #define ws_tab_width(rule)     ((rule) & WS_TAB_WIDTH_MASK)
 
 /* ls-files */
-void overlay_tree_on_cache(const char *tree_name, const char *prefix);
+void overlay_tree_on_index(struct index_state *istate,
+			   const char *tree_name, const char *prefix);
 
 char *alias_lookup(const char *alias);
 int split_cmdline(char *cmdline, const char ***argv);
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 23/32] ls-files: convert write_eolinfo to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (21 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 22/32] ls-files: convert overlay_tree_on_cache to take an index Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 24/32] ls-files: convert show_killed_files " Brandon Williams
                     ` (10 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index a5ceeb052..c37e9de11 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -54,17 +54,16 @@ static const char *tag_modified = "";
 static const char *tag_skip_worktree = "";
 static const char *tag_resolve_undo = "";
 
-static void write_eolinfo(const struct cache_entry *ce, const char *path)
+static void write_eolinfo(const struct index_state *istate,
+			  const struct cache_entry *ce, const char *path)
 {
-	if (!show_eol)
-		return;
-	else {
+	if (show_eol) {
 		struct stat st;
 		const char *i_txt = "";
 		const char *w_txt = "";
 		const char *a_txt = get_convert_attr_ascii(path);
 		if (ce && S_ISREG(ce->ce_mode))
-			i_txt = get_cached_convert_stats_ascii(&the_index,
+			i_txt = get_cached_convert_stats_ascii(istate,
 							       ce->name);
 		if (!lstat(path, &st) && S_ISREG(st.st_mode))
 			w_txt = get_wt_convert_stats_ascii(path);
@@ -106,7 +105,7 @@ static void show_dir_entry(const char *tag, struct dir_entry *ent)
 		return;
 
 	fputs(tag, stdout);
-	write_eolinfo(NULL, ent->name);
+	write_eolinfo(NULL, NULL, ent->name);
 	write_name(ent->name);
 }
 
@@ -276,7 +275,7 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 			       find_unique_abbrev(ce->oid.hash, abbrev),
 			       ce_stage(ce));
 		}
-		write_eolinfo(ce, ce->name);
+		write_eolinfo(&the_index, ce, ce->name);
 		write_name(ce->name);
 		if (debug_mode) {
 			const struct stat_data *sd = &ce->ce_stat_data;
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 24/32] ls-files: convert show_killed_files to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (22 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 23/32] ls-files: convert write_eolinfo " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 25/32] ls-files: convert show_other_files " Brandon Williams
                     ` (9 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index c37e9de11..f9578cf9f 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -121,7 +121,8 @@ static void show_other_files(struct dir_struct *dir)
 	}
 }
 
-static void show_killed_files(struct dir_struct *dir)
+static void show_killed_files(const struct index_state *istate,
+			      const struct dir_struct *dir)
 {
 	int i;
 	for (i = 0; i < dir->nr; i++) {
@@ -135,29 +136,29 @@ static void show_killed_files(struct dir_struct *dir)
 				/* If ent->name is prefix of an entry in the
 				 * cache, it will be killed.
 				 */
-				pos = cache_name_pos(ent->name, ent->len);
+				pos = index_name_pos(istate, ent->name, ent->len);
 				if (0 <= pos)
 					die("BUG: killed-file %.*s not found",
 						ent->len, ent->name);
 				pos = -pos - 1;
-				while (pos < active_nr &&
-				       ce_stage(active_cache[pos]))
+				while (pos < istate->cache_nr &&
+				       ce_stage(istate->cache[pos]))
 					pos++; /* skip unmerged */
-				if (active_nr <= pos)
+				if (istate->cache_nr <= pos)
 					break;
 				/* pos points at a name immediately after
 				 * ent->name in the cache.  Does it expect
 				 * ent->name to be a directory?
 				 */
-				len = ce_namelen(active_cache[pos]);
+				len = ce_namelen(istate->cache[pos]);
 				if ((ent->len < len) &&
-				    !strncmp(active_cache[pos]->name,
+				    !strncmp(istate->cache[pos]->name,
 					     ent->name, ent->len) &&
-				    active_cache[pos]->name[ent->len] == '/')
+				    istate->cache[pos]->name[ent->len] == '/')
 					killed = 1;
 				break;
 			}
-			if (0 <= cache_name_pos(ent->name, sp - ent->name)) {
+			if (0 <= index_name_pos(istate, ent->name, sp - ent->name)) {
 				/* If any of the leading directories in
 				 * ent->name is registered in the cache,
 				 * ent->name will be killed.
@@ -338,7 +339,7 @@ static void show_files(struct dir_struct *dir)
 		if (show_others)
 			show_other_files(dir);
 		if (show_killed)
-			show_killed_files(dir);
+			show_killed_files(&the_index, dir);
 	}
 	if (show_cached || show_stage) {
 		for (i = 0; i < active_nr; i++) {
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 25/32] ls-files: convert show_other_files to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (23 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 24/32] ls-files: convert show_killed_files " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 26/32] ls-files: convert show_ru_info " Brandon Williams
                     ` (8 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f9578cf9f..d00ca7810 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -109,13 +109,14 @@ static void show_dir_entry(const char *tag, struct dir_entry *ent)
 	write_name(ent->name);
 }
 
-static void show_other_files(struct dir_struct *dir)
+static void show_other_files(const struct index_state *istate,
+			     const struct dir_struct *dir)
 {
 	int i;
 
 	for (i = 0; i < dir->nr; i++) {
 		struct dir_entry *ent = dir->entries[i];
-		if (!cache_name_is_other(ent->name, ent->len))
+		if (!index_name_is_other(istate, ent->name, ent->len))
 			continue;
 		show_dir_entry(tag_other, ent);
 	}
@@ -337,7 +338,7 @@ static void show_files(struct dir_struct *dir)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
 		fill_directory(dir, &the_index, &pathspec);
 		if (show_others)
-			show_other_files(dir);
+			show_other_files(&the_index, dir);
 		if (show_killed)
 			show_killed_files(&the_index, dir);
 	}
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 26/32] ls-files: convert show_ru_info to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (24 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 25/32] ls-files: convert show_other_files " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 27/32] ls-files: convert ce_excluded " Brandon Williams
                     ` (7 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index d00ca7810..2838e2f75 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -293,14 +293,14 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 	strbuf_release(&name);
 }
 
-static void show_ru_info(void)
+static void show_ru_info(const struct index_state *istate)
 {
 	struct string_list_item *item;
 
-	if (!the_index.resolve_undo)
+	if (!istate->resolve_undo)
 		return;
 
-	for_each_string_list_item(item, the_index.resolve_undo) {
+	for_each_string_list_item(item, istate->resolve_undo) {
 		const char *path = item->string;
 		struct resolve_undo_info *ui = item->util;
 		int i, len;
@@ -686,7 +686,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	}
 	show_files(&dir);
 	if (show_resolve_undo)
-		show_ru_info();
+		show_ru_info(&the_index);
 
 	if (ps_matched) {
 		int bad;
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 27/32] ls-files: convert ce_excluded to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (25 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 26/32] ls-files: convert show_ru_info " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 28/32] ls-files: convert prune_cache " Brandon Williams
                     ` (6 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 2838e2f75..289c6b2a5 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -322,10 +322,11 @@ static void show_ru_info(const struct index_state *istate)
 	}
 }
 
-static int ce_excluded(struct dir_struct *dir, const struct cache_entry *ce)
+static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
+		       const struct cache_entry *ce)
 {
 	int dtype = ce_to_dtype(ce);
-	return is_excluded(dir, &the_index, ce->name, &dtype);
+	return is_excluded(dir, istate, ce->name, &dtype);
 }
 
 static void show_files(struct dir_struct *dir)
@@ -346,7 +347,7 @@ static void show_files(struct dir_struct *dir)
 		for (i = 0; i < active_nr; i++) {
 			const struct cache_entry *ce = active_cache[i];
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, ce))
+			    !ce_excluded(dir, &the_index, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
@@ -362,7 +363,7 @@ static void show_files(struct dir_struct *dir)
 			struct stat st;
 			int err;
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, ce))
+			    !ce_excluded(dir, &the_index, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 28/32] ls-files: convert prune_cache to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (26 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 27/32] ls-files: convert ce_excluded " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 29/32] ls-files: convert show_files " Brandon Williams
                     ` (5 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 289c6b2a5..e2d8fb7f6 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -381,30 +381,31 @@ static void show_files(struct dir_struct *dir)
 /*
  * Prune the index to only contain stuff starting with "prefix"
  */
-static void prune_cache(const char *prefix, size_t prefixlen)
+static void prune_index(struct index_state *istate,
+			const char *prefix, size_t prefixlen)
 {
 	int pos;
 	unsigned int first, last;
 
 	if (!prefix)
 		return;
-	pos = cache_name_pos(prefix, prefixlen);
+	pos = index_name_pos(istate, prefix, prefixlen);
 	if (pos < 0)
 		pos = -pos-1;
 	first = pos;
-	last = active_nr;
+	last = istate->cache_nr;
 	while (last > first) {
 		int next = (last + first) >> 1;
-		const struct cache_entry *ce = active_cache[next];
+		const struct cache_entry *ce = istate->cache[next];
 		if (!strncmp(ce->name, prefix, prefixlen)) {
 			first = next+1;
 			continue;
 		}
 		last = next;
 	}
-	memmove(active_cache, active_cache + pos,
+	memmove(istate->cache, istate->cache + pos,
 		(last - pos) * sizeof(struct cache_entry *));
-	active_nr = last - pos;
+	istate->cache_nr = last - pos;
 }
 
 static int get_common_prefix_len(const char *common_prefix)
@@ -662,7 +663,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		max_prefix = common_prefix(&pathspec);
 	max_prefix_len = get_common_prefix_len(max_prefix);
 
-	prune_cache(max_prefix, max_prefix_len);
+	prune_index(&the_index, max_prefix, max_prefix_len);
 
 	/* Treat unmatching pathspec elements as errors */
 	if (pathspec.nr && error_unmatch)
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 29/32] ls-files: convert show_files to take an index
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (27 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 28/32] ls-files: convert prune_cache " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 30/32] ls-files: factor out debug info into a function Brandon Williams
                     ` (4 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index e2d8fb7f6..3061af2c5 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -329,7 +329,7 @@ static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
 	return is_excluded(dir, istate, ce->name, &dtype);
 }
 
-static void show_files(struct dir_struct *dir)
+static void show_files(struct index_state *istate, struct dir_struct *dir)
 {
 	int i;
 
@@ -337,17 +337,17 @@ static void show_files(struct dir_struct *dir)
 	if (show_others || show_killed) {
 		if (!show_others)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
-		fill_directory(dir, &the_index, &pathspec);
+		fill_directory(dir, istate, &pathspec);
 		if (show_others)
-			show_other_files(&the_index, dir);
+			show_other_files(istate, dir);
 		if (show_killed)
-			show_killed_files(&the_index, dir);
+			show_killed_files(istate, dir);
 	}
 	if (show_cached || show_stage) {
-		for (i = 0; i < active_nr; i++) {
-			const struct cache_entry *ce = active_cache[i];
+		for (i = 0; i < istate->cache_nr; i++) {
+			const struct cache_entry *ce = istate->cache[i];
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, &the_index, ce))
+			    !ce_excluded(dir, istate, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
@@ -358,12 +358,12 @@ static void show_files(struct dir_struct *dir)
 		}
 	}
 	if (show_deleted || show_modified) {
-		for (i = 0; i < active_nr; i++) {
-			const struct cache_entry *ce = active_cache[i];
+		for (i = 0; i < istate->cache_nr; i++) {
+			const struct cache_entry *ce = istate->cache[i];
 			struct stat st;
 			int err;
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, &the_index, ce))
+			    !ce_excluded(dir, istate, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
@@ -372,7 +372,7 @@ static void show_files(struct dir_struct *dir)
 			err = lstat(ce->name, &st);
 			if (show_deleted && err)
 				show_ce_entry(tag_removed, ce);
-			if (show_modified && ce_modified(ce, &st, 0))
+			if (show_modified && ie_modified(istate, ce, &st, 0))
 				show_ce_entry(tag_modified, ce);
 		}
 	}
@@ -686,7 +686,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 			die("ls-files --with-tree is incompatible with -s or -u");
 		overlay_tree_on_index(&the_index, with_tree, max_prefix);
 	}
-	show_files(&dir);
+	show_files(&the_index, &dir);
 	if (show_resolve_undo)
 		show_ru_info(&the_index);
 
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 30/32] ls-files: factor out debug info into a function
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (28 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 29/32] ls-files: convert show_files " Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:40   ` [PATCH v2 31/32] ls-files: factor out tag calculation Brandon Williams
                     ` (3 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 3061af2c5..2849c9a65 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -94,6 +94,19 @@ static void write_name(const char *name)
 	strbuf_release(&full_name);
 }
 
+static void print_debug(const struct cache_entry *ce)
+{
+	if (debug_mode) {
+		const struct stat_data *sd = &ce->ce_stat_data;
+
+		printf("  ctime: %d:%d\n", sd->sd_ctime.sec, sd->sd_ctime.nsec);
+		printf("  mtime: %d:%d\n", sd->sd_mtime.sec, sd->sd_mtime.nsec);
+		printf("  dev: %d\tino: %d\n", sd->sd_dev, sd->sd_ino);
+		printf("  uid: %d\tgid: %d\n", sd->sd_uid, sd->sd_gid);
+		printf("  size: %d\tflags: %x\n", sd->sd_size, ce->ce_flags);
+	}
+}
+
 static void show_dir_entry(const char *tag, struct dir_entry *ent)
 {
 	int len = max_prefix_len;
@@ -279,15 +292,7 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 		}
 		write_eolinfo(&the_index, ce, ce->name);
 		write_name(ce->name);
-		if (debug_mode) {
-			const struct stat_data *sd = &ce->ce_stat_data;
-
-			printf("  ctime: %d:%d\n", sd->sd_ctime.sec, sd->sd_ctime.nsec);
-			printf("  mtime: %d:%d\n", sd->sd_mtime.sec, sd->sd_mtime.nsec);
-			printf("  dev: %d\tino: %d\n", sd->sd_dev, sd->sd_ino);
-			printf("  uid: %d\tgid: %d\n", sd->sd_uid, sd->sd_gid);
-			printf("  size: %d\tflags: %x\n", sd->sd_size, ce->ce_flags);
-		}
+		print_debug(ce);
 	}
 
 	strbuf_release(&name);
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 31/32] ls-files: factor out tag calculation
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (29 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 30/32] ls-files: factor out debug info into a function Brandon Williams
@ 2017-06-08 23:40   ` Brandon Williams
  2017-06-08 23:41   ` [PATCH v2 32/32] ls-files: use repository object Brandon Williams
                     ` (2 subsequent siblings)
  33 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:40 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c | 41 +++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 2849c9a65..6a0302a28 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -94,6 +94,30 @@ static void write_name(const char *name)
 	strbuf_release(&full_name);
 }
 
+static const char *get_tag(const struct cache_entry *ce, const char *tag)
+{
+	static char alttag[4];
+
+	if (tag && *tag && show_valid_bit && (ce->ce_flags & CE_VALID)) {
+		memcpy(alttag, tag, 3);
+
+		if (isalpha(tag[0])) {
+			alttag[0] = tolower(tag[0]);
+		} else if (tag[0] == '?') {
+			alttag[0] = '!';
+		} else {
+			alttag[0] = 'v';
+			alttag[1] = tag[0];
+			alttag[2] = ' ';
+			alttag[3] = 0;
+		}
+
+		tag = alttag;
+	}
+
+	return tag;
+}
+
 static void print_debug(const struct cache_entry *ce)
 {
 	if (debug_mode) {
@@ -264,22 +288,7 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 				  len, ps_matched,
 				  S_ISDIR(ce->ce_mode) ||
 				  S_ISGITLINK(ce->ce_mode))) {
-		if (tag && *tag && show_valid_bit &&
-		    (ce->ce_flags & CE_VALID)) {
-			static char alttag[4];
-			memcpy(alttag, tag, 3);
-			if (isalpha(tag[0]))
-				alttag[0] = tolower(tag[0]);
-			else if (tag[0] == '?')
-				alttag[0] = '!';
-			else {
-				alttag[0] = 'v';
-				alttag[1] = tag[0];
-				alttag[2] = ' ';
-				alttag[3] = 0;
-			}
-			tag = alttag;
-		}
+		tag = get_tag(ce, tag);
 
 		if (!show_stage) {
 			fputs(tag, stdout);
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 32/32] ls-files: use repository object
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (30 preceding siblings ...)
  2017-06-08 23:40   ` [PATCH v2 31/32] ls-files: factor out tag calculation Brandon Williams
@ 2017-06-08 23:41   ` Brandon Williams
  2017-06-09  0:08     ` Brandon Williams
  2017-06-10  0:40   ` [PATCH v2 00/32] " Jonathan Tan
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
  33 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-08 23:41 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert ls-files to use a repository struct and recurse submodules
inprocess.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c                     | 198 ++++++++++++++-------------------
 git.c                                  |   2 +-
 repository.c                           |   4 +
 repository.h                           |   1 +
 t/t3007-ls-files-recurse-submodules.sh |  39 +++++++
 5 files changed, 129 insertions(+), 115 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 6a0302a28..503fed5a1 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -5,7 +5,9 @@
  *
  * Copyright (C) Linus Torvalds, 2005
  */
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "quote.h"
 #include "dir.h"
@@ -32,10 +34,8 @@ static int line_terminator = '\n';
 static int debug_mode;
 static int show_eol;
 static int recurse_submodules;
-static struct argv_array submodule_options = ARGV_ARRAY_INIT;
 
 static const char *prefix;
-static const char *super_prefix;
 static int max_prefix_len;
 static int prefix_len;
 static struct pathspec pathspec;
@@ -73,25 +73,12 @@ static void write_eolinfo(const struct index_state *istate,
 
 static void write_name(const char *name)
 {
-	/*
-	 * Prepend the super_prefix to name to construct the full_name to be
-	 * written.
-	 */
-	struct strbuf full_name = STRBUF_INIT;
-	if (super_prefix) {
-		strbuf_addstr(&full_name, super_prefix);
-		strbuf_addstr(&full_name, name);
-		name = full_name.buf;
-	}
-
 	/*
 	 * With "--full-name", prefix_len=0; this caller needs to pass
 	 * an empty string in that case (a NULL is good for "").
 	 */
 	write_name_quoted_relative(name, prefix_len ? prefix : NULL,
 				   stdout, line_terminator);
-
-	strbuf_release(&full_name);
 }
 
 static const char *get_tag(const struct cache_entry *ce, const char *tag)
@@ -210,82 +197,41 @@ static void show_killed_files(const struct index_state *istate,
 	}
 }
 
-/*
- * Compile an argv_array with all of the options supported by --recurse_submodules
- */
-static void compile_submodule_options(const char **argv,
-				      const struct dir_struct *dir,
-				      int show_tag)
-{
-	if (line_terminator == '\0')
-		argv_array_push(&submodule_options, "-z");
-	if (show_tag)
-		argv_array_push(&submodule_options, "-t");
-	if (show_valid_bit)
-		argv_array_push(&submodule_options, "-v");
-	if (show_cached)
-		argv_array_push(&submodule_options, "--cached");
-	if (show_eol)
-		argv_array_push(&submodule_options, "--eol");
-	if (debug_mode)
-		argv_array_push(&submodule_options, "--debug");
-
-	/* Add Pathspecs */
-	argv_array_push(&submodule_options, "--");
-	for (; *argv; argv++)
-		argv_array_push(&submodule_options, *argv);
-}
+static void show_files(struct repository *repo, struct dir_struct *dir);
 
-/**
- * Recursively call ls-files on a submodule
- */
-static void show_gitlink(const struct cache_entry *ce)
+static void show_submodule(const struct repository *superproject,
+			   struct dir_struct *dir, const char *path)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-	char *dir;
-
-	prepare_submodule_repo_env(&cp.env_array);
-	argv_array_push(&cp.env_array, GIT_DIR_ENVIRONMENT);
-
-	if (prefix_len)
-		argv_array_pushf(&cp.env_array, "%s=%s",
-				 GIT_TOPLEVEL_PREFIX_ENVIRONMENT,
-				 prefix);
-	argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
-			 super_prefix ? super_prefix : "",
-			 ce->name);
-	argv_array_push(&cp.args, "ls-files");
-	argv_array_push(&cp.args, "--recurse-submodules");
-
-	/* add supported options */
-	argv_array_pushv(&cp.args, submodule_options.argv);
-
-	cp.git_cmd = 1;
-	dir = mkpathdup("%s/%s", get_git_work_tree(), ce->name);
-	cp.dir = dir;
-	status = run_command(&cp);
-	free(dir);
-	if (status)
-		exit(status);
+	struct repository submodule;
+	char *gitdir = mkpathdup("%s/%s", superproject->worktree, path);
+	repo_init(&submodule, gitdir);
+
+	repo_read_index(&submodule);
+	repo_read_gitmodules(&submodule);
+
+	if (superproject->submodule_prefix)
+		submodule.submodule_prefix = xstrfmt("%s%s/", superproject->submodule_prefix, path);
+	else
+		submodule.submodule_prefix = xstrfmt("%s/", path);
+	show_files(&submodule, dir);
+
+	repo_clear(&submodule);
+	free(gitdir);
 }
 
-static void show_ce_entry(const char *tag, const struct cache_entry *ce)
+static void show_ce(struct repository *repo, struct dir_struct *dir,
+		    const struct cache_entry *ce, const char *fullname,
+		    const char *tag)
 {
-	struct strbuf name = STRBUF_INIT;
-	int len = max_prefix_len;
-	if (super_prefix)
-		strbuf_addstr(&name, super_prefix);
-	strbuf_addstr(&name, ce->name);
-
-	if (len > ce_namelen(ce))
+	if (max_prefix_len > strlen(fullname))
 		die("git ls-files: internal error - cache entry not superset of prefix");
 
 	if (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&
-	    submodule_path_match(&pathspec, name.buf, ps_matched)) {
-		show_gitlink(ce);
-	} else if (match_pathspec(&pathspec, name.buf, name.len,
-				  len, ps_matched,
+	    submodule_path_match(&pathspec, fullname, ps_matched) &&
+	    is_submodule_active(repo, ce->name)) {
+		show_submodule(repo, dir, ce->name);
+	} else if (match_pathspec(&pathspec, fullname, strlen(fullname),
+				  max_prefix_len, ps_matched,
 				  S_ISDIR(ce->ce_mode) ||
 				  S_ISGITLINK(ce->ce_mode))) {
 		tag = get_tag(ce, tag);
@@ -299,12 +245,11 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
 			       find_unique_abbrev(ce->oid.hash, abbrev),
 			       ce_stage(ce));
 		}
-		write_eolinfo(&the_index, ce, ce->name);
-		write_name(ce->name);
+
+		write_eolinfo(repo->index, ce, fullname);
+		write_name(fullname);
 		print_debug(ce);
 	}
-
-	strbuf_release(&name);
 }
 
 static void show_ru_info(const struct index_state *istate)
@@ -337,59 +282,79 @@ static void show_ru_info(const struct index_state *istate)
 }
 
 static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
-		       const struct cache_entry *ce)
+		       const char *fullname, const struct cache_entry *ce)
 {
 	int dtype = ce_to_dtype(ce);
-	return is_excluded(dir, istate, ce->name, &dtype);
+	return is_excluded(dir, istate, fullname, &dtype);
+}
+
+static void construct_fullname(struct strbuf *out, const struct repository *repo,
+			       const struct cache_entry *ce)
+{
+	strbuf_reset(out);
+	if (repo->submodule_prefix)
+		strbuf_addstr(out, repo->submodule_prefix);
+	strbuf_addstr(out, ce->name);
 }
 
-static void show_files(struct index_state *istate, struct dir_struct *dir)
+static void show_files(struct repository *repo, struct dir_struct *dir)
 {
 	int i;
+	struct strbuf fullname = STRBUF_INIT;
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
 		if (!show_others)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
-		fill_directory(dir, istate, &pathspec);
+		fill_directory(dir, repo->index, &pathspec);
 		if (show_others)
-			show_other_files(istate, dir);
+			show_other_files(repo->index, dir);
 		if (show_killed)
-			show_killed_files(istate, dir);
+			show_killed_files(repo->index, dir);
 	}
 	if (show_cached || show_stage) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
-			show_ce_entry(ce_stage(ce) ? tag_unmerged :
-				(ce_skip_worktree(ce) ? tag_skip_worktree : tag_cached), ce);
+			show_ce(repo, dir, ce, fullname.buf,
+				ce_stage(ce) ? tag_unmerged :
+				(ce_skip_worktree(ce) ? tag_skip_worktree :
+				 tag_cached));
 		}
 	}
 	if (show_deleted || show_modified) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
 			struct stat st;
 			int err;
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
 			if (ce_skip_worktree(ce))
 				continue;
-			err = lstat(ce->name, &st);
+			err = lstat(fullname.buf, &st);
 			if (show_deleted && err)
-				show_ce_entry(tag_removed, ce);
-			if (show_modified && ie_modified(istate, ce, &st, 0))
-				show_ce_entry(tag_modified, ce);
+				show_ce(repo, dir, ce, fullname.buf, tag_removed);
+			if (show_modified && ie_modified(repo->index, ce, &st, 0))
+				show_ce(repo, dir, ce, fullname.buf, tag_modified);
 		}
 	}
+
+	strbuf_release(&fullname);
 }
 
 /*
@@ -614,10 +579,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	prefix = cmd_prefix;
 	if (prefix)
 		prefix_len = strlen(prefix);
-	super_prefix = get_super_prefix();
 	git_config(git_default_config, NULL);
 
-	if (read_cache() < 0)
+	if (repo_read_index(the_repository))
 		die("index file corrupt");
 
 	argc = parse_options(argc, argv, prefix, builtin_ls_files_options,
@@ -650,8 +614,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	if (require_work_tree && !is_inside_work_tree())
 		setup_work_tree();
 
-	if (recurse_submodules)
-		compile_submodule_options(argv, &dir, show_tag);
+	if (recurse_submodules) {
+		repo_read_gitmodules(the_repository);
+	}
 
 	if (recurse_submodules &&
 	    (show_stage || show_deleted || show_others || show_unmerged ||
@@ -669,7 +634,10 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	/*
 	 * Find common prefix for all pathspec's
 	 * This is used as a performance optimization which unfortunately cannot
-	 * be done when recursing into submodules
+	 * be done when recursing into submodules because when a pathspec is
+	 * given which spans repository boundaries you can't simply remove the
+	 * submodule entry because the pathspec may match something inside the
+	 * submodule.
 	 */
 	if (recurse_submodules)
 		max_prefix = NULL;
@@ -677,7 +645,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		max_prefix = common_prefix(&pathspec);
 	max_prefix_len = get_common_prefix_len(max_prefix);
 
-	prune_index(&the_index, max_prefix, max_prefix_len);
+	prune_index(the_repository->index, max_prefix, max_prefix_len);
 
 	/* Treat unmatching pathspec elements as errors */
 	if (pathspec.nr && error_unmatch)
@@ -698,11 +666,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		 */
 		if (show_stage || show_unmerged)
 			die("ls-files --with-tree is incompatible with -s or -u");
-		overlay_tree_on_index(&the_index, with_tree, max_prefix);
+		overlay_tree_on_index(the_repository->index, with_tree, max_prefix);
 	}
-	show_files(&the_index, &dir);
+
+	show_files(the_repository, &dir);
+
 	if (show_resolve_undo)
-		show_ru_info(&the_index);
+		show_ru_info(the_repository->index);
 
 	if (ps_matched) {
 		int bad;
diff --git a/git.c b/git.c
index 594436e43..16c6e325d 100644
--- a/git.c
+++ b/git.c
@@ -442,7 +442,7 @@ static struct cmd_struct commands[] = {
 	{ "init-db", cmd_init_db },
 	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY },
 	{ "log", cmd_log, RUN_SETUP },
-	{ "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },
+	{ "ls-files", cmd_ls_files, RUN_SETUP },
 	{ "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY },
 	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
 	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY },
diff --git a/repository.c b/repository.c
index 98a851d20..9f899a34a 100644
--- a/repository.c
+++ b/repository.c
@@ -152,6 +152,8 @@ int repo_init(struct repository *repo, const char *gitdir)
 	if (verify_repo_format(&format, repo->commondir))
 		goto error;
 
+	repo_set_worktree(repo, gitdir);
+
 	return 0;
 
 error:
@@ -180,6 +182,8 @@ void repo_clear(struct repository *repo)
 	repo_clear_env(repo);
 	free(repo->worktree);
 	repo->worktree = NULL;
+	free(repo->submodule_prefix);
+	repo->submodule_prefix = NULL;
 
 	if (repo->config) {
 		git_configset_clear(repo->config);
diff --git a/repository.h b/repository.h
index d9f1f721b..eea9a88aa 100644
--- a/repository.h
+++ b/repository.h
@@ -14,6 +14,7 @@ struct repository {
 	char *graft_file;
 	char *namespace;
 	char *worktree;
+	char *submodule_prefix;
 
 	/* Subsystems */
 	/*
diff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh
index ebb956fd1..318b5bce7 100755
--- a/t/t3007-ls-files-recurse-submodules.sh
+++ b/t/t3007-ls-files-recurse-submodules.sh
@@ -135,6 +135,45 @@ test_expect_success '--recurse-submodules and pathspecs setup' '
 	test_cmp expect actual
 '
 
+test_expect_success 'inactive submodule' '
+	test_when_finished "git config --bool submodule.submodule.active true" &&
+	test_when_finished "git -C submodule config --bool submodule.subsub.active true" &&
+	git config --bool submodule.submodule.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual &&
+
+	git config --bool submodule.submodule.active "true" &&
+	git -C submodule config --bool submodule.subsub.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule/.gitmodules
+	submodule/c
+	submodule/f.TXT
+	submodule/g.txt
+	submodule/subsub
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success '--recurse-submodules and pathspecs' '
 	cat >expect <<-\EOF &&
 	h.txt
-- 
2.13.1.508.gb3defc5cc-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 32/32] ls-files: use repository object
  2017-06-08 23:41   ` [PATCH v2 32/32] ls-files: use repository object Brandon Williams
@ 2017-06-09  0:08     ` Brandon Williams
  2017-06-10  5:59       ` Junio C Hamano
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-09  0:08 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

On 06/08, Brandon Williams wrote:
> Convert ls-files to use a repository struct and recurse submodules
> inprocess.
> 
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  builtin/ls-files.c                     | 198 ++++++++++++++-------------------
>  git.c                                  |   2 +-
>  repository.c                           |   4 +
>  repository.h                           |   1 +
>  t/t3007-ls-files-recurse-submodules.sh |  39 +++++++
>  5 files changed, 129 insertions(+), 115 deletions(-)
> 
> diff --git a/builtin/ls-files.c b/builtin/ls-files.c
> index 6a0302a28..503fed5a1 100644
> --- a/builtin/ls-files.c
> +++ b/builtin/ls-files.c
> @@ -5,7 +5,9 @@
>   *
>   * Copyright (C) Linus Torvalds, 2005
>   */
> +#define NO_THE_INDEX_COMPATIBILITY_MACROS
>  #include "cache.h"
> +#include "repository.h"
>  #include "config.h"
>  #include "quote.h"
>  #include "dir.h"
> @@ -32,10 +34,8 @@ static int line_terminator = '\n';
>  static int debug_mode;
>  static int show_eol;
>  static int recurse_submodules;
> -static struct argv_array submodule_options = ARGV_ARRAY_INIT;
>  
>  static const char *prefix;
> -static const char *super_prefix;
>  static int max_prefix_len;
>  static int prefix_len;
>  static struct pathspec pathspec;
> @@ -73,25 +73,12 @@ static void write_eolinfo(const struct index_state *istate,
>  
>  static void write_name(const char *name)
>  {
> -	/*
> -	 * Prepend the super_prefix to name to construct the full_name to be
> -	 * written.
> -	 */
> -	struct strbuf full_name = STRBUF_INIT;
> -	if (super_prefix) {
> -		strbuf_addstr(&full_name, super_prefix);
> -		strbuf_addstr(&full_name, name);
> -		name = full_name.buf;
> -	}
> -
>  	/*
>  	 * With "--full-name", prefix_len=0; this caller needs to pass
>  	 * an empty string in that case (a NULL is good for "").
>  	 */
>  	write_name_quoted_relative(name, prefix_len ? prefix : NULL,
>  				   stdout, line_terminator);
> -
> -	strbuf_release(&full_name);
>  }
>  
>  static const char *get_tag(const struct cache_entry *ce, const char *tag)
> @@ -210,82 +197,41 @@ static void show_killed_files(const struct index_state *istate,
>  	}
>  }
>  
> -/*
> - * Compile an argv_array with all of the options supported by --recurse_submodules
> - */
> -static void compile_submodule_options(const char **argv,
> -				      const struct dir_struct *dir,
> -				      int show_tag)
> -{
> -	if (line_terminator == '\0')
> -		argv_array_push(&submodule_options, "-z");
> -	if (show_tag)
> -		argv_array_push(&submodule_options, "-t");
> -	if (show_valid_bit)
> -		argv_array_push(&submodule_options, "-v");
> -	if (show_cached)
> -		argv_array_push(&submodule_options, "--cached");
> -	if (show_eol)
> -		argv_array_push(&submodule_options, "--eol");
> -	if (debug_mode)
> -		argv_array_push(&submodule_options, "--debug");
> -
> -	/* Add Pathspecs */
> -	argv_array_push(&submodule_options, "--");
> -	for (; *argv; argv++)
> -		argv_array_push(&submodule_options, *argv);
> -}
> +static void show_files(struct repository *repo, struct dir_struct *dir);
>  
> -/**
> - * Recursively call ls-files on a submodule
> - */
> -static void show_gitlink(const struct cache_entry *ce)
> +static void show_submodule(const struct repository *superproject,
> +			   struct dir_struct *dir, const char *path)
>  {
> -	struct child_process cp = CHILD_PROCESS_INIT;
> -	int status;
> -	char *dir;
> -
> -	prepare_submodule_repo_env(&cp.env_array);
> -	argv_array_push(&cp.env_array, GIT_DIR_ENVIRONMENT);
> -
> -	if (prefix_len)
> -		argv_array_pushf(&cp.env_array, "%s=%s",
> -				 GIT_TOPLEVEL_PREFIX_ENVIRONMENT,
> -				 prefix);
> -	argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
> -			 super_prefix ? super_prefix : "",
> -			 ce->name);
> -	argv_array_push(&cp.args, "ls-files");
> -	argv_array_push(&cp.args, "--recurse-submodules");
> -
> -	/* add supported options */
> -	argv_array_pushv(&cp.args, submodule_options.argv);
> -
> -	cp.git_cmd = 1;
> -	dir = mkpathdup("%s/%s", get_git_work_tree(), ce->name);
> -	cp.dir = dir;
> -	status = run_command(&cp);
> -	free(dir);
> -	if (status)
> -		exit(status);
> +	struct repository submodule;
> +	char *gitdir = mkpathdup("%s/%s", superproject->worktree, path);
> +	repo_init(&submodule, gitdir);
> +
> +	repo_read_index(&submodule);
> +	repo_read_gitmodules(&submodule);
> +
> +	if (superproject->submodule_prefix)
> +		submodule.submodule_prefix = xstrfmt("%s%s/", superproject->submodule_prefix, path);
> +	else
> +		submodule.submodule_prefix = xstrfmt("%s/", path);
> +	show_files(&submodule, dir);
> +
> +	repo_clear(&submodule);
> +	free(gitdir);
>  }
>  
> -static void show_ce_entry(const char *tag, const struct cache_entry *ce)
> +static void show_ce(struct repository *repo, struct dir_struct *dir,
> +		    const struct cache_entry *ce, const char *fullname,
> +		    const char *tag)
>  {
> -	struct strbuf name = STRBUF_INIT;
> -	int len = max_prefix_len;
> -	if (super_prefix)
> -		strbuf_addstr(&name, super_prefix);
> -	strbuf_addstr(&name, ce->name);
> -
> -	if (len > ce_namelen(ce))
> +	if (max_prefix_len > strlen(fullname))
>  		die("git ls-files: internal error - cache entry not superset of prefix");
>  
>  	if (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&
> -	    submodule_path_match(&pathspec, name.buf, ps_matched)) {
> -		show_gitlink(ce);
> -	} else if (match_pathspec(&pathspec, name.buf, name.len,
> -				  len, ps_matched,
> +	    submodule_path_match(&pathspec, fullname, ps_matched) &&
> +	    is_submodule_active(repo, ce->name)) {
> +		show_submodule(repo, dir, ce->name);
> +	} else if (match_pathspec(&pathspec, fullname, strlen(fullname),
> +				  max_prefix_len, ps_matched,
>  				  S_ISDIR(ce->ce_mode) ||
>  				  S_ISGITLINK(ce->ce_mode))) {
>  		tag = get_tag(ce, tag);
> @@ -299,12 +245,11 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)
>  			       find_unique_abbrev(ce->oid.hash, abbrev),
>  			       ce_stage(ce));
>  		}
> -		write_eolinfo(&the_index, ce, ce->name);
> -		write_name(ce->name);
> +
> +		write_eolinfo(repo->index, ce, fullname);
> +		write_name(fullname);
>  		print_debug(ce);
>  	}
> -
> -	strbuf_release(&name);
>  }
>  
>  static void show_ru_info(const struct index_state *istate)
> @@ -337,59 +282,79 @@ static void show_ru_info(const struct index_state *istate)
>  }
>  
>  static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
> -		       const struct cache_entry *ce)
> +		       const char *fullname, const struct cache_entry *ce)
>  {
>  	int dtype = ce_to_dtype(ce);
> -	return is_excluded(dir, istate, ce->name, &dtype);
> +	return is_excluded(dir, istate, fullname, &dtype);
> +}
> +
> +static void construct_fullname(struct strbuf *out, const struct repository *repo,
> +			       const struct cache_entry *ce)
> +{
> +	strbuf_reset(out);
> +	if (repo->submodule_prefix)
> +		strbuf_addstr(out, repo->submodule_prefix);
> +	strbuf_addstr(out, ce->name);
>  }
>  
> -static void show_files(struct index_state *istate, struct dir_struct *dir)
> +static void show_files(struct repository *repo, struct dir_struct *dir)
>  {
>  	int i;
> +	struct strbuf fullname = STRBUF_INIT;
>  
>  	/* For cached/deleted files we don't need to even do the readdir */
>  	if (show_others || show_killed) {
>  		if (!show_others)
>  			dir->flags |= DIR_COLLECT_KILLED_ONLY;
> -		fill_directory(dir, istate, &pathspec);
> +		fill_directory(dir, repo->index, &pathspec);
>  		if (show_others)
> -			show_other_files(istate, dir);
> +			show_other_files(repo->index, dir);
>  		if (show_killed)
> -			show_killed_files(istate, dir);
> +			show_killed_files(repo->index, dir);
>  	}
>  	if (show_cached || show_stage) {
> -		for (i = 0; i < istate->cache_nr; i++) {
> -			const struct cache_entry *ce = istate->cache[i];
> +		for (i = 0; i < repo->index->cache_nr; i++) {
> +			const struct cache_entry *ce = repo->index->cache[i];
> +
> +			construct_fullname(&fullname, repo, ce);
> +
>  			if ((dir->flags & DIR_SHOW_IGNORED) &&
> -			    !ce_excluded(dir, istate, ce))
> +			    !ce_excluded(dir, repo->index, fullname.buf, ce))
>  				continue;
>  			if (show_unmerged && !ce_stage(ce))
>  				continue;
>  			if (ce->ce_flags & CE_UPDATE)
>  				continue;
> -			show_ce_entry(ce_stage(ce) ? tag_unmerged :
> -				(ce_skip_worktree(ce) ? tag_skip_worktree : tag_cached), ce);
> +			show_ce(repo, dir, ce, fullname.buf,
> +				ce_stage(ce) ? tag_unmerged :
> +				(ce_skip_worktree(ce) ? tag_skip_worktree :
> +				 tag_cached));
>  		}
>  	}
>  	if (show_deleted || show_modified) {
> -		for (i = 0; i < istate->cache_nr; i++) {
> -			const struct cache_entry *ce = istate->cache[i];
> +		for (i = 0; i < repo->index->cache_nr; i++) {
> +			const struct cache_entry *ce = repo->index->cache[i];
>  			struct stat st;
>  			int err;
> +
> +			construct_fullname(&fullname, repo, ce);
> +
>  			if ((dir->flags & DIR_SHOW_IGNORED) &&
> -			    !ce_excluded(dir, istate, ce))
> +			    !ce_excluded(dir, repo->index, fullname.buf, ce))
>  				continue;
>  			if (ce->ce_flags & CE_UPDATE)
>  				continue;
>  			if (ce_skip_worktree(ce))
>  				continue;
> -			err = lstat(ce->name, &st);
> +			err = lstat(fullname.buf, &st);
>  			if (show_deleted && err)
> -				show_ce_entry(tag_removed, ce);
> -			if (show_modified && ie_modified(istate, ce, &st, 0))
> -				show_ce_entry(tag_modified, ce);
> +				show_ce(repo, dir, ce, fullname.buf, tag_removed);
> +			if (show_modified && ie_modified(repo->index, ce, &st, 0))
> +				show_ce(repo, dir, ce, fullname.buf, tag_modified);
>  		}
>  	}
> +
> +	strbuf_release(&fullname);
>  }
>  
>  /*
> @@ -614,10 +579,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
>  	prefix = cmd_prefix;
>  	if (prefix)
>  		prefix_len = strlen(prefix);
> -	super_prefix = get_super_prefix();
>  	git_config(git_default_config, NULL);
>  
> -	if (read_cache() < 0)
> +	if (repo_read_index(the_repository))

oops, looks like this should have been:

    if (repo_read_index(the_repository) < 0)

>  		die("index file corrupt");
>  
>  	argc = parse_options(argc, argv, prefix, builtin_ls_files_options,
> @@ -650,8 +614,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
>  	if (require_work_tree && !is_inside_work_tree())
>  		setup_work_tree();
>  
> -	if (recurse_submodules)
> -		compile_submodule_options(argv, &dir, show_tag);
> +	if (recurse_submodules) {
> +		repo_read_gitmodules(the_repository);
> +	}
>  
>  	if (recurse_submodules &&
>  	    (show_stage || show_deleted || show_others || show_unmerged ||
> @@ -669,7 +634,10 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
>  	/*
>  	 * Find common prefix for all pathspec's
>  	 * This is used as a performance optimization which unfortunately cannot
> -	 * be done when recursing into submodules
> +	 * be done when recursing into submodules because when a pathspec is
> +	 * given which spans repository boundaries you can't simply remove the
> +	 * submodule entry because the pathspec may match something inside the
> +	 * submodule.
>  	 */
>  	if (recurse_submodules)
>  		max_prefix = NULL;
> @@ -677,7 +645,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
>  		max_prefix = common_prefix(&pathspec);
>  	max_prefix_len = get_common_prefix_len(max_prefix);
>  
> -	prune_index(&the_index, max_prefix, max_prefix_len);
> +	prune_index(the_repository->index, max_prefix, max_prefix_len);
>  
>  	/* Treat unmatching pathspec elements as errors */
>  	if (pathspec.nr && error_unmatch)
> @@ -698,11 +666,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
>  		 */
>  		if (show_stage || show_unmerged)
>  			die("ls-files --with-tree is incompatible with -s or -u");
> -		overlay_tree_on_index(&the_index, with_tree, max_prefix);
> +		overlay_tree_on_index(the_repository->index, with_tree, max_prefix);
>  	}
> -	show_files(&the_index, &dir);
> +
> +	show_files(the_repository, &dir);
> +
>  	if (show_resolve_undo)
> -		show_ru_info(&the_index);
> +		show_ru_info(the_repository->index);
>  
>  	if (ps_matched) {
>  		int bad;
> diff --git a/git.c b/git.c
> index 594436e43..16c6e325d 100644
> --- a/git.c
> +++ b/git.c
> @@ -442,7 +442,7 @@ static struct cmd_struct commands[] = {
>  	{ "init-db", cmd_init_db },
>  	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY },
>  	{ "log", cmd_log, RUN_SETUP },
> -	{ "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },
> +	{ "ls-files", cmd_ls_files, RUN_SETUP },
>  	{ "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY },
>  	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
>  	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY },
> diff --git a/repository.c b/repository.c
> index 98a851d20..9f899a34a 100644
> --- a/repository.c
> +++ b/repository.c
> @@ -152,6 +152,8 @@ int repo_init(struct repository *repo, const char *gitdir)
>  	if (verify_repo_format(&format, repo->commondir))
>  		goto error;
>  
> +	repo_set_worktree(repo, gitdir);
> +
>  	return 0;
>  
>  error:
> @@ -180,6 +182,8 @@ void repo_clear(struct repository *repo)
>  	repo_clear_env(repo);
>  	free(repo->worktree);
>  	repo->worktree = NULL;
> +	free(repo->submodule_prefix);
> +	repo->submodule_prefix = NULL;
>  
>  	if (repo->config) {
>  		git_configset_clear(repo->config);
> diff --git a/repository.h b/repository.h
> index d9f1f721b..eea9a88aa 100644
> --- a/repository.h
> +++ b/repository.h
> @@ -14,6 +14,7 @@ struct repository {
>  	char *graft_file;
>  	char *namespace;
>  	char *worktree;
> +	char *submodule_prefix;
>  
>  	/* Subsystems */
>  	/*
> diff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh
> index ebb956fd1..318b5bce7 100755
> --- a/t/t3007-ls-files-recurse-submodules.sh
> +++ b/t/t3007-ls-files-recurse-submodules.sh
> @@ -135,6 +135,45 @@ test_expect_success '--recurse-submodules and pathspecs setup' '
>  	test_cmp expect actual
>  '
>  
> +test_expect_success 'inactive submodule' '
> +	test_when_finished "git config --bool submodule.submodule.active true" &&
> +	test_when_finished "git -C submodule config --bool submodule.subsub.active true" &&
> +	git config --bool submodule.submodule.active "false" &&
> +
> +	cat >expect <<-\EOF &&
> +	.gitmodules
> +	a
> +	b/b
> +	h.txt
> +	sib/file
> +	sub/file
> +	submodule
> +	EOF
> +
> +	git ls-files --recurse-submodules >actual &&
> +	test_cmp expect actual &&
> +
> +	git config --bool submodule.submodule.active "true" &&
> +	git -C submodule config --bool submodule.subsub.active "false" &&
> +
> +	cat >expect <<-\EOF &&
> +	.gitmodules
> +	a
> +	b/b
> +	h.txt
> +	sib/file
> +	sub/file
> +	submodule/.gitmodules
> +	submodule/c
> +	submodule/f.TXT
> +	submodule/g.txt
> +	submodule/subsub
> +	EOF
> +
> +	git ls-files --recurse-submodules >actual &&
> +	test_cmp expect actual
> +'
> +
>  test_expect_success '--recurse-submodules and pathspecs' '
>  	cat >expect <<-\EOF &&
>  	h.txt
> -- 
> 2.13.1.508.gb3defc5cc-goog
> 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH 0/2] Add a freez() wrapper
  2017-06-08 23:40   ` [PATCH v2 07/32] repository: introduce the repository object Brandon Williams
@ 2017-06-09  8:53     ` Ævar Arnfjörð Bjarmason
  2017-06-09 14:53       ` Brandon Williams
                         ` (3 more replies)
  2017-06-09  8:53     ` [PATCH 1/2] git-compat-util: add a freez() wrapper around free(x); x = NULL Ævar Arnfjörð Bjarmason
  2017-06-09  8:53     ` [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper Ævar Arnfjörð Bjarmason
  2 siblings, 4 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-09  8:53 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams,
	Ævar Arnfjörð Bjarmason

On Fri, Jun 9, 2017 at 1:40 AM, Brandon Williams <bmwill@google.com> wrote:
> Introduce the repository object 'struct repository' which can be used to
> hold all state pertaining to a git repository.
> [...]
> +static void repo_clear_env(struct repository *repo)
> +{
> +       free(repo->gitdir);
> +       repo->gitdir = NULL;
> +       free(repo->commondir);
> +       repo->commondir = NULL;
> +       free(repo->objectdir);
> +       repo->objectdir = NULL;
> +       free(repo->index_file);
> +       repo->index_file = NULL;
> +       free(repo->graft_file);
> +       repo->graft_file = NULL;
> +       free(repo->namespace);
> +       repo->namespace = NULL;
> +}

When seeing this I wondered if it couldn't be less noisy as:

> +       freez(repo->gitdir);
> +       freez(repo->commondir);
> +       freez(repo->objectdir);
> +       freez(repo->index_file);
> +       freez(repo->graft_file);
> +       freez(repo->namespace);

It's not a problem with your patch, but something I thought would be
nice to have in general, so here's a patch series to implement that.

Ævar Arnfjörð Bjarmason (2):
  git-compat-util: add a freez() wrapper around free(x); x = NULL
  *.[ch] refactoring: make use of the freez() wrapper

 alias.c                  |  6 ++----
 apply.c                  |  3 +--
 attr.c                   |  6 ++----
 blame.c                  |  3 +--
 branch.c                 |  3 +--
 builtin/am.c             | 18 +++++-------------
 builtin/clean.c          |  6 ++----
 builtin/config.c         |  6 ++----
 builtin/index-pack.c     |  6 ++----
 builtin/pack-objects.c   | 12 ++++--------
 builtin/unpack-objects.c |  3 +--
 builtin/worktree.c       |  6 ++----
 commit-slab.h            |  3 +--
 commit.c                 |  3 +--
 config.c                 |  3 +--
 credential.c             |  9 +++------
 diff-lib.c               |  3 +--
 diff.c                   |  6 ++----
 diffcore-rename.c        |  6 ++----
 dir.c                    |  9 +++------
 fast-import.c            |  6 ++----
 git-compat-util.h        |  1 +
 gpg-interface.c          | 15 +++++----------
 grep.c                   | 12 ++++--------
 help.c                   |  3 +--
 http-push.c              | 24 ++++++++----------------
 http.c                   | 15 +++++----------
 imap-send.c              |  3 +--
 line-log.c               |  6 ++----
 ll-merge.c               |  3 +--
 mailinfo.c               |  3 +--
 object.c                 |  3 +--
 pathspec.c               |  3 +--
 prio-queue.c             |  3 +--
 read-cache.c             |  6 ++----
 ref-filter.c             |  3 +--
 refs/files-backend.c     |  3 +--
 refs/ref-cache.c         |  3 +--
 remote-testsvn.c         |  3 +--
 rerere.c                 |  3 +--
 sequencer.c              |  3 +--
 sha1-array.c             |  3 +--
 sha1_file.c              |  3 +--
 split-index.c            |  3 +--
 transport-helper.c       | 27 +++++++++------------------
 transport.c              |  3 +--
 tree-diff.c              |  6 ++----
 tree-walk.c              |  3 +--
 tree.c                   |  3 +--
 49 files changed, 98 insertions(+), 197 deletions(-)

-- 
2.13.0.506.g27d5fe0cd


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH 1/2] git-compat-util: add a freez() wrapper around free(x); x = NULL
  2017-06-08 23:40   ` [PATCH v2 07/32] repository: introduce the repository object Brandon Williams
  2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
@ 2017-06-09  8:53     ` Ævar Arnfjörð Bjarmason
  2017-06-09 12:04       ` Christian Couder
  2017-06-09  8:53     ` [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper Ævar Arnfjörð Bjarmason
  2 siblings, 1 reply; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-09  8:53 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams,
	Ævar Arnfjörð Bjarmason

Add a freez() wrapper for the common pattern of freeing a pointer and
assigning NULL to it right afterwards.

The implementation is the same as the (currently unused) XDL_PTRFREE
macro in xdiff/xmacros.h added in commit 3443546f6e ("Use a *real*
built-in diff generator", 2006-03-24).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-compat-util.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index 4b7dcf21ad..21e3710755 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -787,6 +787,7 @@ extern char *xstrdup(const char *str);
 extern void *xmalloc(size_t size);
 extern void *xmallocz(size_t size);
 extern void *xmallocz_gently(size_t size);
+#define freez(p) do { if (p) { free(p); (p) = NULL; } } while (0)
 extern void *xmemdupz(const void *data, size_t len);
 extern char *xstrndup(const char *str, size_t len);
 extern void *xrealloc(void *ptr, size_t size);
-- 
2.13.0.506.g27d5fe0cd


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper
  2017-06-08 23:40   ` [PATCH v2 07/32] repository: introduce the repository object Brandon Williams
  2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
  2017-06-09  8:53     ` [PATCH 1/2] git-compat-util: add a freez() wrapper around free(x); x = NULL Ævar Arnfjörð Bjarmason
@ 2017-06-09  8:53     ` Ævar Arnfjörð Bjarmason
  2017-06-09 10:12       ` Martin Ågren
  2 siblings, 1 reply; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-09  8:53 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams,
	Ævar Arnfjörð Bjarmason

Replace occurrences of `free(p); p = NULL` with `freez(p)`. This
introduces no functional changes, but cuts the number of lines spent
on this cleanup in half.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 alias.c                  |  6 ++----
 apply.c                  |  3 +--
 attr.c                   |  6 ++----
 blame.c                  |  3 +--
 branch.c                 |  3 +--
 builtin/am.c             | 18 +++++-------------
 builtin/clean.c          |  6 ++----
 builtin/config.c         |  6 ++----
 builtin/index-pack.c     |  6 ++----
 builtin/pack-objects.c   | 12 ++++--------
 builtin/unpack-objects.c |  3 +--
 builtin/worktree.c       |  6 ++----
 commit-slab.h            |  3 +--
 commit.c                 |  3 +--
 config.c                 |  3 +--
 credential.c             |  9 +++------
 diff-lib.c               |  3 +--
 diff.c                   |  6 ++----
 diffcore-rename.c        |  6 ++----
 dir.c                    |  9 +++------
 fast-import.c            |  6 ++----
 gpg-interface.c          | 15 +++++----------
 grep.c                   | 12 ++++--------
 help.c                   |  3 +--
 http-push.c              | 24 ++++++++----------------
 http.c                   | 15 +++++----------
 imap-send.c              |  3 +--
 line-log.c               |  6 ++----
 ll-merge.c               |  3 +--
 mailinfo.c               |  3 +--
 object.c                 |  3 +--
 pathspec.c               |  3 +--
 prio-queue.c             |  3 +--
 read-cache.c             |  6 ++----
 ref-filter.c             |  3 +--
 refs/files-backend.c     |  3 +--
 refs/ref-cache.c         |  3 +--
 remote-testsvn.c         |  3 +--
 rerere.c                 |  3 +--
 sequencer.c              |  3 +--
 sha1-array.c             |  3 +--
 sha1_file.c              |  3 +--
 split-index.c            |  3 +--
 transport-helper.c       | 27 +++++++++------------------
 transport.c              |  3 +--
 tree-diff.c              |  6 ++----
 tree-walk.c              |  3 +--
 tree.c                   |  3 +--
 48 files changed, 97 insertions(+), 197 deletions(-)

diff --git a/alias.c b/alias.c
index 3b90397a99..99cd547b6b 100644
--- a/alias.c
+++ b/alias.c
@@ -47,8 +47,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 				src++;
 				c = cmdline[src];
 				if (!c) {
-					free(*argv);
-					*argv = NULL;
+					freez(*argv);
 					return -SPLIT_CMDLINE_BAD_ENDING;
 				}
 			}
@@ -60,8 +59,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 	cmdline[dst] = 0;
 
 	if (quoted) {
-		free(*argv);
-		*argv = NULL;
+		freez(*argv);
 		return -SPLIT_CMDLINE_UNCLOSED_QUOTE;
 	}
 
diff --git a/apply.c b/apply.c
index c49cef0637..f987e1e961 100644
--- a/apply.c
+++ b/apply.c
@@ -3705,8 +3705,7 @@ static int check_preimage(struct apply_state *state,
  is_new:
 	patch->is_new = 1;
 	patch->is_delete = 0;
-	free(patch->old_name);
-	patch->old_name = NULL;
+	freez(patch->old_name);
 	return 0;
 }
 
diff --git a/attr.c b/attr.c
index 7e2134471c..cd92a02e98 100644
--- a/attr.c
+++ b/attr.c
@@ -638,13 +638,11 @@ void attr_check_reset(struct attr_check *check)
 
 void attr_check_clear(struct attr_check *check)
 {
-	free(check->items);
-	check->items = NULL;
+	freez(check->items);
 	check->alloc = 0;
 	check->nr = 0;
 
-	free(check->all_attrs);
-	check->all_attrs = NULL;
+	freez(check->all_attrs);
 	check->all_attrs_nr = 0;
 
 	drop_attr_stack(&check->stack);
diff --git a/blame.c b/blame.c
index 843c845cba..dc1688889c 100644
--- a/blame.c
+++ b/blame.c
@@ -314,8 +314,7 @@ static void fill_origin_blob(struct diff_options *opt,
 static void drop_origin_blob(struct blame_origin *o)
 {
 	if (o->file.ptr) {
-		free(o->file.ptr);
-		o->file.ptr = NULL;
+		freez(o->file.ptr);
 	}
 }
 
diff --git a/branch.c b/branch.c
index 985316eb76..ef2ae0b49e 100644
--- a/branch.c
+++ b/branch.c
@@ -24,8 +24,7 @@ static int find_tracked_branch(struct remote *remote, void *priv)
 		} else {
 			free(tracking->spec.src);
 			if (tracking->src) {
-				free(tracking->src);
-				tracking->src = NULL;
+				freez(tracking->src);
 			}
 		}
 		tracking->spec.src = NULL;
diff --git a/builtin/am.c b/builtin/am.c
index 5ee146bfb3..f5441cc990 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -483,8 +483,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
 	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
 
 	if (!ret) {
-		free(state->msg);
-		state->msg = NULL;
+		freez(state->msg);
 		if (read_commit_msg(state) < 0)
 			die(_("'%s' was deleted by the applypatch-msg hook"),
 				am_path(state, "final-commit"));
@@ -1073,17 +1072,10 @@ static void am_next(struct am_state *state)
 {
 	struct object_id head;
 
-	free(state->author_name);
-	state->author_name = NULL;
-
-	free(state->author_email);
-	state->author_email = NULL;
-
-	free(state->author_date);
-	state->author_date = NULL;
-
-	free(state->msg);
-	state->msg = NULL;
+	freez(state->author_name);
+	freez(state->author_email);
+	freez(state->author_date);
+	freez(state->msg);
 	state->msg_len = 0;
 
 	unlink(am_path(state, "author-script"));
diff --git a/builtin/clean.c b/builtin/clean.c
index 142bf668cf..cdf7b0911e 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -837,8 +837,7 @@ static void interactive_main_loop(void)
 			int ret;
 			ret = menus[*chosen].fn();
 			if (ret != MENU_RETURN_NO_LOOP) {
-				free(chosen);
-				chosen = NULL;
+				freez(chosen);
 				if (!del_list.nr) {
 					clean_print_color(CLEAN_COLOR_ERROR);
 					printf_ln(_("No more files to clean, exiting."));
@@ -851,8 +850,7 @@ static void interactive_main_loop(void)
 			quit_cmd();
 		}
 
-		free(chosen);
-		chosen = NULL;
+		freez(chosen);
 		break;
 	}
 }
diff --git a/builtin/config.c b/builtin/config.c
index 7f6c25d4d9..e9adac9622 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -214,8 +214,7 @@ static int get_value(const char *key_, const char *regex_)
 		key_regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(key_regexp, key, REG_EXTENDED)) {
 			error("invalid key pattern: %s", key_);
-			free(key_regexp);
-			key_regexp = NULL;
+			freez(key_regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
@@ -235,8 +234,7 @@ static int get_value(const char *key_, const char *regex_)
 		regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(regexp, regex_, REG_EXTENDED)) {
 			error("invalid pattern: %s", regex_);
-			free(regexp);
-			regexp = NULL;
+			freez(regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 04b9dcaf0f..4165e99484 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -388,8 +388,7 @@ static struct base_data *alloc_base_data(void)
 static void free_base_data(struct base_data *c)
 {
 	if (c->data) {
-		free(c->data);
-		c->data = NULL;
+		freez(c->data);
 		get_thread_data()->base_cache_used -= c->size;
 	}
 }
@@ -605,8 +604,7 @@ static void *unpack_data(struct object_entry *obj,
 	git_inflate_end(&stream);
 	free(inbuf);
 	if (consume) {
-		free(data);
-		data = NULL;
+		freez(data);
 	}
 	return data;
 }
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f672225def..fb43c1b101 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -264,8 +264,7 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 		 * make sure no cached delta data remains from a
 		 * previous attempt before a pack split occurred.
 		 */
-		free(entry->delta_data);
-		entry->delta_data = NULL;
+		freez(entry->delta_data);
 		entry->z_delta_size = 0;
 	} else if (entry->delta_data) {
 		size = entry->delta_size;
@@ -1375,12 +1374,10 @@ static void cleanup_preferred_base(void)
 		if (!pbase_tree_cache[i])
 			continue;
 		free(pbase_tree_cache[i]->tree_data);
-		free(pbase_tree_cache[i]);
-		pbase_tree_cache[i] = NULL;
+		freez(pbase_tree_cache[i]);
 	}
 
-	free(done_pbase_paths);
-	done_pbase_paths = NULL;
+	freez(done_pbase_paths);
 	done_pbase_paths_num = done_pbase_paths_alloc = 0;
 }
 
@@ -1970,8 +1967,7 @@ static unsigned long free_unpacked(struct unpacked *n)
 	n->index = NULL;
 	if (n->data) {
 		freed_mem += n->entry->size;
-		free(n->data);
-		n->data = NULL;
+		freez(n->data);
 	}
 	n->entry = NULL;
 	n->depth = 0;
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 8bc9997767..236ee6ebfc 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -112,8 +112,7 @@ static void *get_data(unsigned long size)
 			break;
 		if (ret != Z_OK) {
 			error("inflate returned %d", ret);
-			free(buf);
-			buf = NULL;
+			freez(buf);
 			if (!recover)
 				exit(1);
 			has_errors = 1;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 793306ea51..dffed4666e 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -299,10 +299,8 @@ static int add_worktree(const char *path, const char *refname,
 	}
 
 	is_junk = 0;
-	free(junk_work_tree);
-	free(junk_git_dir);
-	junk_work_tree = NULL;
-	junk_git_dir = NULL;
+	freez(junk_work_tree);
+	freez(junk_git_dir);
 
 done:
 	if (ret || !opts->keep_locked) {
diff --git a/commit-slab.h b/commit-slab.h
index 42d16dcded..ffa2a30f53 100644
--- a/commit-slab.h
+++ b/commit-slab.h
@@ -82,8 +82,7 @@ static MAYBE_UNUSED void clear_ ##slabname(struct slabname *s)		\
 	for (i = 0; i < s->slab_count; i++)				\
 		free(s->slab[i]);					\
 	s->slab_count = 0;						\
-	free(s->slab);							\
-	s->slab = NULL;							\
+	freez(s->slab);							\
 }									\
 									\
 static MAYBE_UNUSED elemtype *slabname## _at_peek(struct slabname *s,	\
diff --git a/commit.c b/commit.c
index 713f09feb0..851f1f3516 100644
--- a/commit.c
+++ b/commit.c
@@ -287,8 +287,7 @@ void free_commit_buffer(struct commit *commit)
 {
 	struct commit_buffer *v = buffer_slab_peek(&buffer_slab, commit);
 	if (v) {
-		free(v->buffer);
-		v->buffer = NULL;
+		freez(v->buffer);
 		v->size = 0;
 	}
 }
diff --git a/config.c b/config.c
index 146cb3452a..9ab9260e0f 100644
--- a/config.c
+++ b/config.c
@@ -395,8 +395,7 @@ static int git_config_parse_key_1(const char *key, char **store_key, int *basele
 
 out_free_ret_1:
 	if (store_key) {
-		free(*store_key);
-		*store_key = NULL;
+		freez(*store_key);
 	}
 	return -CONFIG_INVALID_KEY;
 }
diff --git a/credential.c b/credential.c
index aa996669fc..120fb996cb 100644
--- a/credential.c
+++ b/credential.c
@@ -93,8 +93,7 @@ static void credential_apply_config(struct credential *c)
 	c->configured = 1;
 
 	if (!c->use_http_path && proto_is_http(c->protocol)) {
-		free(c->path);
-		c->path = NULL;
+		freez(c->path);
 	}
 }
 
@@ -314,10 +313,8 @@ void credential_reject(struct credential *c)
 	for (i = 0; i < c->helpers.nr; i++)
 		credential_do(c, c->helpers.items[i].string, "erase");
 
-	free(c->username);
-	c->username = NULL;
-	free(c->password);
-	c->password = NULL;
+	freez(c->username);
+	freez(c->password);
 	c->approved = 0;
 }
 
diff --git a/diff-lib.c b/diff-lib.c
index 2982bf055a..d275c2469f 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -179,8 +179,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)
 				free(dpath);
 				continue;
 			}
-			free(dpath);
-			dpath = NULL;
+			freez(dpath);
 
 			/*
 			 * Show the diff for the 'ce' if we found the one
diff --git a/diff.c b/diff.c
index 5275c4b780..8ae4a6325c 100644
--- a/diff.c
+++ b/diff.c
@@ -1218,8 +1218,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)
 			regfree(ecbdata->diff_words->word_regex);
 			free(ecbdata->diff_words->word_regex);
 		}
-		free(ecbdata->diff_words);
-		ecbdata->diff_words = NULL;
+		freez(ecbdata->diff_words);
 	}
 }
 
@@ -2951,8 +2950,7 @@ void diff_free_filespec_blob(struct diff_filespec *s)
 void diff_free_filespec_data(struct diff_filespec *s)
 {
 	diff_free_filespec_blob(s);
-	free(s->cnt_data);
-	s->cnt_data = NULL;
+	freez(s->cnt_data);
 }
 
 static void prep_temp_blob(const char *path, struct diff_tempfile *temp,
diff --git a/diffcore-rename.c b/diffcore-rename.c
index f7444c86bd..99aab63c85 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -667,11 +667,9 @@ void diffcore_rename(struct diff_options *options)
 	for (i = 0; i < rename_dst_nr; i++)
 		free_filespec(rename_dst[i].two);
 
-	free(rename_dst);
-	rename_dst = NULL;
+	freez(rename_dst);
 	rename_dst_nr = rename_dst_alloc = 0;
-	free(rename_src);
-	rename_src = NULL;
+	freez(rename_src);
 	rename_src_nr = rename_src_alloc = 0;
 	return;
 }
diff --git a/dir.c b/dir.c
index 9efcf1eab6..441f98095a 100644
--- a/dir.c
+++ b/dir.c
@@ -2117,8 +2117,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		for (i = j = 0; j < dir->nr; j++) {
 			if (i &&
 			    check_dir_entry_contains(dir->entries[i - 1], dir->entries[j])) {
-				free(dir->entries[j]);
-				dir->entries[j] = NULL;
+				freez(dir->entries[j]);
 			} else {
 				dir->entries[i++] = dir->entries[j];
 			}
@@ -2144,8 +2143,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		     dir->untracked->dir_invalidated))
 			istate->cache_changed |= UNTRACKED_CHANGED;
 		if (dir->untracked != istate->untracked) {
-			free(dir->untracked);
-			dir->untracked = NULL;
+			freez(dir->untracked);
 		}
 	}
 	return dir->nr;
@@ -2488,8 +2486,7 @@ void write_untracked_extension(struct strbuf *out, struct untracked_cache *untra
 	strbuf_addbuf(out, &untracked->ident);
 
 	strbuf_add(out, ouc, ouc_size(len));
-	free(ouc);
-	ouc = NULL;
+	freez(ouc);
 
 	if (!untracked->root) {
 		varint_len = encode_varint(0, varbuf);
diff --git a/fast-import.c b/fast-import.c
index e69d219682..9d8f91eebd 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1063,8 +1063,7 @@ static void end_packfile(void)
 		close(pack_data->pack_fd);
 		unlink_or_warn(pack_data->pack_name);
 	}
-	free(pack_data);
-	pack_data = NULL;
+	freez(pack_data);
 	running = 0;
 
 	/* We can't carry a delta across packfiles. */
@@ -1149,8 +1148,7 @@ static int store_object(
 
 		/* We cannot carry a delta into the new pack. */
 		if (delta) {
-			free(delta);
-			delta = NULL;
+			freez(delta);
 
 			git_deflate_init(&s, pack_compression_level);
 			s.next_in = (void *)dat->buf;
diff --git a/gpg-interface.c b/gpg-interface.c
index e44cc27da1..0baca17293 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -13,16 +13,11 @@ static const char *gpg_program = "gpg";
 
 void signature_check_clear(struct signature_check *sigc)
 {
-	free(sigc->payload);
-	free(sigc->gpg_output);
-	free(sigc->gpg_status);
-	free(sigc->signer);
-	free(sigc->key);
-	sigc->payload = NULL;
-	sigc->gpg_output = NULL;
-	sigc->gpg_status = NULL;
-	sigc->signer = NULL;
-	sigc->key = NULL;
+	freez(sigc->payload);
+	freez(sigc->gpg_output);
+	freez(sigc->gpg_status);
+	freez(sigc->signer);
+	freez(sigc->key);
 }
 
 static struct {
diff --git a/grep.c b/grep.c
index d03d424e5c..c74c5a8270 100644
--- a/grep.c
+++ b/grep.c
@@ -1763,12 +1763,9 @@ void grep_source_init(struct grep_source *gs, enum grep_source_type type,
 
 void grep_source_clear(struct grep_source *gs)
 {
-	free(gs->name);
-	gs->name = NULL;
-	free(gs->path);
-	gs->path = NULL;
-	free(gs->identifier);
-	gs->identifier = NULL;
+	freez(gs->name);
+	freez(gs->path);
+	freez(gs->identifier);
 	grep_source_clear_data(gs);
 }
 
@@ -1778,8 +1775,7 @@ void grep_source_clear_data(struct grep_source *gs)
 	case GREP_SOURCE_FILE:
 	case GREP_SOURCE_SHA1:
 	case GREP_SOURCE_SUBMODULE:
-		free(gs->buf);
-		gs->buf = NULL;
+		freez(gs->buf);
 		gs->size = 0;
 		break;
 	case GREP_SOURCE_BUF:
diff --git a/help.c b/help.c
index db7f3d79a0..6783bb9b4b 100644
--- a/help.c
+++ b/help.c
@@ -267,9 +267,8 @@ static void add_cmd_list(struct cmdnames *cmds, struct cmdnames *old)
 
 	for (i = 0; i < old->cnt; i++)
 		cmds->names[cmds->cnt++] = old->names[i];
-	free(old->names);
+	freez(old->names);
 	old->cnt = 0;
-	old->names = NULL;
 }
 
 /* An empirically derived magic number */
diff --git a/http-push.c b/http-push.c
index 67c4d4b472..89ae33464f 100644
--- a/http-push.c
+++ b/http-push.c
@@ -291,8 +291,7 @@ static void start_mkcol(struct transfer_request *request)
 		request->state = RUN_MKCOL;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		freez(request->url);
 	}
 }
 #endif
@@ -409,8 +408,7 @@ static void start_put(struct transfer_request *request)
 		request->state = RUN_PUT;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		freez(request->url);
 	}
 }
 
@@ -432,8 +430,7 @@ static void start_move(struct transfer_request *request)
 		request->state = RUN_MOVE;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		freez(request->url);
 	}
 }
 
@@ -526,8 +523,7 @@ static void finish_request(struct transfer_request *request)
 
 	/* URL is reused for MOVE after PUT */
 	if (request->state != RUN_PUT) {
-		free(request->url);
-		request->url = NULL;
+		freez(request->url);
 	}
 
 	if (request->state == RUN_MKCOL) {
@@ -803,8 +799,7 @@ xml_start_tag(void *userData, const char *name, const char **atts)
 	}
 	xsnprintf(ctx->name + old_namelen, ctx->len - old_namelen, ".%s", c);
 
-	free(ctx->cdata);
-	ctx->cdata = NULL;
+	freez(ctx->cdata);
 
 	ctx->userFunc(ctx, 0);
 }
@@ -932,8 +927,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)
 		free(lock->token);
 		free(lock->owner);
 		free(url);
-		free(lock);
-		lock = NULL;
+		freez(lock);
 	} else {
 		lock->url = url;
 		lock->start_time = time(NULL);
@@ -1105,8 +1099,7 @@ static void handle_remote_ls_ctx(struct xml_ctx *ctx, int tag_closed)
 			ls->dentry_flags |= IS_DIR;
 		}
 	} else if (!strcmp(ctx->name, DAV_PROPFIND_RESP)) {
-		free(ls->dentry_name);
-		ls->dentry_name = NULL;
+		freez(ls->dentry_name);
 		ls->dentry_flags = 0;
 	}
 }
@@ -1547,8 +1540,7 @@ static void fetch_symref(const char *path, char **symref, struct object_id *oid)
 		    curl_errorstr);
 	free(url);
 
-	free(*symref);
-	*symref = NULL;
+	freez(*symref);
 	oidclr(oid);
 
 	if (buffer.len == 0)
diff --git a/http.c b/http.c
index d2e11ec6f0..28eeb0d5ba 100644
--- a/http.c
+++ b/http.c
@@ -1026,8 +1026,7 @@ void http_cleanup(void)
 
 	if (proxy_auth.password) {
 		memset(proxy_auth.password, 0, strlen(proxy_auth.password));
-		free(proxy_auth.password);
-		proxy_auth.password = NULL;
+		freez(proxy_auth.password);
 	}
 
 	free((void *)curl_proxyuserpwd);
@@ -1038,13 +1037,11 @@ void http_cleanup(void)
 
 	if (cert_auth.password != NULL) {
 		memset(cert_auth.password, 0, strlen(cert_auth.password));
-		free(cert_auth.password);
-		cert_auth.password = NULL;
+		freez(cert_auth.password);
 	}
 	ssl_cert_password_required = 0;
 
-	free(cached_accept_language);
-	cached_accept_language = NULL;
+	freez(cached_accept_language);
 }
 
 struct active_request_slot *get_active_slot(void)
@@ -1896,8 +1893,7 @@ static char *fetch_pack_index(unsigned char *sha1, const char *base_url)
 
 	if (http_get_file(url, tmp, NULL) != HTTP_OK) {
 		error("Unable to get pack index %s", url);
-		free(tmp);
-		tmp = NULL;
+		freez(tmp);
 	}
 
 	free(url);
@@ -2328,8 +2324,7 @@ void release_http_object_request(struct http_object_request *freq)
 		freq->localfile = -1;
 	}
 	if (freq->url != NULL) {
-		free(freq->url);
-		freq->url = NULL;
+		freez(freq->url);
 	}
 	if (freq->slot != NULL) {
 		freq->slot->callback_func = NULL;
diff --git a/imap-send.c b/imap-send.c
index 857591660f..a96facf530 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -776,8 +776,7 @@ static int get_cmd_result(struct imap_store *ctx, struct imap_cmd *tcmd)
 			       offsetof(struct imap_cmd, next));
 			if (cmdp->cb.data) {
 				n = socket_write(&imap->buf.sock, cmdp->cb.data, cmdp->cb.dlen);
-				free(cmdp->cb.data);
-				cmdp->cb.data = NULL;
+				freez(cmdp->cb.data);
 				if (n != (int)cmdp->cb.dlen)
 					return RESP_BAD;
 			} else if (cmdp->cb.cont) {
diff --git a/line-log.c b/line-log.c
index b9087814b8..bc7efb91da 100644
--- a/line-log.c
+++ b/line-log.c
@@ -34,9 +34,8 @@ void range_set_init(struct range_set *rs, size_t prealloc)
 
 void range_set_release(struct range_set *rs)
 {
-	free(rs->ranges);
+	freez(rs->ranges);
 	rs->alloc = rs->nr = 0;
-	rs->ranges = NULL;
 }
 
 /* dst must be uninitialized! */
@@ -610,8 +609,7 @@ parse_lines(struct commit *commit, const char *prefix, struct string_list *args)
 		line_log_data_insert(&ranges, full_name, begin, end);
 
 		free_filespec(spec);
-		free(ends);
-		ends = NULL;
+		freez(ends);
 	}
 
 	for (p = ranges; p; p = p->next)
diff --git a/ll-merge.c b/ll-merge.c
index ac0d4a5d78..279ac6319c 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -209,8 +209,7 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 	result->size = st.st_size;
 	result->ptr = xmallocz(result->size);
 	if (read_in_full(fd, result->ptr, result->size) != result->size) {
-		free(result->ptr);
-		result->ptr = NULL;
+		freez(result->ptr);
 		result->size = 0;
 	}
  close_bad:
diff --git a/mailinfo.c b/mailinfo.c
index f92cb9f729..b26689b56d 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -919,8 +919,7 @@ static int handle_boundary(struct mailinfo *mi, struct strbuf *line)
 		/* we hit an end boundary */
 		/* pop the current boundary off the stack */
 		strbuf_release(*(mi->content_top));
-		free(*(mi->content_top));
-		*(mi->content_top) = NULL;
+		freez(*(mi->content_top));
 
 		/* technically won't happen as is_multipart_boundary()
 		   will fail first.  But just in case..
diff --git a/object.c b/object.c
index 06ba3a11d8..2883d46a0d 100644
--- a/object.c
+++ b/object.c
@@ -377,8 +377,7 @@ void object_array_clear(struct object_array *array)
 	int i;
 	for (i = 0; i < array->nr; i++)
 		object_array_release_entry(&array->objects[i]);
-	free(array->objects);
-	array->objects = NULL;
+	freez(array->objects);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/pathspec.c b/pathspec.c
index 828405021f..1f8ff7f4a9 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -662,7 +662,6 @@ void clear_pathspec(struct pathspec *pathspec)
 			attr_check_free(pathspec->items[i].attr_check);
 	}
 
-	free(pathspec->items);
-	pathspec->items = NULL;
+	freez(pathspec->items);
 	pathspec->nr = 0;
 }
diff --git a/prio-queue.c b/prio-queue.c
index fc3860fdcb..59211eb2e8 100644
--- a/prio-queue.c
+++ b/prio-queue.c
@@ -27,10 +27,9 @@ void prio_queue_reverse(struct prio_queue *queue)
 
 void clear_prio_queue(struct prio_queue *queue)
 {
-	free(queue->array);
+	freez(queue->array);
 	queue->nr = 0;
 	queue->alloc = 0;
-	queue->array = NULL;
 	queue->insertion_ctr = 0;
 }
 
diff --git a/read-cache.c b/read-cache.c
index bc156a133e..8434ef5577 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1888,8 +1888,7 @@ int discard_index(struct index_state *istate)
 	free_name_hash(istate);
 	cache_tree_free(&(istate->cache_tree));
 	istate->initialized = 0;
-	free(istate->cache);
-	istate->cache = NULL;
+	freez(istate->cache);
 	istate->cache_alloc = 0;
 	discard_split_index(istate);
 	free_untracked_cache(istate->untracked);
@@ -2603,8 +2602,7 @@ void *read_blob_data_from_index(const struct index_state *istate,
 
 void stat_validity_clear(struct stat_validity *sv)
 {
-	free(sv->sd);
-	sv->sd = NULL;
+	freez(sv->sd);
 }
 
 int stat_validity_check(struct stat_validity *sv, const char *path)
diff --git a/ref-filter.c b/ref-filter.c
index ab32bc9c31..5b05d93575 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1891,8 +1891,7 @@ void ref_array_clear(struct ref_array *array)
 
 	for (i = 0; i < array->nr; i++)
 		free_array_item(array->items[i]);
-	free(array->items);
-	array->items = NULL;
+	freez(array->items);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index d8b3f73147..dd90b090e8 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2944,8 +2944,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 				       head_oid.hash, &head_type);
 
 	if (head_ref && !(head_type & REF_ISSYMREF)) {
-		free(head_ref);
-		head_ref = NULL;
+		freez(head_ref);
 	}
 
 	/*
diff --git a/refs/ref-cache.c b/refs/ref-cache.c
index af2fcb2c12..ab12e0dc9d 100644
--- a/refs/ref-cache.c
+++ b/refs/ref-cache.c
@@ -82,9 +82,8 @@ static void clear_ref_dir(struct ref_dir *dir)
 	int i;
 	for (i = 0; i < dir->nr; i++)
 		free_ref_entry(dir->entries[i]);
-	free(dir->entries);
+	freez(dir->entries);
 	dir->sorted = dir->nr = dir->alloc = 0;
-	dir->entries = NULL;
 }
 
 struct ref_entry *create_dir_entry(struct ref_cache *cache,
diff --git a/remote-testsvn.c b/remote-testsvn.c
index f87bf851ba..9b893a5f09 100644
--- a/remote-testsvn.c
+++ b/remote-testsvn.c
@@ -66,8 +66,7 @@ static char *read_ref_note(const unsigned char sha1[20])
 	else if (!msglen || type != OBJ_BLOB) {
 		error("Note contains unusable content. "
 			"Is something else using this notes tree? %s", notes_ref);
-		free(msg);
-		msg = NULL;
+		freez(msg);
 	}
 	free_notes(NULL);
 	return msg;
diff --git a/rerere.c b/rerere.c
index 3bd55caf3b..43b62e6135 100644
--- a/rerere.c
+++ b/rerere.c
@@ -39,9 +39,8 @@ static void free_rerere_dirs(void)
 		free(rerere_dir[i]->status);
 		free(rerere_dir[i]);
 	}
-	free(rerere_dir);
+	freez(rerere_dir);
 	rerere_dir_nr = rerere_dir_alloc = 0;
-	rerere_dir = NULL;
 }
 
 static void free_rerere_id(struct string_list_item *item)
diff --git a/sequencer.c b/sequencer.c
index 924fb1d0c3..e542b3bd79 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1211,8 +1211,7 @@ struct todo_list {
 static void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
-	free(todo_list->items);
-	todo_list->items = NULL;
+	freez(todo_list->items);
 	todo_list->nr = todo_list->alloc = 0;
 }
 
diff --git a/sha1-array.c b/sha1-array.c
index 7d646ab5b8..c98506fd77 100644
--- a/sha1-array.c
+++ b/sha1-array.c
@@ -35,8 +35,7 @@ int oid_array_lookup(struct oid_array *array, const struct object_id *oid)
 
 void oid_array_clear(struct oid_array *array)
 {
-	free(array->oid);
-	array->oid = NULL;
+	freez(array->oid);
 	array->nr = 0;
 	array->alloc = 0;
 	array->sorted = 0;
diff --git a/sha1_file.c b/sha1_file.c
index 59a4ed2ed3..a34d904012 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -610,8 +610,7 @@ char *compute_alternate_path(const char *path, struct strbuf *err)
 
 out:
 	if (seen_error) {
-		free(ref_git);
-		ref_git = NULL;
+		freez(ref_git);
 	}
 
 	return ref_git;
diff --git a/split-index.c b/split-index.c
index 49bd197f71..87c197617a 100644
--- a/split-index.c
+++ b/split-index.c
@@ -174,10 +174,9 @@ void merge_base_index(struct index_state *istate)
 
 	ewah_free(si->delete_bitmap);
 	ewah_free(si->replace_bitmap);
-	free(si->saved_cache);
+	freez(si->saved_cache);
 	si->delete_bitmap  = NULL;
 	si->replace_bitmap = NULL;
-	si->saved_cache	   = NULL;
 	si->saved_cache_nr = 0;
 }
 
diff --git a/transport-helper.c b/transport-helper.c
index 36408046eb..a986589a14 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -242,8 +242,7 @@ static int disconnect_helper(struct transport *transport)
 		close(data->helper->out);
 		fclose(data->out);
 		res = finish_command(data->helper);
-		free(data->helper);
-		data->helper = NULL;
+		freez(data->helper);
 	}
 	return res;
 }
@@ -711,43 +710,35 @@ static int push_update_ref_status(struct strbuf *buf,
 
 		if (!strcmp(msg, "no match")) {
 			status = REF_STATUS_NONE;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 		else if (!strcmp(msg, "up to date")) {
 			status = REF_STATUS_UPTODATE;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 		else if (!strcmp(msg, "non-fast forward")) {
 			status = REF_STATUS_REJECT_NONFASTFORWARD;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 		else if (!strcmp(msg, "already exists")) {
 			status = REF_STATUS_REJECT_ALREADY_EXISTS;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 		else if (!strcmp(msg, "fetch first")) {
 			status = REF_STATUS_REJECT_FETCH_FIRST;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 		else if (!strcmp(msg, "needs force")) {
 			status = REF_STATUS_REJECT_NEEDS_FORCE;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 		else if (!strcmp(msg, "stale info")) {
 			status = REF_STATUS_REJECT_STALE;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 		else if (!strcmp(msg, "forced update")) {
 			forced = 1;
-			free(msg);
-			msg = NULL;
+			freez(msg);
 		}
 	}
 
diff --git a/transport.c b/transport.c
index 9bfcf870f9..1023a2d281 100644
--- a/transport.c
+++ b/transport.c
@@ -1145,8 +1145,7 @@ void transport_unlock_pack(struct transport *transport)
 {
 	if (transport->pack_lockfile) {
 		unlink_or_warn(transport->pack_lockfile);
-		free(transport->pack_lockfile);
-		transport->pack_lockfile = NULL;
+		freez(transport->pack_lockfile);
 	}
 }
 
diff --git a/tree-diff.c b/tree-diff.c
index e164e532b2..022745d4bd 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -140,8 +140,7 @@ static struct combine_diff_path *path_appendnew(struct combine_diff_path *last,
 	/* if last->next is !NULL - it is a pre-allocated memory, we can reuse */
 	p = last->next;
 	if (p && (alloclen > (intptr_t)p->next)) {
-		free(p);
-		p = NULL;
+		freez(p);
 	}
 
 	if (!p) {
@@ -559,8 +558,7 @@ struct combine_diff_path *diff_tree_paths(
 	 * (see path_appendnew() for details about why)
 	 */
 	if (p->next) {
-		free(p->next);
-		p->next = NULL;
+		freez(p->next);
 	}
 
 	return p;
diff --git a/tree-walk.c b/tree-walk.c
index 6a42e402b0..9f162155fe 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -479,8 +479,7 @@ int traverse_trees(int n, struct tree_desc *t, struct traverse_info *info)
 	for (i = 0; i < n; i++)
 		free_extended_entry(tx + i);
 	free(tx);
-	free(traverse_path);
-	info->traverse_path = NULL;
+	freez(traverse_path);
 	strbuf_release(&base);
 	return error;
 }
diff --git a/tree.c b/tree.c
index 603b29ee80..c03caabc0f 100644
--- a/tree.c
+++ b/tree.c
@@ -226,8 +226,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)
 
 void free_tree_buffer(struct tree *tree)
 {
-	free(tree->buffer);
-	tree->buffer = NULL;
+	freez(tree->buffer);
 	tree->size = 0;
 	tree->object.parsed = 0;
 }
-- 
2.13.0.506.g27d5fe0cd


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper
  2017-06-09  8:53     ` [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper Ævar Arnfjörð Bjarmason
@ 2017-06-09 10:12       ` Martin Ågren
  2017-06-09 10:58         ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 214+ messages in thread
From: Martin Ågren @ 2017-06-09 10:12 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Git Mailing List, Junio C Hamano, Brandon Williams

On 9 June 2017 at 10:53, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> Replace occurrences of `free(p); p = NULL` with `freez(p)`. This
> introduces no functional changes, but cuts the number of lines spent
> on this cleanup in half.

It's even better than that. ;)

>  48 files changed, 97 insertions(+), 197 deletions(-)

The difference is in builtin/am.c where some empty lines are removed
in am_next(), so no need to be alarmed.

The macro would be dangerous with things like "freez(ptr++)" but I
couldn't find any such side-effects. In hindsight, I guess your commit
message says as much since using "ptr++" for "p" would already be a
bug.

I have no idea whether this conflicts with other topics, or any
opinion on the best strategy for doing the conversion (all-at-once or
"while we're here").

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper
  2017-06-09 10:12       ` Martin Ågren
@ 2017-06-09 10:58         ` Ævar Arnfjörð Bjarmason
  2017-06-09 14:48           ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-09 10:58 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List, Junio C Hamano, Brandon Williams

On Fri, Jun 9, 2017 at 12:12 PM, Martin Ågren <martin.agren@gmail.com> wrote:
> On 9 June 2017 at 10:53, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> Replace occurrences of `free(p); p = NULL` with `freez(p)`. This
>> introduces no functional changes, but cuts the number of lines spent
>> on this cleanup in half.
>
> It's even better than that. ;)
>
>>  48 files changed, 97 insertions(+), 197 deletions(-)
>
> The difference is in builtin/am.c where some empty lines are removed
> in am_next(), so no need to be alarmed.
>
> The macro would be dangerous with things like "freez(ptr++)" but I
> couldn't find any such side-effects. In hindsight, I guess your commit
> message says as much since using "ptr++" for "p" would already be a
> bug.

Yes, although perhaps we should call this FREEZ() or GIT_FREEZ()
instead of freez() to make it clear that it's a macro.

> I have no idea whether this conflicts with other topics, or any
> opinion on the best strategy for doing the conversion (all-at-once or
> "while we're here").

It has no conflicts with pu, so that's something, and passes all tests
with & without that merge.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 1/2] git-compat-util: add a freez() wrapper around free(x); x = NULL
  2017-06-09  8:53     ` [PATCH 1/2] git-compat-util: add a freez() wrapper around free(x); x = NULL Ævar Arnfjörð Bjarmason
@ 2017-06-09 12:04       ` Christian Couder
  0 siblings, 0 replies; 214+ messages in thread
From: Christian Couder @ 2017-06-09 12:04 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Brandon Williams

On Fri, Jun 9, 2017 at 10:53 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:

> --- a/git-compat-util.h
> +++ b/git-compat-util.h
> @@ -787,6 +787,7 @@ extern char *xstrdup(const char *str);
>  extern void *xmalloc(size_t size);
>  extern void *xmallocz(size_t size);
>  extern void *xmallocz_gently(size_t size);
> +#define freez(p) do { if (p) { free(p); (p) = NULL; } } while (0)

I think we already rely on free(NULL) working, see
http://public-inbox.org/git/alpine.DEB.2.20.1608301948310.129229@virtualbox/
for example, so this could just be:

#define freez(p) do { free(p); (p) = NULL; } while (0)

and yeah FREEZ() would be even better than freez().

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper
  2017-06-09 10:58         ` Ævar Arnfjörð Bjarmason
@ 2017-06-09 14:48           ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-09 14:48 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Martin Ågren, Git Mailing List, Junio C Hamano

On 06/09, Ævar Arnfjörð Bjarmason wrote:
> On Fri, Jun 9, 2017 at 12:12 PM, Martin Ågren <martin.agren@gmail.com> wrote:
> > On 9 June 2017 at 10:53, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >> Replace occurrences of `free(p); p = NULL` with `freez(p)`. This
> >> introduces no functional changes, but cuts the number of lines spent
> >> on this cleanup in half.
> >
> > It's even better than that. ;)
> >
> >>  48 files changed, 97 insertions(+), 197 deletions(-)
> >
> > The difference is in builtin/am.c where some empty lines are removed
> > in am_next(), so no need to be alarmed.
> >
> > The macro would be dangerous with things like "freez(ptr++)" but I
> > couldn't find any such side-effects. In hindsight, I guess your commit
> > message says as much since using "ptr++" for "p" would already be a
> > bug.

It also couldn't hurt to add a comment to the macro definition
explaining that side-effect operators would be broken.

> 
> Yes, although perhaps we should call this FREEZ() or GIT_FREEZ()
> instead of freez() to make it clear that it's a macro.
> 
> > I have no idea whether this conflicts with other topics, or any
> > opinion on the best strategy for doing the conversion (all-at-once or
> > "while we're here").
> 
> It has no conflicts with pu, so that's something, and passes all tests
> with & without that merge.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH 0/2] Add a freez() wrapper
  2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
@ 2017-06-09 14:53       ` Brandon Williams
  2017-06-09 22:04       ` [PATCH v2 0/2] Add a FREEZ() wrapper macro Ævar Arnfjörð Bjarmason
                         ` (2 subsequent siblings)
  3 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-09 14:53 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano

On 06/09, Ævar Arnfjörð Bjarmason wrote:
> On Fri, Jun 9, 2017 at 1:40 AM, Brandon Williams <bmwill@google.com> wrote:
> > Introduce the repository object 'struct repository' which can be used to
> > hold all state pertaining to a git repository.
> > [...]
> > +static void repo_clear_env(struct repository *repo)
> > +{
> > +       free(repo->gitdir);
> > +       repo->gitdir = NULL;
> > +       free(repo->commondir);
> > +       repo->commondir = NULL;
> > +       free(repo->objectdir);
> > +       repo->objectdir = NULL;
> > +       free(repo->index_file);
> > +       repo->index_file = NULL;
> > +       free(repo->graft_file);
> > +       repo->graft_file = NULL;
> > +       free(repo->namespace);
> > +       repo->namespace = NULL;
> > +}
> 
> When seeing this I wondered if it couldn't be less noisy as:

One of the other alternatives is using memset, but that's difficult to
only clear only a few fields.

> 
> > +       freez(repo->gitdir);
> > +       freez(repo->commondir);
> > +       freez(repo->objectdir);
> > +       freez(repo->index_file);
> > +       freez(repo->graft_file);
> > +       freez(repo->namespace);
> 
> It's not a problem with your patch, but something I thought would be
> nice to have in general, so here's a patch series to implement that.

This also could makes things a little less error prone.  Thanks!

> 
> Ævar Arnfjörð Bjarmason (2):
>   git-compat-util: add a freez() wrapper around free(x); x = NULL
>   *.[ch] refactoring: make use of the freez() wrapper
> 
>  alias.c                  |  6 ++----
>  apply.c                  |  3 +--
>  attr.c                   |  6 ++----
>  blame.c                  |  3 +--
>  branch.c                 |  3 +--
>  builtin/am.c             | 18 +++++-------------
>  builtin/clean.c          |  6 ++----
>  builtin/config.c         |  6 ++----
>  builtin/index-pack.c     |  6 ++----
>  builtin/pack-objects.c   | 12 ++++--------
>  builtin/unpack-objects.c |  3 +--
>  builtin/worktree.c       |  6 ++----
>  commit-slab.h            |  3 +--
>  commit.c                 |  3 +--
>  config.c                 |  3 +--
>  credential.c             |  9 +++------
>  diff-lib.c               |  3 +--
>  diff.c                   |  6 ++----
>  diffcore-rename.c        |  6 ++----
>  dir.c                    |  9 +++------
>  fast-import.c            |  6 ++----
>  git-compat-util.h        |  1 +
>  gpg-interface.c          | 15 +++++----------
>  grep.c                   | 12 ++++--------
>  help.c                   |  3 +--
>  http-push.c              | 24 ++++++++----------------
>  http.c                   | 15 +++++----------
>  imap-send.c              |  3 +--
>  line-log.c               |  6 ++----
>  ll-merge.c               |  3 +--
>  mailinfo.c               |  3 +--
>  object.c                 |  3 +--
>  pathspec.c               |  3 +--
>  prio-queue.c             |  3 +--
>  read-cache.c             |  6 ++----
>  ref-filter.c             |  3 +--
>  refs/files-backend.c     |  3 +--
>  refs/ref-cache.c         |  3 +--
>  remote-testsvn.c         |  3 +--
>  rerere.c                 |  3 +--
>  sequencer.c              |  3 +--
>  sha1-array.c             |  3 +--
>  sha1_file.c              |  3 +--
>  split-index.c            |  3 +--
>  transport-helper.c       | 27 +++++++++------------------
>  transport.c              |  3 +--
>  tree-diff.c              |  6 ++----
>  tree-walk.c              |  3 +--
>  tree.c                   |  3 +--
>  49 files changed, 98 insertions(+), 197 deletions(-)
> 
> -- 
> 2.13.0.506.g27d5fe0cd
> 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v2 0/2] Add a FREEZ() wrapper macro
  2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
  2017-06-09 14:53       ` Brandon Williams
@ 2017-06-09 22:04       ` Ævar Arnfjörð Bjarmason
  2017-06-09 22:04       ` [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
  2017-06-09 22:04       ` [PATCH v2 2/2] *.[ch] refactoring: make use of the FREEZ() macro Ævar Arnfjörð Bjarmason
  3 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-09 22:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams, Christian Couder,
	Martin Ågren, Ævar Arnfjörð Bjarmason

See <20170609085346.19974-1-avarab@gmail.com>
(https://public-inbox.org/git/20170609085346.19974-1-avarab@gmail.com/)
for v1.

Changes:

 * Now called FREEZ() instead of freez().
 * Documented the caveat of giving it ptr++ both in the commit message
   and in a comment.
 * Moved the definition to be with other similar macros.
 * Just unconditionally call free(ptr), even if ptr is NULL, as we do
   everywhere else.

Ævar Arnfjörð Bjarmason (2):
  git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  *.[ch] refactoring: make use of the FREEZ() macro

 alias.c                  |  6 ++----
 apply.c                  |  3 +--
 attr.c                   |  6 ++----
 blame.c                  |  3 +--
 branch.c                 |  3 +--
 builtin/am.c             | 18 +++++-------------
 builtin/clean.c          |  6 ++----
 builtin/config.c         |  6 ++----
 builtin/index-pack.c     |  6 ++----
 builtin/pack-objects.c   | 12 ++++--------
 builtin/unpack-objects.c |  3 +--
 builtin/worktree.c       |  6 ++----
 commit-slab.h            |  3 +--
 commit.c                 |  3 +--
 config.c                 |  3 +--
 credential.c             |  9 +++------
 diff-lib.c               |  3 +--
 diff.c                   |  6 ++----
 diffcore-rename.c        |  6 ++----
 dir.c                    |  9 +++------
 fast-import.c            |  6 ++----
 git-compat-util.h        |  6 ++++++
 gpg-interface.c          | 15 +++++----------
 grep.c                   | 12 ++++--------
 help.c                   |  3 +--
 http-push.c              | 24 ++++++++----------------
 http.c                   | 15 +++++----------
 imap-send.c              |  3 +--
 line-log.c               |  6 ++----
 ll-merge.c               |  3 +--
 mailinfo.c               |  3 +--
 object.c                 |  3 +--
 pathspec.c               |  3 +--
 prio-queue.c             |  3 +--
 read-cache.c             |  6 ++----
 ref-filter.c             |  3 +--
 refs/files-backend.c     |  3 +--
 refs/ref-cache.c         |  3 +--
 remote-testsvn.c         |  3 +--
 rerere.c                 |  3 +--
 sequencer.c              |  3 +--
 sha1-array.c             |  3 +--
 sha1_file.c              |  3 +--
 split-index.c            |  3 +--
 transport-helper.c       | 27 +++++++++------------------
 transport.c              |  3 +--
 tree-diff.c              |  6 ++----
 tree-walk.c              |  3 +--
 tree.c                   |  3 +--
 49 files changed, 103 insertions(+), 197 deletions(-)

-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
  2017-06-09 14:53       ` Brandon Williams
  2017-06-09 22:04       ` [PATCH v2 0/2] Add a FREEZ() wrapper macro Ævar Arnfjörð Bjarmason
@ 2017-06-09 22:04       ` Ævar Arnfjörð Bjarmason
  2017-06-09 22:27         ` Jonathan Nieder
  2017-06-09 22:04       ` [PATCH v2 2/2] *.[ch] refactoring: make use of the FREEZ() macro Ævar Arnfjörð Bjarmason
  3 siblings, 1 reply; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-09 22:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams, Christian Couder,
	Martin Ågren, Ævar Arnfjörð Bjarmason

Add a FREEZ() wrapper marco for the common pattern of freeing a
pointer and assigning NULL to it right afterwards.

The implementation is similar to the (currently unused) XDL_PTRFREE
macro in xdiff/xmacros.h added in commit 3443546f6e ("Use a *real*
built-in diff generator", 2006-03-24). The only difference is that
free() is called unconditionally, see [1].

1. <alpine.DEB.2.20.1608301948310.129229@virtualbox>
   (http://public-inbox.org/git/alpine.DEB.2.20.1608301948310.129229@virtualbox/)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-compat-util.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index 4b7dcf21ad..ba2d0c8c80 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -805,6 +805,12 @@ extern int xmkstemp_mode(char *template, int mode);
 extern char *xgetcwd(void);
 extern FILE *fopen_for_writing(const char *path);
 
+/*
+ * FREEZ(ptr) is like free(ptr) followed by ptr = NULL. Note that ptr
+ * is used twice, so don't pass e.g. ptr++.
+ */
+#define FREEZ(p) do { free(p); (p) = NULL; } while (0)
+
 #define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), (alloc)))
 #define REALLOC_ARRAY(x, alloc) (x) = xrealloc((x), st_mult(sizeof(*(x)), (alloc)))
 
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v2 2/2] *.[ch] refactoring: make use of the FREEZ() macro
  2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
                         ` (2 preceding siblings ...)
  2017-06-09 22:04       ` [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
@ 2017-06-09 22:04       ` Ævar Arnfjörð Bjarmason
  3 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-09 22:04 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams, Christian Couder,
	Martin Ågren, Ævar Arnfjörð Bjarmason

Replace occurrences of `free(ptr); ptr = NULL` with `FREEZ(ptr)`. This
introduces no functional changes, but reduces the line count and
establishes this pattern as a common idiom with a wrapper macro.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 alias.c                  |  6 ++----
 apply.c                  |  3 +--
 attr.c                   |  6 ++----
 blame.c                  |  3 +--
 branch.c                 |  3 +--
 builtin/am.c             | 18 +++++-------------
 builtin/clean.c          |  6 ++----
 builtin/config.c         |  6 ++----
 builtin/index-pack.c     |  6 ++----
 builtin/pack-objects.c   | 12 ++++--------
 builtin/unpack-objects.c |  3 +--
 builtin/worktree.c       |  6 ++----
 commit-slab.h            |  3 +--
 commit.c                 |  3 +--
 config.c                 |  3 +--
 credential.c             |  9 +++------
 diff-lib.c               |  3 +--
 diff.c                   |  6 ++----
 diffcore-rename.c        |  6 ++----
 dir.c                    |  9 +++------
 fast-import.c            |  6 ++----
 gpg-interface.c          | 15 +++++----------
 grep.c                   | 12 ++++--------
 help.c                   |  3 +--
 http-push.c              | 24 ++++++++----------------
 http.c                   | 15 +++++----------
 imap-send.c              |  3 +--
 line-log.c               |  6 ++----
 ll-merge.c               |  3 +--
 mailinfo.c               |  3 +--
 object.c                 |  3 +--
 pathspec.c               |  3 +--
 prio-queue.c             |  3 +--
 read-cache.c             |  6 ++----
 ref-filter.c             |  3 +--
 refs/files-backend.c     |  3 +--
 refs/ref-cache.c         |  3 +--
 remote-testsvn.c         |  3 +--
 rerere.c                 |  3 +--
 sequencer.c              |  3 +--
 sha1-array.c             |  3 +--
 sha1_file.c              |  3 +--
 split-index.c            |  3 +--
 transport-helper.c       | 27 +++++++++------------------
 transport.c              |  3 +--
 tree-diff.c              |  6 ++----
 tree-walk.c              |  3 +--
 tree.c                   |  3 +--
 48 files changed, 97 insertions(+), 197 deletions(-)

diff --git a/alias.c b/alias.c
index 3b90397a99..a8cef09650 100644
--- a/alias.c
+++ b/alias.c
@@ -47,8 +47,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 				src++;
 				c = cmdline[src];
 				if (!c) {
-					free(*argv);
-					*argv = NULL;
+					FREEZ(*argv);
 					return -SPLIT_CMDLINE_BAD_ENDING;
 				}
 			}
@@ -60,8 +59,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 	cmdline[dst] = 0;
 
 	if (quoted) {
-		free(*argv);
-		*argv = NULL;
+		FREEZ(*argv);
 		return -SPLIT_CMDLINE_UNCLOSED_QUOTE;
 	}
 
diff --git a/apply.c b/apply.c
index c49cef0637..f11330bbfe 100644
--- a/apply.c
+++ b/apply.c
@@ -3705,8 +3705,7 @@ static int check_preimage(struct apply_state *state,
  is_new:
 	patch->is_new = 1;
 	patch->is_delete = 0;
-	free(patch->old_name);
-	patch->old_name = NULL;
+	FREEZ(patch->old_name);
 	return 0;
 }
 
diff --git a/attr.c b/attr.c
index 7e2134471c..0b550695d5 100644
--- a/attr.c
+++ b/attr.c
@@ -638,13 +638,11 @@ void attr_check_reset(struct attr_check *check)
 
 void attr_check_clear(struct attr_check *check)
 {
-	free(check->items);
-	check->items = NULL;
+	FREEZ(check->items);
 	check->alloc = 0;
 	check->nr = 0;
 
-	free(check->all_attrs);
-	check->all_attrs = NULL;
+	FREEZ(check->all_attrs);
 	check->all_attrs_nr = 0;
 
 	drop_attr_stack(&check->stack);
diff --git a/blame.c b/blame.c
index 843c845cba..5b2eac3915 100644
--- a/blame.c
+++ b/blame.c
@@ -314,8 +314,7 @@ static void fill_origin_blob(struct diff_options *opt,
 static void drop_origin_blob(struct blame_origin *o)
 {
 	if (o->file.ptr) {
-		free(o->file.ptr);
-		o->file.ptr = NULL;
+		FREEZ(o->file.ptr);
 	}
 }
 
diff --git a/branch.c b/branch.c
index 985316eb76..b2fce80706 100644
--- a/branch.c
+++ b/branch.c
@@ -24,8 +24,7 @@ static int find_tracked_branch(struct remote *remote, void *priv)
 		} else {
 			free(tracking->spec.src);
 			if (tracking->src) {
-				free(tracking->src);
-				tracking->src = NULL;
+				FREEZ(tracking->src);
 			}
 		}
 		tracking->spec.src = NULL;
diff --git a/builtin/am.c b/builtin/am.c
index 5ee146bfb3..f665239dc3 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -483,8 +483,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
 	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
 
 	if (!ret) {
-		free(state->msg);
-		state->msg = NULL;
+		FREEZ(state->msg);
 		if (read_commit_msg(state) < 0)
 			die(_("'%s' was deleted by the applypatch-msg hook"),
 				am_path(state, "final-commit"));
@@ -1073,17 +1072,10 @@ static void am_next(struct am_state *state)
 {
 	struct object_id head;
 
-	free(state->author_name);
-	state->author_name = NULL;
-
-	free(state->author_email);
-	state->author_email = NULL;
-
-	free(state->author_date);
-	state->author_date = NULL;
-
-	free(state->msg);
-	state->msg = NULL;
+	FREEZ(state->author_name);
+	FREEZ(state->author_email);
+	FREEZ(state->author_date);
+	FREEZ(state->msg);
 	state->msg_len = 0;
 
 	unlink(am_path(state, "author-script"));
diff --git a/builtin/clean.c b/builtin/clean.c
index 142bf668cf..027ce46505 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -837,8 +837,7 @@ static void interactive_main_loop(void)
 			int ret;
 			ret = menus[*chosen].fn();
 			if (ret != MENU_RETURN_NO_LOOP) {
-				free(chosen);
-				chosen = NULL;
+				FREEZ(chosen);
 				if (!del_list.nr) {
 					clean_print_color(CLEAN_COLOR_ERROR);
 					printf_ln(_("No more files to clean, exiting."));
@@ -851,8 +850,7 @@ static void interactive_main_loop(void)
 			quit_cmd();
 		}
 
-		free(chosen);
-		chosen = NULL;
+		FREEZ(chosen);
 		break;
 	}
 }
diff --git a/builtin/config.c b/builtin/config.c
index 7f6c25d4d9..f2b698971c 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -214,8 +214,7 @@ static int get_value(const char *key_, const char *regex_)
 		key_regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(key_regexp, key, REG_EXTENDED)) {
 			error("invalid key pattern: %s", key_);
-			free(key_regexp);
-			key_regexp = NULL;
+			FREEZ(key_regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
@@ -235,8 +234,7 @@ static int get_value(const char *key_, const char *regex_)
 		regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(regexp, regex_, REG_EXTENDED)) {
 			error("invalid pattern: %s", regex_);
-			free(regexp);
-			regexp = NULL;
+			FREEZ(regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 04b9dcaf0f..17c86168fb 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -388,8 +388,7 @@ static struct base_data *alloc_base_data(void)
 static void free_base_data(struct base_data *c)
 {
 	if (c->data) {
-		free(c->data);
-		c->data = NULL;
+		FREEZ(c->data);
 		get_thread_data()->base_cache_used -= c->size;
 	}
 }
@@ -605,8 +604,7 @@ static void *unpack_data(struct object_entry *obj,
 	git_inflate_end(&stream);
 	free(inbuf);
 	if (consume) {
-		free(data);
-		data = NULL;
+		FREEZ(data);
 	}
 	return data;
 }
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f672225def..451079b2f2 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -264,8 +264,7 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 		 * make sure no cached delta data remains from a
 		 * previous attempt before a pack split occurred.
 		 */
-		free(entry->delta_data);
-		entry->delta_data = NULL;
+		FREEZ(entry->delta_data);
 		entry->z_delta_size = 0;
 	} else if (entry->delta_data) {
 		size = entry->delta_size;
@@ -1375,12 +1374,10 @@ static void cleanup_preferred_base(void)
 		if (!pbase_tree_cache[i])
 			continue;
 		free(pbase_tree_cache[i]->tree_data);
-		free(pbase_tree_cache[i]);
-		pbase_tree_cache[i] = NULL;
+		FREEZ(pbase_tree_cache[i]);
 	}
 
-	free(done_pbase_paths);
-	done_pbase_paths = NULL;
+	FREEZ(done_pbase_paths);
 	done_pbase_paths_num = done_pbase_paths_alloc = 0;
 }
 
@@ -1970,8 +1967,7 @@ static unsigned long free_unpacked(struct unpacked *n)
 	n->index = NULL;
 	if (n->data) {
 		freed_mem += n->entry->size;
-		free(n->data);
-		n->data = NULL;
+		FREEZ(n->data);
 	}
 	n->entry = NULL;
 	n->depth = 0;
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 8bc9997767..402232dc4d 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -112,8 +112,7 @@ static void *get_data(unsigned long size)
 			break;
 		if (ret != Z_OK) {
 			error("inflate returned %d", ret);
-			free(buf);
-			buf = NULL;
+			FREEZ(buf);
 			if (!recover)
 				exit(1);
 			has_errors = 1;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 793306ea51..2371fef396 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -299,10 +299,8 @@ static int add_worktree(const char *path, const char *refname,
 	}
 
 	is_junk = 0;
-	free(junk_work_tree);
-	free(junk_git_dir);
-	junk_work_tree = NULL;
-	junk_git_dir = NULL;
+	FREEZ(junk_work_tree);
+	FREEZ(junk_git_dir);
 
 done:
 	if (ret || !opts->keep_locked) {
diff --git a/commit-slab.h b/commit-slab.h
index 42d16dcded..a239112360 100644
--- a/commit-slab.h
+++ b/commit-slab.h
@@ -82,8 +82,7 @@ static MAYBE_UNUSED void clear_ ##slabname(struct slabname *s)		\
 	for (i = 0; i < s->slab_count; i++)				\
 		free(s->slab[i]);					\
 	s->slab_count = 0;						\
-	free(s->slab);							\
-	s->slab = NULL;							\
+	FREEZ(s->slab);							\
 }									\
 									\
 static MAYBE_UNUSED elemtype *slabname## _at_peek(struct slabname *s,	\
diff --git a/commit.c b/commit.c
index 713f09feb0..82c80fdd5e 100644
--- a/commit.c
+++ b/commit.c
@@ -287,8 +287,7 @@ void free_commit_buffer(struct commit *commit)
 {
 	struct commit_buffer *v = buffer_slab_peek(&buffer_slab, commit);
 	if (v) {
-		free(v->buffer);
-		v->buffer = NULL;
+		FREEZ(v->buffer);
 		v->size = 0;
 	}
 }
diff --git a/config.c b/config.c
index 146cb3452a..fa366d1c18 100644
--- a/config.c
+++ b/config.c
@@ -395,8 +395,7 @@ static int git_config_parse_key_1(const char *key, char **store_key, int *basele
 
 out_free_ret_1:
 	if (store_key) {
-		free(*store_key);
-		*store_key = NULL;
+		FREEZ(*store_key);
 	}
 	return -CONFIG_INVALID_KEY;
 }
diff --git a/credential.c b/credential.c
index aa996669fc..73650483b6 100644
--- a/credential.c
+++ b/credential.c
@@ -93,8 +93,7 @@ static void credential_apply_config(struct credential *c)
 	c->configured = 1;
 
 	if (!c->use_http_path && proto_is_http(c->protocol)) {
-		free(c->path);
-		c->path = NULL;
+		FREEZ(c->path);
 	}
 }
 
@@ -314,10 +313,8 @@ void credential_reject(struct credential *c)
 	for (i = 0; i < c->helpers.nr; i++)
 		credential_do(c, c->helpers.items[i].string, "erase");
 
-	free(c->username);
-	c->username = NULL;
-	free(c->password);
-	c->password = NULL;
+	FREEZ(c->username);
+	FREEZ(c->password);
 	c->approved = 0;
 }
 
diff --git a/diff-lib.c b/diff-lib.c
index 2982bf055a..1130a9c9d9 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -179,8 +179,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)
 				free(dpath);
 				continue;
 			}
-			free(dpath);
-			dpath = NULL;
+			FREEZ(dpath);
 
 			/*
 			 * Show the diff for the 'ce' if we found the one
diff --git a/diff.c b/diff.c
index 5275c4b780..db9282d731 100644
--- a/diff.c
+++ b/diff.c
@@ -1218,8 +1218,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)
 			regfree(ecbdata->diff_words->word_regex);
 			free(ecbdata->diff_words->word_regex);
 		}
-		free(ecbdata->diff_words);
-		ecbdata->diff_words = NULL;
+		FREEZ(ecbdata->diff_words);
 	}
 }
 
@@ -2951,8 +2950,7 @@ void diff_free_filespec_blob(struct diff_filespec *s)
 void diff_free_filespec_data(struct diff_filespec *s)
 {
 	diff_free_filespec_blob(s);
-	free(s->cnt_data);
-	s->cnt_data = NULL;
+	FREEZ(s->cnt_data);
 }
 
 static void prep_temp_blob(const char *path, struct diff_tempfile *temp,
diff --git a/diffcore-rename.c b/diffcore-rename.c
index f7444c86bd..f9428082bc 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -667,11 +667,9 @@ void diffcore_rename(struct diff_options *options)
 	for (i = 0; i < rename_dst_nr; i++)
 		free_filespec(rename_dst[i].two);
 
-	free(rename_dst);
-	rename_dst = NULL;
+	FREEZ(rename_dst);
 	rename_dst_nr = rename_dst_alloc = 0;
-	free(rename_src);
-	rename_src = NULL;
+	FREEZ(rename_src);
 	rename_src_nr = rename_src_alloc = 0;
 	return;
 }
diff --git a/dir.c b/dir.c
index 9efcf1eab6..0d4e7e6bfa 100644
--- a/dir.c
+++ b/dir.c
@@ -2117,8 +2117,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		for (i = j = 0; j < dir->nr; j++) {
 			if (i &&
 			    check_dir_entry_contains(dir->entries[i - 1], dir->entries[j])) {
-				free(dir->entries[j]);
-				dir->entries[j] = NULL;
+				FREEZ(dir->entries[j]);
 			} else {
 				dir->entries[i++] = dir->entries[j];
 			}
@@ -2144,8 +2143,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		     dir->untracked->dir_invalidated))
 			istate->cache_changed |= UNTRACKED_CHANGED;
 		if (dir->untracked != istate->untracked) {
-			free(dir->untracked);
-			dir->untracked = NULL;
+			FREEZ(dir->untracked);
 		}
 	}
 	return dir->nr;
@@ -2488,8 +2486,7 @@ void write_untracked_extension(struct strbuf *out, struct untracked_cache *untra
 	strbuf_addbuf(out, &untracked->ident);
 
 	strbuf_add(out, ouc, ouc_size(len));
-	free(ouc);
-	ouc = NULL;
+	FREEZ(ouc);
 
 	if (!untracked->root) {
 		varint_len = encode_varint(0, varbuf);
diff --git a/fast-import.c b/fast-import.c
index e69d219682..4e23caeaad 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1063,8 +1063,7 @@ static void end_packfile(void)
 		close(pack_data->pack_fd);
 		unlink_or_warn(pack_data->pack_name);
 	}
-	free(pack_data);
-	pack_data = NULL;
+	FREEZ(pack_data);
 	running = 0;
 
 	/* We can't carry a delta across packfiles. */
@@ -1149,8 +1148,7 @@ static int store_object(
 
 		/* We cannot carry a delta into the new pack. */
 		if (delta) {
-			free(delta);
-			delta = NULL;
+			FREEZ(delta);
 
 			git_deflate_init(&s, pack_compression_level);
 			s.next_in = (void *)dat->buf;
diff --git a/gpg-interface.c b/gpg-interface.c
index e44cc27da1..93739787f9 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -13,16 +13,11 @@ static const char *gpg_program = "gpg";
 
 void signature_check_clear(struct signature_check *sigc)
 {
-	free(sigc->payload);
-	free(sigc->gpg_output);
-	free(sigc->gpg_status);
-	free(sigc->signer);
-	free(sigc->key);
-	sigc->payload = NULL;
-	sigc->gpg_output = NULL;
-	sigc->gpg_status = NULL;
-	sigc->signer = NULL;
-	sigc->key = NULL;
+	FREEZ(sigc->payload);
+	FREEZ(sigc->gpg_output);
+	FREEZ(sigc->gpg_status);
+	FREEZ(sigc->signer);
+	FREEZ(sigc->key);
 }
 
 static struct {
diff --git a/grep.c b/grep.c
index d03d424e5c..2ec0f7800d 100644
--- a/grep.c
+++ b/grep.c
@@ -1763,12 +1763,9 @@ void grep_source_init(struct grep_source *gs, enum grep_source_type type,
 
 void grep_source_clear(struct grep_source *gs)
 {
-	free(gs->name);
-	gs->name = NULL;
-	free(gs->path);
-	gs->path = NULL;
-	free(gs->identifier);
-	gs->identifier = NULL;
+	FREEZ(gs->name);
+	FREEZ(gs->path);
+	FREEZ(gs->identifier);
 	grep_source_clear_data(gs);
 }
 
@@ -1778,8 +1775,7 @@ void grep_source_clear_data(struct grep_source *gs)
 	case GREP_SOURCE_FILE:
 	case GREP_SOURCE_SHA1:
 	case GREP_SOURCE_SUBMODULE:
-		free(gs->buf);
-		gs->buf = NULL;
+		FREEZ(gs->buf);
 		gs->size = 0;
 		break;
 	case GREP_SOURCE_BUF:
diff --git a/help.c b/help.c
index db7f3d79a0..90d1ff6a48 100644
--- a/help.c
+++ b/help.c
@@ -267,9 +267,8 @@ static void add_cmd_list(struct cmdnames *cmds, struct cmdnames *old)
 
 	for (i = 0; i < old->cnt; i++)
 		cmds->names[cmds->cnt++] = old->names[i];
-	free(old->names);
+	FREEZ(old->names);
 	old->cnt = 0;
-	old->names = NULL;
 }
 
 /* An empirically derived magic number */
diff --git a/http-push.c b/http-push.c
index 67c4d4b472..eda9725f14 100644
--- a/http-push.c
+++ b/http-push.c
@@ -291,8 +291,7 @@ static void start_mkcol(struct transfer_request *request)
 		request->state = RUN_MKCOL;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREEZ(request->url);
 	}
 }
 #endif
@@ -409,8 +408,7 @@ static void start_put(struct transfer_request *request)
 		request->state = RUN_PUT;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREEZ(request->url);
 	}
 }
 
@@ -432,8 +430,7 @@ static void start_move(struct transfer_request *request)
 		request->state = RUN_MOVE;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREEZ(request->url);
 	}
 }
 
@@ -526,8 +523,7 @@ static void finish_request(struct transfer_request *request)
 
 	/* URL is reused for MOVE after PUT */
 	if (request->state != RUN_PUT) {
-		free(request->url);
-		request->url = NULL;
+		FREEZ(request->url);
 	}
 
 	if (request->state == RUN_MKCOL) {
@@ -803,8 +799,7 @@ xml_start_tag(void *userData, const char *name, const char **atts)
 	}
 	xsnprintf(ctx->name + old_namelen, ctx->len - old_namelen, ".%s", c);
 
-	free(ctx->cdata);
-	ctx->cdata = NULL;
+	FREEZ(ctx->cdata);
 
 	ctx->userFunc(ctx, 0);
 }
@@ -932,8 +927,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)
 		free(lock->token);
 		free(lock->owner);
 		free(url);
-		free(lock);
-		lock = NULL;
+		FREEZ(lock);
 	} else {
 		lock->url = url;
 		lock->start_time = time(NULL);
@@ -1105,8 +1099,7 @@ static void handle_remote_ls_ctx(struct xml_ctx *ctx, int tag_closed)
 			ls->dentry_flags |= IS_DIR;
 		}
 	} else if (!strcmp(ctx->name, DAV_PROPFIND_RESP)) {
-		free(ls->dentry_name);
-		ls->dentry_name = NULL;
+		FREEZ(ls->dentry_name);
 		ls->dentry_flags = 0;
 	}
 }
@@ -1547,8 +1540,7 @@ static void fetch_symref(const char *path, char **symref, struct object_id *oid)
 		    curl_errorstr);
 	free(url);
 
-	free(*symref);
-	*symref = NULL;
+	FREEZ(*symref);
 	oidclr(oid);
 
 	if (buffer.len == 0)
diff --git a/http.c b/http.c
index d2e11ec6f0..4ce761f9a0 100644
--- a/http.c
+++ b/http.c
@@ -1026,8 +1026,7 @@ void http_cleanup(void)
 
 	if (proxy_auth.password) {
 		memset(proxy_auth.password, 0, strlen(proxy_auth.password));
-		free(proxy_auth.password);
-		proxy_auth.password = NULL;
+		FREEZ(proxy_auth.password);
 	}
 
 	free((void *)curl_proxyuserpwd);
@@ -1038,13 +1037,11 @@ void http_cleanup(void)
 
 	if (cert_auth.password != NULL) {
 		memset(cert_auth.password, 0, strlen(cert_auth.password));
-		free(cert_auth.password);
-		cert_auth.password = NULL;
+		FREEZ(cert_auth.password);
 	}
 	ssl_cert_password_required = 0;
 
-	free(cached_accept_language);
-	cached_accept_language = NULL;
+	FREEZ(cached_accept_language);
 }
 
 struct active_request_slot *get_active_slot(void)
@@ -1896,8 +1893,7 @@ static char *fetch_pack_index(unsigned char *sha1, const char *base_url)
 
 	if (http_get_file(url, tmp, NULL) != HTTP_OK) {
 		error("Unable to get pack index %s", url);
-		free(tmp);
-		tmp = NULL;
+		FREEZ(tmp);
 	}
 
 	free(url);
@@ -2328,8 +2324,7 @@ void release_http_object_request(struct http_object_request *freq)
 		freq->localfile = -1;
 	}
 	if (freq->url != NULL) {
-		free(freq->url);
-		freq->url = NULL;
+		FREEZ(freq->url);
 	}
 	if (freq->slot != NULL) {
 		freq->slot->callback_func = NULL;
diff --git a/imap-send.c b/imap-send.c
index 857591660f..b4b29c30c8 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -776,8 +776,7 @@ static int get_cmd_result(struct imap_store *ctx, struct imap_cmd *tcmd)
 			       offsetof(struct imap_cmd, next));
 			if (cmdp->cb.data) {
 				n = socket_write(&imap->buf.sock, cmdp->cb.data, cmdp->cb.dlen);
-				free(cmdp->cb.data);
-				cmdp->cb.data = NULL;
+				FREEZ(cmdp->cb.data);
 				if (n != (int)cmdp->cb.dlen)
 					return RESP_BAD;
 			} else if (cmdp->cb.cont) {
diff --git a/line-log.c b/line-log.c
index b9087814b8..272c853ede 100644
--- a/line-log.c
+++ b/line-log.c
@@ -34,9 +34,8 @@ void range_set_init(struct range_set *rs, size_t prealloc)
 
 void range_set_release(struct range_set *rs)
 {
-	free(rs->ranges);
+	FREEZ(rs->ranges);
 	rs->alloc = rs->nr = 0;
-	rs->ranges = NULL;
 }
 
 /* dst must be uninitialized! */
@@ -610,8 +609,7 @@ parse_lines(struct commit *commit, const char *prefix, struct string_list *args)
 		line_log_data_insert(&ranges, full_name, begin, end);
 
 		free_filespec(spec);
-		free(ends);
-		ends = NULL;
+		FREEZ(ends);
 	}
 
 	for (p = ranges; p; p = p->next)
diff --git a/ll-merge.c b/ll-merge.c
index ac0d4a5d78..65067a6a1a 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -209,8 +209,7 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 	result->size = st.st_size;
 	result->ptr = xmallocz(result->size);
 	if (read_in_full(fd, result->ptr, result->size) != result->size) {
-		free(result->ptr);
-		result->ptr = NULL;
+		FREEZ(result->ptr);
 		result->size = 0;
 	}
  close_bad:
diff --git a/mailinfo.c b/mailinfo.c
index f92cb9f729..4531ccad58 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -919,8 +919,7 @@ static int handle_boundary(struct mailinfo *mi, struct strbuf *line)
 		/* we hit an end boundary */
 		/* pop the current boundary off the stack */
 		strbuf_release(*(mi->content_top));
-		free(*(mi->content_top));
-		*(mi->content_top) = NULL;
+		FREEZ(*(mi->content_top));
 
 		/* technically won't happen as is_multipart_boundary()
 		   will fail first.  But just in case..
diff --git a/object.c b/object.c
index 06ba3a11d8..156931f735 100644
--- a/object.c
+++ b/object.c
@@ -377,8 +377,7 @@ void object_array_clear(struct object_array *array)
 	int i;
 	for (i = 0; i < array->nr; i++)
 		object_array_release_entry(&array->objects[i]);
-	free(array->objects);
-	array->objects = NULL;
+	FREEZ(array->objects);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/pathspec.c b/pathspec.c
index 828405021f..ff131a3c02 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -662,7 +662,6 @@ void clear_pathspec(struct pathspec *pathspec)
 			attr_check_free(pathspec->items[i].attr_check);
 	}
 
-	free(pathspec->items);
-	pathspec->items = NULL;
+	FREEZ(pathspec->items);
 	pathspec->nr = 0;
 }
diff --git a/prio-queue.c b/prio-queue.c
index fc3860fdcb..2c16240104 100644
--- a/prio-queue.c
+++ b/prio-queue.c
@@ -27,10 +27,9 @@ void prio_queue_reverse(struct prio_queue *queue)
 
 void clear_prio_queue(struct prio_queue *queue)
 {
-	free(queue->array);
+	FREEZ(queue->array);
 	queue->nr = 0;
 	queue->alloc = 0;
-	queue->array = NULL;
 	queue->insertion_ctr = 0;
 }
 
diff --git a/read-cache.c b/read-cache.c
index bc156a133e..5d3447d618 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1888,8 +1888,7 @@ int discard_index(struct index_state *istate)
 	free_name_hash(istate);
 	cache_tree_free(&(istate->cache_tree));
 	istate->initialized = 0;
-	free(istate->cache);
-	istate->cache = NULL;
+	FREEZ(istate->cache);
 	istate->cache_alloc = 0;
 	discard_split_index(istate);
 	free_untracked_cache(istate->untracked);
@@ -2603,8 +2602,7 @@ void *read_blob_data_from_index(const struct index_state *istate,
 
 void stat_validity_clear(struct stat_validity *sv)
 {
-	free(sv->sd);
-	sv->sd = NULL;
+	FREEZ(sv->sd);
 }
 
 int stat_validity_check(struct stat_validity *sv, const char *path)
diff --git a/ref-filter.c b/ref-filter.c
index ab32bc9c31..c9ca76d0d9 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1891,8 +1891,7 @@ void ref_array_clear(struct ref_array *array)
 
 	for (i = 0; i < array->nr; i++)
 		free_array_item(array->items[i]);
-	free(array->items);
-	array->items = NULL;
+	FREEZ(array->items);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index d8b3f73147..f088c7b9fa 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2944,8 +2944,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 				       head_oid.hash, &head_type);
 
 	if (head_ref && !(head_type & REF_ISSYMREF)) {
-		free(head_ref);
-		head_ref = NULL;
+		FREEZ(head_ref);
 	}
 
 	/*
diff --git a/refs/ref-cache.c b/refs/ref-cache.c
index af2fcb2c12..2726d5fd77 100644
--- a/refs/ref-cache.c
+++ b/refs/ref-cache.c
@@ -82,9 +82,8 @@ static void clear_ref_dir(struct ref_dir *dir)
 	int i;
 	for (i = 0; i < dir->nr; i++)
 		free_ref_entry(dir->entries[i]);
-	free(dir->entries);
+	FREEZ(dir->entries);
 	dir->sorted = dir->nr = dir->alloc = 0;
-	dir->entries = NULL;
 }
 
 struct ref_entry *create_dir_entry(struct ref_cache *cache,
diff --git a/remote-testsvn.c b/remote-testsvn.c
index f87bf851ba..79bfb2a1b7 100644
--- a/remote-testsvn.c
+++ b/remote-testsvn.c
@@ -66,8 +66,7 @@ static char *read_ref_note(const unsigned char sha1[20])
 	else if (!msglen || type != OBJ_BLOB) {
 		error("Note contains unusable content. "
 			"Is something else using this notes tree? %s", notes_ref);
-		free(msg);
-		msg = NULL;
+		FREEZ(msg);
 	}
 	free_notes(NULL);
 	return msg;
diff --git a/rerere.c b/rerere.c
index 3bd55caf3b..7dbb29f224 100644
--- a/rerere.c
+++ b/rerere.c
@@ -39,9 +39,8 @@ static void free_rerere_dirs(void)
 		free(rerere_dir[i]->status);
 		free(rerere_dir[i]);
 	}
-	free(rerere_dir);
+	FREEZ(rerere_dir);
 	rerere_dir_nr = rerere_dir_alloc = 0;
-	rerere_dir = NULL;
 }
 
 static void free_rerere_id(struct string_list_item *item)
diff --git a/sequencer.c b/sequencer.c
index 924fb1d0c3..c3f83e3a01 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1211,8 +1211,7 @@ struct todo_list {
 static void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
-	free(todo_list->items);
-	todo_list->items = NULL;
+	FREEZ(todo_list->items);
 	todo_list->nr = todo_list->alloc = 0;
 }
 
diff --git a/sha1-array.c b/sha1-array.c
index 7d646ab5b8..a683198cd7 100644
--- a/sha1-array.c
+++ b/sha1-array.c
@@ -35,8 +35,7 @@ int oid_array_lookup(struct oid_array *array, const struct object_id *oid)
 
 void oid_array_clear(struct oid_array *array)
 {
-	free(array->oid);
-	array->oid = NULL;
+	FREEZ(array->oid);
 	array->nr = 0;
 	array->alloc = 0;
 	array->sorted = 0;
diff --git a/sha1_file.c b/sha1_file.c
index 59a4ed2ed3..7758a34ff4 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -610,8 +610,7 @@ char *compute_alternate_path(const char *path, struct strbuf *err)
 
 out:
 	if (seen_error) {
-		free(ref_git);
-		ref_git = NULL;
+		FREEZ(ref_git);
 	}
 
 	return ref_git;
diff --git a/split-index.c b/split-index.c
index 49bd197f71..cbcb6eb63f 100644
--- a/split-index.c
+++ b/split-index.c
@@ -174,10 +174,9 @@ void merge_base_index(struct index_state *istate)
 
 	ewah_free(si->delete_bitmap);
 	ewah_free(si->replace_bitmap);
-	free(si->saved_cache);
+	FREEZ(si->saved_cache);
 	si->delete_bitmap  = NULL;
 	si->replace_bitmap = NULL;
-	si->saved_cache	   = NULL;
 	si->saved_cache_nr = 0;
 }
 
diff --git a/transport-helper.c b/transport-helper.c
index 36408046eb..c20725777d 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -242,8 +242,7 @@ static int disconnect_helper(struct transport *transport)
 		close(data->helper->out);
 		fclose(data->out);
 		res = finish_command(data->helper);
-		free(data->helper);
-		data->helper = NULL;
+		FREEZ(data->helper);
 	}
 	return res;
 }
@@ -711,43 +710,35 @@ static int push_update_ref_status(struct strbuf *buf,
 
 		if (!strcmp(msg, "no match")) {
 			status = REF_STATUS_NONE;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 		else if (!strcmp(msg, "up to date")) {
 			status = REF_STATUS_UPTODATE;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 		else if (!strcmp(msg, "non-fast forward")) {
 			status = REF_STATUS_REJECT_NONFASTFORWARD;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 		else if (!strcmp(msg, "already exists")) {
 			status = REF_STATUS_REJECT_ALREADY_EXISTS;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 		else if (!strcmp(msg, "fetch first")) {
 			status = REF_STATUS_REJECT_FETCH_FIRST;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 		else if (!strcmp(msg, "needs force")) {
 			status = REF_STATUS_REJECT_NEEDS_FORCE;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 		else if (!strcmp(msg, "stale info")) {
 			status = REF_STATUS_REJECT_STALE;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 		else if (!strcmp(msg, "forced update")) {
 			forced = 1;
-			free(msg);
-			msg = NULL;
+			FREEZ(msg);
 		}
 	}
 
diff --git a/transport.c b/transport.c
index 9bfcf870f9..e0fd45f614 100644
--- a/transport.c
+++ b/transport.c
@@ -1145,8 +1145,7 @@ void transport_unlock_pack(struct transport *transport)
 {
 	if (transport->pack_lockfile) {
 		unlink_or_warn(transport->pack_lockfile);
-		free(transport->pack_lockfile);
-		transport->pack_lockfile = NULL;
+		FREEZ(transport->pack_lockfile);
 	}
 }
 
diff --git a/tree-diff.c b/tree-diff.c
index e164e532b2..36d7455aa4 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -140,8 +140,7 @@ static struct combine_diff_path *path_appendnew(struct combine_diff_path *last,
 	/* if last->next is !NULL - it is a pre-allocated memory, we can reuse */
 	p = last->next;
 	if (p && (alloclen > (intptr_t)p->next)) {
-		free(p);
-		p = NULL;
+		FREEZ(p);
 	}
 
 	if (!p) {
@@ -559,8 +558,7 @@ struct combine_diff_path *diff_tree_paths(
 	 * (see path_appendnew() for details about why)
 	 */
 	if (p->next) {
-		free(p->next);
-		p->next = NULL;
+		FREEZ(p->next);
 	}
 
 	return p;
diff --git a/tree-walk.c b/tree-walk.c
index 6a42e402b0..26640e00fa 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -479,8 +479,7 @@ int traverse_trees(int n, struct tree_desc *t, struct traverse_info *info)
 	for (i = 0; i < n; i++)
 		free_extended_entry(tx + i);
 	free(tx);
-	free(traverse_path);
-	info->traverse_path = NULL;
+	FREEZ(traverse_path);
 	strbuf_release(&base);
 	return error;
 }
diff --git a/tree.c b/tree.c
index 603b29ee80..1bf175d13d 100644
--- a/tree.c
+++ b/tree.c
@@ -226,8 +226,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)
 
 void free_tree_buffer(struct tree *tree)
 {
-	free(tree->buffer);
-	tree->buffer = NULL;
+	FREEZ(tree->buffer);
 	tree->size = 0;
 	tree->object.parsed = 0;
 }
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-09 22:04       ` [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
@ 2017-06-09 22:27         ` Jonathan Nieder
  2017-06-09 23:37           ` Eric Wong
  2017-06-10  6:55           ` [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL Andreas Schwab
  0 siblings, 2 replies; 214+ messages in thread
From: Jonathan Nieder @ 2017-06-09 22:27 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Brandon Williams, Christian Couder,
	Martin Ågren

Hi,

Ævar Arnfjörð Bjarmason wrote:

> Add a FREEZ() wrapper marco for the common pattern of freeing a
> pointer and assigning NULL to it right afterwards.

I'm conflicted.  On one hand it makes code more concise and makes it
easier for people to remember to assign NULL after freeing a variable.
On the other hand it makes git more of a custom dialect of C, which
may make the code harder to read and hack on for new contributors.

My feeling is that the costs outweigh the benefits, but I haven't
thought it through thoroughly.

> The implementation is similar to the (currently unused) XDL_PTRFREE
> macro in xdiff/xmacros.h added in commit 3443546f6e ("Use a *real*
> built-in diff generator", 2006-03-24). The only difference is that
> free() is called unconditionally, see [1].
>
> 1. <alpine.DEB.2.20.1608301948310.129229@virtualbox>
>    (http://public-inbox.org/git/alpine.DEB.2.20.1608301948310.129229@virtualbox/)

Nit: it's redundant to state the message-id twice.

Can this commit message include a summary of that conversation so
someone trying to understand this patch with "git log" does not have
to open a browser and re-read the thread?  Is that thread being cited
for the point that git uses 'free(NULL)' without worrying?  I think
that's okay to say without citation --- there are lots of examples
throughout the git codebase.

> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  git-compat-util.h | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/git-compat-util.h b/git-compat-util.h
> index 4b7dcf21ad..ba2d0c8c80 100644
> --- a/git-compat-util.h
> +++ b/git-compat-util.h
> @@ -805,6 +805,12 @@ extern int xmkstemp_mode(char *template, int mode);
>  extern char *xgetcwd(void);
>  extern FILE *fopen_for_writing(const char *path);
> 
> +/*
> + * FREEZ(ptr) is like free(ptr) followed by ptr = NULL. Note that ptr
> + * is used twice, so don't pass e.g. ptr++.
> + */
> +#define FREEZ(p) do { free(p); (p) = NULL; } while (0)
> +
>  #define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), (alloc)))
>  #define REALLOC_ARRAY(x, alloc) (x) = xrealloc((x), st_mult(sizeof(*(x)), (alloc)))

Golfing: it's possible to do this without ptr being used twice by
introducing a helper function:

	static inline void freez_impl(void **p) {
		free(*p);
		*p = NULL;
	}
	#define FREEZ(p) freez_impl(&(p))

That way side-effectful callers like FREEZ(func() ? a : b) would
work.

Another request: if rolling out another version of this series, could
it go in a new thread?  The cover letter can link to the old version
for context.  Each time I see a new reply to the repository object
series my heart leaps for a moment, in the hope that we're getting
closer to have a repository object in git.

I kind of wish that 'free' returned NULL so that callers could do

	p = free(p);

without requiring a custom helper.  We could introduce a free_wrapper
that works that way but that is probably not worth it, either.

Thanks for an interesting and pleasant read.

Sincerely,
Jonathan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-09 22:27         ` Jonathan Nieder
@ 2017-06-09 23:37           ` Eric Wong
  2017-06-10  1:40             ` Junio C Hamano
  2017-06-10  6:55           ` [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL Andreas Schwab
  1 sibling, 1 reply; 214+ messages in thread
From: Eric Wong @ 2017-06-09 23:37 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	Brandon Williams, Christian Couder, Martin Ågren

Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi,
> 
> Ævar Arnfjörð Bjarmason wrote:
> 
> > Add a FREEZ() wrapper marco for the common pattern of freeing a
> > pointer and assigning NULL to it right afterwards.
> 
> I'm conflicted.  On one hand it makes code more concise and makes it
> easier for people to remember to assign NULL after freeing a variable.
> On the other hand it makes git more of a custom dialect of C, which
> may make the code harder to read and hack on for new contributors.

I think this problem could be avoided by using a more explicit
name, perhaps: "free_and_null"

Seeing the initial subject, I thought this series was short for
"freeze" (like "creat").

However, I admit FREEZ caught my eye because I thought it was
a way to freeze a repository, somehow :)

> My feeling is that the costs outweigh the benefits, but I haven't
> thought it through thoroughly.

<snip>

> > index 4b7dcf21ad..ba2d0c8c80 100644
> > --- a/git-compat-util.h
> > +++ b/git-compat-util.h
> > @@ -805,6 +805,12 @@ extern int xmkstemp_mode(char *template, int mode);
> >  extern char *xgetcwd(void);
> >  extern FILE *fopen_for_writing(const char *path);
> > 
> > +/*
> > + * FREEZ(ptr) is like free(ptr) followed by ptr = NULL. Note that ptr
> > + * is used twice, so don't pass e.g. ptr++.
> > + */
> > +#define FREEZ(p) do { free(p); (p) = NULL; } while (0)
> > +
> >  #define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), (alloc)))
> >  #define REALLOC_ARRAY(x, alloc) (x) = xrealloc((x), st_mult(sizeof(*(x)), (alloc)))
> 
> Golfing: it's possible to do this without ptr being used twice by
> introducing a helper function:
> 
> 	static inline void freez_impl(void **p) {
> 		free(*p);
> 		*p = NULL;
> 	}

Yes.  I think it's prudent to avoid macros in case there are
side effects.

> 	#define FREEZ(p) freez_impl(&(p))
> 
> That way side-effectful callers like FREEZ(func() ? a : b) would
> work.

I don't see the point of a macro wrapper, forcing the user to
type out the '&' should drive home the point that the pointer
gets set to NULL.  I also find capitalization tiring-to-read
because all characters are the same height.

<snip>

> I kind of wish that 'free' returned NULL so that callers could do
> 
> 	p = free(p);
> 
> without requiring a custom helper.  We could introduce a free_wrapper
> that works that way but that is probably not worth it, either.

Sometimes I have wished similar things, too, but that means the
same identifier shows up twice in one line and camouflages the
code.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (31 preceding siblings ...)
  2017-06-08 23:41   ` [PATCH v2 32/32] ls-files: use repository object Brandon Williams
@ 2017-06-10  0:40   ` Jonathan Tan
  2017-06-10  6:07     ` Jeff King
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
  33 siblings, 1 reply; 214+ messages in thread
From: Jonathan Tan @ 2017-06-10  0:40 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On Thu,  8 Jun 2017 16:40:28 -0700
Brandon Williams <bmwill@google.com> wrote:

> When I sent out my RFC series there seemed to be a lot of interest but I
> haven't seen many people jump to review this series.  Despite lack of review I
> wanted to get out another version which includes some changes to fix things
> that were bugging me about the series.  Hopfully this v2 will prod some more
> people to take a look.
> 
> The meat of the series is really from 04-15 and patch 32 which converts
> ls-files to recurse using a repository object.  So if you're pressed for time
> you can focus on those patches.

Thanks - I can see in patch 32 that one immediate benefit is that
ls-files can now recurse into submodules without needing to invoke an
extra process just to change the GIT_DIR.

Before I get into the details, I have some questions:

1. I am concerned that "struct repository" will end up growing without
bounds as we store more and more repo-specific concerns in it. Could it
be restricted to just the fields populated by repo_init()?
repo_read_index() will then return the index itself, instead of using
"struct repository" as a cache. This means that code using
repo_read_index() will need to maintain its own variable holding the
returned index, but that is likely a positive - it's better for code to
just pass around the specific thing needed between functions anyway, as
opposed to passing a giant "struct repository" (which partially defeats
the purpose of eliminating the usage of globals).

2. If we do the above, another potential benefit is that (I think) the
repo_config_get_.* functions would no longer be necessary, since we
would have a function that generates a "struct config_set" given a repo,
and then we just use the git_configset_get_.* functions on it. This
would also reduce the urgency of extracting the config methods into its
own header file, and reduce the size of this patch set.

(I was also going to suggest that you remove the "convert" patches, but
that is not possible - ls-files uses get_cached_convert_stats_ascii()
from convert.h despite not #include-ing it.)

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-09 23:37           ` Eric Wong
@ 2017-06-10  1:40             ` Junio C Hamano
  2017-06-10  3:21               ` Eric Wong
  0 siblings, 1 reply; 214+ messages in thread
From: Junio C Hamano @ 2017-06-10  1:40 UTC (permalink / raw)
  To: Eric Wong
  Cc: Jonathan Nieder, Ævar Arnfjörð Bjarmason, git,
	Brandon Williams, Christian Couder, Martin Ågren

Eric Wong <e@80x24.org> writes:

> Jonathan Nieder <jrnieder@gmail.com> wrote:
>> Hi,
>> 
>> Ævar Arnfjörð Bjarmason wrote:
>> 
>> > Add a FREEZ() wrapper marco for the common pattern of freeing a
>> > pointer and assigning NULL to it right afterwards.
>> 
>> I'm conflicted.  On one hand it makes code more concise and makes it
>> easier for people to remember to assign NULL after freeing a variable.
>> On the other hand it makes git more of a custom dialect of C, which
>> may make the code harder to read and hack on for new contributors.
>
> I think this problem could be avoided by using a more explicit
> name, perhaps: "free_and_null"
>
> Seeing the initial subject, I thought this series was short for
> "freeze" (like "creat").

Both match my thoughts exactly ;-)

> ...
> I don't see the point of a macro wrapper, forcing the user to
> type out the '&' should drive home the point that the pointer
> gets set to NULL.  I also find capitalization tiring-to-read
> because all characters are the same height.

Sounds sensible.

So make Jonathan's freez_impl a public API and rename it to
free_and_null(), perhaps?

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-10  1:40             ` Junio C Hamano
@ 2017-06-10  3:21               ` Eric Wong
  2017-06-10  7:25                 ` Jeff King
  0 siblings, 1 reply; 214+ messages in thread
From: Eric Wong @ 2017-06-10  3:21 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jonathan Nieder, Ævar Arnfjörð Bjarmason, git,
	Brandon Williams, Christian Couder, Martin Ågren

Junio C Hamano <gitster@pobox.com> wrote:
> Eric Wong <e@80x24.org> writes:
> > I don't see the point of a macro wrapper, forcing the user to
> > type out the '&' should drive home the point that the pointer
> > gets set to NULL.  I also find capitalization tiring-to-read
> > because all characters are the same height.
> 
> Sounds sensible.
> 
> So make Jonathan's freez_impl a public API and rename it to
> free_and_null(), perhaps?

Perhaps...  I think it needs to take "void *" to avoid warnings:

	static inline void free_and_null(void *ptrptr)
	{
		void **tmp = ptrptr;

		free(*tmp);
		*tmp = NULL;
	}

...At least I needed to do that for "mog_free_and_null" in
cmogstored:

	https://bogomips.org/cmogstored.git/plain/alloc.c?h=v1.6.0

But heck, maybe that's covering up for something else I got
wrong in cmogstored *shrug*    I don't know all of C.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 32/32] ls-files: use repository object
  2017-06-09  0:08     ` Brandon Williams
@ 2017-06-10  5:59       ` Junio C Hamano
  2017-06-12 21:24         ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Junio C Hamano @ 2017-06-10  5:59 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, peff, git, avarab, jonathantanmy

Brandon Williams <bmwill@google.com> writes:

>> -	if (read_cache() < 0)
>> +	if (repo_read_index(the_repository))
>
> oops, looks like this should have been:

Queued, with this fix-up already applied.

It had a bit of conflicts with topics in flight that may added some
more globals, so please make sure that the conflict resolution is
sane (i.e. do not look only at bw/repo-object and be happy; rather,
please check "pu^{/^Merge branch 'bw/repo-object' into pu}".

Thanks.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-10  0:40   ` [PATCH v2 00/32] " Jonathan Tan
@ 2017-06-10  6:07     ` Jeff King
  2017-06-10  6:13       ` Jeff King
                         ` (3 more replies)
  0 siblings, 4 replies; 214+ messages in thread
From: Jeff King @ 2017-06-10  6:07 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Brandon Williams, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On Fri, Jun 09, 2017 at 05:40:34PM -0700, Jonathan Tan wrote:

> Before I get into the details, I have some questions:
> 
> 1. I am concerned that "struct repository" will end up growing without
> bounds as we store more and more repo-specific concerns in it. Could it
> be restricted to just the fields populated by repo_init()?
> repo_read_index() will then return the index itself, instead of using
> "struct repository" as a cache. This means that code using
> repo_read_index() will need to maintain its own variable holding the
> returned index, but that is likely a positive - it's better for code to
> just pass around the specific thing needed between functions anyway, as
> opposed to passing a giant "struct repository" (which partially defeats
> the purpose of eliminating the usage of globals).

I think the repository object has to become a kitchen sink of sorts,
because we have tons of global variables representing repo-wide config.
ls-files doesn't respect a lot of config, but what should, e.g.:

  git config core.quotepath true
  git -C submodule config core.quotepath false
  git ls-files --recurse-submodules

do?  Right now, with a separate process, we respect the submodule
version of the config. But in a single process[1] we'd need one copy of
the quote_path_fully variable for each repo object. It's tempting for
this case to say that core.quotepath from the super-project should just
take precedence, as that's where the command is issued from (and why the
heck would anybody have per-repo settings for this anyway?). But I
suspect as we get into more complicated commands that there are likely
to be config variables that are important to match to each repo.

I do agree that "pass just what the sub-function needs" is a good rule
of thumb. But the reason that these are globals in the first place is
that there are a ton of them, and they are used at the lowest levels of
call chains. So I have a feeling that we're always going to need some
big object to hold all that context when doing multi-repo operations in
a single process.

For config, in theory that could be a big "config_set" object, but
that's not quite how we treat our config. We usually parse it once into
actual variables. So really you end up with a big parsed-config object
that gets passed around, I'd think.

-Peff

[1] I wanted to see how Brandon's series behaved for this quotepath
    case, but unfortunately I couldn't get it to work in even a simple
    case.  :(

      $ git ls-files --recurse-submodules
      fatal: index file corrupt

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-10  6:07     ` Jeff King
@ 2017-06-10  6:13       ` Jeff King
  2017-06-11  0:43         ` Brandon Williams
  2017-06-11  0:35       ` Brandon Williams
                         ` (2 subsequent siblings)
  3 siblings, 1 reply; 214+ messages in thread
From: Jeff King @ 2017-06-10  6:13 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Brandon Williams, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On Sat, Jun 10, 2017 at 02:07:12AM -0400, Jeff King wrote:

> I think the repository object has to become a kitchen sink of sorts,
> because we have tons of global variables representing repo-wide config.
> ls-files doesn't respect a lot of config, but what should, e.g.:
> 
>   git config core.quotepath true
>   git -C submodule config core.quotepath false
>   git ls-files --recurse-submodules
>
> [...]
>
> [1] I wanted to see how Brandon's series behaved for this quotepath
>     case, but unfortunately I couldn't get it to work in even a simple
>     case.  :(
> 
>       $ git ls-files --recurse-submodules
>       fatal: index file corrupt

Ah, this was just hitting the bug mentioned later in the thread. With
that fix, I can see that it does indeed behave differently than the
current code:

  git config core.quotepath true
  git -C submodule config core.quotepath false
  (cd submodule &&
   echo hello >buenos_días &&
   git add .
  )
  git ls-files --recurse-submodules

shows:

  submodule/buenos_días

before the patch series, and:

  "submodule/buenos_d\303\255as"

after.

Like I said, I doubt this is a bug that anybody cares much about, but
it's hard to know what other repo-specific global-variable usage is
lurking in low-level code.

-Peff

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-09 22:27         ` Jonathan Nieder
  2017-06-09 23:37           ` Eric Wong
@ 2017-06-10  6:55           ` Andreas Schwab
  1 sibling, 0 replies; 214+ messages in thread
From: Andreas Schwab @ 2017-06-10  6:55 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	Brandon Williams, Christian Couder, Martin Ågren

On Jun 09 2017, Jonathan Nieder <jrnieder@gmail.com> wrote:

> That way side-effectful callers like FREEZ(func() ? a : b) would
> work.

Except that you cannot take the address of a non-lvalue.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-10  3:21               ` Eric Wong
@ 2017-06-10  7:25                 ` Jeff King
  2017-06-15 16:48                   ` Junio C Hamano
  0 siblings, 1 reply; 214+ messages in thread
From: Jeff King @ 2017-06-10  7:25 UTC (permalink / raw)
  To: Eric Wong
  Cc: Junio C Hamano, Jonathan Nieder,
	Ævar Arnfjörð Bjarmason, git, Brandon Williams,
	Christian Couder, Martin Ågren

On Sat, Jun 10, 2017 at 03:21:43AM +0000, Eric Wong wrote:

> > So make Jonathan's freez_impl a public API and rename it to
> > free_and_null(), perhaps?
> 
> Perhaps...  I think it needs to take "void *" to avoid warnings:
> 
> 	static inline void free_and_null(void *ptrptr)
> 	{
> 		void **tmp = ptrptr;
> 
> 		free(*tmp);
> 		*tmp = NULL;
> 	}

That unfortunately makes it very easy to get it wrong in the callers.
Both:

  free_and_null(&p);

and

  free_and_null(p);

would be accepted by the compiler, but one of them causes undefined
behavior.

Unfortunately using "void **" in the declaration doesn't work, because
C's implicit casting rules don't apply to pointer-to-pointer types.

-Peff

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-10  6:07     ` Jeff King
  2017-06-10  6:13       ` Jeff King
@ 2017-06-11  0:35       ` Brandon Williams
  2017-06-12  5:24       ` Stefan Beller
  2017-06-12 19:01       ` Jonathan Tan
  3 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-11  0:35 UTC (permalink / raw)
  To: Jeff King
  Cc: Jonathan Tan, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On 06/10, Jeff King wrote:
> On Fri, Jun 09, 2017 at 05:40:34PM -0700, Jonathan Tan wrote:
> 
> > Before I get into the details, I have some questions:
> > 
> > 1. I am concerned that "struct repository" will end up growing without
> > bounds as we store more and more repo-specific concerns in it. Could it
> > be restricted to just the fields populated by repo_init()?
> > repo_read_index() will then return the index itself, instead of using
> > "struct repository" as a cache. This means that code using
> > repo_read_index() will need to maintain its own variable holding the
> > returned index, but that is likely a positive - it's better for code to
> > just pass around the specific thing needed between functions anyway, as
> > opposed to passing a giant "struct repository" (which partially defeats
> > the purpose of eliminating the usage of globals).
> 
> I think the repository object has to become a kitchen sink of sorts,
> because we have tons of global variables representing repo-wide config.
> ls-files doesn't respect a lot of config, but what should, e.g.:
> 
>   git config core.quotepath true
>   git -C submodule config core.quotepath false
>   git ls-files --recurse-submodules
> 
> do?  Right now, with a separate process, we respect the submodule
> version of the config. But in a single process[1] we'd need one copy of
> the quote_path_fully variable for each repo object. It's tempting for
> this case to say that core.quotepath from the super-project should just
> take precedence, as that's where the command is issued from (and why the
> heck would anybody have per-repo settings for this anyway?). But I
> suspect as we get into more complicated commands that there are likely
> to be config variables that are important to match to each repo.
> 
> I do agree that "pass just what the sub-function needs" is a good rule
> of thumb. But the reason that these are globals in the first place is
> that there are a ton of them, and they are used at the lowest levels of
> call chains. So I have a feeling that we're always going to need some
> big object to hold all that context when doing multi-repo operations in
> a single process.
> 
> For config, in theory that could be a big "config_set" object, but
> that's not quite how we treat our config. We usually parse it once into
> actual variables. So really you end up with a big parsed-config object
> that gets passed around, I'd think.
> 
> -Peff
> 
> [1] I wanted to see how Brandon's series behaved for this quotepath
>     case, but unfortunately I couldn't get it to work in even a simple
>     case.  :(
> 
>       $ git ls-files --recurse-submodules
>       fatal: index file corrupt

Yeah sorry about that...I commented on patch 32 indicating that I made a
small mistake and forgot a '> 0' when checking the index.  I made a last
minute change before sending v2 out and forgot to run tests again (I'm
terrible i know!).

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-10  6:13       ` Jeff King
@ 2017-06-11  0:43         ` Brandon Williams
  2017-06-12 19:10           ` Jonathan Tan
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-11  0:43 UTC (permalink / raw)
  To: Jeff King
  Cc: Jonathan Tan, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On 06/10, Jeff King wrote:
> On Sat, Jun 10, 2017 at 02:07:12AM -0400, Jeff King wrote:
> 
> > I think the repository object has to become a kitchen sink of sorts,
> > because we have tons of global variables representing repo-wide config.
> > ls-files doesn't respect a lot of config, but what should, e.g.:
> > 
> >   git config core.quotepath true
> >   git -C submodule config core.quotepath false
> >   git ls-files --recurse-submodules
> >
> > [...]
> >
> > [1] I wanted to see how Brandon's series behaved for this quotepath
> >     case, but unfortunately I couldn't get it to work in even a simple
> >     case.  :(
> > 
> >       $ git ls-files --recurse-submodules
> >       fatal: index file corrupt
> 
> Ah, this was just hitting the bug mentioned later in the thread. With
> that fix, I can see that it does indeed behave differently than the
> current code:
> 
>   git config core.quotepath true
>   git -C submodule config core.quotepath false
>   (cd submodule &&
>    echo hello >buenos_días &&
>    git add .
>   )
>   git ls-files --recurse-submodules
> 
> shows:
> 
>   submodule/buenos_días
> 
> before the patch series, and:
> 
>   "submodule/buenos_d\303\255as"
> 
> after.
> 
> Like I said, I doubt this is a bug that anybody cares much about, but
> it's hard to know what other repo-specific global-variable usage is
> lurking in low-level code.

I disagree with a few points of what jonathan said (mostly about
removing the config from the repo object, as I like the idea of nothing
knowing about a 'config_set' object) and I think this problem could be
solved in a couple ways.

I don't think that the in-memory global variable 'quotepath' (or
whatever its called) should live in the repository object (I mean it's
already contained in the config) but rather 'quotepath' is specific to
how ls-files handles its output.  So really what should happen is you
pass a pair of objects to the ls-files machinery (or any other command's
machinery) (1) the repository object being operated on and (2) an
options struct which can be configured based on the repository.  So when
recursing you can do something like the following:

  repo_init(submodule, path_to_submodule);
  configure_opts(sub_opts, submodule, super_opts)
  ls_files(submodule, sub_opts);

This eliminates bloating 'struct repository' and would allow you to have
things configured differently in submodules (which is crazy if you ask
me, but people do crazy things).

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-10  6:07     ` Jeff King
  2017-06-10  6:13       ` Jeff King
  2017-06-11  0:35       ` Brandon Williams
@ 2017-06-12  5:24       ` Stefan Beller
  2017-06-12 21:23         ` Jeff King
  2017-06-12 19:01       ` Jonathan Tan
  3 siblings, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-06-12  5:24 UTC (permalink / raw)
  To: Jeff King
  Cc: Jonathan Tan, Brandon Williams, git@vger.kernel.org,
	Jonathan Nieder, Jacob Keller, Johannes Schindelin,
	brian m. carlson, Ben Peart, Duy Nguyen, Junio C Hamano,
	Jeff Hostetler, Ævar Arnfjörð Bjarmason

On Fri, Jun 9, 2017 at 11:07 PM, Jeff King <peff@peff.net> wrote:
> On Fri, Jun 09, 2017 at 05:40:34PM -0700, Jonathan Tan wrote:
>
>> Before I get into the details, I have some questions:
>>
>> 1. I am concerned that "struct repository" will end up growing without
>> bounds as we store more and more repo-specific concerns in it. Could it
>> be restricted to just the fields populated by repo_init()?
>> repo_read_index() will then return the index itself, instead of using
>> "struct repository" as a cache. This means that code using
>> repo_read_index() will need to maintain its own variable holding the
>> returned index, but that is likely a positive - it's better for code to
>> just pass around the specific thing needed between functions anyway, as
>> opposed to passing a giant "struct repository" (which partially defeats
>> the purpose of eliminating the usage of globals).
>
> I think the repository object has to become a kitchen sink of sorts,
> because we have tons of global variables representing repo-wide config.

AFAICT we want to operate on struct 'the_repo' and struct 'the_cmd_options'
eventually. In our use case of submodules the submodules would ignore the
settings of the main repo, but still accept guidance of the_cmd_config or
'the_config.

> So I have a feeling that we're always going to need some
> big object to hold all that context when doing multi-repo operations in
> a single process.

Well not just one big struct, but two. (or more?)

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-10  6:07     ` Jeff King
                         ` (2 preceding siblings ...)
  2017-06-12  5:24       ` Stefan Beller
@ 2017-06-12 19:01       ` Jonathan Tan
  2017-06-12 19:11         ` Brandon Williams
  3 siblings, 1 reply; 214+ messages in thread
From: Jonathan Tan @ 2017-06-12 19:01 UTC (permalink / raw)
  To: Jeff King
  Cc: Brandon Williams, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On Sat, 10 Jun 2017 02:07:12 -0400
Jeff King <peff@peff.net> wrote:

> I do agree that "pass just what the sub-function needs" is a good rule
> of thumb. But the reason that these are globals in the first place is
> that there are a ton of them, and they are used at the lowest levels of
> call chains. So I have a feeling that we're always going to need some
> big object to hold all that context when doing multi-repo operations in
> a single process.

From my experience with the codebase, it seems that most of these config
variables are static (file-local). This means that the lowest levels of
call chains could probably get away with storing per-repo configs in a
static hashmap or associative array keyed by repo (if they cannot just
pass the config around).

Having said that, if it did come to the hashmap, I probably would prefer
just putting the config in the repo object too. So maybe that is the way
to go.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-11  0:43         ` Brandon Williams
@ 2017-06-12 19:10           ` Jonathan Tan
  0 siblings, 0 replies; 214+ messages in thread
From: Jonathan Tan @ 2017-06-12 19:10 UTC (permalink / raw)
  To: Brandon Williams
  Cc: Jeff King, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On Sat, 10 Jun 2017 17:43:29 -0700
Brandon Williams <bmwill@google.com> wrote:

> I disagree with a few points of what jonathan said (mostly about
> removing the config from the repo object, as I like the idea of nothing
> knowing about a 'config_set' object) and I think this problem could be
> solved in a couple ways.

Ah...is the plan to eventually delete the git_configset_.* functions and
only have repo_config_get_.* functions? If yes, I would prefer the
in-between state to be "git_config_set_get_int(repo->configset, ...)"
instead of "repo_config_get_int(repo, ...)" to avoid the parallel set of
functions (which will make it more troublesome, for example, to add
support for a new data type) but I can see the benefits of having the
repo_config_get_.* functions too (conciseness and not having to make one
final find-and-replace when we finally complete the migration, at least)
so I don't feel too strongly about this.

> I don't think that the in-memory global variable 'quotepath' (or
> whatever its called) should live in the repository object (I mean it's
> already contained in the config) but rather 'quotepath' is specific to
> how ls-files handles its output.  So really what should happen is you
> pass a pair of objects to the ls-files machinery (or any other command's
> machinery) (1) the repository object being operated on and (2) an
> options struct which can be configured based on the repository.  So when
> recursing you can do something like the following:
> 
>   repo_init(submodule, path_to_submodule);
>   configure_opts(sub_opts, submodule, super_opts)
>   ls_files(submodule, sub_opts);
> 
> This eliminates bloating 'struct repository' and would allow you to have
> things configured differently in submodules (which is crazy if you ask
> me, but people do crazy things).

This does sound good to me.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-12 19:01       ` Jonathan Tan
@ 2017-06-12 19:11         ` Brandon Williams
  2017-06-12 20:04           ` Jonathan Tan
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-12 19:11 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Jeff King, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On 06/12, Jonathan Tan wrote:
> On Sat, 10 Jun 2017 02:07:12 -0400
> Jeff King <peff@peff.net> wrote:
> 
> > I do agree that "pass just what the sub-function needs" is a good rule
> > of thumb. But the reason that these are globals in the first place is
> > that there are a ton of them, and they are used at the lowest levels of
> > call chains. So I have a feeling that we're always going to need some
> > big object to hold all that context when doing multi-repo operations in
> > a single process.
> 
> From my experience with the codebase, it seems that most of these config
> variables are static (file-local). This means that the lowest levels of
> call chains could probably get away with storing per-repo configs in a
> static hashmap or associative array keyed by repo (if they cannot just
> pass the config around).
> 
> Having said that, if it did come to the hashmap, I probably would prefer
> just putting the config in the repo object too. So maybe that is the way
> to go.

This is how the config is already handled.  A config_set is just a
wrapper around a fancy hashmap.  Callers query using a string as a key
and are returned the value for that config option.  I say fancy because
it does stuff to handle multiple values, etc.

I'm not sure I know what you mean by config variables which are static,
are you referring to the in-memory options which are populated by
querying the config?  Those I wouldn't want to see placed in a
'repository object'.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 04/32] config: don't implicitly use gitdir
  2017-06-08 23:40   ` [PATCH v2 04/32] config: don't implicitly use gitdir Brandon Williams
@ 2017-06-12 19:57     ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-12 19:57 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

On 06/08, Brandon Williams wrote:
> Commit 2185fde56 (config: handle conditional include when $GIT_DIR is
> not set up) added a 'git_dir' field to the config_options struct.  Let's
> use this option field explicitly all the time instead of occasionally
> falling back to calling 'git_pathdup("config")' to get the path to the
> local repository configuration.  This allows 'do_git_config_sequence()'
> to not implicitly rely on global repository state.
> 
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  builtin/config.c | 2 ++
>  config.c         | 4 ++--
>  2 files changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/builtin/config.c b/builtin/config.c
> index 753c40a5c..90f49a6ee 100644
> --- a/builtin/config.c
> +++ b/builtin/config.c
> @@ -539,6 +539,8 @@ int cmd_config(int argc, const char **argv, const char *prefix)
>  		config_options.respect_includes = !given_config_source.file;
>  	else
>  		config_options.respect_includes = respect_includes_opt;
> +	if (have_git_dir())
> +		config_options.git_dir = get_git_common_dir();
>  
>  	if (end_null) {
>  		term = '\0';
> diff --git a/config.c b/config.c
> index 2390f98e3..ff09b27b8 100644
> --- a/config.c
> +++ b/config.c
> @@ -1548,8 +1548,6 @@ static int do_git_config_sequence(const struct config_options *opts,
>  
>  	if (opts->git_dir)
>  		repo_config = mkpathdup("%s/config", opts->git_dir);
> -	else if (have_git_dir())
> -		repo_config = git_pathdup("config");
>  	else
>  		repo_config = NULL;
>  
> @@ -1613,6 +1611,8 @@ static void git_config_raw(config_fn_t fn, void *data)
>  	struct config_options opts = {0};
>  
>  	opts.respect_includes = 1;
> +	if (have_git_dir())
> +		opts.git_dir = get_git_common_dir();
>  	if (git_config_with_options(fn, data, NULL, &opts) < 0)
>  		/*
>  		 * git_config_with_options() normally returns only

I missed a case in 'include_by_gitdir'.

Given that these config changes touch a lot of files (making a config.h
files and including it everyone) I think I'll break this into its own
series so that it can get reviewed and merged in faster.  This is
especially due to this series breaking pu right now.


-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-12 19:11         ` Brandon Williams
@ 2017-06-12 20:04           ` Jonathan Tan
  2017-06-12 21:28             ` Jeff King
  0 siblings, 1 reply; 214+ messages in thread
From: Jonathan Tan @ 2017-06-12 20:04 UTC (permalink / raw)
  To: Brandon Williams
  Cc: Jeff King, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On Mon, 12 Jun 2017 12:11:21 -0700
Brandon Williams <bmwill@google.com> wrote:

> On 06/12, Jonathan Tan wrote:
> > On Sat, 10 Jun 2017 02:07:12 -0400
> > Jeff King <peff@peff.net> wrote:
> > 
> > > I do agree that "pass just what the sub-function needs" is a good rule
> > > of thumb. But the reason that these are globals in the first place is
> > > that there are a ton of them, and they are used at the lowest levels of
> > > call chains. So I have a feeling that we're always going to need some
> > > big object to hold all that context when doing multi-repo operations in
> > > a single process.
> > 
> > From my experience with the codebase, it seems that most of these config
> > variables are static (file-local). This means that the lowest levels of
> > call chains could probably get away with storing per-repo configs in a
> > static hashmap or associative array keyed by repo (if they cannot just
> > pass the config around).
> > 
> > Having said that, if it did come to the hashmap, I probably would prefer
> > just putting the config in the repo object too. So maybe that is the way
> > to go.
> 
> This is how the config is already handled.  A config_set is just a
> wrapper around a fancy hashmap.  Callers query using a string as a key
> and are returned the value for that config option.  I say fancy because
> it does stuff to handle multiple values, etc.

The hashmap I meant is one wrapping the one you describe:

  repo -> configset

Or equivalently:

  repo -> (string -> value(s))

> I'm not sure I know what you mean by config variables which are static,
> are you referring to the in-memory options which are populated by
> querying the config?  Those I wouldn't want to see placed in a
> 'repository object'.

Yes. I agree that I wouldn't want to see them placed in a repository
object, but after reading Peff's e-mail, I was thinking of what happens
if a file repeatedly invokes a config-sensitive function in another
file. For example:

 a.c
  for (i = 0; i < 100; i++) b_output(repo, x);

 b.c
  void b_output(struct repository *repo, int x)
  {
   /* print the configured "b.prefix" followed by x */
  }

We probably wouldn't want to parse the repo's configset every time
b_output() is invoked, but then, where to store our parsed "b.prefix"?
The only alternatives I see is to have a static hashmap in b.c (keyed by
repo, as described above), which would work if such a situation is rare
(or can be made rare), but if it is common, maybe we have no choice but
to put it in struct repository.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-12  5:24       ` Stefan Beller
@ 2017-06-12 21:23         ` Jeff King
  0 siblings, 0 replies; 214+ messages in thread
From: Jeff King @ 2017-06-12 21:23 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Jonathan Tan, Brandon Williams, git@vger.kernel.org,
	Jonathan Nieder, Jacob Keller, Johannes Schindelin,
	brian m. carlson, Ben Peart, Duy Nguyen, Junio C Hamano,
	Jeff Hostetler, Ævar Arnfjörð Bjarmason

On Sun, Jun 11, 2017 at 10:24:12PM -0700, Stefan Beller wrote:

> On Fri, Jun 9, 2017 at 11:07 PM, Jeff King <peff@peff.net> wrote:
> > On Fri, Jun 09, 2017 at 05:40:34PM -0700, Jonathan Tan wrote:
> >
> >> Before I get into the details, I have some questions:
> >>
> >> 1. I am concerned that "struct repository" will end up growing without
> >> bounds as we store more and more repo-specific concerns in it. Could it
> >> be restricted to just the fields populated by repo_init()?
> >> repo_read_index() will then return the index itself, instead of using
> >> "struct repository" as a cache. This means that code using
> >> repo_read_index() will need to maintain its own variable holding the
> >> returned index, but that is likely a positive - it's better for code to
> >> just pass around the specific thing needed between functions anyway, as
> >> opposed to passing a giant "struct repository" (which partially defeats
> >> the purpose of eliminating the usage of globals).
> >
> > I think the repository object has to become a kitchen sink of sorts,
> > because we have tons of global variables representing repo-wide config.
> 
> AFAICT we want to operate on struct 'the_repo' and struct 'the_cmd_options'
> eventually. In our use case of submodules the submodules would ignore the
> settings of the main repo, but still accept guidance of the_cmd_config or
> 'the_config.
> 
> > So I have a feeling that we're always going to need some
> > big object to hold all that context when doing multi-repo operations in
> > a single process.
> 
> Well not just one big struct, but two. (or more?)

Right, I think you could have a separate kitchen-sink struct that isn't
the "repo" one. But now you have to pass both of those around, which is
going to get cumbersome. Almost every function is going to end up
passing around the context struct.

I almost think it would be easier to shove them all of the context into
a big global struct and "push" and "pop" contexts from a stack of
structs. That gets you the in-process benefits, though of course it's
absolutely horrible if you ever want to multi-thread across two contexts.

-Peff

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 32/32] ls-files: use repository object
  2017-06-10  5:59       ` Junio C Hamano
@ 2017-06-12 21:24         ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-12 21:24 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, peff, git, avarab, jonathantanmy

On 06/10, Junio C Hamano wrote:
> Brandon Williams <bmwill@google.com> writes:
> 
> >> -	if (read_cache() < 0)
> >> +	if (repo_read_index(the_repository))
> >
> > oops, looks like this should have been:
> 
> Queued, with this fix-up already applied.
> 
> It had a bit of conflicts with topics in flight that may added some
> more globals, so please make sure that the conflict resolution is
> sane (i.e. do not look only at bw/repo-object and be happy; rather,
> please check "pu^{/^Merge branch 'bw/repo-object' into pu}".
> 
> Thanks.

Thanks for picking this up and merging it to pu.  After some discussion
I think it may be ideal to split this thing up so that the parts that
aren't specific to the 'repository object' can be reviewed more easily
as a 32 long patch set can be daunting to review.  I also need to think
a little bit more about the breakage this series causes in pu.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 00/32] repository object
  2017-06-12 20:04           ` Jonathan Tan
@ 2017-06-12 21:28             ` Jeff King
  0 siblings, 0 replies; 214+ messages in thread
From: Jeff King @ 2017-06-12 21:28 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Brandon Williams, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, gitster, git,
	avarab

On Mon, Jun 12, 2017 at 01:04:09PM -0700, Jonathan Tan wrote:

> > I'm not sure I know what you mean by config variables which are static,
> > are you referring to the in-memory options which are populated by
> > querying the config?  Those I wouldn't want to see placed in a
> > 'repository object'.
> 
> Yes. I agree that I wouldn't want to see them placed in a repository
> object, but after reading Peff's e-mail, I was thinking of what happens
> if a file repeatedly invokes a config-sensitive function in another
> file. For example:
> 
>  a.c
>   for (i = 0; i < 100; i++) b_output(repo, x);
> 
>  b.c
>   void b_output(struct repository *repo, int x)
>   {
>    /* print the configured "b.prefix" followed by x */
>   }
> 
> We probably wouldn't want to parse the repo's configset every time
> b_output() is invoked, but then, where to store our parsed "b.prefix"?
> The only alternatives I see is to have a static hashmap in b.c (keyed by
> repo, as described above), which would work if such a situation is rare
> (or can be made rare), but if it is common, maybe we have no choice but
> to put it in struct repository.

I think besides optimization, we often parse the textual config into
variables and then _modify_ those variables based on other input (most
often command-line arguments provided by the user, but sometimes other
circumstances).

You can move the resolution to the point-of-use instead of
point-of-setting, but that's going to be a big change to how most of the
code already works.

-Peff

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-10  7:25                 ` Jeff King
@ 2017-06-15 16:48                   ` Junio C Hamano
  2017-06-15 17:13                     ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 214+ messages in thread
From: Junio C Hamano @ 2017-06-15 16:48 UTC (permalink / raw)
  To: Jeff King
  Cc: Eric Wong, Jonathan Nieder,
	Ævar Arnfjörð Bjarmason, git, Brandon Williams,
	Christian Couder, Martin Ågren

Jeff King <peff@peff.net> writes:

> On Sat, Jun 10, 2017 at 03:21:43AM +0000, Eric Wong wrote:
>
>> > So make Jonathan's freez_impl a public API and rename it to
>> > free_and_null(), perhaps?
>> 
>> Perhaps...  I think it needs to take "void *" to avoid warnings:
>> 
>> 	static inline void free_and_null(void *ptrptr)
>> 	{
>> 		void **tmp = ptrptr;
>> 
>> 		free(*tmp);
>> 		*tmp = NULL;
>> 	}
>
> That unfortunately makes it very easy to get it wrong in the callers.
> Both:
>
>   free_and_null(&p);
>
> and
>
>   free_and_null(p);
>
> would be accepted by the compiler, but one of them causes undefined
> behavior.
>
> Unfortunately using "void **" in the declaration doesn't work, because
> C's implicit casting rules don't apply to pointer-to-pointer types.

All true.  

I still think the macro FREEZ() is too confusing a name to live;
perhaps we can take Ævar's patch with s/FREEZ/FREE_AND_NULL/ and be
done with it?  By spelling it in all caps, readers will know that
there is something special going on in that macro, and Eric's
"forcing the readers to type & in front to let them be aware that
the ptr variable is being manipulated" may become less necessary.


^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL
  2017-06-15 16:48                   ` Junio C Hamano
@ 2017-06-15 17:13                     ` Ævar Arnfjörð Bjarmason
  2017-06-15 21:06                       ` [PATCH v3 0/2] Add a FREE_AND_NULL() wrapper macro Ævar Arnfjörð Bjarmason
                                         ` (2 more replies)
  0 siblings, 3 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 17:13 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jeff King, Eric Wong, Jonathan Nieder, Git Mailing List,
	Brandon Williams, Christian Couder, Martin Ågren

On Thu, Jun 15, 2017 at 6:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jeff King <peff@peff.net> writes:
>
>> On Sat, Jun 10, 2017 at 03:21:43AM +0000, Eric Wong wrote:
>>
>>> > So make Jonathan's freez_impl a public API and rename it to
>>> > free_and_null(), perhaps?
>>>
>>> Perhaps...  I think it needs to take "void *" to avoid warnings:
>>>
>>>      static inline void free_and_null(void *ptrptr)
>>>      {
>>>              void **tmp = ptrptr;
>>>
>>>              free(*tmp);
>>>              *tmp = NULL;
>>>      }
>>
>> That unfortunately makes it very easy to get it wrong in the callers.
>> Both:
>>
>>   free_and_null(&p);
>>
>> and
>>
>>   free_and_null(p);
>>
>> would be accepted by the compiler, but one of them causes undefined
>> behavior.
>>
>> Unfortunately using "void **" in the declaration doesn't work, because
>> C's implicit casting rules don't apply to pointer-to-pointer types.
>
> All true.
>
> I still think the macro FREEZ() is too confusing a name to live;
> perhaps we can take Ævar's patch with s/FREEZ/FREE_AND_NULL/ and be
> done with it?  By spelling it in all caps, readers will know that
> there is something special going on in that macro, and Eric's
> "forcing the readers to type & in front to let them be aware that
> the ptr variable is being manipulated" may become less necessary.

I'll change it to FREE_AND_NULL and submit my patch as-is, my reading
of the rest of this thread is that making it a function instead of a
macro would be interesting, but has its own caveats that are likely
better considered as part of its own series, whereas this just changes
existing code to its macro-expanded functional equivalent.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v3 0/2] Add a FREE_AND_NULL() wrapper macro
  2017-06-15 17:13                     ` Ævar Arnfjörð Bjarmason
@ 2017-06-15 21:06                       ` Ævar Arnfjörð Bjarmason
  2017-06-15 22:00                         ` Junio C Hamano
  2017-06-15 21:06                       ` [PATCH v3 1/2] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
  2017-06-15 21:07                       ` [PATCH v3 2/2] *.[ch] refactoring: make use of the FREE_AND_NULL() macro Ævar Arnfjörð Bjarmason
  2 siblings, 1 reply; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 21:06 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams, Christian Couder,
	Martin Ågren, Ævar Arnfjörð Bjarmason

On Thu, Jun 15 2017, Ævar Arnfjörð Bjarmason jotted:
> I'll change it to FREE_AND_NULL and submit my patch as-is, my reading
> of the rest of this thread is that making it a function instead of a
> macro would be interesting, but has its own caveats that are likely
> better considered as part of its own series, whereas this just changes
> existing code to its macro-expanded functional equivalent.

Here's v3 with that change. Nothing but the macro name (and comments,
commit messages etc. referring to it) have changed.

Ævar Arnfjörð Bjarmason (2):
  git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr =
    NULL
  *.[ch] refactoring: make use of the FREE_AND_NULL() macro

 alias.c                  |  6 ++----
 apply.c                  |  3 +--
 attr.c                   |  6 ++----
 blame.c                  |  3 +--
 branch.c                 |  3 +--
 builtin/am.c             | 18 +++++-------------
 builtin/clean.c          |  6 ++----
 builtin/config.c         |  6 ++----
 builtin/index-pack.c     |  6 ++----
 builtin/pack-objects.c   | 12 ++++--------
 builtin/unpack-objects.c |  3 +--
 builtin/worktree.c       |  6 ++----
 commit-slab.h            |  3 +--
 commit.c                 |  3 +--
 config.c                 |  3 +--
 credential.c             |  9 +++------
 diff-lib.c               |  3 +--
 diff.c                   |  6 ++----
 diffcore-rename.c        |  6 ++----
 dir.c                    |  9 +++------
 fast-import.c            |  6 ++----
 git-compat-util.h        |  6 ++++++
 gpg-interface.c          | 15 +++++----------
 grep.c                   | 12 ++++--------
 help.c                   |  3 +--
 http-push.c              | 24 ++++++++----------------
 http.c                   | 15 +++++----------
 imap-send.c              |  3 +--
 line-log.c               |  6 ++----
 ll-merge.c               |  3 +--
 mailinfo.c               |  3 +--
 object.c                 |  3 +--
 pathspec.c               |  3 +--
 prio-queue.c             |  3 +--
 read-cache.c             |  6 ++----
 ref-filter.c             |  3 +--
 refs/files-backend.c     |  3 +--
 refs/ref-cache.c         |  3 +--
 remote-testsvn.c         |  3 +--
 rerere.c                 |  3 +--
 sequencer.c              |  3 +--
 sha1-array.c             |  3 +--
 sha1_file.c              |  3 +--
 split-index.c            |  3 +--
 transport-helper.c       | 27 +++++++++------------------
 transport.c              |  3 +--
 tree-diff.c              |  6 ++----
 tree-walk.c              |  3 +--
 tree.c                   |  3 +--
 49 files changed, 103 insertions(+), 197 deletions(-)

-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v3 1/2] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL
  2017-06-15 17:13                     ` Ævar Arnfjörð Bjarmason
  2017-06-15 21:06                       ` [PATCH v3 0/2] Add a FREE_AND_NULL() wrapper macro Ævar Arnfjörð Bjarmason
@ 2017-06-15 21:06                       ` Ævar Arnfjörð Bjarmason
  2017-06-15 21:07                       ` [PATCH v3 2/2] *.[ch] refactoring: make use of the FREE_AND_NULL() macro Ævar Arnfjörð Bjarmason
  2 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 21:06 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams, Christian Couder,
	Martin Ågren, Ævar Arnfjörð Bjarmason

Add a FREE_AND_NULL() wrapper marco for the common pattern of freeing
a pointer and assigning NULL to it right afterwards.

The implementation is similar to the (currently unused) XDL_PTRFREE
macro in xdiff/xmacros.h added in commit 3443546f6e ("Use a *real*
built-in diff generator", 2006-03-24). The only difference is that
free() is called unconditionally, see [1].

See [2] for a suggested alternative which does this via a function
instead of a macro. As covered in replies to that message, while it's
a viable approach, it would introduce caveats which this approach
doesn't have, so that potential change is left to a future follow-up
change.

This merely allows us to translate exactly what we're doing now to a
less verbose & idiomatic form using a macro, while guaranteeing that
we don't introduce any functional changes.

1. <alpine.DEB.2.20.1608301948310.129229@virtualbox>
   (http://public-inbox.org/git/alpine.DEB.2.20.1608301948310.129229@virtualbox/)

2. <20170610032143.GA7880@starla>
   (https://public-inbox.org/git/20170610032143.GA7880@starla/)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-compat-util.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index 51ba4e6b3b..047172d173 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -808,6 +808,12 @@ extern char *xgetcwd(void);
 extern FILE *fopen_for_writing(const char *path);
 extern FILE *fopen_or_warn(const char *path, const char *mode);
 
+/*
+ * FREE_AND_NULL(ptr) is like free(ptr) followed by ptr = NULL. Note
+ * that ptr is used twice, so don't pass e.g. ptr++.
+ */
+#define FREE_AND_NULL(p) do { free(p); (p) = NULL; } while (0)
+
 #define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), (alloc)))
 #define REALLOC_ARRAY(x, alloc) (x) = xrealloc((x), st_mult(sizeof(*(x)), (alloc)))
 
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 2/2] *.[ch] refactoring: make use of the FREE_AND_NULL() macro
  2017-06-15 17:13                     ` Ævar Arnfjörð Bjarmason
  2017-06-15 21:06                       ` [PATCH v3 0/2] Add a FREE_AND_NULL() wrapper macro Ævar Arnfjörð Bjarmason
  2017-06-15 21:06                       ` [PATCH v3 1/2] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
@ 2017-06-15 21:07                       ` Ævar Arnfjörð Bjarmason
  2 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 21:07 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Brandon Williams, Christian Couder,
	Martin Ågren, Ævar Arnfjörð Bjarmason

Replace occurrences of `free(ptr); ptr = NULL` with
`FREE_AND_NULL(ptr)`. This introduces no functional changes, but
reduces the line count and establishes this pattern as a common idiom
with a wrapper macro.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 alias.c                  |  6 ++----
 apply.c                  |  3 +--
 attr.c                   |  6 ++----
 blame.c                  |  3 +--
 branch.c                 |  3 +--
 builtin/am.c             | 18 +++++-------------
 builtin/clean.c          |  6 ++----
 builtin/config.c         |  6 ++----
 builtin/index-pack.c     |  6 ++----
 builtin/pack-objects.c   | 12 ++++--------
 builtin/unpack-objects.c |  3 +--
 builtin/worktree.c       |  6 ++----
 commit-slab.h            |  3 +--
 commit.c                 |  3 +--
 config.c                 |  3 +--
 credential.c             |  9 +++------
 diff-lib.c               |  3 +--
 diff.c                   |  6 ++----
 diffcore-rename.c        |  6 ++----
 dir.c                    |  9 +++------
 fast-import.c            |  6 ++----
 gpg-interface.c          | 15 +++++----------
 grep.c                   | 12 ++++--------
 help.c                   |  3 +--
 http-push.c              | 24 ++++++++----------------
 http.c                   | 15 +++++----------
 imap-send.c              |  3 +--
 line-log.c               |  6 ++----
 ll-merge.c               |  3 +--
 mailinfo.c               |  3 +--
 object.c                 |  3 +--
 pathspec.c               |  3 +--
 prio-queue.c             |  3 +--
 read-cache.c             |  6 ++----
 ref-filter.c             |  3 +--
 refs/files-backend.c     |  3 +--
 refs/ref-cache.c         |  3 +--
 remote-testsvn.c         |  3 +--
 rerere.c                 |  3 +--
 sequencer.c              |  3 +--
 sha1-array.c             |  3 +--
 sha1_file.c              |  3 +--
 split-index.c            |  3 +--
 transport-helper.c       | 27 +++++++++------------------
 transport.c              |  3 +--
 tree-diff.c              |  6 ++----
 tree-walk.c              |  3 +--
 tree.c                   |  3 +--
 48 files changed, 97 insertions(+), 197 deletions(-)

diff --git a/alias.c b/alias.c
index 3b90397a99..911481855f 100644
--- a/alias.c
+++ b/alias.c
@@ -47,8 +47,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 				src++;
 				c = cmdline[src];
 				if (!c) {
-					free(*argv);
-					*argv = NULL;
+					FREE_AND_NULL(*argv);
 					return -SPLIT_CMDLINE_BAD_ENDING;
 				}
 			}
@@ -60,8 +59,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 	cmdline[dst] = 0;
 
 	if (quoted) {
-		free(*argv);
-		*argv = NULL;
+		FREE_AND_NULL(*argv);
 		return -SPLIT_CMDLINE_UNCLOSED_QUOTE;
 	}
 
diff --git a/apply.c b/apply.c
index 854faa6779..e78de0affa 100644
--- a/apply.c
+++ b/apply.c
@@ -3705,8 +3705,7 @@ static int check_preimage(struct apply_state *state,
  is_new:
 	patch->is_new = 1;
 	patch->is_delete = 0;
-	free(patch->old_name);
-	patch->old_name = NULL;
+	FREE_AND_NULL(patch->old_name);
 	return 0;
 }
 
diff --git a/attr.c b/attr.c
index 821203e2a9..ebdcfb0b8a 100644
--- a/attr.c
+++ b/attr.c
@@ -638,13 +638,11 @@ void attr_check_reset(struct attr_check *check)
 
 void attr_check_clear(struct attr_check *check)
 {
-	free(check->items);
-	check->items = NULL;
+	FREE_AND_NULL(check->items);
 	check->alloc = 0;
 	check->nr = 0;
 
-	free(check->all_attrs);
-	check->all_attrs = NULL;
+	FREE_AND_NULL(check->all_attrs);
 	check->all_attrs_nr = 0;
 
 	drop_attr_stack(&check->stack);
diff --git a/blame.c b/blame.c
index 843c845cba..1183943960 100644
--- a/blame.c
+++ b/blame.c
@@ -314,8 +314,7 @@ static void fill_origin_blob(struct diff_options *opt,
 static void drop_origin_blob(struct blame_origin *o)
 {
 	if (o->file.ptr) {
-		free(o->file.ptr);
-		o->file.ptr = NULL;
+		FREE_AND_NULL(o->file.ptr);
 	}
 }
 
diff --git a/branch.c b/branch.c
index 985316eb76..2347cb8649 100644
--- a/branch.c
+++ b/branch.c
@@ -24,8 +24,7 @@ static int find_tracked_branch(struct remote *remote, void *priv)
 		} else {
 			free(tracking->spec.src);
 			if (tracking->src) {
-				free(tracking->src);
-				tracking->src = NULL;
+				FREE_AND_NULL(tracking->src);
 			}
 		}
 		tracking->spec.src = NULL;
diff --git a/builtin/am.c b/builtin/am.c
index 8881d73615..80368b6fe6 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -483,8 +483,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
 	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
 
 	if (!ret) {
-		free(state->msg);
-		state->msg = NULL;
+		FREE_AND_NULL(state->msg);
 		if (read_commit_msg(state) < 0)
 			die(_("'%s' was deleted by the applypatch-msg hook"),
 				am_path(state, "final-commit"));
@@ -1073,17 +1072,10 @@ static void am_next(struct am_state *state)
 {
 	struct object_id head;
 
-	free(state->author_name);
-	state->author_name = NULL;
-
-	free(state->author_email);
-	state->author_email = NULL;
-
-	free(state->author_date);
-	state->author_date = NULL;
-
-	free(state->msg);
-	state->msg = NULL;
+	FREE_AND_NULL(state->author_name);
+	FREE_AND_NULL(state->author_email);
+	FREE_AND_NULL(state->author_date);
+	FREE_AND_NULL(state->msg);
 	state->msg_len = 0;
 
 	unlink(am_path(state, "author-script"));
diff --git a/builtin/clean.c b/builtin/clean.c
index 142bf668cf..bff5a07330 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -837,8 +837,7 @@ static void interactive_main_loop(void)
 			int ret;
 			ret = menus[*chosen].fn();
 			if (ret != MENU_RETURN_NO_LOOP) {
-				free(chosen);
-				chosen = NULL;
+				FREE_AND_NULL(chosen);
 				if (!del_list.nr) {
 					clean_print_color(CLEAN_COLOR_ERROR);
 					printf_ln(_("No more files to clean, exiting."));
@@ -851,8 +850,7 @@ static void interactive_main_loop(void)
 			quit_cmd();
 		}
 
-		free(chosen);
-		chosen = NULL;
+		FREE_AND_NULL(chosen);
 		break;
 	}
 }
diff --git a/builtin/config.c b/builtin/config.c
index 7f6c25d4d9..18c2158630 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -214,8 +214,7 @@ static int get_value(const char *key_, const char *regex_)
 		key_regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(key_regexp, key, REG_EXTENDED)) {
 			error("invalid key pattern: %s", key_);
-			free(key_regexp);
-			key_regexp = NULL;
+			FREE_AND_NULL(key_regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
@@ -235,8 +234,7 @@ static int get_value(const char *key_, const char *regex_)
 		regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(regexp, regex_, REG_EXTENDED)) {
 			error("invalid pattern: %s", regex_);
-			free(regexp);
-			regexp = NULL;
+			FREE_AND_NULL(regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 04b9dcaf0f..f3b16520c2 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -388,8 +388,7 @@ static struct base_data *alloc_base_data(void)
 static void free_base_data(struct base_data *c)
 {
 	if (c->data) {
-		free(c->data);
-		c->data = NULL;
+		FREE_AND_NULL(c->data);
 		get_thread_data()->base_cache_used -= c->size;
 	}
 }
@@ -605,8 +604,7 @@ static void *unpack_data(struct object_entry *obj,
 	git_inflate_end(&stream);
 	free(inbuf);
 	if (consume) {
-		free(data);
-		data = NULL;
+		FREE_AND_NULL(data);
 	}
 	return data;
 }
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f672225def..790615b2c3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -264,8 +264,7 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 		 * make sure no cached delta data remains from a
 		 * previous attempt before a pack split occurred.
 		 */
-		free(entry->delta_data);
-		entry->delta_data = NULL;
+		FREE_AND_NULL(entry->delta_data);
 		entry->z_delta_size = 0;
 	} else if (entry->delta_data) {
 		size = entry->delta_size;
@@ -1375,12 +1374,10 @@ static void cleanup_preferred_base(void)
 		if (!pbase_tree_cache[i])
 			continue;
 		free(pbase_tree_cache[i]->tree_data);
-		free(pbase_tree_cache[i]);
-		pbase_tree_cache[i] = NULL;
+		FREE_AND_NULL(pbase_tree_cache[i]);
 	}
 
-	free(done_pbase_paths);
-	done_pbase_paths = NULL;
+	FREE_AND_NULL(done_pbase_paths);
 	done_pbase_paths_num = done_pbase_paths_alloc = 0;
 }
 
@@ -1970,8 +1967,7 @@ static unsigned long free_unpacked(struct unpacked *n)
 	n->index = NULL;
 	if (n->data) {
 		freed_mem += n->entry->size;
-		free(n->data);
-		n->data = NULL;
+		FREE_AND_NULL(n->data);
 	}
 	n->entry = NULL;
 	n->depth = 0;
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 8bc9997767..60bfe48789 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -112,8 +112,7 @@ static void *get_data(unsigned long size)
 			break;
 		if (ret != Z_OK) {
 			error("inflate returned %d", ret);
-			free(buf);
-			buf = NULL;
+			FREE_AND_NULL(buf);
 			if (!recover)
 				exit(1);
 			has_errors = 1;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 793306ea51..41d1c007a4 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -299,10 +299,8 @@ static int add_worktree(const char *path, const char *refname,
 	}
 
 	is_junk = 0;
-	free(junk_work_tree);
-	free(junk_git_dir);
-	junk_work_tree = NULL;
-	junk_git_dir = NULL;
+	FREE_AND_NULL(junk_work_tree);
+	FREE_AND_NULL(junk_git_dir);
 
 done:
 	if (ret || !opts->keep_locked) {
diff --git a/commit-slab.h b/commit-slab.h
index 42d16dcded..3286c3581b 100644
--- a/commit-slab.h
+++ b/commit-slab.h
@@ -82,8 +82,7 @@ static MAYBE_UNUSED void clear_ ##slabname(struct slabname *s)		\
 	for (i = 0; i < s->slab_count; i++)				\
 		free(s->slab[i]);					\
 	s->slab_count = 0;						\
-	free(s->slab);							\
-	s->slab = NULL;							\
+	FREE_AND_NULL(s->slab);							\
 }									\
 									\
 static MAYBE_UNUSED elemtype *slabname## _at_peek(struct slabname *s,	\
diff --git a/commit.c b/commit.c
index 99846d9bf4..cbfd689939 100644
--- a/commit.c
+++ b/commit.c
@@ -287,8 +287,7 @@ void free_commit_buffer(struct commit *commit)
 {
 	struct commit_buffer *v = buffer_slab_peek(&buffer_slab, commit);
 	if (v) {
-		free(v->buffer);
-		v->buffer = NULL;
+		FREE_AND_NULL(v->buffer);
 		v->size = 0;
 	}
 }
diff --git a/config.c b/config.c
index 34a139c40b..0501a60dd2 100644
--- a/config.c
+++ b/config.c
@@ -395,8 +395,7 @@ static int git_config_parse_key_1(const char *key, char **store_key, int *basele
 
 out_free_ret_1:
 	if (store_key) {
-		free(*store_key);
-		*store_key = NULL;
+		FREE_AND_NULL(*store_key);
 	}
 	return -CONFIG_INVALID_KEY;
 }
diff --git a/credential.c b/credential.c
index aa996669fc..924ab58538 100644
--- a/credential.c
+++ b/credential.c
@@ -93,8 +93,7 @@ static void credential_apply_config(struct credential *c)
 	c->configured = 1;
 
 	if (!c->use_http_path && proto_is_http(c->protocol)) {
-		free(c->path);
-		c->path = NULL;
+		FREE_AND_NULL(c->path);
 	}
 }
 
@@ -314,10 +313,8 @@ void credential_reject(struct credential *c)
 	for (i = 0; i < c->helpers.nr; i++)
 		credential_do(c, c->helpers.items[i].string, "erase");
 
-	free(c->username);
-	c->username = NULL;
-	free(c->password);
-	c->password = NULL;
+	FREE_AND_NULL(c->username);
+	FREE_AND_NULL(c->password);
 	c->approved = 0;
 }
 
diff --git a/diff-lib.c b/diff-lib.c
index 76c8f185cd..7528f4163f 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -179,8 +179,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)
 				free(dpath);
 				continue;
 			}
-			free(dpath);
-			dpath = NULL;
+			FREE_AND_NULL(dpath);
 
 			/*
 			 * Show the diff for the 'ce' if we found the one
diff --git a/diff.c b/diff.c
index e35cf6c704..3561adf49f 100644
--- a/diff.c
+++ b/diff.c
@@ -1218,8 +1218,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)
 			regfree(ecbdata->diff_words->word_regex);
 			free(ecbdata->diff_words->word_regex);
 		}
-		free(ecbdata->diff_words);
-		ecbdata->diff_words = NULL;
+		FREE_AND_NULL(ecbdata->diff_words);
 	}
 }
 
@@ -2951,8 +2950,7 @@ void diff_free_filespec_blob(struct diff_filespec *s)
 void diff_free_filespec_data(struct diff_filespec *s)
 {
 	diff_free_filespec_blob(s);
-	free(s->cnt_data);
-	s->cnt_data = NULL;
+	FREE_AND_NULL(s->cnt_data);
 }
 
 static void prep_temp_blob(const char *path, struct diff_tempfile *temp,
diff --git a/diffcore-rename.c b/diffcore-rename.c
index f7444c86bd..6d8daf9c00 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -667,11 +667,9 @@ void diffcore_rename(struct diff_options *options)
 	for (i = 0; i < rename_dst_nr; i++)
 		free_filespec(rename_dst[i].two);
 
-	free(rename_dst);
-	rename_dst = NULL;
+	FREE_AND_NULL(rename_dst);
 	rename_dst_nr = rename_dst_alloc = 0;
-	free(rename_src);
-	rename_src = NULL;
+	FREE_AND_NULL(rename_src);
 	rename_src_nr = rename_src_alloc = 0;
 	return;
 }
diff --git a/dir.c b/dir.c
index 1759063817..7dc168b806 100644
--- a/dir.c
+++ b/dir.c
@@ -2117,8 +2117,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		for (i = j = 0; j < dir->nr; j++) {
 			if (i &&
 			    check_dir_entry_contains(dir->entries[i - 1], dir->entries[j])) {
-				free(dir->entries[j]);
-				dir->entries[j] = NULL;
+				FREE_AND_NULL(dir->entries[j]);
 			} else {
 				dir->entries[i++] = dir->entries[j];
 			}
@@ -2144,8 +2143,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		     dir->untracked->dir_invalidated))
 			istate->cache_changed |= UNTRACKED_CHANGED;
 		if (dir->untracked != istate->untracked) {
-			free(dir->untracked);
-			dir->untracked = NULL;
+			FREE_AND_NULL(dir->untracked);
 		}
 	}
 	return dir->nr;
@@ -2488,8 +2486,7 @@ void write_untracked_extension(struct strbuf *out, struct untracked_cache *untra
 	strbuf_addbuf(out, &untracked->ident);
 
 	strbuf_add(out, ouc, ouc_size(len));
-	free(ouc);
-	ouc = NULL;
+	FREE_AND_NULL(ouc);
 
 	if (!untracked->root) {
 		varint_len = encode_varint(0, varbuf);
diff --git a/fast-import.c b/fast-import.c
index 9a22fc92c0..eeab927d5a 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1063,8 +1063,7 @@ static void end_packfile(void)
 		close(pack_data->pack_fd);
 		unlink_or_warn(pack_data->pack_name);
 	}
-	free(pack_data);
-	pack_data = NULL;
+	FREE_AND_NULL(pack_data);
 	running = 0;
 
 	/* We can't carry a delta across packfiles. */
@@ -1149,8 +1148,7 @@ static int store_object(
 
 		/* We cannot carry a delta into the new pack. */
 		if (delta) {
-			free(delta);
-			delta = NULL;
+			FREE_AND_NULL(delta);
 
 			git_deflate_init(&s, pack_compression_level);
 			s.next_in = (void *)dat->buf;
diff --git a/gpg-interface.c b/gpg-interface.c
index e44cc27da1..3f377f89d0 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -13,16 +13,11 @@ static const char *gpg_program = "gpg";
 
 void signature_check_clear(struct signature_check *sigc)
 {
-	free(sigc->payload);
-	free(sigc->gpg_output);
-	free(sigc->gpg_status);
-	free(sigc->signer);
-	free(sigc->key);
-	sigc->payload = NULL;
-	sigc->gpg_output = NULL;
-	sigc->gpg_status = NULL;
-	sigc->signer = NULL;
-	sigc->key = NULL;
+	FREE_AND_NULL(sigc->payload);
+	FREE_AND_NULL(sigc->gpg_output);
+	FREE_AND_NULL(sigc->gpg_status);
+	FREE_AND_NULL(sigc->signer);
+	FREE_AND_NULL(sigc->key);
 }
 
 static struct {
diff --git a/grep.c b/grep.c
index d03d424e5c..909f35a984 100644
--- a/grep.c
+++ b/grep.c
@@ -1763,12 +1763,9 @@ void grep_source_init(struct grep_source *gs, enum grep_source_type type,
 
 void grep_source_clear(struct grep_source *gs)
 {
-	free(gs->name);
-	gs->name = NULL;
-	free(gs->path);
-	gs->path = NULL;
-	free(gs->identifier);
-	gs->identifier = NULL;
+	FREE_AND_NULL(gs->name);
+	FREE_AND_NULL(gs->path);
+	FREE_AND_NULL(gs->identifier);
 	grep_source_clear_data(gs);
 }
 
@@ -1778,8 +1775,7 @@ void grep_source_clear_data(struct grep_source *gs)
 	case GREP_SOURCE_FILE:
 	case GREP_SOURCE_SHA1:
 	case GREP_SOURCE_SUBMODULE:
-		free(gs->buf);
-		gs->buf = NULL;
+		FREE_AND_NULL(gs->buf);
 		gs->size = 0;
 		break;
 	case GREP_SOURCE_BUF:
diff --git a/help.c b/help.c
index db7f3d79a0..a2f48c0df3 100644
--- a/help.c
+++ b/help.c
@@ -267,9 +267,8 @@ static void add_cmd_list(struct cmdnames *cmds, struct cmdnames *old)
 
 	for (i = 0; i < old->cnt; i++)
 		cmds->names[cmds->cnt++] = old->names[i];
-	free(old->names);
+	FREE_AND_NULL(old->names);
 	old->cnt = 0;
-	old->names = NULL;
 }
 
 /* An empirically derived magic number */
diff --git a/http-push.c b/http-push.c
index 67c4d4b472..c91f40a610 100644
--- a/http-push.c
+++ b/http-push.c
@@ -291,8 +291,7 @@ static void start_mkcol(struct transfer_request *request)
 		request->state = RUN_MKCOL;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 }
 #endif
@@ -409,8 +408,7 @@ static void start_put(struct transfer_request *request)
 		request->state = RUN_PUT;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 }
 
@@ -432,8 +430,7 @@ static void start_move(struct transfer_request *request)
 		request->state = RUN_MOVE;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 }
 
@@ -526,8 +523,7 @@ static void finish_request(struct transfer_request *request)
 
 	/* URL is reused for MOVE after PUT */
 	if (request->state != RUN_PUT) {
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 
 	if (request->state == RUN_MKCOL) {
@@ -803,8 +799,7 @@ xml_start_tag(void *userData, const char *name, const char **atts)
 	}
 	xsnprintf(ctx->name + old_namelen, ctx->len - old_namelen, ".%s", c);
 
-	free(ctx->cdata);
-	ctx->cdata = NULL;
+	FREE_AND_NULL(ctx->cdata);
 
 	ctx->userFunc(ctx, 0);
 }
@@ -932,8 +927,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)
 		free(lock->token);
 		free(lock->owner);
 		free(url);
-		free(lock);
-		lock = NULL;
+		FREE_AND_NULL(lock);
 	} else {
 		lock->url = url;
 		lock->start_time = time(NULL);
@@ -1105,8 +1099,7 @@ static void handle_remote_ls_ctx(struct xml_ctx *ctx, int tag_closed)
 			ls->dentry_flags |= IS_DIR;
 		}
 	} else if (!strcmp(ctx->name, DAV_PROPFIND_RESP)) {
-		free(ls->dentry_name);
-		ls->dentry_name = NULL;
+		FREE_AND_NULL(ls->dentry_name);
 		ls->dentry_flags = 0;
 	}
 }
@@ -1547,8 +1540,7 @@ static void fetch_symref(const char *path, char **symref, struct object_id *oid)
 		    curl_errorstr);
 	free(url);
 
-	free(*symref);
-	*symref = NULL;
+	FREE_AND_NULL(*symref);
 	oidclr(oid);
 
 	if (buffer.len == 0)
diff --git a/http.c b/http.c
index d2e11ec6f0..77f46bef0f 100644
--- a/http.c
+++ b/http.c
@@ -1026,8 +1026,7 @@ void http_cleanup(void)
 
 	if (proxy_auth.password) {
 		memset(proxy_auth.password, 0, strlen(proxy_auth.password));
-		free(proxy_auth.password);
-		proxy_auth.password = NULL;
+		FREE_AND_NULL(proxy_auth.password);
 	}
 
 	free((void *)curl_proxyuserpwd);
@@ -1038,13 +1037,11 @@ void http_cleanup(void)
 
 	if (cert_auth.password != NULL) {
 		memset(cert_auth.password, 0, strlen(cert_auth.password));
-		free(cert_auth.password);
-		cert_auth.password = NULL;
+		FREE_AND_NULL(cert_auth.password);
 	}
 	ssl_cert_password_required = 0;
 
-	free(cached_accept_language);
-	cached_accept_language = NULL;
+	FREE_AND_NULL(cached_accept_language);
 }
 
 struct active_request_slot *get_active_slot(void)
@@ -1896,8 +1893,7 @@ static char *fetch_pack_index(unsigned char *sha1, const char *base_url)
 
 	if (http_get_file(url, tmp, NULL) != HTTP_OK) {
 		error("Unable to get pack index %s", url);
-		free(tmp);
-		tmp = NULL;
+		FREE_AND_NULL(tmp);
 	}
 
 	free(url);
@@ -2328,8 +2324,7 @@ void release_http_object_request(struct http_object_request *freq)
 		freq->localfile = -1;
 	}
 	if (freq->url != NULL) {
-		free(freq->url);
-		freq->url = NULL;
+		FREE_AND_NULL(freq->url);
 	}
 	if (freq->slot != NULL) {
 		freq->slot->callback_func = NULL;
diff --git a/imap-send.c b/imap-send.c
index 857591660f..40ddb496e6 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -776,8 +776,7 @@ static int get_cmd_result(struct imap_store *ctx, struct imap_cmd *tcmd)
 			       offsetof(struct imap_cmd, next));
 			if (cmdp->cb.data) {
 				n = socket_write(&imap->buf.sock, cmdp->cb.data, cmdp->cb.dlen);
-				free(cmdp->cb.data);
-				cmdp->cb.data = NULL;
+				FREE_AND_NULL(cmdp->cb.data);
 				if (n != (int)cmdp->cb.dlen)
 					return RESP_BAD;
 			} else if (cmdp->cb.cont) {
diff --git a/line-log.c b/line-log.c
index b9087814b8..2d60c347ca 100644
--- a/line-log.c
+++ b/line-log.c
@@ -34,9 +34,8 @@ void range_set_init(struct range_set *rs, size_t prealloc)
 
 void range_set_release(struct range_set *rs)
 {
-	free(rs->ranges);
+	FREE_AND_NULL(rs->ranges);
 	rs->alloc = rs->nr = 0;
-	rs->ranges = NULL;
 }
 
 /* dst must be uninitialized! */
@@ -610,8 +609,7 @@ parse_lines(struct commit *commit, const char *prefix, struct string_list *args)
 		line_log_data_insert(&ranges, full_name, begin, end);
 
 		free_filespec(spec);
-		free(ends);
-		ends = NULL;
+		FREE_AND_NULL(ends);
 	}
 
 	for (p = ranges; p; p = p->next)
diff --git a/ll-merge.c b/ll-merge.c
index ac0d4a5d78..91ff519cef 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -209,8 +209,7 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 	result->size = st.st_size;
 	result->ptr = xmallocz(result->size);
 	if (read_in_full(fd, result->ptr, result->size) != result->size) {
-		free(result->ptr);
-		result->ptr = NULL;
+		FREE_AND_NULL(result->ptr);
 		result->size = 0;
 	}
  close_bad:
diff --git a/mailinfo.c b/mailinfo.c
index f92cb9f729..44f829edb1 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -919,8 +919,7 @@ static int handle_boundary(struct mailinfo *mi, struct strbuf *line)
 		/* we hit an end boundary */
 		/* pop the current boundary off the stack */
 		strbuf_release(*(mi->content_top));
-		free(*(mi->content_top));
-		*(mi->content_top) = NULL;
+		FREE_AND_NULL(*(mi->content_top));
 
 		/* technically won't happen as is_multipart_boundary()
 		   will fail first.  But just in case..
diff --git a/object.c b/object.c
index 06ba3a11d8..f818777412 100644
--- a/object.c
+++ b/object.c
@@ -377,8 +377,7 @@ void object_array_clear(struct object_array *array)
 	int i;
 	for (i = 0; i < array->nr; i++)
 		object_array_release_entry(&array->objects[i]);
-	free(array->objects);
-	array->objects = NULL;
+	FREE_AND_NULL(array->objects);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/pathspec.c b/pathspec.c
index 828405021f..54aab8bc01 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -662,7 +662,6 @@ void clear_pathspec(struct pathspec *pathspec)
 			attr_check_free(pathspec->items[i].attr_check);
 	}
 
-	free(pathspec->items);
-	pathspec->items = NULL;
+	FREE_AND_NULL(pathspec->items);
 	pathspec->nr = 0;
 }
diff --git a/prio-queue.c b/prio-queue.c
index fc3860fdcb..126d096727 100644
--- a/prio-queue.c
+++ b/prio-queue.c
@@ -27,10 +27,9 @@ void prio_queue_reverse(struct prio_queue *queue)
 
 void clear_prio_queue(struct prio_queue *queue)
 {
-	free(queue->array);
+	FREE_AND_NULL(queue->array);
 	queue->nr = 0;
 	queue->alloc = 0;
-	queue->array = NULL;
 	queue->insertion_ctr = 0;
 }
 
diff --git a/read-cache.c b/read-cache.c
index bc156a133e..66c413657e 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1888,8 +1888,7 @@ int discard_index(struct index_state *istate)
 	free_name_hash(istate);
 	cache_tree_free(&(istate->cache_tree));
 	istate->initialized = 0;
-	free(istate->cache);
-	istate->cache = NULL;
+	FREE_AND_NULL(istate->cache);
 	istate->cache_alloc = 0;
 	discard_split_index(istate);
 	free_untracked_cache(istate->untracked);
@@ -2603,8 +2602,7 @@ void *read_blob_data_from_index(const struct index_state *istate,
 
 void stat_validity_clear(struct stat_validity *sv)
 {
-	free(sv->sd);
-	sv->sd = NULL;
+	FREE_AND_NULL(sv->sd);
 }
 
 int stat_validity_check(struct stat_validity *sv, const char *path)
diff --git a/ref-filter.c b/ref-filter.c
index ab32bc9c31..72e6cb8ecc 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1891,8 +1891,7 @@ void ref_array_clear(struct ref_array *array)
 
 	for (i = 0; i < array->nr; i++)
 		free_array_item(array->items[i]);
-	free(array->items);
-	array->items = NULL;
+	FREE_AND_NULL(array->items);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index d8b3f73147..c1bd99d60f 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2944,8 +2944,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 				       head_oid.hash, &head_type);
 
 	if (head_ref && !(head_type & REF_ISSYMREF)) {
-		free(head_ref);
-		head_ref = NULL;
+		FREE_AND_NULL(head_ref);
 	}
 
 	/*
diff --git a/refs/ref-cache.c b/refs/ref-cache.c
index af2fcb2c12..76bb723c86 100644
--- a/refs/ref-cache.c
+++ b/refs/ref-cache.c
@@ -82,9 +82,8 @@ static void clear_ref_dir(struct ref_dir *dir)
 	int i;
 	for (i = 0; i < dir->nr; i++)
 		free_ref_entry(dir->entries[i]);
-	free(dir->entries);
+	FREE_AND_NULL(dir->entries);
 	dir->sorted = dir->nr = dir->alloc = 0;
-	dir->entries = NULL;
 }
 
 struct ref_entry *create_dir_entry(struct ref_cache *cache,
diff --git a/remote-testsvn.c b/remote-testsvn.c
index 50404ef343..078b0c3139 100644
--- a/remote-testsvn.c
+++ b/remote-testsvn.c
@@ -66,8 +66,7 @@ static char *read_ref_note(const unsigned char sha1[20])
 	else if (!msglen || type != OBJ_BLOB) {
 		error("Note contains unusable content. "
 			"Is something else using this notes tree? %s", notes_ref);
-		free(msg);
-		msg = NULL;
+		FREE_AND_NULL(msg);
 	}
 	free_notes(NULL);
 	return msg;
diff --git a/rerere.c b/rerere.c
index c26c29f87a..61f3804a1e 100644
--- a/rerere.c
+++ b/rerere.c
@@ -39,9 +39,8 @@ static void free_rerere_dirs(void)
 		free(rerere_dir[i]->status);
 		free(rerere_dir[i]);
 	}
-	free(rerere_dir);
+	FREE_AND_NULL(rerere_dir);
 	rerere_dir_nr = rerere_dir_alloc = 0;
-	rerere_dir = NULL;
 }
 
 static void free_rerere_id(struct string_list_item *item)
diff --git a/sequencer.c b/sequencer.c
index 5282fb849c..070b42f240 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1211,8 +1211,7 @@ struct todo_list {
 static void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
-	free(todo_list->items);
-	todo_list->items = NULL;
+	FREE_AND_NULL(todo_list->items);
 	todo_list->nr = todo_list->alloc = 0;
 }
 
diff --git a/sha1-array.c b/sha1-array.c
index 7d646ab5b8..838b3bf847 100644
--- a/sha1-array.c
+++ b/sha1-array.c
@@ -35,8 +35,7 @@ int oid_array_lookup(struct oid_array *array, const struct object_id *oid)
 
 void oid_array_clear(struct oid_array *array)
 {
-	free(array->oid);
-	array->oid = NULL;
+	FREE_AND_NULL(array->oid);
 	array->nr = 0;
 	array->alloc = 0;
 	array->sorted = 0;
diff --git a/sha1_file.c b/sha1_file.c
index 59a4ed2ed3..d67c88bb8d 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -610,8 +610,7 @@ char *compute_alternate_path(const char *path, struct strbuf *err)
 
 out:
 	if (seen_error) {
-		free(ref_git);
-		ref_git = NULL;
+		FREE_AND_NULL(ref_git);
 	}
 
 	return ref_git;
diff --git a/split-index.c b/split-index.c
index 49bd197f71..20477c6a48 100644
--- a/split-index.c
+++ b/split-index.c
@@ -174,10 +174,9 @@ void merge_base_index(struct index_state *istate)
 
 	ewah_free(si->delete_bitmap);
 	ewah_free(si->replace_bitmap);
-	free(si->saved_cache);
+	FREE_AND_NULL(si->saved_cache);
 	si->delete_bitmap  = NULL;
 	si->replace_bitmap = NULL;
-	si->saved_cache	   = NULL;
 	si->saved_cache_nr = 0;
 }
 
diff --git a/transport-helper.c b/transport-helper.c
index 36408046eb..33cff38cc0 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -242,8 +242,7 @@ static int disconnect_helper(struct transport *transport)
 		close(data->helper->out);
 		fclose(data->out);
 		res = finish_command(data->helper);
-		free(data->helper);
-		data->helper = NULL;
+		FREE_AND_NULL(data->helper);
 	}
 	return res;
 }
@@ -711,43 +710,35 @@ static int push_update_ref_status(struct strbuf *buf,
 
 		if (!strcmp(msg, "no match")) {
 			status = REF_STATUS_NONE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "up to date")) {
 			status = REF_STATUS_UPTODATE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "non-fast forward")) {
 			status = REF_STATUS_REJECT_NONFASTFORWARD;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "already exists")) {
 			status = REF_STATUS_REJECT_ALREADY_EXISTS;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "fetch first")) {
 			status = REF_STATUS_REJECT_FETCH_FIRST;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "needs force")) {
 			status = REF_STATUS_REJECT_NEEDS_FORCE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "stale info")) {
 			status = REF_STATUS_REJECT_STALE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "forced update")) {
 			forced = 1;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 	}
 
diff --git a/transport.c b/transport.c
index 9bfcf870f9..cf35088001 100644
--- a/transport.c
+++ b/transport.c
@@ -1145,8 +1145,7 @@ void transport_unlock_pack(struct transport *transport)
 {
 	if (transport->pack_lockfile) {
 		unlink_or_warn(transport->pack_lockfile);
-		free(transport->pack_lockfile);
-		transport->pack_lockfile = NULL;
+		FREE_AND_NULL(transport->pack_lockfile);
 	}
 }
 
diff --git a/tree-diff.c b/tree-diff.c
index e164e532b2..1e1b393d08 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -140,8 +140,7 @@ static struct combine_diff_path *path_appendnew(struct combine_diff_path *last,
 	/* if last->next is !NULL - it is a pre-allocated memory, we can reuse */
 	p = last->next;
 	if (p && (alloclen > (intptr_t)p->next)) {
-		free(p);
-		p = NULL;
+		FREE_AND_NULL(p);
 	}
 
 	if (!p) {
@@ -559,8 +558,7 @@ struct combine_diff_path *diff_tree_paths(
 	 * (see path_appendnew() for details about why)
 	 */
 	if (p->next) {
-		free(p->next);
-		p->next = NULL;
+		FREE_AND_NULL(p->next);
 	}
 
 	return p;
diff --git a/tree-walk.c b/tree-walk.c
index 6a42e402b0..9f67b89c87 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -479,8 +479,7 @@ int traverse_trees(int n, struct tree_desc *t, struct traverse_info *info)
 	for (i = 0; i < n; i++)
 		free_extended_entry(tx + i);
 	free(tx);
-	free(traverse_path);
-	info->traverse_path = NULL;
+	FREE_AND_NULL(traverse_path);
 	strbuf_release(&base);
 	return error;
 }
diff --git a/tree.c b/tree.c
index 603b29ee80..614f91586d 100644
--- a/tree.c
+++ b/tree.c
@@ -226,8 +226,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)
 
 void free_tree_buffer(struct tree *tree)
 {
-	free(tree->buffer);
-	tree->buffer = NULL;
+	FREE_AND_NULL(tree->buffer);
 	tree->size = 0;
 	tree->object.parsed = 0;
 }
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 0/2] Add a FREE_AND_NULL() wrapper macro
  2017-06-15 21:06                       ` [PATCH v3 0/2] Add a FREE_AND_NULL() wrapper macro Ævar Arnfjörð Bjarmason
@ 2017-06-15 22:00                         ` Junio C Hamano
  2017-06-15 23:15                           ` [PATCH v4 0/6] " Ævar Arnfjörð Bjarmason
                                             ` (6 more replies)
  0 siblings, 7 replies; 214+ messages in thread
From: Junio C Hamano @ 2017-06-15 22:00 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Brandon Williams, Christian Couder, Martin Ågren

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> On Thu, Jun 15 2017, Ævar Arnfjörð Bjarmason jotted:
>> I'll change it to FREE_AND_NULL and submit my patch as-is, my reading
>> of the rest of this thread is that making it a function instead of a
>> macro would be interesting, but has its own caveats that are likely
>> better considered as part of its own series, whereas this just changes
>> existing code to its macro-expanded functional equivalent.
>
> Here's v3 with that change. Nothing but the macro name (and comments,
> commit messages etc. referring to it) have changed.
>
> Ævar Arnfjörð Bjarmason (2):
>   git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr =
>     NULL
>   *.[ch] refactoring: make use of the FREE_AND_NULL() macro

Thanks.

Perhaps somebody wants to do a follow-up patch on top of these two
patches to add .cocci rule e.g.

	@@
	type T;
	T *ptr;
	@@
	- free(ptr);
	- ptr = NULL;
	+ FREE_AND_NULL(ptr);

so that we can periodically sweep new candidates, to which this
macro can be applied?

^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v4 0/6] Add a FREE_AND_NULL() wrapper macro
  2017-06-15 22:00                         ` Junio C Hamano
@ 2017-06-15 23:15                           ` Ævar Arnfjörð Bjarmason
  2017-06-15 23:15                           ` [PATCH v4 1/6] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
                                             ` (5 subsequent siblings)
  6 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 23:15 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason

I didn't know about the coccinelle semantic patch facility. This adds
coccinelle rules to do the code changes, and adds subsequent follow-up
commits which apply that change, with the series ending with a patch
that I manually authored.

This fixes a bug in earlier versions of the series. I was mistakenly
freeing the wrong thing in the tree-walk.c change, that change is now
gone, and all the changes not made by coccinelle are in one commit,
making it easier to review.

Ævar Arnfjörð Bjarmason (6):
  git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr =
    NULL
  coccinelle: add a rule to make "type" code use FREE_AND_NULL()
  coccinelle: make use of the "type" FREE_AND_NULL() rule
  coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  coccinelle: make use of the "expression" FREE_AND_NULL() rule
  *.[ch] refactoring: make use of the FREE_AND_NULL() macro

 alias.c                       |  6 ++----
 apply.c                       |  3 +--
 attr.c                        |  6 ++----
 blame.c                       |  3 +--
 branch.c                      |  3 +--
 builtin/am.c                  | 18 +++++-------------
 builtin/clean.c               |  6 ++----
 builtin/config.c              |  6 ++----
 builtin/index-pack.c          |  6 ++----
 builtin/pack-objects.c        | 12 ++++--------
 builtin/unpack-objects.c      |  3 +--
 builtin/worktree.c            |  6 ++----
 commit-slab.h                 |  3 +--
 commit.c                      |  3 +--
 config.c                      |  3 +--
 contrib/coccinelle/free.cocci | 15 +++++++++++++++
 credential.c                  |  9 +++------
 diff-lib.c                    |  3 +--
 diff.c                        |  6 ++----
 diffcore-rename.c             |  6 ++----
 dir.c                         |  9 +++------
 fast-import.c                 |  6 ++----
 git-compat-util.h             |  6 ++++++
 gpg-interface.c               | 15 +++++----------
 grep.c                        | 12 ++++--------
 help.c                        |  3 +--
 http-push.c                   | 24 ++++++++----------------
 http.c                        | 15 +++++----------
 imap-send.c                   |  3 +--
 line-log.c                    |  6 ++----
 ll-merge.c                    |  3 +--
 mailinfo.c                    |  3 +--
 object.c                      |  3 +--
 pathspec.c                    |  3 +--
 prio-queue.c                  |  3 +--
 read-cache.c                  |  6 ++----
 ref-filter.c                  |  3 +--
 refs/files-backend.c          |  3 +--
 refs/ref-cache.c              |  3 +--
 remote-testsvn.c              |  3 +--
 rerere.c                      |  3 +--
 sequencer.c                   |  3 +--
 sha1-array.c                  |  3 +--
 sha1_file.c                   |  3 +--
 split-index.c                 |  3 +--
 transport-helper.c            | 27 +++++++++------------------
 transport.c                   |  3 +--
 tree-diff.c                   |  6 ++----
 tree.c                        |  3 +--
 49 files changed, 117 insertions(+), 195 deletions(-)

-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v4 1/6] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL
  2017-06-15 22:00                         ` Junio C Hamano
  2017-06-15 23:15                           ` [PATCH v4 0/6] " Ævar Arnfjörð Bjarmason
@ 2017-06-15 23:15                           ` Ævar Arnfjörð Bjarmason
  2017-06-15 23:15                           ` [PATCH v4 2/6] coccinelle: add a rule to make "type" code use FREE_AND_NULL() Ævar Arnfjörð Bjarmason
                                             ` (4 subsequent siblings)
  6 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 23:15 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason

Add a FREE_AND_NULL() wrapper marco for the common pattern of freeing
a pointer and assigning NULL to it right afterwards.

The implementation is similar to the (currently unused) XDL_PTRFREE
macro in xdiff/xmacros.h added in commit 3443546f6e ("Use a *real*
built-in diff generator", 2006-03-24). The only difference is that
free() is called unconditionally, see [1].

See [2] for a suggested alternative which does this via a function
instead of a macro. As covered in replies to that message, while it's
a viable approach, it would introduce caveats which this approach
doesn't have, so that potential change is left to a future follow-up
change.

This merely allows us to translate exactly what we're doing now to a
less verbose & idiomatic form using a macro, while guaranteeing that
we don't introduce any functional changes.

1. <alpine.DEB.2.20.1608301948310.129229@virtualbox>
   (http://public-inbox.org/git/alpine.DEB.2.20.1608301948310.129229@virtualbox/)

2. <20170610032143.GA7880@starla>
   (https://public-inbox.org/git/20170610032143.GA7880@starla/)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-compat-util.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index 51ba4e6b3b..047172d173 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -808,6 +808,12 @@ extern char *xgetcwd(void);
 extern FILE *fopen_for_writing(const char *path);
 extern FILE *fopen_or_warn(const char *path, const char *mode);
 
+/*
+ * FREE_AND_NULL(ptr) is like free(ptr) followed by ptr = NULL. Note
+ * that ptr is used twice, so don't pass e.g. ptr++.
+ */
+#define FREE_AND_NULL(p) do { free(p); (p) = NULL; } while (0)
+
 #define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), (alloc)))
 #define REALLOC_ARRAY(x, alloc) (x) = xrealloc((x), st_mult(sizeof(*(x)), (alloc)))
 
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 2/6] coccinelle: add a rule to make "type" code use FREE_AND_NULL()
  2017-06-15 22:00                         ` Junio C Hamano
  2017-06-15 23:15                           ` [PATCH v4 0/6] " Ævar Arnfjörð Bjarmason
  2017-06-15 23:15                           ` [PATCH v4 1/6] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
@ 2017-06-15 23:15                           ` Ævar Arnfjörð Bjarmason
  2017-06-15 23:15                           ` [PATCH v4 3/6] coccinelle: make use of the "type" FREE_AND_NULL() rule Ævar Arnfjörð Bjarmason
                                             ` (3 subsequent siblings)
  6 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 23:15 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 contrib/coccinelle/free.cocci | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
index c03ba737e5..35fb992621 100644
--- a/contrib/coccinelle/free.cocci
+++ b/contrib/coccinelle/free.cocci
@@ -9,3 +9,11 @@ expression E;
 @@
 - if (!E)
   free(E);
+
+@@
+type T;
+T *ptr;
+@@
+- free(ptr);
+- ptr = NULL;
++ FREE_AND_NULL(ptr);
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 3/6] coccinelle: make use of the "type" FREE_AND_NULL() rule
  2017-06-15 22:00                         ` Junio C Hamano
                                             ` (2 preceding siblings ...)
  2017-06-15 23:15                           ` [PATCH v4 2/6] coccinelle: add a rule to make "type" code use FREE_AND_NULL() Ævar Arnfjörð Bjarmason
@ 2017-06-15 23:15                           ` Ævar Arnfjörð Bjarmason
  2017-06-15 23:15                           ` [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL() Ævar Arnfjörð Bjarmason
                                             ` (2 subsequent siblings)
  6 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 23:15 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason

Apply the result of the just-added coccinelle rule. This manually
excludes a few occurrences, mostly things that resulted in many
FREE_AND_NULL() on one line, that'll be manually fixed in a subsequent
change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 alias.c                  |  6 ++----
 apply.c                  |  3 +--
 attr.c                   |  6 ++----
 branch.c                 |  3 +--
 builtin/am.c             |  3 +--
 builtin/clean.c          |  6 ++----
 builtin/config.c         |  6 ++----
 builtin/index-pack.c     |  6 ++----
 builtin/pack-objects.c   | 12 ++++--------
 builtin/unpack-objects.c |  3 +--
 commit.c                 |  3 +--
 config.c                 |  3 +--
 credential.c             |  3 +--
 diff-lib.c               |  3 +--
 diff.c                   |  6 ++----
 diffcore-rename.c        |  6 ++----
 dir.c                    |  9 +++------
 fast-import.c            |  6 ++----
 grep.c                   |  3 +--
 http-push.c              | 24 ++++++++----------------
 http.c                   | 15 +++++----------
 imap-send.c              |  3 +--
 line-log.c               |  3 +--
 mailinfo.c               |  3 +--
 object.c                 |  3 +--
 pathspec.c               |  3 +--
 read-cache.c             |  6 ++----
 ref-filter.c             |  3 +--
 refs/files-backend.c     |  3 +--
 remote-testsvn.c         |  3 +--
 sequencer.c              |  3 +--
 sha1-array.c             |  3 +--
 sha1_file.c              |  3 +--
 transport-helper.c       | 27 +++++++++------------------
 transport.c              |  3 +--
 tree-diff.c              |  6 ++----
 tree.c                   |  3 +--
 37 files changed, 71 insertions(+), 142 deletions(-)

diff --git a/alias.c b/alias.c
index 3b90397a99..911481855f 100644
--- a/alias.c
+++ b/alias.c
@@ -47,8 +47,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 				src++;
 				c = cmdline[src];
 				if (!c) {
-					free(*argv);
-					*argv = NULL;
+					FREE_AND_NULL(*argv);
 					return -SPLIT_CMDLINE_BAD_ENDING;
 				}
 			}
@@ -60,8 +59,7 @@ int split_cmdline(char *cmdline, const char ***argv)
 	cmdline[dst] = 0;
 
 	if (quoted) {
-		free(*argv);
-		*argv = NULL;
+		FREE_AND_NULL(*argv);
 		return -SPLIT_CMDLINE_UNCLOSED_QUOTE;
 	}
 
diff --git a/apply.c b/apply.c
index 854faa6779..e78de0affa 100644
--- a/apply.c
+++ b/apply.c
@@ -3705,8 +3705,7 @@ static int check_preimage(struct apply_state *state,
  is_new:
 	patch->is_new = 1;
 	patch->is_delete = 0;
-	free(patch->old_name);
-	patch->old_name = NULL;
+	FREE_AND_NULL(patch->old_name);
 	return 0;
 }
 
diff --git a/attr.c b/attr.c
index 821203e2a9..ebdcfb0b8a 100644
--- a/attr.c
+++ b/attr.c
@@ -638,13 +638,11 @@ void attr_check_reset(struct attr_check *check)
 
 void attr_check_clear(struct attr_check *check)
 {
-	free(check->items);
-	check->items = NULL;
+	FREE_AND_NULL(check->items);
 	check->alloc = 0;
 	check->nr = 0;
 
-	free(check->all_attrs);
-	check->all_attrs = NULL;
+	FREE_AND_NULL(check->all_attrs);
 	check->all_attrs_nr = 0;
 
 	drop_attr_stack(&check->stack);
diff --git a/branch.c b/branch.c
index 985316eb76..2347cb8649 100644
--- a/branch.c
+++ b/branch.c
@@ -24,8 +24,7 @@ static int find_tracked_branch(struct remote *remote, void *priv)
 		} else {
 			free(tracking->spec.src);
 			if (tracking->src) {
-				free(tracking->src);
-				tracking->src = NULL;
+				FREE_AND_NULL(tracking->src);
 			}
 		}
 		tracking->spec.src = NULL;
diff --git a/builtin/am.c b/builtin/am.c
index 8881d73615..138fb98537 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -483,8 +483,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
 	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
 
 	if (!ret) {
-		free(state->msg);
-		state->msg = NULL;
+		FREE_AND_NULL(state->msg);
 		if (read_commit_msg(state) < 0)
 			die(_("'%s' was deleted by the applypatch-msg hook"),
 				am_path(state, "final-commit"));
diff --git a/builtin/clean.c b/builtin/clean.c
index 142bf668cf..bff5a07330 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -837,8 +837,7 @@ static void interactive_main_loop(void)
 			int ret;
 			ret = menus[*chosen].fn();
 			if (ret != MENU_RETURN_NO_LOOP) {
-				free(chosen);
-				chosen = NULL;
+				FREE_AND_NULL(chosen);
 				if (!del_list.nr) {
 					clean_print_color(CLEAN_COLOR_ERROR);
 					printf_ln(_("No more files to clean, exiting."));
@@ -851,8 +850,7 @@ static void interactive_main_loop(void)
 			quit_cmd();
 		}
 
-		free(chosen);
-		chosen = NULL;
+		FREE_AND_NULL(chosen);
 		break;
 	}
 }
diff --git a/builtin/config.c b/builtin/config.c
index 7f6c25d4d9..18c2158630 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -214,8 +214,7 @@ static int get_value(const char *key_, const char *regex_)
 		key_regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(key_regexp, key, REG_EXTENDED)) {
 			error("invalid key pattern: %s", key_);
-			free(key_regexp);
-			key_regexp = NULL;
+			FREE_AND_NULL(key_regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
@@ -235,8 +234,7 @@ static int get_value(const char *key_, const char *regex_)
 		regexp = (regex_t*)xmalloc(sizeof(regex_t));
 		if (regcomp(regexp, regex_, REG_EXTENDED)) {
 			error("invalid pattern: %s", regex_);
-			free(regexp);
-			regexp = NULL;
+			FREE_AND_NULL(regexp);
 			ret = CONFIG_INVALID_PATTERN;
 			goto free_strings;
 		}
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 04b9dcaf0f..f3b16520c2 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -388,8 +388,7 @@ static struct base_data *alloc_base_data(void)
 static void free_base_data(struct base_data *c)
 {
 	if (c->data) {
-		free(c->data);
-		c->data = NULL;
+		FREE_AND_NULL(c->data);
 		get_thread_data()->base_cache_used -= c->size;
 	}
 }
@@ -605,8 +604,7 @@ static void *unpack_data(struct object_entry *obj,
 	git_inflate_end(&stream);
 	free(inbuf);
 	if (consume) {
-		free(data);
-		data = NULL;
+		FREE_AND_NULL(data);
 	}
 	return data;
 }
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f672225def..790615b2c3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -264,8 +264,7 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 		 * make sure no cached delta data remains from a
 		 * previous attempt before a pack split occurred.
 		 */
-		free(entry->delta_data);
-		entry->delta_data = NULL;
+		FREE_AND_NULL(entry->delta_data);
 		entry->z_delta_size = 0;
 	} else if (entry->delta_data) {
 		size = entry->delta_size;
@@ -1375,12 +1374,10 @@ static void cleanup_preferred_base(void)
 		if (!pbase_tree_cache[i])
 			continue;
 		free(pbase_tree_cache[i]->tree_data);
-		free(pbase_tree_cache[i]);
-		pbase_tree_cache[i] = NULL;
+		FREE_AND_NULL(pbase_tree_cache[i]);
 	}
 
-	free(done_pbase_paths);
-	done_pbase_paths = NULL;
+	FREE_AND_NULL(done_pbase_paths);
 	done_pbase_paths_num = done_pbase_paths_alloc = 0;
 }
 
@@ -1970,8 +1967,7 @@ static unsigned long free_unpacked(struct unpacked *n)
 	n->index = NULL;
 	if (n->data) {
 		freed_mem += n->entry->size;
-		free(n->data);
-		n->data = NULL;
+		FREE_AND_NULL(n->data);
 	}
 	n->entry = NULL;
 	n->depth = 0;
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 8bc9997767..60bfe48789 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -112,8 +112,7 @@ static void *get_data(unsigned long size)
 			break;
 		if (ret != Z_OK) {
 			error("inflate returned %d", ret);
-			free(buf);
-			buf = NULL;
+			FREE_AND_NULL(buf);
 			if (!recover)
 				exit(1);
 			has_errors = 1;
diff --git a/commit.c b/commit.c
index 99846d9bf4..cbfd689939 100644
--- a/commit.c
+++ b/commit.c
@@ -287,8 +287,7 @@ void free_commit_buffer(struct commit *commit)
 {
 	struct commit_buffer *v = buffer_slab_peek(&buffer_slab, commit);
 	if (v) {
-		free(v->buffer);
-		v->buffer = NULL;
+		FREE_AND_NULL(v->buffer);
 		v->size = 0;
 	}
 }
diff --git a/config.c b/config.c
index 34a139c40b..0501a60dd2 100644
--- a/config.c
+++ b/config.c
@@ -395,8 +395,7 @@ static int git_config_parse_key_1(const char *key, char **store_key, int *basele
 
 out_free_ret_1:
 	if (store_key) {
-		free(*store_key);
-		*store_key = NULL;
+		FREE_AND_NULL(*store_key);
 	}
 	return -CONFIG_INVALID_KEY;
 }
diff --git a/credential.c b/credential.c
index aa996669fc..0ab247ff40 100644
--- a/credential.c
+++ b/credential.c
@@ -93,8 +93,7 @@ static void credential_apply_config(struct credential *c)
 	c->configured = 1;
 
 	if (!c->use_http_path && proto_is_http(c->protocol)) {
-		free(c->path);
-		c->path = NULL;
+		FREE_AND_NULL(c->path);
 	}
 }
 
diff --git a/diff-lib.c b/diff-lib.c
index 76c8f185cd..7528f4163f 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -179,8 +179,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)
 				free(dpath);
 				continue;
 			}
-			free(dpath);
-			dpath = NULL;
+			FREE_AND_NULL(dpath);
 
 			/*
 			 * Show the diff for the 'ce' if we found the one
diff --git a/diff.c b/diff.c
index e35cf6c704..3561adf49f 100644
--- a/diff.c
+++ b/diff.c
@@ -1218,8 +1218,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)
 			regfree(ecbdata->diff_words->word_regex);
 			free(ecbdata->diff_words->word_regex);
 		}
-		free(ecbdata->diff_words);
-		ecbdata->diff_words = NULL;
+		FREE_AND_NULL(ecbdata->diff_words);
 	}
 }
 
@@ -2951,8 +2950,7 @@ void diff_free_filespec_blob(struct diff_filespec *s)
 void diff_free_filespec_data(struct diff_filespec *s)
 {
 	diff_free_filespec_blob(s);
-	free(s->cnt_data);
-	s->cnt_data = NULL;
+	FREE_AND_NULL(s->cnt_data);
 }
 
 static void prep_temp_blob(const char *path, struct diff_tempfile *temp,
diff --git a/diffcore-rename.c b/diffcore-rename.c
index f7444c86bd..6d8daf9c00 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -667,11 +667,9 @@ void diffcore_rename(struct diff_options *options)
 	for (i = 0; i < rename_dst_nr; i++)
 		free_filespec(rename_dst[i].two);
 
-	free(rename_dst);
-	rename_dst = NULL;
+	FREE_AND_NULL(rename_dst);
 	rename_dst_nr = rename_dst_alloc = 0;
-	free(rename_src);
-	rename_src = NULL;
+	FREE_AND_NULL(rename_src);
 	rename_src_nr = rename_src_alloc = 0;
 	return;
 }
diff --git a/dir.c b/dir.c
index 1759063817..7dc168b806 100644
--- a/dir.c
+++ b/dir.c
@@ -2117,8 +2117,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		for (i = j = 0; j < dir->nr; j++) {
 			if (i &&
 			    check_dir_entry_contains(dir->entries[i - 1], dir->entries[j])) {
-				free(dir->entries[j]);
-				dir->entries[j] = NULL;
+				FREE_AND_NULL(dir->entries[j]);
 			} else {
 				dir->entries[i++] = dir->entries[j];
 			}
@@ -2144,8 +2143,7 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
 		     dir->untracked->dir_invalidated))
 			istate->cache_changed |= UNTRACKED_CHANGED;
 		if (dir->untracked != istate->untracked) {
-			free(dir->untracked);
-			dir->untracked = NULL;
+			FREE_AND_NULL(dir->untracked);
 		}
 	}
 	return dir->nr;
@@ -2488,8 +2486,7 @@ void write_untracked_extension(struct strbuf *out, struct untracked_cache *untra
 	strbuf_addbuf(out, &untracked->ident);
 
 	strbuf_add(out, ouc, ouc_size(len));
-	free(ouc);
-	ouc = NULL;
+	FREE_AND_NULL(ouc);
 
 	if (!untracked->root) {
 		varint_len = encode_varint(0, varbuf);
diff --git a/fast-import.c b/fast-import.c
index 9a22fc92c0..eeab927d5a 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1063,8 +1063,7 @@ static void end_packfile(void)
 		close(pack_data->pack_fd);
 		unlink_or_warn(pack_data->pack_name);
 	}
-	free(pack_data);
-	pack_data = NULL;
+	FREE_AND_NULL(pack_data);
 	running = 0;
 
 	/* We can't carry a delta across packfiles. */
@@ -1149,8 +1148,7 @@ static int store_object(
 
 		/* We cannot carry a delta into the new pack. */
 		if (delta) {
-			free(delta);
-			delta = NULL;
+			FREE_AND_NULL(delta);
 
 			git_deflate_init(&s, pack_compression_level);
 			s.next_in = (void *)dat->buf;
diff --git a/grep.c b/grep.c
index d03d424e5c..11513ec06d 100644
--- a/grep.c
+++ b/grep.c
@@ -1778,8 +1778,7 @@ void grep_source_clear_data(struct grep_source *gs)
 	case GREP_SOURCE_FILE:
 	case GREP_SOURCE_SHA1:
 	case GREP_SOURCE_SUBMODULE:
-		free(gs->buf);
-		gs->buf = NULL;
+		FREE_AND_NULL(gs->buf);
 		gs->size = 0;
 		break;
 	case GREP_SOURCE_BUF:
diff --git a/http-push.c b/http-push.c
index 67c4d4b472..c91f40a610 100644
--- a/http-push.c
+++ b/http-push.c
@@ -291,8 +291,7 @@ static void start_mkcol(struct transfer_request *request)
 		request->state = RUN_MKCOL;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 }
 #endif
@@ -409,8 +408,7 @@ static void start_put(struct transfer_request *request)
 		request->state = RUN_PUT;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 }
 
@@ -432,8 +430,7 @@ static void start_move(struct transfer_request *request)
 		request->state = RUN_MOVE;
 	} else {
 		request->state = ABORTED;
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 }
 
@@ -526,8 +523,7 @@ static void finish_request(struct transfer_request *request)
 
 	/* URL is reused for MOVE after PUT */
 	if (request->state != RUN_PUT) {
-		free(request->url);
-		request->url = NULL;
+		FREE_AND_NULL(request->url);
 	}
 
 	if (request->state == RUN_MKCOL) {
@@ -803,8 +799,7 @@ xml_start_tag(void *userData, const char *name, const char **atts)
 	}
 	xsnprintf(ctx->name + old_namelen, ctx->len - old_namelen, ".%s", c);
 
-	free(ctx->cdata);
-	ctx->cdata = NULL;
+	FREE_AND_NULL(ctx->cdata);
 
 	ctx->userFunc(ctx, 0);
 }
@@ -932,8 +927,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)
 		free(lock->token);
 		free(lock->owner);
 		free(url);
-		free(lock);
-		lock = NULL;
+		FREE_AND_NULL(lock);
 	} else {
 		lock->url = url;
 		lock->start_time = time(NULL);
@@ -1105,8 +1099,7 @@ static void handle_remote_ls_ctx(struct xml_ctx *ctx, int tag_closed)
 			ls->dentry_flags |= IS_DIR;
 		}
 	} else if (!strcmp(ctx->name, DAV_PROPFIND_RESP)) {
-		free(ls->dentry_name);
-		ls->dentry_name = NULL;
+		FREE_AND_NULL(ls->dentry_name);
 		ls->dentry_flags = 0;
 	}
 }
@@ -1547,8 +1540,7 @@ static void fetch_symref(const char *path, char **symref, struct object_id *oid)
 		    curl_errorstr);
 	free(url);
 
-	free(*symref);
-	*symref = NULL;
+	FREE_AND_NULL(*symref);
 	oidclr(oid);
 
 	if (buffer.len == 0)
diff --git a/http.c b/http.c
index d2e11ec6f0..77f46bef0f 100644
--- a/http.c
+++ b/http.c
@@ -1026,8 +1026,7 @@ void http_cleanup(void)
 
 	if (proxy_auth.password) {
 		memset(proxy_auth.password, 0, strlen(proxy_auth.password));
-		free(proxy_auth.password);
-		proxy_auth.password = NULL;
+		FREE_AND_NULL(proxy_auth.password);
 	}
 
 	free((void *)curl_proxyuserpwd);
@@ -1038,13 +1037,11 @@ void http_cleanup(void)
 
 	if (cert_auth.password != NULL) {
 		memset(cert_auth.password, 0, strlen(cert_auth.password));
-		free(cert_auth.password);
-		cert_auth.password = NULL;
+		FREE_AND_NULL(cert_auth.password);
 	}
 	ssl_cert_password_required = 0;
 
-	free(cached_accept_language);
-	cached_accept_language = NULL;
+	FREE_AND_NULL(cached_accept_language);
 }
 
 struct active_request_slot *get_active_slot(void)
@@ -1896,8 +1893,7 @@ static char *fetch_pack_index(unsigned char *sha1, const char *base_url)
 
 	if (http_get_file(url, tmp, NULL) != HTTP_OK) {
 		error("Unable to get pack index %s", url);
-		free(tmp);
-		tmp = NULL;
+		FREE_AND_NULL(tmp);
 	}
 
 	free(url);
@@ -2328,8 +2324,7 @@ void release_http_object_request(struct http_object_request *freq)
 		freq->localfile = -1;
 	}
 	if (freq->url != NULL) {
-		free(freq->url);
-		freq->url = NULL;
+		FREE_AND_NULL(freq->url);
 	}
 	if (freq->slot != NULL) {
 		freq->slot->callback_func = NULL;
diff --git a/imap-send.c b/imap-send.c
index 857591660f..40ddb496e6 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -776,8 +776,7 @@ static int get_cmd_result(struct imap_store *ctx, struct imap_cmd *tcmd)
 			       offsetof(struct imap_cmd, next));
 			if (cmdp->cb.data) {
 				n = socket_write(&imap->buf.sock, cmdp->cb.data, cmdp->cb.dlen);
-				free(cmdp->cb.data);
-				cmdp->cb.data = NULL;
+				FREE_AND_NULL(cmdp->cb.data);
 				if (n != (int)cmdp->cb.dlen)
 					return RESP_BAD;
 			} else if (cmdp->cb.cont) {
diff --git a/line-log.c b/line-log.c
index b9087814b8..79eded7363 100644
--- a/line-log.c
+++ b/line-log.c
@@ -610,8 +610,7 @@ parse_lines(struct commit *commit, const char *prefix, struct string_list *args)
 		line_log_data_insert(&ranges, full_name, begin, end);
 
 		free_filespec(spec);
-		free(ends);
-		ends = NULL;
+		FREE_AND_NULL(ends);
 	}
 
 	for (p = ranges; p; p = p->next)
diff --git a/mailinfo.c b/mailinfo.c
index f92cb9f729..44f829edb1 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -919,8 +919,7 @@ static int handle_boundary(struct mailinfo *mi, struct strbuf *line)
 		/* we hit an end boundary */
 		/* pop the current boundary off the stack */
 		strbuf_release(*(mi->content_top));
-		free(*(mi->content_top));
-		*(mi->content_top) = NULL;
+		FREE_AND_NULL(*(mi->content_top));
 
 		/* technically won't happen as is_multipart_boundary()
 		   will fail first.  But just in case..
diff --git a/object.c b/object.c
index 06ba3a11d8..f818777412 100644
--- a/object.c
+++ b/object.c
@@ -377,8 +377,7 @@ void object_array_clear(struct object_array *array)
 	int i;
 	for (i = 0; i < array->nr; i++)
 		object_array_release_entry(&array->objects[i]);
-	free(array->objects);
-	array->objects = NULL;
+	FREE_AND_NULL(array->objects);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/pathspec.c b/pathspec.c
index 828405021f..54aab8bc01 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -662,7 +662,6 @@ void clear_pathspec(struct pathspec *pathspec)
 			attr_check_free(pathspec->items[i].attr_check);
 	}
 
-	free(pathspec->items);
-	pathspec->items = NULL;
+	FREE_AND_NULL(pathspec->items);
 	pathspec->nr = 0;
 }
diff --git a/read-cache.c b/read-cache.c
index bc156a133e..66c413657e 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1888,8 +1888,7 @@ int discard_index(struct index_state *istate)
 	free_name_hash(istate);
 	cache_tree_free(&(istate->cache_tree));
 	istate->initialized = 0;
-	free(istate->cache);
-	istate->cache = NULL;
+	FREE_AND_NULL(istate->cache);
 	istate->cache_alloc = 0;
 	discard_split_index(istate);
 	free_untracked_cache(istate->untracked);
@@ -2603,8 +2602,7 @@ void *read_blob_data_from_index(const struct index_state *istate,
 
 void stat_validity_clear(struct stat_validity *sv)
 {
-	free(sv->sd);
-	sv->sd = NULL;
+	FREE_AND_NULL(sv->sd);
 }
 
 int stat_validity_check(struct stat_validity *sv, const char *path)
diff --git a/ref-filter.c b/ref-filter.c
index ab32bc9c31..72e6cb8ecc 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1891,8 +1891,7 @@ void ref_array_clear(struct ref_array *array)
 
 	for (i = 0; i < array->nr; i++)
 		free_array_item(array->items[i]);
-	free(array->items);
-	array->items = NULL;
+	FREE_AND_NULL(array->items);
 	array->nr = array->alloc = 0;
 }
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index d8b3f73147..c1bd99d60f 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2944,8 +2944,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 				       head_oid.hash, &head_type);
 
 	if (head_ref && !(head_type & REF_ISSYMREF)) {
-		free(head_ref);
-		head_ref = NULL;
+		FREE_AND_NULL(head_ref);
 	}
 
 	/*
diff --git a/remote-testsvn.c b/remote-testsvn.c
index 50404ef343..078b0c3139 100644
--- a/remote-testsvn.c
+++ b/remote-testsvn.c
@@ -66,8 +66,7 @@ static char *read_ref_note(const unsigned char sha1[20])
 	else if (!msglen || type != OBJ_BLOB) {
 		error("Note contains unusable content. "
 			"Is something else using this notes tree? %s", notes_ref);
-		free(msg);
-		msg = NULL;
+		FREE_AND_NULL(msg);
 	}
 	free_notes(NULL);
 	return msg;
diff --git a/sequencer.c b/sequencer.c
index 5282fb849c..070b42f240 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1211,8 +1211,7 @@ struct todo_list {
 static void todo_list_release(struct todo_list *todo_list)
 {
 	strbuf_release(&todo_list->buf);
-	free(todo_list->items);
-	todo_list->items = NULL;
+	FREE_AND_NULL(todo_list->items);
 	todo_list->nr = todo_list->alloc = 0;
 }
 
diff --git a/sha1-array.c b/sha1-array.c
index 7d646ab5b8..838b3bf847 100644
--- a/sha1-array.c
+++ b/sha1-array.c
@@ -35,8 +35,7 @@ int oid_array_lookup(struct oid_array *array, const struct object_id *oid)
 
 void oid_array_clear(struct oid_array *array)
 {
-	free(array->oid);
-	array->oid = NULL;
+	FREE_AND_NULL(array->oid);
 	array->nr = 0;
 	array->alloc = 0;
 	array->sorted = 0;
diff --git a/sha1_file.c b/sha1_file.c
index 59a4ed2ed3..d67c88bb8d 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -610,8 +610,7 @@ char *compute_alternate_path(const char *path, struct strbuf *err)
 
 out:
 	if (seen_error) {
-		free(ref_git);
-		ref_git = NULL;
+		FREE_AND_NULL(ref_git);
 	}
 
 	return ref_git;
diff --git a/transport-helper.c b/transport-helper.c
index 36408046eb..33cff38cc0 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -242,8 +242,7 @@ static int disconnect_helper(struct transport *transport)
 		close(data->helper->out);
 		fclose(data->out);
 		res = finish_command(data->helper);
-		free(data->helper);
-		data->helper = NULL;
+		FREE_AND_NULL(data->helper);
 	}
 	return res;
 }
@@ -711,43 +710,35 @@ static int push_update_ref_status(struct strbuf *buf,
 
 		if (!strcmp(msg, "no match")) {
 			status = REF_STATUS_NONE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "up to date")) {
 			status = REF_STATUS_UPTODATE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "non-fast forward")) {
 			status = REF_STATUS_REJECT_NONFASTFORWARD;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "already exists")) {
 			status = REF_STATUS_REJECT_ALREADY_EXISTS;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "fetch first")) {
 			status = REF_STATUS_REJECT_FETCH_FIRST;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "needs force")) {
 			status = REF_STATUS_REJECT_NEEDS_FORCE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "stale info")) {
 			status = REF_STATUS_REJECT_STALE;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 		else if (!strcmp(msg, "forced update")) {
 			forced = 1;
-			free(msg);
-			msg = NULL;
+			FREE_AND_NULL(msg);
 		}
 	}
 
diff --git a/transport.c b/transport.c
index 9bfcf870f9..cf35088001 100644
--- a/transport.c
+++ b/transport.c
@@ -1145,8 +1145,7 @@ void transport_unlock_pack(struct transport *transport)
 {
 	if (transport->pack_lockfile) {
 		unlink_or_warn(transport->pack_lockfile);
-		free(transport->pack_lockfile);
-		transport->pack_lockfile = NULL;
+		FREE_AND_NULL(transport->pack_lockfile);
 	}
 }
 
diff --git a/tree-diff.c b/tree-diff.c
index e164e532b2..1e1b393d08 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -140,8 +140,7 @@ static struct combine_diff_path *path_appendnew(struct combine_diff_path *last,
 	/* if last->next is !NULL - it is a pre-allocated memory, we can reuse */
 	p = last->next;
 	if (p && (alloclen > (intptr_t)p->next)) {
-		free(p);
-		p = NULL;
+		FREE_AND_NULL(p);
 	}
 
 	if (!p) {
@@ -559,8 +558,7 @@ struct combine_diff_path *diff_tree_paths(
 	 * (see path_appendnew() for details about why)
 	 */
 	if (p->next) {
-		free(p->next);
-		p->next = NULL;
+		FREE_AND_NULL(p->next);
 	}
 
 	return p;
diff --git a/tree.c b/tree.c
index 603b29ee80..614f91586d 100644
--- a/tree.c
+++ b/tree.c
@@ -226,8 +226,7 @@ int parse_tree_gently(struct tree *item, int quiet_on_missing)
 
 void free_tree_buffer(struct tree *tree)
 {
-	free(tree->buffer);
-	tree->buffer = NULL;
+	FREE_AND_NULL(tree->buffer);
 	tree->size = 0;
 	tree->object.parsed = 0;
 }
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-15 22:00                         ` Junio C Hamano
                                             ` (3 preceding siblings ...)
  2017-06-15 23:15                           ` [PATCH v4 3/6] coccinelle: make use of the "type" FREE_AND_NULL() rule Ævar Arnfjörð Bjarmason
@ 2017-06-15 23:15                           ` Ævar Arnfjörð Bjarmason
  2017-06-16 19:43                             ` Junio C Hamano
  2017-06-15 23:15                           ` [PATCH v4 5/6] coccinelle: make use of the "expression" FREE_AND_NULL() rule Ævar Arnfjörð Bjarmason
  2017-06-15 23:15                           ` [PATCH v4 6/6] *.[ch] refactoring: make use of the FREE_AND_NULL() macro Ævar Arnfjörð Bjarmason
  6 siblings, 1 reply; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 23:15 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason

A follow-up to the existing "type" rule added in an earlier
change. This catches some occurrences that are missed by the previous
rule.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 contrib/coccinelle/free.cocci | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
index 35fb992621..f2d97e755b 100644
--- a/contrib/coccinelle/free.cocci
+++ b/contrib/coccinelle/free.cocci
@@ -17,3 +17,10 @@ T *ptr;
 - free(ptr);
 - ptr = NULL;
 + FREE_AND_NULL(ptr);
+
+@@
+expression E;
+@@
+- free(E);
+- E = NULL;
++ FREE_AND_NULL(E);
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 5/6] coccinelle: make use of the "expression" FREE_AND_NULL() rule
  2017-06-15 22:00                         ` Junio C Hamano
                                             ` (4 preceding siblings ...)
  2017-06-15 23:15                           ` [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL() Ævar Arnfjörð Bjarmason
@ 2017-06-15 23:15                           ` Ævar Arnfjörð Bjarmason
  2017-06-15 23:15                           ` [PATCH v4 6/6] *.[ch] refactoring: make use of the FREE_AND_NULL() macro Ævar Arnfjörð Bjarmason
  6 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 23:15 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason

A follow-up to the existing "expression" rule added in an earlier
change. This manually excludes a few occurrences, mostly things that
resulted in many FREE_AND_NULL() on one line, that'll be manually
fixed in a subsequent change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 blame.c    | 3 +--
 ll-merge.c | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/blame.c b/blame.c
index 843c845cba..1183943960 100644
--- a/blame.c
+++ b/blame.c
@@ -314,8 +314,7 @@ static void fill_origin_blob(struct diff_options *opt,
 static void drop_origin_blob(struct blame_origin *o)
 {
 	if (o->file.ptr) {
-		free(o->file.ptr);
-		o->file.ptr = NULL;
+		FREE_AND_NULL(o->file.ptr);
 	}
 }
 
diff --git a/ll-merge.c b/ll-merge.c
index ac0d4a5d78..91ff519cef 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -209,8 +209,7 @@ static int ll_ext_merge(const struct ll_merge_driver *fn,
 	result->size = st.st_size;
 	result->ptr = xmallocz(result->size);
 	if (read_in_full(fd, result->ptr, result->size) != result->size) {
-		free(result->ptr);
-		result->ptr = NULL;
+		FREE_AND_NULL(result->ptr);
 		result->size = 0;
 	}
  close_bad:
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 6/6] *.[ch] refactoring: make use of the FREE_AND_NULL() macro
  2017-06-15 22:00                         ` Junio C Hamano
                                             ` (5 preceding siblings ...)
  2017-06-15 23:15                           ` [PATCH v4 5/6] coccinelle: make use of the "expression" FREE_AND_NULL() rule Ævar Arnfjörð Bjarmason
@ 2017-06-15 23:15                           ` Ævar Arnfjörð Bjarmason
  6 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-15 23:15 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason

Replace occurrences of `free(ptr); ptr = NULL` which weren't caught by
the coccinelle rule. These fall into two categories:

 - free/NULL assignments one after the other which coccinelle all put
   on one line, which is functionally equivalent code, but very ugly.

 - manually spotted occurrences where the NULL assignment isn't right
   after the free() call.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c       | 15 ++++-----------
 builtin/worktree.c |  6 ++----
 commit-slab.h      |  3 +--
 credential.c       |  6 ++----
 gpg-interface.c    | 15 +++++----------
 grep.c             |  9 +++------
 help.c             |  3 +--
 line-log.c         |  3 +--
 prio-queue.c       |  3 +--
 refs/ref-cache.c   |  3 +--
 rerere.c           |  3 +--
 split-index.c      |  3 +--
 12 files changed, 23 insertions(+), 49 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 138fb98537..80368b6fe6 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1072,17 +1072,10 @@ static void am_next(struct am_state *state)
 {
 	struct object_id head;
 
-	free(state->author_name);
-	state->author_name = NULL;
-
-	free(state->author_email);
-	state->author_email = NULL;
-
-	free(state->author_date);
-	state->author_date = NULL;
-
-	free(state->msg);
-	state->msg = NULL;
+	FREE_AND_NULL(state->author_name);
+	FREE_AND_NULL(state->author_email);
+	FREE_AND_NULL(state->author_date);
+	FREE_AND_NULL(state->msg);
 	state->msg_len = 0;
 
 	unlink(am_path(state, "author-script"));
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 793306ea51..41d1c007a4 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -299,10 +299,8 @@ static int add_worktree(const char *path, const char *refname,
 	}
 
 	is_junk = 0;
-	free(junk_work_tree);
-	free(junk_git_dir);
-	junk_work_tree = NULL;
-	junk_git_dir = NULL;
+	FREE_AND_NULL(junk_work_tree);
+	FREE_AND_NULL(junk_git_dir);
 
 done:
 	if (ret || !opts->keep_locked) {
diff --git a/commit-slab.h b/commit-slab.h
index 42d16dcded..333d81e370 100644
--- a/commit-slab.h
+++ b/commit-slab.h
@@ -82,8 +82,7 @@ static MAYBE_UNUSED void clear_ ##slabname(struct slabname *s)		\
 	for (i = 0; i < s->slab_count; i++)				\
 		free(s->slab[i]);					\
 	s->slab_count = 0;						\
-	free(s->slab);							\
-	s->slab = NULL;							\
+	FREE_AND_NULL(s->slab);						\
 }									\
 									\
 static MAYBE_UNUSED elemtype *slabname## _at_peek(struct slabname *s,	\
diff --git a/credential.c b/credential.c
index 0ab247ff40..924ab58538 100644
--- a/credential.c
+++ b/credential.c
@@ -313,10 +313,8 @@ void credential_reject(struct credential *c)
 	for (i = 0; i < c->helpers.nr; i++)
 		credential_do(c, c->helpers.items[i].string, "erase");
 
-	free(c->username);
-	c->username = NULL;
-	free(c->password);
-	c->password = NULL;
+	FREE_AND_NULL(c->username);
+	FREE_AND_NULL(c->password);
 	c->approved = 0;
 }
 
diff --git a/gpg-interface.c b/gpg-interface.c
index e44cc27da1..3f377f89d0 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -13,16 +13,11 @@ static const char *gpg_program = "gpg";
 
 void signature_check_clear(struct signature_check *sigc)
 {
-	free(sigc->payload);
-	free(sigc->gpg_output);
-	free(sigc->gpg_status);
-	free(sigc->signer);
-	free(sigc->key);
-	sigc->payload = NULL;
-	sigc->gpg_output = NULL;
-	sigc->gpg_status = NULL;
-	sigc->signer = NULL;
-	sigc->key = NULL;
+	FREE_AND_NULL(sigc->payload);
+	FREE_AND_NULL(sigc->gpg_output);
+	FREE_AND_NULL(sigc->gpg_status);
+	FREE_AND_NULL(sigc->signer);
+	FREE_AND_NULL(sigc->key);
 }
 
 static struct {
diff --git a/grep.c b/grep.c
index 11513ec06d..909f35a984 100644
--- a/grep.c
+++ b/grep.c
@@ -1763,12 +1763,9 @@ void grep_source_init(struct grep_source *gs, enum grep_source_type type,
 
 void grep_source_clear(struct grep_source *gs)
 {
-	free(gs->name);
-	gs->name = NULL;
-	free(gs->path);
-	gs->path = NULL;
-	free(gs->identifier);
-	gs->identifier = NULL;
+	FREE_AND_NULL(gs->name);
+	FREE_AND_NULL(gs->path);
+	FREE_AND_NULL(gs->identifier);
 	grep_source_clear_data(gs);
 }
 
diff --git a/help.c b/help.c
index db7f3d79a0..a2f48c0df3 100644
--- a/help.c
+++ b/help.c
@@ -267,9 +267,8 @@ static void add_cmd_list(struct cmdnames *cmds, struct cmdnames *old)
 
 	for (i = 0; i < old->cnt; i++)
 		cmds->names[cmds->cnt++] = old->names[i];
-	free(old->names);
+	FREE_AND_NULL(old->names);
 	old->cnt = 0;
-	old->names = NULL;
 }
 
 /* An empirically derived magic number */
diff --git a/line-log.c b/line-log.c
index 79eded7363..2d60c347ca 100644
--- a/line-log.c
+++ b/line-log.c
@@ -34,9 +34,8 @@ void range_set_init(struct range_set *rs, size_t prealloc)
 
 void range_set_release(struct range_set *rs)
 {
-	free(rs->ranges);
+	FREE_AND_NULL(rs->ranges);
 	rs->alloc = rs->nr = 0;
-	rs->ranges = NULL;
 }
 
 /* dst must be uninitialized! */
diff --git a/prio-queue.c b/prio-queue.c
index fc3860fdcb..126d096727 100644
--- a/prio-queue.c
+++ b/prio-queue.c
@@ -27,10 +27,9 @@ void prio_queue_reverse(struct prio_queue *queue)
 
 void clear_prio_queue(struct prio_queue *queue)
 {
-	free(queue->array);
+	FREE_AND_NULL(queue->array);
 	queue->nr = 0;
 	queue->alloc = 0;
-	queue->array = NULL;
 	queue->insertion_ctr = 0;
 }
 
diff --git a/refs/ref-cache.c b/refs/ref-cache.c
index af2fcb2c12..76bb723c86 100644
--- a/refs/ref-cache.c
+++ b/refs/ref-cache.c
@@ -82,9 +82,8 @@ static void clear_ref_dir(struct ref_dir *dir)
 	int i;
 	for (i = 0; i < dir->nr; i++)
 		free_ref_entry(dir->entries[i]);
-	free(dir->entries);
+	FREE_AND_NULL(dir->entries);
 	dir->sorted = dir->nr = dir->alloc = 0;
-	dir->entries = NULL;
 }
 
 struct ref_entry *create_dir_entry(struct ref_cache *cache,
diff --git a/rerere.c b/rerere.c
index c26c29f87a..61f3804a1e 100644
--- a/rerere.c
+++ b/rerere.c
@@ -39,9 +39,8 @@ static void free_rerere_dirs(void)
 		free(rerere_dir[i]->status);
 		free(rerere_dir[i]);
 	}
-	free(rerere_dir);
+	FREE_AND_NULL(rerere_dir);
 	rerere_dir_nr = rerere_dir_alloc = 0;
-	rerere_dir = NULL;
 }
 
 static void free_rerere_id(struct string_list_item *item)
diff --git a/split-index.c b/split-index.c
index 49bd197f71..20477c6a48 100644
--- a/split-index.c
+++ b/split-index.c
@@ -174,10 +174,9 @@ void merge_base_index(struct index_state *istate)
 
 	ewah_free(si->delete_bitmap);
 	ewah_free(si->replace_bitmap);
-	free(si->saved_cache);
+	FREE_AND_NULL(si->saved_cache);
 	si->delete_bitmap  = NULL;
 	si->replace_bitmap = NULL;
-	si->saved_cache	   = NULL;
 	si->saved_cache_nr = 0;
 }
 
-- 
2.13.1.508.gb3defc5cc


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-15 23:15                           ` [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL() Ævar Arnfjörð Bjarmason
@ 2017-06-16 19:43                             ` Junio C Hamano
  2017-06-25  8:01                               ` René Scharfe
  0 siblings, 1 reply; 214+ messages in thread
From: Junio C Hamano @ 2017-06-16 19:43 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> A follow-up to the existing "type" rule added in an earlier
> change. This catches some occurrences that are missed by the previous
> rule.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---

Hmph, I wonder if the "type" thing is really needed.  Over there,
"ptr" is an expression and we can find "free(ptr); ptr = NULL" with
the rule in this patch already, no?

In any case, I'll queue these patches as-is.  Thanks for working on
this.


>  contrib/coccinelle/free.cocci | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
> index 35fb992621..f2d97e755b 100644
> --- a/contrib/coccinelle/free.cocci
> +++ b/contrib/coccinelle/free.cocci
> @@ -17,3 +17,10 @@ T *ptr;
>  - free(ptr);
>  - ptr = NULL;
>  + FREE_AND_NULL(ptr);
> +
> +@@
> +expression E;
> +@@
> +- free(E);
> +- E = NULL;
> ++ FREE_AND_NULL(E);

^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v3 00/20] repository object
  2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
                     ` (32 preceding siblings ...)
  2017-06-10  0:40   ` [PATCH v2 00/32] " Jonathan Tan
@ 2017-06-20 19:19   ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
                       ` (21 more replies)
  33 siblings, 22 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

I decided to split up the original series into three parts in order to make
review a little bit easier.  As such this series is dependent on on
'bw/config-h' and 'bw/ls-files-sans-the-index' which should be moving into
'next' soon.  Due to this I was able to greatly shrink this series in terms of
number of patches so hopefully it is a little easier to review.

As before you can find this series at:
https://github.com/bmwill/git/tree/repository-object

Chagnes in v3:

 * Once v2 was merged to 'pu' a bug was discovered in how the 'do_git_path'
   function handled worktree's.  v3 addresses this by ensuring to only replace
   'gitdir/index' with 'repository->index_file' when not operating specifically
   on a 'struct worktree'.

 * Added a bunch of path related functions to facilitate construction of paths
   into a repository's gitdir or in a repository's worktree.

 * Added an additional initialization function to allow initializing a 'struct
   repository' as a submodule of another 'struct repository'.

Brandon Williams (20):
  setup: don't perform lazy initialization of repository state
  setup: add comment indicating a hack
  environment: remove namespace_len variable
  repository: introduce the repository object
  environment: place key repository state in the_repository
  environment: store worktree in the_repository
  path: create path.h
  path: always pass in commondir to update_common_dir
  path: convert strbuf_git_common_path to take a 'struct repository'
  path: convert do_git_path to take a 'struct repository'
  path: construct correct path to a worktree's index
  path: add repo_git_path and strbuf_repo_git_path
  path: add repo_worktree_path and strbuf_repo_worktree_path
  config: read config from a repository object
  repository: add index_state to struct repo
  submodule-config: store the_submodule_cache in the_repository
  submodule: add repo_read_gitmodules
  submodule: convert is_submodule_initialized to work on a repository
  repository: enable initialization of submodules
  ls-files: use repository object

 Makefile                               |   1 +
 builtin/grep.c                         |   3 +-
 builtin/ls-files.c                     | 195 +++++++++++---------------
 builtin/submodule--helper.c            |   9 +-
 cache.h                                |  62 +--------
 config.c                               | 216 +++++++++++++++++++----------
 config.h                               |  24 ++++
 environment.c                          |  91 ++++--------
 git.c                                  |   2 +-
 path.c                                 | 136 ++++++++++++++----
 path.h                                 |  82 +++++++++++
 repository.c                           | 245 +++++++++++++++++++++++++++++++++
 repository.h                           |  76 ++++++++++
 setup.c                                |  33 +++++
 submodule-config.c                     |  70 +++++++---
 submodule-config.h                     |  10 ++
 submodule.c                            |  35 +++--
 submodule.h                            |   4 +-
 t/t3007-ls-files-recurse-submodules.sh |  39 ++++++
 worktree.c                             |   3 +-
 20 files changed, 962 insertions(+), 374 deletions(-)
 create mode 100644 path.h
 create mode 100644 repository.c
 create mode 100644 repository.h

-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v3 01/20] setup: don't perform lazy initialization of repository state
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 02/20] setup: add comment indicating a hack Brandon Williams
                       ` (20 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Under some circumstances (bogus GIT_DIR value or the discovered gitdir
is '.git') 'setup_git_directory()' won't initialize key repository
state.  This leads to inconsistent state after running the setup code.
To account for this inconsistent state, lazy initialization is done once
a caller asks for the repository's gitdir or some other piece of
repository state.  This is confusing and can be error prone.

Instead let's tighten the expected outcome of 'setup_git_directory()'
and ensure that it initializes repository state in all cases that would
have been handled by lazy initialization.

This also lets us drop the requirement to have 'have_git_dir()' check if
the environment variable GIT_DIR was set as that will be handled by the
end of the setup code.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  2 ++
 environment.c | 17 ++++++++---------
 setup.c       | 14 ++++++++++++++
 3 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/cache.h b/cache.h
index 96055c222..7c81749a9 100644
--- a/cache.h
+++ b/cache.h
@@ -462,6 +462,8 @@ static inline enum object_type object_type(unsigned int mode)
  */
 extern const char * const local_repo_env[];
 
+extern void setup_git_env(void);
+
 /*
  * Returns true iff we have a configured git repository (either via
  * setup_git_directory, or in the environment via $GIT_DIR).
diff --git a/environment.c b/environment.c
index d40b21fb7..a73b08f5d 100644
--- a/environment.c
+++ b/environment.c
@@ -160,7 +160,7 @@ static char *git_path_from_env(const char *envvar, const char *git_dir,
 	return xstrdup(value);
 }
 
-static void setup_git_env(void)
+void setup_git_env(void)
 {
 	struct strbuf sb = STRBUF_INIT;
 	const char *gitfile;
@@ -205,28 +205,27 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir
-		|| getenv(GIT_DIR_ENVIRONMENT);
+		|| git_dir;
 }
 
 const char *get_git_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_dir;
 }
 
 const char *get_git_common_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_common_dir;
 }
 
 const char *get_git_namespace(void)
 {
 	if (!namespace)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return namespace;
 }
 
@@ -276,7 +275,7 @@ const char *get_git_work_tree(void)
 char *get_object_directory(void)
 {
 	if (!git_object_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_object_dir;
 }
 
@@ -316,14 +315,14 @@ int odb_pack_keep(const char *name)
 char *get_index_file(void)
 {
 	if (!git_index_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_index_file;
 }
 
 char *get_graft_file(void)
 {
 	if (!git_graft_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_graft_file;
 }
 
diff --git a/setup.c b/setup.c
index 358fbc2e5..24a738b0d 100644
--- a/setup.c
+++ b/setup.c
@@ -1091,6 +1091,20 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	startup_info->have_repository = !nongit_ok || !*nongit_ok;
 	startup_info->prefix = prefix;
 
+	/*
+	 * Not all paths through the setup code will call 'set_git_dir()' (which
+	 * directly sets up the environment) so in order to guarantee that the
+	 * environment is in a consistent state after setup, explicitly setup
+	 * the environment if we have a repository.
+	 *
+	 * NEEDSWORK: currently we allow bogus GIT_DIR values to be set in some
+	 * code paths so we also need to explicitly setup the environment if
+	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
+	 * GIT_DIR values at some point in the future.
+	 */
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
+		setup_git_env();
+
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
 
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 02/20] setup: add comment indicating a hack
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 03/20] environment: remove namespace_len variable Brandon Williams
                       ` (19 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

'GIT_TOPLEVEL_PREFIX_ENVIRONMENT' was added in (b58a68c1c setup: allow
for prefix to be passed to git commands) to aid in fixing a bug where
'ls-files' and 'grep' were not able to properly recurse when called from
within a subdirectory.  Add a 'NEEDSWORK' comment indicating that this
envvar should be removed once 'ls-files' and 'grep' can recurse
in-process.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 setup.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/setup.c b/setup.c
index 24a738b0d..b477faa44 100644
--- a/setup.c
+++ b/setup.c
@@ -1079,6 +1079,12 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die("BUG: unhandled setup_git_directory_1() result");
 	}
 
+	/*
+	 * NEEDSWORK: This was a hack in order to get ls-files and grep to have
+	 * properly formated output when recursing submodules.  Once ls-files
+	 * and grep have been changed to perform this recursing in-process this
+	 * needs to be removed.
+	 */
 	env_prefix = getenv(GIT_TOPLEVEL_PREFIX_ENVIRONMENT);
 	if (env_prefix)
 		prefix = env_prefix;
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 03/20] environment: remove namespace_len variable
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 02/20] setup: add comment indicating a hack Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 04/20] repository: introduce the repository object Brandon Williams
                       ` (18 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Use 'skip_prefix' instead of 'starts_with' so that we can drop the need
to keep around 'namespace_len'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index a73b08f5d..e035f6372 100644
--- a/environment.c
+++ b/environment.c
@@ -98,7 +98,6 @@ char *git_work_tree_cfg;
 static char *work_tree;
 
 static const char *namespace;
-static size_t namespace_len;
 
 static const char *super_prefix;
 
@@ -190,7 +189,6 @@ void setup_git_env(void)
 	git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
 							  : "refs/replace/");
 	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
-	namespace_len = strlen(namespace);
 	shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
 	if (shallow_file)
 		set_alternate_shallow_file(shallow_file, 0);
@@ -231,9 +229,10 @@ const char *get_git_namespace(void)
 
 const char *strip_namespace(const char *namespaced_ref)
 {
-	if (!starts_with(namespaced_ref, get_git_namespace()))
-		return NULL;
-	return namespaced_ref + namespace_len;
+	const char *out;
+	if (skip_prefix(namespaced_ref, get_git_namespace(), &out))
+		return out;
+	return NULL;
 }
 
 const char *get_super_prefix(void)
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 04/20] repository: introduce the repository object
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (2 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 03/20] environment: remove namespace_len variable Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:57       ` Stefan Beller
  2017-06-20 21:31       ` Jonathan Tan
  2017-06-20 19:19     ` [PATCH v3 05/20] environment: place key repository state in the_repository Brandon Williams
                       ` (17 subsequent siblings)
  21 siblings, 2 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce the repository object 'struct repository' which can be used to
hold all state pertaining to a git repository.

Some of the benefits of object-ifying a repository are:

  1. Make the code base more readable and easier to reason about.

  2. Allow for working on multiple repositories, specifically
     submodules, within the same process.  Currently the process for
     working on a submodule involves setting up an argv_array of options
     for a particular command and then launching a child process to
     execute the command in the context of the submodule.  This is
     clunky and can require lots of little hacks in order to ensure
     correctness.  Ideally it would be nice to simply pass a repository
     and an options struct to a command.

  3. Eliminating reliance on global state will make it easier to
     enable the use of threading to improve performance.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 Makefile     |   1 +
 repository.c | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 repository.h |  46 ++++++++++++++++
 3 files changed, 219 insertions(+)
 create mode 100644 repository.c
 create mode 100644 repository.h

diff --git a/Makefile b/Makefile
index f48480163..32e4efc71 100644
--- a/Makefile
+++ b/Makefile
@@ -839,6 +839,7 @@ LIB_OBJS += refs/ref-cache.o
 LIB_OBJS += ref-filter.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace_object.o
+LIB_OBJS += repository.o
 LIB_OBJS += rerere.o
 LIB_OBJS += resolve-undo.o
 LIB_OBJS += revision.o
diff --git a/repository.c b/repository.c
new file mode 100644
index 000000000..7c8a7728c
--- /dev/null
+++ b/repository.c
@@ -0,0 +1,172 @@
+#include "cache.h"
+#include "repository.h"
+
+/* The main repository */
+static struct repository the_repo;
+struct repository *the_repository = &the_repo;
+
+static char *git_path_from_env(const char *envvar, const char *git_dir,
+			       const char *path, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(envvar);
+		if (value)
+			return xstrdup(value);
+	}
+
+	return xstrfmt("%s/%s", git_dir, path);
+}
+
+static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
+		if (value) {
+			strbuf_addstr(sb, value);
+			return 1;
+		}
+	}
+
+	return get_common_dir_noenv(sb, gitdir);
+}
+
+/* called after setting gitdir */
+static void repo_setup_env(struct repository *repo)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	if (!repo->gitdir)
+		BUG("gitdir wasn't set before setting up the environment");
+
+	repo->different_commondir = find_common_dir(&sb, repo->gitdir,
+						    !repo->ignore_env);
+	repo->commondir = strbuf_detach(&sb, NULL);
+	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+					    "objects", !repo->ignore_env);
+	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
+					     "info/grafts", !repo->ignore_env);
+	repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
+					     "index", !repo->ignore_env);
+}
+
+void repo_set_gitdir(struct repository *repo, const char *path)
+{
+	const char *gitfile = read_gitfile(path);
+
+	/*
+	 * NEEDSWORK: Eventually we want to be able to free gitdir and the rest
+	 * of the environment before reinitializing it again, but we have some
+	 * crazy code paths where we try to set gitdir with the current gitdir
+	 * and we don't want to free gitdir before copying the passed in value.
+	 */
+	repo->gitdir = xstrdup(gitfile ? gitfile : path);
+
+	repo_setup_env(repo);
+}
+
+/*
+ * Attempt to resolve and set the provided 'gitdir' for repository 'repo'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+static int repo_init_gitdir(struct repository *repo, const char *gitdir)
+{
+	int ret = 0;
+	int error = 0;
+	char *abspath = NULL;
+	char *suspect = NULL;
+	const char *resolved_gitdir;
+
+	abspath = real_pathdup(gitdir, 0);
+	if (!abspath) {
+		ret = -1;
+		goto out;
+	}
+
+	/* First assume 'gitdir' references the gitdir directly */
+	resolved_gitdir = resolve_gitdir_gently(abspath, &error);
+	/* otherwise; try 'gitdir'.git */
+	if (!resolved_gitdir) {
+		suspect = xstrfmt("%s/.git", abspath);
+		resolved_gitdir = resolve_gitdir_gently(suspect, &error);
+		if (!resolved_gitdir) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+	repo_set_gitdir(repo, resolved_gitdir);
+
+out:
+	free(abspath);
+	free(suspect);
+	return ret;
+}
+
+void repo_set_worktree(struct repository *repo, const char *path)
+{
+	repo->worktree = real_pathdup(path, 1);
+}
+
+static int verify_repo_format(struct repository_format *format,
+			      const char *commondir)
+{
+	int ret = 0;
+	struct strbuf sb = STRBUF_INIT;
+
+	strbuf_addf(&sb, "%s/config", commondir);
+	read_repository_format(format, sb.buf);
+	strbuf_reset(&sb);
+
+	if (verify_repository_format(format, &sb) < 0) {
+		warning("%s", sb.buf);
+		ret = -1;
+	}
+
+	strbuf_release(&sb);
+	return ret;
+}
+
+/*
+ * Initialize 'repo' based on the provided 'gitdir'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
+{
+	struct repository_format format;
+	memset(repo, 0, sizeof(*repo));
+
+	repo->ignore_env = 1;
+
+	if (repo_init_gitdir(repo, gitdir))
+		goto error;
+
+	if (verify_repo_format(&format, repo->commondir))
+		goto error;
+
+	if (worktree)
+		repo_set_worktree(repo, worktree);
+
+	return 0;
+
+error:
+	repo_clear(repo);
+	return -1;
+}
+
+void repo_clear(struct repository *repo)
+{
+	free(repo->gitdir);
+	repo->gitdir = NULL;
+	free(repo->commondir);
+	repo->commondir = NULL;
+	free(repo->objectdir);
+	repo->objectdir = NULL;
+	free(repo->graft_file);
+	repo->graft_file = NULL;
+	free(repo->index_file);
+	repo->index_file = NULL;
+	free(repo->worktree);
+	repo->worktree = NULL;
+
+	memset(repo, 0, sizeof(*repo));
+}
diff --git a/repository.h b/repository.h
new file mode 100644
index 000000000..ab1744d51
--- /dev/null
+++ b/repository.h
@@ -0,0 +1,46 @@
+#ifndef REPOSITORY_H
+#define REPOSITORY_H
+
+struct repository {
+	/* Environment */
+	/* Path to the git directory */
+	char *gitdir;
+
+	/* Path to the common git directory */
+	char *commondir;
+
+	/* Path to the repository's object store */
+	char *objectdir;
+
+	/* Path to the repository's graft file */
+	char *graft_file;
+
+	/* Path to the current worktree's index file */
+	char *index_file;
+
+	/* Path to the working directory */
+	char *worktree;
+
+	/* Configurations */
+	/*
+	 * Bit used during initialization to indicate if repository state (like
+	 * the location of the 'objectdir') should be read from the
+	 * environment.  By default this bit will be set at the begining of
+	 * 'repo_init()' so that all repositories will ignore the environment.
+	 * The exception to this is 'the_repository', which doesn't go through
+	 * the normal 'repo_init()' process.
+	 */
+	unsigned ignore_env:1;
+
+	/* Indicate if a repository has a different 'commondir' from 'gitdir' */
+	unsigned different_commondir:1;
+};
+
+extern struct repository *the_repository;
+
+extern void repo_set_gitdir(struct repository *repo, const char *path);
+extern void repo_set_worktree(struct repository *repo, const char *path);
+extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
+extern void repo_clear(struct repository *repo);
+
+#endif /* REPOSITORY_H */
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 05/20] environment: place key repository state in the_repository
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (3 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 04/20] repository: introduce the repository object Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 21:59       ` Jonathan Tan
  2017-06-20 19:19     ` [PATCH v3 06/20] environment: store worktree " Brandon Williams
                       ` (16 subsequent siblings)
  21 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'git_dir', 'git_common_dir', 'git_object_dir', 'git_index_file',
'git_graft_file', and 'namespace' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  1 -
 environment.c | 58 +++++++++++++---------------------------------------------
 path.c        | 11 ++++++-----
 setup.c       | 17 +++++++++++++++--
 4 files changed, 34 insertions(+), 53 deletions(-)

diff --git a/cache.h b/cache.h
index 7c81749a9..cd64cbc81 100644
--- a/cache.h
+++ b/cache.h
@@ -771,7 +771,6 @@ extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
 extern int protect_hfs;
 extern int protect_ntfs;
-extern int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
 
 /*
  * Include broken refs in all ref iterations, which will
diff --git a/environment.c b/environment.c
index e035f6372..aa79ef83e 100644
--- a/environment.c
+++ b/environment.c
@@ -8,6 +8,7 @@
  * are.
  */
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "refs.h"
 #include "fmt-merge-msg.h"
@@ -101,10 +102,6 @@ static const char *namespace;
 
 static const char *super_prefix;
 
-static const char *git_dir, *git_common_dir;
-static char *git_object_dir, *git_index_file, *git_graft_file;
-int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
-
 /*
  * Repository-local GIT_* environment variables; see cache.h for details.
  */
@@ -148,41 +145,11 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-static char *git_path_from_env(const char *envvar, const char *git_dir,
-			       const char *path, int *fromenv)
-{
-	const char *value = getenv(envvar);
-	if (!value)
-		return xstrfmt("%s/%s", git_dir, path);
-	if (fromenv)
-		*fromenv = 1;
-	return xstrdup(value);
-}
-
 void setup_git_env(void)
 {
-	struct strbuf sb = STRBUF_INIT;
-	const char *gitfile;
 	const char *shallow_file;
 	const char *replace_ref_base;
 
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
-	if (!git_dir) {
-		if (!startup_info->have_repository)
-			BUG("setup_git_env called without repository");
-		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
-	}
-	gitfile = read_gitfile(git_dir);
-	git_dir = xstrdup(gitfile ? gitfile : git_dir);
-	if (get_common_dir(&sb, git_dir))
-		git_common_dir_env = 1;
-	git_common_dir = strbuf_detach(&sb, NULL);
-	git_object_dir = git_path_from_env(DB_ENVIRONMENT, git_common_dir,
-					   "objects", &git_db_env);
-	git_index_file = git_path_from_env(INDEX_ENVIRONMENT, git_dir,
-					   "index", &git_index_env);
-	git_graft_file = git_path_from_env(GRAFT_ENVIRONMENT, git_common_dir,
-					   "info/grafts", &git_graft_env);
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		check_replace_refs = 0;
 	replace_ref_base = getenv(GIT_REPLACE_REF_BASE_ENVIRONMENT);
@@ -203,21 +170,21 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir;
+		|| the_repository->gitdir;
 }
 
 const char *get_git_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository->gitdir)
 		BUG("git environment hasn't been setup");
-	return git_dir;
+	return the_repository->gitdir;
 }
 
 const char *get_git_common_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository->commondir)
 		BUG("git environment hasn't been setup");
-	return git_common_dir;
+	return the_repository->commondir;
 }
 
 const char *get_git_namespace(void)
@@ -273,9 +240,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!git_object_dir)
+	if (!the_repository->objectdir)
 		BUG("git environment hasn't been setup");
-	return git_object_dir;
+	return the_repository->objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
@@ -313,22 +280,23 @@ int odb_pack_keep(const char *name)
 
 char *get_index_file(void)
 {
-	if (!git_index_file)
+	if (!the_repository->index_file)
 		BUG("git environment hasn't been setup");
-	return git_index_file;
+	return the_repository->index_file;
 }
 
 char *get_graft_file(void)
 {
-	if (!git_graft_file)
+	if (!the_repository->graft_file)
 		BUG("git environment hasn't been setup");
-	return git_graft_file;
+	return the_repository->graft_file;
 }
 
 int set_git_dir(const char *path)
 {
 	if (setenv(GIT_DIR_ENVIRONMENT, path, 1))
 		return error("Could not set GIT_DIR to '%s'", path);
+	repo_set_gitdir(the_repository, path);
 	setup_git_env();
 	return 0;
 }
diff --git a/path.c b/path.c
index c1cb1cf62..e4abea083 100644
--- a/path.c
+++ b/path.c
@@ -2,6 +2,7 @@
  * Utilities for paths and pathnames
  */
 #include "cache.h"
+#include "repository.h"
 #include "strbuf.h"
 #include "string-list.h"
 #include "dir.h"
@@ -355,7 +356,7 @@ void report_linked_checkout_garbage(void)
 	const struct common_dir *p;
 	int len;
 
-	if (!git_common_dir_env)
+	if (!the_repository->different_commondir)
 		return;
 	strbuf_addf(&sb, "%s/", get_git_dir());
 	len = sb.len;
@@ -374,17 +375,17 @@ void report_linked_checkout_garbage(void)
 static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
-	if (git_graft_env && is_dir_file(base, "info", "grafts"))
+	if (is_dir_file(base, "info", "grafts"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_graft_file(), strlen(get_graft_file()));
-	else if (git_index_env && !strcmp(base, "index"))
+	else if (!strcmp(base, "index"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_index_file(), strlen(get_index_file()));
-	else if (git_db_env && dir_prefix(base, "objects"))
+	else if (dir_prefix(base, "objects"))
 		replace_dir(buf, git_dir_len + 7, get_object_directory());
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
-	else if (git_common_dir_env)
+	else if (the_repository->different_commondir)
 		update_common_dir(buf, git_dir_len, NULL);
 }
 
diff --git a/setup.c b/setup.c
index b477faa44..860507e1f 100644
--- a/setup.c
+++ b/setup.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "dir.h"
 #include "string-list.h"
@@ -398,6 +399,11 @@ void setup_work_tree(void)
 	if (getenv(GIT_WORK_TREE_ENVIRONMENT))
 		setenv(GIT_WORK_TREE_ENVIRONMENT, ".", 1);
 
+	/*
+	 * NEEDSWORK: this call can essentially be set_git_dir(get_git_dir())
+	 * which can cause some problems when trying to free the old value of
+	 * gitdir.
+	 */
 	set_git_dir(remove_leading_path(git_dir, work_tree));
 	initialized = 1;
 }
@@ -1108,8 +1114,15 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
 	 * GIT_DIR values at some point in the future.
 	 */
-	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
-		setup_git_env();
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT)) {
+		if (!the_repository->gitdir) {
+			const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);
+			if (!gitdir)
+				gitdir = DEFAULT_GIT_DIR_ENVIRONMENT;
+			repo_set_gitdir(the_repository, gitdir);
+			setup_git_env();
+		}
+	}
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 06/20] environment: store worktree in the_repository
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (4 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 05/20] environment: place key repository state in the_repository Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 07/20] path: create path.h Brandon Williams
                       ` (15 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'work_tree' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index aa79ef83e..3fd4b1084 100644
--- a/environment.c
+++ b/environment.c
@@ -96,7 +96,6 @@ int ignore_untracked_cache_config;
 
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
 char *git_work_tree_cfg;
-static char *work_tree;
 
 static const char *namespace;
 
@@ -223,19 +222,19 @@ void set_git_work_tree(const char *new_work_tree)
 {
 	if (git_work_tree_initialized) {
 		new_work_tree = real_path(new_work_tree);
-		if (strcmp(new_work_tree, work_tree))
+		if (strcmp(new_work_tree, the_repository->worktree))
 			die("internal error: work tree has already been set\n"
 			    "Current worktree: %s\nNew worktree: %s",
-			    work_tree, new_work_tree);
+			    the_repository->worktree, new_work_tree);
 		return;
 	}
 	git_work_tree_initialized = 1;
-	work_tree = real_pathdup(new_work_tree, 1);
+	repo_set_worktree(the_repository, new_work_tree);
 }
 
 const char *get_git_work_tree(void)
 {
-	return work_tree;
+	return the_repository->worktree;
 }
 
 char *get_object_directory(void)
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 07/20] path: create path.h
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (5 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 06/20] environment: store worktree " Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 08/20] path: always pass in commondir to update_common_dir Brandon Williams
                       ` (14 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Move all path related declarations from cache.h to a new path.h header
file.  This makes cache.h smaller and makes it easier to add new path
related functions.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h | 59 +----------------------------------------------------------
 path.c  |  1 +
 path.h  | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+), 58 deletions(-)
 create mode 100644 path.h

diff --git a/cache.h b/cache.h
index cd64cbc81..c958fc3ce 100644
--- a/cache.h
+++ b/cache.h
@@ -11,6 +11,7 @@
 #include "string-list.h"
 #include "pack-revindex.h"
 #include "hash.h"
+#include "path.h"
 
 #ifndef platform_SHA_CTX
 /*
@@ -892,64 +893,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Return a statically allocated filename, either generically (mkpath), in
- * the repository directory (git_path), or in a submodule's repository
- * directory (git_path_submodule). In all cases, note that the result
- * may be overwritten by another call to _any_ of the functions. Consider
- * using the safer "dup" or "strbuf" formats below (in some cases, the
- * unsafe versions have already been removed).
- */
-extern const char *mkpath(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
-extern const char *git_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
-extern const char *git_common_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
-
-extern char *mksnpath(char *buf, size_t n, const char *fmt, ...)
-	__attribute__((format (printf, 3, 4)));
-extern void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-extern void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-extern char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-extern int strbuf_git_path_submodule(struct strbuf *sb, const char *path,
-				     const char *fmt, ...)
-	__attribute__((format (printf, 3, 4)));
-extern char *git_pathdup(const char *fmt, ...)
-	__attribute__((format (printf, 1, 2)));
-extern char *mkpathdup(const char *fmt, ...)
-	__attribute__((format (printf, 1, 2)));
-extern char *git_pathdup_submodule(const char *path, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-
-extern void report_linked_checkout_garbage(void);
-
-/*
- * You can define a static memoized git path like:
- *
- *    static GIT_PATH_FUNC(git_path_foo, "FOO");
- *
- * or use one of the global ones below.
- */
-#define GIT_PATH_FUNC(func, filename) \
-	const char *func(void) \
-	{ \
-		static char *ret; \
-		if (!ret) \
-			ret = git_pathdup(filename); \
-		return ret; \
-	}
-
-const char *git_path_cherry_pick_head(void);
-const char *git_path_revert_head(void);
-const char *git_path_squash_msg(void);
-const char *git_path_merge_msg(void);
-const char *git_path_merge_rr(void);
-const char *git_path_merge_mode(void);
-const char *git_path_merge_head(void);
-const char *git_path_fetch_head(void);
-const char *git_path_shallow(void);
-
 /*
  * Return the name of the file in the local object database that would
  * be used to store a loose object with the specified sha1.  The
diff --git a/path.c b/path.c
index e4abea083..41c861c96 100644
--- a/path.c
+++ b/path.c
@@ -8,6 +8,7 @@
 #include "dir.h"
 #include "worktree.h"
 #include "submodule-config.h"
+#include "path.h"
 
 static int get_st_mode_bits(const char *path, int *mode)
 {
diff --git a/path.h b/path.h
new file mode 100644
index 000000000..522cd029b
--- /dev/null
+++ b/path.h
@@ -0,0 +1,62 @@
+#ifndef PATH_H
+#define PATH_H
+
+/*
+ * Return a statically allocated filename, either generically (mkpath), in
+ * the repository directory (git_path), or in a submodule's repository
+ * directory (git_path_submodule). In all cases, note that the result
+ * may be overwritten by another call to _any_ of the functions. Consider
+ * using the safer "dup" or "strbuf" formats below (in some cases, the
+ * unsafe versions have already been removed).
+ */
+extern const char *mkpath(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
+extern const char *git_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
+extern const char *git_common_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
+
+extern char *mksnpath(char *buf, size_t n, const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+extern void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern int strbuf_git_path_submodule(struct strbuf *sb, const char *path,
+				     const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+extern char *git_pathdup(const char *fmt, ...)
+	__attribute__((format (printf, 1, 2)));
+extern char *mkpathdup(const char *fmt, ...)
+	__attribute__((format (printf, 1, 2)));
+extern char *git_pathdup_submodule(const char *path, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+
+extern void report_linked_checkout_garbage(void);
+
+/*
+ * You can define a static memoized git path like:
+ *
+ *    static GIT_PATH_FUNC(git_path_foo, "FOO");
+ *
+ * or use one of the global ones below.
+ */
+#define GIT_PATH_FUNC(func, filename) \
+	const char *func(void) \
+	{ \
+		static char *ret; \
+		if (!ret) \
+			ret = git_pathdup(filename); \
+		return ret; \
+	}
+
+const char *git_path_cherry_pick_head(void);
+const char *git_path_revert_head(void);
+const char *git_path_squash_msg(void);
+const char *git_path_merge_msg(void);
+const char *git_path_merge_rr(void);
+const char *git_path_merge_mode(void);
+const char *git_path_merge_head(void);
+const char *git_path_fetch_head(void);
+const char *git_path_shallow(void);
+
+#endif /* PATH_H */
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 08/20] path: always pass in commondir to update_common_dir
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (6 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 07/20] path: create path.h Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 09/20] path: convert strbuf_git_common_path to take a 'struct repository' Brandon Williams
                       ` (13 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Instead of passing in 'NULL' and having 'update_common_dir()' query for
the commondir, have the callers of 'update_common_dir()' be responsible
for providing the commondir.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/path.c b/path.c
index 41c861c96..2434921d8 100644
--- a/path.c
+++ b/path.c
@@ -345,8 +345,6 @@ static void update_common_dir(struct strbuf *buf, int git_dir_len,
 {
 	char *base = buf->buf + git_dir_len;
 	init_common_trie();
-	if (!common_dir)
-		common_dir = get_git_common_dir();
 	if (trie_find(&common_trie, base, check_common, NULL) > 0)
 		replace_dir(buf, git_dir_len, common_dir);
 }
@@ -387,7 +385,7 @@ static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (the_repository->different_commondir)
-		update_common_dir(buf, git_dir_len, NULL);
+		update_common_dir(buf, git_dir_len, get_git_common_dir());
 }
 
 static void do_git_path(const struct worktree *wt, struct strbuf *buf,
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 09/20] path: convert strbuf_git_common_path to take a 'struct repository'
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (7 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 08/20] path: always pass in commondir to update_common_dir Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 10/20] path: convert do_git_path " Brandon Williams
                       ` (12 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c     | 13 ++++++++-----
 path.h     |  8 ++++++--
 worktree.c |  3 ++-
 3 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/path.c b/path.c
index 2434921d8..9be6804a9 100644
--- a/path.c
+++ b/path.c
@@ -524,11 +524,12 @@ int strbuf_git_path_submodule(struct strbuf *buf, const char *path,
 	return err;
 }
 
-static void do_git_common_path(struct strbuf *buf,
+static void do_git_common_path(const struct repository *repo,
+			       struct strbuf *buf,
 			       const char *fmt,
 			       va_list args)
 {
-	strbuf_addstr(buf, get_git_common_dir());
+	strbuf_addstr(buf, repo->commondir);
 	if (buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
 		strbuf_addch(buf, '/');
 	strbuf_vaddf(buf, fmt, args);
@@ -540,16 +541,18 @@ const char *git_common_path(const char *fmt, ...)
 	struct strbuf *pathname = get_pathname();
 	va_list args;
 	va_start(args, fmt);
-	do_git_common_path(pathname, fmt, args);
+	do_git_common_path(the_repository, pathname, fmt, args);
 	va_end(args);
 	return pathname->buf;
 }
 
-void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
+void strbuf_git_common_path(struct strbuf *sb,
+			    const struct repository *repo,
+			    const char *fmt, ...)
 {
 	va_list args;
 	va_start(args, fmt);
-	do_git_common_path(sb, fmt, args);
+	do_git_common_path(repo, sb, fmt, args);
 	va_end(args);
 }
 
diff --git a/path.h b/path.h
index 522cd029b..568d63be5 100644
--- a/path.h
+++ b/path.h
@@ -1,6 +1,8 @@
 #ifndef PATH_H
 #define PATH_H
 
+struct repository;
+
 /*
  * Return a statically allocated filename, either generically (mkpath), in
  * the repository directory (git_path), or in a submodule's repository
@@ -17,8 +19,10 @@ extern char *mksnpath(char *buf, size_t n, const char *fmt, ...)
 	__attribute__((format (printf, 3, 4)));
 extern void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
 	__attribute__((format (printf, 2, 3)));
-extern void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
+extern void strbuf_git_common_path(struct strbuf *sb,
+				   const struct repository *repo,
+				   const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
 extern char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
 	__attribute__((format (printf, 2, 3)));
 extern int strbuf_git_path_submodule(struct strbuf *sb, const char *path,
diff --git a/worktree.c b/worktree.c
index 2801c6d52..e28ffbeb0 100644
--- a/worktree.c
+++ b/worktree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
 #include "strbuf.h"
 #include "worktree.h"
@@ -76,7 +77,7 @@ static struct worktree *get_linked_worktree(const char *id)
 	if (!id)
 		die("Missing linked worktree name");
 
-	strbuf_git_common_path(&path, "worktrees/%s/gitdir", id);
+	strbuf_git_common_path(&path, the_repository, "worktrees/%s/gitdir", id);
 	if (strbuf_read_file(&worktree_path, path.buf, 0) <= 0)
 		/* invalid gitdir file */
 		goto done;
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 10/20] path: convert do_git_path to take a 'struct repository'
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (8 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 09/20] path: convert strbuf_git_common_path to take a 'struct repository' Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 22:23       ` Jonathan Tan
  2017-06-20 19:19     ` [PATCH v3 11/20] path: construct correct path to a worktree's index Brandon Williams
                       ` (11 subsequent siblings)
  21 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

In preparation to adding 'git_path' like functions which operate on a
'struct repository' convert 'do_git_path' to take a 'struct repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 42 ++++++++++++++++++++++++++++--------------
 1 file changed, 28 insertions(+), 14 deletions(-)

diff --git a/path.c b/path.c
index 9be6804a9..76a872297 100644
--- a/path.c
+++ b/path.c
@@ -371,33 +371,47 @@ void report_linked_checkout_garbage(void)
 	strbuf_release(&sb);
 }
 
-static void adjust_git_path(struct strbuf *buf, int git_dir_len)
+static void adjust_git_path(const struct repository *repo,
+			    struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
 	if (is_dir_file(base, "info", "grafts"))
 		strbuf_splice(buf, 0, buf->len,
-			      get_graft_file(), strlen(get_graft_file()));
+			      repo->graft_file, strlen(repo->graft_file));
 	else if (!strcmp(base, "index"))
 		strbuf_splice(buf, 0, buf->len,
-			      get_index_file(), strlen(get_index_file()));
+			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, get_object_directory());
+		replace_dir(buf, git_dir_len + 7, repo->objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
-	else if (the_repository->different_commondir)
-		update_common_dir(buf, git_dir_len, get_git_common_dir());
+	else if (repo->different_commondir)
+		update_common_dir(buf, git_dir_len, repo->commondir);
 }
 
-static void do_git_path(const struct worktree *wt, struct strbuf *buf,
+static void strbuf_worktree_gitdir(struct strbuf *buf,
+				   const struct repository *repo,
+				   const struct worktree *wt)
+{
+	if (!wt)
+		strbuf_addstr(buf, repo->gitdir);
+	else if (!wt->id)
+		strbuf_addstr(buf, repo->commondir);
+	else
+		strbuf_git_common_path(buf, repo, "worktrees/%s", wt->id);
+}
+
+static void do_git_path(const struct repository *repo,
+			const struct worktree *wt, struct strbuf *buf,
 			const char *fmt, va_list args)
 {
 	int gitdir_len;
-	strbuf_addstr(buf, get_worktree_git_dir(wt));
+	strbuf_worktree_gitdir(buf, repo, wt);
 	if (buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
 		strbuf_addch(buf, '/');
 	gitdir_len = buf->len;
 	strbuf_vaddf(buf, fmt, args);
-	adjust_git_path(buf, gitdir_len);
+	adjust_git_path(repo, buf, gitdir_len);
 	strbuf_cleanup_path(buf);
 }
 
@@ -406,7 +420,7 @@ char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
 	va_list args;
 	strbuf_reset(buf);
 	va_start(args, fmt);
-	do_git_path(NULL, buf, fmt, args);
+	do_git_path(the_repository, NULL, buf, fmt, args);
 	va_end(args);
 	return buf->buf;
 }
@@ -415,7 +429,7 @@ void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
 {
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(NULL, sb, fmt, args);
+	do_git_path(the_repository, NULL, sb, fmt, args);
 	va_end(args);
 }
 
@@ -424,7 +438,7 @@ const char *git_path(const char *fmt, ...)
 	struct strbuf *pathname = get_pathname();
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(NULL, pathname, fmt, args);
+	do_git_path(the_repository, NULL, pathname, fmt, args);
 	va_end(args);
 	return pathname->buf;
 }
@@ -434,7 +448,7 @@ char *git_pathdup(const char *fmt, ...)
 	struct strbuf path = STRBUF_INIT;
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(NULL, &path, fmt, args);
+	do_git_path(the_repository, NULL, &path, fmt, args);
 	va_end(args);
 	return strbuf_detach(&path, NULL);
 }
@@ -465,7 +479,7 @@ const char *worktree_git_path(const struct worktree *wt, const char *fmt, ...)
 	struct strbuf *pathname = get_pathname();
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(wt, pathname, fmt, args);
+	do_git_path(the_repository, wt, pathname, fmt, args);
 	va_end(args);
 	return pathname->buf;
 }
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (9 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 10/20] path: convert do_git_path " Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 21:10       ` Stefan Beller
                         ` (2 more replies)
  2017-06-20 19:19     ` [PATCH v3 12/20] path: add repo_git_path and strbuf_repo_git_path Brandon Williams
                       ` (10 subsequent siblings)
  21 siblings, 3 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

When working with worktrees the git directory is split into two part,
the per-worktree gitdir and a commondir which contains things which are
shared among all worktrees (like the object store).  With this notion of
having a split git directory, 557bd833b (git_path(): be aware of file
relocation in $GIT_DIR) and c7b3a3d2f ($GIT_COMMON_DIR: a new
environment variable) changed the way that 'git_path()' functioned so
that paths would be adjusted if they referred to files or directories
that are stored in the commondir or have been changed via an environment
variable.

One interesting problem with this is the index file as it is not shared
across worktrees yet when asking for a path to a specific worktree's
index it will be replaced with a path to the current worktree's index.
In order to prevent this, teach 'adjuct_git_path' to replace the
path to the index with the path recorded in a repository (which would be
the current, active worktree) only when not asked to construct a path
for a specific worktree.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/path.c b/path.c
index 76a872297..c6832a30e 100644
--- a/path.c
+++ b/path.c
@@ -372,13 +372,20 @@ void report_linked_checkout_garbage(void)
 }
 
 static void adjust_git_path(const struct repository *repo,
+			    const struct worktree *wt,
 			    struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
 	if (is_dir_file(base, "info", "grafts"))
 		strbuf_splice(buf, 0, buf->len,
 			      repo->graft_file, strlen(repo->graft_file));
-	else if (!strcmp(base, "index"))
+	/*
+	 * Only try to replace the path '$gitdir/index' with the index file
+	 * recorded in the repository when not constructing a path for a
+	 * worktree.  This way we can retrieve the correct path to a particular
+	 * worktree's index file.
+	 */
+	else if (!wt && !strcmp(base, "index"))
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
@@ -411,7 +418,7 @@ static void do_git_path(const struct repository *repo,
 		strbuf_addch(buf, '/');
 	gitdir_len = buf->len;
 	strbuf_vaddf(buf, fmt, args);
-	adjust_git_path(repo, buf, gitdir_len);
+	adjust_git_path(repo, wt, buf, gitdir_len);
 	strbuf_cleanup_path(buf);
 }
 
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 12/20] path: add repo_git_path and strbuf_repo_git_path
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (10 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 11/20] path: construct correct path to a worktree's index Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path Brandon Williams
                       ` (9 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce 'repo_git_path' and 'strbuf_repo_git_path' which take a
repository struct and constructs a path into the repository's git
directory.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 21 +++++++++++++++++++++
 path.h |  8 ++++++++
 2 files changed, 29 insertions(+)

diff --git a/path.c b/path.c
index c6832a30e..3d223e0b2 100644
--- a/path.c
+++ b/path.c
@@ -422,6 +422,27 @@ static void do_git_path(const struct repository *repo,
 	strbuf_cleanup_path(buf);
 }
 
+char *repo_git_path(const struct repository *repo,
+		    const char *fmt, ...)
+{
+	struct strbuf path = STRBUF_INIT;
+	va_list args;
+	va_start(args, fmt);
+	do_git_path(repo, NULL, &path, fmt, args);
+	va_end(args);
+	return strbuf_detach(&path, NULL);
+}
+
+void strbuf_repo_git_path(struct strbuf *sb,
+			  const struct repository *repo,
+			  const char *fmt, ...)
+{
+	va_list args;
+	va_start(args, fmt);
+	do_git_path(repo, NULL, sb, fmt, args);
+	va_end(args);
+}
+
 char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
 {
 	va_list args;
diff --git a/path.h b/path.h
index 568d63be5..c779c4aa2 100644
--- a/path.h
+++ b/path.h
@@ -35,6 +35,14 @@ extern char *mkpathdup(const char *fmt, ...)
 extern char *git_pathdup_submodule(const char *path, const char *fmt, ...)
 	__attribute__((format (printf, 2, 3)));
 
+extern char *repo_git_path(const struct repository *repo,
+			   const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern void strbuf_repo_git_path(struct strbuf *sb,
+				 const struct repository *repo,
+				 const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+
 extern void report_linked_checkout_garbage(void);
 
 /*
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (11 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 12/20] path: add repo_git_path and strbuf_repo_git_path Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 14/20] config: read config from a repository object Brandon Williams
                       ` (8 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce 'repo_worktree_path' and 'strbuf_repo_worktree_path' which
take a repository struct and constructs a path relative to the
repository's worktree.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 41 +++++++++++++++++++++++++++++++++++++++++
 path.h |  8 ++++++++
 2 files changed, 49 insertions(+)

diff --git a/path.c b/path.c
index 3d223e0b2..a32c19547 100644
--- a/path.c
+++ b/path.c
@@ -512,6 +512,47 @@ const char *worktree_git_path(const struct worktree *wt, const char *fmt, ...)
 	return pathname->buf;
 }
 
+static void do_worktree_path(const struct repository *repo,
+			     struct strbuf *buf,
+			     const char *fmt, va_list args)
+{
+	strbuf_addstr(buf, repo->worktree);
+	if(buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
+		strbuf_addch(buf, '/');
+
+	strbuf_vaddf(buf, fmt, args);
+	strbuf_cleanup_path(buf);
+}
+
+char *repo_worktree_path(const struct repository *repo, const char *fmt, ...)
+{
+	struct strbuf path = STRBUF_INIT;
+	va_list args;
+
+	if (!repo->worktree)
+		return NULL;
+
+	va_start(args, fmt);
+	do_worktree_path(repo, &path, fmt, args);
+	va_end(args);
+
+	return strbuf_detach(&path, NULL);
+}
+
+void strbuf_repo_worktree_path(struct strbuf *sb,
+			       const struct repository *repo,
+			       const char *fmt, ...)
+{
+	va_list args;
+
+	if (!repo->worktree)
+		return;
+
+	va_start(args, fmt);
+	do_worktree_path(repo, sb, fmt, args);
+	va_end(args);
+}
+
 /* Returns 0 on success, negative on failure. */
 static int do_submodule_path(struct strbuf *buf, const char *path,
 			     const char *fmt, va_list args)
diff --git a/path.h b/path.h
index c779c4aa2..9541620c7 100644
--- a/path.h
+++ b/path.h
@@ -43,6 +43,14 @@ extern void strbuf_repo_git_path(struct strbuf *sb,
 				 const char *fmt, ...)
 	__attribute__((format (printf, 3, 4)));
 
+extern char *repo_worktree_path(const struct repository *repo,
+				const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern void strbuf_repo_worktree_path(struct strbuf *sb,
+				      const struct repository *repo,
+				      const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+
 extern void report_linked_checkout_garbage(void);
 
 /*
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 14/20] config: read config from a repository object
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (12 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 15/20] repository: add index_state to struct repo Brandon Williams
                       ` (7 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Teach the config machinery to read config information from a repository
object.  This involves storing a 'struct config_set' inside the
repository object and adding a number of functions (repo_config*) to be
able to query a repository's config.

The current config API enables lazy-loading of the config.  This means
that when 'git_config_get_int()' is called, if the_config_set hasn't
been populated yet, then it will be populated and properly initialized by
reading the necessary config files (system wide .gitconfig, user's home
.gitconfig, and the repository's config).  To maintain this paradigm,
the new API to read from a repository object's config will also perform
this lazy-initialization.

Since both APIs (git_config_get* and repo_config_get*) have the same
semantics we can migrate the default config to be stored within
'the_repository' and just have the 'git_config_get*' family of functions
redirect to the 'repo_config_get*' functions.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 config.c     | 216 +++++++++++++++++++++++++++++++++++++++--------------------
 config.h     |  24 +++++++
 repository.c |   7 ++
 repository.h |  10 +++
 4 files changed, 183 insertions(+), 74 deletions(-)

diff --git a/config.c b/config.c
index 6f0f8b30f..be1c640a4 100644
--- a/config.c
+++ b/config.c
@@ -7,6 +7,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "lockfile.h"
 #include "exec_cmd.h"
 #include "strbuf.h"
@@ -72,13 +73,6 @@ static int core_compression_seen;
 static int pack_compression_seen;
 static int zlib_compression_seen;
 
-/*
- * Default config_set that contains key-value pairs from the usual set of config
- * config files (i.e repo specific .git/config, user wide ~/.gitconfig, XDG
- * config file and the global /etc/gitconfig)
- */
-static struct config_set the_config_set;
-
 static int config_file_fgetc(struct config_source *conf)
 {
 	return getc_unlocked(conf->u.file);
@@ -1605,31 +1599,6 @@ int config_with_options(config_fn_t fn, void *data,
 	return do_git_config_sequence(opts, fn, data);
 }
 
-static void git_config_raw(config_fn_t fn, void *data)
-{
-	struct config_options opts = {0};
-
-	opts.respect_includes = 1;
-	if (have_git_dir()) {
-		opts.commondir = get_git_common_dir();
-		opts.git_dir = get_git_dir();
-	}
-
-	if (config_with_options(fn, data, NULL, &opts) < 0)
-		/*
-		 * config_with_options() normally returns only
-		 * zero, as most errors are fatal, and
-		 * non-fatal potential errors are guarded by "if"
-		 * statements that are entered only when no error is
-		 * possible.
-		 *
-		 * If we ever encounter a non-fatal error, it means
-		 * something went really wrong and we should stop
-		 * immediately.
-		 */
-		die(_("unknown error occurred while reading the configuration files"));
-}
-
 static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 {
 	int i, value_index;
@@ -1683,14 +1652,6 @@ void read_early_config(config_fn_t cb, void *data)
 	strbuf_release(&gitdir);
 }
 
-static void git_config_check_init(void);
-
-void git_config(config_fn_t fn, void *data)
-{
-	git_config_check_init();
-	configset_iter(&the_config_set, fn, data);
-}
-
 static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
 {
 	struct config_set_element k;
@@ -1900,87 +1861,194 @@ int git_configset_get_pathname(struct config_set *cs, const char *key, const cha
 		return 1;
 }
 
-static void git_config_check_init(void)
+/* Functions use to read configuration from a repository */
+static void repo_read_config(struct repository *repo)
 {
-	if (the_config_set.hash_initialized)
+	struct config_options opts;
+
+	opts.respect_includes = 1;
+	opts.commondir = repo->commondir;
+	opts.git_dir = repo->gitdir;
+
+	if (!repo->config)
+		repo->config = xcalloc(1, sizeof(struct config_set));
+	else
+		git_configset_clear(repo->config);
+
+	git_configset_init(repo->config);
+
+	if (config_with_options(config_set_callback, repo->config, NULL, &opts) < 0)
+		/*
+		 * config_with_options() normally returns only
+		 * zero, as most errors are fatal, and
+		 * non-fatal potential errors are guarded by "if"
+		 * statements that are entered only when no error is
+		 * possible.
+		 *
+		 * If we ever encounter a non-fatal error, it means
+		 * something went really wrong and we should stop
+		 * immediately.
+		 */
+		die(_("unknown error occurred while reading the configuration files"));
+}
+
+static void git_config_check_init(struct repository *repo)
+{
+	if (repo->config && repo->config->hash_initialized)
 		return;
-	git_configset_init(&the_config_set);
-	git_config_raw(config_set_callback, &the_config_set);
+	repo_read_config(repo);
 }
 
-void git_config_clear(void)
+static void repo_config_clear(struct repository *repo)
 {
-	if (!the_config_set.hash_initialized)
+	if (!repo->config || !repo->config->hash_initialized)
 		return;
-	git_configset_clear(&the_config_set);
+	git_configset_clear(repo->config);
 }
 
-int git_config_get_value(const char *key, const char **value)
+void repo_config(struct repository *repo, config_fn_t fn, void *data)
 {
-	git_config_check_init();
-	return git_configset_get_value(&the_config_set, key, value);
+	git_config_check_init(repo);
+	configset_iter(repo->config, fn, data);
 }
 
-const struct string_list *git_config_get_value_multi(const char *key)
+int repo_config_get_value(struct repository *repo,
+			  const char *key, const char **value)
 {
-	git_config_check_init();
-	return git_configset_get_value_multi(&the_config_set, key);
+	git_config_check_init(repo);
+	return git_configset_get_value(repo->config, key, value);
 }
 
-int git_config_get_string_const(const char *key, const char **dest)
+const struct string_list *repo_config_get_value_multi(struct repository *repo,
+						      const char *key)
+{
+	git_config_check_init(repo);
+	return git_configset_get_value_multi(repo->config, key);
+}
+
+int repo_config_get_string_const(struct repository *repo,
+				 const char *key, const char **dest)
+{
+	int ret;
+	git_config_check_init(repo);
+	ret = git_configset_get_string_const(repo->config, key, dest);
+	if (ret < 0)
+		git_die_config(key, NULL);
+	return ret;
+}
+
+int repo_config_get_string(struct repository *repo,
+			   const char *key, char **dest)
+{
+	git_config_check_init(repo);
+	return repo_config_get_string_const(repo, key, (const char **)dest);
+}
+
+int repo_config_get_int(struct repository *repo,
+			const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_int(repo->config, key, dest);
+}
+
+int repo_config_get_ulong(struct repository *repo,
+			  const char *key, unsigned long *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_ulong(repo->config, key, dest);
+}
+
+int repo_config_get_bool(struct repository *repo,
+			 const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_bool(repo->config, key, dest);
+}
+
+int repo_config_get_bool_or_int(struct repository *repo,
+				const char *key, int *is_bool, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_bool_or_int(repo->config, key, is_bool, dest);
+}
+
+int repo_config_get_maybe_bool(struct repository *repo,
+			       const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_maybe_bool(repo->config, key, dest);
+}
+
+int repo_config_get_pathname(struct repository *repo,
+			     const char *key, const char **dest)
 {
 	int ret;
-	git_config_check_init();
-	ret = git_configset_get_string_const(&the_config_set, key, dest);
+	git_config_check_init(repo);
+	ret = git_configset_get_pathname(repo->config, key, dest);
 	if (ret < 0)
 		git_die_config(key, NULL);
 	return ret;
 }
 
+/* Functions used historically to read configuration from 'the_repository' */
+void git_config(config_fn_t fn, void *data)
+{
+	repo_config(the_repository, fn, data);
+}
+
+void git_config_clear(void)
+{
+	repo_config_clear(the_repository);
+}
+
+int git_config_get_value(const char *key, const char **value)
+{
+	return repo_config_get_value(the_repository, key, value);
+}
+
+const struct string_list *git_config_get_value_multi(const char *key)
+{
+	return repo_config_get_value_multi(the_repository, key);
+}
+
+int git_config_get_string_const(const char *key, const char **dest)
+{
+	return repo_config_get_string_const(the_repository, key, dest);
+}
+
 int git_config_get_string(const char *key, char **dest)
 {
-	git_config_check_init();
-	return git_config_get_string_const(key, (const char **)dest);
+	return repo_config_get_string(the_repository, key, dest);
 }
 
 int git_config_get_int(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_int(&the_config_set, key, dest);
+	return repo_config_get_int(the_repository, key, dest);
 }
 
 int git_config_get_ulong(const char *key, unsigned long *dest)
 {
-	git_config_check_init();
-	return git_configset_get_ulong(&the_config_set, key, dest);
+	return repo_config_get_ulong(the_repository, key, dest);
 }
 
 int git_config_get_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool(&the_config_set, key, dest);
+	return repo_config_get_bool(the_repository, key, dest);
 }
 
 int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool_or_int(&the_config_set, key, is_bool, dest);
+	return repo_config_get_bool_or_int(the_repository, key, is_bool, dest);
 }
 
 int git_config_get_maybe_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_maybe_bool(&the_config_set, key, dest);
+	return repo_config_get_maybe_bool(the_repository, key, dest);
 }
 
 int git_config_get_pathname(const char *key, const char **dest)
 {
-	int ret;
-	git_config_check_init();
-	ret = git_configset_get_pathname(&the_config_set, key, dest);
-	if (ret < 0)
-		git_die_config(key, NULL);
-	return ret;
+	return repo_config_get_pathname(the_repository, key, dest);
 }
 
 int git_config_get_expiry(const char *key, const char **output)
diff --git a/config.h b/config.h
index 9e038cce2..0352da117 100644
--- a/config.h
+++ b/config.h
@@ -163,6 +163,30 @@ extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key,
 extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
 extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
 
+/* Functions for reading a repository's config */
+struct repository;
+extern void repo_config(struct repository *repo, config_fn_t fn, void *data);
+extern int repo_config_get_value(struct repository *repo,
+				 const char *key, const char **value);
+extern const struct string_list *repo_config_get_value_multi(struct repository *repo,
+							     const char *key);
+extern int repo_config_get_string_const(struct repository *repo,
+					const char *key, const char **dest);
+extern int repo_config_get_string(struct repository *repo,
+				  const char *key, char **dest);
+extern int repo_config_get_int(struct repository *repo,
+			       const char *key, int *dest);
+extern int repo_config_get_ulong(struct repository *repo,
+				 const char *key, unsigned long *dest);
+extern int repo_config_get_bool(struct repository *repo,
+				const char *key, int *dest);
+extern int repo_config_get_bool_or_int(struct repository *repo,
+				       const char *key, int *is_bool, int *dest);
+extern int repo_config_get_maybe_bool(struct repository *repo,
+				      const char *key, int *dest);
+extern int repo_config_get_pathname(struct repository *repo,
+				    const char *key, const char **dest);
+
 extern int git_config_get_value(const char *key, const char **value);
 extern const struct string_list *git_config_get_value_multi(const char *key);
 extern void git_config_clear(void);
diff --git a/repository.c b/repository.c
index 7c8a7728c..3dcfec3b8 100644
--- a/repository.c
+++ b/repository.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "repository.h"
+#include "config.h"
 
 /* The main repository */
 static struct repository the_repo;
@@ -168,5 +169,11 @@ void repo_clear(struct repository *repo)
 	free(repo->worktree);
 	repo->worktree = NULL;
 
+	if (repo->config) {
+		git_configset_clear(repo->config);
+		free(repo->config);
+		repo->config = NULL;
+	}
+
 	memset(repo, 0, sizeof(*repo));
 }
diff --git a/repository.h b/repository.h
index ab1744d51..379b64170 100644
--- a/repository.h
+++ b/repository.h
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+struct config_set;
+
 struct repository {
 	/* Environment */
 	/* Path to the git directory */
@@ -21,6 +23,14 @@ struct repository {
 	/* Path to the working directory */
 	char *worktree;
 
+	/* Subsystems */
+	/*
+	 * Repository's config which contains key-value pairs from the usual
+	 * set of config files (i.e. repo specific .git/config, user wide
+	 * ~/.gitconfig, XDG config file and the global /etc/gitconfig)
+	 */
+	struct config_set *config;
+
 	/* Configurations */
 	/*
 	 * Bit used during initialization to indicate if repository state (like
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 15/20] repository: add index_state to struct repo
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (13 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 14/20] config: read config from a repository object Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-21 22:50       ` Jonathan Tan
  2017-06-20 19:19     ` [PATCH v3 16/20] submodule-config: store the_submodule_cache in the_repository Brandon Williams
                       ` (6 subsequent siblings)
  21 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c | 16 ++++++++++++++++
 repository.h |  6 ++++++
 2 files changed, 22 insertions(+)

diff --git a/repository.c b/repository.c
index 3dcfec3b8..883e6e9e9 100644
--- a/repository.c
+++ b/repository.c
@@ -175,5 +175,21 @@ void repo_clear(struct repository *repo)
 		repo->config = NULL;
 	}
 
+	if (repo->index) {
+		discard_index(repo->index);
+		free(repo->index);
+		repo->index = NULL;
+	}
+
 	memset(repo, 0, sizeof(*repo));
 }
+
+int repo_read_index(struct repository *repo)
+{
+	if (!repo->index)
+		repo->index = xcalloc(1, sizeof(struct index_state));
+	else
+		discard_index(repo->index);
+
+	return read_index_from(repo->index, repo->index_file);
+}
diff --git a/repository.h b/repository.h
index 379b64170..1fa65c42f 100644
--- a/repository.h
+++ b/repository.h
@@ -2,6 +2,7 @@
 #define REPOSITORY_H
 
 struct config_set;
+struct index_state;
 
 struct repository {
 	/* Environment */
@@ -31,6 +32,9 @@ struct repository {
 	 */
 	struct config_set *config;
 
+	/* Repository's in-memory index */
+	struct index_state *index;
+
 	/* Configurations */
 	/*
 	 * Bit used during initialization to indicate if repository state (like
@@ -53,4 +57,6 @@ extern void repo_set_worktree(struct repository *repo, const char *path);
 extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
 extern void repo_clear(struct repository *repo);
 
+extern int repo_read_index(struct repository *repo);
+
 #endif /* REPOSITORY_H */
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 16/20] submodule-config: store the_submodule_cache in the_repository
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (14 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 15/20] repository: add index_state to struct repo Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 17/20] submodule: add repo_read_gitmodules Brandon Williams
                       ` (5 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Refactor how 'the_submodule_cache' is handled so that it can be stored
inside of a repository object.  Also migrate 'the_submodule_cache' to be
stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c       |  6 +++++
 repository.h       |  4 ++++
 submodule-config.c | 70 ++++++++++++++++++++++++++++++++++++++++--------------
 submodule-config.h | 10 ++++++++
 4 files changed, 72 insertions(+), 18 deletions(-)

diff --git a/repository.c b/repository.c
index 883e6e9e9..317041a4a 100644
--- a/repository.c
+++ b/repository.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "submodule-config.h"
 
 /* The main repository */
 static struct repository the_repo;
@@ -181,6 +182,11 @@ void repo_clear(struct repository *repo)
 		repo->index = NULL;
 	}
 
+	if (repo->submodule_cache) {
+		submodule_cache_free(repo->submodule_cache);
+		repo->submodule_cache = NULL;
+	}
+
 	memset(repo, 0, sizeof(*repo));
 }
 
diff --git a/repository.h b/repository.h
index 1fa65c42f..4bc70ebc5 100644
--- a/repository.h
+++ b/repository.h
@@ -3,6 +3,7 @@
 
 struct config_set;
 struct index_state;
+struct submodule_cache;
 
 struct repository {
 	/* Environment */
@@ -32,6 +33,9 @@ struct repository {
 	 */
 	struct config_set *config;
 
+	/* Repository's submodule config as defined by '.gitmodules' */
+	struct submodule_cache *submodule_cache;
+
 	/* Repository's in-memory index */
 	struct index_state *index;
 
diff --git a/submodule-config.c b/submodule-config.c
index d8f8d5ea3..37cfcceb9 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -15,6 +16,7 @@
 struct submodule_cache {
 	struct hashmap for_path;
 	struct hashmap for_name;
+	unsigned initialized:1;
 };
 
 /*
@@ -31,9 +33,6 @@ enum lookup_type {
 	lookup_path
 };
 
-static struct submodule_cache the_submodule_cache;
-static int is_cache_init;
-
 static int config_path_cmp(const struct submodule_entry *a,
 			   const struct submodule_entry *b,
 			   const void *unused)
@@ -50,10 +49,16 @@ static int config_name_cmp(const struct submodule_entry *a,
 	       hashcmp(a->config->gitmodules_sha1, b->config->gitmodules_sha1);
 }
 
-static void cache_init(struct submodule_cache *cache)
+static struct submodule_cache *submodule_cache_alloc(void)
+{
+	return xcalloc(1, sizeof(struct submodule_cache));
+}
+
+static void submodule_cache_init(struct submodule_cache *cache)
 {
 	hashmap_init(&cache->for_path, (hashmap_cmp_fn) config_path_cmp, 0);
 	hashmap_init(&cache->for_name, (hashmap_cmp_fn) config_name_cmp, 0);
+	cache->initialized = 1;
 }
 
 static void free_one_config(struct submodule_entry *entry)
@@ -65,11 +70,14 @@ static void free_one_config(struct submodule_entry *entry)
 	free(entry->config);
 }
 
-static void cache_free(struct submodule_cache *cache)
+static void submodule_cache_clear(struct submodule_cache *cache)
 {
 	struct hashmap_iter iter;
 	struct submodule_entry *entry;
 
+	if (!cache->initialized)
+		return;
+
 	/*
 	 * We iterate over the name hash here to be symmetric with the
 	 * allocation of struct submodule entries. Each is allocated by
@@ -81,6 +89,13 @@ static void cache_free(struct submodule_cache *cache)
 
 	hashmap_free(&cache->for_path, 1);
 	hashmap_free(&cache->for_name, 1);
+	cache->initialized = 0;
+}
+
+void submodule_cache_free(struct submodule_cache *cache)
+{
+	submodule_cache_clear(cache);
+	free(cache);
 }
 
 static unsigned int hash_sha1_string(const unsigned char *sha1,
@@ -494,43 +509,62 @@ static const struct submodule *config_from(struct submodule_cache *cache,
 	return submodule;
 }
 
-static void ensure_cache_init(void)
+static void submodule_cache_check_init(struct repository *repo)
 {
-	if (is_cache_init)
+	if (repo->submodule_cache && repo->submodule_cache->initialized)
 		return;
 
-	cache_init(&the_submodule_cache);
-	is_cache_init = 1;
+	if (!repo->submodule_cache)
+		repo->submodule_cache = submodule_cache_alloc();
+
+	submodule_cache_init(repo->submodule_cache);
 }
 
-int parse_submodule_config_option(const char *var, const char *value)
+int submodule_config_option(struct repository *repo,
+			    const char *var, const char *value)
 {
 	struct parse_config_parameter parameter;
-	parameter.cache = &the_submodule_cache;
+
+	submodule_cache_check_init(repo);
+
+	parameter.cache = repo->submodule_cache;
 	parameter.treeish_name = NULL;
 	parameter.gitmodules_sha1 = null_sha1;
 	parameter.overwrite = 1;
 
-	ensure_cache_init();
 	return parse_config(var, value, &parameter);
 }
 
+int parse_submodule_config_option(const char *var, const char *value)
+{
+	return submodule_config_option(the_repository, var, value);
+}
+
 const struct submodule *submodule_from_name(const unsigned char *treeish_name,
 		const char *name)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, name, lookup_name);
+	submodule_cache_check_init(the_repository);
+	return config_from(the_repository->submodule_cache, treeish_name, name, lookup_name);
 }
 
 const struct submodule *submodule_from_path(const unsigned char *treeish_name,
 		const char *path)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, path, lookup_path);
+	submodule_cache_check_init(the_repository);
+	return config_from(the_repository->submodule_cache, treeish_name, path, lookup_path);
+}
+
+const struct submodule *submodule_from_cache(struct repository *repo,
+					     const unsigned char *treeish_name,
+					     const char *key)
+{
+	submodule_cache_check_init(repo);
+	return config_from(repo->submodule_cache, treeish_name,
+			   key, lookup_path);
 }
 
 void submodule_free(void)
 {
-	cache_free(&the_submodule_cache);
-	is_cache_init = 0;
+	if (the_repository->submodule_cache)
+		submodule_cache_clear(the_repository->submodule_cache);
 }
diff --git a/submodule-config.h b/submodule-config.h
index d434ecdb4..bc45a25e8 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -22,14 +22,24 @@ struct submodule {
 	int recommend_shallow;
 };
 
+struct submodule_cache;
+struct repository;
+
+extern void submodule_cache_free(struct submodule_cache *cache);
+
 extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_submodule_config_option(const char *var, const char *value);
+extern int submodule_config_option(struct repository *repo,
+				   const char *var, const char *value);
 extern const struct submodule *submodule_from_name(
 		const unsigned char *commit_or_tree, const char *name);
 extern const struct submodule *submodule_from_path(
 		const unsigned char *commit_or_tree, const char *path);
+extern const struct submodule *submodule_from_cache(struct repository *repo,
+						    const unsigned char *treeish_name,
+						    const char *key);
 extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,
 				      unsigned char *gitmodules_sha1,
 				      struct strbuf *rev);
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 17/20] submodule: add repo_read_gitmodules
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (15 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 16/20] submodule-config: store the_submodule_cache in the_repository Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 18/20] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
                       ` (4 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Teach the repo object to be able to populate the submodule_cache by
reading the repository's gitmodules file.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 submodule.c | 15 +++++++++++++++
 submodule.h |  2 ++
 2 files changed, 17 insertions(+)

diff --git a/submodule.c b/submodule.c
index da0b80549..d0b894772 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -255,6 +256,20 @@ void gitmodules_config(void)
 	}
 }
 
+static int gitmodules_cb(const char *var, const char *value, void *data)
+{
+	struct repository *repo = data;
+	return submodule_config_option(repo, var, value);
+}
+
+void repo_read_gitmodules(struct repository *repo)
+{
+	char *gitmodules_path = repo_worktree_path(repo, ".gitmodules");
+
+	git_config_from_file(gitmodules_cb, gitmodules_path, repo);
+	free(gitmodules_path);
+}
+
 void gitmodules_config_sha1(const unsigned char *commit_sha1)
 {
 	struct strbuf rev = STRBUF_INIT;
diff --git a/submodule.h b/submodule.h
index cbe5c1726..8a3771ec6 100644
--- a/submodule.h
+++ b/submodule.h
@@ -1,6 +1,7 @@
 #ifndef SUBMODULE_H
 #define SUBMODULE_H
 
+struct repository;
 struct diff_options;
 struct argv_array;
 struct oid_array;
@@ -46,6 +47,7 @@ int option_parse_recurse_submodules_worktree_updater(const struct option *opt,
 						     const char *arg, int unset);
 void load_submodule_cache(void);
 extern void gitmodules_config(void);
+extern void repo_read_gitmodules(struct repository *repo);
 extern void gitmodules_config_sha1(const unsigned char *commit_sha1);
 extern int is_submodule_initialized(const char *path);
 /*
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 18/20] submodule: convert is_submodule_initialized to work on a repository
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (16 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 17/20] submodule: add repo_read_gitmodules Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-20 19:19     ` [PATCH v3 19/20] repository: enable initialization of submodules Brandon Williams
                       ` (3 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert 'is_submodule_initialized()' to take a repository object and
while we're at it, lets rename the function to 'is_submodule_active()'
and remove the NEEDSWORK comment.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/grep.c              |  3 ++-
 builtin/submodule--helper.c |  9 +++++----
 submodule.c                 | 20 ++++++++------------
 submodule.h                 |  2 +-
 4 files changed, 16 insertions(+), 18 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index f61a9d938..e3ba1d98e 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "blob.h"
 #include "tree.h"
@@ -643,7 +644,7 @@ static int grep_submodule_launch(struct grep_opt *opt,
 static int grep_submodule(struct grep_opt *opt, const struct object_id *oid,
 			  const char *filename, const char *path)
 {
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(the_repository, path))
 		return 0;
 	if (!is_submodule_populated_gently(path, NULL)) {
 		/*
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 8517032b3..e1b06c41d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "repository.h"
 #include "cache.h"
 #include "config.h"
 #include "parse-options.h"
@@ -280,7 +281,7 @@ static void module_list_active(struct module_list *list)
 	for (i = 0; i < list->nr; i++) {
 		const struct cache_entry *ce = list->entries[i];
 
-		if (!is_submodule_initialized(ce->name))
+		if (!is_submodule_active(the_repository, ce->name))
 			continue;
 
 		ALLOC_GROW(active_modules.entries,
@@ -362,7 +363,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_initialized(path)) {
+	if (!is_submodule_active(the_repository, path)) {
 		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
@@ -817,7 +818,7 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 	}
 
 	/* Check if the submodule has been initialized. */
-	if (!is_submodule_initialized(ce->name)) {
+	if (!is_submodule_active(the_repository, ce->name)) {
 		next_submodule_warn_missing(suc, out, displaypath);
 		goto cleanup;
 	}
@@ -1193,7 +1194,7 @@ static int is_active(int argc, const char **argv, const char *prefix)
 
 	gitmodules_config();
 
-	return !is_submodule_initialized(argv[1]);
+	return !is_submodule_active(the_repository, argv[1]);
 }
 
 #define SUPPORT_SUPER_PREFIX (1<<0)
diff --git a/submodule.c b/submodule.c
index d0b894772..b23c25311 100644
--- a/submodule.c
+++ b/submodule.c
@@ -283,21 +283,17 @@ void gitmodules_config_sha1(const unsigned char *commit_sha1)
 }
 
 /*
- * NEEDSWORK: With the addition of different configuration options to determine
- * if a submodule is of interests, the validity of this function's name comes
- * into question.  Once the dust has settled and more concrete terminology is
- * decided upon, come up with a more proper name for this function.  One
- * potential candidate could be 'is_submodule_active()'.
- *
  * Determine if a submodule has been initialized at a given 'path'
  */
-int is_submodule_initialized(const char *path)
+int is_submodule_active(struct repository *repo, const char *path)
 {
 	int ret = 0;
 	char *key = NULL;
 	char *value = NULL;
 	const struct string_list *sl;
-	const struct submodule *module = submodule_from_path(null_sha1, path);
+	const struct submodule *module;
+
+	module = submodule_from_cache(repo, null_sha1, path);
 
 	/* early return if there isn't a path->module mapping */
 	if (!module)
@@ -305,14 +301,14 @@ int is_submodule_initialized(const char *path)
 
 	/* submodule.<name>.active is set */
 	key = xstrfmt("submodule.%s.active", module->name);
-	if (!git_config_get_bool(key, &ret)) {
+	if (!repo_config_get_bool(repo, key, &ret)) {
 		free(key);
 		return ret;
 	}
 	free(key);
 
 	/* submodule.active is set */
-	sl = git_config_get_value_multi("submodule.active");
+	sl = repo_config_get_value_multi(repo, "submodule.active");
 	if (sl) {
 		struct pathspec ps;
 		struct argv_array args = ARGV_ARRAY_INIT;
@@ -332,7 +328,7 @@ int is_submodule_initialized(const char *path)
 
 	/* fallback to checking if the URL is set */
 	key = xstrfmt("submodule.%s.url", module->name);
-	ret = !git_config_get_string(key, &value);
+	ret = !repo_config_get_string(repo, key, &value);
 
 	free(value);
 	free(key);
@@ -1532,7 +1528,7 @@ int submodule_move_head(const char *path,
 	const struct submodule *sub;
 	int *error_code_ptr, error_code;
 
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(the_repository, path))
 		return 0;
 
 	if (flags & SUBMODULE_MOVE_HEAD_FORCE)
diff --git a/submodule.h b/submodule.h
index 8a3771ec6..623ce6ad7 100644
--- a/submodule.h
+++ b/submodule.h
@@ -49,7 +49,7 @@ void load_submodule_cache(void);
 extern void gitmodules_config(void);
 extern void repo_read_gitmodules(struct repository *repo);
 extern void gitmodules_config_sha1(const unsigned char *commit_sha1);
-extern int is_submodule_initialized(const char *path);
+extern int is_submodule_active(struct repository *repo, const char *path);
 /*
  * Determine if a submodule has been populated at a given 'path' by checking if
  * the <path>/.git resolves to a valid git repository.
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 19/20] repository: enable initialization of submodules
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (17 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 18/20] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-21 23:00       ` Jonathan Tan
  2017-06-20 19:19     ` [PATCH v3 20/20] ls-files: use repository object Brandon Williams
                       ` (2 subsequent siblings)
  21 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce 'repo_submodule_init()' which performs initialization of a
'struct repository' as a submodule of another 'struct repository'.

The resulting submodule can be in one of three states:

  1. The submodule is initialized and has a worktree.

  2. The submodule is initialized but does not have a worktree.  This
     would occur when the submodule's gitdir is present in the
     superproject's 'gitdir/modules/' directory yet the submodule has not
     been checked out in superproject's worktree.

  3. The submodule remains uninitialized due to an error in the
     initialization process or there is no matching submodule at the
     provided path in the superproject.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 repository.h | 10 ++++++++++
 2 files changed, 54 insertions(+)

diff --git a/repository.c b/repository.c
index 317041a4a..673b35ca5 100644
--- a/repository.c
+++ b/repository.c
@@ -155,6 +155,48 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 	return -1;
 }
 
+/*
+ * Initialize 'submodule' as the submodule given by 'path' in parent repository
+ * 'superproject'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+int repo_submodule_init(struct repository *submodule,
+			struct repository *superproject,
+			const char *path)
+{
+	const struct submodule *sub;
+	struct strbuf submodule_path = STRBUF_INIT;
+	int ret = 0;
+
+	sub = submodule_from_cache(superproject, null_sha1, path);
+	if (!sub) {
+		ret = -1;
+		goto out;
+	}
+
+	strbuf_repo_worktree_path(&submodule_path, superproject, "%s", path);
+
+	if (repo_init(submodule, submodule_path.buf, submodule_path.buf)) {
+		strbuf_reset(&submodule_path);
+		strbuf_repo_git_path(&submodule_path, superproject,
+				     "modules/%s", sub->name);
+
+		if (repo_init(submodule, submodule_path.buf, NULL)) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+	submodule->submodule_prefix = xstrfmt("%s%s/",
+					      superproject->submodule_prefix ?
+					      superproject->submodule_prefix :
+					      "", path);
+
+out:
+	strbuf_release(&submodule_path);
+	return ret;
+}
+
 void repo_clear(struct repository *repo)
 {
 	free(repo->gitdir);
@@ -169,6 +211,8 @@ void repo_clear(struct repository *repo)
 	repo->index_file = NULL;
 	free(repo->worktree);
 	repo->worktree = NULL;
+	free(repo->submodule_prefix);
+	repo->submodule_prefix = NULL;
 
 	if (repo->config) {
 		git_configset_clear(repo->config);
diff --git a/repository.h b/repository.h
index 4bc70ebc5..7b352accb 100644
--- a/repository.h
+++ b/repository.h
@@ -25,6 +25,13 @@ struct repository {
 	/* Path to the working directory */
 	char *worktree;
 
+	/*
+	 * Path from the root of the top-level superproject down to this
+	 * repository.  This is only non-NULL if the repository is initialized
+	 * as a submodule of another repository.
+	 */
+	char *submodule_prefix;
+
 	/* Subsystems */
 	/*
 	 * Repository's config which contains key-value pairs from the usual
@@ -59,6 +66,9 @@ extern struct repository *the_repository;
 extern void repo_set_gitdir(struct repository *repo, const char *path);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
+extern int repo_submodule_init(struct repository *submodule,
+			       struct repository *superproject,
+			       const char *path);
 extern void repo_clear(struct repository *repo);
 
 extern int repo_read_index(struct repository *repo);
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v3 20/20] ls-files: use repository object
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (18 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 19/20] repository: enable initialization of submodules Brandon Williams
@ 2017-06-20 19:19     ` Brandon Williams
  2017-06-21 22:48       ` Jonathan Tan
  2017-06-20 19:23     ` [PATCH v3 00/20] " Stefan Beller
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
  21 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 19:19 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert ls-files to use a repository struct and recurse submodules
inprocess.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c                     | 195 ++++++++++++++-------------------
 git.c                                  |   2 +-
 t/t3007-ls-files-recurse-submodules.sh |  39 +++++++
 3 files changed, 120 insertions(+), 116 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index b12d0bb61..f5f36ddb4 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -5,7 +5,9 @@
  *
  * Copyright (C) Linus Torvalds, 2005
  */
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "quote.h"
 #include "dir.h"
@@ -32,10 +34,8 @@ static int line_terminator = '\n';
 static int debug_mode;
 static int show_eol;
 static int recurse_submodules;
-static struct argv_array submodule_options = ARGV_ARRAY_INIT;
 
 static const char *prefix;
-static const char *super_prefix;
 static int max_prefix_len;
 static int prefix_len;
 static struct pathspec pathspec;
@@ -73,25 +73,12 @@ static void write_eolinfo(const struct index_state *istate,
 
 static void write_name(const char *name)
 {
-	/*
-	 * Prepend the super_prefix to name to construct the full_name to be
-	 * written.
-	 */
-	struct strbuf full_name = STRBUF_INIT;
-	if (super_prefix) {
-		strbuf_addstr(&full_name, super_prefix);
-		strbuf_addstr(&full_name, name);
-		name = full_name.buf;
-	}
-
 	/*
 	 * With "--full-name", prefix_len=0; this caller needs to pass
 	 * an empty string in that case (a NULL is good for "").
 	 */
 	write_name_quoted_relative(name, prefix_len ? prefix : NULL,
 				   stdout, line_terminator);
-
-	strbuf_release(&full_name);
 }
 
 static const char *get_tag(const struct cache_entry *ce, const char *tag)
@@ -210,83 +197,38 @@ static void show_killed_files(const struct index_state *istate,
 	}
 }
 
-/*
- * Compile an argv_array with all of the options supported by --recurse_submodules
- */
-static void compile_submodule_options(const char **argv,
-				      const struct dir_struct *dir,
-				      int show_tag)
-{
-	if (line_terminator == '\0')
-		argv_array_push(&submodule_options, "-z");
-	if (show_tag)
-		argv_array_push(&submodule_options, "-t");
-	if (show_valid_bit)
-		argv_array_push(&submodule_options, "-v");
-	if (show_cached)
-		argv_array_push(&submodule_options, "--cached");
-	if (show_eol)
-		argv_array_push(&submodule_options, "--eol");
-	if (debug_mode)
-		argv_array_push(&submodule_options, "--debug");
-
-	/* Add Pathspecs */
-	argv_array_push(&submodule_options, "--");
-	for (; *argv; argv++)
-		argv_array_push(&submodule_options, *argv);
-}
+static void show_files(struct repository *repo, struct dir_struct *dir);
 
-/**
- * Recursively call ls-files on a submodule
- */
-static void show_gitlink(const struct cache_entry *ce)
+static void show_submodule(struct repository *superproject,
+			   struct dir_struct *dir, const char *path)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-	char *dir;
-
-	prepare_submodule_repo_env(&cp.env_array);
-	argv_array_push(&cp.env_array, GIT_DIR_ENVIRONMENT);
-
-	if (prefix_len)
-		argv_array_pushf(&cp.env_array, "%s=%s",
-				 GIT_TOPLEVEL_PREFIX_ENVIRONMENT,
-				 prefix);
-	argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
-			 super_prefix ? super_prefix : "",
-			 ce->name);
-	argv_array_push(&cp.args, "ls-files");
-	argv_array_push(&cp.args, "--recurse-submodules");
-
-	/* add supported options */
-	argv_array_pushv(&cp.args, submodule_options.argv);
-
-	cp.git_cmd = 1;
-	dir = mkpathdup("%s/%s", get_git_work_tree(), ce->name);
-	cp.dir = dir;
-	status = run_command(&cp);
-	free(dir);
-	if (status)
-		exit(status);
+	struct repository submodule;
+
+	if (repo_submodule_init(&submodule, superproject, path))
+		return;
+
+	if (repo_read_index(&submodule) < 0)
+		die("index file corrupt");
+
+	repo_read_gitmodules(&submodule);
+
+	show_files(&submodule, dir);
+
+	repo_clear(&submodule);
 }
 
-static void show_ce_entry(const struct index_state *istate,
-			  const char *tag, const struct cache_entry *ce)
+static void show_ce(struct repository *repo, struct dir_struct *dir,
+		    const struct cache_entry *ce, const char *fullname,
+		    const char *tag)
 {
-	struct strbuf name = STRBUF_INIT;
-	int len = max_prefix_len;
-	if (super_prefix)
-		strbuf_addstr(&name, super_prefix);
-	strbuf_addstr(&name, ce->name);
-
-	if (len > ce_namelen(ce))
+	if (max_prefix_len > strlen(fullname))
 		die("git ls-files: internal error - cache entry not superset of prefix");
 
 	if (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&
-	    submodule_path_match(&pathspec, name.buf, ps_matched)) {
-		show_gitlink(ce);
-	} else if (match_pathspec(&pathspec, name.buf, name.len,
-				  len, ps_matched,
+	    is_submodule_active(repo, ce->name)) {
+		show_submodule(repo, dir, ce->name);
+	} else if (match_pathspec(&pathspec, fullname, strlen(fullname),
+				  max_prefix_len, ps_matched,
 				  S_ISDIR(ce->ce_mode) ||
 				  S_ISGITLINK(ce->ce_mode))) {
 		tag = get_tag(ce, tag);
@@ -300,12 +242,10 @@ static void show_ce_entry(const struct index_state *istate,
 			       find_unique_abbrev(ce->oid.hash, abbrev),
 			       ce_stage(ce));
 		}
-		write_eolinfo(istate, ce, ce->name);
-		write_name(ce->name);
+		write_eolinfo(repo->index, ce, fullname);
+		write_name(fullname);
 		print_debug(ce);
 	}
-
-	strbuf_release(&name);
 }
 
 static void show_ru_info(const struct index_state *istate)
@@ -338,59 +278,79 @@ static void show_ru_info(const struct index_state *istate)
 }
 
 static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
-		       const struct cache_entry *ce)
+		       const char *fullname, const struct cache_entry *ce)
 {
 	int dtype = ce_to_dtype(ce);
-	return is_excluded(dir, istate, ce->name, &dtype);
+	return is_excluded(dir, istate, fullname, &dtype);
 }
 
-static void show_files(struct index_state *istate, struct dir_struct *dir)
+static void construct_fullname(struct strbuf *out, const struct repository *repo,
+			       const struct cache_entry *ce)
+{
+	strbuf_reset(out);
+	if (repo->submodule_prefix)
+		strbuf_addstr(out, repo->submodule_prefix);
+	strbuf_addstr(out, ce->name);
+}
+
+static void show_files(struct repository *repo, struct dir_struct *dir)
 {
 	int i;
+	struct strbuf fullname = STRBUF_INIT;
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
 		if (!show_others)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
-		fill_directory(dir, istate, &pathspec);
+		fill_directory(dir, repo->index, &pathspec);
 		if (show_others)
-			show_other_files(istate, dir);
+			show_other_files(repo->index, dir);
 		if (show_killed)
-			show_killed_files(istate, dir);
+			show_killed_files(repo->index, dir);
 	}
 	if (show_cached || show_stage) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
-			show_ce_entry(istate, ce_stage(ce) ? tag_unmerged :
-				(ce_skip_worktree(ce) ? tag_skip_worktree : tag_cached), ce);
+			show_ce(repo, dir, ce, fullname.buf,
+				ce_stage(ce) ? tag_unmerged :
+				(ce_skip_worktree(ce) ? tag_skip_worktree :
+				 tag_cached));
 		}
 	}
 	if (show_deleted || show_modified) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
 			struct stat st;
 			int err;
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
 			if (ce_skip_worktree(ce))
 				continue;
-			err = lstat(ce->name, &st);
+			err = lstat(fullname.buf, &st);
 			if (show_deleted && err)
-				show_ce_entry(istate, tag_removed, ce);
-			if (show_modified && ie_modified(istate, ce, &st, 0))
-				show_ce_entry(istate, tag_modified, ce);
+				show_ce(repo, dir, ce, fullname.buf, tag_removed);
+			if (show_modified && ie_modified(repo->index, ce, &st, 0))
+				show_ce(repo, dir, ce, fullname.buf, tag_modified);
 		}
 	}
+
+	strbuf_release(&fullname);
 }
 
 /*
@@ -615,10 +575,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	prefix = cmd_prefix;
 	if (prefix)
 		prefix_len = strlen(prefix);
-	super_prefix = get_super_prefix();
 	git_config(git_default_config, NULL);
 
-	if (read_cache() < 0)
+	if (repo_read_index(the_repository) < 0)
 		die("index file corrupt");
 
 	argc = parse_options(argc, argv, prefix, builtin_ls_files_options,
@@ -651,8 +610,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	if (require_work_tree && !is_inside_work_tree())
 		setup_work_tree();
 
-	if (recurse_submodules)
-		compile_submodule_options(argv, &dir, show_tag);
+	if (recurse_submodules) {
+		repo_read_gitmodules(the_repository);
+	}
 
 	if (recurse_submodules &&
 	    (show_stage || show_deleted || show_others || show_unmerged ||
@@ -670,7 +630,10 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	/*
 	 * Find common prefix for all pathspec's
 	 * This is used as a performance optimization which unfortunately cannot
-	 * be done when recursing into submodules
+	 * be done when recursing into submodules because when a pathspec is
+	 * given which spans repository boundaries you can't simply remove the
+	 * submodule entry because the pathspec may match something inside the
+	 * submodule.
 	 */
 	if (recurse_submodules)
 		max_prefix = NULL;
@@ -678,7 +641,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		max_prefix = common_prefix(&pathspec);
 	max_prefix_len = get_common_prefix_len(max_prefix);
 
-	prune_index(&the_index, max_prefix, max_prefix_len);
+	prune_index(the_repository->index, max_prefix, max_prefix_len);
 
 	/* Treat unmatching pathspec elements as errors */
 	if (pathspec.nr && error_unmatch)
@@ -699,11 +662,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		 */
 		if (show_stage || show_unmerged)
 			die("ls-files --with-tree is incompatible with -s or -u");
-		overlay_tree_on_index(&the_index, with_tree, max_prefix);
+		overlay_tree_on_index(the_repository->index, with_tree, max_prefix);
 	}
-	show_files(&the_index, &dir);
+
+	show_files(the_repository, &dir);
+
 	if (show_resolve_undo)
-		show_ru_info(&the_index);
+		show_ru_info(the_repository->index);
 
 	if (ps_matched) {
 		int bad;
diff --git a/git.c b/git.c
index 5be27b07e..489aab4d8 100644
--- a/git.c
+++ b/git.c
@@ -400,7 +400,7 @@ static struct cmd_struct commands[] = {
 	{ "init-db", cmd_init_db },
 	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY },
 	{ "log", cmd_log, RUN_SETUP },
-	{ "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },
+	{ "ls-files", cmd_ls_files, RUN_SETUP },
 	{ "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY },
 	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
 	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY },
diff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh
index ebb956fd1..318b5bce7 100755
--- a/t/t3007-ls-files-recurse-submodules.sh
+++ b/t/t3007-ls-files-recurse-submodules.sh
@@ -135,6 +135,45 @@ test_expect_success '--recurse-submodules and pathspecs setup' '
 	test_cmp expect actual
 '
 
+test_expect_success 'inactive submodule' '
+	test_when_finished "git config --bool submodule.submodule.active true" &&
+	test_when_finished "git -C submodule config --bool submodule.subsub.active true" &&
+	git config --bool submodule.submodule.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual &&
+
+	git config --bool submodule.submodule.active "true" &&
+	git -C submodule config --bool submodule.subsub.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule/.gitmodules
+	submodule/c
+	submodule/f.TXT
+	submodule/g.txt
+	submodule/subsub
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success '--recurse-submodules and pathspecs' '
 	cat >expect <<-\EOF &&
 	h.txt
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 00/20] repository object
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (19 preceding siblings ...)
  2017-06-20 19:19     ` [PATCH v3 20/20] ls-files: use repository object Brandon Williams
@ 2017-06-20 19:23     ` Stefan Beller
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
  21 siblings, 0 replies; 214+ messages in thread
From: Stefan Beller @ 2017-06-20 19:23 UTC (permalink / raw)
  To: Brandon Williams, Prathamesh Chavan; +Cc: git@vger.kernel.org

On Tue, Jun 20, 2017 at 12:19 PM, Brandon Williams <bmwill@google.com> wrote:
> I decided to split up the original series into three parts in order to make
> review a little bit easier.  As such this series is dependent on on
> 'bw/config-h' and 'bw/ls-files-sans-the-index' which should be moving into
> 'next' soon.  Due to this I was able to greatly shrink this series in terms of
> number of patches so hopefully it is a little easier to review.
>
> As before you can find this series at:
> https://github.com/bmwill/git/tree/repository-object

If you plan on building 'foreach' on top of Brandons series,
maybe start off the latest version here.

>
> Chagnes in v3:
...
>
>  * Added an additional initialization function to allow initializing a 'struct
>    repository' as a submodule of another 'struct repository'.

This one seems relevant for 'foreach'

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 04/20] repository: introduce the repository object
  2017-06-20 19:19     ` [PATCH v3 04/20] repository: introduce the repository object Brandon Williams
@ 2017-06-20 19:57       ` Stefan Beller
  2017-06-21 21:27         ` Brandon Williams
  2017-06-20 21:31       ` Jonathan Tan
  1 sibling, 1 reply; 214+ messages in thread
From: Stefan Beller @ 2017-06-20 19:57 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Tue, Jun 20, 2017 at 12:19 PM, Brandon Williams <bmwill@google.com> wrote:
> Introduce the repository object 'struct repository' which can be used to
> hold all state pertaining to a git repository.
>
> Some of the benefits of object-ifying a repository are:
>
>   1. Make the code base more readable and easier to reason about.
>
>   2. Allow for working on multiple repositories, specifically
>      submodules, within the same process.  Currently the process for
>      working on a submodule involves setting up an argv_array of options
>      for a particular command and then launching a child process to
>      execute the command in the context of the submodule.  This is
>      clunky and can require lots of little hacks in order to ensure
>      correctness.  Ideally it would be nice to simply pass a repository
>      and an options struct to a command.
>
>   3. Eliminating reliance on global state will make it easier to
>      enable the use of threading to improve performance.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>

> +/*
> + * Initialize 'repo' based on the provided 'gitdir'.
> + * Return 0 upon success and a non-zero value upon failure.

Non zero or negative? The point of the question is if we want to
ask users of this function to be cautious early on. So in the future,
do we want to rather see

    if (repo_init(...))
        die("you're doomed");

or rather

    int x = repo_init(...);
    if (x < 0)
        die("you're doomed");
    else if (x == 1)
        warning("you're not doomed, but close."\
             "Not distimming the gostaks.")
    else
        ; /* we're fine, carry on with life */

I guess we can still refactor later, it's just one
thing to thing about when introducing an API
that will likely be used a lot down the road.

> +struct repository {
> +       /* Environment */
> +       /* Path to the git directory */
> +       char *gitdir;
> +
> +       /* Path to the common git directory */
> +       char *commondir;
> +
> +       /* Path to the repository's object store */
> +       char *objectdir;
> +
> +       /* Path to the repository's graft file */
> +       char *graft_file;
> +
> +       /* Path to the current worktree's index file */
> +       char *index_file;
> +
> +       /* Path to the working directory */
> +       char *worktree;
> +
> +       /* Configurations */
> +       /*
> +        * Bit used during initialization to indicate if repository state (like
> +        * the location of the 'objectdir') should be read from the
> +        * environment.  By default this bit will be set at the begining of
> +        * 'repo_init()' so that all repositories will ignore the environment.
> +        * The exception to this is 'the_repository', which doesn't go through
> +        * the normal 'repo_init()' process.
> +        */
> +       unsigned ignore_env:1;
> +
> +       /* Indicate if a repository has a different 'commondir' from 'gitdir' */
> +       unsigned different_commondir:1;
> +};

I applaud the effort towards documenting what each variable is
supposed to contain. But some of them read like

    /* increments i by one */
    i++;

which is considered bad comment style (it doesn't add
more information, it just wastes a line), so specifically for
all the "Path to X" comments:
* Are they absolute path, or relative path?
  If relative, then relative to what?
* Can they be NULL? When?

(* Why do we need so many path?
    Could one of them be constructed using
    another and then hardcoding a string relative to it?
    This question may rather be answered in the commit
    message)

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-20 19:19     ` [PATCH v3 11/20] path: construct correct path to a worktree's index Brandon Williams
@ 2017-06-20 21:10       ` Stefan Beller
  2017-06-20 23:02       ` Jonathan Nieder
  2017-06-21  2:10       ` Jonathan Nieder
  2 siblings, 0 replies; 214+ messages in thread
From: Stefan Beller @ 2017-06-20 21:10 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Tue, Jun 20, 2017 at 12:19 PM, Brandon Williams <bmwill@google.com> wrote:
> When working with worktrees the git directory is split into two part,
> the per-worktree gitdir and a commondir which contains things which are
> shared among all worktrees (like the object store).  With this notion of
> having a split git directory, 557bd833b (git_path(): be aware of file
> relocation in $GIT_DIR) and c7b3a3d2f ($GIT_COMMON_DIR: a new
> environment variable) changed the way that 'git_path()' functioned so
> that paths would be adjusted if they referred to files or directories
> that are stored in the commondir or have been changed via an environment
> variable.
>
> One interesting problem with this is the index file as it is not shared
> across worktrees yet when asking for a path to a specific worktree's
> index it will be replaced with a path to the current worktree's index.
> In order to prevent this, teach 'adjuct_git_path' to replace the

adjust_git_path

> path to the index with the path recorded in a repository (which would be
> the current, active worktree) only when not asked to construct a path
> for a specific worktree.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  path.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/path.c b/path.c
> index 76a872297..c6832a30e 100644
> --- a/path.c
> +++ b/path.c
> @@ -372,13 +372,20 @@ void report_linked_checkout_garbage(void)
>  }
>
>  static void adjust_git_path(const struct repository *repo,
> +                           const struct worktree *wt,
>                             struct strbuf *buf, int git_dir_len)
>  {
>         const char *base = buf->buf + git_dir_len;
>         if (is_dir_file(base, "info", "grafts"))
>                 strbuf_splice(buf, 0, buf->len,
>                               repo->graft_file, strlen(repo->graft_file));
> -       else if (!strcmp(base, "index"))
> +       /*
> +        * Only try to replace the path '$gitdir/index' with the index file
> +        * recorded in the repository when not constructing a path for a
> +        * worktree.  This way we can retrieve the correct path to a particular
> +        * worktree's index file.
> +        */
> +       else if (!wt && !strcmp(base, "index"))
>                 strbuf_splice(buf, 0, buf->len,
>                               repo->index_file, strlen(repo->index_file));
>         else if (dir_prefix(base, "objects"))
> @@ -411,7 +418,7 @@ static void do_git_path(const struct repository *repo,
>                 strbuf_addch(buf, '/');
>         gitdir_len = buf->len;
>         strbuf_vaddf(buf, fmt, args);
> -       adjust_git_path(repo, buf, gitdir_len);
> +       adjust_git_path(repo, wt, buf, gitdir_len);
>         strbuf_cleanup_path(buf);
>  }
>
> --
> 2.13.1.611.g7e3b11ae1-goog
>

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 04/20] repository: introduce the repository object
  2017-06-20 19:19     ` [PATCH v3 04/20] repository: introduce the repository object Brandon Williams
  2017-06-20 19:57       ` Stefan Beller
@ 2017-06-20 21:31       ` Jonathan Tan
  2017-06-20 21:56         ` Brandon Williams
  1 sibling, 1 reply; 214+ messages in thread
From: Jonathan Tan @ 2017-06-20 21:31 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On Tue, 20 Jun 2017 12:19:35 -0700
Brandon Williams <bmwill@google.com> wrote:

> Introduce the repository object 'struct repository' which can be used to
> hold all state pertaining to a git repository.
> 
> Some of the benefits of object-ifying a repository are:
> 
>   1. Make the code base more readable and easier to reason about.
> 
>   2. Allow for working on multiple repositories, specifically
>      submodules, within the same process.  Currently the process for
>      working on a submodule involves setting up an argv_array of options
>      for a particular command and then launching a child process to
>      execute the command in the context of the submodule.  This is
>      clunky and can require lots of little hacks in order to ensure
>      correctness.  Ideally it would be nice to simply pass a repository
>      and an options struct to a command.
> 
>   3. Eliminating reliance on global state will make it easier to
>      enable the use of threading to improve performance.

These would indeed be nice to have.

> +/* called after setting gitdir */
> +static void repo_setup_env(struct repository *repo)
> +{
> +	struct strbuf sb = STRBUF_INIT;
> +
> +	if (!repo->gitdir)
> +		BUG("gitdir wasn't set before setting up the environment");
> +
> +	repo->different_commondir = find_common_dir(&sb, repo->gitdir,
> +						    !repo->ignore_env);
> +	repo->commondir = strbuf_detach(&sb, NULL);
> +	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
> +					    "objects", !repo->ignore_env);
> +	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
> +					     "info/grafts", !repo->ignore_env);
> +	repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
> +					     "index", !repo->ignore_env);
> +}

It seems that this function is only used once in repo_set_gitdir() -
could this be inlined there instead? Then you wouldn't need the comment
and the !repo->gitdir check.

> +static int verify_repo_format(struct repository_format *format,
> +			      const char *commondir)
> +{
> +	int ret = 0;
> +	struct strbuf sb = STRBUF_INIT;
> +
> +	strbuf_addf(&sb, "%s/config", commondir);
> +	read_repository_format(format, sb.buf);
> +	strbuf_reset(&sb);
> +
> +	if (verify_repository_format(format, &sb) < 0) {
> +		warning("%s", sb.buf);
> +		ret = -1;
> +	}
> +
> +	strbuf_release(&sb);
> +	return ret;
> +}

This function is confusingly named - firstly, there is already a
verify_repository_format(), and secondly, this function both reads and
verifies it.

> +void repo_clear(struct repository *repo)
> +{
> +	free(repo->gitdir);
> +	repo->gitdir = NULL;
> +	free(repo->commondir);
> +	repo->commondir = NULL;
> +	free(repo->objectdir);
> +	repo->objectdir = NULL;
> +	free(repo->graft_file);
> +	repo->graft_file = NULL;
> +	free(repo->index_file);
> +	repo->index_file = NULL;
> +	free(repo->worktree);
> +	repo->worktree = NULL;
> +
> +	memset(repo, 0, sizeof(*repo));
> +}

If you're going to memset, you probably don't need to set everything to
NULL.

> +	/* Configurations */
> +	/*
> +	 * Bit used during initialization to indicate if repository state (like
> +	 * the location of the 'objectdir') should be read from the
> +	 * environment.  By default this bit will be set at the begining of
> +	 * 'repo_init()' so that all repositories will ignore the environment.
> +	 * The exception to this is 'the_repository', which doesn't go through
> +	 * the normal 'repo_init()' process.
> +	 */
> +	unsigned ignore_env:1;

If this is only used during initialization, could this be passed as a
separate parameter internally instead of having it here?

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 04/20] repository: introduce the repository object
  2017-06-20 21:31       ` Jonathan Tan
@ 2017-06-20 21:56         ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-20 21:56 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On 06/20, Jonathan Tan wrote:
> On Tue, 20 Jun 2017 12:19:35 -0700
> Brandon Williams <bmwill@google.com> wrote:
> 
> > Introduce the repository object 'struct repository' which can be used to
> > hold all state pertaining to a git repository.
> > 
> > Some of the benefits of object-ifying a repository are:
> > 
> >   1. Make the code base more readable and easier to reason about.
> > 
> >   2. Allow for working on multiple repositories, specifically
> >      submodules, within the same process.  Currently the process for
> >      working on a submodule involves setting up an argv_array of options
> >      for a particular command and then launching a child process to
> >      execute the command in the context of the submodule.  This is
> >      clunky and can require lots of little hacks in order to ensure
> >      correctness.  Ideally it would be nice to simply pass a repository
> >      and an options struct to a command.
> > 
> >   3. Eliminating reliance on global state will make it easier to
> >      enable the use of threading to improve performance.
> 
> These would indeed be nice to have.
> 
> > +/* called after setting gitdir */
> > +static void repo_setup_env(struct repository *repo)
> > +{
> > +	struct strbuf sb = STRBUF_INIT;
> > +
> > +	if (!repo->gitdir)
> > +		BUG("gitdir wasn't set before setting up the environment");
> > +
> > +	repo->different_commondir = find_common_dir(&sb, repo->gitdir,
> > +						    !repo->ignore_env);
> > +	repo->commondir = strbuf_detach(&sb, NULL);
> > +	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
> > +					    "objects", !repo->ignore_env);
> > +	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
> > +					     "info/grafts", !repo->ignore_env);
> > +	repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
> > +					     "index", !repo->ignore_env);
> > +}
> 
> It seems that this function is only used once in repo_set_gitdir() -
> could this be inlined there instead? Then you wouldn't need the comment
> and the !repo->gitdir check.

I'd like to keep them separate as it makes things a little clearer in my
opinion, smaller functions and the like.

> 
> > +static int verify_repo_format(struct repository_format *format,
> > +			      const char *commondir)
> > +{
> > +	int ret = 0;
> > +	struct strbuf sb = STRBUF_INIT;
> > +
> > +	strbuf_addf(&sb, "%s/config", commondir);
> > +	read_repository_format(format, sb.buf);
> > +	strbuf_reset(&sb);
> > +
> > +	if (verify_repository_format(format, &sb) < 0) {
> > +		warning("%s", sb.buf);
> > +		ret = -1;
> > +	}
> > +
> > +	strbuf_release(&sb);
> > +	return ret;
> > +}
> 
> This function is confusingly named - firstly, there is already a
> verify_repository_format(), and secondly, this function both reads and
> verifies it.

This one is static to the file, and i don't think its named confusingly
as it does just what it says it does. I'm trying to limit how much other
code I change so I had to make this function.

> 
> > +void repo_clear(struct repository *repo)
> > +{
> > +	free(repo->gitdir);
> > +	repo->gitdir = NULL;
> > +	free(repo->commondir);
> > +	repo->commondir = NULL;
> > +	free(repo->objectdir);
> > +	repo->objectdir = NULL;
> > +	free(repo->graft_file);
> > +	repo->graft_file = NULL;
> > +	free(repo->index_file);
> > +	repo->index_file = NULL;
> > +	free(repo->worktree);
> > +	repo->worktree = NULL;
> > +
> > +	memset(repo, 0, sizeof(*repo));
> > +}
> 
> If you're going to memset, you probably don't need to set everything to
> NULL.
> 
> > +	/* Configurations */
> > +	/*
> > +	 * Bit used during initialization to indicate if repository state (like
> > +	 * the location of the 'objectdir') should be read from the
> > +	 * environment.  By default this bit will be set at the begining of
> > +	 * 'repo_init()' so that all repositories will ignore the environment.
> > +	 * The exception to this is 'the_repository', which doesn't go through
> > +	 * the normal 'repo_init()' process.
> > +	 */
> > +	unsigned ignore_env:1;
> 
> If this is only used during initialization, could this be passed as a
> separate parameter internally instead of having it here?

It would feel a little wonky to be a parameter as 'the_repository' is
initialized differently than a repository would normally be.  Theres
been a lot of cleanup to the setup logic but it would need to be cleaned
up even more to have this be a param.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 05/20] environment: place key repository state in the_repository
  2017-06-20 19:19     ` [PATCH v3 05/20] environment: place key repository state in the_repository Brandon Williams
@ 2017-06-20 21:59       ` Jonathan Tan
  0 siblings, 0 replies; 214+ messages in thread
From: Jonathan Tan @ 2017-06-20 21:59 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On Tue, 20 Jun 2017 12:19:36 -0700
Brandon Williams <bmwill@google.com> wrote:

> Migrate 'git_dir', 'git_common_dir', 'git_object_dir', 'git_index_file',
> 'git_graft_file', and 'namespace' to be stored in 'the_repository'.
> 
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  cache.h       |  1 -
>  environment.c | 58 +++++++++++++---------------------------------------------
>  path.c        | 11 ++++++-----
>  setup.c       | 17 +++++++++++++++--
>  4 files changed, 34 insertions(+), 53 deletions(-)
> 
> diff --git a/cache.h b/cache.h
> index 7c81749a9..cd64cbc81 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -771,7 +771,6 @@ extern int core_apply_sparse_checkout;
>  extern int precomposed_unicode;
>  extern int protect_hfs;
>  extern int protect_ntfs;
> -extern int git_db_env, git_index_env, git_graft_env, git_common_dir_env;

In the commit message, it is probably worth mentioning commit 557bd83
which added these fields to attempt rewriting a path in do_git_path()
only if the appropriate _env flag is set, and that this patch removes
this optimization.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 10/20] path: convert do_git_path to take a 'struct repository'
  2017-06-20 19:19     ` [PATCH v3 10/20] path: convert do_git_path " Brandon Williams
@ 2017-06-20 22:23       ` Jonathan Tan
  2017-06-21 21:20         ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Jonathan Tan @ 2017-06-20 22:23 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On Tue, 20 Jun 2017 12:19:41 -0700
Brandon Williams <bmwill@google.com> wrote:

> +static void do_git_path(const struct repository *repo,
> +			const struct worktree *wt, struct strbuf *buf,
>  			const char *fmt, va_list args)
>  {
>  	int gitdir_len;
> -	strbuf_addstr(buf, get_worktree_git_dir(wt));

With this change, the get_worktree_git_dir() function no longer seems to
be used from outside - could it be marked static?

> +	strbuf_worktree_gitdir(buf, repo, wt);
>  	if (buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
>  		strbuf_addch(buf, '/');
>  	gitdir_len = buf->len;
>  	strbuf_vaddf(buf, fmt, args);
> -	adjust_git_path(buf, gitdir_len);
> +	adjust_git_path(repo, buf, gitdir_len);
>  	strbuf_cleanup_path(buf);
>  }

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-20 19:19     ` [PATCH v3 11/20] path: construct correct path to a worktree's index Brandon Williams
  2017-06-20 21:10       ` Stefan Beller
@ 2017-06-20 23:02       ` Jonathan Nieder
  2017-06-21  0:39         ` Brandon Williams
  2017-06-21  2:10       ` Jonathan Nieder
  2 siblings, 1 reply; 214+ messages in thread
From: Jonathan Nieder @ 2017-06-20 23:02 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

Hi,

Brandon Williams wrote:

> When working with worktrees the git directory is split into two part,
> the per-worktree gitdir and a commondir which contains things which are
> shared among all worktrees (like the object store).  With this notion of
> having a split git directory, 557bd833b (git_path(): be aware of file
> relocation in $GIT_DIR) and c7b3a3d2f ($GIT_COMMON_DIR: a new
> environment variable) changed the way that 'git_path()' functioned so
> that paths would be adjusted if they referred to files or directories
> that are stored in the commondir or have been changed via an environment
> variable.
>
> One interesting problem with this is the index file as it is not shared
> across worktrees yet when asking for a path to a specific worktree's
> index it will be replaced with a path to the current worktree's index.
> In order to prevent this, teach 'adjuct_git_path' to replace the
> path to the index with the path recorded in a repository (which would be
> the current, active worktree) only when not asked to construct a path
> for a specific worktree.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  path.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)

Thanks --- this is subtle.  I don't think that what this patch does is
right.  Commenting below.

> --- a/path.c
> +++ b/path.c
> @@ -372,13 +372,20 @@ void report_linked_checkout_garbage(void)
>  }
>  
>  static void adjust_git_path(const struct repository *repo,
> +			    const struct worktree *wt,
>  			    struct strbuf *buf, int git_dir_len)
>  {
>  	const char *base = buf->buf + git_dir_len;
>  	if (is_dir_file(base, "info", "grafts"))
>  		strbuf_splice(buf, 0, buf->len,
>  			      repo->graft_file, strlen(repo->graft_file));
> -	else if (!strcmp(base, "index"))
> +	/*
> +	 * Only try to replace the path '$gitdir/index' with the index file
> +	 * recorded in the repository when not constructing a path for a
> +	 * worktree.  This way we can retrieve the correct path to a particular
> +	 * worktree's index file.
> +	 */
> +	else if (!wt && !strcmp(base, "index"))
>  		strbuf_splice(buf, 0, buf->len,
>  			      repo->index_file, strlen(repo->index_file));

Some context that may have been missing: GIT_INDEX_FILE is a low-level
tool to allow script authors to specify an alternate index file to use
when running commands like "git read-tree" or "git checkout-index".

The above would make it not take effect for git_path() callers when
'wt != NULL'.  As a result, if any caller reaches this code path, then
scripts specifying GIT_INDEX_FILE would stop working when run from a
worktree that borrows refs and objects from a separate repository.
I'm pretty sure that such a subtle flip in behavior based on whether
they are in "git worktree" created worktree or the main working tree
is not what the end user would intend, so this looks like a step in
the wrong direction.

Fortunately this code path doesn't actually get called.

In fact, rewriting git_path("index") in this function feels to me like
a layering violation.  Shouldn't callers be using get_index_file() to
express their intent more clearly?  That's what all current callers
do.

IIUC this came up when a patch from nd/prune-in-worktree (e7a6a3b15,
revision.c: --indexed-objects add objects from all worktrees), which
is currently not in pu, introduced a caller that does call
git_path("index").  The old behavior of replacing git_path("index")
with $GIT_INDEX_FILE when the latter is set was mostly harmless
because typically GIT_INDEX_FILE is not set, especially when people
are running "git prune".  Patch 05/20 (environment: place key
repository state in the_repository) made the substitution harmful: now
we would use repo->index_file unconditionally instead of allowing the
ordinary worktree-relative resolution as a fallback when
GIT_INDEX_FILE is unset.

Possible next steps:

 1. I think we should make git_path less magical and discourage
    callers from relying on it to handle the GIT_INDEX_FILE envvar.
    We can do that by removing the !strcmp(base, "index") case
    completely.

 2. Optionally, it is possible to be more cautious by keeping the
    !strcmp(base, "index") case and making it call BUG() to force
    people not to use it.  This would help steer callers toward
    get_index_file().  But given that the only caller did not actually
    want GIT_INDEX_FILE substitution, I don't think that that is
    necessary or useful.

 3. A docstring for git_path should explain the substitutions it
    currently makes and more straightforward alternatives that callers
    can use.

 4. Specifying GIT_INDEX_FILE when running "git prune" is a
    meaningless combination.  It would be nice for "git prune" to
    error out to save the user from confusion.

 5. There are likely other commands that don't make sense with
    GIT_INDEX_FILE.  Tracking them all down to make them print a
    meaningful error message might be a bit of a slog, though.

>  	else if (dir_prefix(base, "objects"))
> @@ -411,7 +418,7 @@ static void do_git_path(const struct repository *repo,
>  		strbuf_addch(buf, '/');
>  	gitdir_len = buf->len;
>  	strbuf_vaddf(buf, fmt, args);
> -	adjust_git_path(repo, buf, gitdir_len);
> +	adjust_git_path(repo, wt, buf, gitdir_len);
>  	strbuf_cleanup_path(buf);
>  }

Thanks and hope that helps,
Jonathan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-20 23:02       ` Jonathan Nieder
@ 2017-06-21  0:39         ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-21  0:39 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: git, sbeller, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

On 06/20, Jonathan Nieder wrote:
> Hi,
> 
> Brandon Williams wrote:
> 
> > When working with worktrees the git directory is split into two part,
> > the per-worktree gitdir and a commondir which contains things which are
> > shared among all worktrees (like the object store).  With this notion of
> > having a split git directory, 557bd833b (git_path(): be aware of file
> > relocation in $GIT_DIR) and c7b3a3d2f ($GIT_COMMON_DIR: a new
> > environment variable) changed the way that 'git_path()' functioned so
> > that paths would be adjusted if they referred to files or directories
> > that are stored in the commondir or have been changed via an environment
> > variable.
> >
> > One interesting problem with this is the index file as it is not shared
> > across worktrees yet when asking for a path to a specific worktree's
> > index it will be replaced with a path to the current worktree's index.
> > In order to prevent this, teach 'adjuct_git_path' to replace the
> > path to the index with the path recorded in a repository (which would be
> > the current, active worktree) only when not asked to construct a path
> > for a specific worktree.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> > ---
> >  path.c | 11 +++++++++--
> >  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> Thanks --- this is subtle.  I don't think that what this patch does is
> right.  Commenting below.
> 
> > --- a/path.c
> > +++ b/path.c
> > @@ -372,13 +372,20 @@ void report_linked_checkout_garbage(void)
> >  }
> >  
> >  static void adjust_git_path(const struct repository *repo,
> > +			    const struct worktree *wt,
> >  			    struct strbuf *buf, int git_dir_len)
> >  {
> >  	const char *base = buf->buf + git_dir_len;
> >  	if (is_dir_file(base, "info", "grafts"))
> >  		strbuf_splice(buf, 0, buf->len,
> >  			      repo->graft_file, strlen(repo->graft_file));
> > -	else if (!strcmp(base, "index"))
> > +	/*
> > +	 * Only try to replace the path '$gitdir/index' with the index file
> > +	 * recorded in the repository when not constructing a path for a
> > +	 * worktree.  This way we can retrieve the correct path to a particular
> > +	 * worktree's index file.
> > +	 */
> > +	else if (!wt && !strcmp(base, "index"))
> >  		strbuf_splice(buf, 0, buf->len,
> >  			      repo->index_file, strlen(repo->index_file));
> 
> Some context that may have been missing: GIT_INDEX_FILE is a low-level
> tool to allow script authors to specify an alternate index file to use
> when running commands like "git read-tree" or "git checkout-index".
> 
> The above would make it not take effect for git_path() callers when
> 'wt != NULL'.  As a result, if any caller reaches this code path, then
> scripts specifying GIT_INDEX_FILE would stop working when run from a
> worktree that borrows refs and objects from a separate repository.
> I'm pretty sure that such a subtle flip in behavior based on whether
> they are in "git worktree" created worktree or the main working tree
> is not what the end user would intend, so this looks like a step in
> the wrong direction.
> 
> Fortunately this code path doesn't actually get called.
> 
> In fact, rewriting git_path("index") in this function feels to me like
> a layering violation.  Shouldn't callers be using get_index_file() to
> express their intent more clearly?  That's what all current callers
> do.
> 
> IIUC this came up when a patch from nd/prune-in-worktree (e7a6a3b15,
> revision.c: --indexed-objects add objects from all worktrees), which
> is currently not in pu, introduced a caller that does call
> git_path("index").  The old behavior of replacing git_path("index")
> with $GIT_INDEX_FILE when the latter is set was mostly harmless
> because typically GIT_INDEX_FILE is not set, especially when people
> are running "git prune".  Patch 05/20 (environment: place key
> repository state in the_repository) made the substitution harmful: now
> we would use repo->index_file unconditionally instead of allowing the
> ordinary worktree-relative resolution as a fallback when
> GIT_INDEX_FILE is unset.
> 
> Possible next steps:
> 
>  1. I think we should make git_path less magical and discourage
>     callers from relying on it to handle the GIT_INDEX_FILE envvar.
>     We can do that by removing the !strcmp(base, "index") case
>     completely.

From our conversation off-line it seems like we agree that this is
probably the best course of action.  So I'll plan to drop the case
completely since it isn't even used currently.  I'll also go ahead and
drop the graft_file one as well since it doesn't make much sense to keep
it around when it is only queried for once.

> 
>  2. Optionally, it is possible to be more cautious by keeping the
>     !strcmp(base, "index") case and making it call BUG() to force
>     people not to use it.  This would help steer callers toward
>     get_index_file().  But given that the only caller did not actually
>     want GIT_INDEX_FILE substitution, I don't think that that is
>     necessary or useful.
> 
>  3. A docstring for git_path should explain the substitutions it
>     currently makes and more straightforward alternatives that callers
>     can use.
> 
>  4. Specifying GIT_INDEX_FILE when running "git prune" is a
>     meaningless combination.  It would be nice for "git prune" to
>     error out to save the user from confusion.
> 
>  5. There are likely other commands that don't make sense with
>     GIT_INDEX_FILE.  Tracking them all down to make them print a
>     meaningful error message might be a bit of a slog, though.
> 
> >  	else if (dir_prefix(base, "objects"))
> > @@ -411,7 +418,7 @@ static void do_git_path(const struct repository *repo,
> >  		strbuf_addch(buf, '/');
> >  	gitdir_len = buf->len;
> >  	strbuf_vaddf(buf, fmt, args);
> > -	adjust_git_path(repo, buf, gitdir_len);
> > +	adjust_git_path(repo, wt, buf, gitdir_len);
> >  	strbuf_cleanup_path(buf);
> >  }
> 
> Thanks and hope that helps,
> Jonathan

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-20 19:19     ` [PATCH v3 11/20] path: construct correct path to a worktree's index Brandon Williams
  2017-06-20 21:10       ` Stefan Beller
  2017-06-20 23:02       ` Jonathan Nieder
@ 2017-06-21  2:10       ` Jonathan Nieder
  2017-06-21  2:30         ` Jonathan Nieder
  2017-06-21 15:43         ` Brandon Williams
  2 siblings, 2 replies; 214+ messages in thread
From: Jonathan Nieder @ 2017-06-21  2:10 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

Hi again,

Brandon Williams wrote:

> When working with worktrees the git directory is split into two part,
> the per-worktree gitdir and a commondir which contains things which are
> shared among all worktrees (like the object store).  With this notion of
> having a split git directory, 557bd833b (git_path(): be aware of file
> relocation in $GIT_DIR) and c7b3a3d2f ($GIT_COMMON_DIR: a new
> environment variable) changed the way that 'git_path()' functioned so
> that paths would be adjusted if they referred to files or directories
> that are stored in the commondir or have been changed via an environment
> variable.
>
> One interesting problem with this is the index file as it is not shared
> across worktrees yet when asking for a path to a specific worktree's
> index it will be replaced with a path to the current worktree's index.
> In order to prevent this, teach 'adjuct_git_path' to replace the
> path to the index with the path recorded in a repository (which would be
> the current, active worktree) only when not asked to construct a path
> for a specific worktree.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  path.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)

Digging more into it with your help, I ran into v2.5.0-rc0~143^2~34
and v2.11.0-rc0~15^2 (git-svn: "git worktree" awareness, 2016-10-14),
which uses this function:

 -	my $index = $ENV{GIT_INDEX_FILE} || "$ENV{GIT_DIR}/index";
 +	my $index = command_oneline(qw(rev-parse --git-path index));

That rules out my hope of making git_path stop being aware of the index
at all.

That also made it a little clearer to me what is going on with this
function.  I had previously misread !wt as meaning that git_dir is
not under worktrees/ (or in other words as !wt->id).  Instead, it
means that the caller does not want to use some other work tree in
preference to git_dir.

With that piece in place, the patch starts making more sense to me.
When !wt, repo->index_file is going to have the same value as
getenv("GIT_INDEX_FILE") ?: buf->buf already, since repo->index_file
refers to the current work tree git dir's index file.  When wt != NULL,
we don't want to use that file and have requested to grab the index
from a specific work tree git dir instead.

And patch 05/20 (environment: place key repository state in
the_repository) had no effect since no one calls worktree_git_path
with argument "index", nor with a user-specified path.

How about the following?  I found it a little easier to understand.

-- >8 --
From: Brandon Williams <bmwill@google.com>
Subject: worktree_git_path: do not let GIT_INDEX_FILE override path to index

git_path (and variants like strbuf_git_path) is designed to behave
like a convenience function that produces a string $GIT_DIR/<path>.
In the process, callers automatically get support for path relocation
variables like $GIT_OBJECT_DIRECTORY:

- git_path("index") is $GIT_INDEX_FILE when set
- git_path("info/grafts") is $GIT_GRAFTS_FILE when set
- git_path("objects/<foo>") is $GIT_OBJECT_DIRECTORY/<foo> when set
- git_path("hooks/<foo>") is <foo> under core.hookspath when set
- git_path("refs/<foo>") etc (see path.c::common_list) is relative
  to $GIT_COMMON_DIR instead of $GIT_DIR

worktree_git_path, by comparison, is designed to resolve files in a
specific worktree's git dir and should not perform such relocation.
Do so by skipping the relocation step in worktree_git_path.

Fortunately no current callers of worktree_git_path pass such
arguments.  Noticed due to an interaction between two patches under
review, one of which introduced such a caller:

* One made "git prune" check the index file in each worktree's git dir
  (using worktree_git_path(wt, "index")) for objects not to prune,
  triggering the unwanted relocation code by mistake and allowing
  objects reachable from worktree indices to be pruned if
  GIT_INDEX_FILE is set.

* The other simplified the relocation logic for index, info/grafts,
  objects, and hooks to happen unconditionally instead of based on
  whether environment or configuration variables are set, causing the
  relocation to trigger even when GIT_INDEX_FILE is not set.

[jn: rewrote commit message; skipped all relocations instead of just
 the index]

Change-Id: I2ba0a48a48b7e9a9c2e3ef97648cf53cb913bdd9
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 path.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/path.c b/path.c
index c1cb1cf62..4c3a27a8e 100644
--- a/path.c
+++ b/path.c
@@ -397,7 +397,8 @@ static void do_git_path(const struct worktree *wt, struct strbuf *buf,
 		strbuf_addch(buf, '/');
 	gitdir_len = buf->len;
 	strbuf_vaddf(buf, fmt, args);
-	adjust_git_path(buf, gitdir_len);
+	if (!wt)
+		adjust_git_path(buf, gitdir_len);
 	strbuf_cleanup_path(buf);
 }
 
-- 
2.13.1.611.g7e3b11ae1


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-21  2:10       ` Jonathan Nieder
@ 2017-06-21  2:30         ` Jonathan Nieder
  2017-06-21 15:43         ` Brandon Williams
  1 sibling, 0 replies; 214+ messages in thread
From: Jonathan Nieder @ 2017-06-21  2:30 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

Subject: worktree_git_path() should not use file relocation

git_path is a convenience function that usually produces a string
$GIT_DIR/<path>.  Since v2.5.0-rc0~143^2~35 (git_path(): be aware of
file relocation in $GIT_DIR, 2014-11-30), as a side benefit callers
get support for path relocation variables like $GIT_OBJECT_DIRECTORY:

- git_path("index") is $GIT_INDEX_FILE when set
- git_path("info/grafts") is $GIT_GRAFTS_FILE when set
- git_path("objects/<foo>") is $GIT_OBJECT_DIRECTORY/<foo> when set
- git_path("hooks/<foo>") is <foo> under core.hookspath when set
- git_path("refs/<foo>") etc (see path.c::common_list) is relative
  to $GIT_COMMON_DIR instead of $GIT_DIR

worktree_git_path, by comparison, is designed to resolve files in a
specific worktree's git dir.  Unfortunately, it shares code with
git_path and performs the same relocation.  The result is that paths
that are meant to be relative to the specified worktree's git dir end
up replaced by paths from environment variables within the current git
dir.

Luckily, no current callers pass such arguments.  The relocation was
noticed when testing the result of merging two patches under review,
one of which introduces a caller:

* The first patch made git prune check the index file in each
  worktree's git dir (using worktree_git_path(wt, "index")) for
  objects not to prune.  This would trigger the unwanted relocation
  when GIT_INDEX_FILE is set, causing objects reachable from the
  index to be pruned.

* The second patch simplified the relocation logic for index,
  info/grafts, objects, and hooks to happen unconditionally instead of
  based on whether environment or configuration variables are set.
  This caused the relocation to trigger even when GIT_INDEX_FILE is
  not set.

[jn: rewrote commit message; skipping all relocation instead of just
 GIT_INDEX_FILE]

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Jonathan Nieder wrote:

> How about the following?  I found it a little easier to understand.
>
> -- >8 --
> From: Brandon Williams <bmwill@google.com>
> Subject: worktree_git_path: do not let GIT_INDEX_FILE override path to index
[...]
> Change-Id: I2ba0a48a48b7e9a9c2e3ef97648cf53cb913bdd9

Gah, sorry about the stray Change-Id line.  While we're fixing that,
here's a version with a slightly clearer commit message.

 path.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/path.c b/path.c
index c1cb1cf62..4c3a27a8e 100644
--- a/path.c
+++ b/path.c
@@ -397,7 +397,8 @@ static void do_git_path(const struct worktree *wt, struct strbuf *buf,
 		strbuf_addch(buf, '/');
 	gitdir_len = buf->len;
 	strbuf_vaddf(buf, fmt, args);
-	adjust_git_path(buf, gitdir_len);
+	if (!wt)
+		adjust_git_path(buf, gitdir_len);
 	strbuf_cleanup_path(buf);
 }
 
-- 
2.13.1.611.g7e3b11ae1-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-21  2:10       ` Jonathan Nieder
  2017-06-21  2:30         ` Jonathan Nieder
@ 2017-06-21 15:43         ` Brandon Williams
  2017-06-21 17:57           ` Jonathan Nieder
  1 sibling, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-21 15:43 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: git, sbeller, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

On 06/20, Jonathan Nieder wrote:
> Hi again,
> 
> Brandon Williams wrote:
> 
> > When working with worktrees the git directory is split into two part,
> > the per-worktree gitdir and a commondir which contains things which are
> > shared among all worktrees (like the object store).  With this notion of
> > having a split git directory, 557bd833b (git_path(): be aware of file
> > relocation in $GIT_DIR) and c7b3a3d2f ($GIT_COMMON_DIR: a new
> > environment variable) changed the way that 'git_path()' functioned so
> > that paths would be adjusted if they referred to files or directories
> > that are stored in the commondir or have been changed via an environment
> > variable.
> >
> > One interesting problem with this is the index file as it is not shared
> > across worktrees yet when asking for a path to a specific worktree's
> > index it will be replaced with a path to the current worktree's index.
> > In order to prevent this, teach 'adjuct_git_path' to replace the
> > path to the index with the path recorded in a repository (which would be
> > the current, active worktree) only when not asked to construct a path
> > for a specific worktree.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> > ---
> >  path.c | 11 +++++++++--
> >  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> Digging more into it with your help, I ran into v2.5.0-rc0~143^2~34
> and v2.11.0-rc0~15^2 (git-svn: "git worktree" awareness, 2016-10-14),
> which uses this function:
> 
>  -	my $index = $ENV{GIT_INDEX_FILE} || "$ENV{GIT_DIR}/index";
>  +	my $index = command_oneline(qw(rev-parse --git-path index));
> 
> That rules out my hope of making git_path stop being aware of the index
> at all.
> 
> That also made it a little clearer to me what is going on with this
> function.  I had previously misread !wt as meaning that git_dir is
> not under worktrees/ (or in other words as !wt->id).  Instead, it
> means that the caller does not want to use some other work tree in
> preference to git_dir.
> 
> With that piece in place, the patch starts making more sense to me.
> When !wt, repo->index_file is going to have the same value as
> getenv("GIT_INDEX_FILE") ?: buf->buf already, since repo->index_file
> refers to the current work tree git dir's index file.  When wt != NULL,
> we don't want to use that file and have requested to grab the index
> from a specific work tree git dir instead.
> 
> And patch 05/20 (environment: place key repository state in
> the_repository) had no effect since no one calls worktree_git_path
> with argument "index", nor with a user-specified path.
> 
> How about the following?  I found it a little easier to understand.

So your suggestion is to completely avoid doing any location when asking
for a worktree_git_path, I guess those code paths which request those
paths should be aware enough that if they need something in commondir to
use git_common_path instead.  My only worry is that it may be difficult
to catch misuse of worktree_git_path during code review, at least that
was one of the motivating factors for originally respecting
GIT_INDEX_FILE and the like.

> 
> -- >8 --
> From: Brandon Williams <bmwill@google.com>
> Subject: worktree_git_path: do not let GIT_INDEX_FILE override path to index
> 
> git_path (and variants like strbuf_git_path) is designed to behave
> like a convenience function that produces a string $GIT_DIR/<path>.
> In the process, callers automatically get support for path relocation
> variables like $GIT_OBJECT_DIRECTORY:
> 
> - git_path("index") is $GIT_INDEX_FILE when set
> - git_path("info/grafts") is $GIT_GRAFTS_FILE when set
> - git_path("objects/<foo>") is $GIT_OBJECT_DIRECTORY/<foo> when set
> - git_path("hooks/<foo>") is <foo> under core.hookspath when set
> - git_path("refs/<foo>") etc (see path.c::common_list) is relative
>   to $GIT_COMMON_DIR instead of $GIT_DIR
> 
> worktree_git_path, by comparison, is designed to resolve files in a
> specific worktree's git dir and should not perform such relocation.
> Do so by skipping the relocation step in worktree_git_path.
> 
> Fortunately no current callers of worktree_git_path pass such
> arguments.  Noticed due to an interaction between two patches under
> review, one of which introduced such a caller:
> 
> * One made "git prune" check the index file in each worktree's git dir
>   (using worktree_git_path(wt, "index")) for objects not to prune,
>   triggering the unwanted relocation code by mistake and allowing
>   objects reachable from worktree indices to be pruned if
>   GIT_INDEX_FILE is set.
> 
> * The other simplified the relocation logic for index, info/grafts,
>   objects, and hooks to happen unconditionally instead of based on
>   whether environment or configuration variables are set, causing the
>   relocation to trigger even when GIT_INDEX_FILE is not set.
> 
> [jn: rewrote commit message; skipped all relocations instead of just
>  the index]
> 
> Change-Id: I2ba0a48a48b7e9a9c2e3ef97648cf53cb913bdd9
> Signed-off-by: Brandon Williams <bmwill@google.com>
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
> ---
>  path.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/path.c b/path.c
> index c1cb1cf62..4c3a27a8e 100644
> --- a/path.c
> +++ b/path.c
> @@ -397,7 +397,8 @@ static void do_git_path(const struct worktree *wt, struct strbuf *buf,
>  		strbuf_addch(buf, '/');
>  	gitdir_len = buf->len;
>  	strbuf_vaddf(buf, fmt, args);
> -	adjust_git_path(buf, gitdir_len);
> +	if (!wt)
> +		adjust_git_path(buf, gitdir_len);
>  	strbuf_cleanup_path(buf);
>  }
>  
> -- 
> 2.13.1.611.g7e3b11ae1
> 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-21 15:43         ` Brandon Williams
@ 2017-06-21 17:57           ` Jonathan Nieder
  2017-06-21 18:48             ` Junio C Hamano
  0 siblings, 1 reply; 214+ messages in thread
From: Jonathan Nieder @ 2017-06-21 17:57 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy

Brandon Williams wrote:

> So your suggestion is to completely avoid doing any location when asking
> for a worktree_git_path, I guess those code paths which request those
> paths should be aware enough that if they need something in commondir to
> use git_common_path instead.  My only worry is that it may be difficult
> to catch misuse of worktree_git_path during code review, at least that
> was one of the motivating factors for originally respecting
> GIT_INDEX_FILE and the like.

Correct: I'm saying that when someone calls worktree_git_path, the
intent is to resolve a path within the worktree git directory.  File
relocation just gets in the way of that.

I am not too worried about misuse because the only reason to call
worktree_git_path is to access a worktree-specific file like HEAD or
index.

Thanks,
Jonathan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 11/20] path: construct correct path to a worktree's index
  2017-06-21 17:57           ` Jonathan Nieder
@ 2017-06-21 18:48             ` Junio C Hamano
  0 siblings, 0 replies; 214+ messages in thread
From: Junio C Hamano @ 2017-06-21 18:48 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: Brandon Williams, git, sbeller, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, peff, git, avarab, jonathantanmy

Jonathan Nieder <jrnieder@gmail.com> writes:

> Brandon Williams wrote:
>
>> So your suggestion is to completely avoid doing any location when asking
>> for a worktree_git_path, I guess those code paths which request those
>> paths should be aware enough that if they need something in commondir to
>> use git_common_path instead.  My only worry is that it may be difficult
>> to catch misuse of worktree_git_path during code review, at least that
>> was one of the motivating factors for originally respecting
>> GIT_INDEX_FILE and the like.
>
> Correct: I'm saying that when someone calls worktree_git_path, the
> intent is to resolve a path within the worktree git directory.  File
> relocation just gets in the way of that.
>
> I am not too worried about misuse because the only reason to call
> worktree_git_path is to access a worktree-specific file like HEAD or
> index.

Until somebody has a brilliant idea "git_path() can be implemented
in terms of worktree_git_path()---give it the current worktree!" ;-)

Just joking.  I agree with the general direction you've shown in the
thread.

Thanks.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 10/20] path: convert do_git_path to take a 'struct repository'
  2017-06-20 22:23       ` Jonathan Tan
@ 2017-06-21 21:20         ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-21 21:20 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On 06/20, Jonathan Tan wrote:
> On Tue, 20 Jun 2017 12:19:41 -0700
> Brandon Williams <bmwill@google.com> wrote:
> 
> > +static void do_git_path(const struct repository *repo,
> > +			const struct worktree *wt, struct strbuf *buf,
> >  			const char *fmt, va_list args)
> >  {
> >  	int gitdir_len;
> > -	strbuf_addstr(buf, get_worktree_git_dir(wt));
> 
> With this change, the get_worktree_git_dir() function no longer seems to
> be used from outside - could it be marked static?

Probably?  But you could argue that it is ok to be part of the public
API.  As in it may be ok to leave as is because some future caller may
want it at some point.

> 
> > +	strbuf_worktree_gitdir(buf, repo, wt);
> >  	if (buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
> >  		strbuf_addch(buf, '/');
> >  	gitdir_len = buf->len;
> >  	strbuf_vaddf(buf, fmt, args);
> > -	adjust_git_path(buf, gitdir_len);
> > +	adjust_git_path(repo, buf, gitdir_len);
> >  	strbuf_cleanup_path(buf);
> >  }

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 04/20] repository: introduce the repository object
  2017-06-20 19:57       ` Stefan Beller
@ 2017-06-21 21:27         ` Brandon Williams
  2017-06-21 21:31           ` Stefan Beller
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-21 21:27 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On 06/20, Stefan Beller wrote:
> On Tue, Jun 20, 2017 at 12:19 PM, Brandon Williams <bmwill@google.com> wrote:
> > Introduce the repository object 'struct repository' which can be used to
> > hold all state pertaining to a git repository.
> >
> > Some of the benefits of object-ifying a repository are:
> >
> >   1. Make the code base more readable and easier to reason about.
> >
> >   2. Allow for working on multiple repositories, specifically
> >      submodules, within the same process.  Currently the process for
> >      working on a submodule involves setting up an argv_array of options
> >      for a particular command and then launching a child process to
> >      execute the command in the context of the submodule.  This is
> >      clunky and can require lots of little hacks in order to ensure
> >      correctness.  Ideally it would be nice to simply pass a repository
> >      and an options struct to a command.
> >
> >   3. Eliminating reliance on global state will make it easier to
> >      enable the use of threading to improve performance.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> 
> > +/*
> > + * Initialize 'repo' based on the provided 'gitdir'.
> > + * Return 0 upon success and a non-zero value upon failure.
> 
> Non zero or negative? The point of the question is if we want to
> ask users of this function to be cautious early on. So in the future,
> do we want to rather see
> 
>     if (repo_init(...))
>         die("you're doomed");
> 
> or rather
> 
>     int x = repo_init(...);
>     if (x < 0)
>         die("you're doomed");
>     else if (x == 1)
>         warning("you're not doomed, but close."\
>              "Not distimming the gostaks.")
>     else
>         ; /* we're fine, carry on with life */
> 
> I guess we can still refactor later, it's just one
> thing to thing about when introducing an API
> that will likely be used a lot down the road.

I'm not sure what we want right now, hence why I left it a little more
vague.  At this point in time all the relevant callers I can think of
(or rather potential callers) don't care about the failure and just want
to know if it succeeded.  I think it would be ok to do a small refactor
at a later time if we really needed to provide the reason for the
failure.  Unless of course someone feels strongly enough that it needs
to be addressed right now.  If we did address it now then we would need
a group of #define's or maybe an enum to describe the failure modes.

> 
> > +struct repository {
> > +       /* Environment */
> > +       /* Path to the git directory */
> > +       char *gitdir;
> > +
> > +       /* Path to the common git directory */
> > +       char *commondir;
> > +
> > +       /* Path to the repository's object store */
> > +       char *objectdir;
> > +
> > +       /* Path to the repository's graft file */
> > +       char *graft_file;
> > +
> > +       /* Path to the current worktree's index file */
> > +       char *index_file;
> > +
> > +       /* Path to the working directory */
> > +       char *worktree;
> > +
> > +       /* Configurations */
> > +       /*
> > +        * Bit used during initialization to indicate if repository state (like
> > +        * the location of the 'objectdir') should be read from the
> > +        * environment.  By default this bit will be set at the begining of
> > +        * 'repo_init()' so that all repositories will ignore the environment.
> > +        * The exception to this is 'the_repository', which doesn't go through
> > +        * the normal 'repo_init()' process.
> > +        */
> > +       unsigned ignore_env:1;
> > +
> > +       /* Indicate if a repository has a different 'commondir' from 'gitdir' */
> > +       unsigned different_commondir:1;
> > +};
> 
> I applaud the effort towards documenting what each variable is
> supposed to contain. But some of them read like
> 
>     /* increments i by one */
>     i++;
> 
> which is considered bad comment style (it doesn't add
> more information, it just wastes a line), so specifically for
> all the "Path to X" comments:
> * Are they absolute path, or relative path?
>   If relative, then relative to what?
> * Can they be NULL? When?
> 
> (* Why do we need so many path?
>     Could one of them be constructed using
>     another and then hardcoding a string relative to it?
>     This question may rather be answered in the commit
>     message)

Thanks for pointing this out.  I'll work a little bit more on the
comments to be more descriptive.  I do think that all field names should
probably be commented though.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 04/20] repository: introduce the repository object
  2017-06-21 21:27         ` Brandon Williams
@ 2017-06-21 21:31           ` Stefan Beller
  0 siblings, 0 replies; 214+ messages in thread
From: Stefan Beller @ 2017-06-21 21:31 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

>> I guess we can still refactor later, it's just one
>> thing to thing about when introducing an API
>> that will likely be used a lot down the road.
>
> I'm not sure what we want right now, hence why I left it a little more
> vague.  At this point in time all the relevant callers I can think of
> (or rather potential callers) don't care about the failure and just want
> to know if it succeeded.  I think it would be ok to do a small refactor
> at a later time if we really needed to provide the reason for the
> failure.  Unless of course someone feels strongly enough that it needs
> to be addressed right now.  If we did address it now then we would need
> a group of #define's or maybe an enum to describe the failure modes.

I do not feel strongly, just wanted to draw your attention to it.
And having thought about it, refactoring down the road is likely quite cheap,
so this was a useless bikeshedding attempt.

>>
>> I applaud the effort towards documenting what each variable is
>> supposed to contain. But some of them read like
>>
>>     /* increments i by one */
>>     i++;
>>
>> which is considered bad comment style (it doesn't add
>> more information, it just wastes a line), so specifically for
>> all the "Path to X" comments:
>> * Are they absolute path, or relative path?
>>   If relative, then relative to what?
>> * Can they be NULL? When?
>>
>> (* Why do we need so many path?
>>     Could one of them be constructed using
>>     another and then hardcoding a string relative to it?
>>     This question may rather be answered in the commit
>>     message)
>
> Thanks for pointing this out.  I'll work a little bit more on the
> comments to be more descriptive.  I do think that all field names should
> probably be commented though.

Thanks!
Stefan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 20/20] ls-files: use repository object
  2017-06-20 19:19     ` [PATCH v3 20/20] ls-files: use repository object Brandon Williams
@ 2017-06-21 22:48       ` Jonathan Tan
  0 siblings, 0 replies; 214+ messages in thread
From: Jonathan Tan @ 2017-06-21 22:48 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On Tue, 20 Jun 2017 12:19:51 -0700
Brandon Williams <bmwill@google.com> wrote:

> -static void show_ce_entry(const struct index_state *istate,
> -			  const char *tag, const struct cache_entry *ce)
> +static void show_ce(struct repository *repo, struct dir_struct *dir,
> +		    const struct cache_entry *ce, const char *fullname,
> +		    const char *tag)

This function is getting complicated - in particular, it's getting a
"fullname" which is, in a sense, redundant with "repo" and "ce" (but
that seems necessary for efficiency). Maybe add a comment describing the
function?

[snip]

> @@ -651,8 +610,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
>  	if (require_work_tree && !is_inside_work_tree())
>  		setup_work_tree();
>  
> -	if (recurse_submodules)
> -		compile_submodule_options(argv, &dir, show_tag);
> +	if (recurse_submodules) {
> +		repo_read_gitmodules(the_repository);
> +	}

No need for braces.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 15/20] repository: add index_state to struct repo
  2017-06-20 19:19     ` [PATCH v3 15/20] repository: add index_state to struct repo Brandon Williams
@ 2017-06-21 22:50       ` Jonathan Tan
  2017-06-21 22:54         ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Jonathan Tan @ 2017-06-21 22:50 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On Tue, 20 Jun 2017 12:19:46 -0700
Brandon Williams <bmwill@google.com> wrote:

> +int repo_read_index(struct repository *repo)
> +{
> +	if (!repo->index)
> +		repo->index = xcalloc(1, sizeof(struct index_state));

sizeof(*repo->index)?

[snip]

> +	/* Repository's in-memory index */
> +	struct index_state *index;
> +

Might be worth commenting that repo_read_index() can populate this.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 15/20] repository: add index_state to struct repo
  2017-06-21 22:50       ` Jonathan Tan
@ 2017-06-21 22:54         ` Brandon Williams
  2017-06-21 23:00           ` Stefan Beller
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-21 22:54 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On 06/21, Jonathan Tan wrote:
> On Tue, 20 Jun 2017 12:19:46 -0700
> Brandon Williams <bmwill@google.com> wrote:
> 
> > +int repo_read_index(struct repository *repo)
> > +{
> > +	if (!repo->index)
> > +		repo->index = xcalloc(1, sizeof(struct index_state));
> 
> sizeof(*repo->index)?

Is there a reason to prefer one versus the other? Either way I can
change.

> 
> [snip]
> 
> > +	/* Repository's in-memory index */
> > +	struct index_state *index;
> > +
> 
> Might be worth commenting that repo_read_index() can populate this.

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 19/20] repository: enable initialization of submodules
  2017-06-20 19:19     ` [PATCH v3 19/20] repository: enable initialization of submodules Brandon Williams
@ 2017-06-21 23:00       ` Jonathan Tan
  2017-06-21 23:09         ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Jonathan Tan @ 2017-06-21 23:00 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On Tue, 20 Jun 2017 12:19:50 -0700
Brandon Williams <bmwill@google.com> wrote:

> Introduce 'repo_submodule_init()' which performs initialization of a
> 'struct repository' as a submodule of another 'struct repository'.
> 
> The resulting submodule can be in one of three states:
> 
>   1. The submodule is initialized and has a worktree.
> 
>   2. The submodule is initialized but does not have a worktree.  This
>      would occur when the submodule's gitdir is present in the
>      superproject's 'gitdir/modules/' directory yet the submodule has not
>      been checked out in superproject's worktree.

In a recent proposal [1] to update the submodule documentation, an
"initialized submodule" is one that has a working directory, which seems
to have a different meaning of "initialized" (to the paragraphs above).

Or did you mean the "struct repository" is initialized etc.? In which
case, it does not seem strange to me that a repository is initialized
but does not have a worktree, since bare repositories are like that too.

[1] https://public-inbox.org/git/20170621173756.4444-1-sbeller@google.com/

>   3. The submodule remains uninitialized due to an error in the
>      initialization process or there is no matching submodule at the
>      provided path in the superproject.
> 
> Signed-off-by: Brandon Williams <bmwill@google.com>

[snip]

> +/*
> + * Initialize 'submodule' as the submodule given by 'path' in parent repository
> + * 'superproject'.
> + * Return 0 upon success and a non-zero value upon failure.
> + */
> +int repo_submodule_init(struct repository *submodule,
> +			struct repository *superproject,
> +			const char *path)
> +{
> +	const struct submodule *sub;
> +	struct strbuf submodule_path = STRBUF_INIT;
> +	int ret = 0;
> +
> +	sub = submodule_from_cache(superproject, null_sha1, path);
> +	if (!sub) {
> +		ret = -1;
> +		goto out;
> +	}
> +
> +	strbuf_repo_worktree_path(&submodule_path, superproject, "%s", path);
> +
> +	if (repo_init(submodule, submodule_path.buf, submodule_path.buf)) {

This works because the 2nd parameter (git_dir) can take in either the
Git directory itself or its parent, but it does make the call site look
strange. Would it be a good idea to make it mandatory to specify the Git
directory? That would make call sites clearer but require more code
there.

> +		strbuf_reset(&submodule_path);
> +		strbuf_repo_git_path(&submodule_path, superproject,
> +				     "modules/%s", sub->name);
> +
> +		if (repo_init(submodule, submodule_path.buf, NULL)) {
> +			ret = -1;
> +			goto out;
> +		}
> +	}
> +
> +	submodule->submodule_prefix = xstrfmt("%s%s/",
> +					      superproject->submodule_prefix ?
> +					      superproject->submodule_prefix :
> +					      "", path);
> +
> +out:
> +	strbuf_release(&submodule_path);
> +	return ret;
> +}

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 15/20] repository: add index_state to struct repo
  2017-06-21 22:54         ` Brandon Williams
@ 2017-06-21 23:00           ` Stefan Beller
  0 siblings, 0 replies; 214+ messages in thread
From: Stefan Beller @ 2017-06-21 23:00 UTC (permalink / raw)
  To: Brandon Williams
  Cc: Jonathan Tan, git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason

On Wed, Jun 21, 2017 at 3:54 PM, Brandon Williams <bmwill@google.com> wrote:
> On 06/21, Jonathan Tan wrote:
>> On Tue, 20 Jun 2017 12:19:46 -0700
>> Brandon Williams <bmwill@google.com> wrote:
>>
>> > +int repo_read_index(struct repository *repo)
>> > +{
>> > +   if (!repo->index)
>> > +           repo->index = xcalloc(1, sizeof(struct index_state));
>>
>> sizeof(*repo->index)?
>
> Is there a reason to prefer one versus the other? Either way I can
> change.

If repo->index changes its type, then we do not need to fix it here
strictly. We may fix surrounding code, but not necessarily, e.g.
when the type change is a cast to a wider type (subclass so to speak),
so e.g. *repo->index is made to be:

    struct new_index {
      struct index_state index;
      int more_state
    };

then this area of code would just work fine, no maintenance burden.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v3 19/20] repository: enable initialization of submodules
  2017-06-21 23:00       ` Jonathan Tan
@ 2017-06-21 23:09         ` Brandon Williams
  0 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-21 23:09 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, gitster, peff, git, avarab

On 06/21, Jonathan Tan wrote:
> On Tue, 20 Jun 2017 12:19:50 -0700
> Brandon Williams <bmwill@google.com> wrote:
> 
> > Introduce 'repo_submodule_init()' which performs initialization of a
> > 'struct repository' as a submodule of another 'struct repository'.
> > 
> > The resulting submodule can be in one of three states:
> > 
> >   1. The submodule is initialized and has a worktree.
> > 
> >   2. The submodule is initialized but does not have a worktree.  This
> >      would occur when the submodule's gitdir is present in the
> >      superproject's 'gitdir/modules/' directory yet the submodule has not
> >      been checked out in superproject's worktree.
> 
> In a recent proposal [1] to update the submodule documentation, an
> "initialized submodule" is one that has a working directory, which seems
> to have a different meaning of "initialized" (to the paragraphs above).
> 
> Or did you mean the "struct repository" is initialized etc.? In which
> case, it does not seem strange to me that a repository is initialized
> but does not have a worktree, since bare repositories are like that too.

Yes "initialization" only refers to the state of the 'struct
repository'.

> 
> [1] https://public-inbox.org/git/20170621173756.4444-1-sbeller@google.com/
> 
> >   3. The submodule remains uninitialized due to an error in the
> >      initialization process or there is no matching submodule at the
> >      provided path in the superproject.
> > 
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> 
> [snip]
> 
> > +/*
> > + * Initialize 'submodule' as the submodule given by 'path' in parent repository
> > + * 'superproject'.
> > + * Return 0 upon success and a non-zero value upon failure.
> > + */
> > +int repo_submodule_init(struct repository *submodule,
> > +			struct repository *superproject,
> > +			const char *path)
> > +{
> > +	const struct submodule *sub;
> > +	struct strbuf submodule_path = STRBUF_INIT;
> > +	int ret = 0;
> > +
> > +	sub = submodule_from_cache(superproject, null_sha1, path);
> > +	if (!sub) {
> > +		ret = -1;
> > +		goto out;
> > +	}
> > +
> > +	strbuf_repo_worktree_path(&submodule_path, superproject, "%s", path);
> > +
> > +	if (repo_init(submodule, submodule_path.buf, submodule_path.buf)) {
> 
> This works because the 2nd parameter (git_dir) can take in either the
> Git directory itself or its parent, but it does make the call site look
> strange. Would it be a good idea to make it mandatory to specify the Git
> directory? That would make call sites clearer but require more code
> there.

Correct, The idea was to make it easy for callers to initialize
repositories...but you may have convinced me to change that and require
and exact path to the gitdir.  That would actually make the repo_init
code cleaner too.  Originally I was worried that extra boiler plate code
would be needed everytime we wanted to init a submodule, but then I
introduced this funciton so the extra logic would be contained in this
function.

> 
> > +		strbuf_reset(&submodule_path);
> > +		strbuf_repo_git_path(&submodule_path, superproject,
> > +				     "modules/%s", sub->name);
> > +
> > +		if (repo_init(submodule, submodule_path.buf, NULL)) {
> > +			ret = -1;
> > +			goto out;
> > +		}
> > +	}
> > +
> > +	submodule->submodule_prefix = xstrfmt("%s%s/",
> > +					      superproject->submodule_prefix ?
> > +					      superproject->submodule_prefix :
> > +					      "", path);
> > +
> > +out:
> > +	strbuf_release(&submodule_path);
> > +	return ret;
> > +}

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v4 00/20] repository object
  2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
                       ` (20 preceding siblings ...)
  2017-06-20 19:23     ` [PATCH v3 00/20] " Stefan Beller
@ 2017-06-22 18:43     ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
                         ` (21 more replies)
  21 siblings, 22 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

As before you can find this series at:
https://github.com/bmwill/git/tree/repository-object

Changes in v4:

* Patch 11 is slightly different and turns off all path relocation when a
  worktree is provided instead of just for the index file (Thanks for the help
  Jonathan Nieder).
* 'repo_init()' has a tighter API and now requires that the provided gitdir is
  a path to the gitdir instead of either a path to the gitdir or path to the
  worktree (which has a .git file or directory) (Thanks Jonathan Tan).
* Minor comment and commit message chagnes

Note: Like v3 this series is dependent on on 'bw/config-h' and
      'bw/ls-files-sans-the-index'

Brandon Williams (20):
  setup: don't perform lazy initialization of repository state
  setup: add comment indicating a hack
  environment: remove namespace_len variable
  repository: introduce the repository object
  environment: place key repository state in the_repository
  environment: store worktree in the_repository
  path: create path.h
  path: always pass in commondir to update_common_dir
  path: convert strbuf_git_common_path to take a 'struct repository'
  path: convert do_git_path to take a 'struct repository'
  path: worktree_git_path() should not use file relocation
  path: add repo_git_path and strbuf_repo_git_path
  path: add repo_worktree_path and strbuf_repo_worktree_path
  config: read config from a repository object
  repository: add index_state to struct repo
  submodule-config: store the_submodule_cache in the_repository
  submodule: add repo_read_gitmodules
  submodule: convert is_submodule_initialized to work on a repository
  repository: enable initialization of submodules
  ls-files: use repository object

 Makefile                               |   1 +
 builtin/grep.c                         |   3 +-
 builtin/ls-files.c                     | 192 +++++++++++---------------
 builtin/submodule--helper.c            |   9 +-
 cache.h                                |  62 +--------
 config.c                               | 216 +++++++++++++++++++----------
 config.h                               |  24 ++++
 environment.c                          |  91 ++++---------
 git.c                                  |   2 +-
 path.c                                 | 130 ++++++++++++++----
 path.h                                 |  82 +++++++++++
 repository.c                           | 242 +++++++++++++++++++++++++++++++++
 repository.h                           |  97 +++++++++++++
 setup.c                                |  33 +++++
 submodule-config.c                     |  70 +++++++---
 submodule-config.h                     |  10 ++
 submodule.c                            |  35 +++--
 submodule.h                            |   4 +-
 t/t3007-ls-files-recurse-submodules.sh |  39 ++++++
 worktree.c                             |   3 +-
 20 files changed, 972 insertions(+), 373 deletions(-)
 create mode 100644 path.h
 create mode 100644 repository.c
 create mode 100644 repository.h

-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply	[flat|nested] 214+ messages in thread

* [PATCH v4 01/20] setup: don't perform lazy initialization of repository state
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 02/20] setup: add comment indicating a hack Brandon Williams
                         ` (20 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Under some circumstances (bogus GIT_DIR value or the discovered gitdir
is '.git') 'setup_git_directory()' won't initialize key repository
state.  This leads to inconsistent state after running the setup code.
To account for this inconsistent state, lazy initialization is done once
a caller asks for the repository's gitdir or some other piece of
repository state.  This is confusing and can be error prone.

Instead let's tighten the expected outcome of 'setup_git_directory()'
and ensure that it initializes repository state in all cases that would
have been handled by lazy initialization.

This also lets us drop the requirement to have 'have_git_dir()' check if
the environment variable GIT_DIR was set as that will be handled by the
end of the setup code.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  2 ++
 environment.c | 17 ++++++++---------
 setup.c       | 14 ++++++++++++++
 3 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/cache.h b/cache.h
index 96055c222..7c81749a9 100644
--- a/cache.h
+++ b/cache.h
@@ -462,6 +462,8 @@ static inline enum object_type object_type(unsigned int mode)
  */
 extern const char * const local_repo_env[];
 
+extern void setup_git_env(void);
+
 /*
  * Returns true iff we have a configured git repository (either via
  * setup_git_directory, or in the environment via $GIT_DIR).
diff --git a/environment.c b/environment.c
index d40b21fb7..a73b08f5d 100644
--- a/environment.c
+++ b/environment.c
@@ -160,7 +160,7 @@ static char *git_path_from_env(const char *envvar, const char *git_dir,
 	return xstrdup(value);
 }
 
-static void setup_git_env(void)
+void setup_git_env(void)
 {
 	struct strbuf sb = STRBUF_INIT;
 	const char *gitfile;
@@ -205,28 +205,27 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir
-		|| getenv(GIT_DIR_ENVIRONMENT);
+		|| git_dir;
 }
 
 const char *get_git_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_dir;
 }
 
 const char *get_git_common_dir(void)
 {
 	if (!git_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_common_dir;
 }
 
 const char *get_git_namespace(void)
 {
 	if (!namespace)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return namespace;
 }
 
@@ -276,7 +275,7 @@ const char *get_git_work_tree(void)
 char *get_object_directory(void)
 {
 	if (!git_object_dir)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_object_dir;
 }
 
@@ -316,14 +315,14 @@ int odb_pack_keep(const char *name)
 char *get_index_file(void)
 {
 	if (!git_index_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_index_file;
 }
 
 char *get_graft_file(void)
 {
 	if (!git_graft_file)
-		setup_git_env();
+		BUG("git environment hasn't been setup");
 	return git_graft_file;
 }
 
diff --git a/setup.c b/setup.c
index 358fbc2e5..24a738b0d 100644
--- a/setup.c
+++ b/setup.c
@@ -1091,6 +1091,20 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	startup_info->have_repository = !nongit_ok || !*nongit_ok;
 	startup_info->prefix = prefix;
 
+	/*
+	 * Not all paths through the setup code will call 'set_git_dir()' (which
+	 * directly sets up the environment) so in order to guarantee that the
+	 * environment is in a consistent state after setup, explicitly setup
+	 * the environment if we have a repository.
+	 *
+	 * NEEDSWORK: currently we allow bogus GIT_DIR values to be set in some
+	 * code paths so we also need to explicitly setup the environment if
+	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
+	 * GIT_DIR values at some point in the future.
+	 */
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
+		setup_git_env();
+
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
 
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 02/20] setup: add comment indicating a hack
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 03/20] environment: remove namespace_len variable Brandon Williams
                         ` (19 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

'GIT_TOPLEVEL_PREFIX_ENVIRONMENT' was added in (b58a68c1c setup: allow
for prefix to be passed to git commands) to aid in fixing a bug where
'ls-files' and 'grep' were not able to properly recurse when called from
within a subdirectory.  Add a 'NEEDSWORK' comment indicating that this
envvar should be removed once 'ls-files' and 'grep' can recurse
in-process.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 setup.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/setup.c b/setup.c
index 24a738b0d..b477faa44 100644
--- a/setup.c
+++ b/setup.c
@@ -1079,6 +1079,12 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die("BUG: unhandled setup_git_directory_1() result");
 	}
 
+	/*
+	 * NEEDSWORK: This was a hack in order to get ls-files and grep to have
+	 * properly formated output when recursing submodules.  Once ls-files
+	 * and grep have been changed to perform this recursing in-process this
+	 * needs to be removed.
+	 */
 	env_prefix = getenv(GIT_TOPLEVEL_PREFIX_ENVIRONMENT);
 	if (env_prefix)
 		prefix = env_prefix;
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 03/20] environment: remove namespace_len variable
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 02/20] setup: add comment indicating a hack Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 04/20] repository: introduce the repository object Brandon Williams
                         ` (18 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Use 'skip_prefix' instead of 'starts_with' so that we can drop the need
to keep around 'namespace_len'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index a73b08f5d..e035f6372 100644
--- a/environment.c
+++ b/environment.c
@@ -98,7 +98,6 @@ char *git_work_tree_cfg;
 static char *work_tree;
 
 static const char *namespace;
-static size_t namespace_len;
 
 static const char *super_prefix;
 
@@ -190,7 +189,6 @@ void setup_git_env(void)
 	git_replace_ref_base = xstrdup(replace_ref_base ? replace_ref_base
 							  : "refs/replace/");
 	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
-	namespace_len = strlen(namespace);
 	shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT);
 	if (shallow_file)
 		set_alternate_shallow_file(shallow_file, 0);
@@ -231,9 +229,10 @@ const char *get_git_namespace(void)
 
 const char *strip_namespace(const char *namespaced_ref)
 {
-	if (!starts_with(namespaced_ref, get_git_namespace()))
-		return NULL;
-	return namespaced_ref + namespace_len;
+	const char *out;
+	if (skip_prefix(namespaced_ref, get_git_namespace(), &out))
+		return out;
+	return NULL;
 }
 
 const char *get_super_prefix(void)
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 04/20] repository: introduce the repository object
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (2 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 03/20] environment: remove namespace_len variable Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 05/20] environment: place key repository state in the_repository Brandon Williams
                         ` (17 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce the repository object 'struct repository' which can be used to
hold all state pertaining to a git repository.

Some of the benefits of object-ifying a repository are:

  1. Make the code base more readable and easier to reason about.

  2. Allow for working on multiple repositories, specifically
     submodules, within the same process.  Currently the process for
     working on a submodule involves setting up an argv_array of options
     for a particular command and then launching a child process to
     execute the command in the context of the submodule.  This is
     clunky and can require lots of little hacks in order to ensure
     correctness.  Ideally it would be nice to simply pass a repository
     and an options struct to a command.

  3. Eliminating reliance on global state will make it easier to
     enable the use of threading to improve performance.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 Makefile     |   1 +
 repository.c | 159 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 repository.h |  64 ++++++++++++++++++++++++
 3 files changed, 224 insertions(+)
 create mode 100644 repository.c
 create mode 100644 repository.h

diff --git a/Makefile b/Makefile
index f48480163..32e4efc71 100644
--- a/Makefile
+++ b/Makefile
@@ -839,6 +839,7 @@ LIB_OBJS += refs/ref-cache.o
 LIB_OBJS += ref-filter.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace_object.o
+LIB_OBJS += repository.o
 LIB_OBJS += rerere.o
 LIB_OBJS += resolve-undo.o
 LIB_OBJS += revision.o
diff --git a/repository.c b/repository.c
new file mode 100644
index 000000000..cf440405a
--- /dev/null
+++ b/repository.c
@@ -0,0 +1,159 @@
+#include "cache.h"
+#include "repository.h"
+
+/* The main repository */
+static struct repository the_repo;
+struct repository *the_repository = &the_repo;
+
+static char *git_path_from_env(const char *envvar, const char *git_dir,
+			       const char *path, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(envvar);
+		if (value)
+			return xstrdup(value);
+	}
+
+	return xstrfmt("%s/%s", git_dir, path);
+}
+
+static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)
+{
+	if (fromenv) {
+		const char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);
+		if (value) {
+			strbuf_addstr(sb, value);
+			return 1;
+		}
+	}
+
+	return get_common_dir_noenv(sb, gitdir);
+}
+
+static void repo_setup_env(struct repository *repo)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	repo->different_commondir = find_common_dir(&sb, repo->gitdir,
+						    !repo->ignore_env);
+	repo->commondir = strbuf_detach(&sb, NULL);
+	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+					    "objects", !repo->ignore_env);
+	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
+					     "info/grafts", !repo->ignore_env);
+	repo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,
+					     "index", !repo->ignore_env);
+}
+
+void repo_set_gitdir(struct repository *repo, const char *path)
+{
+	const char *gitfile = read_gitfile(path);
+
+	/*
+	 * NEEDSWORK: Eventually we want to be able to free gitdir and the rest
+	 * of the environment before reinitializing it again, but we have some
+	 * crazy code paths where we try to set gitdir with the current gitdir
+	 * and we don't want to free gitdir before copying the passed in value.
+	 */
+	repo->gitdir = xstrdup(gitfile ? gitfile : path);
+
+	repo_setup_env(repo);
+}
+
+/*
+ * Attempt to resolve and set the provided 'gitdir' for repository 'repo'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+static int repo_init_gitdir(struct repository *repo, const char *gitdir)
+{
+	int ret = 0;
+	int error = 0;
+	char *abspath = NULL;
+	const char *resolved_gitdir;
+
+	abspath = real_pathdup(gitdir, 0);
+	if (!abspath) {
+		ret = -1;
+		goto out;
+	}
+
+	/* 'gitdir' must reference the gitdir directly */
+	resolved_gitdir = resolve_gitdir_gently(abspath, &error);
+	if (!resolved_gitdir) {
+		ret = -1;
+		goto out;
+	}
+
+	repo_set_gitdir(repo, resolved_gitdir);
+
+out:
+	free(abspath);
+	return ret;
+}
+
+void repo_set_worktree(struct repository *repo, const char *path)
+{
+	repo->worktree = real_pathdup(path, 1);
+}
+
+static int read_and_verify_repository_format(struct repository_format *format,
+					     const char *commondir)
+{
+	int ret = 0;
+	struct strbuf sb = STRBUF_INIT;
+
+	strbuf_addf(&sb, "%s/config", commondir);
+	read_repository_format(format, sb.buf);
+	strbuf_reset(&sb);
+
+	if (verify_repository_format(format, &sb) < 0) {
+		warning("%s", sb.buf);
+		ret = -1;
+	}
+
+	strbuf_release(&sb);
+	return ret;
+}
+
+/*
+ * Initialize 'repo' based on the provided 'gitdir'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
+{
+	struct repository_format format;
+	memset(repo, 0, sizeof(*repo));
+
+	repo->ignore_env = 1;
+
+	if (repo_init_gitdir(repo, gitdir))
+		goto error;
+
+	if (read_and_verify_repository_format(&format, repo->commondir))
+		goto error;
+
+	if (worktree)
+		repo_set_worktree(repo, worktree);
+
+	return 0;
+
+error:
+	repo_clear(repo);
+	return -1;
+}
+
+void repo_clear(struct repository *repo)
+{
+	free(repo->gitdir);
+	repo->gitdir = NULL;
+	free(repo->commondir);
+	repo->commondir = NULL;
+	free(repo->objectdir);
+	repo->objectdir = NULL;
+	free(repo->graft_file);
+	repo->graft_file = NULL;
+	free(repo->index_file);
+	repo->index_file = NULL;
+	free(repo->worktree);
+	repo->worktree = NULL;
+}
diff --git a/repository.h b/repository.h
new file mode 100644
index 000000000..0a1db9633
--- /dev/null
+++ b/repository.h
@@ -0,0 +1,64 @@
+#ifndef REPOSITORY_H
+#define REPOSITORY_H
+
+struct repository {
+	/* Environment */
+	/*
+	 * Path to the git directory.
+	 * Cannot be NULL after initialization.
+	 */
+	char *gitdir;
+
+	/*
+	 * Path to the common git directory.
+	 * Cannot be NULL after initialization.
+	 */
+	char *commondir;
+
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+
+	/*
+	 * Path to the repository's graft file.
+	 * Cannot be NULL after initialization.
+	 */
+	char *graft_file;
+
+	/*
+	 * Path to the current worktree's index file.
+	 * Cannot be NULL after initialization.
+	 */
+	char *index_file;
+
+	/*
+	 * Path to the working directory.
+	 * A NULL value indicates that there is no working directory.
+	 */
+	char *worktree;
+
+	/* Configurations */
+	/*
+	 * Bit used during initialization to indicate if repository state (like
+	 * the location of the 'objectdir') should be read from the
+	 * environment.  By default this bit will be set at the begining of
+	 * 'repo_init()' so that all repositories will ignore the environment.
+	 * The exception to this is 'the_repository', which doesn't go through
+	 * the normal 'repo_init()' process.
+	 */
+	unsigned ignore_env:1;
+
+	/* Indicate if a repository has a different 'commondir' from 'gitdir' */
+	unsigned different_commondir:1;
+};
+
+extern struct repository *the_repository;
+
+extern void repo_set_gitdir(struct repository *repo, const char *path);
+extern void repo_set_worktree(struct repository *repo, const char *path);
+extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
+extern void repo_clear(struct repository *repo);
+
+#endif /* REPOSITORY_H */
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 05/20] environment: place key repository state in the_repository
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (3 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 04/20] repository: introduce the repository object Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 06/20] environment: store worktree " Brandon Williams
                         ` (16 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'git_dir', 'git_common_dir', 'git_object_dir', 'git_index_file',
'git_graft_file', and 'namespace' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h       |  1 -
 environment.c | 58 +++++++++++++---------------------------------------------
 path.c        | 11 ++++++-----
 setup.c       | 17 +++++++++++++++--
 4 files changed, 34 insertions(+), 53 deletions(-)

diff --git a/cache.h b/cache.h
index 7c81749a9..cd64cbc81 100644
--- a/cache.h
+++ b/cache.h
@@ -771,7 +771,6 @@ extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
 extern int protect_hfs;
 extern int protect_ntfs;
-extern int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
 
 /*
  * Include broken refs in all ref iterations, which will
diff --git a/environment.c b/environment.c
index e035f6372..aa79ef83e 100644
--- a/environment.c
+++ b/environment.c
@@ -8,6 +8,7 @@
  * are.
  */
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "refs.h"
 #include "fmt-merge-msg.h"
@@ -101,10 +102,6 @@ static const char *namespace;
 
 static const char *super_prefix;
 
-static const char *git_dir, *git_common_dir;
-static char *git_object_dir, *git_index_file, *git_graft_file;
-int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
-
 /*
  * Repository-local GIT_* environment variables; see cache.h for details.
  */
@@ -148,41 +145,11 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-static char *git_path_from_env(const char *envvar, const char *git_dir,
-			       const char *path, int *fromenv)
-{
-	const char *value = getenv(envvar);
-	if (!value)
-		return xstrfmt("%s/%s", git_dir, path);
-	if (fromenv)
-		*fromenv = 1;
-	return xstrdup(value);
-}
-
 void setup_git_env(void)
 {
-	struct strbuf sb = STRBUF_INIT;
-	const char *gitfile;
 	const char *shallow_file;
 	const char *replace_ref_base;
 
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
-	if (!git_dir) {
-		if (!startup_info->have_repository)
-			BUG("setup_git_env called without repository");
-		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
-	}
-	gitfile = read_gitfile(git_dir);
-	git_dir = xstrdup(gitfile ? gitfile : git_dir);
-	if (get_common_dir(&sb, git_dir))
-		git_common_dir_env = 1;
-	git_common_dir = strbuf_detach(&sb, NULL);
-	git_object_dir = git_path_from_env(DB_ENVIRONMENT, git_common_dir,
-					   "objects", &git_db_env);
-	git_index_file = git_path_from_env(INDEX_ENVIRONMENT, git_dir,
-					   "index", &git_index_env);
-	git_graft_file = git_path_from_env(GRAFT_ENVIRONMENT, git_common_dir,
-					   "info/grafts", &git_graft_env);
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		check_replace_refs = 0;
 	replace_ref_base = getenv(GIT_REPLACE_REF_BASE_ENVIRONMENT);
@@ -203,21 +170,21 @@ int is_bare_repository(void)
 int have_git_dir(void)
 {
 	return startup_info->have_repository
-		|| git_dir;
+		|| the_repository->gitdir;
 }
 
 const char *get_git_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository->gitdir)
 		BUG("git environment hasn't been setup");
-	return git_dir;
+	return the_repository->gitdir;
 }
 
 const char *get_git_common_dir(void)
 {
-	if (!git_dir)
+	if (!the_repository->commondir)
 		BUG("git environment hasn't been setup");
-	return git_common_dir;
+	return the_repository->commondir;
 }
 
 const char *get_git_namespace(void)
@@ -273,9 +240,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!git_object_dir)
+	if (!the_repository->objectdir)
 		BUG("git environment hasn't been setup");
-	return git_object_dir;
+	return the_repository->objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
@@ -313,22 +280,23 @@ int odb_pack_keep(const char *name)
 
 char *get_index_file(void)
 {
-	if (!git_index_file)
+	if (!the_repository->index_file)
 		BUG("git environment hasn't been setup");
-	return git_index_file;
+	return the_repository->index_file;
 }
 
 char *get_graft_file(void)
 {
-	if (!git_graft_file)
+	if (!the_repository->graft_file)
 		BUG("git environment hasn't been setup");
-	return git_graft_file;
+	return the_repository->graft_file;
 }
 
 int set_git_dir(const char *path)
 {
 	if (setenv(GIT_DIR_ENVIRONMENT, path, 1))
 		return error("Could not set GIT_DIR to '%s'", path);
+	repo_set_gitdir(the_repository, path);
 	setup_git_env();
 	return 0;
 }
diff --git a/path.c b/path.c
index c1cb1cf62..e4abea083 100644
--- a/path.c
+++ b/path.c
@@ -2,6 +2,7 @@
  * Utilities for paths and pathnames
  */
 #include "cache.h"
+#include "repository.h"
 #include "strbuf.h"
 #include "string-list.h"
 #include "dir.h"
@@ -355,7 +356,7 @@ void report_linked_checkout_garbage(void)
 	const struct common_dir *p;
 	int len;
 
-	if (!git_common_dir_env)
+	if (!the_repository->different_commondir)
 		return;
 	strbuf_addf(&sb, "%s/", get_git_dir());
 	len = sb.len;
@@ -374,17 +375,17 @@ void report_linked_checkout_garbage(void)
 static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
-	if (git_graft_env && is_dir_file(base, "info", "grafts"))
+	if (is_dir_file(base, "info", "grafts"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_graft_file(), strlen(get_graft_file()));
-	else if (git_index_env && !strcmp(base, "index"))
+	else if (!strcmp(base, "index"))
 		strbuf_splice(buf, 0, buf->len,
 			      get_index_file(), strlen(get_index_file()));
-	else if (git_db_env && dir_prefix(base, "objects"))
+	else if (dir_prefix(base, "objects"))
 		replace_dir(buf, git_dir_len + 7, get_object_directory());
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
-	else if (git_common_dir_env)
+	else if (the_repository->different_commondir)
 		update_common_dir(buf, git_dir_len, NULL);
 }
 
diff --git a/setup.c b/setup.c
index b477faa44..860507e1f 100644
--- a/setup.c
+++ b/setup.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "dir.h"
 #include "string-list.h"
@@ -398,6 +399,11 @@ void setup_work_tree(void)
 	if (getenv(GIT_WORK_TREE_ENVIRONMENT))
 		setenv(GIT_WORK_TREE_ENVIRONMENT, ".", 1);
 
+	/*
+	 * NEEDSWORK: this call can essentially be set_git_dir(get_git_dir())
+	 * which can cause some problems when trying to free the old value of
+	 * gitdir.
+	 */
 	set_git_dir(remove_leading_path(git_dir, work_tree));
 	initialized = 1;
 }
@@ -1108,8 +1114,15 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	 * the user has set GIT_DIR.  It may be beneficial to disallow bogus
 	 * GIT_DIR values at some point in the future.
 	 */
-	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT))
-		setup_git_env();
+	if (startup_info->have_repository || getenv(GIT_DIR_ENVIRONMENT)) {
+		if (!the_repository->gitdir) {
+			const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);
+			if (!gitdir)
+				gitdir = DEFAULT_GIT_DIR_ENVIRONMENT;
+			repo_set_gitdir(the_repository, gitdir);
+			setup_git_env();
+		}
+	}
 
 	strbuf_release(&dir);
 	strbuf_release(&gitdir);
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 06/20] environment: store worktree in the_repository
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (4 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 05/20] environment: place key repository state in the_repository Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 07/20] path: create path.h Brandon Williams
                         ` (15 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Migrate 'work_tree' to be stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 environment.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/environment.c b/environment.c
index aa79ef83e..3fd4b1084 100644
--- a/environment.c
+++ b/environment.c
@@ -96,7 +96,6 @@ int ignore_untracked_cache_config;
 
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
 char *git_work_tree_cfg;
-static char *work_tree;
 
 static const char *namespace;
 
@@ -223,19 +222,19 @@ void set_git_work_tree(const char *new_work_tree)
 {
 	if (git_work_tree_initialized) {
 		new_work_tree = real_path(new_work_tree);
-		if (strcmp(new_work_tree, work_tree))
+		if (strcmp(new_work_tree, the_repository->worktree))
 			die("internal error: work tree has already been set\n"
 			    "Current worktree: %s\nNew worktree: %s",
-			    work_tree, new_work_tree);
+			    the_repository->worktree, new_work_tree);
 		return;
 	}
 	git_work_tree_initialized = 1;
-	work_tree = real_pathdup(new_work_tree, 1);
+	repo_set_worktree(the_repository, new_work_tree);
 }
 
 const char *get_git_work_tree(void)
 {
-	return work_tree;
+	return the_repository->worktree;
 }
 
 char *get_object_directory(void)
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 07/20] path: create path.h
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (5 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 06/20] environment: store worktree " Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 08/20] path: always pass in commondir to update_common_dir Brandon Williams
                         ` (14 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Move all path related declarations from cache.h to a new path.h header
file.  This makes cache.h smaller and makes it easier to add new path
related functions.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 cache.h | 59 +----------------------------------------------------------
 path.c  |  1 +
 path.h  | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+), 58 deletions(-)
 create mode 100644 path.h

diff --git a/cache.h b/cache.h
index cd64cbc81..c958fc3ce 100644
--- a/cache.h
+++ b/cache.h
@@ -11,6 +11,7 @@
 #include "string-list.h"
 #include "pack-revindex.h"
 #include "hash.h"
+#include "path.h"
 
 #ifndef platform_SHA_CTX
 /*
@@ -892,64 +893,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Return a statically allocated filename, either generically (mkpath), in
- * the repository directory (git_path), or in a submodule's repository
- * directory (git_path_submodule). In all cases, note that the result
- * may be overwritten by another call to _any_ of the functions. Consider
- * using the safer "dup" or "strbuf" formats below (in some cases, the
- * unsafe versions have already been removed).
- */
-extern const char *mkpath(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
-extern const char *git_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
-extern const char *git_common_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
-
-extern char *mksnpath(char *buf, size_t n, const char *fmt, ...)
-	__attribute__((format (printf, 3, 4)));
-extern void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-extern void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-extern char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-extern int strbuf_git_path_submodule(struct strbuf *sb, const char *path,
-				     const char *fmt, ...)
-	__attribute__((format (printf, 3, 4)));
-extern char *git_pathdup(const char *fmt, ...)
-	__attribute__((format (printf, 1, 2)));
-extern char *mkpathdup(const char *fmt, ...)
-	__attribute__((format (printf, 1, 2)));
-extern char *git_pathdup_submodule(const char *path, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
-
-extern void report_linked_checkout_garbage(void);
-
-/*
- * You can define a static memoized git path like:
- *
- *    static GIT_PATH_FUNC(git_path_foo, "FOO");
- *
- * or use one of the global ones below.
- */
-#define GIT_PATH_FUNC(func, filename) \
-	const char *func(void) \
-	{ \
-		static char *ret; \
-		if (!ret) \
-			ret = git_pathdup(filename); \
-		return ret; \
-	}
-
-const char *git_path_cherry_pick_head(void);
-const char *git_path_revert_head(void);
-const char *git_path_squash_msg(void);
-const char *git_path_merge_msg(void);
-const char *git_path_merge_rr(void);
-const char *git_path_merge_mode(void);
-const char *git_path_merge_head(void);
-const char *git_path_fetch_head(void);
-const char *git_path_shallow(void);
-
 /*
  * Return the name of the file in the local object database that would
  * be used to store a loose object with the specified sha1.  The
diff --git a/path.c b/path.c
index e4abea083..41c861c96 100644
--- a/path.c
+++ b/path.c
@@ -8,6 +8,7 @@
 #include "dir.h"
 #include "worktree.h"
 #include "submodule-config.h"
+#include "path.h"
 
 static int get_st_mode_bits(const char *path, int *mode)
 {
diff --git a/path.h b/path.h
new file mode 100644
index 000000000..522cd029b
--- /dev/null
+++ b/path.h
@@ -0,0 +1,62 @@
+#ifndef PATH_H
+#define PATH_H
+
+/*
+ * Return a statically allocated filename, either generically (mkpath), in
+ * the repository directory (git_path), or in a submodule's repository
+ * directory (git_path_submodule). In all cases, note that the result
+ * may be overwritten by another call to _any_ of the functions. Consider
+ * using the safer "dup" or "strbuf" formats below (in some cases, the
+ * unsafe versions have already been removed).
+ */
+extern const char *mkpath(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
+extern const char *git_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
+extern const char *git_common_path(const char *fmt, ...) __attribute__((format (printf, 1, 2)));
+
+extern char *mksnpath(char *buf, size_t n, const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+extern void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern int strbuf_git_path_submodule(struct strbuf *sb, const char *path,
+				     const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+extern char *git_pathdup(const char *fmt, ...)
+	__attribute__((format (printf, 1, 2)));
+extern char *mkpathdup(const char *fmt, ...)
+	__attribute__((format (printf, 1, 2)));
+extern char *git_pathdup_submodule(const char *path, const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+
+extern void report_linked_checkout_garbage(void);
+
+/*
+ * You can define a static memoized git path like:
+ *
+ *    static GIT_PATH_FUNC(git_path_foo, "FOO");
+ *
+ * or use one of the global ones below.
+ */
+#define GIT_PATH_FUNC(func, filename) \
+	const char *func(void) \
+	{ \
+		static char *ret; \
+		if (!ret) \
+			ret = git_pathdup(filename); \
+		return ret; \
+	}
+
+const char *git_path_cherry_pick_head(void);
+const char *git_path_revert_head(void);
+const char *git_path_squash_msg(void);
+const char *git_path_merge_msg(void);
+const char *git_path_merge_rr(void);
+const char *git_path_merge_mode(void);
+const char *git_path_merge_head(void);
+const char *git_path_fetch_head(void);
+const char *git_path_shallow(void);
+
+#endif /* PATH_H */
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 08/20] path: always pass in commondir to update_common_dir
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (6 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 07/20] path: create path.h Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 09/20] path: convert strbuf_git_common_path to take a 'struct repository' Brandon Williams
                         ` (13 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Instead of passing in 'NULL' and having 'update_common_dir()' query for
the commondir, have the callers of 'update_common_dir()' be responsible
for providing the commondir.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/path.c b/path.c
index 41c861c96..2434921d8 100644
--- a/path.c
+++ b/path.c
@@ -345,8 +345,6 @@ static void update_common_dir(struct strbuf *buf, int git_dir_len,
 {
 	char *base = buf->buf + git_dir_len;
 	init_common_trie();
-	if (!common_dir)
-		common_dir = get_git_common_dir();
 	if (trie_find(&common_trie, base, check_common, NULL) > 0)
 		replace_dir(buf, git_dir_len, common_dir);
 }
@@ -387,7 +385,7 @@ static void adjust_git_path(struct strbuf *buf, int git_dir_len)
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (the_repository->different_commondir)
-		update_common_dir(buf, git_dir_len, NULL);
+		update_common_dir(buf, git_dir_len, get_git_common_dir());
 }
 
 static void do_git_path(const struct worktree *wt, struct strbuf *buf,
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 09/20] path: convert strbuf_git_common_path to take a 'struct repository'
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (7 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 08/20] path: always pass in commondir to update_common_dir Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 10/20] path: convert do_git_path " Brandon Williams
                         ` (12 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c     | 13 ++++++++-----
 path.h     |  8 ++++++--
 worktree.c |  3 ++-
 3 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/path.c b/path.c
index 2434921d8..9be6804a9 100644
--- a/path.c
+++ b/path.c
@@ -524,11 +524,12 @@ int strbuf_git_path_submodule(struct strbuf *buf, const char *path,
 	return err;
 }
 
-static void do_git_common_path(struct strbuf *buf,
+static void do_git_common_path(const struct repository *repo,
+			       struct strbuf *buf,
 			       const char *fmt,
 			       va_list args)
 {
-	strbuf_addstr(buf, get_git_common_dir());
+	strbuf_addstr(buf, repo->commondir);
 	if (buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
 		strbuf_addch(buf, '/');
 	strbuf_vaddf(buf, fmt, args);
@@ -540,16 +541,18 @@ const char *git_common_path(const char *fmt, ...)
 	struct strbuf *pathname = get_pathname();
 	va_list args;
 	va_start(args, fmt);
-	do_git_common_path(pathname, fmt, args);
+	do_git_common_path(the_repository, pathname, fmt, args);
 	va_end(args);
 	return pathname->buf;
 }
 
-void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
+void strbuf_git_common_path(struct strbuf *sb,
+			    const struct repository *repo,
+			    const char *fmt, ...)
 {
 	va_list args;
 	va_start(args, fmt);
-	do_git_common_path(sb, fmt, args);
+	do_git_common_path(repo, sb, fmt, args);
 	va_end(args);
 }
 
diff --git a/path.h b/path.h
index 522cd029b..568d63be5 100644
--- a/path.h
+++ b/path.h
@@ -1,6 +1,8 @@
 #ifndef PATH_H
 #define PATH_H
 
+struct repository;
+
 /*
  * Return a statically allocated filename, either generically (mkpath), in
  * the repository directory (git_path), or in a submodule's repository
@@ -17,8 +19,10 @@ extern char *mksnpath(char *buf, size_t n, const char *fmt, ...)
 	__attribute__((format (printf, 3, 4)));
 extern void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
 	__attribute__((format (printf, 2, 3)));
-extern void strbuf_git_common_path(struct strbuf *sb, const char *fmt, ...)
-	__attribute__((format (printf, 2, 3)));
+extern void strbuf_git_common_path(struct strbuf *sb,
+				   const struct repository *repo,
+				   const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
 extern char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
 	__attribute__((format (printf, 2, 3)));
 extern int strbuf_git_path_submodule(struct strbuf *sb, const char *path,
diff --git a/worktree.c b/worktree.c
index 2801c6d52..e28ffbeb0 100644
--- a/worktree.c
+++ b/worktree.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
 #include "strbuf.h"
 #include "worktree.h"
@@ -76,7 +77,7 @@ static struct worktree *get_linked_worktree(const char *id)
 	if (!id)
 		die("Missing linked worktree name");
 
-	strbuf_git_common_path(&path, "worktrees/%s/gitdir", id);
+	strbuf_git_common_path(&path, the_repository, "worktrees/%s/gitdir", id);
 	if (strbuf_read_file(&worktree_path, path.buf, 0) <= 0)
 		/* invalid gitdir file */
 		goto done;
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 10/20] path: convert do_git_path to take a 'struct repository'
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (8 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 09/20] path: convert strbuf_git_common_path to take a 'struct repository' Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 11/20] path: worktree_git_path() should not use file relocation Brandon Williams
                         ` (11 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

In preparation to adding 'git_path' like functions which operate on a
'struct repository' convert 'do_git_path' to take a 'struct repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 42 ++++++++++++++++++++++++++++--------------
 1 file changed, 28 insertions(+), 14 deletions(-)

diff --git a/path.c b/path.c
index 9be6804a9..76a872297 100644
--- a/path.c
+++ b/path.c
@@ -371,33 +371,47 @@ void report_linked_checkout_garbage(void)
 	strbuf_release(&sb);
 }
 
-static void adjust_git_path(struct strbuf *buf, int git_dir_len)
+static void adjust_git_path(const struct repository *repo,
+			    struct strbuf *buf, int git_dir_len)
 {
 	const char *base = buf->buf + git_dir_len;
 	if (is_dir_file(base, "info", "grafts"))
 		strbuf_splice(buf, 0, buf->len,
-			      get_graft_file(), strlen(get_graft_file()));
+			      repo->graft_file, strlen(repo->graft_file));
 	else if (!strcmp(base, "index"))
 		strbuf_splice(buf, 0, buf->len,
-			      get_index_file(), strlen(get_index_file()));
+			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, get_object_directory());
+		replace_dir(buf, git_dir_len + 7, repo->objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
-	else if (the_repository->different_commondir)
-		update_common_dir(buf, git_dir_len, get_git_common_dir());
+	else if (repo->different_commondir)
+		update_common_dir(buf, git_dir_len, repo->commondir);
 }
 
-static void do_git_path(const struct worktree *wt, struct strbuf *buf,
+static void strbuf_worktree_gitdir(struct strbuf *buf,
+				   const struct repository *repo,
+				   const struct worktree *wt)
+{
+	if (!wt)
+		strbuf_addstr(buf, repo->gitdir);
+	else if (!wt->id)
+		strbuf_addstr(buf, repo->commondir);
+	else
+		strbuf_git_common_path(buf, repo, "worktrees/%s", wt->id);
+}
+
+static void do_git_path(const struct repository *repo,
+			const struct worktree *wt, struct strbuf *buf,
 			const char *fmt, va_list args)
 {
 	int gitdir_len;
-	strbuf_addstr(buf, get_worktree_git_dir(wt));
+	strbuf_worktree_gitdir(buf, repo, wt);
 	if (buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
 		strbuf_addch(buf, '/');
 	gitdir_len = buf->len;
 	strbuf_vaddf(buf, fmt, args);
-	adjust_git_path(buf, gitdir_len);
+	adjust_git_path(repo, buf, gitdir_len);
 	strbuf_cleanup_path(buf);
 }
 
@@ -406,7 +420,7 @@ char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
 	va_list args;
 	strbuf_reset(buf);
 	va_start(args, fmt);
-	do_git_path(NULL, buf, fmt, args);
+	do_git_path(the_repository, NULL, buf, fmt, args);
 	va_end(args);
 	return buf->buf;
 }
@@ -415,7 +429,7 @@ void strbuf_git_path(struct strbuf *sb, const char *fmt, ...)
 {
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(NULL, sb, fmt, args);
+	do_git_path(the_repository, NULL, sb, fmt, args);
 	va_end(args);
 }
 
@@ -424,7 +438,7 @@ const char *git_path(const char *fmt, ...)
 	struct strbuf *pathname = get_pathname();
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(NULL, pathname, fmt, args);
+	do_git_path(the_repository, NULL, pathname, fmt, args);
 	va_end(args);
 	return pathname->buf;
 }
@@ -434,7 +448,7 @@ char *git_pathdup(const char *fmt, ...)
 	struct strbuf path = STRBUF_INIT;
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(NULL, &path, fmt, args);
+	do_git_path(the_repository, NULL, &path, fmt, args);
 	va_end(args);
 	return strbuf_detach(&path, NULL);
 }
@@ -465,7 +479,7 @@ const char *worktree_git_path(const struct worktree *wt, const char *fmt, ...)
 	struct strbuf *pathname = get_pathname();
 	va_list args;
 	va_start(args, fmt);
-	do_git_path(wt, pathname, fmt, args);
+	do_git_path(the_repository, wt, pathname, fmt, args);
 	va_end(args);
 	return pathname->buf;
 }
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 11/20] path: worktree_git_path() should not use file relocation
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (9 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 10/20] path: convert do_git_path " Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 12/20] path: add repo_git_path and strbuf_repo_git_path Brandon Williams
                         ` (10 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

git_path is a convenience function that usually produces a string
$GIT_DIR/<path>.  Since v2.5.0-rc0~143^2~35 (git_path(): be aware of
file relocation in $GIT_DIR, 2014-11-30), as a side benefit callers
get support for path relocation variables like $GIT_OBJECT_DIRECTORY:

- git_path("index") is $GIT_INDEX_FILE when set
- git_path("info/grafts") is $GIT_GRAFTS_FILE when set
- git_path("objects/<foo>") is $GIT_OBJECT_DIRECTORY/<foo> when set
- git_path("hooks/<foo>") is <foo> under core.hookspath when set
- git_path("refs/<foo>") etc (see path.c::common_list) is relative
  to $GIT_COMMON_DIR instead of $GIT_DIR

worktree_git_path, by comparison, is designed to resolve files in a
specific worktree's git dir.  Unfortunately, it shares code with
git_path and performs the same relocation.  The result is that paths
that are meant to be relative to the specified worktree's git dir end
up replaced by paths from environment variables within the current git
dir.

Luckily, no current callers pass such arguments.  The relocation was
noticed when testing the result of merging two patches under review,
one of which introduces a caller:

* The first patch made git prune check the index file in each
  worktree's git dir (using worktree_git_path(wt, "index")) for
  objects not to prune.  This would trigger the unwanted relocation
  when GIT_INDEX_FILE is set, causing objects reachable from the
  index to be pruned.

* The second patch simplified the relocation logic for index,
  info/grafts, objects, and hooks to happen unconditionally instead of
  based on whether environment or configuration variables are set.
  This caused the relocation to trigger even when GIT_INDEX_FILE is
  not set.

[jn: rewrote commit message; skipping all relocation instead of just
 GIT_INDEX_FILE]

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 path.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/path.c b/path.c
index 76a872297..2bdd0044f 100644
--- a/path.c
+++ b/path.c
@@ -411,7 +411,8 @@ static void do_git_path(const struct repository *repo,
 		strbuf_addch(buf, '/');
 	gitdir_len = buf->len;
 	strbuf_vaddf(buf, fmt, args);
-	adjust_git_path(repo, buf, gitdir_len);
+	if (!wt)
+		adjust_git_path(repo, buf, gitdir_len);
 	strbuf_cleanup_path(buf);
 }
 
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 12/20] path: add repo_git_path and strbuf_repo_git_path
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (10 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 11/20] path: worktree_git_path() should not use file relocation Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path Brandon Williams
                         ` (9 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce 'repo_git_path' and 'strbuf_repo_git_path' which take a
repository struct and constructs a path into the repository's git
directory.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 21 +++++++++++++++++++++
 path.h |  8 ++++++++
 2 files changed, 29 insertions(+)

diff --git a/path.c b/path.c
index 2bdd0044f..ffc0f10fd 100644
--- a/path.c
+++ b/path.c
@@ -416,6 +416,27 @@ static void do_git_path(const struct repository *repo,
 	strbuf_cleanup_path(buf);
 }
 
+char *repo_git_path(const struct repository *repo,
+		    const char *fmt, ...)
+{
+	struct strbuf path = STRBUF_INIT;
+	va_list args;
+	va_start(args, fmt);
+	do_git_path(repo, NULL, &path, fmt, args);
+	va_end(args);
+	return strbuf_detach(&path, NULL);
+}
+
+void strbuf_repo_git_path(struct strbuf *sb,
+			  const struct repository *repo,
+			  const char *fmt, ...)
+{
+	va_list args;
+	va_start(args, fmt);
+	do_git_path(repo, NULL, sb, fmt, args);
+	va_end(args);
+}
+
 char *git_path_buf(struct strbuf *buf, const char *fmt, ...)
 {
 	va_list args;
diff --git a/path.h b/path.h
index 568d63be5..c779c4aa2 100644
--- a/path.h
+++ b/path.h
@@ -35,6 +35,14 @@ extern char *mkpathdup(const char *fmt, ...)
 extern char *git_pathdup_submodule(const char *path, const char *fmt, ...)
 	__attribute__((format (printf, 2, 3)));
 
+extern char *repo_git_path(const struct repository *repo,
+			   const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern void strbuf_repo_git_path(struct strbuf *sb,
+				 const struct repository *repo,
+				 const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+
 extern void report_linked_checkout_garbage(void);
 
 /*
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (11 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 12/20] path: add repo_git_path and strbuf_repo_git_path Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 14/20] config: read config from a repository object Brandon Williams
                         ` (8 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce 'repo_worktree_path' and 'strbuf_repo_worktree_path' which
take a repository struct and constructs a path relative to the
repository's worktree.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 path.c | 41 +++++++++++++++++++++++++++++++++++++++++
 path.h |  8 ++++++++
 2 files changed, 49 insertions(+)

diff --git a/path.c b/path.c
index ffc0f10fd..e485f9f93 100644
--- a/path.c
+++ b/path.c
@@ -506,6 +506,47 @@ const char *worktree_git_path(const struct worktree *wt, const char *fmt, ...)
 	return pathname->buf;
 }
 
+static void do_worktree_path(const struct repository *repo,
+			     struct strbuf *buf,
+			     const char *fmt, va_list args)
+{
+	strbuf_addstr(buf, repo->worktree);
+	if(buf->len && !is_dir_sep(buf->buf[buf->len - 1]))
+		strbuf_addch(buf, '/');
+
+	strbuf_vaddf(buf, fmt, args);
+	strbuf_cleanup_path(buf);
+}
+
+char *repo_worktree_path(const struct repository *repo, const char *fmt, ...)
+{
+	struct strbuf path = STRBUF_INIT;
+	va_list args;
+
+	if (!repo->worktree)
+		return NULL;
+
+	va_start(args, fmt);
+	do_worktree_path(repo, &path, fmt, args);
+	va_end(args);
+
+	return strbuf_detach(&path, NULL);
+}
+
+void strbuf_repo_worktree_path(struct strbuf *sb,
+			       const struct repository *repo,
+			       const char *fmt, ...)
+{
+	va_list args;
+
+	if (!repo->worktree)
+		return;
+
+	va_start(args, fmt);
+	do_worktree_path(repo, sb, fmt, args);
+	va_end(args);
+}
+
 /* Returns 0 on success, negative on failure. */
 static int do_submodule_path(struct strbuf *buf, const char *path,
 			     const char *fmt, va_list args)
diff --git a/path.h b/path.h
index c779c4aa2..9541620c7 100644
--- a/path.h
+++ b/path.h
@@ -43,6 +43,14 @@ extern void strbuf_repo_git_path(struct strbuf *sb,
 				 const char *fmt, ...)
 	__attribute__((format (printf, 3, 4)));
 
+extern char *repo_worktree_path(const struct repository *repo,
+				const char *fmt, ...)
+	__attribute__((format (printf, 2, 3)));
+extern void strbuf_repo_worktree_path(struct strbuf *sb,
+				      const struct repository *repo,
+				      const char *fmt, ...)
+	__attribute__((format (printf, 3, 4)));
+
 extern void report_linked_checkout_garbage(void);
 
 /*
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 14/20] config: read config from a repository object
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (12 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 15/20] repository: add index_state to struct repo Brandon Williams
                         ` (7 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Teach the config machinery to read config information from a repository
object.  This involves storing a 'struct config_set' inside the
repository object and adding a number of functions (repo_config*) to be
able to query a repository's config.

The current config API enables lazy-loading of the config.  This means
that when 'git_config_get_int()' is called, if the_config_set hasn't
been populated yet, then it will be populated and properly initialized by
reading the necessary config files (system wide .gitconfig, user's home
.gitconfig, and the repository's config).  To maintain this paradigm,
the new API to read from a repository object's config will also perform
this lazy-initialization.

Since both APIs (git_config_get* and repo_config_get*) have the same
semantics we can migrate the default config to be stored within
'the_repository' and just have the 'git_config_get*' family of functions
redirect to the 'repo_config_get*' functions.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 config.c     | 216 +++++++++++++++++++++++++++++++++++++++--------------------
 config.h     |  24 +++++++
 repository.c |   7 ++
 repository.h |  10 +++
 4 files changed, 183 insertions(+), 74 deletions(-)

diff --git a/config.c b/config.c
index 6f0f8b30f..be1c640a4 100644
--- a/config.c
+++ b/config.c
@@ -7,6 +7,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "lockfile.h"
 #include "exec_cmd.h"
 #include "strbuf.h"
@@ -72,13 +73,6 @@ static int core_compression_seen;
 static int pack_compression_seen;
 static int zlib_compression_seen;
 
-/*
- * Default config_set that contains key-value pairs from the usual set of config
- * config files (i.e repo specific .git/config, user wide ~/.gitconfig, XDG
- * config file and the global /etc/gitconfig)
- */
-static struct config_set the_config_set;
-
 static int config_file_fgetc(struct config_source *conf)
 {
 	return getc_unlocked(conf->u.file);
@@ -1605,31 +1599,6 @@ int config_with_options(config_fn_t fn, void *data,
 	return do_git_config_sequence(opts, fn, data);
 }
 
-static void git_config_raw(config_fn_t fn, void *data)
-{
-	struct config_options opts = {0};
-
-	opts.respect_includes = 1;
-	if (have_git_dir()) {
-		opts.commondir = get_git_common_dir();
-		opts.git_dir = get_git_dir();
-	}
-
-	if (config_with_options(fn, data, NULL, &opts) < 0)
-		/*
-		 * config_with_options() normally returns only
-		 * zero, as most errors are fatal, and
-		 * non-fatal potential errors are guarded by "if"
-		 * statements that are entered only when no error is
-		 * possible.
-		 *
-		 * If we ever encounter a non-fatal error, it means
-		 * something went really wrong and we should stop
-		 * immediately.
-		 */
-		die(_("unknown error occurred while reading the configuration files"));
-}
-
 static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 {
 	int i, value_index;
@@ -1683,14 +1652,6 @@ void read_early_config(config_fn_t cb, void *data)
 	strbuf_release(&gitdir);
 }
 
-static void git_config_check_init(void);
-
-void git_config(config_fn_t fn, void *data)
-{
-	git_config_check_init();
-	configset_iter(&the_config_set, fn, data);
-}
-
 static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
 {
 	struct config_set_element k;
@@ -1900,87 +1861,194 @@ int git_configset_get_pathname(struct config_set *cs, const char *key, const cha
 		return 1;
 }
 
-static void git_config_check_init(void)
+/* Functions use to read configuration from a repository */
+static void repo_read_config(struct repository *repo)
 {
-	if (the_config_set.hash_initialized)
+	struct config_options opts;
+
+	opts.respect_includes = 1;
+	opts.commondir = repo->commondir;
+	opts.git_dir = repo->gitdir;
+
+	if (!repo->config)
+		repo->config = xcalloc(1, sizeof(struct config_set));
+	else
+		git_configset_clear(repo->config);
+
+	git_configset_init(repo->config);
+
+	if (config_with_options(config_set_callback, repo->config, NULL, &opts) < 0)
+		/*
+		 * config_with_options() normally returns only
+		 * zero, as most errors are fatal, and
+		 * non-fatal potential errors are guarded by "if"
+		 * statements that are entered only when no error is
+		 * possible.
+		 *
+		 * If we ever encounter a non-fatal error, it means
+		 * something went really wrong and we should stop
+		 * immediately.
+		 */
+		die(_("unknown error occurred while reading the configuration files"));
+}
+
+static void git_config_check_init(struct repository *repo)
+{
+	if (repo->config && repo->config->hash_initialized)
 		return;
-	git_configset_init(&the_config_set);
-	git_config_raw(config_set_callback, &the_config_set);
+	repo_read_config(repo);
 }
 
-void git_config_clear(void)
+static void repo_config_clear(struct repository *repo)
 {
-	if (!the_config_set.hash_initialized)
+	if (!repo->config || !repo->config->hash_initialized)
 		return;
-	git_configset_clear(&the_config_set);
+	git_configset_clear(repo->config);
 }
 
-int git_config_get_value(const char *key, const char **value)
+void repo_config(struct repository *repo, config_fn_t fn, void *data)
 {
-	git_config_check_init();
-	return git_configset_get_value(&the_config_set, key, value);
+	git_config_check_init(repo);
+	configset_iter(repo->config, fn, data);
 }
 
-const struct string_list *git_config_get_value_multi(const char *key)
+int repo_config_get_value(struct repository *repo,
+			  const char *key, const char **value)
 {
-	git_config_check_init();
-	return git_configset_get_value_multi(&the_config_set, key);
+	git_config_check_init(repo);
+	return git_configset_get_value(repo->config, key, value);
 }
 
-int git_config_get_string_const(const char *key, const char **dest)
+const struct string_list *repo_config_get_value_multi(struct repository *repo,
+						      const char *key)
+{
+	git_config_check_init(repo);
+	return git_configset_get_value_multi(repo->config, key);
+}
+
+int repo_config_get_string_const(struct repository *repo,
+				 const char *key, const char **dest)
+{
+	int ret;
+	git_config_check_init(repo);
+	ret = git_configset_get_string_const(repo->config, key, dest);
+	if (ret < 0)
+		git_die_config(key, NULL);
+	return ret;
+}
+
+int repo_config_get_string(struct repository *repo,
+			   const char *key, char **dest)
+{
+	git_config_check_init(repo);
+	return repo_config_get_string_const(repo, key, (const char **)dest);
+}
+
+int repo_config_get_int(struct repository *repo,
+			const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_int(repo->config, key, dest);
+}
+
+int repo_config_get_ulong(struct repository *repo,
+			  const char *key, unsigned long *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_ulong(repo->config, key, dest);
+}
+
+int repo_config_get_bool(struct repository *repo,
+			 const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_bool(repo->config, key, dest);
+}
+
+int repo_config_get_bool_or_int(struct repository *repo,
+				const char *key, int *is_bool, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_bool_or_int(repo->config, key, is_bool, dest);
+}
+
+int repo_config_get_maybe_bool(struct repository *repo,
+			       const char *key, int *dest)
+{
+	git_config_check_init(repo);
+	return git_configset_get_maybe_bool(repo->config, key, dest);
+}
+
+int repo_config_get_pathname(struct repository *repo,
+			     const char *key, const char **dest)
 {
 	int ret;
-	git_config_check_init();
-	ret = git_configset_get_string_const(&the_config_set, key, dest);
+	git_config_check_init(repo);
+	ret = git_configset_get_pathname(repo->config, key, dest);
 	if (ret < 0)
 		git_die_config(key, NULL);
 	return ret;
 }
 
+/* Functions used historically to read configuration from 'the_repository' */
+void git_config(config_fn_t fn, void *data)
+{
+	repo_config(the_repository, fn, data);
+}
+
+void git_config_clear(void)
+{
+	repo_config_clear(the_repository);
+}
+
+int git_config_get_value(const char *key, const char **value)
+{
+	return repo_config_get_value(the_repository, key, value);
+}
+
+const struct string_list *git_config_get_value_multi(const char *key)
+{
+	return repo_config_get_value_multi(the_repository, key);
+}
+
+int git_config_get_string_const(const char *key, const char **dest)
+{
+	return repo_config_get_string_const(the_repository, key, dest);
+}
+
 int git_config_get_string(const char *key, char **dest)
 {
-	git_config_check_init();
-	return git_config_get_string_const(key, (const char **)dest);
+	return repo_config_get_string(the_repository, key, dest);
 }
 
 int git_config_get_int(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_int(&the_config_set, key, dest);
+	return repo_config_get_int(the_repository, key, dest);
 }
 
 int git_config_get_ulong(const char *key, unsigned long *dest)
 {
-	git_config_check_init();
-	return git_configset_get_ulong(&the_config_set, key, dest);
+	return repo_config_get_ulong(the_repository, key, dest);
 }
 
 int git_config_get_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool(&the_config_set, key, dest);
+	return repo_config_get_bool(the_repository, key, dest);
 }
 
 int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_bool_or_int(&the_config_set, key, is_bool, dest);
+	return repo_config_get_bool_or_int(the_repository, key, is_bool, dest);
 }
 
 int git_config_get_maybe_bool(const char *key, int *dest)
 {
-	git_config_check_init();
-	return git_configset_get_maybe_bool(&the_config_set, key, dest);
+	return repo_config_get_maybe_bool(the_repository, key, dest);
 }
 
 int git_config_get_pathname(const char *key, const char **dest)
 {
-	int ret;
-	git_config_check_init();
-	ret = git_configset_get_pathname(&the_config_set, key, dest);
-	if (ret < 0)
-		git_die_config(key, NULL);
-	return ret;
+	return repo_config_get_pathname(the_repository, key, dest);
 }
 
 int git_config_get_expiry(const char *key, const char **output)
diff --git a/config.h b/config.h
index 9e038cce2..0352da117 100644
--- a/config.h
+++ b/config.h
@@ -163,6 +163,30 @@ extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key,
 extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
 extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
 
+/* Functions for reading a repository's config */
+struct repository;
+extern void repo_config(struct repository *repo, config_fn_t fn, void *data);
+extern int repo_config_get_value(struct repository *repo,
+				 const char *key, const char **value);
+extern const struct string_list *repo_config_get_value_multi(struct repository *repo,
+							     const char *key);
+extern int repo_config_get_string_const(struct repository *repo,
+					const char *key, const char **dest);
+extern int repo_config_get_string(struct repository *repo,
+				  const char *key, char **dest);
+extern int repo_config_get_int(struct repository *repo,
+			       const char *key, int *dest);
+extern int repo_config_get_ulong(struct repository *repo,
+				 const char *key, unsigned long *dest);
+extern int repo_config_get_bool(struct repository *repo,
+				const char *key, int *dest);
+extern int repo_config_get_bool_or_int(struct repository *repo,
+				       const char *key, int *is_bool, int *dest);
+extern int repo_config_get_maybe_bool(struct repository *repo,
+				      const char *key, int *dest);
+extern int repo_config_get_pathname(struct repository *repo,
+				    const char *key, const char **dest);
+
 extern int git_config_get_value(const char *key, const char **value);
 extern const struct string_list *git_config_get_value_multi(const char *key);
 extern void git_config_clear(void);
diff --git a/repository.c b/repository.c
index cf440405a..686a964ad 100644
--- a/repository.c
+++ b/repository.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "repository.h"
+#include "config.h"
 
 /* The main repository */
 static struct repository the_repo;
@@ -156,4 +157,10 @@ void repo_clear(struct repository *repo)
 	repo->index_file = NULL;
 	free(repo->worktree);
 	repo->worktree = NULL;
+
+	if (repo->config) {
+		git_configset_clear(repo->config);
+		free(repo->config);
+		repo->config = NULL;
+	}
 }
diff --git a/repository.h b/repository.h
index 0a1db9633..8ae5e8653 100644
--- a/repository.h
+++ b/repository.h
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+struct config_set;
+
 struct repository {
 	/* Environment */
 	/*
@@ -39,6 +41,14 @@ struct repository {
 	 */
 	char *worktree;
 
+	/* Subsystems */
+	/*
+	 * Repository's config which contains key-value pairs from the usual
+	 * set of config files (i.e. repo specific .git/config, user wide
+	 * ~/.gitconfig, XDG config file and the global /etc/gitconfig)
+	 */
+	struct config_set *config;
+
 	/* Configurations */
 	/*
 	 * Bit used during initialization to indicate if repository state (like
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 15/20] repository: add index_state to struct repo
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (13 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 14/20] config: read config from a repository object Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 20:16         ` Junio C Hamano
  2017-06-22 18:43       ` [PATCH v4 16/20] submodule-config: store the_submodule_cache in the_repository Brandon Williams
                         ` (6 subsequent siblings)
  21 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c | 16 ++++++++++++++++
 repository.h |  9 +++++++++
 2 files changed, 25 insertions(+)

diff --git a/repository.c b/repository.c
index 686a964ad..6f6f4d91e 100644
--- a/repository.c
+++ b/repository.c
@@ -163,4 +163,20 @@ void repo_clear(struct repository *repo)
 		free(repo->config);
 		repo->config = NULL;
 	}
+
+	if (repo->index) {
+		discard_index(repo->index);
+		free(repo->index);
+		repo->index = NULL;
+	}
+}
+
+int repo_read_index(struct repository *repo)
+{
+	if (!repo->index)
+		repo->index = xcalloc(1, sizeof(*repo->index));
+	else
+		discard_index(repo->index);
+
+	return read_index_from(repo->index, repo->index_file);
 }
diff --git a/repository.h b/repository.h
index 8ae5e8653..3a41568aa 100644
--- a/repository.h
+++ b/repository.h
@@ -2,6 +2,7 @@
 #define REPOSITORY_H
 
 struct config_set;
+struct index_state;
 
 struct repository {
 	/* Environment */
@@ -49,6 +50,12 @@ struct repository {
 	 */
 	struct config_set *config;
 
+	/*
+	 * Repository's in-memory index.
+	 * 'repo_read_index()' can be used to populate 'index'.
+	 */
+	struct index_state *index;
+
 	/* Configurations */
 	/*
 	 * Bit used during initialization to indicate if repository state (like
@@ -71,4 +78,6 @@ extern void repo_set_worktree(struct repository *repo, const char *path);
 extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
 extern void repo_clear(struct repository *repo);
 
+extern int repo_read_index(struct repository *repo);
+
 #endif /* REPOSITORY_H */
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 16/20] submodule-config: store the_submodule_cache in the_repository
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (14 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 15/20] repository: add index_state to struct repo Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 17/20] submodule: add repo_read_gitmodules Brandon Williams
                         ` (5 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Refactor how 'the_submodule_cache' is handled so that it can be stored
inside of a repository object.  Also migrate 'the_submodule_cache' to be
stored in 'the_repository'.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c       |  6 +++++
 repository.h       |  4 ++++
 submodule-config.c | 70 ++++++++++++++++++++++++++++++++++++++++--------------
 submodule-config.h | 10 ++++++++
 4 files changed, 72 insertions(+), 18 deletions(-)

diff --git a/repository.c b/repository.c
index 6f6f4d91e..358c17517 100644
--- a/repository.c
+++ b/repository.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "submodule-config.h"
 
 /* The main repository */
 static struct repository the_repo;
@@ -164,6 +165,11 @@ void repo_clear(struct repository *repo)
 		repo->config = NULL;
 	}
 
+	if (repo->submodule_cache) {
+		submodule_cache_free(repo->submodule_cache);
+		repo->submodule_cache = NULL;
+	}
+
 	if (repo->index) {
 		discard_index(repo->index);
 		free(repo->index);
diff --git a/repository.h b/repository.h
index 3a41568aa..c40738ceb 100644
--- a/repository.h
+++ b/repository.h
@@ -3,6 +3,7 @@
 
 struct config_set;
 struct index_state;
+struct submodule_cache;
 
 struct repository {
 	/* Environment */
@@ -50,6 +51,9 @@ struct repository {
 	 */
 	struct config_set *config;
 
+	/* Repository's submodule config as defined by '.gitmodules' */
+	struct submodule_cache *submodule_cache;
+
 	/*
 	 * Repository's in-memory index.
 	 * 'repo_read_index()' can be used to populate 'index'.
diff --git a/submodule-config.c b/submodule-config.c
index d8f8d5ea3..37cfcceb9 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -15,6 +16,7 @@
 struct submodule_cache {
 	struct hashmap for_path;
 	struct hashmap for_name;
+	unsigned initialized:1;
 };
 
 /*
@@ -31,9 +33,6 @@ enum lookup_type {
 	lookup_path
 };
 
-static struct submodule_cache the_submodule_cache;
-static int is_cache_init;
-
 static int config_path_cmp(const struct submodule_entry *a,
 			   const struct submodule_entry *b,
 			   const void *unused)
@@ -50,10 +49,16 @@ static int config_name_cmp(const struct submodule_entry *a,
 	       hashcmp(a->config->gitmodules_sha1, b->config->gitmodules_sha1);
 }
 
-static void cache_init(struct submodule_cache *cache)
+static struct submodule_cache *submodule_cache_alloc(void)
+{
+	return xcalloc(1, sizeof(struct submodule_cache));
+}
+
+static void submodule_cache_init(struct submodule_cache *cache)
 {
 	hashmap_init(&cache->for_path, (hashmap_cmp_fn) config_path_cmp, 0);
 	hashmap_init(&cache->for_name, (hashmap_cmp_fn) config_name_cmp, 0);
+	cache->initialized = 1;
 }
 
 static void free_one_config(struct submodule_entry *entry)
@@ -65,11 +70,14 @@ static void free_one_config(struct submodule_entry *entry)
 	free(entry->config);
 }
 
-static void cache_free(struct submodule_cache *cache)
+static void submodule_cache_clear(struct submodule_cache *cache)
 {
 	struct hashmap_iter iter;
 	struct submodule_entry *entry;
 
+	if (!cache->initialized)
+		return;
+
 	/*
 	 * We iterate over the name hash here to be symmetric with the
 	 * allocation of struct submodule entries. Each is allocated by
@@ -81,6 +89,13 @@ static void cache_free(struct submodule_cache *cache)
 
 	hashmap_free(&cache->for_path, 1);
 	hashmap_free(&cache->for_name, 1);
+	cache->initialized = 0;
+}
+
+void submodule_cache_free(struct submodule_cache *cache)
+{
+	submodule_cache_clear(cache);
+	free(cache);
 }
 
 static unsigned int hash_sha1_string(const unsigned char *sha1,
@@ -494,43 +509,62 @@ static const struct submodule *config_from(struct submodule_cache *cache,
 	return submodule;
 }
 
-static void ensure_cache_init(void)
+static void submodule_cache_check_init(struct repository *repo)
 {
-	if (is_cache_init)
+	if (repo->submodule_cache && repo->submodule_cache->initialized)
 		return;
 
-	cache_init(&the_submodule_cache);
-	is_cache_init = 1;
+	if (!repo->submodule_cache)
+		repo->submodule_cache = submodule_cache_alloc();
+
+	submodule_cache_init(repo->submodule_cache);
 }
 
-int parse_submodule_config_option(const char *var, const char *value)
+int submodule_config_option(struct repository *repo,
+			    const char *var, const char *value)
 {
 	struct parse_config_parameter parameter;
-	parameter.cache = &the_submodule_cache;
+
+	submodule_cache_check_init(repo);
+
+	parameter.cache = repo->submodule_cache;
 	parameter.treeish_name = NULL;
 	parameter.gitmodules_sha1 = null_sha1;
 	parameter.overwrite = 1;
 
-	ensure_cache_init();
 	return parse_config(var, value, &parameter);
 }
 
+int parse_submodule_config_option(const char *var, const char *value)
+{
+	return submodule_config_option(the_repository, var, value);
+}
+
 const struct submodule *submodule_from_name(const unsigned char *treeish_name,
 		const char *name)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, name, lookup_name);
+	submodule_cache_check_init(the_repository);
+	return config_from(the_repository->submodule_cache, treeish_name, name, lookup_name);
 }
 
 const struct submodule *submodule_from_path(const unsigned char *treeish_name,
 		const char *path)
 {
-	ensure_cache_init();
-	return config_from(&the_submodule_cache, treeish_name, path, lookup_path);
+	submodule_cache_check_init(the_repository);
+	return config_from(the_repository->submodule_cache, treeish_name, path, lookup_path);
+}
+
+const struct submodule *submodule_from_cache(struct repository *repo,
+					     const unsigned char *treeish_name,
+					     const char *key)
+{
+	submodule_cache_check_init(repo);
+	return config_from(repo->submodule_cache, treeish_name,
+			   key, lookup_path);
 }
 
 void submodule_free(void)
 {
-	cache_free(&the_submodule_cache);
-	is_cache_init = 0;
+	if (the_repository->submodule_cache)
+		submodule_cache_clear(the_repository->submodule_cache);
 }
diff --git a/submodule-config.h b/submodule-config.h
index d434ecdb4..bc45a25e8 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -22,14 +22,24 @@ struct submodule {
 	int recommend_shallow;
 };
 
+struct submodule_cache;
+struct repository;
+
+extern void submodule_cache_free(struct submodule_cache *cache);
+
 extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
 extern int parse_submodule_config_option(const char *var, const char *value);
+extern int submodule_config_option(struct repository *repo,
+				   const char *var, const char *value);
 extern const struct submodule *submodule_from_name(
 		const unsigned char *commit_or_tree, const char *name);
 extern const struct submodule *submodule_from_path(
 		const unsigned char *commit_or_tree, const char *path);
+extern const struct submodule *submodule_from_cache(struct repository *repo,
+						    const unsigned char *treeish_name,
+						    const char *key);
 extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,
 				      unsigned char *gitmodules_sha1,
 				      struct strbuf *rev);
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 17/20] submodule: add repo_read_gitmodules
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (15 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 16/20] submodule-config: store the_submodule_cache in the_repository Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 18/20] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
                         ` (4 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Teach the repo object to be able to populate the submodule_cache by
reading the repository's gitmodules file.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 submodule.c | 15 +++++++++++++++
 submodule.h |  2 ++
 2 files changed, 17 insertions(+)

diff --git a/submodule.c b/submodule.c
index da0b80549..d0b894772 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "submodule-config.h"
 #include "submodule.h"
@@ -255,6 +256,20 @@ void gitmodules_config(void)
 	}
 }
 
+static int gitmodules_cb(const char *var, const char *value, void *data)
+{
+	struct repository *repo = data;
+	return submodule_config_option(repo, var, value);
+}
+
+void repo_read_gitmodules(struct repository *repo)
+{
+	char *gitmodules_path = repo_worktree_path(repo, ".gitmodules");
+
+	git_config_from_file(gitmodules_cb, gitmodules_path, repo);
+	free(gitmodules_path);
+}
+
 void gitmodules_config_sha1(const unsigned char *commit_sha1)
 {
 	struct strbuf rev = STRBUF_INIT;
diff --git a/submodule.h b/submodule.h
index cbe5c1726..8a3771ec6 100644
--- a/submodule.h
+++ b/submodule.h
@@ -1,6 +1,7 @@
 #ifndef SUBMODULE_H
 #define SUBMODULE_H
 
+struct repository;
 struct diff_options;
 struct argv_array;
 struct oid_array;
@@ -46,6 +47,7 @@ int option_parse_recurse_submodules_worktree_updater(const struct option *opt,
 						     const char *arg, int unset);
 void load_submodule_cache(void);
 extern void gitmodules_config(void);
+extern void repo_read_gitmodules(struct repository *repo);
 extern void gitmodules_config_sha1(const unsigned char *commit_sha1);
 extern int is_submodule_initialized(const char *path);
 /*
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 18/20] submodule: convert is_submodule_initialized to work on a repository
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (16 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 17/20] submodule: add repo_read_gitmodules Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 19/20] repository: enable initialization of submodules Brandon Williams
                         ` (3 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert 'is_submodule_initialized()' to take a repository object and
while we're at it, lets rename the function to 'is_submodule_active()'
and remove the NEEDSWORK comment.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/grep.c              |  3 ++-
 builtin/submodule--helper.c |  9 +++++----
 submodule.c                 | 20 ++++++++------------
 submodule.h                 |  2 +-
 4 files changed, 16 insertions(+), 18 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index f61a9d938..e3ba1d98e 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -4,6 +4,7 @@
  * Copyright (c) 2006 Junio C Hamano
  */
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "blob.h"
 #include "tree.h"
@@ -643,7 +644,7 @@ static int grep_submodule_launch(struct grep_opt *opt,
 static int grep_submodule(struct grep_opt *opt, const struct object_id *oid,
 			  const char *filename, const char *path)
 {
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(the_repository, path))
 		return 0;
 	if (!is_submodule_populated_gently(path, NULL)) {
 		/*
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 8517032b3..e1b06c41d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "repository.h"
 #include "cache.h"
 #include "config.h"
 #include "parse-options.h"
@@ -280,7 +281,7 @@ static void module_list_active(struct module_list *list)
 	for (i = 0; i < list->nr; i++) {
 		const struct cache_entry *ce = list->entries[i];
 
-		if (!is_submodule_initialized(ce->name))
+		if (!is_submodule_active(the_repository, ce->name))
 			continue;
 
 		ALLOC_GROW(active_modules.entries,
@@ -362,7 +363,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_initialized(path)) {
+	if (!is_submodule_active(the_repository, path)) {
 		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
@@ -817,7 +818,7 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 	}
 
 	/* Check if the submodule has been initialized. */
-	if (!is_submodule_initialized(ce->name)) {
+	if (!is_submodule_active(the_repository, ce->name)) {
 		next_submodule_warn_missing(suc, out, displaypath);
 		goto cleanup;
 	}
@@ -1193,7 +1194,7 @@ static int is_active(int argc, const char **argv, const char *prefix)
 
 	gitmodules_config();
 
-	return !is_submodule_initialized(argv[1]);
+	return !is_submodule_active(the_repository, argv[1]);
 }
 
 #define SUPPORT_SUPER_PREFIX (1<<0)
diff --git a/submodule.c b/submodule.c
index d0b894772..b23c25311 100644
--- a/submodule.c
+++ b/submodule.c
@@ -283,21 +283,17 @@ void gitmodules_config_sha1(const unsigned char *commit_sha1)
 }
 
 /*
- * NEEDSWORK: With the addition of different configuration options to determine
- * if a submodule is of interests, the validity of this function's name comes
- * into question.  Once the dust has settled and more concrete terminology is
- * decided upon, come up with a more proper name for this function.  One
- * potential candidate could be 'is_submodule_active()'.
- *
  * Determine if a submodule has been initialized at a given 'path'
  */
-int is_submodule_initialized(const char *path)
+int is_submodule_active(struct repository *repo, const char *path)
 {
 	int ret = 0;
 	char *key = NULL;
 	char *value = NULL;
 	const struct string_list *sl;
-	const struct submodule *module = submodule_from_path(null_sha1, path);
+	const struct submodule *module;
+
+	module = submodule_from_cache(repo, null_sha1, path);
 
 	/* early return if there isn't a path->module mapping */
 	if (!module)
@@ -305,14 +301,14 @@ int is_submodule_initialized(const char *path)
 
 	/* submodule.<name>.active is set */
 	key = xstrfmt("submodule.%s.active", module->name);
-	if (!git_config_get_bool(key, &ret)) {
+	if (!repo_config_get_bool(repo, key, &ret)) {
 		free(key);
 		return ret;
 	}
 	free(key);
 
 	/* submodule.active is set */
-	sl = git_config_get_value_multi("submodule.active");
+	sl = repo_config_get_value_multi(repo, "submodule.active");
 	if (sl) {
 		struct pathspec ps;
 		struct argv_array args = ARGV_ARRAY_INIT;
@@ -332,7 +328,7 @@ int is_submodule_initialized(const char *path)
 
 	/* fallback to checking if the URL is set */
 	key = xstrfmt("submodule.%s.url", module->name);
-	ret = !git_config_get_string(key, &value);
+	ret = !repo_config_get_string(repo, key, &value);
 
 	free(value);
 	free(key);
@@ -1532,7 +1528,7 @@ int submodule_move_head(const char *path,
 	const struct submodule *sub;
 	int *error_code_ptr, error_code;
 
-	if (!is_submodule_initialized(path))
+	if (!is_submodule_active(the_repository, path))
 		return 0;
 
 	if (flags & SUBMODULE_MOVE_HEAD_FORCE)
diff --git a/submodule.h b/submodule.h
index 8a3771ec6..623ce6ad7 100644
--- a/submodule.h
+++ b/submodule.h
@@ -49,7 +49,7 @@ void load_submodule_cache(void);
 extern void gitmodules_config(void);
 extern void repo_read_gitmodules(struct repository *repo);
 extern void gitmodules_config_sha1(const unsigned char *commit_sha1);
-extern int is_submodule_initialized(const char *path);
+extern int is_submodule_active(struct repository *repo, const char *path);
 /*
  * Determine if a submodule has been populated at a given 'path' by checking if
  * the <path>/.git resolves to a valid git repository.
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 19/20] repository: enable initialization of submodules
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (17 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 18/20] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 18:43       ` [PATCH v4 20/20] ls-files: use repository object Brandon Williams
                         ` (2 subsequent siblings)
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Introduce 'repo_submodule_init()' which performs initialization of a
'struct repository' as a submodule of another 'struct repository'.

The resulting submodule 'struct repository' can be in one of three states:

  1. The submodule is initialized and has a worktree.

  2. The submodule is initialized but does not have a worktree.  This
     would occur when the submodule's gitdir is present in the
     superproject's 'gitdir/modules/' directory yet the submodule has not
     been checked out in superproject's worktree.

  3. The submodule remains uninitialized due to an error in the
     initialization process or there is no matching submodule at the
     provided path in the superproject.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 repository.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 repository.h | 10 ++++++++++
 2 files changed, 64 insertions(+)

diff --git a/repository.c b/repository.c
index 358c17517..edca90740 100644
--- a/repository.c
+++ b/repository.c
@@ -144,6 +144,58 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 	return -1;
 }
 
+/*
+ * Initialize 'submodule' as the submodule given by 'path' in parent repository
+ * 'superproject'.
+ * Return 0 upon success and a non-zero value upon failure.
+ */
+int repo_submodule_init(struct repository *submodule,
+			struct repository *superproject,
+			const char *path)
+{
+	const struct submodule *sub;
+	struct strbuf gitdir = STRBUF_INIT;
+	struct strbuf worktree = STRBUF_INIT;
+	int ret = 0;
+
+	sub = submodule_from_cache(superproject, null_sha1, path);
+	if (!sub) {
+		ret = -1;
+		goto out;
+	}
+
+	strbuf_repo_worktree_path(&gitdir, superproject, "%s/.git", path);
+	strbuf_repo_worktree_path(&worktree, superproject, "%s", path);
+
+	if (repo_init(submodule, gitdir.buf, worktree.buf)) {
+		/*
+		 * If initilization fails then it may be due to the submodule
+		 * not being populated in the superproject's worktree.  Instead
+		 * we can try to initilize the submodule by finding it's gitdir
+		 * in the superproject's 'modules' directory.  In this case the
+		 * submodule would not have a worktree.
+		 */
+		strbuf_reset(&gitdir);
+		strbuf_repo_git_path(&gitdir, superproject,
+				     "modules/%s", sub->name);
+
+		if (repo_init(submodule, gitdir.buf, NULL)) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+	submodule->submodule_prefix = xstrfmt("%s%s/",
+					      superproject->submodule_prefix ?
+					      superproject->submodule_prefix :
+					      "", path);
+
+out:
+	strbuf_release(&gitdir);
+	strbuf_release(&worktree);
+	return ret;
+}
+
 void repo_clear(struct repository *repo)
 {
 	free(repo->gitdir);
@@ -158,6 +210,8 @@ void repo_clear(struct repository *repo)
 	repo->index_file = NULL;
 	free(repo->worktree);
 	repo->worktree = NULL;
+	free(repo->submodule_prefix);
+	repo->submodule_prefix = NULL;
 
 	if (repo->config) {
 		git_configset_clear(repo->config);
diff --git a/repository.h b/repository.h
index c40738ceb..417787f3e 100644
--- a/repository.h
+++ b/repository.h
@@ -43,6 +43,13 @@ struct repository {
 	 */
 	char *worktree;
 
+	/*
+	 * Path from the root of the top-level superproject down to this
+	 * repository.  This is only non-NULL if the repository is initialized
+	 * as a submodule of another repository.
+	 */
+	char *submodule_prefix;
+
 	/* Subsystems */
 	/*
 	 * Repository's config which contains key-value pairs from the usual
@@ -80,6 +87,9 @@ extern struct repository *the_repository;
 extern void repo_set_gitdir(struct repository *repo, const char *path);
 extern void repo_set_worktree(struct repository *repo, const char *path);
 extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
+extern int repo_submodule_init(struct repository *submodule,
+			       struct repository *superproject,
+			       const char *path);
 extern void repo_clear(struct repository *repo);
 
 extern int repo_read_index(struct repository *repo);
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* [PATCH v4 20/20] ls-files: use repository object
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (18 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 19/20] repository: enable initialization of submodules Brandon Williams
@ 2017-06-22 18:43       ` Brandon Williams
  2017-06-22 19:42       ` [PATCH v4 00/20] " Stefan Beller
  2017-06-23 16:44       ` Jeff Hostetler
  21 siblings, 0 replies; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 18:43 UTC (permalink / raw)
  To: git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, git, avarab, jonathantanmy,
	Brandon Williams

Convert ls-files to use a repository struct and recurse submodules
inprocess.

Signed-off-by: Brandon Williams <bmwill@google.com>
---
 builtin/ls-files.c                     | 192 ++++++++++++++-------------------
 git.c                                  |   2 +-
 t/t3007-ls-files-recurse-submodules.sh |  39 +++++++
 3 files changed, 118 insertions(+), 115 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index b12d0bb61..b8514a002 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -5,7 +5,9 @@
  *
  * Copyright (C) Linus Torvalds, 2005
  */
+#define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "quote.h"
 #include "dir.h"
@@ -32,10 +34,8 @@ static int line_terminator = '\n';
 static int debug_mode;
 static int show_eol;
 static int recurse_submodules;
-static struct argv_array submodule_options = ARGV_ARRAY_INIT;
 
 static const char *prefix;
-static const char *super_prefix;
 static int max_prefix_len;
 static int prefix_len;
 static struct pathspec pathspec;
@@ -73,25 +73,12 @@ static void write_eolinfo(const struct index_state *istate,
 
 static void write_name(const char *name)
 {
-	/*
-	 * Prepend the super_prefix to name to construct the full_name to be
-	 * written.
-	 */
-	struct strbuf full_name = STRBUF_INIT;
-	if (super_prefix) {
-		strbuf_addstr(&full_name, super_prefix);
-		strbuf_addstr(&full_name, name);
-		name = full_name.buf;
-	}
-
 	/*
 	 * With "--full-name", prefix_len=0; this caller needs to pass
 	 * an empty string in that case (a NULL is good for "").
 	 */
 	write_name_quoted_relative(name, prefix_len ? prefix : NULL,
 				   stdout, line_terminator);
-
-	strbuf_release(&full_name);
 }
 
 static const char *get_tag(const struct cache_entry *ce, const char *tag)
@@ -210,83 +197,38 @@ static void show_killed_files(const struct index_state *istate,
 	}
 }
 
-/*
- * Compile an argv_array with all of the options supported by --recurse_submodules
- */
-static void compile_submodule_options(const char **argv,
-				      const struct dir_struct *dir,
-				      int show_tag)
-{
-	if (line_terminator == '\0')
-		argv_array_push(&submodule_options, "-z");
-	if (show_tag)
-		argv_array_push(&submodule_options, "-t");
-	if (show_valid_bit)
-		argv_array_push(&submodule_options, "-v");
-	if (show_cached)
-		argv_array_push(&submodule_options, "--cached");
-	if (show_eol)
-		argv_array_push(&submodule_options, "--eol");
-	if (debug_mode)
-		argv_array_push(&submodule_options, "--debug");
-
-	/* Add Pathspecs */
-	argv_array_push(&submodule_options, "--");
-	for (; *argv; argv++)
-		argv_array_push(&submodule_options, *argv);
-}
+static void show_files(struct repository *repo, struct dir_struct *dir);
 
-/**
- * Recursively call ls-files on a submodule
- */
-static void show_gitlink(const struct cache_entry *ce)
+static void show_submodule(struct repository *superproject,
+			   struct dir_struct *dir, const char *path)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-	char *dir;
-
-	prepare_submodule_repo_env(&cp.env_array);
-	argv_array_push(&cp.env_array, GIT_DIR_ENVIRONMENT);
-
-	if (prefix_len)
-		argv_array_pushf(&cp.env_array, "%s=%s",
-				 GIT_TOPLEVEL_PREFIX_ENVIRONMENT,
-				 prefix);
-	argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
-			 super_prefix ? super_prefix : "",
-			 ce->name);
-	argv_array_push(&cp.args, "ls-files");
-	argv_array_push(&cp.args, "--recurse-submodules");
-
-	/* add supported options */
-	argv_array_pushv(&cp.args, submodule_options.argv);
-
-	cp.git_cmd = 1;
-	dir = mkpathdup("%s/%s", get_git_work_tree(), ce->name);
-	cp.dir = dir;
-	status = run_command(&cp);
-	free(dir);
-	if (status)
-		exit(status);
+	struct repository submodule;
+
+	if (repo_submodule_init(&submodule, superproject, path))
+		return;
+
+	if (repo_read_index(&submodule) < 0)
+		die("index file corrupt");
+
+	repo_read_gitmodules(&submodule);
+
+	show_files(&submodule, dir);
+
+	repo_clear(&submodule);
 }
 
-static void show_ce_entry(const struct index_state *istate,
-			  const char *tag, const struct cache_entry *ce)
+static void show_ce(struct repository *repo, struct dir_struct *dir,
+		    const struct cache_entry *ce, const char *fullname,
+		    const char *tag)
 {
-	struct strbuf name = STRBUF_INIT;
-	int len = max_prefix_len;
-	if (super_prefix)
-		strbuf_addstr(&name, super_prefix);
-	strbuf_addstr(&name, ce->name);
-
-	if (len > ce_namelen(ce))
+	if (max_prefix_len > strlen(fullname))
 		die("git ls-files: internal error - cache entry not superset of prefix");
 
 	if (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&
-	    submodule_path_match(&pathspec, name.buf, ps_matched)) {
-		show_gitlink(ce);
-	} else if (match_pathspec(&pathspec, name.buf, name.len,
-				  len, ps_matched,
+	    is_submodule_active(repo, ce->name)) {
+		show_submodule(repo, dir, ce->name);
+	} else if (match_pathspec(&pathspec, fullname, strlen(fullname),
+				  max_prefix_len, ps_matched,
 				  S_ISDIR(ce->ce_mode) ||
 				  S_ISGITLINK(ce->ce_mode))) {
 		tag = get_tag(ce, tag);
@@ -300,12 +242,10 @@ static void show_ce_entry(const struct index_state *istate,
 			       find_unique_abbrev(ce->oid.hash, abbrev),
 			       ce_stage(ce));
 		}
-		write_eolinfo(istate, ce, ce->name);
-		write_name(ce->name);
+		write_eolinfo(repo->index, ce, fullname);
+		write_name(fullname);
 		print_debug(ce);
 	}
-
-	strbuf_release(&name);
 }
 
 static void show_ru_info(const struct index_state *istate)
@@ -338,59 +278,79 @@ static void show_ru_info(const struct index_state *istate)
 }
 
 static int ce_excluded(struct dir_struct *dir, struct index_state *istate,
-		       const struct cache_entry *ce)
+		       const char *fullname, const struct cache_entry *ce)
 {
 	int dtype = ce_to_dtype(ce);
-	return is_excluded(dir, istate, ce->name, &dtype);
+	return is_excluded(dir, istate, fullname, &dtype);
+}
+
+static void construct_fullname(struct strbuf *out, const struct repository *repo,
+			       const struct cache_entry *ce)
+{
+	strbuf_reset(out);
+	if (repo->submodule_prefix)
+		strbuf_addstr(out, repo->submodule_prefix);
+	strbuf_addstr(out, ce->name);
 }
 
-static void show_files(struct index_state *istate, struct dir_struct *dir)
+static void show_files(struct repository *repo, struct dir_struct *dir)
 {
 	int i;
+	struct strbuf fullname = STRBUF_INIT;
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
 		if (!show_others)
 			dir->flags |= DIR_COLLECT_KILLED_ONLY;
-		fill_directory(dir, istate, &pathspec);
+		fill_directory(dir, repo->index, &pathspec);
 		if (show_others)
-			show_other_files(istate, dir);
+			show_other_files(repo->index, dir);
 		if (show_killed)
-			show_killed_files(istate, dir);
+			show_killed_files(repo->index, dir);
 	}
 	if (show_cached || show_stage) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
-			show_ce_entry(istate, ce_stage(ce) ? tag_unmerged :
-				(ce_skip_worktree(ce) ? tag_skip_worktree : tag_cached), ce);
+			show_ce(repo, dir, ce, fullname.buf,
+				ce_stage(ce) ? tag_unmerged :
+				(ce_skip_worktree(ce) ? tag_skip_worktree :
+				 tag_cached));
 		}
 	}
 	if (show_deleted || show_modified) {
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
+		for (i = 0; i < repo->index->cache_nr; i++) {
+			const struct cache_entry *ce = repo->index->cache[i];
 			struct stat st;
 			int err;
+
+			construct_fullname(&fullname, repo, ce);
+
 			if ((dir->flags & DIR_SHOW_IGNORED) &&
-			    !ce_excluded(dir, istate, ce))
+			    !ce_excluded(dir, repo->index, fullname.buf, ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
 			if (ce_skip_worktree(ce))
 				continue;
-			err = lstat(ce->name, &st);
+			err = lstat(fullname.buf, &st);
 			if (show_deleted && err)
-				show_ce_entry(istate, tag_removed, ce);
-			if (show_modified && ie_modified(istate, ce, &st, 0))
-				show_ce_entry(istate, tag_modified, ce);
+				show_ce(repo, dir, ce, fullname.buf, tag_removed);
+			if (show_modified && ie_modified(repo->index, ce, &st, 0))
+				show_ce(repo, dir, ce, fullname.buf, tag_modified);
 		}
 	}
+
+	strbuf_release(&fullname);
 }
 
 /*
@@ -615,10 +575,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	prefix = cmd_prefix;
 	if (prefix)
 		prefix_len = strlen(prefix);
-	super_prefix = get_super_prefix();
 	git_config(git_default_config, NULL);
 
-	if (read_cache() < 0)
+	if (repo_read_index(the_repository) < 0)
 		die("index file corrupt");
 
 	argc = parse_options(argc, argv, prefix, builtin_ls_files_options,
@@ -652,7 +611,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		setup_work_tree();
 
 	if (recurse_submodules)
-		compile_submodule_options(argv, &dir, show_tag);
+		repo_read_gitmodules(the_repository);
 
 	if (recurse_submodules &&
 	    (show_stage || show_deleted || show_others || show_unmerged ||
@@ -670,7 +629,10 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	/*
 	 * Find common prefix for all pathspec's
 	 * This is used as a performance optimization which unfortunately cannot
-	 * be done when recursing into submodules
+	 * be done when recursing into submodules because when a pathspec is
+	 * given which spans repository boundaries you can't simply remove the
+	 * submodule entry because the pathspec may match something inside the
+	 * submodule.
 	 */
 	if (recurse_submodules)
 		max_prefix = NULL;
@@ -678,7 +640,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		max_prefix = common_prefix(&pathspec);
 	max_prefix_len = get_common_prefix_len(max_prefix);
 
-	prune_index(&the_index, max_prefix, max_prefix_len);
+	prune_index(the_repository->index, max_prefix, max_prefix_len);
 
 	/* Treat unmatching pathspec elements as errors */
 	if (pathspec.nr && error_unmatch)
@@ -699,11 +661,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		 */
 		if (show_stage || show_unmerged)
 			die("ls-files --with-tree is incompatible with -s or -u");
-		overlay_tree_on_index(&the_index, with_tree, max_prefix);
+		overlay_tree_on_index(the_repository->index, with_tree, max_prefix);
 	}
-	show_files(&the_index, &dir);
+
+	show_files(the_repository, &dir);
+
 	if (show_resolve_undo)
-		show_ru_info(&the_index);
+		show_ru_info(the_repository->index);
 
 	if (ps_matched) {
 		int bad;
diff --git a/git.c b/git.c
index 5be27b07e..489aab4d8 100644
--- a/git.c
+++ b/git.c
@@ -400,7 +400,7 @@ static struct cmd_struct commands[] = {
 	{ "init-db", cmd_init_db },
 	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY },
 	{ "log", cmd_log, RUN_SETUP },
-	{ "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },
+	{ "ls-files", cmd_ls_files, RUN_SETUP },
 	{ "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY },
 	{ "ls-tree", cmd_ls_tree, RUN_SETUP },
 	{ "mailinfo", cmd_mailinfo, RUN_SETUP_GENTLY },
diff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh
index ebb956fd1..318b5bce7 100755
--- a/t/t3007-ls-files-recurse-submodules.sh
+++ b/t/t3007-ls-files-recurse-submodules.sh
@@ -135,6 +135,45 @@ test_expect_success '--recurse-submodules and pathspecs setup' '
 	test_cmp expect actual
 '
 
+test_expect_success 'inactive submodule' '
+	test_when_finished "git config --bool submodule.submodule.active true" &&
+	test_when_finished "git -C submodule config --bool submodule.subsub.active true" &&
+	git config --bool submodule.submodule.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual &&
+
+	git config --bool submodule.submodule.active "true" &&
+	git -C submodule config --bool submodule.subsub.active "false" &&
+
+	cat >expect <<-\EOF &&
+	.gitmodules
+	a
+	b/b
+	h.txt
+	sib/file
+	sub/file
+	submodule/.gitmodules
+	submodule/c
+	submodule/f.TXT
+	submodule/g.txt
+	submodule/subsub
+	EOF
+
+	git ls-files --recurse-submodules >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success '--recurse-submodules and pathspecs' '
 	cat >expect <<-\EOF &&
 	h.txt
-- 
2.13.1.704.gde00cce3c-goog


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 00/20] repository object
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (19 preceding siblings ...)
  2017-06-22 18:43       ` [PATCH v4 20/20] ls-files: use repository object Brandon Williams
@ 2017-06-22 19:42       ` Stefan Beller
  2017-06-23 16:44       ` Jeff Hostetler
  21 siblings, 0 replies; 214+ messages in thread
From: Stefan Beller @ 2017-06-22 19:42 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git@vger.kernel.org, Jonathan Nieder, Jacob Keller,
	Johannes Schindelin, brian m. carlson, Ben Peart, Duy Nguyen,
	Junio C Hamano, Jeff King, Jeff Hostetler,
	Ævar Arnfjörð Bjarmason, Jonathan Tan

On Thu, Jun 22, 2017 at 11:43 AM, Brandon Williams <bmwill@google.com> wrote:
> As before you can find this series at:
> https://github.com/bmwill/git/tree/repository-object
>
> Changes in v4:
>
> * Patch 11 is slightly different and turns off all path relocation when a
>   worktree is provided instead of just for the index file (Thanks for the help
>   Jonathan Nieder).
> * 'repo_init()' has a tighter API and now requires that the provided gitdir is
>   a path to the gitdir instead of either a path to the gitdir or path to the
>   worktree (which has a .git file or directory) (Thanks Jonathan Tan).
> * Minor comment and commit message chagnes
>
> Note: Like v3 this series is dependent on on 'bw/config-h' and
>       'bw/ls-files-sans-the-index'

I read the whole series and I consider it
Reviewed-by: Stefan Beller <sbeller@google.com>

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 15/20] repository: add index_state to struct repo
  2017-06-22 18:43       ` [PATCH v4 15/20] repository: add index_state to struct repo Brandon Williams
@ 2017-06-22 20:16         ` Junio C Hamano
  2017-06-22 20:35           ` Brandon Williams
  0 siblings, 1 reply; 214+ messages in thread
From: Junio C Hamano @ 2017-06-22 20:16 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, peff, git, avarab, jonathantanmy

Brandon Williams <bmwill@google.com> writes:

>  struct repository {
>  	/* Environment */
> @@ -49,6 +50,12 @@ struct repository {
>  	 */
>  	struct config_set *config;
>  
> +	/*
> +	 * Repository's in-memory index.
> +	 * 'repo_read_index()' can be used to populate 'index'.
> +	 */
> +	struct index_state *index;
> +
>  	/* Configurations */
>  	/*
>  	 * Bit used during initialization to indicate if repository state (like
> @@ -71,4 +78,6 @@ extern void repo_set_worktree(struct repository *repo, const char *path);
>  extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
>  extern void repo_clear(struct repository *repo);
>  
> +extern int repo_read_index(struct repository *repo);
> +
>  #endif /* REPOSITORY_H */

While you are working on a simple read-only operation like
"ls-files", you can get away with just a singleton (the equivalent
to "the_index" in the repository object world) in-core index
instance, and having a way to tell the system to read things into
the default thing without having to name what that default thing is
is a very useful thing to have.  It is a good thing that this only
needs "struct repository *" and no "struct index_state *" parameter.

But you will need a way to read, update, write it out as a tree,
etc. to a non-default in-core index, once you start doing more
complex things.  The function signature of the lowest level
primitive helper for them will be:

    extern int (*)(struct repository *, struct index_state *);

And you would want to reserve "index" suffix for such a function,
following the "if you use the default in-core thing, you do not have
to pass it as a parameter---just call _cache() convenience macro;
but you can explicitly pass it to the underlying _index() function"
convention we established long time ago.

So I'd suggest renaming the above one that uses the default in-core
index instance to

	extern int repo_read_cache(struct repository *);

or you will regret later.


^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 15/20] repository: add index_state to struct repo
  2017-06-22 20:16         ` Junio C Hamano
@ 2017-06-22 20:35           ` Brandon Williams
  2017-06-22 22:10             ` Junio C Hamano
  0 siblings, 1 reply; 214+ messages in thread
From: Brandon Williams @ 2017-06-22 20:35 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, peff, git, avarab, jonathantanmy

On 06/22, Junio C Hamano wrote:
> Brandon Williams <bmwill@google.com> writes:
> 
> >  struct repository {
> >  	/* Environment */
> > @@ -49,6 +50,12 @@ struct repository {
> >  	 */
> >  	struct config_set *config;
> >  
> > +	/*
> > +	 * Repository's in-memory index.
> > +	 * 'repo_read_index()' can be used to populate 'index'.
> > +	 */
> > +	struct index_state *index;
> > +
> >  	/* Configurations */
> >  	/*
> >  	 * Bit used during initialization to indicate if repository state (like
> > @@ -71,4 +78,6 @@ extern void repo_set_worktree(struct repository *repo, const char *path);
> >  extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);
> >  extern void repo_clear(struct repository *repo);
> >  
> > +extern int repo_read_index(struct repository *repo);
> > +
> >  #endif /* REPOSITORY_H */
> 
> While you are working on a simple read-only operation like
> "ls-files", you can get away with just a singleton (the equivalent
> to "the_index" in the repository object world) in-core index
> instance, and having a way to tell the system to read things into
> the default thing without having to name what that default thing is
> is a very useful thing to have.  It is a good thing that this only
> needs "struct repository *" and no "struct index_state *" parameter.
> 
> But you will need a way to read, update, write it out as a tree,
> etc. to a non-default in-core index, once you start doing more
> complex things.  The function signature of the lowest level
> primitive helper for them will be:
> 
>     extern int (*)(struct repository *, struct index_state *);
> 
> And you would want to reserve "index" suffix for such a function,
> following the "if you use the default in-core thing, you do not have
> to pass it as a parameter---just call _cache() convenience macro;
> but you can explicitly pass it to the underlying _index() function"
> convention we established long time ago.
> 
> So I'd suggest renaming the above one that uses the default in-core
> index instance to
> 
> 	extern int repo_read_cache(struct repository *);
> 
> or you will regret later.
> 

That makes sense.  While at it, would it make sense to ensure that the
'struct index_state *' which is stored in 'the_repository.index' be
'&the_index'?

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 15/20] repository: add index_state to struct repo
  2017-06-22 20:35           ` Brandon Williams
@ 2017-06-22 22:10             ` Junio C Hamano
  0 siblings, 0 replies; 214+ messages in thread
From: Junio C Hamano @ 2017-06-22 22:10 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, sbeller, jrnieder, jacob.keller, Johannes.Schindelin,
	sandals, peartben, pclouds, peff, git, avarab, jonathantanmy

Brandon Williams <bmwill@google.com> writes:

> That makes sense.  While at it, would it make sense to ensure that the
> 'struct index_state *' which is stored in 'the_repository.index' be
> '&the_index'?

I was imagining (read: speculating one possible future, without
thinking things through to judge if it makes sense at all) that
conceptually most of the current API that works on things in the
current repository to be implicitly working on the_repository
singleton instance in the "repository object" world.

When that happens, the_index can become a CPP macro that translates
to "the_repository.index", I would think.


^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 00/20] repository object
  2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
                         ` (20 preceding siblings ...)
  2017-06-22 19:42       ` [PATCH v4 00/20] " Stefan Beller
@ 2017-06-23 16:44       ` Jeff Hostetler
  2017-06-23 17:38         ` Junio C Hamano
  21 siblings, 1 reply; 214+ messages in thread
From: Jeff Hostetler @ 2017-06-23 16:44 UTC (permalink / raw)
  To: Brandon Williams, git
  Cc: sbeller, jrnieder, jacob.keller, Johannes.Schindelin, sandals,
	peartben, pclouds, gitster, peff, avarab, jonathantanmy



On 6/22/2017 2:43 PM, Brandon Williams wrote:
> As before you can find this series at:
> https://github.com/bmwill/git/tree/repository-object
> 
> Changes in v4:
> 
> * Patch 11 is slightly different and turns off all path relocation when a
>    worktree is provided instead of just for the index file (Thanks for the help
>    Jonathan Nieder).
> * 'repo_init()' has a tighter API and now requires that the provided gitdir is
>    a path to the gitdir instead of either a path to the gitdir or path to the
>    worktree (which has a .git file or directory) (Thanks Jonathan Tan).
> * Minor comment and commit message chagnes
> 
> Note: Like v3 this series is dependent on on 'bw/config-h' and
>        'bw/ls-files-sans-the-index'
> 
> Brandon Williams (20):

I read thru the v1 and v4 versions.  Very nice.
And thanks for splitting the earlier parts out
into independent patches.

I didn't have any complaints, but did want to ACK
that I had looked at it.

Jeff


^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 00/20] repository object
  2017-06-23 16:44       ` Jeff Hostetler
@ 2017-06-23 17:38         ` Junio C Hamano
  0 siblings, 0 replies; 214+ messages in thread
From: Junio C Hamano @ 2017-06-23 17:38 UTC (permalink / raw)
  To: Jeff Hostetler
  Cc: Brandon Williams, git, sbeller, jrnieder, jacob.keller,
	Johannes.Schindelin, sandals, peartben, pclouds, peff, avarab,
	jonathantanmy

Jeff Hostetler <git@jeffhostetler.com> writes:

> On 6/22/2017 2:43 PM, Brandon Williams wrote:
>> As before you can find this series at:
>> https://github.com/bmwill/git/tree/repository-object
>>
>> Changes in v4:
>>
>> * Patch 11 is slightly different and turns off all path relocation when a
>>    worktree is provided instead of just for the index file (Thanks for the help
>>    Jonathan Nieder).
>> * 'repo_init()' has a tighter API and now requires that the provided gitdir is
>>    a path to the gitdir instead of either a path to the gitdir or path to the
>>    worktree (which has a .git file or directory) (Thanks Jonathan Tan).
>> * Minor comment and commit message chagnes
>>
>> Note: Like v3 this series is dependent on on 'bw/config-h' and
>>        'bw/ls-files-sans-the-index'
>>
>> Brandon Williams (20):
>
> I read thru the v1 and v4 versions.  Very nice.
> And thanks for splitting the earlier parts out
> into independent patches.
>
> I didn't have any complaints, but did want to ACK
> that I had looked at it.

Thanks.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-16 19:43                             ` Junio C Hamano
@ 2017-06-25  8:01                               ` René Scharfe
  2017-06-28 21:39                                 ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 214+ messages in thread
From: René Scharfe @ 2017-06-25  8:01 UTC (permalink / raw)
  To: Junio C Hamano, Ævar Arnfjörð Bjarmason; +Cc: git

Am 16.06.2017 um 21:43 schrieb Junio C Hamano:
> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
> 
>> A follow-up to the existing "type" rule added in an earlier
>> change. This catches some occurrences that are missed by the previous
>> rule.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
> 
> Hmph, I wonder if the "type" thing is really needed.  Over there,
> "ptr" is an expression and we can find "free(ptr); ptr = NULL" with
> the rule in this patch already, no?

Indeed.  How about this on top of master?

-- >8 --
Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules

There are two rules for using FREE_AND_NULL in free.cocci, one for
pointer types and one for expressions.  Both cause coccinelle to remove
empty lines and even newline characters between replacements for some
reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
FREE_AND_NULL calls on the same time.

Remove the type rule, as the expression rule already covers it, and
rearrange the lines of the latter to place the addition of FREE_AND_NULL
between the removals, which causes coccinelle to leave surrounding
whitespace untouched.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 contrib/coccinelle/free.cocci | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)

diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
index f2d97e755b..4490069df9 100644
--- a/contrib/coccinelle/free.cocci
+++ b/contrib/coccinelle/free.cocci
@@ -11,16 +11,8 @@ expression E;
   free(E);
 
 @@
-type T;
-T *ptr;
-@@
-- free(ptr);
-- ptr = NULL;
-+ FREE_AND_NULL(ptr);
-
-@@
 expression E;
 @@
 - free(E);
-- E = NULL;
 + FREE_AND_NULL(E);
+- E = NULL;
-- 
2.13.2


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-25  8:01                               ` René Scharfe
@ 2017-06-28 21:39                                 ` Ævar Arnfjörð Bjarmason
  2017-06-28 22:00                                   ` Junio C Hamano
  2017-06-28 22:21                                   ` René Scharfe
  0 siblings, 2 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-28 21:39 UTC (permalink / raw)
  To: René Scharfe; +Cc: Junio C Hamano, git


On Sun, Jun 25 2017, René Scharfe jotted:

> Am 16.06.2017 um 21:43 schrieb Junio C Hamano:
>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>
>>> A follow-up to the existing "type" rule added in an earlier
>>> change. This catches some occurrences that are missed by the previous
>>> rule.
>>>
>>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>> ---
>>
>> Hmph, I wonder if the "type" thing is really needed.  Over there,
>> "ptr" is an expression and we can find "free(ptr); ptr = NULL" with
>> the rule in this patch already, no?
>
> Indeed.  How about this on top of master?
>
> -- >8 --
> Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules
>
> There are two rules for using FREE_AND_NULL in free.cocci, one for
> pointer types and one for expressions.  Both cause coccinelle to remove
> empty lines and even newline characters between replacements for some
> reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
> FREE_AND_NULL calls on the same time.
>
> Remove the type rule, as the expression rule already covers it, and
> rearrange the lines of the latter to place the addition of FREE_AND_NULL
> between the removals, which causes coccinelle to leave surrounding
> whitespace untouched.
>
> Signed-off-by: Rene Scharfe <l.s.r@web.de>
> ---
>  contrib/coccinelle/free.cocci | 10 +---------
>  1 file changed, 1 insertion(+), 9 deletions(-)
>
> diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
> index f2d97e755b..4490069df9 100644
> --- a/contrib/coccinelle/free.cocci
> +++ b/contrib/coccinelle/free.cocci
> @@ -11,16 +11,8 @@ expression E;
>    free(E);
>
>  @@
> -type T;
> -T *ptr;
> -@@
> -- free(ptr);
> -- ptr = NULL;
> -+ FREE_AND_NULL(ptr);
> -
> -@@
>  expression E;
>  @@
>  - free(E);
> -- E = NULL;
>  + FREE_AND_NULL(E);
> +- E = NULL;

Late reply, sorry. What version of coccinelle are you running? I have
1.0.4 (from Debian) and can't get this to produce the same results as
what I have.

On top of next, I did:

        Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
        Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
        Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"

And then generated the patch as usual with `make coccicheck`, and
applied it. This has your change.

I then re-applied the manual "*.[ch] refactoring" changes

This results in this diff with next:

    $ git diff --stat origin/next.. -- '*.[ch]'
     builtin/am.c             |  3 ++-
     builtin/clean.c          |  6 ++++--
     builtin/config.c         |  6 ++++--
     builtin/index-pack.c     |  6 ++++--
     builtin/pack-objects.c   | 12 ++++++++----
     builtin/unpack-objects.c |  3 ++-
     fast-import.c            |  6 ++++--
     http-push.c              | 24 ++++++++++++++++--------
     http.c                   | 15 ++++++++++-----
     imap-send.c              |  3 ++-
     ref-filter.c             |  3 ++-
     refs/files-backend.c     |  3 ++-
     remote-testsvn.c         |  3 ++-
     sequencer.c              |  3 ++-
     sha1-array.c             |  3 ++-
     sha1_file.c              |  3 ++-
     transport-helper.c       | 27 ++++++++++++++++++---------
     transport.c              |  3 ++-
     tree-diff.c              |  6 ++++--
     tree.c                   |  3 ++-
     20 files changed, 94 insertions(+), 47 deletions(-)

These are all cases where we now miss things that should use
FREE_AND_NULL(), e.g.:

    diff --git a/builtin/am.c b/builtin/am.c
    index c973bd96dc..2f89338ed7 100644
    --- a/builtin/am.c
    +++ b/builtin/am.c
    @@ -484,7 +484,8 @@ static int run_applypatch_msg_hook(struct am_state *state)
            ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);

            if (!ret) {
    -               FREE_AND_NULL(state->msg);
    +               free(state->msg);
    +               state->msg = NULL;
                    if (read_commit_msg(state) < 0)
                            die(_("'%s' was deleted by the applypatch-msg hook"),
                                    am_path(state, "final-commit"));

So it looks to me like removing the "type T" rule breaks a lot of
things, but that the change you made to the expression rule is good, and
we should do that for the "type" rule as well. Your commit says the
"expression rule already covers it", but this doesn't seem to be the
case at all.

As an aside: Junio, did you mean to apply f8bb4631fb to next this way?
Looks like a mis-applied scissor commit.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 21:39                                 ` Ævar Arnfjörð Bjarmason
@ 2017-06-28 22:00                                   ` Junio C Hamano
  2017-06-28 22:17                                     ` Ævar Arnfjörð Bjarmason
  2017-06-28 22:21                                   ` René Scharfe
  1 sibling, 1 reply; 214+ messages in thread
From: Junio C Hamano @ 2017-06-28 22:00 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: René Scharfe, git

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> Late reply, sorry. What version of coccinelle are you running? I have
> 1.0.4 (from Debian) and can't get this to produce the same results as
> what I have.
>
> On top of next, I did:
>
>         Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
>         Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
>         Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"
>
> And then generated the patch as usual with `make coccicheck`, and
> applied it. This has your change.
>
> I then re-applied the manual "*.[ch] refactoring" changes
>
> This results in this diff with next:
> ...
>
> These are all cases where we now miss things that should use
> FREE_AND_NULL(), e.g.:
>
>     diff --git a/builtin/am.c b/builtin/am.c
>     index c973bd96dc..2f89338ed7 100644
>     --- a/builtin/am.c
>     +++ b/builtin/am.c
>     @@ -484,7 +484,8 @@ static int run_applypatch_msg_hook(struct am_state *state)
>             ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
>
>             if (!ret) {
>     -               FREE_AND_NULL(state->msg);
>     +               free(state->msg);
>     +               state->msg = NULL;
>                     if (read_commit_msg(state) < 0)
>                             die(_("'%s' was deleted by the applypatch-msg hook"),
>                                     am_path(state, "final-commit"));
>
> So it looks to me like removing the "type T" rule breaks a lot of
> things, but that the change you made to the expression rule is good, and
> we should do that for the "type" rule as well. Your commit says the
> "expression rule already covers it", but this doesn't seem to be the
> case at all.

OK, if an older version of the tool that is otherwise still usable
needs two rules, let's keep the "type T" one by reverting the change
out of 'next' and then have a patch that only rearranges the lines,
something like the attached.

-- >8 --
From: René Scharfe <l.s.r@web.de>
Date: Sun, 25 Jun 2017 10:01:04 +0200
Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules

There are two rules for using FREE_AND_NULL in free.cocci, one for
pointer types and one for expressions.  Both cause coccinelle to remove
empty lines and even newline characters between replacements for some
reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
FREE_AND_NULL calls on the same time.

Rearrange the lines to place the addition of FREE_AND_NULL between
the removals, which causes coccinelle to leave surrounding
whitespace untouched.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 contrib/coccinelle/free.cocci | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
index f2d97e755b..179c185d85 100644
--- a/contrib/coccinelle/free.cocci
+++ b/contrib/coccinelle/free.cocci
@@ -15,12 +15,12 @@ type T;
 T *ptr;
 @@
 - free(ptr);
-- ptr = NULL;
 + FREE_AND_NULL(ptr);
+- ptr = NULL;
 
 @@
 expression E;
 @@
 - free(E);
-- E = NULL;
 + FREE_AND_NULL(E);
+- E = NULL;
-- 
2.13.2-659-g904a3dfd54


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 22:00                                   ` Junio C Hamano
@ 2017-06-28 22:17                                     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-28 22:17 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: René Scharfe, git


On Wed, Jun 28 2017, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> Late reply, sorry. What version of coccinelle are you running? I have
>> 1.0.4 (from Debian) and can't get this to produce the same results as
>> what I have.
>>
>> On top of next, I did:
>>
>>         Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
>>         Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
>>         Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"
>>
>> And then generated the patch as usual with `make coccicheck`, and
>> applied it. This has your change.
>>
>> I then re-applied the manual "*.[ch] refactoring" changes
>>
>> This results in this diff with next:
>> ...
>>
>> These are all cases where we now miss things that should use
>> FREE_AND_NULL(), e.g.:
>>
>>     diff --git a/builtin/am.c b/builtin/am.c
>>     index c973bd96dc..2f89338ed7 100644
>>     --- a/builtin/am.c
>>     +++ b/builtin/am.c
>>     @@ -484,7 +484,8 @@ static int run_applypatch_msg_hook(struct am_state *state)
>>             ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
>>
>>             if (!ret) {
>>     -               FREE_AND_NULL(state->msg);
>>     +               free(state->msg);
>>     +               state->msg = NULL;
>>                     if (read_commit_msg(state) < 0)
>>                             die(_("'%s' was deleted by the applypatch-msg hook"),
>>                                     am_path(state, "final-commit"));
>>
>> So it looks to me like removing the "type T" rule breaks a lot of
>> things, but that the change you made to the expression rule is good, and
>> we should do that for the "type" rule as well. Your commit says the
>> "expression rule already covers it", but this doesn't seem to be the
>> case at all.
>
> OK, if an older version of the tool that is otherwise still usable
> needs two rules, let's keep the "type T" one by reverting the change
> out of 'next' and then have a patch that only rearranges the lines,
> something like the attached.
>
> -- >8 --
> From: René Scharfe <l.s.r@web.de>
> Date: Sun, 25 Jun 2017 10:01:04 +0200
> Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules
>
> There are two rules for using FREE_AND_NULL in free.cocci, one for
> pointer types and one for expressions.  Both cause coccinelle to remove
> empty lines and even newline characters between replacements for some
> reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
> FREE_AND_NULL calls on the same time.
>
> Rearrange the lines to place the addition of FREE_AND_NULL between
> the removals, which causes coccinelle to leave surrounding
> whitespace untouched.
>
> Signed-off-by: Rene Scharfe <l.s.r@web.de>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  contrib/coccinelle/free.cocci | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
> index f2d97e755b..179c185d85 100644
> --- a/contrib/coccinelle/free.cocci
> +++ b/contrib/coccinelle/free.cocci
> @@ -15,12 +15,12 @@ type T;
>  T *ptr;
>  @@
>  - free(ptr);
> -- ptr = NULL;
>  + FREE_AND_NULL(ptr);
> +- ptr = NULL;
>
>  @@
>  expression E;
>  @@
>  - free(E);
> -- E = NULL;
>  + FREE_AND_NULL(E);
> +- E = NULL;

Looks good.

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 21:39                                 ` Ævar Arnfjörð Bjarmason
  2017-06-28 22:00                                   ` Junio C Hamano
@ 2017-06-28 22:21                                   ` René Scharfe
  2017-06-28 22:30                                     ` Ævar Arnfjörð Bjarmason
                                                       ` (2 more replies)
  1 sibling, 3 replies; 214+ messages in thread
From: René Scharfe @ 2017-06-28 22:21 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Junio C Hamano, git

Am 28.06.2017 um 23:39 schrieb Ævar Arnfjörð Bjarmason:
> 
> On Sun, Jun 25 2017, René Scharfe jotted:
> 
>> Am 16.06.2017 um 21:43 schrieb Junio C Hamano:
>>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>>
>>>> A follow-up to the existing "type" rule added in an earlier
>>>> change. This catches some occurrences that are missed by the previous
>>>> rule.
>>>>
>>>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>>> ---
>>>
>>> Hmph, I wonder if the "type" thing is really needed.  Over there,
>>> "ptr" is an expression and we can find "free(ptr); ptr = NULL" with
>>> the rule in this patch already, no?
>>
>> Indeed.  How about this on top of master?
>>
>> -- >8 --
>> Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules
>>
>> There are two rules for using FREE_AND_NULL in free.cocci, one for
>> pointer types and one for expressions.  Both cause coccinelle to remove
>> empty lines and even newline characters between replacements for some
>> reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
>> FREE_AND_NULL calls on the same time.
>>
>> Remove the type rule, as the expression rule already covers it, and
>> rearrange the lines of the latter to place the addition of FREE_AND_NULL
>> between the removals, which causes coccinelle to leave surrounding
>> whitespace untouched.
>>
>> Signed-off-by: Rene Scharfe <l.s.r@web.de>
>> ---
>>   contrib/coccinelle/free.cocci | 10 +---------
>>   1 file changed, 1 insertion(+), 9 deletions(-)
>>
>> diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
>> index f2d97e755b..4490069df9 100644
>> --- a/contrib/coccinelle/free.cocci
>> +++ b/contrib/coccinelle/free.cocci
>> @@ -11,16 +11,8 @@ expression E;
>>     free(E);
>>
>>   @@
>> -type T;
>> -T *ptr;
>> -@@
>> -- free(ptr);
>> -- ptr = NULL;
>> -+ FREE_AND_NULL(ptr);
>> -
>> -@@
>>   expression E;
>>   @@
>>   - free(E);
>> -- E = NULL;
>>   + FREE_AND_NULL(E);
>> +- E = NULL;
> 
> Late reply, sorry. What version of coccinelle are you running? I have
> 1.0.4 (from Debian) and can't get this to produce the same results as
> what I have.
> 
> On top of next, I did:
> 
>          Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
>          Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
>          Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"
> 
> And then generated the patch as usual with `make coccicheck`, and
> applied it. This has your change.
> 
> I then re-applied the manual "*.[ch] refactoring" changes
> 
> This results in this diff with next:
> 
>      $ git diff --stat origin/next.. -- '*.[ch]'
>       builtin/am.c             |  3 ++-
>       builtin/clean.c          |  6 ++++--
>       builtin/config.c         |  6 ++++--
>       builtin/index-pack.c     |  6 ++++--
>       builtin/pack-objects.c   | 12 ++++++++----
>       builtin/unpack-objects.c |  3 ++-
>       fast-import.c            |  6 ++++--
>       http-push.c              | 24 ++++++++++++++++--------
>       http.c                   | 15 ++++++++++-----
>       imap-send.c              |  3 ++-
>       ref-filter.c             |  3 ++-
>       refs/files-backend.c     |  3 ++-
>       remote-testsvn.c         |  3 ++-
>       sequencer.c              |  3 ++-
>       sha1-array.c             |  3 ++-
>       sha1_file.c              |  3 ++-
>       transport-helper.c       | 27 ++++++++++++++++++---------
>       transport.c              |  3 ++-
>       tree-diff.c              |  6 ++++--
>       tree.c                   |  3 ++-
>       20 files changed, 94 insertions(+), 47 deletions(-)
> 
> These are all cases where we now miss things that should use
> FREE_AND_NULL(), e.g.:
> 
>      diff --git a/builtin/am.c b/builtin/am.c
>      index c973bd96dc..2f89338ed7 100644
>      --- a/builtin/am.c
>      +++ b/builtin/am.c
>      @@ -484,7 +484,8 @@ static int run_applypatch_msg_hook(struct am_state *state)
>              ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
> 
>              if (!ret) {
>      -               FREE_AND_NULL(state->msg);
>      +               free(state->msg);
>      +               state->msg = NULL;
>                      if (read_commit_msg(state) < 0)
>                              die(_("'%s' was deleted by the applypatch-msg hook"),
>                                      am_path(state, "final-commit"));
> 
> So it looks to me like removing the "type T" rule breaks a lot of
> things, but that the change you made to the expression rule is good, and
> we should do that for the "type" rule as well. Your commit says the
> "expression rule already covers it", but this doesn't seem to be the
> case at all.
> 
> As an aside: Junio, did you mean to apply f8bb4631fb to next this way?
> Looks like a mis-applied scissor commit.

I can't reproduce that strange result with master, but get a scissors
line into the commit message as well.  The diff at the end looks
reasonable (contains just the manual stuff).  Here's what I did:

$ uname -a
Linux debian 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2 (2017-06-12) x86_64 GNU/Linux

$ spatch -version
spatch version 1.0.4 with Python support and with PCRE support

$ git checkout -b cocci-free-null origin/master
Branch cocci-free-null set up to track remote branch master from origin.
Switched to a new branch 'cocci-free-null'

$ git am
# pasted my email
Applying: coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
# results in commit message with scissor line, interesting..

$ git revert --no-edit 88ce3ef636 e140f7afdd 6a83d90207
[cocci-free-null b50d5eca5f] Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
 12 files changed, 49 insertions(+), 23 deletions(-)
[cocci-free-null cf9072ce4e] Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
 2 files changed, 4 insertions(+), 2 deletions(-)
[cocci-free-null 97675d4dc2] Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"
 37 files changed, 142 insertions(+), 71 deletions(-)

$ make contrib/coccinelle/free.cocci.patch
GIT_VERSION = 2.13.2.537.g97675d4dc2
     SPATCH contrib/coccinelle/free.cocci
     SPATCH result: contrib/coccinelle/free.cocci.patch

$ patch -p0 <contrib/coccinelle/free.cocci.patch
patching file alias.c
patching file apply.c
patching file attr.c
patching file blame.c
patching file branch.c
patching file commit.c
patching file config.c
patching file credential.c
patching file diffcore-rename.c
patching file diff-lib.c
patching file diff.c
patching file dir.c
patching file grep.c
patching file line-log.c
patching file ll-merge.c
patching file mailinfo.c
patching file object.c
patching file pathspec.c
patching file read-cache.c
patching file refs/files-backend.c
patching file ref-filter.c
patching file sequencer.c
patching file sha1-array.c
patching file sha1_file.c
patching file transport.c
patching file transport-helper.c
patching file tree-diff.c
patching file tree.c
patching file builtin/am.c
patching file builtin/clean.c
patching file builtin/config.c
patching file builtin/index-pack.c
patching file builtin/pack-objects.c
patching file builtin/unpack-objects.c
patching file fast-import.c
patching file imap-send.c
patching file remote-testsvn.c
patching file http-push.c
patching file http.c

$ git diff HEAD^^^
diff --git a/builtin/am.c b/builtin/am.c
index c973bd96dc..eb24eb7b08 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1074,8 +1074,11 @@ static void am_next(struct am_state *state)
 	struct object_id head;
 
 	FREE_AND_NULL(state->author_name);
+
 	FREE_AND_NULL(state->author_email);
+
 	FREE_AND_NULL(state->author_date);
+
 	FREE_AND_NULL(state->msg);
 	state->msg_len = 0;
 
diff --git a/builtin/worktree.c b/builtin/worktree.c
index c98e2ce5f5..0c5476ee9d 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -300,8 +300,10 @@ static int add_worktree(const char *path, const char *refname,
 	}
 
 	is_junk = 0;
-	FREE_AND_NULL(junk_work_tree);
-	FREE_AND_NULL(junk_git_dir);
+	free(junk_work_tree);
+	free(junk_git_dir);
+	junk_work_tree = NULL;
+	junk_git_dir = NULL;
 
 done:
 	if (ret || !opts->keep_locked) {
diff --git a/commit-slab.h b/commit-slab.h
index 333d81e370..42d16dcded 100644
--- a/commit-slab.h
+++ b/commit-slab.h
@@ -82,7 +82,8 @@ static MAYBE_UNUSED void clear_ ##slabname(struct slabname *s)		\
 	for (i = 0; i < s->slab_count; i++)				\
 		free(s->slab[i]);					\
 	s->slab_count = 0;						\
-	FREE_AND_NULL(s->slab);						\
+	free(s->slab);							\
+	s->slab = NULL;							\
 }									\
 									\
 static MAYBE_UNUSED elemtype *slabname## _at_peek(struct slabname *s,	\
diff --git a/gpg-interface.c b/gpg-interface.c
index d936f3a32f..8ab32df457 100644
--- a/gpg-interface.c
+++ b/gpg-interface.c
@@ -14,11 +14,16 @@ static const char *gpg_program = "gpg";
 
 void signature_check_clear(struct signature_check *sigc)
 {
-	FREE_AND_NULL(sigc->payload);
-	FREE_AND_NULL(sigc->gpg_output);
-	FREE_AND_NULL(sigc->gpg_status);
-	FREE_AND_NULL(sigc->signer);
-	FREE_AND_NULL(sigc->key);
+	free(sigc->payload);
+	free(sigc->gpg_output);
+	free(sigc->gpg_status);
+	free(sigc->signer);
+	free(sigc->key);
+	sigc->payload = NULL;
+	sigc->gpg_output = NULL;
+	sigc->gpg_status = NULL;
+	sigc->signer = NULL;
+	sigc->key = NULL;
 }
 
 static struct {
diff --git a/help.c b/help.c
index 88a3aeaeb9..5af5319caf 100644
--- a/help.c
+++ b/help.c
@@ -269,8 +269,9 @@ static void add_cmd_list(struct cmdnames *cmds, struct cmdnames *old)
 
 	for (i = 0; i < old->cnt; i++)
 		cmds->names[cmds->cnt++] = old->names[i];
-	FREE_AND_NULL(old->names);
+	free(old->names);
 	old->cnt = 0;
+	old->names = NULL;
 }
 
 /* An empirically derived magic number */
diff --git a/line-log.c b/line-log.c
index ab0709f9ae..f225447ebb 100644
--- a/line-log.c
+++ b/line-log.c
@@ -34,8 +34,9 @@ void range_set_init(struct range_set *rs, size_t prealloc)
 
 void range_set_release(struct range_set *rs)
 {
-	FREE_AND_NULL(rs->ranges);
+	free(rs->ranges);
 	rs->alloc = rs->nr = 0;
+	rs->ranges = NULL;
 }
 
 /* dst must be uninitialized! */
diff --git a/prio-queue.c b/prio-queue.c
index 126d096727..fc3860fdcb 100644
--- a/prio-queue.c
+++ b/prio-queue.c
@@ -27,9 +27,10 @@ void prio_queue_reverse(struct prio_queue *queue)
 
 void clear_prio_queue(struct prio_queue *queue)
 {
-	FREE_AND_NULL(queue->array);
+	free(queue->array);
 	queue->nr = 0;
 	queue->alloc = 0;
+	queue->array = NULL;
 	queue->insertion_ctr = 0;
 }
 
diff --git a/refs/ref-cache.c b/refs/ref-cache.c
index 76bb723c86..af2fcb2c12 100644
--- a/refs/ref-cache.c
+++ b/refs/ref-cache.c
@@ -82,8 +82,9 @@ static void clear_ref_dir(struct ref_dir *dir)
 	int i;
 	for (i = 0; i < dir->nr; i++)
 		free_ref_entry(dir->entries[i]);
-	FREE_AND_NULL(dir->entries);
+	free(dir->entries);
 	dir->sorted = dir->nr = dir->alloc = 0;
+	dir->entries = NULL;
 }
 
 struct ref_entry *create_dir_entry(struct ref_cache *cache,
diff --git a/rerere.c b/rerere.c
index 70634d456c..829b3b0f08 100644
--- a/rerere.c
+++ b/rerere.c
@@ -40,8 +40,9 @@ static void free_rerere_dirs(void)
 		free(rerere_dir[i]->status);
 		free(rerere_dir[i]);
 	}
-	FREE_AND_NULL(rerere_dir);
+	free(rerere_dir);
 	rerere_dir_nr = rerere_dir_alloc = 0;
+	rerere_dir = NULL;
 }
 
 static void free_rerere_id(struct string_list_item *item)
diff --git a/split-index.c b/split-index.c
index 83e39ec8d7..f519e60f87 100644
--- a/split-index.c
+++ b/split-index.c
@@ -167,9 +167,10 @@ void merge_base_index(struct index_state *istate)
 
 	ewah_free(si->delete_bitmap);
 	ewah_free(si->replace_bitmap);
-	FREE_AND_NULL(si->saved_cache);
+	free(si->saved_cache);
 	si->delete_bitmap  = NULL;
 	si->replace_bitmap = NULL;
+	si->saved_cache	   = NULL;
 	si->saved_cache_nr = 0;
 }
 


^ permalink raw reply related	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 22:21                                   ` René Scharfe
@ 2017-06-28 22:30                                     ` Ævar Arnfjörð Bjarmason
  2017-06-28 23:15                                       ` Junio C Hamano
  2017-06-28 22:30                                     ` René Scharfe
  2017-06-28 22:44                                     ` René Scharfe
  2 siblings, 1 reply; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-28 22:30 UTC (permalink / raw)
  To: René Scharfe; +Cc: Junio C Hamano, git


On Wed, Jun 28 2017, René Scharfe jotted:

> Am 28.06.2017 um 23:39 schrieb Ævar Arnfjörð Bjarmason:
>>
>> On Sun, Jun 25 2017, René Scharfe jotted:
>>
>>> Am 16.06.2017 um 21:43 schrieb Junio C Hamano:
>>>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>>>
>>>>> A follow-up to the existing "type" rule added in an earlier
>>>>> change. This catches some occurrences that are missed by the previous
>>>>> rule.
>>>>>
>>>>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>>>> ---
>>>>
>>>> Hmph, I wonder if the "type" thing is really needed.  Over there,
>>>> "ptr" is an expression and we can find "free(ptr); ptr = NULL" with
>>>> the rule in this patch already, no?
>>>
>>> Indeed.  How about this on top of master?
>>>
>>> -- >8 --
>>> Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules
>>>
>>> There are two rules for using FREE_AND_NULL in free.cocci, one for
>>> pointer types and one for expressions.  Both cause coccinelle to remove
>>> empty lines and even newline characters between replacements for some
>>> reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
>>> FREE_AND_NULL calls on the same time.
>>>
>>> Remove the type rule, as the expression rule already covers it, and
>>> rearrange the lines of the latter to place the addition of FREE_AND_NULL
>>> between the removals, which causes coccinelle to leave surrounding
>>> whitespace untouched.
>>>
>>> Signed-off-by: Rene Scharfe <l.s.r@web.de>
>>> ---
>>>   contrib/coccinelle/free.cocci | 10 +---------
>>>   1 file changed, 1 insertion(+), 9 deletions(-)
>>>
>>> diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
>>> index f2d97e755b..4490069df9 100644
>>> --- a/contrib/coccinelle/free.cocci
>>> +++ b/contrib/coccinelle/free.cocci
>>> @@ -11,16 +11,8 @@ expression E;
>>>     free(E);
>>>
>>>   @@
>>> -type T;
>>> -T *ptr;
>>> -@@
>>> -- free(ptr);
>>> -- ptr = NULL;
>>> -+ FREE_AND_NULL(ptr);
>>> -
>>> -@@
>>>   expression E;
>>>   @@
>>>   - free(E);
>>> -- E = NULL;
>>>   + FREE_AND_NULL(E);
>>> +- E = NULL;
>>
>> Late reply, sorry. What version of coccinelle are you running? I have
>> 1.0.4 (from Debian) and can't get this to produce the same results as
>> what I have.
>>
>> On top of next, I did:
>>
>>          Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
>>          Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
>>          Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"
>>
>> And then generated the patch as usual with `make coccicheck`, and
>> applied it. This has your change.
>>
>> I then re-applied the manual "*.[ch] refactoring" changes
>>
>> This results in this diff with next:
>>
>>      $ git diff --stat origin/next.. -- '*.[ch]'
>>       builtin/am.c             |  3 ++-
>>       builtin/clean.c          |  6 ++++--
>>       builtin/config.c         |  6 ++++--
>>       builtin/index-pack.c     |  6 ++++--
>>       builtin/pack-objects.c   | 12 ++++++++----
>>       builtin/unpack-objects.c |  3 ++-
>>       fast-import.c            |  6 ++++--
>>       http-push.c              | 24 ++++++++++++++++--------
>>       http.c                   | 15 ++++++++++-----
>>       imap-send.c              |  3 ++-
>>       ref-filter.c             |  3 ++-
>>       refs/files-backend.c     |  3 ++-
>>       remote-testsvn.c         |  3 ++-
>>       sequencer.c              |  3 ++-
>>       sha1-array.c             |  3 ++-
>>       sha1_file.c              |  3 ++-
>>       transport-helper.c       | 27 ++++++++++++++++++---------
>>       transport.c              |  3 ++-
>>       tree-diff.c              |  6 ++++--
>>       tree.c                   |  3 ++-
>>       20 files changed, 94 insertions(+), 47 deletions(-)
>>
>> These are all cases where we now miss things that should use
>> FREE_AND_NULL(), e.g.:
>>
>>      diff --git a/builtin/am.c b/builtin/am.c
>>      index c973bd96dc..2f89338ed7 100644
>>      --- a/builtin/am.c
>>      +++ b/builtin/am.c
>>      @@ -484,7 +484,8 @@ static int run_applypatch_msg_hook(struct am_state *state)
>>              ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
>>
>>              if (!ret) {
>>      -               FREE_AND_NULL(state->msg);
>>      +               free(state->msg);
>>      +               state->msg = NULL;
>>                      if (read_commit_msg(state) < 0)
>>                              die(_("'%s' was deleted by the applypatch-msg hook"),
>>                                      am_path(state, "final-commit"));
>>
>> So it looks to me like removing the "type T" rule breaks a lot of
>> things, but that the change you made to the expression rule is good, and
>> we should do that for the "type" rule as well. Your commit says the
>> "expression rule already covers it", but this doesn't seem to be the
>> case at all.
>>
>> As an aside: Junio, did you mean to apply f8bb4631fb to next this way?
>> Looks like a mis-applied scissor commit.
>
> I can't reproduce that strange result with master, but get a scissors
> line into the commit message as well.  The diff at the end looks
> reasonable (contains just the manual stuff).  Here's what I did:

I get exactly the same thing as you do below when following these
steps. So it seems your patch in
2122b01f-7627-cd1b-c7df-751c076f90ff@web.de is just fine as-is and I
just screwed something up when testing this.

Sorry about the noise. Since this works your original patch is obviously
preferrable since it's not duplicating the rule.

> $ uname -a
> Linux debian 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2 (2017-06-12) x86_64 GNU/Linux
>
> $ spatch -version
> spatch version 1.0.4 with Python support and with PCRE support
>
> $ git checkout -b cocci-free-null origin/master
> Branch cocci-free-null set up to track remote branch master from origin.
> Switched to a new branch 'cocci-free-null'
>
> $ git am
> # pasted my email
> Applying: coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
> # results in commit message with scissor line, interesting..
>
> $ git revert --no-edit 88ce3ef636 e140f7afdd 6a83d90207
> [cocci-free-null b50d5eca5f] Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
>  12 files changed, 49 insertions(+), 23 deletions(-)
> [cocci-free-null cf9072ce4e] Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
>  2 files changed, 4 insertions(+), 2 deletions(-)
> [cocci-free-null 97675d4dc2] Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"
>  37 files changed, 142 insertions(+), 71 deletions(-)
>
> $ make contrib/coccinelle/free.cocci.patch
> GIT_VERSION = 2.13.2.537.g97675d4dc2
>      SPATCH contrib/coccinelle/free.cocci
>      SPATCH result: contrib/coccinelle/free.cocci.patch
>
> $ patch -p0 <contrib/coccinelle/free.cocci.patch
> patching file alias.c
> patching file apply.c
> patching file attr.c
> patching file blame.c
> patching file branch.c
> patching file commit.c
> patching file config.c
> patching file credential.c
> patching file diffcore-rename.c
> patching file diff-lib.c
> patching file diff.c
> patching file dir.c
> patching file grep.c
> patching file line-log.c
> patching file ll-merge.c
> patching file mailinfo.c
> patching file object.c
> patching file pathspec.c
> patching file read-cache.c
> patching file refs/files-backend.c
> patching file ref-filter.c
> patching file sequencer.c
> patching file sha1-array.c
> patching file sha1_file.c
> patching file transport.c
> patching file transport-helper.c
> patching file tree-diff.c
> patching file tree.c
> patching file builtin/am.c
> patching file builtin/clean.c
> patching file builtin/config.c
> patching file builtin/index-pack.c
> patching file builtin/pack-objects.c
> patching file builtin/unpack-objects.c
> patching file fast-import.c
> patching file imap-send.c
> patching file remote-testsvn.c
> patching file http-push.c
> patching file http.c
>
> $ git diff HEAD^^^
> diff --git a/builtin/am.c b/builtin/am.c
> index c973bd96dc..eb24eb7b08 100644
> --- a/builtin/am.c
> +++ b/builtin/am.c
> @@ -1074,8 +1074,11 @@ static void am_next(struct am_state *state)
>  	struct object_id head;
>
>  	FREE_AND_NULL(state->author_name);
> +
>  	FREE_AND_NULL(state->author_email);
> +
>  	FREE_AND_NULL(state->author_date);
> +
>  	FREE_AND_NULL(state->msg);
>  	state->msg_len = 0;
>
> diff --git a/builtin/worktree.c b/builtin/worktree.c
> index c98e2ce5f5..0c5476ee9d 100644
> --- a/builtin/worktree.c
> +++ b/builtin/worktree.c
> @@ -300,8 +300,10 @@ static int add_worktree(const char *path, const char *refname,
>  	}
>
>  	is_junk = 0;
> -	FREE_AND_NULL(junk_work_tree);
> -	FREE_AND_NULL(junk_git_dir);
> +	free(junk_work_tree);
> +	free(junk_git_dir);
> +	junk_work_tree = NULL;
> +	junk_git_dir = NULL;
>
>  done:
>  	if (ret || !opts->keep_locked) {
> diff --git a/commit-slab.h b/commit-slab.h
> index 333d81e370..42d16dcded 100644
> --- a/commit-slab.h
> +++ b/commit-slab.h
> @@ -82,7 +82,8 @@ static MAYBE_UNUSED void clear_ ##slabname(struct slabname *s)		\
>  	for (i = 0; i < s->slab_count; i++)				\
>  		free(s->slab[i]);					\
>  	s->slab_count = 0;						\
> -	FREE_AND_NULL(s->slab);						\
> +	free(s->slab);							\
> +	s->slab = NULL;							\
>  }									\
>  									\
>  static MAYBE_UNUSED elemtype *slabname## _at_peek(struct slabname *s,	\
> diff --git a/gpg-interface.c b/gpg-interface.c
> index d936f3a32f..8ab32df457 100644
> --- a/gpg-interface.c
> +++ b/gpg-interface.c
> @@ -14,11 +14,16 @@ static const char *gpg_program = "gpg";
>
>  void signature_check_clear(struct signature_check *sigc)
>  {
> -	FREE_AND_NULL(sigc->payload);
> -	FREE_AND_NULL(sigc->gpg_output);
> -	FREE_AND_NULL(sigc->gpg_status);
> -	FREE_AND_NULL(sigc->signer);
> -	FREE_AND_NULL(sigc->key);
> +	free(sigc->payload);
> +	free(sigc->gpg_output);
> +	free(sigc->gpg_status);
> +	free(sigc->signer);
> +	free(sigc->key);
> +	sigc->payload = NULL;
> +	sigc->gpg_output = NULL;
> +	sigc->gpg_status = NULL;
> +	sigc->signer = NULL;
> +	sigc->key = NULL;
>  }
>
>  static struct {
> diff --git a/help.c b/help.c
> index 88a3aeaeb9..5af5319caf 100644
> --- a/help.c
> +++ b/help.c
> @@ -269,8 +269,9 @@ static void add_cmd_list(struct cmdnames *cmds, struct cmdnames *old)
>
>  	for (i = 0; i < old->cnt; i++)
>  		cmds->names[cmds->cnt++] = old->names[i];
> -	FREE_AND_NULL(old->names);
> +	free(old->names);
>  	old->cnt = 0;
> +	old->names = NULL;
>  }
>
>  /* An empirically derived magic number */
> diff --git a/line-log.c b/line-log.c
> index ab0709f9ae..f225447ebb 100644
> --- a/line-log.c
> +++ b/line-log.c
> @@ -34,8 +34,9 @@ void range_set_init(struct range_set *rs, size_t prealloc)
>
>  void range_set_release(struct range_set *rs)
>  {
> -	FREE_AND_NULL(rs->ranges);
> +	free(rs->ranges);
>  	rs->alloc = rs->nr = 0;
> +	rs->ranges = NULL;
>  }
>
>  /* dst must be uninitialized! */
> diff --git a/prio-queue.c b/prio-queue.c
> index 126d096727..fc3860fdcb 100644
> --- a/prio-queue.c
> +++ b/prio-queue.c
> @@ -27,9 +27,10 @@ void prio_queue_reverse(struct prio_queue *queue)
>
>  void clear_prio_queue(struct prio_queue *queue)
>  {
> -	FREE_AND_NULL(queue->array);
> +	free(queue->array);
>  	queue->nr = 0;
>  	queue->alloc = 0;
> +	queue->array = NULL;
>  	queue->insertion_ctr = 0;
>  }
>
> diff --git a/refs/ref-cache.c b/refs/ref-cache.c
> index 76bb723c86..af2fcb2c12 100644
> --- a/refs/ref-cache.c
> +++ b/refs/ref-cache.c
> @@ -82,8 +82,9 @@ static void clear_ref_dir(struct ref_dir *dir)
>  	int i;
>  	for (i = 0; i < dir->nr; i++)
>  		free_ref_entry(dir->entries[i]);
> -	FREE_AND_NULL(dir->entries);
> +	free(dir->entries);
>  	dir->sorted = dir->nr = dir->alloc = 0;
> +	dir->entries = NULL;
>  }
>
>  struct ref_entry *create_dir_entry(struct ref_cache *cache,
> diff --git a/rerere.c b/rerere.c
> index 70634d456c..829b3b0f08 100644
> --- a/rerere.c
> +++ b/rerere.c
> @@ -40,8 +40,9 @@ static void free_rerere_dirs(void)
>  		free(rerere_dir[i]->status);
>  		free(rerere_dir[i]);
>  	}
> -	FREE_AND_NULL(rerere_dir);
> +	free(rerere_dir);
>  	rerere_dir_nr = rerere_dir_alloc = 0;
> +	rerere_dir = NULL;
>  }
>
>  static void free_rerere_id(struct string_list_item *item)
> diff --git a/split-index.c b/split-index.c
> index 83e39ec8d7..f519e60f87 100644
> --- a/split-index.c
> +++ b/split-index.c
> @@ -167,9 +167,10 @@ void merge_base_index(struct index_state *istate)
>
>  	ewah_free(si->delete_bitmap);
>  	ewah_free(si->replace_bitmap);
> -	FREE_AND_NULL(si->saved_cache);
> +	free(si->saved_cache);
>  	si->delete_bitmap  = NULL;
>  	si->replace_bitmap = NULL;
> +	si->saved_cache	   = NULL;
>  	si->saved_cache_nr = 0;
>  }
>

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 22:21                                   ` René Scharfe
  2017-06-28 22:30                                     ` Ævar Arnfjörð Bjarmason
@ 2017-06-28 22:30                                     ` René Scharfe
  2017-06-28 22:35                                       ` Ævar Arnfjörð Bjarmason
  2017-06-28 22:44                                     ` René Scharfe
  2 siblings, 1 reply; 214+ messages in thread
From: René Scharfe @ 2017-06-28 22:30 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Junio C Hamano, git

Am 29.06.2017 um 00:21 schrieb René Scharfe:
> Am 28.06.2017 um 23:39 schrieb Ævar Arnfjörð Bjarmason:
>>
>> On Sun, Jun 25 2017, René Scharfe jotted:
>>
>>> Am 16.06.2017 um 21:43 schrieb Junio C Hamano:
>>>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>>>
>>>>> A follow-up to the existing "type" rule added in an earlier
>>>>> change. This catches some occurrences that are missed by the previous
>>>>> rule.
>>>>>
>>>>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>>>> ---
>>>>
>>>> Hmph, I wonder if the "type" thing is really needed.  Over there,
>>>> "ptr" is an expression and we can find "free(ptr); ptr = NULL" with
>>>> the rule in this patch already, no?
>>>
>>> Indeed.  How about this on top of master?
>>>
>>> -- >8 --
>>> Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules
>>>
>>> There are two rules for using FREE_AND_NULL in free.cocci, one for
>>> pointer types and one for expressions.  Both cause coccinelle to remove
>>> empty lines and even newline characters between replacements for some
>>> reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
>>> FREE_AND_NULL calls on the same time.
>>>
>>> Remove the type rule, as the expression rule already covers it, and
>>> rearrange the lines of the latter to place the addition of FREE_AND_NULL
>>> between the removals, which causes coccinelle to leave surrounding
>>> whitespace untouched.
>>>
>>> Signed-off-by: Rene Scharfe <l.s.r@web.de>
>>> ---
>>>    contrib/coccinelle/free.cocci | 10 +---------
>>>    1 file changed, 1 insertion(+), 9 deletions(-)
>>>
>>> diff --git a/contrib/coccinelle/free.cocci b/contrib/coccinelle/free.cocci
>>> index f2d97e755b..4490069df9 100644
>>> --- a/contrib/coccinelle/free.cocci
>>> +++ b/contrib/coccinelle/free.cocci
>>> @@ -11,16 +11,8 @@ expression E;
>>>      free(E);
>>>
>>>    @@
>>> -type T;
>>> -T *ptr;
>>> -@@
>>> -- free(ptr);
>>> -- ptr = NULL;
>>> -+ FREE_AND_NULL(ptr);
>>> -
>>> -@@
>>>    expression E;
>>>    @@
>>>    - free(E);
>>> -- E = NULL;
>>>    + FREE_AND_NULL(E);
>>> +- E = NULL;
>>
>> Late reply, sorry. What version of coccinelle are you running? I have
>> 1.0.4 (from Debian) and can't get this to produce the same results as
>> what I have.
>>
>> On top of next, I did:
>>
>>           Revert "*.[ch] refactoring: make use of the FREE_AND_NULL() macro"
>>           Revert "coccinelle: make use of the "expression" FREE_AND_NULL() rule"
>>           Revert "coccinelle: make use of the "type" FREE_AND_NULL() rule"
>>
>> And then generated the patch as usual with `make coccicheck`, and
>> applied it. This has your change.
>>
>> I then re-applied the manual "*.[ch] refactoring" changes
>>
>> This results in this diff with next:
>>
>>       $ git diff --stat origin/next.. -- '*.[ch]'
>>        builtin/am.c             |  3 ++-
>>        builtin/clean.c          |  6 ++++--
>>        builtin/config.c         |  6 ++++--
>>        builtin/index-pack.c     |  6 ++++--
>>        builtin/pack-objects.c   | 12 ++++++++----
>>        builtin/unpack-objects.c |  3 ++-
>>        fast-import.c            |  6 ++++--
>>        http-push.c              | 24 ++++++++++++++++--------
>>        http.c                   | 15 ++++++++++-----
>>        imap-send.c              |  3 ++-
>>        ref-filter.c             |  3 ++-
>>        refs/files-backend.c     |  3 ++-
>>        remote-testsvn.c         |  3 ++-
>>        sequencer.c              |  3 ++-
>>        sha1-array.c             |  3 ++-
>>        sha1_file.c              |  3 ++-
>>        transport-helper.c       | 27 ++++++++++++++++++---------
>>        transport.c              |  3 ++-
>>        tree-diff.c              |  6 ++++--
>>        tree.c                   |  3 ++-
>>        20 files changed, 94 insertions(+), 47 deletions(-)
>>
>> These are all cases where we now miss things that should use
>> FREE_AND_NULL(), e.g.:
>>
>>       diff --git a/builtin/am.c b/builtin/am.c
>>       index c973bd96dc..2f89338ed7 100644
>>       --- a/builtin/am.c
>>       +++ b/builtin/am.c
>>       @@ -484,7 +484,8 @@ static int run_applypatch_msg_hook(struct am_state *state)
>>               ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
>>
>>               if (!ret) {
>>       -               FREE_AND_NULL(state->msg);
>>       +               free(state->msg);
>>       +               state->msg = NULL;
>>                       if (read_commit_msg(state) < 0)
>>                               die(_("'%s' was deleted by the applypatch-msg hook"),
>>                                       am_path(state, "final-commit"));
>>
>> So it looks to me like removing the "type T" rule breaks a lot of
>> things, but that the change you made to the expression rule is good, and
>> we should do that for the "type" rule as well. Your commit says the
>> "expression rule already covers it", but this doesn't seem to be the
>> case at all.
>>
>> As an aside: Junio, did you mean to apply f8bb4631fb to next this way?
>> Looks like a mis-applied scissor commit.
> 
> I can't reproduce that strange result with master, but get a scissors
> line into the commit message as well.  The diff at the end looks
> reasonable (contains just the manual stuff).  Here's what I did:

On top of next it looks basically the same for me, only difference
being several new converted cases in repository.c.

Did you commit before diff?

René

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 22:30                                     ` René Scharfe
@ 2017-06-28 22:35                                       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 214+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-28 22:35 UTC (permalink / raw)
  To: René Scharfe; +Cc: Junio C Hamano, Git Mailing List

On Thu, Jun 29, 2017 at 12:30 AM, René Scharfe <l.s.r@web.de> wrote:
> Am 29.06.2017 um 00:21 schrieb René Scharfe:
>>
>> Am 28.06.2017 um 23:39 schrieb Ævar Arnfjörð Bjarmason:
>>>
>>>
>>> On Sun, Jun 25 2017, René Scharfe jotted:
>>>
>>>> Am 16.06.2017 um 21:43 schrieb Junio C Hamano:
>>>>>
>>>>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>>>>
>>>>>> A follow-up to the existing "type" rule added in an earlier
>>>>>> change. This catches some occurrences that are missed by the previous
>>>>>> rule.
>>>>>>
>>>>>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>>>>> ---
>>>>>
>>>>>
>>>>> Hmph, I wonder if the "type" thing is really needed.  Over there,
>>>>> "ptr" is an expression and we can find "free(ptr); ptr = NULL" with
>>>>> the rule in this patch already, no?
>>>>
>>>>
>>>> Indeed.  How about this on top of master?
>>>>
>>>> -- >8 --
>>>> Subject: [PATCH] coccinelle: polish FREE_AND_NULL rules
>>>>
>>>> There are two rules for using FREE_AND_NULL in free.cocci, one for
>>>> pointer types and one for expressions.  Both cause coccinelle to remove
>>>> empty lines and even newline characters between replacements for some
>>>> reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
>>>> FREE_AND_NULL calls on the same time.
>>>>
>>>> Remove the type rule, as the expression rule already covers it, and
>>>> rearrange the lines of the latter to place the addition of FREE_AND_NULL
>>>> between the removals, which causes coccinelle to leave surrounding
>>>> whitespace untouched.
>>>>
>>>> Signed-off-by: Rene Scharfe <l.s.r@web.de>
>>>> ---
>>>>    contrib/coccinelle/free.cocci | 10 +---------
>>>>    1 file changed, 1 insertion(+), 9 deletions(-)
>>>>
>>>> diff --git a/contrib/coccinelle/free.cocci
>>>> b/contrib/coccinelle/free.cocci
>>>> index f2d97e755b..4490069df9 100644
>>>> --- a/contrib/coccinelle/free.cocci
>>>> +++ b/contrib/coccinelle/free.cocci
>>>> @@ -11,16 +11,8 @@ expression E;
>>>>      free(E);
>>>>
>>>>    @@
>>>> -type T;
>>>> -T *ptr;
>>>> -@@
>>>> -- free(ptr);
>>>> -- ptr = NULL;
>>>> -+ FREE_AND_NULL(ptr);
>>>> -
>>>> -@@
>>>>    expression E;
>>>>    @@
>>>>    - free(E);
>>>> -- E = NULL;
>>>>    + FREE_AND_NULL(E);
>>>> +- E = NULL;
>>>
>>>
>>> Late reply, sorry. What version of coccinelle are you running? I have
>>> 1.0.4 (from Debian) and can't get this to produce the same results as
>>> what I have.
>>>
>>> On top of next, I did:
>>>
>>>           Revert "*.[ch] refactoring: make use of the FREE_AND_NULL()
>>> macro"
>>>           Revert "coccinelle: make use of the "expression"
>>> FREE_AND_NULL() rule"
>>>           Revert "coccinelle: make use of the "type" FREE_AND_NULL()
>>> rule"
>>>
>>> And then generated the patch as usual with `make coccicheck`, and
>>> applied it. This has your change.
>>>
>>> I then re-applied the manual "*.[ch] refactoring" changes
>>>
>>> This results in this diff with next:
>>>
>>>       $ git diff --stat origin/next.. -- '*.[ch]'
>>>        builtin/am.c             |  3 ++-
>>>        builtin/clean.c          |  6 ++++--
>>>        builtin/config.c         |  6 ++++--
>>>        builtin/index-pack.c     |  6 ++++--
>>>        builtin/pack-objects.c   | 12 ++++++++----
>>>        builtin/unpack-objects.c |  3 ++-
>>>        fast-import.c            |  6 ++++--
>>>        http-push.c              | 24 ++++++++++++++++--------
>>>        http.c                   | 15 ++++++++++-----
>>>        imap-send.c              |  3 ++-
>>>        ref-filter.c             |  3 ++-
>>>        refs/files-backend.c     |  3 ++-
>>>        remote-testsvn.c         |  3 ++-
>>>        sequencer.c              |  3 ++-
>>>        sha1-array.c             |  3 ++-
>>>        sha1_file.c              |  3 ++-
>>>        transport-helper.c       | 27 ++++++++++++++++++---------
>>>        transport.c              |  3 ++-
>>>        tree-diff.c              |  6 ++++--
>>>        tree.c                   |  3 ++-
>>>        20 files changed, 94 insertions(+), 47 deletions(-)
>>>
>>> These are all cases where we now miss things that should use
>>> FREE_AND_NULL(), e.g.:
>>>
>>>       diff --git a/builtin/am.c b/builtin/am.c
>>>       index c973bd96dc..2f89338ed7 100644
>>>       --- a/builtin/am.c
>>>       +++ b/builtin/am.c
>>>       @@ -484,7 +484,8 @@ static int run_applypatch_msg_hook(struct
>>> am_state *state)
>>>               ret = run_hook_le(NULL, "applypatch-msg", am_path(state,
>>> "final-commit"), NULL);
>>>
>>>               if (!ret) {
>>>       -               FREE_AND_NULL(state->msg);
>>>       +               free(state->msg);
>>>       +               state->msg = NULL;
>>>                       if (read_commit_msg(state) < 0)
>>>                               die(_("'%s' was deleted by the
>>> applypatch-msg hook"),
>>>                                       am_path(state, "final-commit"));
>>>
>>> So it looks to me like removing the "type T" rule breaks a lot of
>>> things, but that the change you made to the expression rule is good, and
>>> we should do that for the "type" rule as well. Your commit says the
>>> "expression rule already covers it", but this doesn't seem to be the
>>> case at all.
>>>
>>> As an aside: Junio, did you mean to apply f8bb4631fb to next this way?
>>> Looks like a mis-applied scissor commit.
>>
>>
>> I can't reproduce that strange result with master, but get a scissors
>> line into the commit message as well.  The diff at the end looks
>> reasonable (contains just the manual stuff).  Here's what I did:
>
>
> On top of next it looks basically the same for me, only difference
> being several new converted cases in repository.c.
>
> Did you commit before diff?

At this point I have no idea what I did wrong, and no time/want to
really find out. But whatever it was was the wrong thing, obviously :)

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 22:21                                   ` René Scharfe
  2017-06-28 22:30                                     ` Ævar Arnfjörð Bjarmason
  2017-06-28 22:30                                     ` René Scharfe
@ 2017-06-28 22:44                                     ` René Scharfe
  2 siblings, 0 replies; 214+ messages in thread
From: René Scharfe @ 2017-06-28 22:44 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Junio C Hamano, git

Am 29.06.2017 um 00:21 schrieb René Scharfe:
> $ git am
> # pasted my email
> Applying: coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
> # results in commit message with scissor line, interesting..

"git am --scissors" commits with the correct message.  I'll do a
"git config --local --add mailinfo.scissors true" real quick..

René

^ permalink raw reply	[flat|nested] 214+ messages in thread

* Re: [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
  2017-06-28 22:30                                     ` Ævar Arnfjörð Bjarmason
@ 2017-06-28 23:15                                       ` Junio C Hamano
  0 siblings, 0 replies; 214+ messages in thread
From: Junio C Hamano @ 2017-06-28 23:15 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: René Scharfe, git

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> I get exactly the same thing as you do below when following these
> steps. So it seems your patch in
> 2122b01f-7627-cd1b-c7df-751c076f90ff@web.de is just fine as-is and I
> just screwed something up when testing this.
>
> Sorry about the noise. Since this works your original patch is obviously
> preferrable since it's not duplicating the rule.

OK.  Unfortunately I screwed up and merged the revert already to
'next'.  So I'll queue René's original again to 'next' and we'll
only have one rule at the end.

Thanks, both.

^ permalink raw reply	[flat|nested] 214+ messages in thread

end of thread, other threads:[~2017-06-28 23:16 UTC | newest]

Thread overview: 214+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-31 21:43 [PATCH 00/31] repository object Brandon Williams
2017-05-31 21:43 ` [PATCH 01/31] config: create config.h Brandon Williams
2017-05-31 21:43 ` [PATCH 02/31] config: don't include config.h by default Brandon Williams
2017-05-31 21:43 ` [PATCH 03/31] config: don't implicitly use gitdir Brandon Williams
2017-06-08 10:41   ` Johannes Schindelin
2017-06-08 16:37     ` Brandon Williams
2017-06-08 18:58       ` Johannes Schindelin
2017-06-08 19:19         ` Brandon Williams
2017-05-31 21:43 ` [PATCH 04/31] setup: don't perform lazy initialization of repository state Brandon Williams
2017-06-01 19:23   ` Stefan Beller
2017-06-02 18:39     ` Jeff King
2017-06-05 17:43       ` Brandon Williams
2017-06-05 18:20         ` Jeff King
2017-06-05 18:44           ` Brandon Williams
2017-05-31 21:43 ` [PATCH 05/31] environment: remove namespace_len variable Brandon Williams
2017-06-01 19:28   ` Stefan Beller
2017-06-01 21:09     ` Brandon Williams
2017-05-31 21:43 ` [PATCH 06/31] repo: introduce the repository object Brandon Williams
2017-06-01 19:53   ` Stefan Beller
2017-06-05 17:53     ` Brandon Williams
2017-06-05 18:31       ` Stefan Beller
2017-05-31 21:43 ` [PATCH 07/31] environment: place key repository state in the_repository Brandon Williams
2017-05-31 21:43 ` [PATCH 08/31] environment: store worktree " Brandon Williams
2017-05-31 21:43 ` [PATCH 09/31] setup: add comment indicating a hack Brandon Williams
2017-05-31 21:43 ` [PATCH 10/31] config: migrate the_configset to the_repository Brandon Williams
2017-05-31 21:43 ` [PATCH 11/31] repo: add index_state to struct repo Brandon Williams
2017-05-31 21:43 ` [PATCH 12/31] submodule-config: store the_submodule_cache in the_repository Brandon Williams
2017-05-31 21:43 ` [PATCH 13/31] repo: add repo_read_gitmodules Brandon Williams
2017-05-31 21:44 ` [PATCH 14/31] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
2017-05-31 21:44 ` [PATCH 15/31] convert: convert get_cached_convert_stats_ascii to take an index Brandon Williams
2017-05-31 21:44 ` [PATCH 16/31] convert: convert crlf_to_git " Brandon Williams
2017-05-31 21:44 ` [PATCH 17/31] convert: convert convert_to_git_filter_fd " Brandon Williams
2017-05-31 21:44 ` [PATCH 18/31] convert: convert convert_to_git " Brandon Williams
2017-05-31 21:44 ` [PATCH 19/31] convert: convert renormalize_buffer " Brandon Williams
2017-05-31 21:44 ` [PATCH 20/31] tree: convert read_tree to take an index parameter Brandon Williams
2017-05-31 21:44 ` [PATCH 21/31] ls-files: convert overlay_tree_on_cache to take an index Brandon Williams
2017-05-31 21:44 ` [PATCH 22/31] ls-files: convert write_eolinfo " Brandon Williams
2017-05-31 21:44 ` [PATCH 23/31] ls-files: convert show_killed_files " Brandon Williams
2017-05-31 21:44 ` [PATCH 24/31] ls-files: convert show_other_files " Brandon Williams
2017-05-31 21:44 ` [PATCH 25/31] ls-files: convert show_ru_info " Brandon Williams
2017-05-31 21:44 ` [PATCH 26/31] ls-files: convert ce_excluded " Brandon Williams
2017-05-31 21:44 ` [PATCH 27/31] ls-files: convert prune_cache " Brandon Williams
2017-05-31 21:44 ` [PATCH 28/31] ls-files: convert show_files " Brandon Williams
2017-05-31 21:44 ` [PATCH 29/31] ls-files: factor out debug info into a function Brandon Williams
2017-05-31 21:44 ` [PATCH 30/31] ls-files: factor out tag calculation Brandon Williams
2017-05-31 21:44 ` [PATCH 31/31] ls-files: use repository object Brandon Williams
2017-06-01 20:36   ` Stefan Beller
2017-06-05 17:46     ` Brandon Williams
2017-05-31 22:56 ` [PATCH 00/31] " Stefan Beller
2017-05-31 23:01   ` Brandon Williams
2017-06-01 18:10 ` Brandon Williams
2017-06-01 18:28   ` Stefan Beller
2017-06-01 20:03     ` Jacob Keller
2017-06-08 23:40 ` [PATCH v2 00/32] " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 01/32] config: create config.h Brandon Williams
2017-06-08 23:40   ` [PATCH v2 02/32] config: remove git_config_iter Brandon Williams
2017-06-08 23:40   ` [PATCH v2 03/32] config: don't include config.h by default Brandon Williams
2017-06-08 23:40   ` [PATCH v2 04/32] config: don't implicitly use gitdir Brandon Williams
2017-06-12 19:57     ` Brandon Williams
2017-06-08 23:40   ` [PATCH v2 05/32] setup: don't perform lazy initialization of repository state Brandon Williams
2017-06-08 23:40   ` [PATCH v2 06/32] environment: remove namespace_len variable Brandon Williams
2017-06-08 23:40   ` [PATCH v2 07/32] repository: introduce the repository object Brandon Williams
2017-06-09  8:53     ` [PATCH 0/2] Add a freez() wrapper Ævar Arnfjörð Bjarmason
2017-06-09 14:53       ` Brandon Williams
2017-06-09 22:04       ` [PATCH v2 0/2] Add a FREEZ() wrapper macro Ævar Arnfjörð Bjarmason
2017-06-09 22:04       ` [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
2017-06-09 22:27         ` Jonathan Nieder
2017-06-09 23:37           ` Eric Wong
2017-06-10  1:40             ` Junio C Hamano
2017-06-10  3:21               ` Eric Wong
2017-06-10  7:25                 ` Jeff King
2017-06-15 16:48                   ` Junio C Hamano
2017-06-15 17:13                     ` Ævar Arnfjörð Bjarmason
2017-06-15 21:06                       ` [PATCH v3 0/2] Add a FREE_AND_NULL() wrapper macro Ævar Arnfjörð Bjarmason
2017-06-15 22:00                         ` Junio C Hamano
2017-06-15 23:15                           ` [PATCH v4 0/6] " Ævar Arnfjörð Bjarmason
2017-06-15 23:15                           ` [PATCH v4 1/6] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
2017-06-15 23:15                           ` [PATCH v4 2/6] coccinelle: add a rule to make "type" code use FREE_AND_NULL() Ævar Arnfjörð Bjarmason
2017-06-15 23:15                           ` [PATCH v4 3/6] coccinelle: make use of the "type" FREE_AND_NULL() rule Ævar Arnfjörð Bjarmason
2017-06-15 23:15                           ` [PATCH v4 4/6] coccinelle: add a rule to make "expression" code use FREE_AND_NULL() Ævar Arnfjörð Bjarmason
2017-06-16 19:43                             ` Junio C Hamano
2017-06-25  8:01                               ` René Scharfe
2017-06-28 21:39                                 ` Ævar Arnfjörð Bjarmason
2017-06-28 22:00                                   ` Junio C Hamano
2017-06-28 22:17                                     ` Ævar Arnfjörð Bjarmason
2017-06-28 22:21                                   ` René Scharfe
2017-06-28 22:30                                     ` Ævar Arnfjörð Bjarmason
2017-06-28 23:15                                       ` Junio C Hamano
2017-06-28 22:30                                     ` René Scharfe
2017-06-28 22:35                                       ` Ævar Arnfjörð Bjarmason
2017-06-28 22:44                                     ` René Scharfe
2017-06-15 23:15                           ` [PATCH v4 5/6] coccinelle: make use of the "expression" FREE_AND_NULL() rule Ævar Arnfjörð Bjarmason
2017-06-15 23:15                           ` [PATCH v4 6/6] *.[ch] refactoring: make use of the FREE_AND_NULL() macro Ævar Arnfjörð Bjarmason
2017-06-15 21:06                       ` [PATCH v3 1/2] git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL Ævar Arnfjörð Bjarmason
2017-06-15 21:07                       ` [PATCH v3 2/2] *.[ch] refactoring: make use of the FREE_AND_NULL() macro Ævar Arnfjörð Bjarmason
2017-06-10  6:55           ` [PATCH v2 1/2] git-compat-util: add a FREEZ() wrapper around free(ptr); ptr = NULL Andreas Schwab
2017-06-09 22:04       ` [PATCH v2 2/2] *.[ch] refactoring: make use of the FREEZ() macro Ævar Arnfjörð Bjarmason
2017-06-09  8:53     ` [PATCH 1/2] git-compat-util: add a freez() wrapper around free(x); x = NULL Ævar Arnfjörð Bjarmason
2017-06-09 12:04       ` Christian Couder
2017-06-09  8:53     ` [PATCH 2/2] *.[ch] refactoring: make use of the freez() wrapper Ævar Arnfjörð Bjarmason
2017-06-09 10:12       ` Martin Ågren
2017-06-09 10:58         ` Ævar Arnfjörð Bjarmason
2017-06-09 14:48           ` Brandon Williams
2017-06-08 23:40   ` [PATCH v2 08/32] environment: place key repository state in the_repository Brandon Williams
2017-06-08 23:40   ` [PATCH v2 09/32] environment: store worktree " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 10/32] setup: add comment indicating a hack Brandon Williams
2017-06-08 23:40   ` [PATCH v2 11/32] config: read config from a repository object Brandon Williams
2017-06-08 23:40   ` [PATCH v2 12/32] repository: add index_state to struct repo Brandon Williams
2017-06-08 23:40   ` [PATCH v2 13/32] submodule-config: store the_submodule_cache in the_repository Brandon Williams
2017-06-08 23:40   ` [PATCH v2 14/32] submodule: add repo_read_gitmodules Brandon Williams
2017-06-08 23:40   ` [PATCH v2 15/32] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
2017-06-08 23:40   ` [PATCH v2 16/32] convert: convert get_cached_convert_stats_ascii to take an index Brandon Williams
2017-06-08 23:40   ` [PATCH v2 17/32] convert: convert crlf_to_git " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 18/32] convert: convert convert_to_git_filter_fd " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 19/32] convert: convert convert_to_git " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 20/32] convert: convert renormalize_buffer " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 21/32] tree: convert read_tree to take an index parameter Brandon Williams
2017-06-08 23:40   ` [PATCH v2 22/32] ls-files: convert overlay_tree_on_cache to take an index Brandon Williams
2017-06-08 23:40   ` [PATCH v2 23/32] ls-files: convert write_eolinfo " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 24/32] ls-files: convert show_killed_files " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 25/32] ls-files: convert show_other_files " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 26/32] ls-files: convert show_ru_info " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 27/32] ls-files: convert ce_excluded " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 28/32] ls-files: convert prune_cache " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 29/32] ls-files: convert show_files " Brandon Williams
2017-06-08 23:40   ` [PATCH v2 30/32] ls-files: factor out debug info into a function Brandon Williams
2017-06-08 23:40   ` [PATCH v2 31/32] ls-files: factor out tag calculation Brandon Williams
2017-06-08 23:41   ` [PATCH v2 32/32] ls-files: use repository object Brandon Williams
2017-06-09  0:08     ` Brandon Williams
2017-06-10  5:59       ` Junio C Hamano
2017-06-12 21:24         ` Brandon Williams
2017-06-10  0:40   ` [PATCH v2 00/32] " Jonathan Tan
2017-06-10  6:07     ` Jeff King
2017-06-10  6:13       ` Jeff King
2017-06-11  0:43         ` Brandon Williams
2017-06-12 19:10           ` Jonathan Tan
2017-06-11  0:35       ` Brandon Williams
2017-06-12  5:24       ` Stefan Beller
2017-06-12 21:23         ` Jeff King
2017-06-12 19:01       ` Jonathan Tan
2017-06-12 19:11         ` Brandon Williams
2017-06-12 20:04           ` Jonathan Tan
2017-06-12 21:28             ` Jeff King
2017-06-20 19:19   ` [PATCH v3 00/20] " Brandon Williams
2017-06-20 19:19     ` [PATCH v3 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
2017-06-20 19:19     ` [PATCH v3 02/20] setup: add comment indicating a hack Brandon Williams
2017-06-20 19:19     ` [PATCH v3 03/20] environment: remove namespace_len variable Brandon Williams
2017-06-20 19:19     ` [PATCH v3 04/20] repository: introduce the repository object Brandon Williams
2017-06-20 19:57       ` Stefan Beller
2017-06-21 21:27         ` Brandon Williams
2017-06-21 21:31           ` Stefan Beller
2017-06-20 21:31       ` Jonathan Tan
2017-06-20 21:56         ` Brandon Williams
2017-06-20 19:19     ` [PATCH v3 05/20] environment: place key repository state in the_repository Brandon Williams
2017-06-20 21:59       ` Jonathan Tan
2017-06-20 19:19     ` [PATCH v3 06/20] environment: store worktree " Brandon Williams
2017-06-20 19:19     ` [PATCH v3 07/20] path: create path.h Brandon Williams
2017-06-20 19:19     ` [PATCH v3 08/20] path: always pass in commondir to update_common_dir Brandon Williams
2017-06-20 19:19     ` [PATCH v3 09/20] path: convert strbuf_git_common_path to take a 'struct repository' Brandon Williams
2017-06-20 19:19     ` [PATCH v3 10/20] path: convert do_git_path " Brandon Williams
2017-06-20 22:23       ` Jonathan Tan
2017-06-21 21:20         ` Brandon Williams
2017-06-20 19:19     ` [PATCH v3 11/20] path: construct correct path to a worktree's index Brandon Williams
2017-06-20 21:10       ` Stefan Beller
2017-06-20 23:02       ` Jonathan Nieder
2017-06-21  0:39         ` Brandon Williams
2017-06-21  2:10       ` Jonathan Nieder
2017-06-21  2:30         ` Jonathan Nieder
2017-06-21 15:43         ` Brandon Williams
2017-06-21 17:57           ` Jonathan Nieder
2017-06-21 18:48             ` Junio C Hamano
2017-06-20 19:19     ` [PATCH v3 12/20] path: add repo_git_path and strbuf_repo_git_path Brandon Williams
2017-06-20 19:19     ` [PATCH v3 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path Brandon Williams
2017-06-20 19:19     ` [PATCH v3 14/20] config: read config from a repository object Brandon Williams
2017-06-20 19:19     ` [PATCH v3 15/20] repository: add index_state to struct repo Brandon Williams
2017-06-21 22:50       ` Jonathan Tan
2017-06-21 22:54         ` Brandon Williams
2017-06-21 23:00           ` Stefan Beller
2017-06-20 19:19     ` [PATCH v3 16/20] submodule-config: store the_submodule_cache in the_repository Brandon Williams
2017-06-20 19:19     ` [PATCH v3 17/20] submodule: add repo_read_gitmodules Brandon Williams
2017-06-20 19:19     ` [PATCH v3 18/20] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
2017-06-20 19:19     ` [PATCH v3 19/20] repository: enable initialization of submodules Brandon Williams
2017-06-21 23:00       ` Jonathan Tan
2017-06-21 23:09         ` Brandon Williams
2017-06-20 19:19     ` [PATCH v3 20/20] ls-files: use repository object Brandon Williams
2017-06-21 22:48       ` Jonathan Tan
2017-06-20 19:23     ` [PATCH v3 00/20] " Stefan Beller
2017-06-22 18:43     ` [PATCH v4 " Brandon Williams
2017-06-22 18:43       ` [PATCH v4 01/20] setup: don't perform lazy initialization of repository state Brandon Williams
2017-06-22 18:43       ` [PATCH v4 02/20] setup: add comment indicating a hack Brandon Williams
2017-06-22 18:43       ` [PATCH v4 03/20] environment: remove namespace_len variable Brandon Williams
2017-06-22 18:43       ` [PATCH v4 04/20] repository: introduce the repository object Brandon Williams
2017-06-22 18:43       ` [PATCH v4 05/20] environment: place key repository state in the_repository Brandon Williams
2017-06-22 18:43       ` [PATCH v4 06/20] environment: store worktree " Brandon Williams
2017-06-22 18:43       ` [PATCH v4 07/20] path: create path.h Brandon Williams
2017-06-22 18:43       ` [PATCH v4 08/20] path: always pass in commondir to update_common_dir Brandon Williams
2017-06-22 18:43       ` [PATCH v4 09/20] path: convert strbuf_git_common_path to take a 'struct repository' Brandon Williams
2017-06-22 18:43       ` [PATCH v4 10/20] path: convert do_git_path " Brandon Williams
2017-06-22 18:43       ` [PATCH v4 11/20] path: worktree_git_path() should not use file relocation Brandon Williams
2017-06-22 18:43       ` [PATCH v4 12/20] path: add repo_git_path and strbuf_repo_git_path Brandon Williams
2017-06-22 18:43       ` [PATCH v4 13/20] path: add repo_worktree_path and strbuf_repo_worktree_path Brandon Williams
2017-06-22 18:43       ` [PATCH v4 14/20] config: read config from a repository object Brandon Williams
2017-06-22 18:43       ` [PATCH v4 15/20] repository: add index_state to struct repo Brandon Williams
2017-06-22 20:16         ` Junio C Hamano
2017-06-22 20:35           ` Brandon Williams
2017-06-22 22:10             ` Junio C Hamano
2017-06-22 18:43       ` [PATCH v4 16/20] submodule-config: store the_submodule_cache in the_repository Brandon Williams
2017-06-22 18:43       ` [PATCH v4 17/20] submodule: add repo_read_gitmodules Brandon Williams
2017-06-22 18:43       ` [PATCH v4 18/20] submodule: convert is_submodule_initialized to work on a repository Brandon Williams
2017-06-22 18:43       ` [PATCH v4 19/20] repository: enable initialization of submodules Brandon Williams
2017-06-22 18:43       ` [PATCH v4 20/20] ls-files: use repository object Brandon Williams
2017-06-22 19:42       ` [PATCH v4 00/20] " Stefan Beller
2017-06-23 16:44       ` Jeff Hostetler
2017-06-23 17:38         ` Junio C Hamano

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).