[PATCH/RFC 0/7] Pie-in-the-sky attempt to fix the early config

[PATCH/RFC 0/7] Pie-in-the-sky attempt to fix the early config

[Johannes Schindelin]

Hopefully these patches will lead to something that we can integrate,
and that eventually will make Git's startup sequence much less
surprising.

The idea here is to discover the .git/ directory gently (i.e. without
changing the current working directory), and to use it to read the
.git/config file early, before we actually called setup_git_directory()
(if we ever do that).

Notes:

- I find the diff pretty ugly: I wish there was a more elegant way to
  *disable* discovery of .git/ *just* for `init` and `clone`. I
  considered a function `about_to_create_git_dir()` that is called in a
  hard-coded manner *only* for `init` and `clone`, but that would
  introduce another magic side effect, when all I want is to reduce those.

- For the moment, I do not handle dashed invocations of `init` and
  `clone` correctly. The real problem is the continued existence of
  the dashed git-init and git-clone, of course.

- There is still duplicated code. For the sake of this RFC, I did not
  address that yet.

- The read_early_config() function is called multiple times, re-reading
  all the config files and re-discovering the .git/ directory multiple
  times, which is quite wasteful. For the sake of this RFC, I did not
  address that yet.

- t7006 fails and the error message is a bit cryptic (not to mention the
  involved function trace, which is mind-boggling for what is supposed
  to be a simply, shell script-based test suite). I did not have time to
  look into that yet.

- after discover_git_directory_gently() did its work, the code happily
  uses its result *only* for the current read_early_config() run, and
  lets setup_git_dir_gently() do the whole work *again*. For the sake of
  this RFC, I did not address that yet.


Johannes Schindelin (7):
  Make read_early_config() reusable
  read_early_config(): avoid .git/config hack when unneeded
  Mark builtins that create .git/ directories
  read_early_config(): special-case `init` and `clone`
  read_early_config(): really discover .git/
  WIP read_config_early(): respect ceiling directories
  WIP: read_early_config(): add tests

 builtin/am.c            |   2 +-
 builtin/blame.c         |   2 +-
 builtin/grep.c          |   4 +-
 builtin/log.c           |   4 +-
 builtin/var.c           |   2 +-
 cache.h                 |   8 ++--
 config.c                | 110 ++++++++++++++++++++++++++++++++++++++++++++++++
 diff.c                  |   4 +-
 git.c                   |  25 +++++------
 pager.c                 |  44 +++----------------
 t/helper/test-config.c  |  15 +++++++
 t/t1309-early-config.sh |  50 ++++++++++++++++++++++
 12 files changed, 209 insertions(+), 61 deletions(-)
 create mode 100755 t/t1309-early-config.sh


base-commit: 8d7a455ed52e2a96debc080dfc011b6bb00db5d2
Published-As: https://github.com/dscho/git/releases/tag/early-config-v1
Fetch-It-Via: git fetch https://github.com/dscho/git early-config-v1

-- 
2.11.0.rc3.windows.1

[PATCH/RFC 1/7] Make read_early_config() reusable

[Johannes Schindelin]

The pager configuration needs to be read early, possibly before
discovering any .git/ directory.

Let's not hide this function in pager.c, but make it available to other
callers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h  |  1 +
 config.c | 31 +++++++++++++++++++++++++++++++
 pager.c  | 31 -------------------------------
 3 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/cache.h b/cache.h
index a50a61a197..96e0cb2523 100644
--- a/cache.h
+++ b/cache.h
@@ -1692,6 +1692,7 @@ extern int git_config_from_mem(config_fn_t fn, const enum config_origin_type,
 					const char *name, const char *buf, size_t len, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
diff --git a/config.c b/config.c
index 83fdecb1bc..7dcd8d8622 100644
--- a/config.c
+++ b/config.c
@@ -1385,6 +1385,37 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
+void read_early_config(config_fn_t cb, void *data)
+{
+	git_config_with_options(cb, data, NULL, 1);
+
+	/*
+	 * Note that this is a really dirty hack that does the wrong thing in
+	 * many cases. The crux of the problem is that we cannot run
+	 * setup_git_directory() early on in git's setup, so we have no idea if
+	 * we are in a repository or not, and therefore are not sure whether
+	 * and how to read repository-local config.
+	 *
+	 * So if we _aren't_ in a repository (or we are but we would reject its
+	 * core.repositoryformatversion), we'll read whatever is in .git/config
+	 * blindly. Similarly, if we _are_ in a repository, but not at the
+	 * root, we'll fail to find .git/config (because it's really
+	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 *
+	 * However, we have historically provided this hack because it does
+	 * work some of the time (namely when you are at the top-level of a
+	 * valid repository), and would rarely make things worse (i.e., you do
+	 * not generally have a .git/config file sitting around).
+	 */
+	if (!startup_info->have_repository) {
+		struct git_config_source repo_config;
+
+		memset(&repo_config, 0, sizeof(repo_config));
+		repo_config.file = ".git/config";
+		git_config_with_options(cb, data, &repo_config, 1);
+	}
+}
+
 static void git_config_check_init(void);
 
 void git_config(config_fn_t fn, void *data)
diff --git a/pager.c b/pager.c
index ae79643363..73ca8bc3b1 100644
--- a/pager.c
+++ b/pager.c
@@ -43,37 +43,6 @@ static int core_pager_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
-static void read_early_config(config_fn_t cb, void *data)
-{
-	git_config_with_options(cb, data, NULL, 1);
-
-	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
-	 */
-	if (!startup_info->have_repository) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
-		git_config_with_options(cb, data, &repo_config, 1);
-	}
-}
-
 const char *git_pager(int stdout_is_tty)
 {
 	const char *pager;
-- 
2.11.0.rc3.windows.1

[PATCH/RFC 2/7] read_early_config(): avoid .git/config hack when unneeded

[Johannes Schindelin]

So far, we only look whether the startup_info claims to have seen a
git_dir.

However, do_git_config_sequence() (and consequently the
git_config_with_options() call used by read_early_config() asks the
have_git_dir() function whether we have a .git/ directory, which in turn
also looks at git_dir and at the environment variable GIT_DIR.

Let's just use the same function, have_git_dir(), to determine whether we
have to look for .git/config specifically.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/config.c b/config.c
index 7dcd8d8622..104c3c3dcd 100644
--- a/config.c
+++ b/config.c
@@ -1407,7 +1407,7 @@ void read_early_config(config_fn_t cb, void *data)
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->have_repository) {
+	if (!have_git_dir()) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-- 
2.11.0.rc3.windows.1

[PATCH/RFC 3/7] Mark builtins that create .git/ directories

[Johannes Schindelin]

To refactor read_early_config() so that it discovers .git/config
properly, we have to make certain that commands such as `git init` (i.e.
commands that create their own .git/ and therefore do *not* want to
be affected by any other .git/ directory) skip this discovery.

Let's introduce a flag that states for every builtin whether it creates
its own .git/ directory or not.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/git.c b/git.c
index dce529fcbf..61df78afc8 100644
--- a/git.c
+++ b/git.c
@@ -318,12 +318,13 @@ static int handle_alias(int *argcp, const char ***argv)
 #define RUN_SETUP		(1<<0)
 #define RUN_SETUP_GENTLY	(1<<1)
 #define USE_PAGER		(1<<2)
+#define CREATES_GIT_DIR         (1<<3)
 /*
  * require working tree to be present -- anything uses this needs
  * RUN_SETUP for reading from the configuration file.
  */
-#define NEED_WORK_TREE		(1<<3)
-#define SUPPORT_SUPER_PREFIX	(1<<4)
+#define NEED_WORK_TREE		(1<<4)
+#define SUPPORT_SUPER_PREFIX	(1<<5)
 
 struct cmd_struct {
 	const char *cmd;
@@ -412,7 +413,7 @@ static struct cmd_struct commands[] = {
 	{ "cherry", cmd_cherry, RUN_SETUP },
 	{ "cherry-pick", cmd_cherry_pick, RUN_SETUP | NEED_WORK_TREE },
 	{ "clean", cmd_clean, RUN_SETUP | NEED_WORK_TREE },
-	{ "clone", cmd_clone },
+	{ "clone", cmd_clone, CREATES_GIT_DIR },
 	{ "column", cmd_column, RUN_SETUP_GENTLY },
 	{ "commit", cmd_commit, RUN_SETUP | NEED_WORK_TREE },
 	{ "commit-tree", cmd_commit_tree, RUN_SETUP },
@@ -438,7 +439,7 @@ static struct cmd_struct commands[] = {
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY },
-	{ "init", cmd_init_db },
+	{ "init", cmd_init_db, CREATES_GIT_DIR },
 	{ "init-db", cmd_init_db },
 	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY },
 	{ "log", cmd_log, RUN_SETUP },
-- 
2.11.0.rc3.windows.1

[PATCH/RFC 4/7] read_early_config(): special-case `init` and `clone`

[Johannes Schindelin]

The `init` and `clone` commands create their own .git/ directory,
therefore we must be careful not to read any repository config when
determining the pager settings.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/am.c    |  2 +-
 builtin/blame.c |  2 +-
 builtin/grep.c  |  4 ++--
 builtin/log.c   |  4 ++--
 builtin/var.c   |  2 +-
 cache.h         |  9 +++++----
 config.c        |  4 ++--
 diff.c          |  4 ++--
 git.c           | 16 ++++++++--------
 pager.c         | 13 +++++++------
 10 files changed, 31 insertions(+), 29 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 6981f42ce9..e6c2ee01bc 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1791,7 +1791,7 @@ static int do_interactive(struct am_state *state)
 			}
 			strbuf_release(&msg);
 		} else if (*reply == 'v' || *reply == 'V') {
-			const char *pager = git_pager(1);
+			const char *pager = git_pager(1, 1);
 			struct child_process cp = CHILD_PROCESS_INIT;
 
 			if (!pager)
diff --git a/builtin/blame.c b/builtin/blame.c
index 4ddfadb71f..5b7daa3686 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -2913,7 +2913,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
 	assign_blame(&sb, opt);
 
 	if (!incremental)
-		setup_pager();
+		setup_pager(1);
 
 	free(final_commit_name);
 
diff --git a/builtin/grep.c b/builtin/grep.c
index 8887b6addb..363a753369 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -800,7 +800,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	}
 
 	if (show_in_pager == default_pager)
-		show_in_pager = git_pager(1);
+		show_in_pager = git_pager(1, 1);
 	if (show_in_pager) {
 		opt.color = 0;
 		opt.name_only = 1;
@@ -896,7 +896,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	}
 
 	if (!show_in_pager && !opt.status_only)
-		setup_pager();
+		setup_pager(1);
 
 	if (!use_index && (untracked || cached))
 		die(_("--cached or --untracked cannot be used with --no-index."));
diff --git a/builtin/log.c b/builtin/log.c
index 55d20cc2d8..96618d38cb 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -203,7 +203,7 @@ static void cmd_log_init_finish(int argc, const char **argv, const char *prefix,
 	if (rev->line_level_traverse)
 		line_log_init(rev, line_cb.prefix, &line_cb.args);
 
-	setup_pager();
+	setup_pager(1);
 }
 
 static void cmd_log_init(int argc, const char **argv, const char *prefix,
@@ -1600,7 +1600,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_stdout)
 		output_directory = set_outdir(prefix, output_directory);
 	else
-		setup_pager();
+		setup_pager(1);
 
 	if (output_directory) {
 		if (rev.diffopt.use_color != GIT_COLOR_ALWAYS)
diff --git a/builtin/var.c b/builtin/var.c
index aedbb53a2d..879867b842 100644
--- a/builtin/var.c
+++ b/builtin/var.c
@@ -19,7 +19,7 @@ static const char *editor(int flag)
 
 static const char *pager(int flag)
 {
-	const char *pgm = git_pager(1);
+	const char *pgm = git_pager(1, 1);
 
 	if (!pgm)
 		pgm = "cat";
diff --git a/cache.h b/cache.h
index 96e0cb2523..6627239de8 100644
--- a/cache.h
+++ b/cache.h
@@ -1325,7 +1325,7 @@ extern const char *fmt_name(const char *name, const char *email);
 extern const char *ident_default_name(void);
 extern const char *ident_default_email(void);
 extern const char *git_editor(void);
-extern const char *git_pager(int stdout_is_tty);
+extern const char *git_pager(int stdout_is_tty, int discover_git_dir);
 extern int git_ident_config(const char *, const char *, void *);
 extern void reset_ident_date(void);
 
@@ -1692,7 +1692,8 @@ extern int git_config_from_mem(config_fn_t fn, const enum config_origin_type,
 					const char *name, const char *buf, size_t len, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
-extern void read_early_config(config_fn_t cb, void *data);
+extern void read_early_config(config_fn_t cb, void *data,
+			      int discover_git_dir);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
@@ -1888,12 +1889,12 @@ __attribute__((format (printf, 2, 3)))
 extern void write_file(const char *path, const char *fmt, ...);
 
 /* pager.c */
-extern void setup_pager(void);
+extern void setup_pager(int discover_git_dir);
 extern int pager_in_use(void);
 extern int pager_use_color;
 extern int term_columns(void);
 extern int decimal_width(uintmax_t);
-extern int check_pager_config(const char *cmd);
+extern int check_pager_config(const char *cmd, int discover_git_dir);
 extern void prepare_pager_args(struct child_process *, const char *pager);
 
 extern const char *editor_program;
diff --git a/config.c b/config.c
index 104c3c3dcd..4c756edca1 100644
--- a/config.c
+++ b/config.c
@@ -1385,7 +1385,7 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
-void read_early_config(config_fn_t cb, void *data)
+void read_early_config(config_fn_t cb, void *data, int discover_git_dir)
 {
 	git_config_with_options(cb, data, NULL, 1);
 
@@ -1407,7 +1407,7 @@ void read_early_config(config_fn_t cb, void *data)
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!have_git_dir()) {
+	if (discover_git_dir && !have_git_dir()) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
diff --git a/diff.c b/diff.c
index ec8728362d..0769c0590a 100644
--- a/diff.c
+++ b/diff.c
@@ -5267,6 +5267,6 @@ void setup_diff_pager(struct diff_options *opt)
 	 * --exit-code" in hooks and other scripts, we do not do so.
 	 */
 	if (!DIFF_OPT_TST(opt, EXIT_WITH_STATUS) &&
-	    check_pager_config("diff") != 0)
-		setup_pager();
+	    check_pager_config("diff", 1) != 0)
+		setup_pager(1);
 }
diff --git a/git.c b/git.c
index 61df78afc8..2b007b83ec 100644
--- a/git.c
+++ b/git.c
@@ -61,13 +61,13 @@ static void restore_env(int external_alias)
 	}
 }
 
-static void commit_pager_choice(void) {
+static void commit_pager_choice(int discover_git_dir) {
 	switch (use_pager) {
 	case 0:
 		setenv("GIT_PAGER", "cat", 1);
 		break;
 	case 1:
-		setup_pager();
+		setup_pager(discover_git_dir);
 		break;
 	default:
 		break;
@@ -261,7 +261,7 @@ static int handle_alias(int *argcp, const char ***argv)
 		if (alias_string[0] == '!') {
 			struct child_process child = CHILD_PROCESS_INIT;
 
-			commit_pager_choice();
+			commit_pager_choice(1);
 			restore_env(1);
 
 			child.use_shell = 1;
@@ -349,7 +349,7 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
 		}
 
 		if (use_pager == -1 && p->option & (RUN_SETUP | RUN_SETUP_GENTLY))
-			use_pager = check_pager_config(p->cmd);
+			use_pager = check_pager_config(p->cmd, !(p->option & CREATES_GIT_DIR));
 		if (use_pager == -1 && p->option & USE_PAGER)
 			use_pager = 1;
 
@@ -357,7 +357,7 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
 		    startup_info->have_repository) /* get_git_dir() may set up repo, avoid that */
 			trace_repo_setup(prefix);
 	}
-	commit_pager_choice();
+	commit_pager_choice(!(p->option & CREATES_GIT_DIR));
 
 	if (!help && get_super_prefix()) {
 		if (!(p->option & SUPPORT_SUPER_PREFIX))
@@ -584,8 +584,8 @@ static void execv_dashed_external(const char **argv)
 		die("%s doesn't support --super-prefix", argv[0]);
 
 	if (use_pager == -1)
-		use_pager = check_pager_config(argv[0]);
-	commit_pager_choice();
+		use_pager = check_pager_config(argv[0], 1);
+	commit_pager_choice(1);
 
 	strbuf_addf(&cmd, "git-%s", argv[0]);
 
@@ -688,7 +688,7 @@ int cmd_main(int argc, const char **argv)
 		skip_prefix(argv[0], "--", &argv[0]);
 	} else {
 		/* The user didn't specify a command; give them help */
-		commit_pager_choice();
+		commit_pager_choice(1);
 		printf("usage: %s\n\n", git_usage_string);
 		list_common_cmds_help();
 		printf("\n%s\n", _(git_more_info_string));
diff --git a/pager.c b/pager.c
index 73ca8bc3b1..16b3cbe232 100644
--- a/pager.c
+++ b/pager.c
@@ -43,7 +43,7 @@ static int core_pager_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
-const char *git_pager(int stdout_is_tty)
+const char *git_pager(int stdout_is_tty, int discover_git_dir)
 {
 	const char *pager;
 
@@ -53,7 +53,8 @@ const char *git_pager(int stdout_is_tty)
 	pager = getenv("GIT_PAGER");
 	if (!pager) {
 		if (!pager_program)
-			read_early_config(core_pager_config, NULL);
+			read_early_config(core_pager_config, NULL,
+					  discover_git_dir);
 		pager = pager_program;
 	}
 	if (!pager)
@@ -100,9 +101,9 @@ void prepare_pager_args(struct child_process *pager_process, const char *pager)
 	setup_pager_env(&pager_process->env_array);
 }
 
-void setup_pager(void)
+void setup_pager(int discover_git_dir)
 {
-	const char *pager = git_pager(isatty(1));
+	const char *pager = git_pager(isatty(1), discover_git_dir);
 
 	if (!pager)
 		return;
@@ -208,7 +209,7 @@ static int pager_command_config(const char *var, const char *value, void *vdata)
 }
 
 /* returns 0 for "no pager", 1 for "use pager", and -1 for "not specified" */
-int check_pager_config(const char *cmd)
+int check_pager_config(const char *cmd, int discover_git_dir)
 {
 	struct pager_command_config_data data;
 
@@ -216,7 +217,7 @@ int check_pager_config(const char *cmd)
 	data.want = -1;
 	data.value = NULL;
 
-	read_early_config(pager_command_config, &data);
+	read_early_config(pager_command_config, &data, discover_git_dir);
 
 	if (data.value)
 		pager_program = data.value;
-- 
2.11.0.rc3.windows.1

[PATCH/RFC 5/7] read_early_config(): really discover .git/

[Johannes Schindelin]

Earlier, we punted and simply assumed that we are in the top-level
directory of the project, and that there is no .git file but a .git/
directory so that we can read directly from .git/config.

Let's discover the .git/ directory correctly.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 69 +++++++++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 49 insertions(+), 20 deletions(-)

diff --git a/config.c b/config.c
index 4c756edca1..286d3cad66 100644
--- a/config.c
+++ b/config.c
@@ -1385,35 +1385,64 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
+/*
+ * Note that this is a really dirty hack that replicates what the
+ * setup_git_directory() function does, without changing the current
+ * working directory. The crux of the problem is that we cannot run
+ * setup_git_directory() early on in git's setup, so we have to
+ * duplicate the work that setup_git_directory() would otherwise do.
+ */
+static int discover_git_directory_gently(struct strbuf *result)
+{
+	const char *p;
+
+	if (strbuf_getcwd(result) < 0)
+		return -1;
+	p = real_path_if_valid(result->buf);
+	if (!p)
+		return -1;
+	strbuf_reset(result);
+	strbuf_addstr(result, p);
+	for (;;) {
+		int len = result->len, i;
+
+		strbuf_addstr(result, "/" DEFAULT_GIT_DIR_ENVIRONMENT);
+		p = read_gitfile_gently(result->buf, &i);
+		if (p) {
+			strbuf_reset(result);
+			strbuf_addstr(result, p);
+			return 0;
+		}
+		if (is_git_directory(result->buf))
+			return 0;
+		strbuf_setlen(result, len);
+		if (is_git_directory(result->buf))
+			return 0;
+		for (i = len; i > 0; )
+			if (is_dir_sep(result->buf[--i]))
+				break;
+		if (!i)
+			return -1;
+		strbuf_setlen(result, i);
+	}
+}
+
 void read_early_config(config_fn_t cb, void *data, int discover_git_dir)
 {
+	struct strbuf buf = STRBUF_INIT;
+
 	git_config_with_options(cb, data, NULL, 1);
 
-	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
-	 */
-	if (discover_git_dir && !have_git_dir()) {
+	if (discover_git_dir && !have_git_dir() &&
+	    !discover_git_directory_gently(&buf)) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
+		strbuf_addstr(&buf, "/config");
+		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, 1);
 	}
+	strbuf_release(&buf);
 }
 
 static void git_config_check_init(void);
-- 
2.11.0.rc3.windows.1

[PATCH/RFC 6/7] WIP read_config_early(): respect ceiling directories

[Johannes Schindelin]

This ports the part from setup_git_directory_gently_1() where the
GIT_CEILING_DIRECTORIES environment variable is handled.

TODO: DRY up code again (exporting canonicalize_ceiling_directories()?)

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 53 insertions(+), 3 deletions(-)

diff --git a/config.c b/config.c
index 286d3cad66..eda3546491 100644
--- a/config.c
+++ b/config.c
@@ -1386,6 +1386,37 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 }
 
 /*
+ * A "string_list_each_func_t" function that canonicalizes an entry
+ * from GIT_CEILING_DIRECTORIES using real_path_if_valid(), or
+ * discards it if unusable.  The presence of an empty entry in
+ * GIT_CEILING_DIRECTORIES turns off canonicalization for all
+ * subsequent entries.
+ */
+static int canonicalize_ceiling_entry(struct string_list_item *item,
+				      void *cb_data)
+{
+	int *empty_entry_found = cb_data;
+	char *ceil = item->string;
+
+	if (!*ceil) {
+		*empty_entry_found = 1;
+		return 0;
+	} else if (!is_absolute_path(ceil)) {
+		return 0;
+	} else if (*empty_entry_found) {
+		/* Keep entry but do not canonicalize it */
+		return 1;
+	} else {
+		const char *real_path = real_path_if_valid(ceil);
+		if (!real_path)
+			return 0;
+		free(item->string);
+		item->string = xstrdup(real_path);
+		return 1;
+	}
+}
+
+/*
  * Note that this is a really dirty hack that replicates what the
  * setup_git_directory() function does, without changing the current
  * working directory. The crux of the problem is that we cannot run
@@ -1394,6 +1425,8 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
  */
 static int discover_git_directory_gently(struct strbuf *result)
 {
+	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
+	int ceiling_offset = -1;
 	const char *p;
 
 	if (strbuf_getcwd(result) < 0)
@@ -1403,6 +1436,23 @@ static int discover_git_directory_gently(struct strbuf *result)
 		return -1;
 	strbuf_reset(result);
 	strbuf_addstr(result, p);
+
+	if (env_ceiling_dirs) {
+		struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
+		int empty_entry_found = 0;
+
+		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP,
+				  -1);
+		filter_string_list(&ceiling_dirs, 0, canonicalize_ceiling_entry,
+				   &empty_entry_found);
+		ceiling_offset = longest_ancestor_length(result->buf,
+							 &ceiling_dirs);
+		string_list_clear(&ceiling_dirs, 0);
+	}
+
+	if (ceiling_offset < 0 && has_dos_drive_prefix(result->buf))
+		ceiling_offset = 1;
+
 	for (;;) {
 		int len = result->len, i;
 
@@ -1418,10 +1468,10 @@ static int discover_git_directory_gently(struct strbuf *result)
 		strbuf_setlen(result, len);
 		if (is_git_directory(result->buf))
 			return 0;
-		for (i = len; i > 0; )
-			if (is_dir_sep(result->buf[--i]))
+		for (i = len; --i > ceiling_offset; )
+			if (is_dir_sep(result->buf[i]))
 				break;
-		if (!i)
+		if (i <= ceiling_offset)
 			return -1;
 		strbuf_setlen(result, i);
 	}
-- 
2.11.0.rc3.windows.1

[PATCH/RFC 7/7] WIP: read_early_config(): add tests

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-config.c  | 15 +++++++++++++++
 t/t1309-early-config.sh | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 t/t1309-early-config.sh

diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 83a4f2ab86..5105069587 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -66,6 +66,16 @@ static int iterate_cb(const char *var, const char *value, void *data)
 	return 0;
 }
 
+static int early_config_cb(const char *var, const char *value, void *vdata)
+{
+	const char *key = vdata;
+
+	if (!strcmp(key, var))
+		printf("%s\n", value);
+
+	return 0;
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	int i, val;
@@ -73,6 +83,11 @@ int cmd_main(int argc, const char **argv)
 	const struct string_list *strptr;
 	struct config_set cs;
 
+	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		read_early_config(early_config_cb, (void *)argv[2], 1);
+		return 0;
+	}
+
 	setup_git_directory();
 
 	git_configset_init(&cs);
diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
new file mode 100755
index 0000000000..0c55dee514
--- /dev/null
+++ b/t/t1309-early-config.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+test_description='Test read_early_config()'
+
+. ./test-lib.sh
+
+test_expect_success 'read early config' '
+	test_config early.config correct &&
+	test-config read_early_config early.config >output &&
+	test correct = "$(cat output)"
+'
+
+test_expect_success 'in a sub-directory' '
+	test_config early.config sub &&
+	mkdir -p sub &&
+	(
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test sub = "$(cat output)"
+'
+
+test_expect_success 'ceiling' '
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test -z "$(cat output)"
+'
+
+test_expect_success 'ceiling #2' '
+	mkdir -p xdg/git &&
+	git config -f xdg/git/config early.config xdg &&
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		XDG_CONFIG_HOME="$PWD"/xdg &&
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES XDG_CONFIG_HOME &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test xdg = "$(cat output)"
+'
+
+test_done
-- 
2.11.0.rc3.windows.1

Re: [PATCH/RFC 0/7] Pie-in-the-sky attempt to fix the early config

[Jeff King]

On Thu, Dec 08, 2016 at 04:35:56PM +0100, Johannes Schindelin wrote:

> The idea here is to discover the .git/ directory gently (i.e. without
> changing the current working directory), and to use it to read the
> .git/config file early, before we actually called setup_git_directory()
> (if we ever do that).

Great. Thanks for taking a stab at this.

> Notes:
> 
> - I find the diff pretty ugly: I wish there was a more elegant way to
>   *disable* discovery of .git/ *just* for `init` and `clone`. I
>   considered a function `about_to_create_git_dir()` that is called in a
>   hard-coded manner *only* for `init` and `clone`, but that would
>   introduce another magic side effect, when all I want is to reduce those.

It looks like a lot of that ugliness comes from passing around the "are
we OK to peek at git-dir config" flag through the various pager-related
calls.  I don't think it would be bad to use a global for "we do not
want a repo".  After all, it's just modifying the _existing_ global for
"are we in a repo". And I do not see that global going away anytime
soon. Sometimes it's good to make incremental steps towards an end goal,
but in this case it seems like we just pay the cost of the step without
any real plan for reaping the benefit in the long term.

As an alternative, I also think it would be OK to just always have the
pager config read from local repo, even for init/clone. In other words,
to loosen the idea that git-init can _never_ look in the current
git-dir, and declare that there is a stage before the command is
initiated, and during which git may read local-repo config. Aliases
would fall into this, too, so:

  git config --local alias.foo init
  git foo /some/other/dir

would work (as it must, because we cannot know that "foo" is "init"
until we read the config!).

I have a feeling you may declare that too magical, but I think it's
consistent and practical.

> - For the moment, I do not handle dashed invocations of `init` and
>   `clone` correctly. The real problem is the continued existence of
>   the dashed git-init and git-clone, of course.

I assume you mean setting the CREATES_GIT_DIR flag here? I think it
would apply to the dashed invocations, too. They strip off the "git-"
and end up in handle_builtin() just like "git clone" does. I didn't test
it, though.

> - There is still duplicated code. For the sake of this RFC, I did not
>   address that yet.

Yeah, I did not read your discover function very carefully. Because I
think the one thing we really don't want to do here is introduce a
separate lookup process that is not shared by setup_git_directory(). The
only sane path, I think, is to refactor setup_git_directory() to build
on top of discover_git_directory(), which implies that the latter
handles all of the cases.

> - The read_early_config() function is called multiple times, re-reading
>   all the config files and re-discovering the .git/ directory multiple
>   times, which is quite wasteful. For the sake of this RFC, I did not
>   address that yet.

We already have a config-caching system. If we went with a global
"config_discover_refs", then I think the sequence for something like
git-init would become:

  1. When git.c starts, config_discover_refs is set to "true". Pager and
     alias lookup may look in .git/config if it's available, even if
     they go through the configset cache.

  2. As soon as git-init starts, it disables config_discover_refs, and
     it flushes the config cache. Any configset lookups will now examine
     the reduced config.

  3. When git-init has set up the real repo it is operating on, it can
     reenable config_discover_refs (though it may not even need to; that
     flag probably wouldn't have any effect once we've entered the
     repository and have_git_dir() returns true).

> - t7006 fails and the error message is a bit cryptic (not to mention the
>   involved function trace, which is mind-boggling for what is supposed
>   to be a simply, shell script-based test suite). I did not have time to
>   look into that yet.

Running t7006 I see a lot of old failures turned into successes, which
is good (because running from a subdirectory now actually respects local
pager config). The one failure looks like it is testing the wrong thing.

It is checking that we _don't_ respect a local core.pager in some cases,
as a proxy for whether or not we are breaking things by doing setup too
early. But the whole point of your series is to fix that, and respect
the config without causing the setup breakage. After your patches, the
proxy behavior and the failure case are disconnected. The test should be
flipped, and ideally another one added that confirms we didn't actually
run setup_git_directory(), but I'm not sure how to test that directly.

> - after discover_git_directory_gently() did its work, the code happily
>   uses its result *only* for the current read_early_config() run, and
>   lets setup_git_dir_gently() do the whole work *again*. For the sake of
>   this RFC, I did not address that yet.

If caching happens at the config layer, then we'd probably only call
this once anyway (or if we did call it again after a config flush, it
would be a good sign that we should compute its value again).

-Peff

Re: [PATCH/RFC 0/7] Pie-in-the-sky attempt to fix the early config

[Johannes Schindelin]

Hi Peff,

On Thu, 8 Dec 2016, Jeff King wrote:

> On Thu, Dec 08, 2016 at 04:35:56PM +0100, Johannes Schindelin wrote:
> 
> > The idea here is to discover the .git/ directory gently (i.e. without
> > changing the current working directory), and to use it to read the
> > .git/config file early, before we actually called setup_git_directory()
> > (if we ever do that).
> 
> Great. Thanks for taking a stab at this.

Well, I figured that I can go through you to get this integrated into
git.git.

> > Notes:
> > 
> > - I find the diff pretty ugly: I wish there was a more elegant way to
> >   *disable* discovery of .git/ *just* for `init` and `clone`. I
> >   considered a function `about_to_create_git_dir()` that is called in a
> >   hard-coded manner *only* for `init` and `clone`, but that would
> >   introduce another magic side effect, when all I want is to reduce those.
> 
> It looks like a lot of that ugliness comes from passing around the "are
> we OK to peek at git-dir config" flag through the various pager-related
> calls.

Correct.

> I don't think it would be bad to use a global for "we do not want a
> repo".

I would think it would be bad, as the entire reason for this patch series
is that we have global state that gets messed up too early (I am speaking
from the point of view of somebody who patched Git locally so that it does
read config variables *before* launching builtins).

> After all, it's just modifying the _existing_ global for "are we in a
> repo".

No it does not.

The read_early_config() function specifically does not leave any traces in
the global namespace. It calls the git_config_with_options() function
without touching the_config_set.

Of course we could introduce a new config set, just for early use. It
would share all the downsides with the current config set.

I would look more favorably on this idea if we were to teach
the_config_set to record a little bit more about the state from which it
was constructed, and to auto-flush-and-re-read when it detects that, say,
git_dir was changed in the meantime.

> And I do not see that global going away anytime soon.

And that is really, really sad.

> Sometimes it's good to make incremental steps towards an end goal, but
> in this case it seems like we just pay the cost of the step without any
> real plan for reaping the benefit in the long term.

Fine. Let's roll with this for now, then.

I just hope that we do not repeat the "slam the door shut to a better
solution" mistake that led to this situation, as the chdir() way did.

> As an alternative, I also think it would be OK to just always have the
> pager config read from local repo, even for init/clone.

For the purpose of this current discussion, I am utterly uninterested in
the pager config. What I want to use the early config for is substantially
different and more relevant: I need to configure some command to run
before, and after, every single Git command here. And this configuration
needs to be per-repository. And no, I do not want to hardcode anything.

So now the cat is out of the bag, I have ulterior motives.

These motives serve a greater good, though: designing read_early_config()
around a very specific use case will always fall short of a well-designed
read_early_config() that could then be used for any use case that
requires, well, reading the config early.

> In other words, to loosen the idea that git-init can _never_ look in the
> current git-dir, and declare that there is a stage before the command is
> initiated, and during which git may read local-repo config. Aliases
> would fall into this, too, so:
> 
>   git config --local alias.foo init
>   git foo /some/other/dir
> 
> would work (as it must, because we cannot know that "foo" is "init"
> until we read the config!).

True.

But is this a good excuse to just shrug our shoulders and let git-init
(which we do know very well) fall into the same trap?

> > - For the moment, I do not handle dashed invocations of `init` and
> >   `clone` correctly. The real problem is the continued existence of
> >   the dashed git-init and git-clone, of course.
> 
> I assume you mean setting the CREATES_GIT_DIR flag here? I think it
> would apply to the dashed invocations, too. They strip off the "git-"
> and end up in handle_builtin() just like "git clone" does. I didn't test
> it, though.

You are correct. I misread the code to expect it to spawn another instance
of git.exe.

> > - There is still duplicated code. For the sake of this RFC, I did not
> >   address that yet.
> 
> Yeah, I did not read your discover function very carefully. Because I
> think the one thing we really don't want to do here is introduce a
> separate lookup process that is not shared by setup_git_directory(). The
> only sane path, I think, is to refactor setup_git_directory() to build
> on top of discover_git_directory(), which implies that the latter
> handles all of the cases.

This could maybe happen later, but for now I do not concern myself with
building the prefix, as the config machinery does not require that
knowledge.

If we go that route, we should of course cache the cwd, git_dir and prefix
of earlier runs, and avoid the entire discovery if we start from the same
cwd.

> > - The read_early_config() function is called multiple times, re-reading
> >   all the config files and re-discovering the .git/ directory multiple
> >   times, which is quite wasteful. For the sake of this RFC, I did not
> >   address that yet.
> 
> We already have a config-caching system. If we went with a global
> "config_discover_refs",

Why "_refs"?

> then I think the sequence for something like git-init would become:
> 
>   1. When git.c starts, config_discover_refs is set to "true". Pager and
>      alias lookup may look in .git/config if it's available, even if
>      they go through the configset cache.
> 
>   2. As soon as git-init starts, it disables config_discover_refs, and
>      it flushes the config cache. Any configset lookups will now examine
>      the reduced config.
> 
>   3. When git-init has set up the real repo it is operating on, it can
>      reenable config_discover_refs (though it may not even need to; that
>      flag probably wouldn't have any effect once we've entered the
>      repository and have_git_dir() returns true).

That is a bit fiddly, don't you think? The callers have to have very
intimate knowledge of the config reading to remember to set, and re-set,
that global. And to flush when appropriate.

How much nicer would the code be if the call to git_config() would realize
what needs to be done, don't you agree?

Less chance for bugs to be introduced by contributors with flakey
understanding of the config/git_dir machinery, myself included.

> > - t7006 fails and the error message is a bit cryptic (not to mention the
> >   involved function trace, which is mind-boggling for what is supposed
> >   to be a simply, shell script-based test suite). I did not have time to
> >   look into that yet.
> 
> Running t7006 I see a lot of old failures turned into successes, which
> is good (because running from a subdirectory now actually respects local
> pager config). The one failure looks like it is testing the wrong thing.

Yeah, but due to the redirections I was not able to figure out what to
change to "fix" this.

> It is checking that we _don't_ respect a local core.pager in some cases,
> as a proxy for whether or not we are breaking things by doing setup too
> early. But the whole point of your series is to fix that, and respect
> the config without causing the setup breakage. After your patches, the
> proxy behavior and the failure case are disconnected. The test should be
> flipped, and ideally another one added that confirms we didn't actually
> run setup_git_directory(), but I'm not sure how to test that directly.

Some dirty tricks come to mind, but I am not even sure that I want to test
for this. Why exactly do we need to avoid calling setup_git_directory() in
that case?

> > - after discover_git_directory_gently() did its work, the code happily
> >   uses its result *only* for the current read_early_config() run, and
> >   lets setup_git_dir_gently() do the whole work *again*. For the sake of
> >   this RFC, I did not address that yet.
> 
> If caching happens at the config layer, then we'd probably only call
> this once anyway (or if we did call it again after a config flush, it
> would be a good sign that we should compute its value again).

I meant the git_dir, not the config... The git_dir is discovered, but a
subsequent setup_git_dir_gently() discovers it *again*, in a different
way.

Ciao,
Dscho

Re: [PATCH/RFC 0/7] Pie-in-the-sky attempt to fix the early config

[Jeff King]

On Fri, Dec 09, 2016 at 06:28:10PM +0100, Johannes Schindelin wrote:

> > Great. Thanks for taking a stab at this.
> 
> Well, I figured that I can go through you to get this integrated into
> git.git.

I am not sure what you mean here, but it _sounds_ like you are
continuing to be negative about the chances of fixes going into git.git
here. I really don't think that negativity is merited, but even if it
is, making snide comments does not help and mostly just makes the
conversation less pleasant.

> > I don't think it would be bad to use a global for "we do not want a
> > repo".
> 
> I would think it would be bad, as the entire reason for this patch series
> is that we have global state that gets messed up too early (I am speaking
> from the point of view of somebody who patched Git locally so that it does
> read config variables *before* launching builtins).

I think those are two different things. One is global state that is
munged as a side effect of setup_git_directory(). The other is global
state that the process sets to say "this is an invariant" so it does not
have to deal with passing it through a huge call chain.

> > After all, it's just modifying the _existing_ global for "are we in a
> > repo".
> 
> No it does not.

Perhaps I wasn't clear on my "it" here. I mean that we will continue to
have startup_info->have_repository as a global (and if not that, then
certainly the environment variable GIT_DIR and the cwd are process
globals). I'm proposing a global flag to modify how to interpret those
globals. I don't think that really makes anything worse.

> The read_early_config() function specifically does not leave any traces in
> the global namespace. It calls the git_config_with_options() function
> without touching the_config_set.

I'm not talking about read_early_config() modifying the global state.
I'm talking about main() modifying it so that lower-level functions like
read_early_config() can make use of it.

> I would look more favorably on this idea if we were to teach
> the_config_set to record a little bit more about the state from which it
> was constructed, and to auto-flush-and-re-read when it detects that, say,
> git_dir was changed in the meantime.

I considered that when fixing some bugs in git-init a few months ago,
but I think the cure becomes worse than the problem. Automatic cache
invalidation can have some tricky corner cases, and there really
_aren't_ that many places where we need to do it. Just dropping
git_config_clear() in those places is ugly, but practical.

> > And I do not see that global going away anytime soon.
> 
> And that is really, really sad.

I think we differ on that.

> > As an alternative, I also think it would be OK to just always have the
> > pager config read from local repo, even for init/clone.
> 
> For the purpose of this current discussion, I am utterly uninterested in
> the pager config. What I want to use the early config for is substantially
> different and more relevant: I need to configure some command to run
> before, and after, every single Git command here. And this configuration
> needs to be per-repository. And no, I do not want to hardcode anything.

I do not see how that changes things. If there is the concept of a
"before the command is run" phase during which we would read from the
current-directory config even for git-init, it seems to me that applies
logically to pagers, aliases, _and_ whatever pre-command magic you are
interested in adding.

> > In other words, to loosen the idea that git-init can _never_ look in the
> > current git-dir, and declare that there is a stage before the command is
> > initiated, and during which git may read local-repo config. Aliases
> > would fall into this, too, so:
> > 
> >   git config --local alias.foo init
> >   git foo /some/other/dir
> > 
> > would work (as it must, because we cannot know that "foo" is "init"
> > until we read the config!).
> 
> True.
> 
> But is this a good excuse to just shrug our shoulders and let git-init
> (which we do know very well) fall into the same trap?

I'm not sure I agree it is a trap. There is a consistent and reasonable
mental model where it is the natural thing. I understand that's not the
one you prefer, but it seems like a practical one to me.

> > We already have a config-caching system. If we went with a global
> > "config_discover_refs",
> 
> Why "_refs"?

Er, sorry, slip of the tongue. I think I meant config_discover_git, and
for some reason managed to repeat it over and over. Hopefully you
figured out what I meant.

> > then I think the sequence for something like git-init would become:
> > 
> >   1. When git.c starts, config_discover_refs is set to "true". Pager and
> >      alias lookup may look in .git/config if it's available, even if
> >      they go through the configset cache.
> > 
> >   2. As soon as git-init starts, it disables config_discover_refs, and
> >      it flushes the config cache. Any configset lookups will now examine
> >      the reduced config.
> > 
> >   3. When git-init has set up the real repo it is operating on, it can
> >      reenable config_discover_refs (though it may not even need to; that
> >      flag probably wouldn't have any effect once we've entered the
> >      repository and have_git_dir() returns true).
> 
> That is a bit fiddly, don't you think? The callers have to have very
> intimate knowledge of the config reading to remember to set, and re-set,
> that global. And to flush when appropriate.

Sure, but I think there are literally 2 callers who care about this.

> How much nicer would the code be if the call to git_config() would realize
> what needs to be done, don't you agree?

No, I wouldn't agree until I had seen a very elegant patch that
implements this and handles any corner cases. I looked into making a
patch before and found that it got a bit ugly (sorry, I don't really
remember the details). I am happy to have you show me that patch, but
pardon me if I remain skeptical until I see it. ;)

> > Running t7006 I see a lot of old failures turned into successes, which
> > is good (because running from a subdirectory now actually respects local
> > pager config). The one failure looks like it is testing the wrong thing.
> 
> Yeah, but due to the redirections I was not able to figure out what to
> change to "fix" this.

I think test_local_config_ignored would go away, and its callers become
test_core_pager_overrides. More or less a revert of 73e25e7cc (git
--paginate: do not commit pager choice too early, 2010-06-26).

> > proxy behavior and the failure case are disconnected. The test should be
> > flipped, and ideally another one added that confirms we didn't actually
> > run setup_git_directory(), but I'm not sure how to test that directly.
> 
> Some dirty tricks come to mind, but I am not even sure that I want to test
> for this. Why exactly do we need to avoid calling setup_git_directory() in
> that case?

I imagine that "git -p init /path/to/new/repo" would start from the
wrong directory, for instance (or any other builtin which does not have
RUN_SETUP would probably get confused).

> > > - after discover_git_directory_gently() did its work, the code happily
> > >   uses its result *only* for the current read_early_config() run, and
> > >   lets setup_git_dir_gently() do the whole work *again*. For the sake of
> > >   this RFC, I did not address that yet.
> > 
> > If caching happens at the config layer, then we'd probably only call
> > this once anyway (or if we did call it again after a config flush, it
> > would be a good sign that we should compute its value again).
> 
> I meant the git_dir, not the config... The git_dir is discovered, but a
> subsequent setup_git_dir_gently() discovers it *again*, in a different
> way.

I don't think it's such a huge amount of work that it really matters
from an optimization standpoint. The really ugly thing to me is if it
returns a different answer. And I think the best way to deal with that
is to literally use the same code (and then if you want to cache the
result and not do the work again, it's pretty easy).

-Peff

Re: [PATCH/RFC 0/7] Pie-in-the-sky attempt to fix the early config

[Duy Nguyen]

On Thu, Dec 8, 2016 at 10:35 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> Hopefully these patches will lead to something that we can integrate,
> and that eventually will make Git's startup sequence much less
> surprising.

What did it surprise you with? Just curious. I can see that I
disrespect the ceiling directory setting, perhaps that's that.
-- 
Duy

Re: [PATCH/RFC 0/7] Pie-in-the-sky attempt to fix the early config

[Johannes Schindelin]

Hi Duy,

On Fri, 9 Dec 2016, Duy Nguyen wrote:

> On Thu, Dec 8, 2016 at 10:35 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > Hopefully these patches will lead to something that we can integrate,
> > and that eventually will make Git's startup sequence much less
> > surprising.
> 
> What did it surprise you with?

I mentioned a couple of WTFs elsewhere:

- that it changes the working directory (personally, I am not surprised,
  just because I was exposed to the Git source code for 10+ years,
  however, I know developers who do not share that experience and find that
  feature... interesting)

- that there are multiple code paths to do essentially the same thing, but
  in an incompatible manner: setup_git_directory() and setup_git_env()
  (and possibly more)

- that some environment variables are set, and need to be unset (or even
  possibly reset to no-longer-known values) to allow subprocesses to
  access different repositories

- that git_path() can be used before setup_git_directory() without
  failure, but wreaks havoc with subsequent git_config() calls.

- that the setup_git_directory() cascade is completely oblivious of
  whatever config has already been read (possibly requiring clearing)

- that setup_git_directory() *returns* the prefix, even stores it in
  startup_info, but a subsequent call to setup_git_directory() returns
  NULL

- that as a consequence, builtins that require the prefix to work if there
  is any, but do not necessarily require a repository to begin with, think
  `git diff`, *must not* be marked with RUN_SETUP_GENTLY

- that check_repository_format() sets have_repository=1

- that setup_git_directory() is a oneliner, calling
  setup_git_directory_gently(NULL), when it would be more logical to
  simply make setup_git_directory() accept the "nongit_ok" parameter

- that setup_git_directory_gently() is not at all gentle when the
  parameter is NULL

- that resolve_gitdir() does not, in fact, resolve any git directory, but
  only tests a specified path whether it refers to a .git/ directory or to
  a .git file

- that resolve_gitdir() actually tests for a .git *file*

- that resolve_gitdir() is not used in setup_git_directory_gently_1()

- that resolve_gitdir() tries the order directory,file and
  setup_git_directory_gently_1() tries the opposite order

- that the handling of the ceiling directories is hardcoded into the
  setup_git_directory_gently_1() function

- that a ceiling directory of /home/hello/world does not prevent Git from
  looking at /home/hello/world/.git/

- that canonicalize_ceiling_entry()' relationship to ceiling directories
  is rather coincidental when the name suggests that it is very specific

- that canonicalize_ceiling_entry() does not, in fact, canonicalize
  non-absolute entries

Need I go on? I could, you know...

> I can see that I disrespect the ceiling directory setting, perhaps
> that's that.

No, I actually see a lot of good reasons for the ceiling directories.

Ciao,
Dscho

[PATCH v2 0/9] Fix the early config

[Johannes Schindelin]

These patches are an attempt to make Git's startup sequence a bit less
surprising.

The idea here is to discover the .git/ directory gently (i.e. without
changing the current working directory, or global variables), and to use
it to read the .git/config file early, before we actually called
setup_git_directory() (if we ever do that).

This also allows us to fix the early config e.g. to determine the pager
or to resolve aliases in a non-surprising manner.

The first iteration of this patch series still tried to be clever and to
avoid having to disentangle the side effects from the
setup_git_directory_gently_1() function simply by duplicating the logic.

However, Peff suggested in a very short sentence that this would not fly
well.

Little did I know that I would spend the better part of an entire week
on trying to address that innocuous comment! There are simply *so many*
side effects in that code. Who would have thought that a function
called check_repository_format() would set global variables?

But after all that work, I am actually quite a bit satisfied with the
way things turned out.

My dirty little secret is that I actually need this for something else
entirely. I need to patch an internal version of Git to gather
statistics, and to that end I need to read the config before and after
running every Git command. Hence the need for a gentle, and correct
early config.

Notes:

- I do not handle dashed invocations of `init` and `clone` correctly.
  That is, even if `git-init` and `git-clone` clearly do not want to
  read the local config, they do. It does not matter all that much
  because they do not use a pager, but still. It is a wart.

- The read_early_config() function is still called multiple times,
  re-reading all the config files and re-discovering the .git/ directory
  multiple times, which is quite wasteful. I was tempted to take care of
  that but I must not run the danger to spread myself even thinner these
  days. If a patch adding that caching were to fly my way, I'd gladly
  integrate it, of course... ;-)

Changes since v1:

- the discover_git_directory() function is no longer completely separate
  from setup_git_directory(), but a callee of the latter.

- t7006 succeeds now (I removed the incorrect test case in favor of one
  that verifies that setup_git_directory() was not run via the tell-tale
  that the current working directory has not changed when the pager
  runs).


Johannes Schindelin (9):
  t7006: replace dubious test
  setup_git_directory(): use is_dir_sep() helper
  setup_git_directory(): avoid changing global state during discovery
  Export the discover_git_directory() function
  Make read_early_config() reusable
  read_early_config(): special-case builtins that create a repository
  read_early_config(): avoid .git/config hack when unneeded
  read_early_config(): really discover .git/
  Test read_early_config()

 cache.h                 |   4 +-
 config.c                |  36 +++++++++
 git.c                   |   3 +
 pager.c                 |  31 -------
 setup.c                 | 211 +++++++++++++++++++++++++++++++-----------------
 t/helper/test-config.c  |  15 ++++
 t/t1309-early-config.sh |  50 ++++++++++++
 t/t7006-pager.sh        |  18 ++++-
 8 files changed, 257 insertions(+), 111 deletions(-)
 create mode 100755 t/t1309-early-config.sh


base-commit: 3bc53220cb2dcf709f7a027a3f526befd021d858
Published-As: https://github.com/dscho/git/releases/tag/early-config-v2
Fetch-It-Via: git fetch https://github.com/dscho/git early-config-v2

Interdiff vs v1:

 diff --git a/builtin/am.c b/builtin/am.c
 index 2b81dabddd9..f7a7a971fbe 100644
 --- a/builtin/am.c
 +++ b/builtin/am.c
 @@ -1791,7 +1791,7 @@ static int do_interactive(struct am_state *state)
  			}
  			strbuf_release(&msg);
  		} else if (*reply == 'v' || *reply == 'V') {
 -			const char *pager = git_pager(1, 1);
 +			const char *pager = git_pager(1);
  			struct child_process cp = CHILD_PROCESS_INIT;
  
  			if (!pager)
 diff --git a/builtin/blame.c b/builtin/blame.c
 index 628ca237da6..cffc6265408 100644
 --- a/builtin/blame.c
 +++ b/builtin/blame.c
 @@ -2919,7 +2919,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
  	assign_blame(&sb, opt);
  
  	if (!incremental)
 -		setup_pager(1);
 +		setup_pager();
  
  	free(final_commit_name);
  
 diff --git a/builtin/grep.c b/builtin/grep.c
 index f820e4b1c4d..9304c33e750 100644
 --- a/builtin/grep.c
 +++ b/builtin/grep.c
 @@ -1133,7 +1133,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
  	}
  
  	if (show_in_pager == default_pager)
 -		show_in_pager = git_pager(1, 1);
 +		show_in_pager = git_pager(1);
  	if (show_in_pager) {
  		opt.color = 0;
  		opt.name_only = 1;
 @@ -1268,7 +1268,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
  		die(_("option not supported with --recurse-submodules."));
  
  	if (!show_in_pager && !opt.status_only)
 -		setup_pager(1);
 +		setup_pager();
  
  	if (!use_index && (untracked || cached))
  		die(_("--cached or --untracked cannot be used with --no-index."));
 diff --git a/builtin/log.c b/builtin/log.c
 index 96618d38cbf..55d20cc2d88 100644
 --- a/builtin/log.c
 +++ b/builtin/log.c
 @@ -203,7 +203,7 @@ static void cmd_log_init_finish(int argc, const char **argv, const char *prefix,
  	if (rev->line_level_traverse)
  		line_log_init(rev, line_cb.prefix, &line_cb.args);
  
 -	setup_pager(1);
 +	setup_pager();
  }
  
  static void cmd_log_init(int argc, const char **argv, const char *prefix,
 @@ -1600,7 +1600,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
  	if (!use_stdout)
  		output_directory = set_outdir(prefix, output_directory);
  	else
 -		setup_pager(1);
 +		setup_pager();
  
  	if (output_directory) {
  		if (rev.diffopt.use_color != GIT_COLOR_ALWAYS)
 diff --git a/builtin/var.c b/builtin/var.c
 index 879867b8427..aedbb53a2da 100644
 --- a/builtin/var.c
 +++ b/builtin/var.c
 @@ -19,7 +19,7 @@ static const char *editor(int flag)
  
  static const char *pager(int flag)
  {
 -	const char *pgm = git_pager(1, 1);
 +	const char *pgm = git_pager(1);
  
  	if (!pgm)
  		pgm = "cat";
 diff --git a/cache.h b/cache.h
 index 4d89966d711..0af7141242f 100644
 --- a/cache.h
 +++ b/cache.h
 @@ -518,6 +518,7 @@ extern void set_git_work_tree(const char *tree);
  #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
  
  extern void setup_work_tree(void);
 +extern const char *discover_git_directory(struct strbuf *gitdir);
  extern const char *setup_git_directory_gently(int *);
  extern const char *setup_git_directory(void);
  extern char *prefix_path(const char *prefix, int len, const char *path);
 @@ -1428,7 +1429,7 @@ extern const char *fmt_name(const char *name, const char *email);
  extern const char *ident_default_name(void);
  extern const char *ident_default_email(void);
  extern const char *git_editor(void);
 -extern const char *git_pager(int stdout_is_tty, int discover_git_dir);
 +extern const char *git_pager(int stdout_is_tty);
  extern int git_ident_config(const char *, const char *, void *);
  extern void reset_ident_date(void);
  
 @@ -1797,8 +1798,7 @@ extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
  				     const unsigned char *sha1, void *data);
  extern void git_config_push_parameter(const char *text);
  extern int git_config_from_parameters(config_fn_t fn, void *data);
 -extern void read_early_config(config_fn_t cb, void *data,
 -			      int discover_git_dir);
 +extern void read_early_config(config_fn_t cb, void *data);
  extern void git_config(config_fn_t fn, void *);
  extern int git_config_with_options(config_fn_t fn, void *,
  				   struct git_config_source *config_source,
 @@ -1994,12 +1994,12 @@ __attribute__((format (printf, 2, 3)))
  extern void write_file(const char *path, const char *fmt, ...);
  
  /* pager.c */
 -extern void setup_pager(int discover_git_dir);
 +extern void setup_pager(void);
  extern int pager_in_use(void);
  extern int pager_use_color;
  extern int term_columns(void);
  extern int decimal_width(uintmax_t);
 -extern int check_pager_config(const char *cmd, int discover_git_dir);
 +extern int check_pager_config(const char *cmd);
  extern void prepare_pager_args(struct child_process *, const char *pager);
  
  extern const char *editor_program;
 @@ -2070,7 +2070,7 @@ const char *split_cmdline_strerror(int cmdline_errno);
  
  /* setup.c */
  struct startup_info {
 -	int have_repository;
 +	int have_repository, creating_repository;
  	const char *prefix;
  };
  extern struct startup_info *startup_info;
 diff --git a/config.c b/config.c
 index c9f191e1fe3..bcda397d42e 100644
 --- a/config.c
 +++ b/config.c
 @@ -1412,106 +1412,32 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
  	}
  }
  
 -/*
 - * A "string_list_each_func_t" function that canonicalizes an entry
 - * from GIT_CEILING_DIRECTORIES using real_path_if_valid(), or
 - * discards it if unusable.  The presence of an empty entry in
 - * GIT_CEILING_DIRECTORIES turns off canonicalization for all
 - * subsequent entries.
 - */
 -static int canonicalize_ceiling_entry(struct string_list_item *item,
 -				      void *cb_data)
 -{
 -	int *empty_entry_found = cb_data;
 -	char *ceil = item->string;
 -
 -	if (!*ceil) {
 -		*empty_entry_found = 1;
 -		return 0;
 -	} else if (!is_absolute_path(ceil)) {
 -		return 0;
 -	} else if (*empty_entry_found) {
 -		/* Keep entry but do not canonicalize it */
 -		return 1;
 -	} else {
 -		const char *real_path = real_path_if_valid(ceil);
 -		if (!real_path)
 -			return 0;
 -		free(item->string);
 -		item->string = xstrdup(real_path);
 -		return 1;
 -	}
 -}
 -
 -/*
 - * Note that this is a really dirty hack that replicates what the
 - * setup_git_directory() function does, without changing the current
 - * working directory. The crux of the problem is that we cannot run
 - * setup_git_directory() early on in git's setup, so we have to
 - * duplicate the work that setup_git_directory() would otherwise do.
 - */
 -static int discover_git_directory_gently(struct strbuf *result)
 -{
 -	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 -	int ceiling_offset = -1;
 -	const char *p;
 -
 -	if (strbuf_getcwd(result) < 0)
 -		return -1;
 -	p = real_path_if_valid(result->buf);
 -	if (!p)
 -		return -1;
 -	strbuf_reset(result);
 -	strbuf_addstr(result, p);
 -
 -	if (env_ceiling_dirs) {
 -		struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
 -		int empty_entry_found = 0;
 -
 -		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP,
 -				  -1);
 -		filter_string_list(&ceiling_dirs, 0, canonicalize_ceiling_entry,
 -				   &empty_entry_found);
 -		ceiling_offset = longest_ancestor_length(result->buf,
 -							 &ceiling_dirs);
 -		string_list_clear(&ceiling_dirs, 0);
 -	}
 -
 -	if (ceiling_offset < 0 && has_dos_drive_prefix(result->buf))
 -		ceiling_offset = 1;
 -
 -	for (;;) {
 -		int len = result->len, i;
 -
 -		strbuf_addstr(result, "/" DEFAULT_GIT_DIR_ENVIRONMENT);
 -		p = read_gitfile_gently(result->buf, &i);
 -		if (p) {
 -			strbuf_reset(result);
 -			strbuf_addstr(result, p);
 -			return 0;
 -		}
 -		if (is_git_directory(result->buf))
 -			return 0;
 -		strbuf_setlen(result, len);
 -		if (is_git_directory(result->buf))
 -			return 0;
 -		for (i = len; --i > ceiling_offset; )
 -			if (is_dir_sep(result->buf[i]))
 -				break;
 -		if (i <= ceiling_offset)
 -			return -1;
 -		strbuf_setlen(result, i);
 -	}
 -}
 -
 -void read_early_config(config_fn_t cb, void *data, int discover_git_dir)
 +void read_early_config(config_fn_t cb, void *data)
  {
  	struct strbuf buf = STRBUF_INIT;
  
  	git_config_with_options(cb, data, NULL, 1);
  
 -	if (discover_git_dir && !have_git_dir() &&
 -	    !discover_git_directory_gently(&buf)) {
 +	/*
 +	 * Note that this is a really dirty hack that does the wrong thing in
 +	 * many cases. The crux of the problem is that we cannot run
 +	 * setup_git_directory() early on in git's setup, so we have no idea if
 +	 * we are in a repository or not, and therefore are not sure whether
 +	 * and how to read repository-local config.
 +	 *
 +	 * So if we _aren't_ in a repository (or we are but we would reject its
 +	 * core.repositoryformatversion), we'll read whatever is in .git/config
 +	 * blindly. Similarly, if we _are_ in a repository, but not at the
 +	 * root, we'll fail to find .git/config (because it's really
 +	 * ../.git/config, etc). See t7006 for a complete set of failures.
 +	 *
 +	 * However, we have historically provided this hack because it does
 +	 * work some of the time (namely when you are at the top-level of a
 +	 * valid repository), and would rarely make things worse (i.e., you do
 +	 * not generally have a .git/config file sitting around).
 +	 */
 +	if (!startup_info->creating_repository && !have_git_dir() &&
 +	    discover_git_directory(&buf)) {
  		struct git_config_source repo_config;
  
  		memset(&repo_config, 0, sizeof(repo_config));
 diff --git a/diff.c b/diff.c
 index 1e1d3b85c2a..051761be405 100644
 --- a/diff.c
 +++ b/diff.c
 @@ -5259,6 +5259,6 @@ void setup_diff_pager(struct diff_options *opt)
  	 * --exit-code" in hooks and other scripts, we do not do so.
  	 */
  	if (!DIFF_OPT_TST(opt, EXIT_WITH_STATUS) &&
 -	    check_pager_config("diff", 1) != 0)
 -		setup_pager(1);
 +	    check_pager_config("diff") != 0)
 +		setup_pager();
  }
 diff --git a/git.c b/git.c
 index d4712b25fee..9fb9bb90a21 100644
 --- a/git.c
 +++ b/git.c
 @@ -61,13 +61,13 @@ static void restore_env(int external_alias)
  	}
  }
  
 -static void commit_pager_choice(int discover_git_dir) {
 +static void commit_pager_choice(void) {
  	switch (use_pager) {
  	case 0:
  		setenv("GIT_PAGER", "cat", 1);
  		break;
  	case 1:
 -		setup_pager(discover_git_dir);
 +		setup_pager();
  		break;
  	default:
  		break;
 @@ -261,7 +261,7 @@ static int handle_alias(int *argcp, const char ***argv)
  		if (alias_string[0] == '!') {
  			struct child_process child = CHILD_PROCESS_INIT;
  
 -			commit_pager_choice(1);
 +			commit_pager_choice();
  			restore_env(1);
  
  			child.use_shell = 1;
 @@ -318,13 +318,12 @@ static int handle_alias(int *argcp, const char ***argv)
  #define RUN_SETUP		(1<<0)
  #define RUN_SETUP_GENTLY	(1<<1)
  #define USE_PAGER		(1<<2)
 -#define CREATES_GIT_DIR         (1<<3)
  /*
   * require working tree to be present -- anything uses this needs
   * RUN_SETUP for reading from the configuration file.
   */
 -#define NEED_WORK_TREE		(1<<4)
 -#define SUPPORT_SUPER_PREFIX	(1<<5)
 +#define NEED_WORK_TREE		(1<<3)
 +#define SUPPORT_SUPER_PREFIX	(1<<4)
  
  struct cmd_struct {
  	const char *cmd;
 @@ -338,6 +337,9 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
  	struct stat st;
  	const char *prefix;
  
 +	if (p->fn == cmd_init_db || p->fn == cmd_clone)
 +		startup_info->creating_repository = 1;
 +
  	prefix = NULL;
  	help = argc == 2 && !strcmp(argv[1], "-h");
  	if (!help) {
 @@ -349,7 +351,7 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
  		}
  
  		if (use_pager == -1 && p->option & (RUN_SETUP | RUN_SETUP_GENTLY))
 -			use_pager = check_pager_config(p->cmd, !(p->option & CREATES_GIT_DIR));
 +			use_pager = check_pager_config(p->cmd);
  		if (use_pager == -1 && p->option & USE_PAGER)
  			use_pager = 1;
  
 @@ -357,7 +359,7 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
  		    startup_info->have_repository) /* get_git_dir() may set up repo, avoid that */
  			trace_repo_setup(prefix);
  	}
 -	commit_pager_choice(!(p->option & CREATES_GIT_DIR));
 +	commit_pager_choice();
  
  	if (!help && get_super_prefix()) {
  		if (!(p->option & SUPPORT_SUPER_PREFIX))
 @@ -413,7 +415,7 @@ static struct cmd_struct commands[] = {
  	{ "cherry", cmd_cherry, RUN_SETUP },
  	{ "cherry-pick", cmd_cherry_pick, RUN_SETUP | NEED_WORK_TREE },
  	{ "clean", cmd_clean, RUN_SETUP | NEED_WORK_TREE },
 -	{ "clone", cmd_clone, CREATES_GIT_DIR },
 +	{ "clone", cmd_clone },
  	{ "column", cmd_column, RUN_SETUP_GENTLY },
  	{ "commit", cmd_commit, RUN_SETUP | NEED_WORK_TREE },
  	{ "commit-tree", cmd_commit_tree, RUN_SETUP },
 @@ -440,7 +442,7 @@ static struct cmd_struct commands[] = {
  	{ "hash-object", cmd_hash_object },
  	{ "help", cmd_help },
  	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY },
 -	{ "init", cmd_init_db, CREATES_GIT_DIR },
 +	{ "init", cmd_init_db },
  	{ "init-db", cmd_init_db },
  	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY },
  	{ "log", cmd_log, RUN_SETUP },
 @@ -585,8 +587,8 @@ static void execv_dashed_external(const char **argv)
  		die("%s doesn't support --super-prefix", argv[0]);
  
  	if (use_pager == -1)
 -		use_pager = check_pager_config(argv[0], 1);
 -	commit_pager_choice(1);
 +		use_pager = check_pager_config(argv[0]);
 +	commit_pager_choice();
  
  	argv_array_pushf(&cmd.args, "git-%s", argv[0]);
  	argv_array_pushv(&cmd.args, argv + 1);
 @@ -684,7 +686,7 @@ int cmd_main(int argc, const char **argv)
  		skip_prefix(argv[0], "--", &argv[0]);
  	} else {
  		/* The user didn't specify a command; give them help */
 -		commit_pager_choice(1);
 +		commit_pager_choice();
  		printf("usage: %s\n\n", git_usage_string);
  		list_common_cmds_help();
  		printf("\n%s\n", _(git_more_info_string));
 diff --git a/pager.c b/pager.c
 index 16b3cbe2320..73ca8bc3b17 100644
 --- a/pager.c
 +++ b/pager.c
 @@ -43,7 +43,7 @@ static int core_pager_config(const char *var, const char *value, void *data)
  	return 0;
  }
  
 -const char *git_pager(int stdout_is_tty, int discover_git_dir)
 +const char *git_pager(int stdout_is_tty)
  {
  	const char *pager;
  
 @@ -53,8 +53,7 @@ const char *git_pager(int stdout_is_tty, int discover_git_dir)
  	pager = getenv("GIT_PAGER");
  	if (!pager) {
  		if (!pager_program)
 -			read_early_config(core_pager_config, NULL,
 -					  discover_git_dir);
 +			read_early_config(core_pager_config, NULL);
  		pager = pager_program;
  	}
  	if (!pager)
 @@ -101,9 +100,9 @@ void prepare_pager_args(struct child_process *pager_process, const char *pager)
  	setup_pager_env(&pager_process->env_array);
  }
  
 -void setup_pager(int discover_git_dir)
 +void setup_pager(void)
  {
 -	const char *pager = git_pager(isatty(1), discover_git_dir);
 +	const char *pager = git_pager(isatty(1));
  
  	if (!pager)
  		return;
 @@ -209,7 +208,7 @@ static int pager_command_config(const char *var, const char *value, void *vdata)
  }
  
  /* returns 0 for "no pager", 1 for "use pager", and -1 for "not specified" */
 -int check_pager_config(const char *cmd, int discover_git_dir)
 +int check_pager_config(const char *cmd)
  {
  	struct pager_command_config_data data;
  
 @@ -217,7 +216,7 @@ int check_pager_config(const char *cmd, int discover_git_dir)
  	data.want = -1;
  	data.value = NULL;
  
 -	read_early_config(pager_command_config, &data, discover_git_dir);
 +	read_early_config(pager_command_config, &data);
  
  	if (data.value)
  		pager_program = data.value;
 diff --git a/setup.c b/setup.c
 index 967f289f1ef..7ceca6cc6ef 100644
 --- a/setup.c
 +++ b/setup.c
 @@ -816,50 +816,49 @@ static int canonicalize_ceiling_entry(struct string_list_item *item,
  	}
  }
  
 +enum discovery_result {
 +	GIT_DIR_NONE = 0,
 +	GIT_DIR_EXPLICIT,
 +	GIT_DIR_DISCOVERED,
 +	GIT_DIR_BARE,
 +	/* these are errors */
 +	GIT_DIR_HIT_CEILING = -1,
 +	GIT_DIR_HIT_MOUNT_POINT = -2
 +};
 +
  /*
   * We cannot decide in this function whether we are in the work tree or
   * not, since the config can only be read _after_ this function was called.
 + *
 + * Also, we avoid changing any global state (such as the current working
 + * directory) to allow early callers.
 + *
 + * The directory where the search should start needs to be passed in via the
 + * `dir` parameter; upon return, the `dir` buffer will contain the path of
 + * the directory where the search ended, and `gitdir` will contain the path of
 + * the discovered .git/ directory, if any. This path may be relative against
 + * `dir` (i.e. *not* necessarily the cwd).
   */
 -static const char *setup_git_directory_gently_1(int *nongit_ok)
 +static enum discovery_result discover_git_directory_1(struct strbuf *dir,
 +						      struct strbuf *gitdir)
  {
  	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
  	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
 -	static struct strbuf cwd = STRBUF_INIT;
 -	const char *gitdirenv, *ret;
 -	char *gitfile;
 -	int offset, offset_parent, ceil_offset = -1;
 +	const char *gitdirenv;
 +	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
  	dev_t current_device = 0;
  	int one_filesystem = 1;
  
  	/*
 -	 * We may have read an incomplete configuration before
 -	 * setting-up the git directory. If so, clear the cache so
 -	 * that the next queries to the configuration reload complete
 -	 * configuration (including the per-repo config file that we
 -	 * ignored previously).
 -	 */
 -	git_config_clear();
 -
 -	/*
 -	 * Let's assume that we are in a git repository.
 -	 * If it turns out later that we are somewhere else, the value will be
 -	 * updated accordingly.
 -	 */
 -	if (nongit_ok)
 -		*nongit_ok = 0;
 -
 -	if (strbuf_getcwd(&cwd))
 -		die_errno(_("Unable to read current working directory"));
 -	offset = cwd.len;
 -
 -	/*
  	 * If GIT_DIR is set explicitly, we're not going
  	 * to do any discovery, but we still do repository
  	 * validation.
  	 */
  	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
 -	if (gitdirenv)
 -		return setup_explicit_git_dir(gitdirenv, &cwd, nongit_ok);
 +	if (gitdirenv) {
 +		strbuf_addstr(gitdir, gitdirenv);
 +		return GIT_DIR_EXPLICIT;
 +	}
  
  	if (env_ceiling_dirs) {
  		int empty_entry_found = 0;
 @@ -867,15 +866,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
  		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP, -1);
  		filter_string_list(&ceiling_dirs, 0,
  				   canonicalize_ceiling_entry, &empty_entry_found);
 -		ceil_offset = longest_ancestor_length(cwd.buf, &ceiling_dirs);
 +		ceil_offset = longest_ancestor_length(dir->buf, &ceiling_dirs);
  		string_list_clear(&ceiling_dirs, 0);
  	}
  
 -	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
 -		ceil_offset = 1;
 +	if (ceil_offset < 0)
 +		ceil_offset = min_offset - 2;
  
  	/*
 -	 * Test in the following order (relative to the cwd):
 +	 * Test in the following order (relative to the dir):
  	 * - .git (file containing "gitdir: <path>")
  	 * - .git/
  	 * - ./ (bare)
 @@ -887,61 +886,123 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
  	 */
  	one_filesystem = !git_env_bool("GIT_DISCOVERY_ACROSS_FILESYSTEM", 0);
  	if (one_filesystem)
 -		current_device = get_device_or_die(".", NULL, 0);
 +		current_device = get_device_or_die(dir->buf, NULL, 0);
  	for (;;) {
 -		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
 -		if (gitfile)
 -			gitdirenv = gitfile = xstrdup(gitfile);
 -		else {
 -			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
 -				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 -		}
 -
 +		int offset = dir->len;
 +
 +		if (offset > min_offset)
 +			strbuf_addch(dir, '/');
 +		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
 +		gitdirenv = read_gitfile(dir->buf);
 +		if (!gitdirenv && is_git_directory(dir->buf))
 +			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 +		strbuf_setlen(dir, offset);
  		if (gitdirenv) {
 -			ret = setup_discovered_git_dir(gitdirenv,
 -						       &cwd, offset,
 -						       nongit_ok);
 -			free(gitfile);
 -			return ret;
 +			strbuf_addstr(gitdir, gitdirenv);
 +			return GIT_DIR_DISCOVERED;
  		}
 -		free(gitfile);
  
 -		if (is_git_directory("."))
 -			return setup_bare_git_dir(&cwd, offset, nongit_ok);
 -
 -		offset_parent = offset;
 -		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
 -		if (offset_parent <= ceil_offset)
 -			return setup_nongit(cwd.buf, nongit_ok);
 -		if (one_filesystem) {
 -			dev_t parent_device = get_device_or_die("..", cwd.buf,
 -								offset);
 -			if (parent_device != current_device) {
 -				if (nongit_ok) {
 -					if (chdir(cwd.buf))
 -						die_errno(_("Cannot come back to cwd"));
 -					*nongit_ok = 1;
 -					return NULL;
 -				}
 -				strbuf_setlen(&cwd, offset);
 -				die(_("Not a git repository (or any parent up to mount point %s)\n"
 -				"Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
 -				    cwd.buf);
 -			}
 -		}
 -		if (chdir("..")) {
 -			strbuf_setlen(&cwd, offset);
 -			die_errno(_("Cannot change to '%s/..'"), cwd.buf);
 +		if (is_git_directory(dir->buf)) {
 +			strbuf_addstr(gitdir, ".");
 +			return GIT_DIR_BARE;
  		}
 -		offset = offset_parent;
 +
 +		if (offset <= min_offset)
 +			return GIT_DIR_HIT_CEILING;
 +
 +		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]));
 +		if (offset <= ceil_offset)
 +			return GIT_DIR_HIT_CEILING;
 +
 +		strbuf_setlen(dir, offset > min_offset ?  offset : min_offset);
 +		if (one_filesystem &&
 +		    current_device != get_device_or_die(dir->buf, NULL, offset))
 +			return GIT_DIR_HIT_MOUNT_POINT;
 +	}
 +}
 +
 +const char *discover_git_directory(struct strbuf *gitdir)
 +{
 +	struct strbuf dir = STRBUF_INIT;
 +	int len;
 +
 +	if (strbuf_getcwd(&dir))
 +		return NULL;
 +
 +	len = dir.len;
 +	if (discover_git_directory_1(&dir, gitdir) < 0) {
 +		strbuf_release(&dir);
 +		return NULL;
 +	}
 +
 +	if (dir.len < len && !is_absolute_path(gitdir->buf)) {
 +		strbuf_addch(&dir, '/');
 +		strbuf_insert(gitdir, 0, dir.buf, dir.len);
  	}
 +	strbuf_release(&dir);
 +
 +	return gitdir->buf;
  }
  
  const char *setup_git_directory_gently(int *nongit_ok)
  {
 +	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
  	const char *prefix;
  
 -	prefix = setup_git_directory_gently_1(nongit_ok);
 +	/*
 +	 * We may have read an incomplete configuration before
 +	 * setting-up the git directory. If so, clear the cache so
 +	 * that the next queries to the configuration reload complete
 +	 * configuration (including the per-repo config file that we
 +	 * ignored previously).
 +	 */
 +	git_config_clear();
 +
 +	/*
 +	 * Let's assume that we are in a git repository.
 +	 * If it turns out later that we are somewhere else, the value will be
 +	 * updated accordingly.
 +	 */
 +	if (nongit_ok)
 +		*nongit_ok = 0;
 +
 +	if (strbuf_getcwd(&cwd))
 +		die_errno(_("Unable to read current working directory"));
 +	strbuf_addbuf(&dir, &cwd);
 +
 +	switch (discover_git_directory_1(&dir, &gitdir)) {
 +	case GIT_DIR_NONE:
 +		prefix = NULL;
 +		break;
 +	case GIT_DIR_EXPLICIT:
 +		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
 +		break;
 +	case GIT_DIR_DISCOVERED:
 +		if (dir.len < cwd.len && chdir(dir.buf))
 +			die(_("Cannot change to '%s'"), dir.buf);
 +		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
 +						  nongit_ok);
 +		break;
 +	case GIT_DIR_BARE:
 +		if (dir.len < cwd.len && chdir(dir.buf))
 +			die(_("Cannot change to '%s'"), dir.buf);
 +		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
 +		break;
 +	case GIT_DIR_HIT_CEILING:
 +		prefix = setup_nongit(cwd.buf, nongit_ok);
 +		break;
 +	case GIT_DIR_HIT_MOUNT_POINT:
 +		if (nongit_ok) {
 +			*nongit_ok = 1;
 +			return NULL;
 +		}
 +		die(_("Not a git repository (or any parent up to mount point %s)\n"
 +		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
 +		    dir.buf);
 +	default:
 +		die("BUG: unhandled discover_git_directory() result");
 +	}
 +
  	if (prefix)
  		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
  	else
 diff --git a/t/helper/test-config.c b/t/helper/test-config.c
 index 51050695876..8e3ed6a76cb 100644
 --- a/t/helper/test-config.c
 +++ b/t/helper/test-config.c
 @@ -84,7 +84,7 @@ int cmd_main(int argc, const char **argv)
  	struct config_set cs;
  
  	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
 -		read_early_config(early_config_cb, (void *)argv[2], 1);
 +		read_early_config(early_config_cb, (void *)argv[2]);
  		return 0;
  	}
  
 diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
 index c8dc665f2fd..bf89340988b 100755
 --- a/t/t7006-pager.sh
 +++ b/t/t7006-pager.sh
 @@ -360,27 +360,37 @@ test_pager_choices                       'git aliasedlog'
  test_default_pager        expect_success 'git -p aliasedlog'
  test_PAGER_overrides      expect_success 'git -p aliasedlog'
  test_core_pager_overrides expect_success 'git -p aliasedlog'
 -test_core_pager_subdir    expect_failure 'git -p aliasedlog'
 +test_core_pager_subdir    expect_success 'git -p aliasedlog'
  test_GIT_PAGER_overrides  expect_success 'git -p aliasedlog'
  
  test_default_pager        expect_success 'git -p true'
  test_PAGER_overrides      expect_success 'git -p true'
  test_core_pager_overrides expect_success 'git -p true'
 -test_core_pager_subdir    expect_failure 'git -p true'
 +test_core_pager_subdir    expect_success 'git -p true'
  test_GIT_PAGER_overrides  expect_success 'git -p true'
  
  test_default_pager        expect_success test_must_fail 'git -p request-pull'
  test_PAGER_overrides      expect_success test_must_fail 'git -p request-pull'
  test_core_pager_overrides expect_success test_must_fail 'git -p request-pull'
 -test_core_pager_subdir    expect_failure test_must_fail 'git -p request-pull'
 +test_core_pager_subdir    expect_success test_must_fail 'git -p request-pull'
  test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
  
  test_default_pager        expect_success test_must_fail 'git -p'
  test_PAGER_overrides      expect_success test_must_fail 'git -p'
  test_local_config_ignored expect_failure test_must_fail 'git -p'
 -test_no_local_config_subdir expect_success test_must_fail 'git -p'
  test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
  
 +test_expect_success TTY 'core.pager in repo config works and retains cwd' '
 +	sane_unset GIT_PAGER &&
 +	test_config core.pager "cat >cwd-retained" &&
 +	(
 +		cd sub &&
 +		rm -f cwd-retained &&
 +		test_terminal git -p rev-parse HEAD &&
 +		test -e cwd-retained
 +	)
 +'
 +
  test_doesnt_paginate      expect_failure test_must_fail 'git -p nonsense'
  
  test_pager_choices                       'git shortlog'

-- 
2.12.0.windows.1.3.g8a117c48243

[PATCH v2 1/9] t7006: replace dubious test

[Johannes Schindelin]

The idea of the test case "git -p - core.pager is not used from
subdirectory" was to verify that the setup_git_directory() function had
not been called just to obtain the core.pager setting.

However, we are about to fix the early config machinery so that it
*does* work, without messing up the global state.

Once that is done, the core.pager setting *will* be used, even when
running from a subdirectory, and that is a Good Thing.

The intention of that test case, however, was to verify that the
setup_git_directory() function has not run, because it changes global
state such as the current working directory.

To keep that spirit, but fix the incorrect assumption, this patch
replaces that test case by a new one that verifies that the pager is
run in the subdirectory, i.e. that the current working directory has
not been changed at the time the pager is configured and launched, even
if the `rev-parse` command requires a .git/ directory and *will* change
the working directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t7006-pager.sh | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index c8dc665f2fd..427bfc605ad 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -378,9 +378,19 @@ test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 test_default_pager        expect_success test_must_fail 'git -p'
 test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
-test_no_local_config_subdir expect_success test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
+test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+	sane_unset GIT_PAGER &&
+	test_config core.pager "cat >cwd-retained" &&
+	(
+		cd sub &&
+		rm -f cwd-retained &&
+		test_terminal git -p rev-parse HEAD &&
+		test -e cwd-retained
+	)
+'
+
 test_doesnt_paginate      expect_failure test_must_fail 'git -p nonsense'
 
 test_pager_choices                       'git shortlog'
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 1/9] t7006: replace dubious test

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:02AM +0100, Johannes Schindelin wrote:

> The intention of that test case, however, was to verify that the
> setup_git_directory() function has not run, because it changes global
> state such as the current working directory.
> 
> To keep that spirit, but fix the incorrect assumption, this patch
> replaces that test case by a new one that verifies that the pager is
> run in the subdirectory, i.e. that the current working directory has
> not been changed at the time the pager is configured and launched, even
> if the `rev-parse` command requires a .git/ directory and *will* change
> the working directory.

This is a great solution to the question I had in v1 ("how do we know
when setup_git_directory() is run?"). It not only checks the
internal-code case that we care about, but it also shows how real users
would get bit if we do the wrong thing.

> +test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
> +	sane_unset GIT_PAGER &&
> +	test_config core.pager "cat >cwd-retained" &&
> +	(
> +		cd sub &&
> +		rm -f cwd-retained &&
> +		test_terminal git -p rev-parse HEAD &&
> +		test -e cwd-retained
> +	)
> +'

Does this actually need TTY and test_terminal? You replace the pager
with something that doesn't care about the terminal, and "-p" should
unconditionally turn it on.

(Also a minor nit: we usually prefer test_path_is_file to "test -e"
these days).

-Peff

Re: [PATCH v2 1/9] t7006: replace dubious test

[Johannes Schindelin]

Hi Peff,

On Thu, 2 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:04:02AM +0100, Johannes Schindelin wrote:
> 
> > The intention of that test case, however, was to verify that the
> > setup_git_directory() function has not run, because it changes global
> > state such as the current working directory.
> > 
> > To keep that spirit, but fix the incorrect assumption, this patch
> > replaces that test case by a new one that verifies that the pager is
> > run in the subdirectory, i.e. that the current working directory has
> > not been changed at the time the pager is configured and launched,
> > even if the `rev-parse` command requires a .git/ directory and *will*
> > change the working directory.
> 
> This is a great solution to the question I had in v1 ("how do we know
> when setup_git_directory() is run?"). It not only checks the
> internal-code case that we care about, but it also shows how real users
> would get bit if we do the wrong thing.

Thanks!

> > +test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
> > +	sane_unset GIT_PAGER &&
> > +	test_config core.pager "cat >cwd-retained" &&
> > +	(
> > +		cd sub &&
> > +		rm -f cwd-retained &&
> > +		test_terminal git -p rev-parse HEAD &&
> > +		test -e cwd-retained
> > +	)
> > +'
> 
> Does this actually need TTY and test_terminal?

Sadly, yes. If git.c sees a --paginate or -p, it sets use_pager to 1 which
is later used as indicator that setup_pager() should be run. And the first
line of that function reads:

	const char *pager = git_pager(isatty(1));

i.e. it does *not* turn on the pager unconditionally.

> (Also a minor nit: we usually prefer test_path_is_file to "test -e"
> these days).

Thanks, fixed.

Ciao,
Dscho

[PATCH v2 2/9] setup_git_directory(): use is_dir_sep() helper

[Johannes Schindelin]

It is okay in practice to test for forward slashes in the output of
getcwd(), because we go out of our way to convert backslashes to forward
slashes in getcwd()'s output on Windows.

Still, the correct way to test for a dir separator is by using the
helper function we introduced for that very purpose. It also serves as a
good documentation what the code tries to do (not "how").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index 967f289f1ef..89a0cef9231 100644
--- a/setup.c
+++ b/setup.c
@@ -910,7 +910,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 			return setup_bare_git_dir(&cwd, offset, nongit_ok);
 
 		offset_parent = offset;
-		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
+		while (--offset_parent > ceil_offset &&
+		       !is_dir_sep(dir->buf[offset_parent]));
 		if (offset_parent <= ceil_offset)
 			return setup_nongit(cwd.buf, nongit_ok);
 		if (one_filesystem) {
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 2/9] setup_git_directory(): use is_dir_sep() helper

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:07AM +0100, Johannes Schindelin wrote:

> It is okay in practice to test for forward slashes in the output of
> getcwd(), because we go out of our way to convert backslashes to forward
> slashes in getcwd()'s output on Windows.
> 
> Still, the correct way to test for a dir separator is by using the
> helper function we introduced for that very purpose. It also serves as a
> good documentation what the code tries to do (not "how").

Makes sense, but...

> @@ -910,7 +910,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
>  			return setup_bare_git_dir(&cwd, offset, nongit_ok);
>  
>  		offset_parent = offset;
> -		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
> +		while (--offset_parent > ceil_offset &&
> +		       !is_dir_sep(dir->buf[offset_parent]));

What is "dir"? I'm guessing this patch got reordered and it should stay
as cwd.buf?

-Peff

Re: [PATCH v2 2/9] setup_git_directory(): use is_dir_sep() helper

[Johannes Schindelin]

Hi Peff,

On Thu, 2 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:04:07AM +0100, Johannes Schindelin wrote:
> 
> > It is okay in practice to test for forward slashes in the output of
> > getcwd(), because we go out of our way to convert backslashes to
> > forward slashes in getcwd()'s output on Windows.
> > 
> > Still, the correct way to test for a dir separator is by using the
> > helper function we introduced for that very purpose. It also serves as
> > a good documentation what the code tries to do (not "how").
> 
> Makes sense, but...
> 
> > @@ -910,7 +910,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
> >  			return setup_bare_git_dir(&cwd, offset, nongit_ok);
> >  
> >  		offset_parent = offset;
> > -		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
> > +		while (--offset_parent > ceil_offset &&
> > +		       !is_dir_sep(dir->buf[offset_parent]));
> 
> What is "dir"? I'm guessing this patch got reordered and it should stay
> as cwd.buf?

Oh drats. Usually I do a final `git rebase -x "make test" upstream/master`
run before submitting, but I was really, really tired by the end of that
stretch.

Thanks for being thorough (and I fixed it, of course),
Dscho

Re: [PATCH v2 2/9] setup_git_directory(): use is_dir_sep() helper

[Jeff King]

On Fri, Mar 03, 2017 at 12:16:31PM +0100, Johannes Schindelin wrote:

> > What is "dir"? I'm guessing this patch got reordered and it should stay
> > as cwd.buf?
> 
> Oh drats. Usually I do a final `git rebase -x "make test" upstream/master`
> run before submitting, but I was really, really tired by the end of that
> stretch.

I usually do the same, and have done the "too tired" thing, too, only to
have it bite me. That's why I so readily recognized the problem. :)

I've recently switched to using Michael's "git test" program[1], which
caches the test results for each tree in a git-note. That makes the
final "rebase -x" a lot less painful if you've left the early commits
alone.

The python dependency might be a blocker for you, but I suspect the
caching parts would be easy to hack together with shell.

-Peff

[1] https://github.com/mhagger/git-test

Re: [PATCH v2 2/9] setup_git_directory(): use is_dir_sep() helper

[Johannes Schindelin]

Hi Peff,

On Fri, 3 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 12:16:31PM +0100, Johannes Schindelin wrote:
> 
> > > What is "dir"? I'm guessing this patch got reordered and it should
> > > stay as cwd.buf?
> > 
> > Oh drats. Usually I do a final `git rebase -x "make test"
> > upstream/master` run before submitting, but I was really, really tired
> > by the end of that stretch.
> 
> I usually do the same, and have done the "too tired" thing, too, only to
> have it bite me. That's why I so readily recognized the problem. :)

:-)

> I've recently switched to using Michael's "git test" program[1], which
> caches the test results for each tree in a git-note. That makes the
> final "rebase -x" a lot less painful if you've left the early commits
> alone.

Good point. I meant to have a look, but got really overwhelmed with other
things.

> The python dependency might be a blocker for you, but I suspect the
> caching parts would be easy to hack together with shell.

No, personally I have no problem with Python. If you asked me to include
it in Git for Windows' installer, increasing the size noticably, that
would be a totally different matter, of course.

Ciao,
Dscho

[PATCH v2 3/9] setup_git_directory(): avoid changing global state during discovery

[Johannes Schindelin]

For historical reasons, Git searches for the .git/ directory (or the
.git file) by changing the working directory successively to the parent
directory of the current directory, until either anything was found or
until a ceiling or a mount point is hit.

Further global state may be changed, depending on the actual type of
discovery, e.g. the global variable `repository_format_precious_objects`
is set in the `check_repository_format_gently()` function (which is a
bit surprising, given the function name).

We do have a use case where we would like to find the .git/ directory
without having any global state touched, though: when we read the early
config e.g. for the pager or for alias expansion.

Let's just rename the function `setup_git_directory_gently_1()` to
`discover_git_directory()` and move all code that changes any global
state back into `setup_git_directory_gently()`.

In subsequent patches, we will export the `discover_git_directory()`
function and make use of it.

Note: the new loop is a *little* tricky, as we have to handle the root
directory specially: we cannot simply strip away the last component
including the slash, as the root directory only has that slash. To remedy
that, we introduce the `min_offset` variable that holds the minimal length
of an absolute path, and using that to special-case the root directory,
including an early exit before trying to find the parent of the root
directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 187 ++++++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 112 insertions(+), 75 deletions(-)

diff --git a/setup.c b/setup.c
index 89a0cef9231..edac3c27dc1 100644
--- a/setup.c
+++ b/setup.c
@@ -816,50 +816,49 @@ static int canonicalize_ceiling_entry(struct string_list_item *item,
 	}
 }
 
+enum discovery_result {
+	GIT_DIR_NONE = 0,
+	GIT_DIR_EXPLICIT,
+	GIT_DIR_DISCOVERED,
+	GIT_DIR_BARE,
+	/* these are errors */
+	GIT_DIR_HIT_CEILING = -1,
+	GIT_DIR_HIT_MOUNT_POINT = -2
+};
+
 /*
  * We cannot decide in this function whether we are in the work tree or
  * not, since the config can only be read _after_ this function was called.
+ *
+ * Also, we avoid changing any global state (such as the current working
+ * directory) to allow early callers.
+ *
+ * The directory where the search should start needs to be passed in via the
+ * `dir` parameter; upon return, the `dir` buffer will contain the path of
+ * the directory where the search ended, and `gitdir` will contain the path of
+ * the discovered .git/ directory, if any. This path may be relative against
+ * `dir` (i.e. *not* necessarily the cwd).
  */
-static const char *setup_git_directory_gently_1(int *nongit_ok)
+static enum discovery_result discover_git_directory(struct strbuf *dir,
+						    struct strbuf *gitdir)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
-	static struct strbuf cwd = STRBUF_INIT;
-	const char *gitdirenv, *ret;
-	char *gitfile;
-	int offset, offset_parent, ceil_offset = -1;
+	const char *gitdirenv;
+	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
 	dev_t current_device = 0;
 	int one_filesystem = 1;
 
 	/*
-	 * We may have read an incomplete configuration before
-	 * setting-up the git directory. If so, clear the cache so
-	 * that the next queries to the configuration reload complete
-	 * configuration (including the per-repo config file that we
-	 * ignored previously).
-	 */
-	git_config_clear();
-
-	/*
-	 * Let's assume that we are in a git repository.
-	 * If it turns out later that we are somewhere else, the value will be
-	 * updated accordingly.
-	 */
-	if (nongit_ok)
-		*nongit_ok = 0;
-
-	if (strbuf_getcwd(&cwd))
-		die_errno(_("Unable to read current working directory"));
-	offset = cwd.len;
-
-	/*
 	 * If GIT_DIR is set explicitly, we're not going
 	 * to do any discovery, but we still do repository
 	 * validation.
 	 */
 	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
-	if (gitdirenv)
-		return setup_explicit_git_dir(gitdirenv, &cwd, nongit_ok);
+	if (gitdirenv) {
+		strbuf_addstr(gitdir, gitdirenv);
+		return GIT_DIR_EXPLICIT;
+	}
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -867,15 +866,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP, -1);
 		filter_string_list(&ceiling_dirs, 0,
 				   canonicalize_ceiling_entry, &empty_entry_found);
-		ceil_offset = longest_ancestor_length(cwd.buf, &ceiling_dirs);
+		ceil_offset = longest_ancestor_length(dir->buf, &ceiling_dirs);
 		string_list_clear(&ceiling_dirs, 0);
 	}
 
-	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
-		ceil_offset = 1;
+	if (ceil_offset < 0)
+		ceil_offset = min_offset - 2;
 
 	/*
-	 * Test in the following order (relative to the cwd):
+	 * Test in the following order (relative to the dir):
 	 * - .git (file containing "gitdir: <path>")
 	 * - .git/
 	 * - ./ (bare)
@@ -887,62 +886,100 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	 */
 	one_filesystem = !git_env_bool("GIT_DISCOVERY_ACROSS_FILESYSTEM", 0);
 	if (one_filesystem)
-		current_device = get_device_or_die(".", NULL, 0);
+		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
-		if (gitfile)
-			gitdirenv = gitfile = xstrdup(gitfile);
-		else {
-			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
-				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
-		}
+		int offset = dir->len;
 
+		if (offset > min_offset)
+			strbuf_addch(dir, '/');
+		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
+		gitdirenv = read_gitfile(dir->buf);
+		if (!gitdirenv && is_git_directory(dir->buf))
+			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		strbuf_setlen(dir, offset);
 		if (gitdirenv) {
-			ret = setup_discovered_git_dir(gitdirenv,
-						       &cwd, offset,
-						       nongit_ok);
-			free(gitfile);
-			return ret;
+			strbuf_addstr(gitdir, gitdirenv);
+			return GIT_DIR_DISCOVERED;
 		}
-		free(gitfile);
 
-		if (is_git_directory("."))
-			return setup_bare_git_dir(&cwd, offset, nongit_ok);
-
-		offset_parent = offset;
-		while (--offset_parent > ceil_offset &&
-		       !is_dir_sep(dir->buf[offset_parent]));
-		if (offset_parent <= ceil_offset)
-			return setup_nongit(cwd.buf, nongit_ok);
-		if (one_filesystem) {
-			dev_t parent_device = get_device_or_die("..", cwd.buf,
-								offset);
-			if (parent_device != current_device) {
-				if (nongit_ok) {
-					if (chdir(cwd.buf))
-						die_errno(_("Cannot come back to cwd"));
-					*nongit_ok = 1;
-					return NULL;
-				}
-				strbuf_setlen(&cwd, offset);
-				die(_("Not a git repository (or any parent up to mount point %s)\n"
-				"Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
-				    cwd.buf);
-			}
-		}
-		if (chdir("..")) {
-			strbuf_setlen(&cwd, offset);
-			die_errno(_("Cannot change to '%s/..'"), cwd.buf);
+		if (is_git_directory(dir->buf)) {
+			strbuf_addstr(gitdir, ".");
+			return GIT_DIR_BARE;
 		}
-		offset = offset_parent;
+
+		if (offset <= min_offset)
+			return GIT_DIR_HIT_CEILING;
+
+		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]));
+		if (offset <= ceil_offset)
+			return GIT_DIR_HIT_CEILING;
+
+		strbuf_setlen(dir, offset > min_offset ?  offset : min_offset);
+		if (one_filesystem &&
+		    current_device != get_device_or_die(dir->buf, NULL, offset))
+			return GIT_DIR_HIT_MOUNT_POINT;
 	}
 }
 
 const char *setup_git_directory_gently(int *nongit_ok)
 {
+	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
 	const char *prefix;
 
-	prefix = setup_git_directory_gently_1(nongit_ok);
+	/*
+	 * We may have read an incomplete configuration before
+	 * setting-up the git directory. If so, clear the cache so
+	 * that the next queries to the configuration reload complete
+	 * configuration (including the per-repo config file that we
+	 * ignored previously).
+	 */
+	git_config_clear();
+
+	/*
+	 * Let's assume that we are in a git repository.
+	 * If it turns out later that we are somewhere else, the value will be
+	 * updated accordingly.
+	 */
+	if (nongit_ok)
+		*nongit_ok = 0;
+
+	if (strbuf_getcwd(&cwd))
+		die_errno(_("Unable to read current working directory"));
+	strbuf_addbuf(&dir, &cwd);
+
+	switch (discover_git_directory(&dir, &gitdir)) {
+	case GIT_DIR_NONE:
+		prefix = NULL;
+		break;
+	case GIT_DIR_EXPLICIT:
+		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
+		break;
+	case GIT_DIR_DISCOVERED:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
+						  nongit_ok);
+		break;
+	case GIT_DIR_BARE:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
+		break;
+	case GIT_DIR_HIT_CEILING:
+		prefix = setup_nongit(cwd.buf, nongit_ok);
+		break;
+	case GIT_DIR_HIT_MOUNT_POINT:
+		if (nongit_ok) {
+			*nongit_ok = 1;
+			return NULL;
+		}
+		die(_("Not a git repository (or any parent up to mount point %s)\n"
+		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
+		    dir.buf);
+	default:
+		die("BUG: unhandled discover_git_directory() result");
+	}
+
 	if (prefix)
 		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
 	else
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 3/9] setup_git_directory(): avoid changing global state during discovery

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:11AM +0100, Johannes Schindelin wrote:

> For historical reasons, Git searches for the .git/ directory (or the
> .git file) by changing the working directory successively to the parent
> directory of the current directory, until either anything was found or
> until a ceiling or a mount point is hit.

This is starting to get into the meat of changes we've been putting off
writing for ages. I'll read with my fingers crossed. :)

> Further global state may be changed, depending on the actual type of
> discovery, e.g. the global variable `repository_format_precious_objects`
> is set in the `check_repository_format_gently()` function (which is a
> bit surprising, given the function name).

It's gentle in the sense that if it does not find a valid repo, it
touches no state. I do suspect the functions you want are the ones it
builds on: read_repository_format() and verify_repository_format(),
which I added not too long ago for the exact purpose of checking repo
config without touching anything global.

> We do have a use case where we would like to find the .git/ directory
> without having any global state touched, though: when we read the early
> config e.g. for the pager or for alias expansion.
> 
> Let's just rename the function `setup_git_directory_gently_1()` to
> `discover_git_directory()` and move all code that changes any global
> state back into `setup_git_directory_gently()`.

Given the earlier paragraph, it sounds like you want to move the
global-state-changing parts out of check_repository_format_gently(). But
that wouldn't be right; it's triggered separate from the discovery
process by things like enter_repo().

However, I don't see that happening in the patch, which is good. I just
wonder if the earlier paragraph should really be complaining about how
setup_git_directory() (and its callees) touches a lot of global state,
not check_repository_format_gently(), whose use is but one of multiple
global-state modifications.

> Note: the new loop is a *little* tricky, as we have to handle the root
> directory specially: we cannot simply strip away the last component
> including the slash, as the root directory only has that slash. To remedy
> that, we introduce the `min_offset` variable that holds the minimal length
> of an absolute path, and using that to special-case the root directory,
> including an early exit before trying to find the parent of the root
> directory.

I wondered how t1509 fared with this, as it is the only test of
repositories at the root directory (and it is not run by default because
it involves a bunch of flaky and expensive chroot setup). Unfortunately
it seems to fail with your patch:

  expecting success: 
  		test_cmp_val "foo/" "$(git rev-parse --show-prefix)"
  	
  --- expected
  +++ result
  @@ -1 +1 @@
  -foo/
  +oo/
  not ok 66 - auto gitdir, foo: prefix

Could the problem be this:

> +	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
> [...]
> -	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
> -		ceil_offset = 1;
> +	if (ceil_offset < 0)
> +		ceil_offset = min_offset - 2;

It works the same as before in the dos-drive case, but we'd get
ceil_offset = -1 otherwise.  Which seems weird, but maybe I don't
understand how ceil_offset is supposed to work.

Interestingly, I don't think this is the bug, though. We still correctly
find /.git as a repository. The problem seems to happen later, in
setup_discovered_git_dir(), which does this:

  /* Make "offset" point to past the '/', and add a '/' at the end */
  offset++;
  strbuf_addch(cwd, '/');
  return cwd->buf + offset;

Here, "offset" is the length of the working tree path. The root
directory case differs from normal in that "offset" already accounts for
the trailing slash.

So I think the bug comes from:

> -			ret = setup_discovered_git_dir(gitdirenv,
> -						       &cwd, offset,
> -						       nongit_ok);
> [...]
> +		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
> +						  nongit_ok);

The original knew that "offset" took into account the off-by-one for the
root, but that's lost when we use dir.len. I haven't studied it enough
to know the best solution, but I suspect you will.

Other than this bug, I very much like the direction that this patch is
taking us.

-Peff

Re: [PATCH v2 3/9] setup_git_directory(): avoid changing global state during discovery

[Johannes Schindelin]

Hi Peff,

On Thu, 2 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:04:11AM +0100, Johannes Schindelin wrote:
> 
> > For historical reasons, Git searches for the .git/ directory (or the
> > .git file) by changing the working directory successively to the
> > parent directory of the current directory, until either anything was
> > found or until a ceiling or a mount point is hit.
> 
> This is starting to get into the meat of changes we've been putting off
> writing for ages. I'll read with my fingers crossed. :)

Heh.

> > Further global state may be changed, depending on the actual type of
> > discovery, e.g. the global variable
> > `repository_format_precious_objects` is set in the
> > `check_repository_format_gently()` function (which is a bit
> > surprising, given the function name).
> 
> It's gentle in the sense that if it does not find a valid repo, it
> touches no state. I do suspect the functions you want are the ones it
> builds on: read_repository_format() and verify_repository_format(),
> which I added not too long ago for the exact purpose of checking repo
> config without touching anything global.

Okay. But I think that my interpretation of the word "gently" is as valid
as Git's source code's.

> > We do have a use case where we would like to find the .git/ directory
> > without having any global state touched, though: when we read the early
> > config e.g. for the pager or for alias expansion.
> > 
> > Let's just rename the function `setup_git_directory_gently_1()` to
> > `discover_git_directory()` and move all code that changes any global
> > state back into `setup_git_directory_gently()`.
> 
> Given the earlier paragraph, it sounds like you want to move the
> global-state-changing parts out of check_repository_format_gently(). But
> that wouldn't be right; it's triggered separate from the discovery
> process by things like enter_repo().

Oh, right. I really only meant to move the global-state-changing parts out
of the discover_git_directory().

> However, I don't see that happening in the patch, which is good. I just
> wonder if the earlier paragraph should really be complaining about how
> setup_git_directory() (and its callees) touches a lot of global state,
> not check_repository_format_gently(), whose use is but one of multiple
> global-state modifications.

Okay, I'll try my best to rephrase the commit message.

For good measure, I will also keep the name setup_git_directory_gently_1()
because it won't get exported directly (I made up my mind about wrapping
that function to allow for an easier interface that does *not* return the
"cdup").

> > Note: the new loop is a *little* tricky, as we have to handle the root
> > directory specially: we cannot simply strip away the last component
> > including the slash, as the root directory only has that slash. To
> > remedy that, we introduce the `min_offset` variable that holds the
> > minimal length of an absolute path, and using that to special-case the
> > root directory, including an early exit before trying to find the
> > parent of the root directory.
> 
> I wondered how t1509 fared with this, as it is the only test of
> repositories at the root directory (and it is not run by default because
> it involves a bunch of flaky and expensive chroot setup).

Oh, thanks. I allowed myself to forget about that test script (and did a
lot of testing by hand, but of course *during* the development of v2, not
when I had finished...).

> Unfortunately it seems to fail with your patch:
> 
>   expecting success: 
>   		test_cmp_val "foo/" "$(git rev-parse --show-prefix)"
>   	
>   --- expected
>   +++ result
>   @@ -1 +1 @@
>   -foo/
>   +oo/
>   not ok 66 - auto gitdir, foo: prefix

I can reproduce this failure here.

Side note: it took a while until I realized that the prepare-chroot.sh
script has to be run *every time* I change *anything* in either Git's
source code or in the test script.

> Could the problem be this:
> 
> > +	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
> > [...]
> > -	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
> > -		ceil_offset = 1;
> > +	if (ceil_offset < 0)
> > +		ceil_offset = min_offset - 2;

Yes. The previous code did not need cwd.buf[0..offset] to be a valid path,
but it needed the offset to point to the trailing slash, if any.

> Interestingly, I don't think this is the bug, though. We still correctly
> find /.git as a repository. The problem seems to happen later, in
> setup_discovered_git_dir(), which does this:
> 
>   /* Make "offset" point to past the '/', and add a '/' at the end */
>   offset++;
>   strbuf_addch(cwd, '/');
>   return cwd->buf + offset;

I fixed this by ensuring that we only increment the offset if it is not
already pointing at the end of the first offset (which handles Windows
paths correctly, too).

> Other than this bug, I very much like the direction that this patch is
> taking us.

Awesome. I was anxious to hear something like that.

Ciao,
Dscho

[PATCH v2 4/9] Export the discover_git_directory() function

[Johannes Schindelin]

The function we introduced earlier needs to return both the path to the
.git/ directory as well as the "cd-up" path to allow
setup_git_directory() to retain its previous behavior as if it changed
the current working directory on its quest for the .git/ directory.

Let's rename it and export a function with an easier signature that
*just* discovers the .git/ directory.

We will use it in a subsequent patch to support early config reading
better.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h |  1 +
 setup.c | 29 ++++++++++++++++++++++++++---
 2 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index 80b6372cf76..a104b76c02e 100644
--- a/cache.h
+++ b/cache.h
@@ -518,6 +518,7 @@ extern void set_git_work_tree(const char *tree);
 #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
 
 extern void setup_work_tree(void);
+extern const char *discover_git_directory(struct strbuf *gitdir);
 extern const char *setup_git_directory_gently(int *);
 extern const char *setup_git_directory(void);
 extern char *prefix_path(const char *prefix, int len, const char *path);
diff --git a/setup.c b/setup.c
index edac3c27dc1..7ceca6cc6ef 100644
--- a/setup.c
+++ b/setup.c
@@ -839,8 +839,8 @@ enum discovery_result {
  * the discovered .git/ directory, if any. This path may be relative against
  * `dir` (i.e. *not* necessarily the cwd).
  */
-static enum discovery_result discover_git_directory(struct strbuf *dir,
-						    struct strbuf *gitdir)
+static enum discovery_result discover_git_directory_1(struct strbuf *dir,
+						      struct strbuf *gitdir)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
@@ -921,6 +921,29 @@ static enum discovery_result discover_git_directory(struct strbuf *dir,
 	}
 }
 
+const char *discover_git_directory(struct strbuf *gitdir)
+{
+	struct strbuf dir = STRBUF_INIT;
+	int len;
+
+	if (strbuf_getcwd(&dir))
+		return NULL;
+
+	len = dir.len;
+	if (discover_git_directory_1(&dir, gitdir) < 0) {
+		strbuf_release(&dir);
+		return NULL;
+	}
+
+	if (dir.len < len && !is_absolute_path(gitdir->buf)) {
+		strbuf_addch(&dir, '/');
+		strbuf_insert(gitdir, 0, dir.buf, dir.len);
+	}
+	strbuf_release(&dir);
+
+	return gitdir->buf;
+}
+
 const char *setup_git_directory_gently(int *nongit_ok)
 {
 	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
@@ -947,7 +970,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die_errno(_("Unable to read current working directory"));
 	strbuf_addbuf(&dir, &cwd);
 
-	switch (discover_git_directory(&dir, &gitdir)) {
+	switch (discover_git_directory_1(&dir, &gitdir)) {
 	case GIT_DIR_NONE:
 		prefix = NULL;
 		break;
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 4/9] Export the discover_git_directory() function

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:15AM +0100, Johannes Schindelin wrote:

> The function we introduced earlier needs to return both the path to the
> .git/ directory as well as the "cd-up" path to allow
> setup_git_directory() to retain its previous behavior as if it changed
> the current working directory on its quest for the .git/ directory.
> 
> Let's rename it and export a function with an easier signature that
> *just* discovers the .git/ directory.
> 
> We will use it in a subsequent patch to support early config reading
> better.
> 

Seems like an obviously good next step.

> diff --git a/cache.h b/cache.h
> index 80b6372cf76..a104b76c02e 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -518,6 +518,7 @@ extern void set_git_work_tree(const char *tree);
>  #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
>  
>  extern void setup_work_tree(void);
> +extern const char *discover_git_directory(struct strbuf *gitdir);

Perhaps it's worth adding a short docstring describing the function. I
know that would make it unlike all of its neighbors, but it is not
immediately obvious to me what the return value is (or whether gitdir is
an input or output parameter).

> +const char *discover_git_directory(struct strbuf *gitdir)
> +{
> +	struct strbuf dir = STRBUF_INIT;
> +	int len;

Nit: please use size_t for storing strbuf lengths.

> +	if (strbuf_getcwd(&dir))
> +		return NULL;
> +
> +	len = dir.len;
> +	if (discover_git_directory_1(&dir, gitdir) < 0) {
> +		strbuf_release(&dir);
> +		return NULL;
> +	}
> +
> +	if (dir.len < len && !is_absolute_path(gitdir->buf)) {
> +		strbuf_addch(&dir, '/');
> +		strbuf_insert(gitdir, 0, dir.buf, dir.len);
> +	}
> +	strbuf_release(&dir);

I was confused by two things here.

One is that because I was wondering whether "gitdir" was supposed to be
passed empty or not, it wasn't clear that this insert is doing the right
thing.  If there _is_ something in it, then discover_git_directory_1()
would just append to it, which makes sense. But then this insert blindly
sticks the absolute-path bit at the front of the string, leaving
whatever was originally there spliced into the middle of the path.
Doing:

  size_t base = gitdir->len;
  ...
  strbuf_insert(gitdir, base, dir.buf, dir.len);

would solve that. It's probably not that likely for somebody to do:

  strbuf_addstr(&buf, "my git dir is ");
  discover_git_directory(&buf);

but since it's not much effort, it might be worth making it work.

The second is that I don't quite understand why we only make the result
absolute when we walked upwards. In git.git, the result of the function
in various directories is:

  - ".git", when we're at the root of the worktree
  - "/home/peff/git/.git, when we're in "t/"
  - "." when we're already in ".git"
  - "/home/peff/git/.git/." when we're in ".git/objects"

I'm not sure if some of those cases are not behaving as intended, or
there's some reason for the differences that I don't quite understand.

-Peff

Re: [PATCH v2 4/9] Export the discover_git_directory() function

[Johannes Schindelin]

Hi Peff,

On Thu, 2 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:04:15AM +0100, Johannes Schindelin wrote:
> 
> > diff --git a/cache.h b/cache.h
> > index 80b6372cf76..a104b76c02e 100644
> > --- a/cache.h
> > +++ b/cache.h
> > @@ -518,6 +518,7 @@ extern void set_git_work_tree(const char *tree);
> >  #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
> >  
> >  extern void setup_work_tree(void);
> > +extern const char *discover_git_directory(struct strbuf *gitdir);
> 
> Perhaps it's worth adding a short docstring describing the function.

Okay.

> > +const char *discover_git_directory(struct strbuf *gitdir)
> > +{
> > +	struct strbuf dir = STRBUF_INIT;
> > +	int len;
> 
> Nit: please use size_t for storing strbuf lengths.

Okay.

> > +	if (strbuf_getcwd(&dir))
> > +		return NULL;
> > +
> > +	len = dir.len;
> > +	if (discover_git_directory_1(&dir, gitdir) < 0) {
> > +		strbuf_release(&dir);
> > +		return NULL;
> > +	}
> > +
> > +	if (dir.len < len && !is_absolute_path(gitdir->buf)) {
> > +		strbuf_addch(&dir, '/');
> > +		strbuf_insert(gitdir, 0, dir.buf, dir.len);
> > +	}
> > +	strbuf_release(&dir);
> 
> I was confused by two things here.
> 
> One is that because I was wondering whether "gitdir" was supposed to be
> passed empty or not, it wasn't clear that this insert is doing the right
> thing.  If there _is_ something in it, then discover_git_directory_1()
> would just append to it, which makes sense. But then this insert blindly
> sticks the absolute-path bit at the front of the string, leaving
> whatever was originally there spliced into the middle of the path.
> Doing:
> 
>   size_t base = gitdir->len;
>   ...
>   strbuf_insert(gitdir, base, dir.buf, dir.len);
> 
> would solve that.

And of course the is_absolute_path() call also needs to offset `gitdir->buf
+ base`.

> It's probably not that likely for somebody to do:
> 
>   strbuf_addstr(&buf, "my git dir is ");
>   discover_git_directory(&buf);
> 
> but since it's not much effort, it might be worth making it work.

Plus, I have no assert()s in place to ensure any expectation to the
contrary. So I fixed it as you suggested.

> The second is that I don't quite understand why we only make the result
> absolute when we walked upwards. In git.git, the result of the function
> in various directories is:
> 
>   - ".git", when we're at the root of the worktree
>   - "/home/peff/git/.git, when we're in "t/"
>   - "." when we're already in ".git"
>   - "/home/peff/git/.git/." when we're in ".git/objects"
> > I'm not sure if some of those cases are not behaving as intended, or
> there's some reason for the differences that I don't quite understand.

Yes, some of those cases do not behave as intended: it is true that
setup_git_directory() sets git_dir to "/home/peff/git/.git" when we
(actually, you, given that my home directory is different) are in "t/",
but setup_git_directory_gently_1() would set gitdir to ".git" and dir to
"/home/peff/git". But the current directory is still what cwd says it is:
"/home/peff/git/t".

I added a comment:

        /*
         * The returned gitdir is relative to dir, and if dir does not reflect
         * the current working directory, we simply make the gitdir
         * absolute.
         */

And thanks: you reminded me of another issue I wanted to address but
forgot: if gitdir is ".", I do not want the resulting absolute path to
have a trailing "/.". I fixed that, too.

Ciao,
Dscho

[PATCH v2 5/9] Make read_early_config() reusable

[Johannes Schindelin]

The pager configuration needs to be read early, possibly before
discovering any .git/ directory.

Let's not hide this function in pager.c, but make it available to other
callers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h  |  1 +
 config.c | 31 +++++++++++++++++++++++++++++++
 pager.c  | 31 -------------------------------
 3 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/cache.h b/cache.h
index a104b76c02e..6b6780064f0 100644
--- a/cache.h
+++ b/cache.h
@@ -1798,6 +1798,7 @@ extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
 				     const unsigned char *sha1, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
diff --git a/config.c b/config.c
index c6b874a7bf7..9cfbeafd04c 100644
--- a/config.c
+++ b/config.c
@@ -1412,6 +1412,37 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
+void read_early_config(config_fn_t cb, void *data)
+{
+	git_config_with_options(cb, data, NULL, 1);
+
+	/*
+	 * Note that this is a really dirty hack that does the wrong thing in
+	 * many cases. The crux of the problem is that we cannot run
+	 * setup_git_directory() early on in git's setup, so we have no idea if
+	 * we are in a repository or not, and therefore are not sure whether
+	 * and how to read repository-local config.
+	 *
+	 * So if we _aren't_ in a repository (or we are but we would reject its
+	 * core.repositoryformatversion), we'll read whatever is in .git/config
+	 * blindly. Similarly, if we _are_ in a repository, but not at the
+	 * root, we'll fail to find .git/config (because it's really
+	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 *
+	 * However, we have historically provided this hack because it does
+	 * work some of the time (namely when you are at the top-level of a
+	 * valid repository), and would rarely make things worse (i.e., you do
+	 * not generally have a .git/config file sitting around).
+	 */
+	if (!startup_info->have_repository) {
+		struct git_config_source repo_config;
+
+		memset(&repo_config, 0, sizeof(repo_config));
+		repo_config.file = ".git/config";
+		git_config_with_options(cb, data, &repo_config, 1);
+	}
+}
+
 static void git_config_check_init(void);
 
 void git_config(config_fn_t fn, void *data)
diff --git a/pager.c b/pager.c
index ae796433630..73ca8bc3b17 100644
--- a/pager.c
+++ b/pager.c
@@ -43,37 +43,6 @@ static int core_pager_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
-static void read_early_config(config_fn_t cb, void *data)
-{
-	git_config_with_options(cb, data, NULL, 1);
-
-	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
-	 */
-	if (!startup_info->have_repository) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
-		git_config_with_options(cb, data, &repo_config, 1);
-	}
-}
-
 const char *git_pager(int stdout_is_tty)
 {
 	const char *pager;
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 5/9] Make read_early_config() reusable

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:20AM +0100, Johannes Schindelin wrote:

> The pager configuration needs to be read early, possibly before
> discovering any .git/ directory.
> 
> Let's not hide this function in pager.c, but make it available to other
> callers.
> [...]
> +	 * Note that this is a really dirty hack that does the wrong thing in
> +	 * many cases. The crux of the problem is that we cannot run

Makes sense. I'll assume the words "dirty hack" disappear from this
now-public function as you fix it up in a future patch. :)

-Peff

Re: [PATCH v2 5/9] Make read_early_config() reusable

[Johannes Schindelin]

Hi Peff,

On Thu, 2 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:04:20AM +0100, Johannes Schindelin wrote:
> 
> > The pager configuration needs to be read early, possibly before
> > discovering any .git/ directory.
> > 
> > Let's not hide this function in pager.c, but make it available to other
> > callers.
> > [...]
> > +	 * Note that this is a really dirty hack that does the wrong thing in
> > +	 * many cases. The crux of the problem is that we cannot run
> 
> Makes sense. I'll assume the words "dirty hack" disappear from this
> now-public function as you fix it up in a future patch. :)

Oops. I did not even think about that.

Fixed,
Dscho

[PATCH v2 6/9] read_early_config(): special-case builtins that create a repository

[Johannes Schindelin]

When the command we are about to execute wants to create a repository
(i.e. the command is `init` or `clone`), we *must not* look for a
repository config.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h  | 2 +-
 config.c | 3 ++-
 git.c    | 3 +++
 3 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/cache.h b/cache.h
index 6b6780064f0..0af7141242f 100644
--- a/cache.h
+++ b/cache.h
@@ -2070,7 +2070,7 @@ const char *split_cmdline_strerror(int cmdline_errno);
 
 /* setup.c */
 struct startup_info {
-	int have_repository;
+	int have_repository, creating_repository;
 	const char *prefix;
 };
 extern struct startup_info *startup_info;
diff --git a/config.c b/config.c
index 9cfbeafd04c..980fcc6ff2e 100644
--- a/config.c
+++ b/config.c
@@ -1434,7 +1434,8 @@ void read_early_config(config_fn_t cb, void *data)
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->have_repository) {
+	if (!startup_info->creating_repository &&
+	    !startup_info->have_repository) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
diff --git a/git.c b/git.c
index 33f52acbcc8..9fb9bb90a21 100644
--- a/git.c
+++ b/git.c
@@ -337,6 +337,9 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
 	struct stat st;
 	const char *prefix;
 
+	if (p->fn == cmd_init_db || p->fn == cmd_clone)
+		startup_info->creating_repository = 1;
+
 	prefix = NULL;
 	help = argc == 2 && !strcmp(argv[1], "-h");
 	if (!help) {
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 6/9] read_early_config(): special-case builtins that create a repository

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:24AM +0100, Johannes Schindelin wrote:

> When the command we are about to execute wants to create a repository
> (i.e. the command is `init` or `clone`), we *must not* look for a
> repository config.

Hmm. I'm not sure if this one is worth the hackery here.

Yes, it would be wrong for init or clone to read something like
core.sharedrepository from a repo it happens to be in. But I wonder if
it would be cleaner to consider calls to read_early_config their own
"pre-command" stage that may respect global config, or config in a repo
directory you happen to be sitting in.

Because I think for aliases, we're going to end up having to do that
anyway (you won't know that your alias is "clone" until you've resolved
it!). And I think the pager fits into this "pre-command" concept, too
(we already have "-p" as a pre-command option on the command-line).

I dunno. It probably doesn't matter _too much_ either way. But it's one
less hack to maintain going forward, and it also makes your "git-init
respects the pager" into the normal, consistent thing.

-Peff

Re: [PATCH v2 6/9] read_early_config(): special-case builtins that create a repository

[Johannes Schindelin]

Hi Peff,

On Thu, 2 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:04:24AM +0100, Johannes Schindelin wrote:
> 
> > When the command we are about to execute wants to create a repository
> > (i.e. the command is `init` or `clone`), we *must not* look for a
> > repository config.
> 
> Hmm. I'm not sure if this one is worth the hackery here.

I guess you're right. Let me think about this for a while in the back of
my head while I work on other parts of the patch series.

Ciao,
Dscho

[PATCH v2 7/9] read_early_config(): avoid .git/config hack when unneeded

[Johannes Schindelin]

So far, we only look whether the startup_info claims to have seen a
git_dir.

However, do_git_config_sequence() (and consequently the
git_config_with_options() call used by read_early_config() asks the
have_git_dir() function whether we have a .git/ directory, which in turn
also looks at git_dir and at the environment variable GIT_DIR. And when
this is the case, the repository config is handled already, so we do not
have to do that again explicitly.

Let's just use the same function, have_git_dir(), to determine whether we
have to handle .git/config explicitly.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/config.c b/config.c
index 980fcc6ff2e..3ee5e5a15d6 100644
--- a/config.c
+++ b/config.c
@@ -1434,8 +1434,7 @@ void read_early_config(config_fn_t cb, void *data)
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->creating_repository &&
-	    !startup_info->have_repository) {
+	if (!startup_info->creating_repository && !have_git_dir()) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 7/9] read_early_config(): avoid .git/config hack when unneeded

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:28AM +0100, Johannes Schindelin wrote:

> So far, we only look whether the startup_info claims to have seen a
> git_dir.
> 
> However, do_git_config_sequence() (and consequently the
> git_config_with_options() call used by read_early_config() asks the
> have_git_dir() function whether we have a .git/ directory, which in turn
> also looks at git_dir and at the environment variable GIT_DIR. And when
> this is the case, the repository config is handled already, so we do not
> have to do that again explicitly.
> 
> Let's just use the same function, have_git_dir(), to determine whether we
> have to handle .git/config explicitly.

Good, this makes sense.

-Peff

[PATCH v2 8/9] read_early_config(): really discover .git/

[Johannes Schindelin]

Earlier, we punted and simply assumed that we are in the top-level
directory of the project, and that there is no .git file but a .git/
directory so that we can read directly from .git/config.

However, that is not necessarily true. We may be in a subdirectory. Or
.git may be a gitfile. Or the environment variable GIT_DIR may be set.

To remedy this situation, we just refactored the way
setup_git_directory() discovers the .git/ directory, to make it
reusable, and more importantly, to leave all global variables and the
current working directory alone.

Let's discover the .git/ directory correctly in read_early_config() by
using that new function.

This fixes 4 known breakages in t7006.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c         | 9 +++++++--
 t/t7006-pager.sh | 8 ++++----
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/config.c b/config.c
index 3ee5e5a15d6..bcda397d42e 100644
--- a/config.c
+++ b/config.c
@@ -1414,6 +1414,8 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 
 void read_early_config(config_fn_t cb, void *data)
 {
+	struct strbuf buf = STRBUF_INIT;
+
 	git_config_with_options(cb, data, NULL, 1);
 
 	/*
@@ -1434,13 +1436,16 @@ void read_early_config(config_fn_t cb, void *data)
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->creating_repository && !have_git_dir()) {
+	if (!startup_info->creating_repository && !have_git_dir() &&
+	    discover_git_directory(&buf)) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
+		strbuf_addstr(&buf, "/config");
+		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, 1);
 	}
+	strbuf_release(&buf);
 }
 
 static void git_config_check_init(void);
diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index 427bfc605ad..bf89340988b 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -360,19 +360,19 @@ test_pager_choices                       'git aliasedlog'
 test_default_pager        expect_success 'git -p aliasedlog'
 test_PAGER_overrides      expect_success 'git -p aliasedlog'
 test_core_pager_overrides expect_success 'git -p aliasedlog'
-test_core_pager_subdir    expect_failure 'git -p aliasedlog'
+test_core_pager_subdir    expect_success 'git -p aliasedlog'
 test_GIT_PAGER_overrides  expect_success 'git -p aliasedlog'
 
 test_default_pager        expect_success 'git -p true'
 test_PAGER_overrides      expect_success 'git -p true'
 test_core_pager_overrides expect_success 'git -p true'
-test_core_pager_subdir    expect_failure 'git -p true'
+test_core_pager_subdir    expect_success 'git -p true'
 test_GIT_PAGER_overrides  expect_success 'git -p true'
 
 test_default_pager        expect_success test_must_fail 'git -p request-pull'
 test_PAGER_overrides      expect_success test_must_fail 'git -p request-pull'
 test_core_pager_overrides expect_success test_must_fail 'git -p request-pull'
-test_core_pager_subdir    expect_failure test_must_fail 'git -p request-pull'
+test_core_pager_subdir    expect_success test_must_fail 'git -p request-pull'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 
 test_default_pager        expect_success test_must_fail 'git -p'
@@ -380,7 +380,7 @@ test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
-test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+test_expect_success TTY 'core.pager in repo config works and retains cwd' '
 	sane_unset GIT_PAGER &&
 	test_config core.pager "cat >cwd-retained" &&
 	(
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 8/9] read_early_config(): really discover .git/

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:32AM +0100, Johannes Schindelin wrote:

> Earlier, we punted and simply assumed that we are in the top-level
> directory of the project, and that there is no .git file but a .git/
> directory so that we can read directly from .git/config.
> 
> However, that is not necessarily true. We may be in a subdirectory. Or
> .git may be a gitfile. Or the environment variable GIT_DIR may be set.
> 
> To remedy this situation, we just refactored the way
> setup_git_directory() discovers the .git/ directory, to make it
> reusable, and more importantly, to leave all global variables and the
> current working directory alone.
> 
> Let's discover the .git/ directory correctly in read_early_config() by
> using that new function.
> 
> This fixes 4 known breakages in t7006.

Yay, this is much nicer.

I think the "dirty hack..." comment in read_early_config() can be
condensed (or go away) now.

I think we _could_ do away with read_early_config() entirely, and just
have the regular config code do this lookup when we're not already in a
repo. But then we'd really need to depend on the "creating_repository"
flag being managed correctly.

I think I prefer the idea that a few "early" spots like pager and alias
config need to use this special function to access the config. That's
less likely to cause surprises when some config option is picked up
before we have run setup_git_directory().

There is one surprising case that I think we need to deal with even now,
though. If I do:

  git init repo
  git -C repo config pager.upload-pack 'echo whoops'
  git upload-pack repo
  cd repo && git upload-pack .

the first one is OK, but the second reads and executes the pager config
from the repo, even though we usually consider upload-pack to be OK to
run in an untrusted repo. This _isn't_ a new thing in your patch, but
just something I noticed while we are here.

And maybe it is a non-issue. The early-config bits all happen via the
git wrapper, and normally we use the direct dashed "git-upload-pack" to
access the other repositories. Certainly I have been known to use "git
-c ... upload-pack" while debugging various things.

So I dunno. You really have to jump through some hoops for it to matter,
but I just wonder if the git wrapper should be special-casing
upload-pack, too.

-Peff

Re: [PATCH v2 8/9] read_early_config(): really discover .git/

[Johannes Schindelin]

Hi Peff,

On Fri, 3 Mar 2017, Jeff King wrote:

> I think the "dirty hack..." comment in read_early_config() can be
> condensed (or go away) now.

Yes, I made that change in response to a comment you made about an earlier
patch in this series.

> I think we _could_ do away with read_early_config() entirely, and just
> have the regular config code do this lookup when we're not already in a
> repo. But then we'd really need to depend on the "creating_repository"
> flag being managed correctly.

Well, that would be a major design change. I'm not really all that
comfortable with that...

> I think I prefer the idea that a few "early" spots like pager and alias
> config need to use this special function to access the config. That's
> less likely to cause surprises when some config option is picked up
> before we have run setup_git_directory().

Exactly. There is semantic meaning in calling read_early_config() vs
git_config().

> There is one surprising case that I think we need to deal with even now,
> though. If I do:
> 
>   git init repo
>   git -C repo config pager.upload-pack 'echo whoops'
>   git upload-pack repo
>   cd repo && git upload-pack .
> 
> the first one is OK, but the second reads and executes the pager config
> from the repo, even though we usually consider upload-pack to be OK to
> run in an untrusted repo. This _isn't_ a new thing in your patch, but
> just something I noticed while we are here.
> 
> And maybe it is a non-issue. The early-config bits all happen via the
> git wrapper, and normally we use the direct dashed "git-upload-pack" to
> access the other repositories. Certainly I have been known to use "git
> -c ... upload-pack" while debugging various things.
> 
> So I dunno. You really have to jump through some hoops for it to matter,
> but I just wonder if the git wrapper should be special-casing
> upload-pack, too.

While I agree that it may sound surprising, I would like to point out that
there *is* a semantic difference between `git upload-pack repo` and `git
-C repo upload-pack .`: the working directory is different. And given that
so much in Git's operations depend on the working directory, it makes
sense that those two invocations have slightly different semantics, too.

I also agree, obviously, that this is something that would need a separate
patch series to tackle ;-)

Ciao,
Dscho

[PATCH v2 9/9] Test read_early_config()

[Johannes Schindelin]

So far, we had no explicit tests of that function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-config.c  | 15 +++++++++++++++
 t/t1309-early-config.sh | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 t/t1309-early-config.sh

diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 83a4f2ab869..8e3ed6a76cb 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -66,6 +66,16 @@ static int iterate_cb(const char *var, const char *value, void *data)
 	return 0;
 }
 
+static int early_config_cb(const char *var, const char *value, void *vdata)
+{
+	const char *key = vdata;
+
+	if (!strcmp(key, var))
+		printf("%s\n", value);
+
+	return 0;
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	int i, val;
@@ -73,6 +83,11 @@ int cmd_main(int argc, const char **argv)
 	const struct string_list *strptr;
 	struct config_set cs;
 
+	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		read_early_config(early_config_cb, (void *)argv[2]);
+		return 0;
+	}
+
 	setup_git_directory();
 
 	git_configset_init(&cs);
diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
new file mode 100755
index 00000000000..0c55dee514c
--- /dev/null
+++ b/t/t1309-early-config.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+test_description='Test read_early_config()'
+
+. ./test-lib.sh
+
+test_expect_success 'read early config' '
+	test_config early.config correct &&
+	test-config read_early_config early.config >output &&
+	test correct = "$(cat output)"
+'
+
+test_expect_success 'in a sub-directory' '
+	test_config early.config sub &&
+	mkdir -p sub &&
+	(
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test sub = "$(cat output)"
+'
+
+test_expect_success 'ceiling' '
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test -z "$(cat output)"
+'
+
+test_expect_success 'ceiling #2' '
+	mkdir -p xdg/git &&
+	git config -f xdg/git/config early.config xdg &&
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		XDG_CONFIG_HOME="$PWD"/xdg &&
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES XDG_CONFIG_HOME &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test xdg = "$(cat output)"
+'
+
+test_done
-- 
2.12.0.windows.1.3.g8a117c48243

Re: [PATCH v2 9/9] Test read_early_config()

[Jeff King]

On Fri, Mar 03, 2017 at 03:04:36AM +0100, Johannes Schindelin wrote:

> So far, we had no explicit tests of that function.

Makes sense. The pager tests fixed in an earlier commit were effectively
checking those, but I don't mind making it explicit.

-Peff

Re: [PATCH v2 9/9] Test read_early_config()

[Johannes Schindelin]

Hi Peff,

On Fri, 3 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:04:36AM +0100, Johannes Schindelin wrote:
> 
> > So far, we had no explicit tests of that function.
> 
> Makes sense. The pager tests fixed in an earlier commit were effectively
> checking those, but I don't mind making it explicit.

Well, TBH I am a bit uncomfortable with t7006 doing those tests. Just
imagine that something breaks in that script, say, when working on
exporting the read_early_config() function. You probably see this coming:
debugging those breakages is half a nightmare. There are multiple levels
of shell script functions, a Perl script, and an isatty() call between the
bug hunter and the bug.

With the new tests, it all becomes much more straight-forward to debug.
And also less surprising (think about the fun you can have with test cases
that fail when running `make t7006-pager.sh` but not when running `bash
t7006-pager.sh -i -v -x`, just because of a forgotten `test_terminal`...).

And yes, you can debug t7006 with the good old "insert debug print
statements here and there, then compile and run, rinse & repeat" method.
But you know, getting to use a real IDE with real debugger/intellisense
integration after years of working on C code in the terminal with vim and
gdb [*1*], I kinda feel a bit pampered and ask myself how I could possibly
have put up with the awkwardness. ;-)

Ciao,
Dscho

Footnote *1*: for some years I developed Java code in Eclipse and already
then did I notice just how much faster development becomes when you have
powerful tools to help you...

Re: [PATCH v2 0/9] Fix the early config

[Jeff King]

On Fri, Mar 03, 2017 at 03:03:56AM +0100, Johannes Schindelin wrote:

> These patches are an attempt to make Git's startup sequence a bit less
> surprising.
> 
> The idea here is to discover the .git/ directory gently (i.e. without
> changing the current working directory, or global variables), and to use
> it to read the .git/config file early, before we actually called
> setup_git_directory() (if we ever do that).

Thanks for working on this. I think it's a huge improvement over the
status quo, and over the earlier attempt. I don't see anything hugely
wrong with this series, though I did note one bug, along with some minor
refinements.

> My dirty little secret is that I actually need this for something else
> entirely. I need to patch an internal version of Git to gather
> statistics, and to that end I need to read the config before and after
> running every Git command. Hence the need for a gentle, and correct
> early config.

We do something similar at GitHub, but it falls into two categories:

  - stat-gathering that's on all the time, so doesn't need to look at
    config (I'm not sure in your case if you want to trigger the
    gathering with config, or if config is just one of the things you
    are gathering).

  - logging that is turned on selectively for some repos, but which
    doesn't have to be looked up until we know we are in a repo

I looked into making something upstream-able, and my approach was to
allow GIT_TRACE_* variables to be specified in the config. But of course
that ran into the early-config problem (and I really wanted repo-level
config there, because I'd like to be able to turn on tracing for just a
single problematic repo).

So not really something you need to work on, but maybe food for thought
as you work on your internal project.

-Peff

Re: [PATCH v2 0/9] Fix the early config

[Johannes Schindelin]

Hi Peff,

On Fri, 3 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 03:03:56AM +0100, Johannes Schindelin wrote:
> 
> > These patches are an attempt to make Git's startup sequence a bit less
> > surprising.
> > 
> > The idea here is to discover the .git/ directory gently (i.e. without
> > changing the current working directory, or global variables), and to
> > use it to read the .git/config file early, before we actually called
> > setup_git_directory() (if we ever do that).
> 
> Thanks for working on this. I think it's a huge improvement over the
> status quo, and over the earlier attempt. I don't see anything hugely
> wrong with this series, though I did note one bug, along with some minor
> refinements.

Thank you very much for your review. You did point out a couple of things
I overlooked, and while it was a pain in the back to go from v1 to v2
(i.e. avoiding code duplication, which immediately put the changed code to
a huge test by having the very central setup_git_directory() use it), I
really believe that we are better off because the patch series pays off
more technical debt now than it introduces.

> > My dirty little secret is that I actually need this for something else
> > entirely. I need to patch an internal version of Git to gather
> > statistics, and to that end I need to read the config before and after
> > running every Git command. Hence the need for a gentle, and correct
> > early config.
> 
> We do something similar at GitHub, but it falls into two categories:
> 
>   - stat-gathering that's on all the time, so doesn't need to look at
>     config (I'm not sure in your case if you want to trigger the
>     gathering with config, or if config is just one of the things you
>     are gathering).
> 
>   - logging that is turned on selectively for some repos, but which
>     doesn't have to be looked up until we know we are in a repo

You probably guessed what I need to do, anyway: for our GVFS usage, we
need some telemetry (i.e. usage statistics), and I basically want to run a
hook whenever any Git command is called, but only on GVFS-enabled
repositories.

So it is somewhat related, but slightly different from your usage.

> I looked into making something upstream-able, and my approach was to
> allow GIT_TRACE_* variables to be specified in the config. But of course
> that ran into the early-config problem (and I really wanted repo-level
> config there, because I'd like to be able to turn on tracing for just a
> single problematic repo).
> 
> So not really something you need to work on, but maybe food for thought
> as you work on your internal project.

Right, GIT_TRACE_* settings in the config should be doable now.

Ciao,
Dscho

[PATCH v3 0/9] Fix the early config

[Johannes Schindelin]

These patches are an attempt to make Git's startup sequence a bit less
surprising.

The idea here is to discover the .git/ directory gently (i.e. without
changing the current working directory, or global variables), and to use
it to read the .git/config file early, before we actually called
setup_git_directory() (if we ever do that).

This also allows us to fix the early config e.g. to determine the pager
or to resolve aliases in a non-surprising manner.

My dirty little secret is that I need to discover the Git directory
early, without changing global state, for usage statistics gathering in
the GVFS Git project, so I actually do not care all that much about the
early config, although it is a welcome fallout (and a good reason for
accepting these patches and thereby releasing me of more maintenance
burden :-)).

Notable notes:

- In contrast to earlier versions, I no longer special-case init and
  clone. Peff pointed out that this adds technical debt, and that we can
  actually argue (for consistency's sake) that early config reads the
  current repository config (if any) even for init and clone.

- The read_early_config() function does not cache Git directory
  discovery nor read values. If needed, this can be implemented later,
  in a separate patch series.

- The alias handling in git.c could possibly benefit from this work, but
  again, this is a separate topic from the current patch series.

Changes since v2:

- replaced `test -e` by `test_path_is_file`

- fixed premature "cwd -> dir" in 2/9

- the setup_git_directory_gently_1() function is no longer renamed
  because it is not exported directly, anyway

- fixed the way setup_discovered_git_dir() expected the offset parameter
  to exclude the trailing slash (which is not true for root
  directories); also verified that setup_bare_git_dir() does not require
  a corresponding patch

- switched to using size_t instead of int to save the length of the
  strbuf in discover_git_directory()

- ensured that discover_git_directory() turns a relative gitdir into an
  absolute one even if there is already some text in the strbuf

- clarified under which circumstances we turn a relative gitdir into an
  absolute one

- avoided absolute gitdir with trailing "/." to be returned

- the commit that fixes the "really dirty hack" now rewords that comment
  to reflect that it is no longer a really dirty hack

- dropped the special-casing of init and clone

- the discover_git_directory() function now correctly checks the
  repository version, warning (and returning NULL) in case of a problem


Johannes Schindelin (9):
  t7006: replace dubious test
  setup_git_directory(): use is_dir_sep() helper
  Prepare setup_discovered_git_directory() the root directory
  setup_git_directory_1(): avoid changing global state
  Export the discover_git_directory() function
  Make read_early_config() reusable
  read_early_config(): avoid .git/config hack when unneeded
  read_early_config(): really discover .git/
  Test read_early_config()

 cache.h                 |   3 +
 config.c                |  27 ++++++
 pager.c                 |  31 -------
 setup.c                 | 237 ++++++++++++++++++++++++++++++++----------------
 t/helper/test-config.c  |  15 +++
 t/t1309-early-config.sh |  50 ++++++++++
 t/t7006-pager.sh        |  18 +++-
 7 files changed, 269 insertions(+), 112 deletions(-)
 create mode 100755 t/t1309-early-config.sh


base-commit: 3bc53220cb2dcf709f7a027a3f526befd021d858
Published-As: https://github.com/dscho/git/releases/tag/early-config-v3
Fetch-It-Via: git fetch https://github.com/dscho/git early-config-v3

Interdiff vs v2:

 diff --git a/cache.h b/cache.h
 index 0af7141242f..8a4580f921d 100644
 --- a/cache.h
 +++ b/cache.h
 @@ -518,6 +518,7 @@ extern void set_git_work_tree(const char *tree);
  #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
  
  extern void setup_work_tree(void);
 +/* Find GIT_DIR without changing the working directory or other global state */
  extern const char *discover_git_directory(struct strbuf *gitdir);
  extern const char *setup_git_directory_gently(int *);
  extern const char *setup_git_directory(void);
 @@ -2070,7 +2071,7 @@ const char *split_cmdline_strerror(int cmdline_errno);
  
  /* setup.c */
  struct startup_info {
 -	int have_repository, creating_repository;
 +	int have_repository;
  	const char *prefix;
  };
  extern struct startup_info *startup_info;
 diff --git a/config.c b/config.c
 index bcda397d42e..749623a9649 100644
 --- a/config.c
 +++ b/config.c
 @@ -1419,25 +1419,16 @@ void read_early_config(config_fn_t cb, void *data)
  	git_config_with_options(cb, data, NULL, 1);
  
  	/*
 -	 * Note that this is a really dirty hack that does the wrong thing in
 -	 * many cases. The crux of the problem is that we cannot run
 -	 * setup_git_directory() early on in git's setup, so we have no idea if
 -	 * we are in a repository or not, and therefore are not sure whether
 -	 * and how to read repository-local config.
 -	 *
 -	 * So if we _aren't_ in a repository (or we are but we would reject its
 -	 * core.repositoryformatversion), we'll read whatever is in .git/config
 -	 * blindly. Similarly, if we _are_ in a repository, but not at the
 -	 * root, we'll fail to find .git/config (because it's really
 -	 * ../.git/config, etc). See t7006 for a complete set of failures.
 -	 *
 -	 * However, we have historically provided this hack because it does
 -	 * work some of the time (namely when you are at the top-level of a
 -	 * valid repository), and would rarely make things worse (i.e., you do
 -	 * not generally have a .git/config file sitting around).
 +	 * When we are not about to create a repository ourselves (init or
 +	 * clone) and when no .git/ directory was set up yet (in which case
 +	 * git_config_with_options() would already have picked up the
 +	 * repository config), we ask discover_git_directory() to figure out
 +	 * whether there is any repository config we should use (but unlike
 +	 * setup_git_directory_gently(), no global state is changed, most
 +	 * notably, the current working directory is still the same after
 +	 * the call).
  	 */
 -	if (!startup_info->creating_repository && !have_git_dir() &&
 -	    discover_git_directory(&buf)) {
 +	if (!have_git_dir() && discover_git_directory(&buf)) {
  		struct git_config_source repo_config;
  
  		memset(&repo_config, 0, sizeof(repo_config));
 diff --git a/git.c b/git.c
 index 9fb9bb90a21..33f52acbcc8 100644
 --- a/git.c
 +++ b/git.c
 @@ -337,9 +337,6 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
  	struct stat st;
  	const char *prefix;
  
 -	if (p->fn == cmd_init_db || p->fn == cmd_clone)
 -		startup_info->creating_repository = 1;
 -
  	prefix = NULL;
  	help = argc == 2 && !strcmp(argv[1], "-h");
  	if (!help) {
 diff --git a/setup.c b/setup.c
 index 7ceca6cc6ef..5320ae37314 100644
 --- a/setup.c
 +++ b/setup.c
 @@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
  	if (offset == cwd->len)
  		return NULL;
  
 -	/* Make "offset" point to past the '/', and add a '/' at the end */
 -	offset++;
 +	/* Make "offset" point past the '/' (already the case for root dirs) */
 +	if (offset != offset_1st_component(cwd->buf))
 +		offset++;
 +	/* Add a '/' at the end */
  	strbuf_addch(cwd, '/');
  	return cwd->buf + offset;
  }
 @@ -839,8 +841,8 @@ enum discovery_result {
   * the discovered .git/ directory, if any. This path may be relative against
   * `dir` (i.e. *not* necessarily the cwd).
   */
 -static enum discovery_result discover_git_directory_1(struct strbuf *dir,
 -						      struct strbuf *gitdir)
 +static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 +							  struct strbuf *gitdir)
  {
  	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
  	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
 @@ -923,24 +925,44 @@ static enum discovery_result discover_git_directory_1(struct strbuf *dir,
  
  const char *discover_git_directory(struct strbuf *gitdir)
  {
 -	struct strbuf dir = STRBUF_INIT;
 -	int len;
 +	struct strbuf dir = STRBUF_INIT, err = STRBUF_INIT;
 +	size_t gitdir_offset = gitdir->len, cwd_len;
 +	struct repository_format candidate;
  
  	if (strbuf_getcwd(&dir))
  		return NULL;
  
 -	len = dir.len;
 -	if (discover_git_directory_1(&dir, gitdir) < 0) {
 +	cwd_len = dir.len;
 +	if (setup_git_directory_gently_1(&dir, gitdir) < 0) {
  		strbuf_release(&dir);
  		return NULL;
  	}
  
 -	if (dir.len < len && !is_absolute_path(gitdir->buf)) {
 -		strbuf_addch(&dir, '/');
 -		strbuf_insert(gitdir, 0, dir.buf, dir.len);
 +	/*
 +	 * The returned gitdir is relative to dir, and if dir does not reflect
 +	 * the current working directory, we simply make the gitdir absolute.
 +	 */
 +	if (dir.len < cwd_len && !is_absolute_path(gitdir->buf + gitdir_offset)) {
 +		/* Avoid a trailing "/." */
 +		if (!strcmp(".", gitdir->buf + gitdir_offset))
 +			strbuf_setlen(gitdir, gitdir_offset);
 +		else
 +			strbuf_addch(&dir, '/');
 +		strbuf_insert(gitdir, gitdir_offset, dir.buf, dir.len);
  	}
 +
 +	strbuf_reset(&dir);
 +	strbuf_addf(&dir, "%s/config", gitdir->buf + gitdir_offset);
 +	read_repository_format(&candidate, dir.buf);
  	strbuf_release(&dir);
  
 +	if (verify_repository_format(&candidate, &err) < 0) {
 +		warning("ignoring git dir '%s': %s",
 +			gitdir->buf + gitdir_offset, err.buf);
 +		strbuf_release(&err);
 +		return NULL;
 +	}
 +
  	return gitdir->buf;
  }
  
 @@ -970,7 +992,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
  		die_errno(_("Unable to read current working directory"));
  	strbuf_addbuf(&dir, &cwd);
  
 -	switch (discover_git_directory_1(&dir, &gitdir)) {
 +	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
  	case GIT_DIR_NONE:
  		prefix = NULL;
  		break;
 @@ -1000,7 +1022,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
  		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
  		    dir.buf);
  	default:
 -		die("BUG: unhandled discover_git_directory() result");
 +		die("BUG: unhandled setup_git_directory_1() result");
  	}
  
  	if (prefix)
 diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
 index bf89340988b..4f3794d415e 100755
 --- a/t/t7006-pager.sh
 +++ b/t/t7006-pager.sh
 @@ -387,7 +387,7 @@ test_expect_success TTY 'core.pager in repo config works and retains cwd' '
  		cd sub &&
  		rm -f cwd-retained &&
  		test_terminal git -p rev-parse HEAD &&
 -		test -e cwd-retained
 +		test_path_is_file cwd-retained
  	)
  '
  

-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 1/9] t7006: replace dubious test

[Johannes Schindelin]

The idea of the test case "git -p - core.pager is not used from
subdirectory" was to verify that the setup_git_directory() function had
not been called just to obtain the core.pager setting.

However, we are about to fix the early config machinery so that it
*does* work, without messing up the global state.

Once that is done, the core.pager setting *will* be used, even when
running from a subdirectory, and that is a Good Thing.

The intention of that test case, however, was to verify that the
setup_git_directory() function has not run, because it changes global
state such as the current working directory.

To keep that spirit, but fix the incorrect assumption, this patch
replaces that test case by a new one that verifies that the pager is
run in the subdirectory, i.e. that the current working directory has
not been changed at the time the pager is configured and launched, even
if the `rev-parse` command requires a .git/ directory and *will* change
the working directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t7006-pager.sh | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index c8dc665f2fd..304ae06c600 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -378,9 +378,19 @@ test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 test_default_pager        expect_success test_must_fail 'git -p'
 test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
-test_no_local_config_subdir expect_success test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
+test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+	sane_unset GIT_PAGER &&
+	test_config core.pager "cat >cwd-retained" &&
+	(
+		cd sub &&
+		rm -f cwd-retained &&
+		test_terminal git -p rev-parse HEAD &&
+		test_path_is_file cwd-retained
+	)
+'
+
 test_doesnt_paginate      expect_failure test_must_fail 'git -p nonsense'
 
 test_pager_choices                       'git shortlog'
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 2/9] setup_git_directory(): use is_dir_sep() helper

[Johannes Schindelin]

It is okay in practice to test for forward slashes in the output of
getcwd(), because we go out of our way to convert backslashes to forward
slashes in getcwd()'s output on Windows.

Still, the correct way to test for a dir separator is by using the
helper function we introduced for that very purpose. It also serves as a
good documentation what the code tries to do (not "how").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index 967f289f1ef..c47619605fb 100644
--- a/setup.c
+++ b/setup.c
@@ -910,7 +910,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 			return setup_bare_git_dir(&cwd, offset, nongit_ok);
 
 		offset_parent = offset;
-		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
+		while (--offset_parent > ceil_offset &&
+		       !is_dir_sep(cwd.buf[offset_parent]));
 		if (offset_parent <= ceil_offset)
 			return setup_nongit(cwd.buf, nongit_ok);
 		if (one_filesystem) {
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 3/9] Prepare setup_discovered_git_directory() the root directory

[Johannes Schindelin]

Currently, the offset parameter (indicating what part of the cwd
parameter corresponds to the current directory after discovering the
.git/ directory) is set to 0 when we are running in the root directory.

However, in the next patches we will avoid changing the current working
directory while searching for the .git/ directory, meaning that the
offset corresponding to the root directory will have to be 1 to reflect
that this directory is characterized by the path "/" (and not "").

So let's make sure that setup_discovered_git_directory() only tries to
append the trailing slash to non-root directories.

Note: the setup_bare_git_directory() does not need a corresponding
change, as it does not want to return a prefix.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/setup.c b/setup.c
index c47619605fb..91d884b6746 100644
--- a/setup.c
+++ b/setup.c
@@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 	if (offset == cwd->len)
 		return NULL;
 
-	/* Make "offset" point to past the '/', and add a '/' at the end */
-	offset++;
+	/* Make "offset" point past the '/' (already the case for root dirs) */
+	if (offset != offset_1st_component(cwd->buf))
+		offset++;
+	/* Add a '/' at the end */
 	strbuf_addch(cwd, '/');
 	return cwd->buf + offset;
 }
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 4/9] setup_git_directory_1(): avoid changing global state

[Johannes Schindelin]

For historical reasons, Git searches for the .git/ directory (or the
.git file) by changing the working directory successively to the parent
directory of the current directory, until either anything was found or
until a ceiling or a mount point is hit.

Further global state may be changed in case a .git/ directory was found.

We do have a use case, though, where we would like to find the .git/
directory without having any global state touched, though: when we read
the early config e.g. for the pager or for alias expansion.

Let's just move all of code that changes any global state out of the
function `setup_git_directory_gently_1()` into
`setup_git_directory_gently()`.

In subsequent patches, we will use the _1() function in a new
`discover_git_directory()` function that we will then use for the early
config code.

Note: the new loop is a *little* tricky, as we have to handle the root
directory specially: we cannot simply strip away the last component
including the slash, as the root directory only has that slash. To remedy
that, we introduce the `min_offset` variable that holds the minimal length
of an absolute path, and using that to special-case the root directory,
including an early exit before trying to find the parent of the root
directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 189 ++++++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 113 insertions(+), 76 deletions(-)

diff --git a/setup.c b/setup.c
index 91d884b6746..9a09bb41ab5 100644
--- a/setup.c
+++ b/setup.c
@@ -818,50 +818,49 @@ static int canonicalize_ceiling_entry(struct string_list_item *item,
 	}
 }
 
+enum discovery_result {
+	GIT_DIR_NONE = 0,
+	GIT_DIR_EXPLICIT,
+	GIT_DIR_DISCOVERED,
+	GIT_DIR_BARE,
+	/* these are errors */
+	GIT_DIR_HIT_CEILING = -1,
+	GIT_DIR_HIT_MOUNT_POINT = -2
+};
+
 /*
  * We cannot decide in this function whether we are in the work tree or
  * not, since the config can only be read _after_ this function was called.
+ *
+ * Also, we avoid changing any global state (such as the current working
+ * directory) to allow early callers.
+ *
+ * The directory where the search should start needs to be passed in via the
+ * `dir` parameter; upon return, the `dir` buffer will contain the path of
+ * the directory where the search ended, and `gitdir` will contain the path of
+ * the discovered .git/ directory, if any. This path may be relative against
+ * `dir` (i.e. *not* necessarily the cwd).
  */
-static const char *setup_git_directory_gently_1(int *nongit_ok)
+static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
+							  struct strbuf *gitdir)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
-	static struct strbuf cwd = STRBUF_INIT;
-	const char *gitdirenv, *ret;
-	char *gitfile;
-	int offset, offset_parent, ceil_offset = -1;
+	const char *gitdirenv;
+	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
 	dev_t current_device = 0;
 	int one_filesystem = 1;
 
 	/*
-	 * We may have read an incomplete configuration before
-	 * setting-up the git directory. If so, clear the cache so
-	 * that the next queries to the configuration reload complete
-	 * configuration (including the per-repo config file that we
-	 * ignored previously).
-	 */
-	git_config_clear();
-
-	/*
-	 * Let's assume that we are in a git repository.
-	 * If it turns out later that we are somewhere else, the value will be
-	 * updated accordingly.
-	 */
-	if (nongit_ok)
-		*nongit_ok = 0;
-
-	if (strbuf_getcwd(&cwd))
-		die_errno(_("Unable to read current working directory"));
-	offset = cwd.len;
-
-	/*
 	 * If GIT_DIR is set explicitly, we're not going
 	 * to do any discovery, but we still do repository
 	 * validation.
 	 */
 	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
-	if (gitdirenv)
-		return setup_explicit_git_dir(gitdirenv, &cwd, nongit_ok);
+	if (gitdirenv) {
+		strbuf_addstr(gitdir, gitdirenv);
+		return GIT_DIR_EXPLICIT;
+	}
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -869,15 +868,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP, -1);
 		filter_string_list(&ceiling_dirs, 0,
 				   canonicalize_ceiling_entry, &empty_entry_found);
-		ceil_offset = longest_ancestor_length(cwd.buf, &ceiling_dirs);
+		ceil_offset = longest_ancestor_length(dir->buf, &ceiling_dirs);
 		string_list_clear(&ceiling_dirs, 0);
 	}
 
-	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
-		ceil_offset = 1;
+	if (ceil_offset < 0)
+		ceil_offset = min_offset - 2;
 
 	/*
-	 * Test in the following order (relative to the cwd):
+	 * Test in the following order (relative to the dir):
 	 * - .git (file containing "gitdir: <path>")
 	 * - .git/
 	 * - ./ (bare)
@@ -889,62 +888,100 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	 */
 	one_filesystem = !git_env_bool("GIT_DISCOVERY_ACROSS_FILESYSTEM", 0);
 	if (one_filesystem)
-		current_device = get_device_or_die(".", NULL, 0);
+		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
-		if (gitfile)
-			gitdirenv = gitfile = xstrdup(gitfile);
-		else {
-			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
-				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
-		}
-
+		int offset = dir->len;
+
+		if (offset > min_offset)
+			strbuf_addch(dir, '/');
+		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
+		gitdirenv = read_gitfile(dir->buf);
+		if (!gitdirenv && is_git_directory(dir->buf))
+			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		strbuf_setlen(dir, offset);
 		if (gitdirenv) {
-			ret = setup_discovered_git_dir(gitdirenv,
-						       &cwd, offset,
-						       nongit_ok);
-			free(gitfile);
-			return ret;
+			strbuf_addstr(gitdir, gitdirenv);
+			return GIT_DIR_DISCOVERED;
 		}
-		free(gitfile);
 
-		if (is_git_directory("."))
-			return setup_bare_git_dir(&cwd, offset, nongit_ok);
-
-		offset_parent = offset;
-		while (--offset_parent > ceil_offset &&
-		       !is_dir_sep(cwd.buf[offset_parent]));
-		if (offset_parent <= ceil_offset)
-			return setup_nongit(cwd.buf, nongit_ok);
-		if (one_filesystem) {
-			dev_t parent_device = get_device_or_die("..", cwd.buf,
-								offset);
-			if (parent_device != current_device) {
-				if (nongit_ok) {
-					if (chdir(cwd.buf))
-						die_errno(_("Cannot come back to cwd"));
-					*nongit_ok = 1;
-					return NULL;
-				}
-				strbuf_setlen(&cwd, offset);
-				die(_("Not a git repository (or any parent up to mount point %s)\n"
-				"Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
-				    cwd.buf);
-			}
-		}
-		if (chdir("..")) {
-			strbuf_setlen(&cwd, offset);
-			die_errno(_("Cannot change to '%s/..'"), cwd.buf);
+		if (is_git_directory(dir->buf)) {
+			strbuf_addstr(gitdir, ".");
+			return GIT_DIR_BARE;
 		}
-		offset = offset_parent;
+
+		if (offset <= min_offset)
+			return GIT_DIR_HIT_CEILING;
+
+		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]));
+		if (offset <= ceil_offset)
+			return GIT_DIR_HIT_CEILING;
+
+		strbuf_setlen(dir, offset > min_offset ?  offset : min_offset);
+		if (one_filesystem &&
+		    current_device != get_device_or_die(dir->buf, NULL, offset))
+			return GIT_DIR_HIT_MOUNT_POINT;
 	}
 }
 
 const char *setup_git_directory_gently(int *nongit_ok)
 {
+	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
 	const char *prefix;
 
-	prefix = setup_git_directory_gently_1(nongit_ok);
+	/*
+	 * We may have read an incomplete configuration before
+	 * setting-up the git directory. If so, clear the cache so
+	 * that the next queries to the configuration reload complete
+	 * configuration (including the per-repo config file that we
+	 * ignored previously).
+	 */
+	git_config_clear();
+
+	/*
+	 * Let's assume that we are in a git repository.
+	 * If it turns out later that we are somewhere else, the value will be
+	 * updated accordingly.
+	 */
+	if (nongit_ok)
+		*nongit_ok = 0;
+
+	if (strbuf_getcwd(&cwd))
+		die_errno(_("Unable to read current working directory"));
+	strbuf_addbuf(&dir, &cwd);
+
+	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
+	case GIT_DIR_NONE:
+		prefix = NULL;
+		break;
+	case GIT_DIR_EXPLICIT:
+		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
+		break;
+	case GIT_DIR_DISCOVERED:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
+						  nongit_ok);
+		break;
+	case GIT_DIR_BARE:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
+		break;
+	case GIT_DIR_HIT_CEILING:
+		prefix = setup_nongit(cwd.buf, nongit_ok);
+		break;
+	case GIT_DIR_HIT_MOUNT_POINT:
+		if (nongit_ok) {
+			*nongit_ok = 1;
+			return NULL;
+		}
+		die(_("Not a git repository (or any parent up to mount point %s)\n"
+		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
+		    dir.buf);
+	default:
+		die("BUG: unhandled setup_git_directory_1() result");
+	}
+
 	if (prefix)
 		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
 	else
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 5/9] Export the discover_git_directory() function

[Johannes Schindelin]

The setup_git_directory_gently_1() function we modified earlier still
needs to return both the path to the .git/ directory as well as the
"cd-up" path to allow setup_git_directory() to retain its previous
behavior as if it changed the current working directory on its quest for
the .git/ directory.

This is a bit cumbersome to use if you only need to figure out the
(possibly absolute) path of the .git/ directory. Let's just provide
a convenient wrapper function with an easier signature that *just*
discovers the .git/ directory.

We will use it in a subsequent patch to support early config reading
better.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h |  2 ++
 setup.c | 31 +++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/cache.h b/cache.h
index 80b6372cf76..4d8eb38de74 100644
--- a/cache.h
+++ b/cache.h
@@ -518,6 +518,8 @@ extern void set_git_work_tree(const char *tree);
 #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
 
 extern void setup_work_tree(void);
+/* Find GIT_DIR without changing the working directory or other global state */
+extern const char *discover_git_directory(struct strbuf *gitdir);
 extern const char *setup_git_directory_gently(int *);
 extern const char *setup_git_directory(void);
 extern char *prefix_path(const char *prefix, int len, const char *path);
diff --git a/setup.c b/setup.c
index 9a09bb41ab5..32ce023638a 100644
--- a/setup.c
+++ b/setup.c
@@ -923,6 +923,37 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 	}
 }
 
+const char *discover_git_directory(struct strbuf *gitdir)
+{
+	struct strbuf dir = STRBUF_INIT;
+	size_t gitdir_offset = gitdir->len, cwd_len;
+
+	if (strbuf_getcwd(&dir))
+		return NULL;
+
+	cwd_len = dir.len;
+	if (setup_git_directory_gently_1(&dir, gitdir) < 0) {
+		strbuf_release(&dir);
+		return NULL;
+	}
+
+	/*
+	 * The returned gitdir is relative to dir, and if dir does not reflect
+	 * the current working directory, we simply make the gitdir absolute.
+	 */
+	if (dir.len < cwd_len && !is_absolute_path(gitdir->buf + gitdir_offset)) {
+		/* Avoid a trailing "/." */
+		if (!strcmp(".", gitdir->buf + gitdir_offset))
+			strbuf_setlen(gitdir, gitdir_offset);
+		else
+			strbuf_addch(&dir, '/');
+		strbuf_insert(gitdir, gitdir_offset, dir.buf, dir.len);
+	}
+	strbuf_release(&dir);
+
+	return gitdir->buf;
+}
+
 const char *setup_git_directory_gently(int *nongit_ok)
 {
 	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 6/9] Make read_early_config() reusable

[Johannes Schindelin]

The pager configuration needs to be read early, possibly before
discovering any .git/ directory.

Let's not hide this function in pager.c, but make it available to other
callers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h  |  1 +
 config.c | 31 +++++++++++++++++++++++++++++++
 pager.c  | 31 -------------------------------
 3 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/cache.h b/cache.h
index 4d8eb38de74..8a4580f921d 100644
--- a/cache.h
+++ b/cache.h
@@ -1799,6 +1799,7 @@ extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
 				     const unsigned char *sha1, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
diff --git a/config.c b/config.c
index c6b874a7bf7..9cfbeafd04c 100644
--- a/config.c
+++ b/config.c
@@ -1412,6 +1412,37 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
+void read_early_config(config_fn_t cb, void *data)
+{
+	git_config_with_options(cb, data, NULL, 1);
+
+	/*
+	 * Note that this is a really dirty hack that does the wrong thing in
+	 * many cases. The crux of the problem is that we cannot run
+	 * setup_git_directory() early on in git's setup, so we have no idea if
+	 * we are in a repository or not, and therefore are not sure whether
+	 * and how to read repository-local config.
+	 *
+	 * So if we _aren't_ in a repository (or we are but we would reject its
+	 * core.repositoryformatversion), we'll read whatever is in .git/config
+	 * blindly. Similarly, if we _are_ in a repository, but not at the
+	 * root, we'll fail to find .git/config (because it's really
+	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 *
+	 * However, we have historically provided this hack because it does
+	 * work some of the time (namely when you are at the top-level of a
+	 * valid repository), and would rarely make things worse (i.e., you do
+	 * not generally have a .git/config file sitting around).
+	 */
+	if (!startup_info->have_repository) {
+		struct git_config_source repo_config;
+
+		memset(&repo_config, 0, sizeof(repo_config));
+		repo_config.file = ".git/config";
+		git_config_with_options(cb, data, &repo_config, 1);
+	}
+}
+
 static void git_config_check_init(void);
 
 void git_config(config_fn_t fn, void *data)
diff --git a/pager.c b/pager.c
index ae796433630..73ca8bc3b17 100644
--- a/pager.c
+++ b/pager.c
@@ -43,37 +43,6 @@ static int core_pager_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
-static void read_early_config(config_fn_t cb, void *data)
-{
-	git_config_with_options(cb, data, NULL, 1);
-
-	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
-	 */
-	if (!startup_info->have_repository) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
-		git_config_with_options(cb, data, &repo_config, 1);
-	}
-}
-
 const char *git_pager(int stdout_is_tty)
 {
 	const char *pager;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 7/9] read_early_config(): avoid .git/config hack when unneeded

[Johannes Schindelin]

So far, we only look whether the startup_info claims to have seen a
git_dir.

However, do_git_config_sequence() (and consequently the
git_config_with_options() call used by read_early_config() asks the
have_git_dir() function whether we have a .git/ directory, which in turn
also looks at git_dir and at the environment variable GIT_DIR. And when
this is the case, the repository config is handled already, so we do not
have to do that again explicitly.

Let's just use the same function, have_git_dir(), to determine whether we
have to handle .git/config explicitly.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.c b/config.c
index 9cfbeafd04c..068fa4dcfa6 100644
--- a/config.c
+++ b/config.c
@@ -1427,14 +1427,15 @@ void read_early_config(config_fn_t cb, void *data)
 	 * core.repositoryformatversion), we'll read whatever is in .git/config
 	 * blindly. Similarly, if we _are_ in a repository, but not at the
 	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 * ../.git/config, etc), unless setup_git_directory() was already called.
+	 * See t7006 for a complete set of failures.
 	 *
 	 * However, we have historically provided this hack because it does
 	 * work some of the time (namely when you are at the top-level of a
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->have_repository) {
+	if (!have_git_dir()) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 8/9] read_early_config(): really discover .git/

[Johannes Schindelin]

Earlier, we punted and simply assumed that we are in the top-level
directory of the project, and that there is no .git file but a .git/
directory so that we can read directly from .git/config.

However, that is not necessarily true. We may be in a subdirectory. Or
.git may be a gitfile. Or the environment variable GIT_DIR may be set.

To remedy this situation, we just refactored the way
setup_git_directory() discovers the .git/ directory, to make it
reusable, and more importantly, to leave all global variables and the
current working directory alone.

Let's discover the .git/ directory correctly in read_early_config() by
using that new function.

This fixes 4 known breakages in t7006.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c         | 33 ++++++++++++++-------------------
 setup.c          | 14 +++++++++++++-
 t/t7006-pager.sh |  8 ++++----
 3 files changed, 31 insertions(+), 24 deletions(-)

diff --git a/config.c b/config.c
index 068fa4dcfa6..749623a9649 100644
--- a/config.c
+++ b/config.c
@@ -1414,34 +1414,29 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 
 void read_early_config(config_fn_t cb, void *data)
 {
+	struct strbuf buf = STRBUF_INIT;
+
 	git_config_with_options(cb, data, NULL, 1);
 
 	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc), unless setup_git_directory() was already called.
-	 * See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
+	 * When we are not about to create a repository ourselves (init or
+	 * clone) and when no .git/ directory was set up yet (in which case
+	 * git_config_with_options() would already have picked up the
+	 * repository config), we ask discover_git_directory() to figure out
+	 * whether there is any repository config we should use (but unlike
+	 * setup_git_directory_gently(), no global state is changed, most
+	 * notably, the current working directory is still the same after
+	 * the call).
 	 */
-	if (!have_git_dir()) {
+	if (!have_git_dir() && discover_git_directory(&buf)) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
+		strbuf_addstr(&buf, "/config");
+		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, 1);
 	}
+	strbuf_release(&buf);
 }
 
 static void git_config_check_init(void);
diff --git a/setup.c b/setup.c
index 32ce023638a..5320ae37314 100644
--- a/setup.c
+++ b/setup.c
@@ -925,8 +925,9 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 
 const char *discover_git_directory(struct strbuf *gitdir)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT, err = STRBUF_INIT;
 	size_t gitdir_offset = gitdir->len, cwd_len;
+	struct repository_format candidate;
 
 	if (strbuf_getcwd(&dir))
 		return NULL;
@@ -949,8 +950,19 @@ const char *discover_git_directory(struct strbuf *gitdir)
 			strbuf_addch(&dir, '/');
 		strbuf_insert(gitdir, gitdir_offset, dir.buf, dir.len);
 	}
+
+	strbuf_reset(&dir);
+	strbuf_addf(&dir, "%s/config", gitdir->buf + gitdir_offset);
+	read_repository_format(&candidate, dir.buf);
 	strbuf_release(&dir);
 
+	if (verify_repository_format(&candidate, &err) < 0) {
+		warning("ignoring git dir '%s': %s",
+			gitdir->buf + gitdir_offset, err.buf);
+		strbuf_release(&err);
+		return NULL;
+	}
+
 	return gitdir->buf;
 }
 
diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index 304ae06c600..4f3794d415e 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -360,19 +360,19 @@ test_pager_choices                       'git aliasedlog'
 test_default_pager        expect_success 'git -p aliasedlog'
 test_PAGER_overrides      expect_success 'git -p aliasedlog'
 test_core_pager_overrides expect_success 'git -p aliasedlog'
-test_core_pager_subdir    expect_failure 'git -p aliasedlog'
+test_core_pager_subdir    expect_success 'git -p aliasedlog'
 test_GIT_PAGER_overrides  expect_success 'git -p aliasedlog'
 
 test_default_pager        expect_success 'git -p true'
 test_PAGER_overrides      expect_success 'git -p true'
 test_core_pager_overrides expect_success 'git -p true'
-test_core_pager_subdir    expect_failure 'git -p true'
+test_core_pager_subdir    expect_success 'git -p true'
 test_GIT_PAGER_overrides  expect_success 'git -p true'
 
 test_default_pager        expect_success test_must_fail 'git -p request-pull'
 test_PAGER_overrides      expect_success test_must_fail 'git -p request-pull'
 test_core_pager_overrides expect_success test_must_fail 'git -p request-pull'
-test_core_pager_subdir    expect_failure test_must_fail 'git -p request-pull'
+test_core_pager_subdir    expect_success test_must_fail 'git -p request-pull'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 
 test_default_pager        expect_success test_must_fail 'git -p'
@@ -380,7 +380,7 @@ test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
-test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+test_expect_success TTY 'core.pager in repo config works and retains cwd' '
 	sane_unset GIT_PAGER &&
 	test_config core.pager "cat >cwd-retained" &&
 	(
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v3 9/9] Test read_early_config()

[Johannes Schindelin]

So far, we had no explicit tests of that function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-config.c  | 15 +++++++++++++++
 t/t1309-early-config.sh | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 t/t1309-early-config.sh

diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 83a4f2ab869..8e3ed6a76cb 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -66,6 +66,16 @@ static int iterate_cb(const char *var, const char *value, void *data)
 	return 0;
 }
 
+static int early_config_cb(const char *var, const char *value, void *vdata)
+{
+	const char *key = vdata;
+
+	if (!strcmp(key, var))
+		printf("%s\n", value);
+
+	return 0;
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	int i, val;
@@ -73,6 +83,11 @@ int cmd_main(int argc, const char **argv)
 	const struct string_list *strptr;
 	struct config_set cs;
 
+	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		read_early_config(early_config_cb, (void *)argv[2]);
+		return 0;
+	}
+
 	setup_git_directory();
 
 	git_configset_init(&cs);
diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
new file mode 100755
index 00000000000..0c55dee514c
--- /dev/null
+++ b/t/t1309-early-config.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+test_description='Test read_early_config()'
+
+. ./test-lib.sh
+
+test_expect_success 'read early config' '
+	test_config early.config correct &&
+	test-config read_early_config early.config >output &&
+	test correct = "$(cat output)"
+'
+
+test_expect_success 'in a sub-directory' '
+	test_config early.config sub &&
+	mkdir -p sub &&
+	(
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test sub = "$(cat output)"
+'
+
+test_expect_success 'ceiling' '
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test -z "$(cat output)"
+'
+
+test_expect_success 'ceiling #2' '
+	mkdir -p xdg/git &&
+	git config -f xdg/git/config early.config xdg &&
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		XDG_CONFIG_HOME="$PWD"/xdg &&
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES XDG_CONFIG_HOME &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test xdg = "$(cat output)"
+'
+
+test_done
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v3 0/9] Fix the early config

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Notable notes:
>
> - In contrast to earlier versions, I no longer special-case init and
>   clone. Peff pointed out that this adds technical debt, and that we can
>   actually argue (for consistency's sake) that early config reads the
>   current repository config (if any) even for init and clone.
>
> - The read_early_config() function does not cache Git directory
>   discovery nor read values. If needed, this can be implemented later,
>   in a separate patch series.
>
> - The alias handling in git.c could possibly benefit from this work, but
>   again, this is a separate topic from the current patch series.

As Peff said in his review, I too find the result of this series a
more pleasant read than than original.

2/9 and corresponding 4/9 triggers "ERROR: trailing statements
should be on next line" from ../linux/scripts/checkpatch.pl because
of a line inherited from the original; I'll queue them with an
obvious style fix to work it around.

Thanks.

Re: [PATCH v3 0/9] Fix the early config

[Johannes Schindelin]

Hi Junio,

On Fri, 3 Mar 2017, Junio C Hamano wrote:

> 2/9 and corresponding 4/9 triggers "ERROR: trailing statements
> should be on next line" from ../linux/scripts/checkpatch.pl because
> of a line inherited from the original; I'll queue them with an
> obvious style fix to work it around.

Wow, it seems that script requires the entire Linux kernel repository to
be cloned, you cannot simply download just the script and run it.

In any case, as you pointed out, this style was inherited from the
original.

I squashed that style fix in, as you probably would have (your
js/early-config does not have anything beyond v2.12.0). But I have to
point out that it is conflating the purpose of this patch series (its goal
is *not* to fix any style). I am absolutely not a fan of that.

Ciao,
Johannes

Re: [PATCH v3 0/9] Fix the early config

[Johannes Schindelin]

Hi Junio,

On Fri, 3 Mar 2017, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Notable notes:
> >
> > - In contrast to earlier versions, I no longer special-case init and
> >   clone. Peff pointed out that this adds technical debt, and that we can
> >   actually argue (for consistency's sake) that early config reads the
> >   current repository config (if any) even for init and clone.
> >
> > - The read_early_config() function does not cache Git directory
> >   discovery nor read values. If needed, this can be implemented later,
> >   in a separate patch series.
> >
> > - The alias handling in git.c could possibly benefit from this work, but
> >   again, this is a separate topic from the current patch series.
> 
> As Peff said in his review, I too find the result of this series a
> more pleasant read than than original.

As do I.

> 2/9 and corresponding 4/9 triggers "ERROR: trailing statements
> should be on next line" from ../linux/scripts/checkpatch.pl because
> of a line inherited from the original; I'll queue them with an
> obvious style fix to work it around.

Thank you. I'll try to pick it up for v3 (which is needed, as I found
another issue that needs to be fixed).

Ciao,
Johannes

Re: [PATCH v3 0/9] Fix the early config

[Jeff King]

On Fri, Mar 03, 2017 at 06:31:55PM +0100, Johannes Schindelin wrote:

> Interdiff vs v2:
> [...]
>  +	 * When we are not about to create a repository ourselves (init or
>  +	 * clone) and when no .git/ directory was set up yet (in which case
>  +	 * git_config_with_options() would already have picked up the
>  +	 * repository config), we ask discover_git_directory() to figure out
>  +	 * whether there is any repository config we should use (but unlike
>  +	 * setup_git_directory_gently(), no global state is changed, most
>  +	 * notably, the current working directory is still the same after
>  +	 * the call).
>   	 */
>  -	if (!startup_info->creating_repository && !have_git_dir() &&
>  -	    discover_git_directory(&buf)) {
>  +	if (!have_git_dir() && discover_git_directory(&buf)) {

I think this "when we are not about to..." part of the comment is no
longer true, given the second part of the hunk.

>  @@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
>   	if (offset == cwd->len)
>   		return NULL;
>   
>  -	/* Make "offset" point to past the '/', and add a '/' at the end */
>  -	offset++;
>  +	/* Make "offset" point past the '/' (already the case for root dirs) */
>  +	if (offset != offset_1st_component(cwd->buf))
>  +		offset++;

Nice. I was worried we would have to have a hacky "well, sometimes we
don't add one here..." code, but using offset_1st_component says
exactly what we mean.

> +/* Find GIT_DIR without changing the working directory or other global state */
>  extern const char *discover_git_directory(struct strbuf *gitdir);

The parts that actually confused me were the parameters (mostly whether
gitdir was a directory to start looking in, or an output parameter). So
maybe:

  /*
   * Find GIT_DIR of the repository that contains the current working
   * directory, without changing the working directory or other global
   * state. The result is appended to gitdir. The return value is NULL
   * if no repository was found, or gitdir->buf otherwise.
   */

This looks good to me aside from those few comment nits. I'm still not
sure I understand how ceil_offset works in setup_git_directory_gently_1(),
but I don't think your patch actually changed it. I can live with my
confusion.

-Peff

Re: [PATCH v3 0/9] Fix the early config

[Junio C Hamano]

On Fri, Mar 3, 2017 at 11:39 PM, Jeff King <peff@peff.net> wrote:
>>  +     * When we are not about to create a repository ourselves (init or
>>  +     * clone) and when no .git/ directory was set up yet (in which case
>>  +     * git_config_with_options() would already have picked up the
>>  +     * repository config), we ask discover_git_directory() to figure out
>>  +     * whether there is any repository config we should use (but unlike
>>  +     * setup_git_directory_gently(), no global state is changed, most
>>  +     * notably, the current working directory is still the same after
>>  +     * the call).
>>        */
>>  -    if (!startup_info->creating_repository && !have_git_dir() &&
>>  -        discover_git_directory(&buf)) {
>>  +    if (!have_git_dir() && discover_git_directory(&buf)) {
>
> I think this "when we are not about to..." part of the comment is no
> longer true, given the second part of the hunk.

Good point.

> The parts that actually confused me were the parameters (mostly whether
> gitdir was a directory to start looking in, or an output parameter). So
> maybe:
>
>   /*
>    * Find GIT_DIR of the repository that contains the current working
>    * directory, without changing the working directory or other global
>    * state. The result is appended to gitdir. The return value is NULL
>    * if no repository was found, or gitdir->buf otherwise.
>    */

This, too.

> This looks good to me aside from those few comment nits. I'm still not
> sure I understand how ceil_offset works in setup_git_directory_gently_1(),
> but I don't think your patch actually changed it. I can live with my
> confusion.

I'll wait to see if Dscho wants to clarify the code and comments further.

Thanks.

Re: [PATCH v3 0/9] Fix the early config

[Johannes Schindelin]

Hi Peff,

On Sat, 4 Mar 2017, Jeff King wrote:

> On Fri, Mar 03, 2017 at 06:31:55PM +0100, Johannes Schindelin wrote:
> 
> > Interdiff vs v2:
> > [...]
> >  +	 * When we are not about to create a repository ourselves (init or
> >  +	 * clone) and when no .git/ directory was set up yet (in which case
> >  +	 * git_config_with_options() would already have picked up the
> >  +	 * repository config), we ask discover_git_directory() to figure out
> >  +	 * whether there is any repository config we should use (but unlike
> >  +	 * setup_git_directory_gently(), no global state is changed, most
> >  +	 * notably, the current working directory is still the same after
> >  +	 * the call).
> >   	 */
> >  -	if (!startup_info->creating_repository && !have_git_dir() &&
> >  -	    discover_git_directory(&buf)) {
> >  +	if (!have_git_dir() && discover_git_directory(&buf)) {
> 
> I think this "when we are not about to..." part of the comment is no
> longer true, given the second part of the hunk.

Yep, that was a stale part of that patch. Thanks for noticing!

> >  @@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
> >   	if (offset == cwd->len)
> >   		return NULL;
> >   
> >  -	/* Make "offset" point to past the '/', and add a '/' at the end */
> >  -	offset++;
> >  +	/* Make "offset" point past the '/' (already the case for root dirs) */
> >  +	if (offset != offset_1st_component(cwd->buf))
> >  +		offset++;
> 
> Nice. I was worried we would have to have a hacky "well, sometimes we
> don't add one here..." code, but using offset_1st_component says
> exactly what we mean.

Right. I also wanted to avoid that very, very much. My initial version
actually tried to detect whether cwd already has a trailing slash, but
then I figured that we can be much, much more precise here (and I am
really pleased how offset_1st_component() is *semantically* precise, i.e.
it describes very well what the code is supposed to do here).

> > +/* Find GIT_DIR without changing the working directory or other global state */
> >  extern const char *discover_git_directory(struct strbuf *gitdir);
> 
> The parts that actually confused me were the parameters (mostly whether
> gitdir was a directory to start looking in, or an output parameter). So
> maybe:
> 
>   /*
>    * Find GIT_DIR of the repository that contains the current working
>    * directory, without changing the working directory or other global
>    * state. The result is appended to gitdir. The return value is NULL
>    * if no repository was found, or gitdir->buf otherwise.
>    */

I changed it a little bit more. In particular, I changed the
discover_git_directory() function to return the pointer to the path
itself: it provides additional value, and if that is not what the caller
wants, they can use git_dir->buf just as well.

> This looks good to me aside from those few comment nits.

Thanks.

It is not obvious from the interdiff, but I had an incorrect fixup to 8/9
that actually wanted to go to 5/9: the code in
discover_git_repository() tests the repository version should be part of
the initial version of this function, of course.

There is one more thing I included in v4: when I (re-)implemented that
pre-command/post-command hook I was hinting at earlier, the test suite
identified a problem where an invalid .git file would prevent even `git
init` from working (it was actually much more complicated than that, but
the gist is that `git -p init` would fail, no matter how much sense it
may make to you to paginate an `init` run, it should still not fail,
right?). So I added a patch on top to fix that.

And another change: the GIT_DIR_NONE value was handled incorrectly in
discover_git_directory().

I am slightly disappointed that the these additional problems were not
spotted in any review but my own. And I had not even included a Duck.

> I'm still not sure I understand how ceil_offset works in
> setup_git_directory_gently_1(), but I don't think your patch actually
> changed it. I can live with my confusion.

Yes, that code is very confusing. It also does not help that the naming is
inconsistent in that it abbreviates "ceiling" but not "offset". What makes
it even worse is that the function name `longest_ancestor_length()` is
highly misleading: in Git's context, "ancestor" of sth pretty much always
refers to a commit reachable from sth, but in this context it refers to
the path of a directory containing sth.

So basically, we set ceil_offset to the offset of the last directory
separator in our path that corresponds to the most precise match in
GIT_CEILING_DIRECTORIES.

Example: given GIT_CEILING_DIRECTORIES /foo:/:/bar and a path of /foo/bar,
ceil_offset would be 4, pointing to the slash at the end of /foo/ because
that is the most precise match in GIT_CEILING_DIRECTORIES ("/" would also
match, but is less precise).

Later, setup_git_directory_gently_1() ensures that it does not go beyond
ceil_offset when looking for the parent directory as the next candidate to
test for .git/.

Hopefully that clears up the picture?

Ciao,
Dscho

Re: [PATCH v3 0/9] Fix the early config

[Jeff King]

On Tue, Mar 07, 2017 at 03:31:43PM +0100, Johannes Schindelin wrote:

> >   /*
> >    * Find GIT_DIR of the repository that contains the current working
> >    * directory, without changing the working directory or other global
> >    * state. The result is appended to gitdir. The return value is NULL
> >    * if no repository was found, or gitdir->buf otherwise.
> >    */
> 
> I changed it a little bit more. In particular, I changed the
> discover_git_directory() function to return the pointer to the path
> itself: it provides additional value, and if that is not what the caller
> wants, they can use git_dir->buf just as well.

Yes, that makes much more sense.

> There is one more thing I included in v4: when I (re-)implemented that
> pre-command/post-command hook I was hinting at earlier, the test suite
> identified a problem where an invalid .git file would prevent even `git
> init` from working (it was actually much more complicated than that, but
> the gist is that `git -p init` would fail, no matter how much sense it
> may make to you to paginate an `init` run, it should still not fail,
> right?). So I added a patch on top to fix that.

Good catch. Another "non-gentle" thing I noticed here while looking at
another thread: the repository-format version check uses the config
parser, which will die() in certain circumstances. So for instance:

  $ git init
  $ git rev-parse && echo ok
  ok

  $ echo '[core]repositoryformatversion = 10' >.git/config
  $ git rev-parse && echo ok
  fatal: Expected git repo version <= 1, found 10

  $ echo '[core]repositoryformatversion = foobar' >.git/config
  $ git rev-parse && echo ok
  fatal: bad numeric config value 'foobar' for 'core.repositoryformatversion' in file .git/config: invalid unit

  $ echo '[co' >.git/config
  $ git rev-parse && echo ok
  fatal: bad config line 1 in file .git/config

Your series correctly avoids the first failure by calling the
read/verify_repository_format functions correctly. But I think it would
get tripped up by the other two.

Fixing the first one is probably not too hard; check_repo_format()
should use a more forgiving parser than git_config_int().

The second one I thought would be tricky, but it looks like we added a
die_on_error flag in b2dc09455. That does what we want, but it needs to
be plumbed through to git_config_from_file().

> And another change: the GIT_DIR_NONE value was handled incorrectly in
> discover_git_directory().

This is the "if (setup_git_directory_1() <= 0)" change from the
interdiff? That's subtle. The compiler could have noticed if we used a
switch statement here. But then any new error conditions would have to
be added to that switch statement.

> I am slightly disappointed that the these additional problems were not
> spotted in any review but my own. And I had not even included a Duck.

Get used to being disappointed, I guess. A non-zero number of bugs will
slip through when writing code _and_ when reviewing it.

> [ceil_offset]
> Hopefully that clears up the picture?

Yes, it does. Thanks.

-Peff

Re: [PATCH v3 0/9] Fix the early config

[Johannes Schindelin]

Hi Peff,

On Wed, 8 Mar 2017, Jeff King wrote:

> Another "non-gentle" thing I noticed here while looking at
> another thread: the repository-format version check uses the config
> parser, which will die() in certain circumstances. [...]

Yes, that is part of the reason why I was not eager to add that check to
discover_git_directory(). The config code is die()-happy.

This is a much bigger problem, of course, and related to a constant gripe
of mine: I *always* get the quoting wrong in my aliases. Always. And when
I want to fix it, `git config -e` simply errors out because of an invalid
config. Yes, Git, I know, that is the exact reason why I want to edit the
config in the first place.

I am certain you will agree that this is a different topic, therefore
subject to a separate patch series.

In any case, I am fairly certain that the examples you showed demonstrate
that the config has to be rather broken for this patch series to have a
negative impact. And it still would report the broken config so that the
user is not blocked (she can fix the config and call the paginating
command again).

> On Tue, Mar 07, 2017 at 03:31:43PM +0100, Johannes Schindelin wrote:
> 
> > And another change: the GIT_DIR_NONE value was handled incorrectly in
> > discover_git_directory().
> 
> This is the "if (setup_git_directory_1() <= 0)" change from the
> interdiff? That's subtle.

Yes, it is subtle.

> The compiler could have noticed if we used a switch statement here. But
> then any new error conditions would have to be added to that switch
> statement.

We could still do that.

> > I am slightly disappointed that the these additional problems were not
> > spotted in any review but my own. And I had not even included a Duck.
> 
> Get used to being disappointed, I guess. A non-zero number of bugs will
> slip through when writing code _and_ when reviewing it.

I know that. I know that bugs are prone to come in through code
contributions. I don't go for 100%. But I would hope for a better rate
than we have right now: we pride ourselves in the OSS community to make bugs
shallow. I really would like to believe that we catch bugs rather than
discuss formatting (which should be automated) or white-space mangling
(which should not even be a problem if we were truly open for
contributions).

Maybe I am just a grumpy old guy. But I *hate* seeing how much buggy code
we get into Git, despite having a review process that does a very good job
of deterring seasoned developers from contributing.

I really wish it were different. I really wish that we did a better job at
catching bugs before they enter `master`. I really wish that I could be
proud of our code review process.

> > [ceil_offset]
> > Hopefully that clears up the picture?
> 
> Yes, it does. Thanks.

Perfect. Then the time I spent trying to figure all of this out was not
spent in vain.

Ciao,
Dscho

Re: [PATCH v3 0/9] Fix the early config

[Jeff King]

On Wed, Mar 08, 2017 at 05:18:46PM +0100, Johannes Schindelin wrote:

> On Wed, 8 Mar 2017, Jeff King wrote:
> 
> > Another "non-gentle" thing I noticed here while looking at
> > another thread: the repository-format version check uses the config
> > parser, which will die() in certain circumstances. [...]
> 
> Yes, that is part of the reason why I was not eager to add that check to
> discover_git_directory(). The config code is die()-happy.
> 
> This is a much bigger problem, of course, and related to a constant gripe
> of mine: I *always* get the quoting wrong in my aliases. Always. And when
> I want to fix it, `git config -e` simply errors out because of an invalid
> config. Yes, Git, I know, that is the exact reason why I want to edit the
> config in the first place.
> 
> I am certain you will agree that this is a different topic, therefore
> subject to a separate patch series.

I agree that in general it is a separate issue. Technically it could
cause a regression because with your series we are more eager to call
discover_git_directory() even when the command would not otherwise need
to even look at the current git directory.

But I don't think that is the case in practice. Your new function is
only called when we would try to read the config anyway. So it would
only affect cases which were previously so broken they did not properly
read the config (like running a command without RUN_SETUP from a
subdirectory of the working tree, which was _supposed_ to respect the
config but failed to do so).

So yeah, I'm happy to leave it for now.

-Peff

Re: [PATCH v3 0/9] Fix the early config

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Good catch. Another "non-gentle" thing I noticed here while looking at
> another thread: the repository-format version check uses the config
> parser, which will die() in certain circumstances. So for instance:
>
>   $ git init
>   $ git rev-parse && echo ok
>   ok
>
>   $ echo '[core]repositoryformatversion = 10' >.git/config
>   $ git rev-parse && echo ok
>   fatal: Expected git repo version <= 1, found 10

Just to set my expectation straight.  Do you expect/wish this not to
fail because of this in cmd_rev_parse()?

	/* No options; just report on whether we're in a git repo or not. */
	if (argc == 1) {
		setup_git_directory();
		git_config(git_default_config, NULL);
		return 0;
	}

Because we do not have anything other than yes/no to the question
"Are we in Git repository?", I'd expect that the expected answer to
the question would be "no" (if we could say "Yes, we are in a Git
repository but its version and layout is unknown to us so we are not
supposed to look at or touch it", that is a different matter).

So "fatal:" may be bad, but I think not seeing "ok" is what we
want to happen in this case.

Having said that, I am not sure asking for default-config is what we
wanted to do in the above code.  Perhaps a more modern way to write
the above code would be to do the "gently" version of setup, without
calling git_config() ourselves, and return the resulting value
returned in *nongit_ok?  If we can do so without triggering "fatal:"
and still return "no, we are not in a Git repository we are supposed
to touch", that would be good.

Or are you discussing a more general issue, iow, anything that can
work without repository (i.e. those who do _gently version of the
setup and act on *nongit_ok) should pretend as if there were no
(broken) repository and take the "no we are not in a repository"
codepath?

Re: [PATCH v3 0/9] Fix the early config

[Jeff King]

On Wed, Mar 08, 2017 at 09:09:31AM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Good catch. Another "non-gentle" thing I noticed here while looking at
> > another thread: the repository-format version check uses the config
> > parser, which will die() in certain circumstances. So for instance:
> >
> >   $ git init
> >   $ git rev-parse && echo ok
> >   ok
> >
> >   $ echo '[core]repositoryformatversion = 10' >.git/config
> >   $ git rev-parse && echo ok
> >   fatal: Expected git repo version <= 1, found 10
> 
> Just to set my expectation straight.  Do you expect/wish this not to
> fail because of this in cmd_rev_parse()?

No, I was just using "rev-parse" as a sample command that tried to do
repo setup. I meant the above snippet that you quoted to both be fine
and expected outputs. The problem is the _other_ two cases where the
config code dies before we even get to the version-number check.

> Or are you discussing a more general issue, iow, anything that can
> work without repository (i.e. those who do _gently version of the
> setup and act on *nongit_ok) should pretend as if there were no
> (broken) repository and take the "no we are not in a repository"
> codepath?

Yes, exactly.  It would have been less confusing if I picked something
that passed nongit_ok. Like hash-object:

  $ git init
  $ echo content >file
  $ git hash-object file
  d95f3ad14dee633a758d2e331151e950dd13e4ed

  $ echo '[core]repositoryformatversion = 10' >.git/config
  $ git hash-object file
  warning: Expected git repo version <= 1, found 10
  d95f3ad14dee633a758d2e331151e950dd13e4ed

The warning is fine and reasonable here. But then:

  $ echo '[core]repositoryformatversion = foobar' >.git/config
  $ git hash-object file
  fatal: bad numeric config value 'foobar' for 'core.repositoryformatversion' in file .git/config: invalid unit

That's wrong. We're supposed to be gentle. And ditto:

  $ echo '[co' >.git/config
  $ git hash-object file
  fatal: bad config line 1 in file .git/config

Those last two should issue a warning at most, and then let the command
continue.

-Peff

Re: [PATCH v3 0/9] Fix the early config

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> Or are you discussing a more general issue, iow, anything that can
>> work without repository (i.e. those who do _gently version of the
>> setup and act on *nongit_ok) should pretend as if there were no
>> (broken) repository and take the "no we are not in a repository"
>> codepath?
>
> Yes, exactly.  It would have been less confusing if I picked something
> that passed nongit_ok. Like hash-object:
>
>   $ git init
>   $ echo content >file
>   $ git hash-object file
>   d95f3ad14dee633a758d2e331151e950dd13e4ed
>
>   $ echo '[core]repositoryformatversion = 10' >.git/config
>   $ git hash-object file
>   warning: Expected git repo version <= 1, found 10
>   d95f3ad14dee633a758d2e331151e950dd13e4ed
>
> The warning is fine and reasonable here. But then:
>
>   $ echo '[core]repositoryformatversion = foobar' >.git/config
>   $ git hash-object file
>   fatal: bad numeric config value 'foobar' for 'core.repositoryformatversion' in file .git/config: invalid unit
>
> That's wrong. We're supposed to be gentle. And ditto:
>
>   $ echo '[co' >.git/config
>   $ git hash-object file
>   fatal: bad config line 1 in file .git/config
>
> Those last two should issue a warning at most, and then let the command
> continue.

Yeah, I agree with that as one of the worthy goals.  IIUC, we
decided to leave that outside of this series and later fix on top,
which is fine by me, too.

Re: [PATCH v3 0/9] Fix the early config

[Johannes Schindelin]

Hi,

On Wed, 8 Mar 2017, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> >> Or are you discussing a more general issue, iow, anything that can
> >> work without repository (i.e. those who do _gently version of the
> >> setup and act on *nongit_ok) should pretend as if there were no
> >> (broken) repository and take the "no we are not in a repository"
> >> codepath?
> >
> > Yes, exactly.  It would have been less confusing if I picked something
> > that passed nongit_ok. Like hash-object:

... or like testing the early config directly?

> >   $ git init
> >   $ echo content >file
> >   $ git hash-object file
> >   d95f3ad14dee633a758d2e331151e950dd13e4ed
> >
> >   $ echo '[core]repositoryformatversion = 10' >.git/config
> >   $ git hash-object file
> >   warning: Expected git repo version <= 1, found 10
> >   d95f3ad14dee633a758d2e331151e950dd13e4ed
> >
> > The warning is fine and reasonable here. But then:
> >
> >   $ echo '[core]repositoryformatversion = foobar' >.git/config
> >   $ git hash-object file
> >   fatal: bad numeric config value 'foobar' for 'core.repositoryformatversion' in file .git/config: invalid unit
> >
> > That's wrong. We're supposed to be gentle. And ditto:
> >
> >   $ echo '[co' >.git/config
> >   $ git hash-object file
> >   fatal: bad config line 1 in file .git/config
> >
> > Those last two should issue a warning at most, and then let the command
> > continue.
> 
> Yeah, I agree with that as one of the worthy goals.  IIUC, we
> decided to leave that outside of this series and later fix on top,
> which is fine by me, too.

How about this on top, then:

-- snipsnap --
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: [PATCH] t1309: document cases where we would want early config not to
 die()

Jeff King came up with a couple examples that demonstrate how the new
read_early_config() that looks harder for the current .git/ directory
could die() in an undesirable way.

Let's add those cases to the test script, to document what we would like
to happen when early config encounters problems.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t1309-early-config.sh | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
index 0c55dee514c..027eca63a3c 100755
--- a/t/t1309-early-config.sh
+++ b/t/t1309-early-config.sh
@@ -47,4 +47,29 @@ test_expect_success 'ceiling #2' '
 	test xdg = "$(cat output)"
 '
 
+test_with_config ()
+{
+	rm -rf throwaway &&
+	git init throwaway &&
+	(
+		cd throwaway &&
+		echo "$*" >.git/config &&
+		test-config read_early_config early.config
+	)
+}
+
+test_expect_success 'ignore .git/ with incompatible repository version' '
+	test_with_config "[core]repositoryformatversion = 999999" 2>err &&
+	grep "warning:.* Expected git repo version <= [1-9]" err
+'
+
+test_expect_failure 'ignore .git/ with invalid repository version' '
+	test_with_config "[core]repositoryformatversion = invalid"
+'
+
+
+test_expect_failure 'ignore .git/ with invalid config' '
+	test_with_config "["
+'
+
 test_done
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v3 0/9] Fix the early config

[Jeff King]

On Thu, Mar 09, 2017 at 12:51:06PM +0100, Johannes Schindelin wrote:

> On Wed, 8 Mar 2017, Junio C Hamano wrote:
> 
> > Jeff King <peff@peff.net> writes:
> > 
> > >> Or are you discussing a more general issue, iow, anything that can
> > >> work without repository (i.e. those who do _gently version of the
> > >> setup and act on *nongit_ok) should pretend as if there were no
> > >> (broken) repository and take the "no we are not in a repository"
> > >> codepath?
> > >
> > > Yes, exactly.  It would have been less confusing if I picked something
> > > that passed nongit_ok. Like hash-object:
> 
> ... or like testing the early config directly?

I was trying to demonstrate that the problem existed already without
your patch series.

> -- snipsnap --
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> Subject: [PATCH] t1309: document cases where we would want early config not to
>  die()
> 
> Jeff King came up with a couple examples that demonstrate how the new
> read_early_config() that looks harder for the current .git/ directory
> could die() in an undesirable way.
> 
> Let's add those cases to the test script, to document what we would like
> to happen when early config encounters problems.

Yep, these all look fine.

-Peff

Re: [PATCH v3 0/9] Fix the early config

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> > > Yes, exactly.  It would have been less confusing if I picked something
>> > > that passed nongit_ok. Like hash-object:
>> 
>> ... or like testing the early config directly?
>
> I was trying to demonstrate that the problem existed already without
> your patch series.

Yup, they are not new breakages introduced by Dscho's change.

>> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>> Subject: [PATCH] t1309: document cases where we would want early config not to
>>  die()
>> 
>> Jeff King came up with a couple examples that demonstrate how the new
>> read_early_config() that looks harder for the current .git/ directory
>> could die() in an undesirable way.
>> 
>> Let's add those cases to the test script, to document what we would like
>> to happen when early config encounters problems.
>
> Yep, these all look fine.

OK.  Thanks, both.

[PATCH v4 00/10] Fix the early config

[Johannes Schindelin]

These patches are an attempt to make Git's startup sequence a bit less
surprising.

The idea here is to discover the .git/ directory gently (i.e. without
changing the current working directory, nor any global variables), and
to use it to read the .git/config file early, before we actually called
setup_git_directory() (if we ever do that).

This also allows us to fix the early config e.g. to determine the pager
or to resolve aliases in a non-surprising manner.

My own use case: in the GVFS Git fork, we need to execute pre-command
and post-command hooks before and after *every* Git command. A previous
version of the pre-command/post-command hook support was broken, as it
used run_hook() which implicitly called setup_git_directory() too early.
The discover_git_directory() function (and due to core.hooksPath also
the read_early_config() function) helped me fix this.

Notable notes:

- Even if it can cause surprising problems, `init` and `clone` are not
  special-cased. Rationale: it would introduce technical debt and
  violate the Principle Of Least Astonishment.

- The read_early_config() function does not cache Git directory
  discovery nor read values. This is left for another patch series, if
  it ever becomes necessary.

- The alias handling in git.c could possibly benefit from this work, but
  again, this is a separate topic from the current patch series.

Changes since v3:

- instead of just fixing the `== '/'` comparison, 2/9 now *also* changes
  the style of the original to something Linux' checkpatch.pl prefers.

- fixed the comment added in 8/9 which was stale from an earlier
  iteration of this patch series.

- adjusted the commit above discover_git_directory()'s declaration, to
  make it more understandable.

- touched up the commit message of 5/9 to make the flow more natural.

- moved a fault fixup: by mistake, the repository_format check was added
  to 8/9 when it really needed to go to 5/9, where the
  discover_git_directory() function was introduced (I noticed this while
  rebasing).

- fixed discover_git_directory() when setup_git_directory_gently_1()
  returned GIT_DIR_NONE (the previous iteration would *not* return NULL
  in that case).


Johannes Schindelin (10):
  t7006: replace dubious test
  setup_git_directory(): use is_dir_sep() helper
  Prepare setup_discovered_git_directory() the root directory
  setup_git_directory_1(): avoid changing global state
  Introduce the discover_git_directory() function
  Make read_early_config() reusable
  read_early_config(): avoid .git/config hack when unneeded
  read_early_config(): really discover .git/
  Test read_early_config()
  setup_git_directory_gently_1(): avoid die()ing

 cache.h                 |   8 ++
 config.c                |  25 +++++
 pager.c                 |  31 ------
 setup.c                 | 246 +++++++++++++++++++++++++++++++++---------------
 t/helper/test-config.c  |  15 +++
 t/t1309-early-config.sh |  50 ++++++++++
 t/t7006-pager.sh        |  18 +++-
 7 files changed, 282 insertions(+), 111 deletions(-)
 create mode 100755 t/t1309-early-config.sh


base-commit: 3bc53220cb2dcf709f7a027a3f526befd021d858
Published-As: https://github.com/dscho/git/releases/tag/early-config-v4
Fetch-It-Via: git fetch https://github.com/dscho/git early-config-v4

Interdiff vs v3:

 diff --git a/cache.h b/cache.h
 index 8a4580f921d..e7b57457e73 100644
 --- a/cache.h
 +++ b/cache.h
 @@ -518,7 +518,12 @@ extern void set_git_work_tree(const char *tree);
  #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
  
  extern void setup_work_tree(void);
 -/* Find GIT_DIR without changing the working directory or other global state */
 +/*
 + * Find GIT_DIR of the repository that contains the current working directory,
 + * without changing the working directory or other global state. The result is
 + * appended to gitdir. The return value is either NULL if no repository was
 + * found, or pointing to the path inside gitdir's buffer.
 + */
  extern const char *discover_git_directory(struct strbuf *gitdir);
  extern const char *setup_git_directory_gently(int *);
  extern const char *setup_git_directory(void);
 diff --git a/config.c b/config.c
 index 749623a9649..a88df53fdbc 100644
 --- a/config.c
 +++ b/config.c
 @@ -1419,14 +1419,12 @@ void read_early_config(config_fn_t cb, void *data)
  	git_config_with_options(cb, data, NULL, 1);
  
  	/*
 -	 * When we are not about to create a repository ourselves (init or
 -	 * clone) and when no .git/ directory was set up yet (in which case
 -	 * git_config_with_options() would already have picked up the
 -	 * repository config), we ask discover_git_directory() to figure out
 -	 * whether there is any repository config we should use (but unlike
 +	 * When setup_git_directory() was not yet asked to discover the
 +	 * GIT_DIR, we ask discover_git_directory() to figure out whether there
 +	 * is any repository config we should use (but unlike
  	 * setup_git_directory_gently(), no global state is changed, most
 -	 * notably, the current working directory is still the same after
 -	 * the call).
 +	 * notably, the current working directory is still the same after the
 +	 * call).
  	 */
  	if (!have_git_dir() && discover_git_directory(&buf)) {
  		struct git_config_source repo_config;
 diff --git a/setup.c b/setup.c
 index 5320ae37314..9118b48590a 100644
 --- a/setup.c
 +++ b/setup.c
 @@ -825,7 +825,8 @@ enum discovery_result {
  	GIT_DIR_BARE,
  	/* these are errors */
  	GIT_DIR_HIT_CEILING = -1,
 -	GIT_DIR_HIT_MOUNT_POINT = -2
 +	GIT_DIR_HIT_MOUNT_POINT = -2,
 +	GIT_DIR_INVALID_GITFILE = -3
  };
  
  /*
 @@ -842,7 +843,8 @@ enum discovery_result {
   * `dir` (i.e. *not* necessarily the cwd).
   */
  static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 -							  struct strbuf *gitdir)
 +							  struct strbuf *gitdir,
 +							  int die_on_error)
  {
  	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
  	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
 @@ -890,14 +892,22 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
  	if (one_filesystem)
  		current_device = get_device_or_die(dir->buf, NULL, 0);
  	for (;;) {
 -		int offset = dir->len;
 +		int offset = dir->len, error_code = 0;
  
  		if (offset > min_offset)
  			strbuf_addch(dir, '/');
  		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
 -		gitdirenv = read_gitfile(dir->buf);
 -		if (!gitdirenv && is_git_directory(dir->buf))
 -			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 +		gitdirenv = read_gitfile_gently(dir->buf, die_on_error ?
 +						NULL : &error_code);
 +		if (!gitdirenv) {
 +			if (die_on_error ||
 +			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
 +				if (is_git_directory(dir->buf))
 +				    gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 +			} else if (error_code &&
 +				   error_code != READ_GITFILE_ERR_STAT_FAILED)
 +				return GIT_DIR_INVALID_GITFILE;
 +		}
  		strbuf_setlen(dir, offset);
  		if (gitdirenv) {
  			strbuf_addstr(gitdir, gitdirenv);
 @@ -912,7 +922,8 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
  		if (offset <= min_offset)
  			return GIT_DIR_HIT_CEILING;
  
 -		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]));
 +		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]))
 +			; /* continue */
  		if (offset <= ceil_offset)
  			return GIT_DIR_HIT_CEILING;
  
 @@ -933,7 +944,7 @@ const char *discover_git_directory(struct strbuf *gitdir)
  		return NULL;
  
  	cwd_len = dir.len;
 -	if (setup_git_directory_gently_1(&dir, gitdir) < 0) {
 +	if (setup_git_directory_gently_1(&dir, gitdir, 0) <= 0) {
  		strbuf_release(&dir);
  		return NULL;
  	}
 @@ -963,7 +974,7 @@ const char *discover_git_directory(struct strbuf *gitdir)
  		return NULL;
  	}
  
 -	return gitdir->buf;
 +	return gitdir->buf + gitdir_offset;
  }
  
  const char *setup_git_directory_gently(int *nongit_ok)
 @@ -992,7 +1003,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
  		die_errno(_("Unable to read current working directory"));
  	strbuf_addbuf(&dir, &cwd);
  
 -	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
 +	switch (setup_git_directory_gently_1(&dir, &gitdir, 1)) {
  	case GIT_DIR_NONE:
  		prefix = NULL;
  		break;

-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 01/10] t7006: replace dubious test

[Johannes Schindelin]

The idea of the test case "git -p - core.pager is not used from
subdirectory" was to verify that the setup_git_directory() function had
not been called just to obtain the core.pager setting.

However, we are about to fix the early config machinery so that it
*does* work, without messing up the global state.

Once that is done, the core.pager setting *will* be used, even when
running from a subdirectory, and that is a Good Thing.

The intention of that test case, however, was to verify that the
setup_git_directory() function has not run, because it changes global
state such as the current working directory.

To keep that spirit, but fix the incorrect assumption, this patch
replaces that test case by a new one that verifies that the pager is
run in the subdirectory, i.e. that the current working directory has
not been changed at the time the pager is configured and launched, even
if the `rev-parse` command requires a .git/ directory and *will* change
the working directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t7006-pager.sh | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index c8dc665f2fd..304ae06c600 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -378,9 +378,19 @@ test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 test_default_pager        expect_success test_must_fail 'git -p'
 test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
-test_no_local_config_subdir expect_success test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
+test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+	sane_unset GIT_PAGER &&
+	test_config core.pager "cat >cwd-retained" &&
+	(
+		cd sub &&
+		rm -f cwd-retained &&
+		test_terminal git -p rev-parse HEAD &&
+		test_path_is_file cwd-retained
+	)
+'
+
 test_doesnt_paginate      expect_failure test_must_fail 'git -p nonsense'
 
 test_pager_choices                       'git shortlog'
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 02/10] setup_git_directory(): use is_dir_sep() helper

[Johannes Schindelin]

It is okay in practice to test for forward slashes in the output of
getcwd(), because we go out of our way to convert backslashes to forward
slashes in getcwd()'s output on Windows.

Still, the correct way to test for a dir separator is by using the
helper function we introduced for that very purpose. It also serves as a
good documentation what the code tries to do (not "how").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index 967f289f1ef..4a15b105676 100644
--- a/setup.c
+++ b/setup.c
@@ -910,7 +910,9 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 			return setup_bare_git_dir(&cwd, offset, nongit_ok);
 
 		offset_parent = offset;
-		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
+		while (--offset_parent > ceil_offset &&
+		       !is_dir_sep(cwd.buf[offset_parent]))
+			; /* continue */
 		if (offset_parent <= ceil_offset)
 			return setup_nongit(cwd.buf, nongit_ok);
 		if (one_filesystem) {
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 03/10] Prepare setup_discovered_git_directory() the root directory

[Johannes Schindelin]

Currently, the offset parameter (indicating what part of the cwd
parameter corresponds to the current directory after discovering the
.git/ directory) is set to 0 when we are running in the root directory.

However, in the next patches we will avoid changing the current working
directory while searching for the .git/ directory, meaning that the
offset corresponding to the root directory will have to be 1 to reflect
that this directory is characterized by the path "/" (and not "").

So let's make sure that setup_discovered_git_directory() only tries to
append the trailing slash to non-root directories.

Note: the setup_bare_git_directory() does not need a corresponding
change, as it does not want to return a prefix.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/setup.c b/setup.c
index 4a15b105676..20a1f0f870e 100644
--- a/setup.c
+++ b/setup.c
@@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 	if (offset == cwd->len)
 		return NULL;
 
-	/* Make "offset" point to past the '/', and add a '/' at the end */
-	offset++;
+	/* Make "offset" point past the '/' (already the case for root dirs) */
+	if (offset != offset_1st_component(cwd->buf))
+		offset++;
+	/* Add a '/' at the end */
 	strbuf_addch(cwd, '/');
 	return cwd->buf + offset;
 }
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 04/10] setup_git_directory_1(): avoid changing global state

[Johannes Schindelin]

For historical reasons, Git searches for the .git/ directory (or the
.git file) by changing the working directory successively to the parent
directory of the current directory, until either anything was found or
until a ceiling or a mount point is hit.

Further global state may be changed in case a .git/ directory was found.

We do have a use case, though, where we would like to find the .git/
directory without having any global state touched, though: when we read
the early config e.g. for the pager or for alias expansion.

Let's just move all of code that changes any global state out of the
function `setup_git_directory_gently_1()` into
`setup_git_directory_gently()`.

In subsequent patches, we will use the _1() function in a new
`discover_git_directory()` function that we will then use for the early
config code.

Note: the new loop is a *little* tricky, as we have to handle the root
directory specially: we cannot simply strip away the last component
including the slash, as the root directory only has that slash. To remedy
that, we introduce the `min_offset` variable that holds the minimal length
of an absolute path, and using that to special-case the root directory,
including an early exit before trying to find the parent of the root
directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 187 ++++++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 112 insertions(+), 75 deletions(-)

diff --git a/setup.c b/setup.c
index 20a1f0f870e..d7af343d14e 100644
--- a/setup.c
+++ b/setup.c
@@ -818,50 +818,49 @@ static int canonicalize_ceiling_entry(struct string_list_item *item,
 	}
 }
 
+enum discovery_result {
+	GIT_DIR_NONE = 0,
+	GIT_DIR_EXPLICIT,
+	GIT_DIR_DISCOVERED,
+	GIT_DIR_BARE,
+	/* these are errors */
+	GIT_DIR_HIT_CEILING = -1,
+	GIT_DIR_HIT_MOUNT_POINT = -2
+};
+
 /*
  * We cannot decide in this function whether we are in the work tree or
  * not, since the config can only be read _after_ this function was called.
+ *
+ * Also, we avoid changing any global state (such as the current working
+ * directory) to allow early callers.
+ *
+ * The directory where the search should start needs to be passed in via the
+ * `dir` parameter; upon return, the `dir` buffer will contain the path of
+ * the directory where the search ended, and `gitdir` will contain the path of
+ * the discovered .git/ directory, if any. This path may be relative against
+ * `dir` (i.e. *not* necessarily the cwd).
  */
-static const char *setup_git_directory_gently_1(int *nongit_ok)
+static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
+							  struct strbuf *gitdir)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
-	static struct strbuf cwd = STRBUF_INIT;
-	const char *gitdirenv, *ret;
-	char *gitfile;
-	int offset, offset_parent, ceil_offset = -1;
+	const char *gitdirenv;
+	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
 	dev_t current_device = 0;
 	int one_filesystem = 1;
 
 	/*
-	 * We may have read an incomplete configuration before
-	 * setting-up the git directory. If so, clear the cache so
-	 * that the next queries to the configuration reload complete
-	 * configuration (including the per-repo config file that we
-	 * ignored previously).
-	 */
-	git_config_clear();
-
-	/*
-	 * Let's assume that we are in a git repository.
-	 * If it turns out later that we are somewhere else, the value will be
-	 * updated accordingly.
-	 */
-	if (nongit_ok)
-		*nongit_ok = 0;
-
-	if (strbuf_getcwd(&cwd))
-		die_errno(_("Unable to read current working directory"));
-	offset = cwd.len;
-
-	/*
 	 * If GIT_DIR is set explicitly, we're not going
 	 * to do any discovery, but we still do repository
 	 * validation.
 	 */
 	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
-	if (gitdirenv)
-		return setup_explicit_git_dir(gitdirenv, &cwd, nongit_ok);
+	if (gitdirenv) {
+		strbuf_addstr(gitdir, gitdirenv);
+		return GIT_DIR_EXPLICIT;
+	}
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -869,15 +868,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP, -1);
 		filter_string_list(&ceiling_dirs, 0,
 				   canonicalize_ceiling_entry, &empty_entry_found);
-		ceil_offset = longest_ancestor_length(cwd.buf, &ceiling_dirs);
+		ceil_offset = longest_ancestor_length(dir->buf, &ceiling_dirs);
 		string_list_clear(&ceiling_dirs, 0);
 	}
 
-	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
-		ceil_offset = 1;
+	if (ceil_offset < 0)
+		ceil_offset = min_offset - 2;
 
 	/*
-	 * Test in the following order (relative to the cwd):
+	 * Test in the following order (relative to the dir):
 	 * - .git (file containing "gitdir: <path>")
 	 * - .git/
 	 * - ./ (bare)
@@ -889,63 +888,101 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	 */
 	one_filesystem = !git_env_bool("GIT_DISCOVERY_ACROSS_FILESYSTEM", 0);
 	if (one_filesystem)
-		current_device = get_device_or_die(".", NULL, 0);
+		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
-		if (gitfile)
-			gitdirenv = gitfile = xstrdup(gitfile);
-		else {
-			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
-				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		int offset = dir->len;
+
+		if (offset > min_offset)
+			strbuf_addch(dir, '/');
+		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
+		gitdirenv = read_gitfile(dir->buf);
+		if (!gitdirenv && is_git_directory(dir->buf))
+			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		strbuf_setlen(dir, offset);
+		if (gitdirenv) {
+			strbuf_addstr(gitdir, gitdirenv);
+			return GIT_DIR_DISCOVERED;
 		}
 
-		if (gitdirenv) {
-			ret = setup_discovered_git_dir(gitdirenv,
-						       &cwd, offset,
-						       nongit_ok);
-			free(gitfile);
-			return ret;
+		if (is_git_directory(dir->buf)) {
+			strbuf_addstr(gitdir, ".");
+			return GIT_DIR_BARE;
 		}
-		free(gitfile);
 
-		if (is_git_directory("."))
-			return setup_bare_git_dir(&cwd, offset, nongit_ok);
+		if (offset <= min_offset)
+			return GIT_DIR_HIT_CEILING;
 
-		offset_parent = offset;
-		while (--offset_parent > ceil_offset &&
-		       !is_dir_sep(cwd.buf[offset_parent]))
+		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]))
 			; /* continue */
-		if (offset_parent <= ceil_offset)
-			return setup_nongit(cwd.buf, nongit_ok);
-		if (one_filesystem) {
-			dev_t parent_device = get_device_or_die("..", cwd.buf,
-								offset);
-			if (parent_device != current_device) {
-				if (nongit_ok) {
-					if (chdir(cwd.buf))
-						die_errno(_("Cannot come back to cwd"));
-					*nongit_ok = 1;
-					return NULL;
-				}
-				strbuf_setlen(&cwd, offset);
-				die(_("Not a git repository (or any parent up to mount point %s)\n"
-				"Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
-				    cwd.buf);
-			}
-		}
-		if (chdir("..")) {
-			strbuf_setlen(&cwd, offset);
-			die_errno(_("Cannot change to '%s/..'"), cwd.buf);
-		}
-		offset = offset_parent;
+		if (offset <= ceil_offset)
+			return GIT_DIR_HIT_CEILING;
+
+		strbuf_setlen(dir, offset > min_offset ?  offset : min_offset);
+		if (one_filesystem &&
+		    current_device != get_device_or_die(dir->buf, NULL, offset))
+			return GIT_DIR_HIT_MOUNT_POINT;
 	}
 }
 
 const char *setup_git_directory_gently(int *nongit_ok)
 {
+	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
 	const char *prefix;
 
-	prefix = setup_git_directory_gently_1(nongit_ok);
+	/*
+	 * We may have read an incomplete configuration before
+	 * setting-up the git directory. If so, clear the cache so
+	 * that the next queries to the configuration reload complete
+	 * configuration (including the per-repo config file that we
+	 * ignored previously).
+	 */
+	git_config_clear();
+
+	/*
+	 * Let's assume that we are in a git repository.
+	 * If it turns out later that we are somewhere else, the value will be
+	 * updated accordingly.
+	 */
+	if (nongit_ok)
+		*nongit_ok = 0;
+
+	if (strbuf_getcwd(&cwd))
+		die_errno(_("Unable to read current working directory"));
+	strbuf_addbuf(&dir, &cwd);
+
+	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
+	case GIT_DIR_NONE:
+		prefix = NULL;
+		break;
+	case GIT_DIR_EXPLICIT:
+		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
+		break;
+	case GIT_DIR_DISCOVERED:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
+						  nongit_ok);
+		break;
+	case GIT_DIR_BARE:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
+		break;
+	case GIT_DIR_HIT_CEILING:
+		prefix = setup_nongit(cwd.buf, nongit_ok);
+		break;
+	case GIT_DIR_HIT_MOUNT_POINT:
+		if (nongit_ok) {
+			*nongit_ok = 1;
+			return NULL;
+		}
+		die(_("Not a git repository (or any parent up to mount point %s)\n"
+		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
+		    dir.buf);
+	default:
+		die("BUG: unhandled setup_git_directory_1() result");
+	}
+
 	if (prefix)
 		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
 	else
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v4 04/10] setup_git_directory_1(): avoid changing global state

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
> +	case GIT_DIR_NONE:
> +		prefix = NULL;
> +		break;
> +	case GIT_DIR_EXPLICIT:
> +		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
> +		break;
> +	case GIT_DIR_DISCOVERED:
> +		if (dir.len < cwd.len && chdir(dir.buf))
> +			die(_("Cannot change to '%s'"), dir.buf);
> +		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
> +						  nongit_ok);
> +		break;
> +	case GIT_DIR_BARE:
> +		if (dir.len < cwd.len && chdir(dir.buf))
> +			die(_("Cannot change to '%s'"), dir.buf);
> +		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
> +		break;
> +	case GIT_DIR_HIT_CEILING:
> +		prefix = setup_nongit(cwd.buf, nongit_ok);
> +		break;
> +	case GIT_DIR_HIT_MOUNT_POINT:
> +		if (nongit_ok) {
> +			*nongit_ok = 1;
> +			return NULL;
> +		}
> +		die(_("Not a git repository (or any parent up to mount point %s)\n"
> +		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
> +		    dir.buf);
> +	default:
> +		die("BUG: unhandled setup_git_directory_1() result");
> +	}


I _might_ find niggles in other patches (and other parts of this
patch) that enables the above clean implementation, but this
switch() statement speaks of the value of this entire series ;-)

Very nicely done.

Re: [PATCH v4 04/10] setup_git_directory_1(): avoid changing global state

[Brandon Williams]

On 03/07, Johannes Schindelin wrote:
>  const char *setup_git_directory_gently(int *nongit_ok)
>  {
> +	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;

I couldn't see any strbuf_release() calls for these strbufs so there may
be some memory leaking here.

>  	const char *prefix;
>  
> -	prefix = setup_git_directory_gently_1(nongit_ok);
> +	/*
> +	 * We may have read an incomplete configuration before
> +	 * setting-up the git directory. If so, clear the cache so
> +	 * that the next queries to the configuration reload complete
> +	 * configuration (including the per-repo config file that we
> +	 * ignored previously).
> +	 */
> +	git_config_clear();
> +
> +	/*
> +	 * Let's assume that we are in a git repository.
> +	 * If it turns out later that we are somewhere else, the value will be
> +	 * updated accordingly.
> +	 */
> +	if (nongit_ok)
> +		*nongit_ok = 0;
> +
> +	if (strbuf_getcwd(&cwd))
> +		die_errno(_("Unable to read current working directory"));
> +	strbuf_addbuf(&dir, &cwd);
> +
> +	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
> +	case GIT_DIR_NONE:
> +		prefix = NULL;
> +		break;
> +	case GIT_DIR_EXPLICIT:
> +		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
> +		break;
> +	case GIT_DIR_DISCOVERED:
> +		if (dir.len < cwd.len && chdir(dir.buf))
> +			die(_("Cannot change to '%s'"), dir.buf);
> +		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
> +						  nongit_ok);
> +		break;
> +	case GIT_DIR_BARE:
> +		if (dir.len < cwd.len && chdir(dir.buf))
> +			die(_("Cannot change to '%s'"), dir.buf);
> +		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
> +		break;
> +	case GIT_DIR_HIT_CEILING:
> +		prefix = setup_nongit(cwd.buf, nongit_ok);
> +		break;
> +	case GIT_DIR_HIT_MOUNT_POINT:
> +		if (nongit_ok) {
> +			*nongit_ok = 1;
> +			return NULL;
> +		}
> +		die(_("Not a git repository (or any parent up to mount point %s)\n"
> +		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
> +		    dir.buf);
> +	default:
> +		die("BUG: unhandled setup_git_directory_1() result");
> +	}
> +
>  	if (prefix)
>  		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
>  	else
> -- 
> 2.12.0.windows.1.7.g94dafc3b124
> 
> 

-- 
Brandon Williams

Re: [PATCH v4 04/10] setup_git_directory_1(): avoid changing global state

[Johannes Schindelin]

Hi Brandon,

On Tue, 7 Mar 2017, Brandon Williams wrote:

> On 03/07, Johannes Schindelin wrote:
> >  const char *setup_git_directory_gently(int *nongit_ok)
> >  {
> > +	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
> 
> I couldn't see any strbuf_release() calls for these strbufs so there may
> be some memory leaking here.

You are correct, of course. Something like this may work:

-- snipsnap --
diff --git a/setup.c b/setup.c
index 9118b48590a..c822582b96e 100644
--- a/setup.c
+++ b/setup.c
@@ -1027,6 +1027,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	case GIT_DIR_HIT_MOUNT_POINT:
 		if (nongit_ok) {
 			*nongit_ok = 1;
+			strbuf_release(&cwd);
+			strbuf_release(&dir);
 			return NULL;
 		}
 		die(_("Not a git repository (or any parent up to mount point %s)\n"
@@ -1044,6 +1046,10 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	startup_info->have_repository = !nongit_ok || !*nongit_ok;
 	startup_info->prefix = prefix;
 
+	strbuf_release(&cwd);
+	strbuf_release(&dir);
+	strbuf_release(&gitdir);
+
 	return prefix;
 }
 

Re: [PATCH v4 04/10] setup_git_directory_1(): avoid changing global state

[Brandon Williams]

On 03/08, Johannes Schindelin wrote:
> Hi Brandon,
> 
> On Tue, 7 Mar 2017, Brandon Williams wrote:
> 
> > On 03/07, Johannes Schindelin wrote:
> > >  const char *setup_git_directory_gently(int *nongit_ok)
> > >  {
> > > +	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
> > 
> > I couldn't see any strbuf_release() calls for these strbufs so there may
> > be some memory leaking here.
> 
> You are correct, of course. Something like this may work:

Yep that should fix it!

> 
> -- snipsnap --
> diff --git a/setup.c b/setup.c
> index 9118b48590a..c822582b96e 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -1027,6 +1027,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
>  	case GIT_DIR_HIT_MOUNT_POINT:
>  		if (nongit_ok) {
>  			*nongit_ok = 1;
> +			strbuf_release(&cwd);
> +			strbuf_release(&dir);
>  			return NULL;
>  		}
>  		die(_("Not a git repository (or any parent up to mount point %s)\n"
> @@ -1044,6 +1046,10 @@ const char *setup_git_directory_gently(int *nongit_ok)
>  	startup_info->have_repository = !nongit_ok || !*nongit_ok;
>  	startup_info->prefix = prefix;
>  
> +	strbuf_release(&cwd);
> +	strbuf_release(&dir);
> +	strbuf_release(&gitdir);
> +
>  	return prefix;
>  }
>  

-- 
Brandon Williams

[PATCH v4 05/10] Introduce the discover_git_directory() function

[Johannes Schindelin]

We modified the setup_git_directory_gently_1() function earlier to make
it possible to discover the GIT_DIR without changing global state.

However, it is still a bit cumbersome to use if you only need to figure
out the (possibly absolute) path of the .git/ directory. Let's just
provide a convenient wrapper function with an easier signature that
*just* discovers the .git/ directory.

We will use it in a subsequent patch to fix the early config.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h |  7 +++++++
 setup.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/cache.h b/cache.h
index 80b6372cf76..5218726cf88 100644
--- a/cache.h
+++ b/cache.h
@@ -518,6 +518,13 @@ extern void set_git_work_tree(const char *tree);
 #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
 
 extern void setup_work_tree(void);
+/*
+ * Find GIT_DIR of the repository that contains the current working directory,
+ * without changing the working directory or other global state. The result is
+ * appended to gitdir. The return value is either NULL if no repository was
+ * found, or pointing to the path inside gitdir's buffer.
+ */
+extern const char *discover_git_directory(struct strbuf *gitdir);
 extern const char *setup_git_directory_gently(int *);
 extern const char *setup_git_directory(void);
 extern char *prefix_path(const char *prefix, int len, const char *path);
diff --git a/setup.c b/setup.c
index d7af343d14e..486acda2054 100644
--- a/setup.c
+++ b/setup.c
@@ -924,6 +924,49 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 	}
 }
 
+const char *discover_git_directory(struct strbuf *gitdir)
+{
+	struct strbuf dir = STRBUF_INIT, err = STRBUF_INIT;
+	size_t gitdir_offset = gitdir->len, cwd_len;
+	struct repository_format candidate;
+
+	if (strbuf_getcwd(&dir))
+		return NULL;
+
+	cwd_len = dir.len;
+	if (setup_git_directory_gently_1(&dir, gitdir) <= 0) {
+		strbuf_release(&dir);
+		return NULL;
+	}
+
+	/*
+	 * The returned gitdir is relative to dir, and if dir does not reflect
+	 * the current working directory, we simply make the gitdir absolute.
+	 */
+	if (dir.len < cwd_len && !is_absolute_path(gitdir->buf + gitdir_offset)) {
+		/* Avoid a trailing "/." */
+		if (!strcmp(".", gitdir->buf + gitdir_offset))
+			strbuf_setlen(gitdir, gitdir_offset);
+		else
+			strbuf_addch(&dir, '/');
+		strbuf_insert(gitdir, gitdir_offset, dir.buf, dir.len);
+	}
+
+	strbuf_reset(&dir);
+	strbuf_addf(&dir, "%s/config", gitdir->buf + gitdir_offset);
+	read_repository_format(&candidate, dir.buf);
+	strbuf_release(&dir);
+
+	if (verify_repository_format(&candidate, &err) < 0) {
+		warning("ignoring git dir '%s': %s",
+			gitdir->buf + gitdir_offset, err.buf);
+		strbuf_release(&err);
+		return NULL;
+	}
+
+	return gitdir->buf + gitdir_offset;
+}
+
 const char *setup_git_directory_gently(int *nongit_ok)
 {
 	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 06/10] Make read_early_config() reusable

[Johannes Schindelin]

The pager configuration needs to be read early, possibly before
discovering any .git/ directory.

Let's not hide this function in pager.c, but make it available to other
callers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h  |  1 +
 config.c | 31 +++++++++++++++++++++++++++++++
 pager.c  | 31 -------------------------------
 3 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/cache.h b/cache.h
index 5218726cf88..e7b57457e73 100644
--- a/cache.h
+++ b/cache.h
@@ -1804,6 +1804,7 @@ extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
 				     const unsigned char *sha1, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
diff --git a/config.c b/config.c
index c6b874a7bf7..9cfbeafd04c 100644
--- a/config.c
+++ b/config.c
@@ -1412,6 +1412,37 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
+void read_early_config(config_fn_t cb, void *data)
+{
+	git_config_with_options(cb, data, NULL, 1);
+
+	/*
+	 * Note that this is a really dirty hack that does the wrong thing in
+	 * many cases. The crux of the problem is that we cannot run
+	 * setup_git_directory() early on in git's setup, so we have no idea if
+	 * we are in a repository or not, and therefore are not sure whether
+	 * and how to read repository-local config.
+	 *
+	 * So if we _aren't_ in a repository (or we are but we would reject its
+	 * core.repositoryformatversion), we'll read whatever is in .git/config
+	 * blindly. Similarly, if we _are_ in a repository, but not at the
+	 * root, we'll fail to find .git/config (because it's really
+	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 *
+	 * However, we have historically provided this hack because it does
+	 * work some of the time (namely when you are at the top-level of a
+	 * valid repository), and would rarely make things worse (i.e., you do
+	 * not generally have a .git/config file sitting around).
+	 */
+	if (!startup_info->have_repository) {
+		struct git_config_source repo_config;
+
+		memset(&repo_config, 0, sizeof(repo_config));
+		repo_config.file = ".git/config";
+		git_config_with_options(cb, data, &repo_config, 1);
+	}
+}
+
 static void git_config_check_init(void);
 
 void git_config(config_fn_t fn, void *data)
diff --git a/pager.c b/pager.c
index ae796433630..73ca8bc3b17 100644
--- a/pager.c
+++ b/pager.c
@@ -43,37 +43,6 @@ static int core_pager_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
-static void read_early_config(config_fn_t cb, void *data)
-{
-	git_config_with_options(cb, data, NULL, 1);
-
-	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
-	 */
-	if (!startup_info->have_repository) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
-		git_config_with_options(cb, data, &repo_config, 1);
-	}
-}
-
 const char *git_pager(int stdout_is_tty)
 {
 	const char *pager;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 07/10] read_early_config(): avoid .git/config hack when unneeded

[Johannes Schindelin]

So far, we only look whether the startup_info claims to have seen a
git_dir.

However, do_git_config_sequence() (and consequently the
git_config_with_options() call used by read_early_config() asks the
have_git_dir() function whether we have a .git/ directory, which in turn
also looks at git_dir and at the environment variable GIT_DIR. And when
this is the case, the repository config is handled already, so we do not
have to do that again explicitly.

Let's just use the same function, have_git_dir(), to determine whether we
have to handle .git/config explicitly.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.c b/config.c
index 9cfbeafd04c..068fa4dcfa6 100644
--- a/config.c
+++ b/config.c
@@ -1427,14 +1427,15 @@ void read_early_config(config_fn_t cb, void *data)
 	 * core.repositoryformatversion), we'll read whatever is in .git/config
 	 * blindly. Similarly, if we _are_ in a repository, but not at the
 	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 * ../.git/config, etc), unless setup_git_directory() was already called.
+	 * See t7006 for a complete set of failures.
 	 *
 	 * However, we have historically provided this hack because it does
 	 * work some of the time (namely when you are at the top-level of a
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->have_repository) {
+	if (!have_git_dir()) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 08/10] read_early_config(): really discover .git/

[Johannes Schindelin]

Earlier, we punted and simply assumed that we are in the top-level
directory of the project, and that there is no .git file but a .git/
directory so that we can read directly from .git/config.

However, that is not necessarily true. We may be in a subdirectory. Or
.git may be a gitfile. Or the environment variable GIT_DIR may be set.

To remedy this situation, we just refactored the way
setup_git_directory() discovers the .git/ directory, to make it
reusable, and more importantly, to leave all global variables and the
current working directory alone.

Let's discover the .git/ directory correctly in read_early_config() by
using that new function.

This fixes 4 known breakages in t7006.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c         | 31 ++++++++++++-------------------
 t/t7006-pager.sh |  8 ++++----
 2 files changed, 16 insertions(+), 23 deletions(-)

diff --git a/config.c b/config.c
index 068fa4dcfa6..a88df53fdbc 100644
--- a/config.c
+++ b/config.c
@@ -1414,34 +1414,27 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 
 void read_early_config(config_fn_t cb, void *data)
 {
+	struct strbuf buf = STRBUF_INIT;
+
 	git_config_with_options(cb, data, NULL, 1);
 
 	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc), unless setup_git_directory() was already called.
-	 * See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
+	 * When setup_git_directory() was not yet asked to discover the
+	 * GIT_DIR, we ask discover_git_directory() to figure out whether there
+	 * is any repository config we should use (but unlike
+	 * setup_git_directory_gently(), no global state is changed, most
+	 * notably, the current working directory is still the same after the
+	 * call).
 	 */
-	if (!have_git_dir()) {
+	if (!have_git_dir() && discover_git_directory(&buf)) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
+		strbuf_addstr(&buf, "/config");
+		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, 1);
 	}
+	strbuf_release(&buf);
 }
 
 static void git_config_check_init(void);
diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index 304ae06c600..4f3794d415e 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -360,19 +360,19 @@ test_pager_choices                       'git aliasedlog'
 test_default_pager        expect_success 'git -p aliasedlog'
 test_PAGER_overrides      expect_success 'git -p aliasedlog'
 test_core_pager_overrides expect_success 'git -p aliasedlog'
-test_core_pager_subdir    expect_failure 'git -p aliasedlog'
+test_core_pager_subdir    expect_success 'git -p aliasedlog'
 test_GIT_PAGER_overrides  expect_success 'git -p aliasedlog'
 
 test_default_pager        expect_success 'git -p true'
 test_PAGER_overrides      expect_success 'git -p true'
 test_core_pager_overrides expect_success 'git -p true'
-test_core_pager_subdir    expect_failure 'git -p true'
+test_core_pager_subdir    expect_success 'git -p true'
 test_GIT_PAGER_overrides  expect_success 'git -p true'
 
 test_default_pager        expect_success test_must_fail 'git -p request-pull'
 test_PAGER_overrides      expect_success test_must_fail 'git -p request-pull'
 test_core_pager_overrides expect_success test_must_fail 'git -p request-pull'
-test_core_pager_subdir    expect_failure test_must_fail 'git -p request-pull'
+test_core_pager_subdir    expect_success test_must_fail 'git -p request-pull'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 
 test_default_pager        expect_success test_must_fail 'git -p'
@@ -380,7 +380,7 @@ test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
-test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+test_expect_success TTY 'core.pager in repo config works and retains cwd' '
 	sane_unset GIT_PAGER &&
 	test_config core.pager "cat >cwd-retained" &&
 	(
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 09/10] Test read_early_config()

[Johannes Schindelin]

So far, we had no explicit tests of that function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-config.c  | 15 +++++++++++++++
 t/t1309-early-config.sh | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 t/t1309-early-config.sh

diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 83a4f2ab869..8e3ed6a76cb 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -66,6 +66,16 @@ static int iterate_cb(const char *var, const char *value, void *data)
 	return 0;
 }
 
+static int early_config_cb(const char *var, const char *value, void *vdata)
+{
+	const char *key = vdata;
+
+	if (!strcmp(key, var))
+		printf("%s\n", value);
+
+	return 0;
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	int i, val;
@@ -73,6 +83,11 @@ int cmd_main(int argc, const char **argv)
 	const struct string_list *strptr;
 	struct config_set cs;
 
+	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		read_early_config(early_config_cb, (void *)argv[2]);
+		return 0;
+	}
+
 	setup_git_directory();
 
 	git_configset_init(&cs);
diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
new file mode 100755
index 00000000000..0c55dee514c
--- /dev/null
+++ b/t/t1309-early-config.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+test_description='Test read_early_config()'
+
+. ./test-lib.sh
+
+test_expect_success 'read early config' '
+	test_config early.config correct &&
+	test-config read_early_config early.config >output &&
+	test correct = "$(cat output)"
+'
+
+test_expect_success 'in a sub-directory' '
+	test_config early.config sub &&
+	mkdir -p sub &&
+	(
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test sub = "$(cat output)"
+'
+
+test_expect_success 'ceiling' '
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test -z "$(cat output)"
+'
+
+test_expect_success 'ceiling #2' '
+	mkdir -p xdg/git &&
+	git config -f xdg/git/config early.config xdg &&
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		XDG_CONFIG_HOME="$PWD"/xdg &&
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES XDG_CONFIG_HOME &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test xdg = "$(cat output)"
+'
+
+test_done
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v4 10/10] setup_git_directory_gently_1(): avoid die()ing

[Johannes Schindelin]

This function now has a new caller in addition to setup_git_directory():
the newly introduced discover_git_directory(). That function wants to
discover the current .git/ directory, and in case of a corrupted one
simply pretend that there is none to be found.

Example: if a stale .git file exists in the parent directory, and the
user calls `git -p init`, we want Git to simply *not* read any
repository config for the pager (instead of aborting with a message that
the .git file is corrupt).

Let's actually pretend that there was no GIT_DIR to be found in that case
when being called from discover_git_directory(), but keep the previous
behavior (i.e. to die()) for the setup_git_directory() case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/setup.c b/setup.c
index 486acda2054..9118b48590a 100644
--- a/setup.c
+++ b/setup.c
@@ -825,7 +825,8 @@ enum discovery_result {
 	GIT_DIR_BARE,
 	/* these are errors */
 	GIT_DIR_HIT_CEILING = -1,
-	GIT_DIR_HIT_MOUNT_POINT = -2
+	GIT_DIR_HIT_MOUNT_POINT = -2,
+	GIT_DIR_INVALID_GITFILE = -3
 };
 
 /*
@@ -842,7 +843,8 @@ enum discovery_result {
  * `dir` (i.e. *not* necessarily the cwd).
  */
 static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
-							  struct strbuf *gitdir)
+							  struct strbuf *gitdir,
+							  int die_on_error)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
@@ -890,14 +892,22 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 	if (one_filesystem)
 		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		int offset = dir->len;
+		int offset = dir->len, error_code = 0;
 
 		if (offset > min_offset)
 			strbuf_addch(dir, '/');
 		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
-		gitdirenv = read_gitfile(dir->buf);
-		if (!gitdirenv && is_git_directory(dir->buf))
-			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		gitdirenv = read_gitfile_gently(dir->buf, die_on_error ?
+						NULL : &error_code);
+		if (!gitdirenv) {
+			if (die_on_error ||
+			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
+				if (is_git_directory(dir->buf))
+				    gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+			} else if (error_code &&
+				   error_code != READ_GITFILE_ERR_STAT_FAILED)
+				return GIT_DIR_INVALID_GITFILE;
+		}
 		strbuf_setlen(dir, offset);
 		if (gitdirenv) {
 			strbuf_addstr(gitdir, gitdirenv);
@@ -934,7 +944,7 @@ const char *discover_git_directory(struct strbuf *gitdir)
 		return NULL;
 
 	cwd_len = dir.len;
-	if (setup_git_directory_gently_1(&dir, gitdir) <= 0) {
+	if (setup_git_directory_gently_1(&dir, gitdir, 0) <= 0) {
 		strbuf_release(&dir);
 		return NULL;
 	}
@@ -993,7 +1003,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die_errno(_("Unable to read current working directory"));
 	strbuf_addbuf(&dir, &cwd);
 
-	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
+	switch (setup_git_directory_gently_1(&dir, &gitdir, 1)) {
 	case GIT_DIR_NONE:
 		prefix = NULL;
 		break;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 00/11] Fix the early config

[Johannes Schindelin]

These patches are an attempt to make Git's startup sequence a bit less
surprising.

The idea here is to discover the .git/ directory gently (i.e. without
changing the current working directory, nor any global variables), and
to use it to read the .git/config file early, before we actually called
setup_git_directory() (if we ever do that).

This also allows us to fix the early config e.g. to determine the pager
or to resolve aliases in a non-surprising manner.

My own use case: in the GVFS Git fork, we need to execute pre-command
and post-command hooks before and after *every* Git command. A previous
version of the pre-command/post-command hook support was broken, as it
used run_hook() which implicitly called setup_git_directory() too early.
The discover_git_directory() function (and due to core.hooksPath also
the read_early_config() function) helped me fix this.

Notable notes:

- Even if it can cause surprising problems, `init` and `clone` are not
  special-cased. Rationale: it would introduce technical debt and
  violate the Principle Of Least Astonishment.

- The read_early_config() function does not cache Git directory
  discovery nor read values. This is left for another patch series, if
  it ever becomes necessary.

- The alias handling in git.c could possibly benefit from this work, but
  again, this is a separate topic from the current patch series.

Changes since v4:

- plugged memory leaks in setup_git_directory() (thanks, Brandon)

- added some tests that demonstrate how early config currently can fail
  in undesired ways

- fixed indentation (this change was snuck into gitster/git's
  js/early-config branch, and I happened to notice the difference by
  pure random chance)


Johannes Schindelin (11):
  t7006: replace dubious test
  setup_git_directory(): use is_dir_sep() helper
  Prepare setup_discovered_git_directory() the root directory
  setup_git_directory_1(): avoid changing global state
  Introduce the discover_git_directory() function
  Make read_early_config() reusable
  read_early_config(): avoid .git/config hack when unneeded
  read_early_config(): really discover .git/
  Test read_early_config()
  setup_git_directory_gently_1(): avoid die()ing
  t1309: document cases where we would want early config not to die()

 cache.h                 |   8 ++
 config.c                |  25 +++++
 pager.c                 |  31 ------
 setup.c                 | 252 +++++++++++++++++++++++++++++++++---------------
 t/helper/test-config.c  |  15 +++
 t/t1309-early-config.sh |  75 ++++++++++++++
 t/t7006-pager.sh        |  18 +++-
 7 files changed, 313 insertions(+), 111 deletions(-)
 create mode 100755 t/t1309-early-config.sh


base-commit: e0688e9b28f2c5ff711460ee8b62077be5df2360
Published-As: https://github.com/dscho/git/releases/tag/early-config-v5
Fetch-It-Via: git fetch https://github.com/dscho/git early-config-v5

Interdiff vs v4:

 diff --git a/setup.c b/setup.c
 index 9118b48590a..b0a28f609e2 100644
 --- a/setup.c
 +++ b/setup.c
 @@ -903,7 +903,7 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
  			if (die_on_error ||
  			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
  				if (is_git_directory(dir->buf))
 -				    gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 +					gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
  			} else if (error_code &&
  				   error_code != READ_GITFILE_ERR_STAT_FAILED)
  				return GIT_DIR_INVALID_GITFILE;
 @@ -979,7 +979,8 @@ const char *discover_git_directory(struct strbuf *gitdir)
  
  const char *setup_git_directory_gently(int *nongit_ok)
  {
 -	struct strbuf cwd = STRBUF_INIT, dir = STRBUF_INIT, gitdir = STRBUF_INIT;
 +	static struct strbuf cwd = STRBUF_INIT;
 +	struct strbuf dir = STRBUF_INIT, gitdir = STRBUF_INIT;
  	const char *prefix;
  
  	/*
 @@ -1027,6 +1028,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
  	case GIT_DIR_HIT_MOUNT_POINT:
  		if (nongit_ok) {
  			*nongit_ok = 1;
 +			strbuf_release(&cwd);
 +			strbuf_release(&dir);
  			return NULL;
  		}
  		die(_("Not a git repository (or any parent up to mount point %s)\n"
 @@ -1044,6 +1047,9 @@ const char *setup_git_directory_gently(int *nongit_ok)
  	startup_info->have_repository = !nongit_ok || !*nongit_ok;
  	startup_info->prefix = prefix;
  
 +	strbuf_release(&dir);
 +	strbuf_release(&gitdir);
 +
  	return prefix;
  }
  
 diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
 index 0c55dee514c..027eca63a3c 100755
 --- a/t/t1309-early-config.sh
 +++ b/t/t1309-early-config.sh
 @@ -47,4 +47,29 @@ test_expect_success 'ceiling #2' '
  	test xdg = "$(cat output)"
  '
  
 +test_with_config ()
 +{
 +	rm -rf throwaway &&
 +	git init throwaway &&
 +	(
 +		cd throwaway &&
 +		echo "$*" >.git/config &&
 +		test-config read_early_config early.config
 +	)
 +}
 +
 +test_expect_success 'ignore .git/ with incompatible repository version' '
 +	test_with_config "[core]repositoryformatversion = 999999" 2>err &&
 +	grep "warning:.* Expected git repo version <= [1-9]" err
 +'
 +
 +test_expect_failure 'ignore .git/ with invalid repository version' '
 +	test_with_config "[core]repositoryformatversion = invalid"
 +'
 +
 +
 +test_expect_failure 'ignore .git/ with invalid config' '
 +	test_with_config "["
 +'
 +
  test_done

-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 01/11] t7006: replace dubious test

[Johannes Schindelin]

The idea of the test case "git -p - core.pager is not used from
subdirectory" was to verify that the setup_git_directory() function had
not been called just to obtain the core.pager setting.

However, we are about to fix the early config machinery so that it
*does* work, without messing up the global state.

Once that is done, the core.pager setting *will* be used, even when
running from a subdirectory, and that is a Good Thing.

The intention of that test case, however, was to verify that the
setup_git_directory() function has not run, because it changes global
state such as the current working directory.

To keep that spirit, but fix the incorrect assumption, this patch
replaces that test case by a new one that verifies that the pager is
run in the subdirectory, i.e. that the current working directory has
not been changed at the time the pager is configured and launched, even
if the `rev-parse` command requires a .git/ directory and *will* change
the working directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t7006-pager.sh | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index c8dc665f2fd..304ae06c600 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -378,9 +378,19 @@ test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 test_default_pager        expect_success test_must_fail 'git -p'
 test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
-test_no_local_config_subdir expect_success test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
+test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+	sane_unset GIT_PAGER &&
+	test_config core.pager "cat >cwd-retained" &&
+	(
+		cd sub &&
+		rm -f cwd-retained &&
+		test_terminal git -p rev-parse HEAD &&
+		test_path_is_file cwd-retained
+	)
+'
+
 test_doesnt_paginate      expect_failure test_must_fail 'git -p nonsense'
 
 test_pager_choices                       'git shortlog'
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 02/11] setup_git_directory(): use is_dir_sep() helper

[Johannes Schindelin]

It is okay in practice to test for forward slashes in the output of
getcwd(), because we go out of our way to convert backslashes to forward
slashes in getcwd()'s output on Windows.

Still, the correct way to test for a dir separator is by using the
helper function we introduced for that very purpose. It also serves as a
good documentation what the code tries to do (not "how").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index 967f289f1ef..4a15b105676 100644
--- a/setup.c
+++ b/setup.c
@@ -910,7 +910,9 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 			return setup_bare_git_dir(&cwd, offset, nongit_ok);
 
 		offset_parent = offset;
-		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
+		while (--offset_parent > ceil_offset &&
+		       !is_dir_sep(cwd.buf[offset_parent]))
+			; /* continue */
 		if (offset_parent <= ceil_offset)
 			return setup_nongit(cwd.buf, nongit_ok);
 		if (one_filesystem) {
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 03/11] Prepare setup_discovered_git_directory() the root directory

[Johannes Schindelin]

Currently, the offset parameter (indicating what part of the cwd
parameter corresponds to the current directory after discovering the
.git/ directory) is set to 0 when we are running in the root directory.

However, in the next patches we will avoid changing the current working
directory while searching for the .git/ directory, meaning that the
offset corresponding to the root directory will have to be 1 to reflect
that this directory is characterized by the path "/" (and not "").

So let's make sure that setup_discovered_git_directory() only tries to
append the trailing slash to non-root directories.

Note: the setup_bare_git_directory() does not need a corresponding
change, as it does not want to return a prefix.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/setup.c b/setup.c
index 4a15b105676..20a1f0f870e 100644
--- a/setup.c
+++ b/setup.c
@@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 	if (offset == cwd->len)
 		return NULL;
 
-	/* Make "offset" point to past the '/', and add a '/' at the end */
-	offset++;
+	/* Make "offset" point past the '/' (already the case for root dirs) */
+	if (offset != offset_1st_component(cwd->buf))
+		offset++;
+	/* Add a '/' at the end */
 	strbuf_addch(cwd, '/');
 	return cwd->buf + offset;
 }
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 04/11] setup_git_directory_1(): avoid changing global state

[Johannes Schindelin]

For historical reasons, Git searches for the .git/ directory (or the
.git file) by changing the working directory successively to the parent
directory of the current directory, until either anything was found or
until a ceiling or a mount point is hit.

Further global state may be changed in case a .git/ directory was found.

We do have a use case, though, where we would like to find the .git/
directory without having any global state touched, though: when we read
the early config e.g. for the pager or for alias expansion.

Let's just move all of code that changes any global state out of the
function `setup_git_directory_gently_1()` into
`setup_git_directory_gently()`.

In subsequent patches, we will use the _1() function in a new
`discover_git_directory()` function that we will then use for the early
config code.

Note: the new loop is a *little* tricky, as we have to handle the root
directory specially: we cannot simply strip away the last component
including the slash, as the root directory only has that slash. To remedy
that, we introduce the `min_offset` variable that holds the minimal length
of an absolute path, and using that to special-case the root directory,
including an early exit before trying to find the parent of the root
directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 193 +++++++++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 118 insertions(+), 75 deletions(-)

diff --git a/setup.c b/setup.c
index 20a1f0f870e..a7bb09608c0 100644
--- a/setup.c
+++ b/setup.c
@@ -818,50 +818,49 @@ static int canonicalize_ceiling_entry(struct string_list_item *item,
 	}
 }
 
+enum discovery_result {
+	GIT_DIR_NONE = 0,
+	GIT_DIR_EXPLICIT,
+	GIT_DIR_DISCOVERED,
+	GIT_DIR_BARE,
+	/* these are errors */
+	GIT_DIR_HIT_CEILING = -1,
+	GIT_DIR_HIT_MOUNT_POINT = -2
+};
+
 /*
  * We cannot decide in this function whether we are in the work tree or
  * not, since the config can only be read _after_ this function was called.
+ *
+ * Also, we avoid changing any global state (such as the current working
+ * directory) to allow early callers.
+ *
+ * The directory where the search should start needs to be passed in via the
+ * `dir` parameter; upon return, the `dir` buffer will contain the path of
+ * the directory where the search ended, and `gitdir` will contain the path of
+ * the discovered .git/ directory, if any. This path may be relative against
+ * `dir` (i.e. *not* necessarily the cwd).
  */
-static const char *setup_git_directory_gently_1(int *nongit_ok)
+static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
+							  struct strbuf *gitdir)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
-	static struct strbuf cwd = STRBUF_INIT;
-	const char *gitdirenv, *ret;
-	char *gitfile;
-	int offset, offset_parent, ceil_offset = -1;
+	const char *gitdirenv;
+	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
 	dev_t current_device = 0;
 	int one_filesystem = 1;
 
 	/*
-	 * We may have read an incomplete configuration before
-	 * setting-up the git directory. If so, clear the cache so
-	 * that the next queries to the configuration reload complete
-	 * configuration (including the per-repo config file that we
-	 * ignored previously).
-	 */
-	git_config_clear();
-
-	/*
-	 * Let's assume that we are in a git repository.
-	 * If it turns out later that we are somewhere else, the value will be
-	 * updated accordingly.
-	 */
-	if (nongit_ok)
-		*nongit_ok = 0;
-
-	if (strbuf_getcwd(&cwd))
-		die_errno(_("Unable to read current working directory"));
-	offset = cwd.len;
-
-	/*
 	 * If GIT_DIR is set explicitly, we're not going
 	 * to do any discovery, but we still do repository
 	 * validation.
 	 */
 	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
-	if (gitdirenv)
-		return setup_explicit_git_dir(gitdirenv, &cwd, nongit_ok);
+	if (gitdirenv) {
+		strbuf_addstr(gitdir, gitdirenv);
+		return GIT_DIR_EXPLICIT;
+	}
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -869,15 +868,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP, -1);
 		filter_string_list(&ceiling_dirs, 0,
 				   canonicalize_ceiling_entry, &empty_entry_found);
-		ceil_offset = longest_ancestor_length(cwd.buf, &ceiling_dirs);
+		ceil_offset = longest_ancestor_length(dir->buf, &ceiling_dirs);
 		string_list_clear(&ceiling_dirs, 0);
 	}
 
-	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
-		ceil_offset = 1;
+	if (ceil_offset < 0)
+		ceil_offset = min_offset - 2;
 
 	/*
-	 * Test in the following order (relative to the cwd):
+	 * Test in the following order (relative to the dir):
 	 * - .git (file containing "gitdir: <path>")
 	 * - .git/
 	 * - ./ (bare)
@@ -889,63 +888,104 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	 */
 	one_filesystem = !git_env_bool("GIT_DISCOVERY_ACROSS_FILESYSTEM", 0);
 	if (one_filesystem)
-		current_device = get_device_or_die(".", NULL, 0);
+		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
-		if (gitfile)
-			gitdirenv = gitfile = xstrdup(gitfile);
-		else {
-			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
-				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		int offset = dir->len;
+
+		if (offset > min_offset)
+			strbuf_addch(dir, '/');
+		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
+		gitdirenv = read_gitfile(dir->buf);
+		if (!gitdirenv && is_git_directory(dir->buf))
+			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		strbuf_setlen(dir, offset);
+		if (gitdirenv) {
+			strbuf_addstr(gitdir, gitdirenv);
+			return GIT_DIR_DISCOVERED;
 		}
 
-		if (gitdirenv) {
-			ret = setup_discovered_git_dir(gitdirenv,
-						       &cwd, offset,
-						       nongit_ok);
-			free(gitfile);
-			return ret;
+		if (is_git_directory(dir->buf)) {
+			strbuf_addstr(gitdir, ".");
+			return GIT_DIR_BARE;
 		}
-		free(gitfile);
 
-		if (is_git_directory("."))
-			return setup_bare_git_dir(&cwd, offset, nongit_ok);
+		if (offset <= min_offset)
+			return GIT_DIR_HIT_CEILING;
 
-		offset_parent = offset;
-		while (--offset_parent > ceil_offset &&
-		       !is_dir_sep(cwd.buf[offset_parent]))
+		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]))
 			; /* continue */
-		if (offset_parent <= ceil_offset)
-			return setup_nongit(cwd.buf, nongit_ok);
-		if (one_filesystem) {
-			dev_t parent_device = get_device_or_die("..", cwd.buf,
-								offset);
-			if (parent_device != current_device) {
-				if (nongit_ok) {
-					if (chdir(cwd.buf))
-						die_errno(_("Cannot come back to cwd"));
-					*nongit_ok = 1;
-					return NULL;
-				}
-				strbuf_setlen(&cwd, offset);
-				die(_("Not a git repository (or any parent up to mount point %s)\n"
-				"Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
-				    cwd.buf);
-			}
-		}
-		if (chdir("..")) {
-			strbuf_setlen(&cwd, offset);
-			die_errno(_("Cannot change to '%s/..'"), cwd.buf);
-		}
-		offset = offset_parent;
+		if (offset <= ceil_offset)
+			return GIT_DIR_HIT_CEILING;
+
+		strbuf_setlen(dir, offset > min_offset ?  offset : min_offset);
+		if (one_filesystem &&
+		    current_device != get_device_or_die(dir->buf, NULL, offset))
+			return GIT_DIR_HIT_MOUNT_POINT;
 	}
 }
 
 const char *setup_git_directory_gently(int *nongit_ok)
 {
+	static struct strbuf cwd = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT, gitdir = STRBUF_INIT;
 	const char *prefix;
 
-	prefix = setup_git_directory_gently_1(nongit_ok);
+	/*
+	 * We may have read an incomplete configuration before
+	 * setting-up the git directory. If so, clear the cache so
+	 * that the next queries to the configuration reload complete
+	 * configuration (including the per-repo config file that we
+	 * ignored previously).
+	 */
+	git_config_clear();
+
+	/*
+	 * Let's assume that we are in a git repository.
+	 * If it turns out later that we are somewhere else, the value will be
+	 * updated accordingly.
+	 */
+	if (nongit_ok)
+		*nongit_ok = 0;
+
+	if (strbuf_getcwd(&cwd))
+		die_errno(_("Unable to read current working directory"));
+	strbuf_addbuf(&dir, &cwd);
+
+	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
+	case GIT_DIR_NONE:
+		prefix = NULL;
+		break;
+	case GIT_DIR_EXPLICIT:
+		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
+		break;
+	case GIT_DIR_DISCOVERED:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
+						  nongit_ok);
+		break;
+	case GIT_DIR_BARE:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
+		break;
+	case GIT_DIR_HIT_CEILING:
+		prefix = setup_nongit(cwd.buf, nongit_ok);
+		break;
+	case GIT_DIR_HIT_MOUNT_POINT:
+		if (nongit_ok) {
+			*nongit_ok = 1;
+			strbuf_release(&cwd);
+			strbuf_release(&dir);
+			return NULL;
+		}
+		die(_("Not a git repository (or any parent up to mount point %s)\n"
+		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
+		    dir.buf);
+	default:
+		die("BUG: unhandled setup_git_directory_1() result");
+	}
+
 	if (prefix)
 		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
 	else
@@ -954,6 +994,9 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	startup_info->have_repository = !nongit_ok || !*nongit_ok;
 	startup_info->prefix = prefix;
 
+	strbuf_release(&dir);
+	strbuf_release(&gitdir);
+
 	return prefix;
 }
 
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v5 04/11] setup_git_directory_1(): avoid changing global state

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

>  /*
>   * We cannot decide in this function whether we are in the work tree or
>   * not, since the config can only be read _after_ this function was called.
> + *
> + * Also, we avoid changing any global state (such as the current working
> + * directory) to allow early callers.
> + *
> + * The directory where the search should start needs to be passed in via the
> + * `dir` parameter; upon return, the `dir` buffer will contain the path of
> + * the directory where the search ended, and `gitdir` will contain the path of
> + * the discovered .git/ directory, if any. This path may be relative against
> + * `dir` (i.e. *not* necessarily the cwd).

I had to read the last sentence three times because my earlier two
attempts misread/misinterpreted as "this may be relative to cwd, but
not necessarily, because this may be relative to dir".  The correct
reading is "when this is relative, it is relative to dir.  It would
not be relative to cwd if you start with dir that is not cwd".

It would be nicer if all readers can get to that without re-reading
three times.

> -static const char *setup_git_directory_gently_1(int *nongit_ok)
> +static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
> ...
> @@ -889,63 +888,104 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
>  	 */
>  	one_filesystem = !git_env_bool("GIT_DISCOVERY_ACROSS_FILESYSTEM", 0);
>  	if (one_filesystem)
> -		current_device = get_device_or_die(".", NULL, 0);
> +		current_device = get_device_or_die(dir->buf, NULL, 0);
>  	for (;;) {
> -		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
> -		if (gitfile)
> -			gitdirenv = gitfile = xstrdup(gitfile);
> -		else {
> -			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
> -				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
> +		int offset = dir->len;
> +
> +		if (offset > min_offset)
> +			strbuf_addch(dir, '/');
> +		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
> +		gitdirenv = read_gitfile(dir->buf);
> +		if (!gitdirenv && is_git_directory(dir->buf))
> +			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
> +		strbuf_setlen(dir, offset);
> +		if (gitdirenv) {
> +			strbuf_addstr(gitdir, gitdirenv);
> +			return GIT_DIR_DISCOVERED;
>  		}

I commented on this part more extensively in a later step of the
series after the read_gitfile() call is replaced with the non-dying
version.  I see that issues in corner cases are inherited from the
code even before this step.  

We'd either want to at least document the corner cases that are not
handled with /* NEEDSWORK: */ in-code comments , or address them in
a follow-up series before we forget.  They are not new problems, so
I am OK if we choose to leave them broken, as people haven't triggered
them in the current code.

Thanks.

[PATCH v5 05/11] Introduce the discover_git_directory() function

[Johannes Schindelin]

We modified the setup_git_directory_gently_1() function earlier to make
it possible to discover the GIT_DIR without changing global state.

However, it is still a bit cumbersome to use if you only need to figure
out the (possibly absolute) path of the .git/ directory. Let's just
provide a convenient wrapper function with an easier signature that
*just* discovers the .git/ directory.

We will use it in a subsequent patch to fix the early config.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h |  7 +++++++
 setup.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/cache.h b/cache.h
index 80b6372cf76..5218726cf88 100644
--- a/cache.h
+++ b/cache.h
@@ -518,6 +518,13 @@ extern void set_git_work_tree(const char *tree);
 #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
 
 extern void setup_work_tree(void);
+/*
+ * Find GIT_DIR of the repository that contains the current working directory,
+ * without changing the working directory or other global state. The result is
+ * appended to gitdir. The return value is either NULL if no repository was
+ * found, or pointing to the path inside gitdir's buffer.
+ */
+extern const char *discover_git_directory(struct strbuf *gitdir);
 extern const char *setup_git_directory_gently(int *);
 extern const char *setup_git_directory(void);
 extern char *prefix_path(const char *prefix, int len, const char *path);
diff --git a/setup.c b/setup.c
index a7bb09608c0..43f522fa996 100644
--- a/setup.c
+++ b/setup.c
@@ -924,6 +924,49 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 	}
 }
 
+const char *discover_git_directory(struct strbuf *gitdir)
+{
+	struct strbuf dir = STRBUF_INIT, err = STRBUF_INIT;
+	size_t gitdir_offset = gitdir->len, cwd_len;
+	struct repository_format candidate;
+
+	if (strbuf_getcwd(&dir))
+		return NULL;
+
+	cwd_len = dir.len;
+	if (setup_git_directory_gently_1(&dir, gitdir) <= 0) {
+		strbuf_release(&dir);
+		return NULL;
+	}
+
+	/*
+	 * The returned gitdir is relative to dir, and if dir does not reflect
+	 * the current working directory, we simply make the gitdir absolute.
+	 */
+	if (dir.len < cwd_len && !is_absolute_path(gitdir->buf + gitdir_offset)) {
+		/* Avoid a trailing "/." */
+		if (!strcmp(".", gitdir->buf + gitdir_offset))
+			strbuf_setlen(gitdir, gitdir_offset);
+		else
+			strbuf_addch(&dir, '/');
+		strbuf_insert(gitdir, gitdir_offset, dir.buf, dir.len);
+	}
+
+	strbuf_reset(&dir);
+	strbuf_addf(&dir, "%s/config", gitdir->buf + gitdir_offset);
+	read_repository_format(&candidate, dir.buf);
+	strbuf_release(&dir);
+
+	if (verify_repository_format(&candidate, &err) < 0) {
+		warning("ignoring git dir '%s': %s",
+			gitdir->buf + gitdir_offset, err.buf);
+		strbuf_release(&err);
+		return NULL;
+	}
+
+	return gitdir->buf + gitdir_offset;
+}
+
 const char *setup_git_directory_gently(int *nongit_ok)
 {
 	static struct strbuf cwd = STRBUF_INIT;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 06/11] Make read_early_config() reusable

[Johannes Schindelin]

The pager configuration needs to be read early, possibly before
discovering any .git/ directory.

Let's not hide this function in pager.c, but make it available to other
callers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h  |  1 +
 config.c | 31 +++++++++++++++++++++++++++++++
 pager.c  | 31 -------------------------------
 3 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/cache.h b/cache.h
index 5218726cf88..e7b57457e73 100644
--- a/cache.h
+++ b/cache.h
@@ -1804,6 +1804,7 @@ extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
 				     const unsigned char *sha1, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
diff --git a/config.c b/config.c
index c6b874a7bf7..9cfbeafd04c 100644
--- a/config.c
+++ b/config.c
@@ -1412,6 +1412,37 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
+void read_early_config(config_fn_t cb, void *data)
+{
+	git_config_with_options(cb, data, NULL, 1);
+
+	/*
+	 * Note that this is a really dirty hack that does the wrong thing in
+	 * many cases. The crux of the problem is that we cannot run
+	 * setup_git_directory() early on in git's setup, so we have no idea if
+	 * we are in a repository or not, and therefore are not sure whether
+	 * and how to read repository-local config.
+	 *
+	 * So if we _aren't_ in a repository (or we are but we would reject its
+	 * core.repositoryformatversion), we'll read whatever is in .git/config
+	 * blindly. Similarly, if we _are_ in a repository, but not at the
+	 * root, we'll fail to find .git/config (because it's really
+	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 *
+	 * However, we have historically provided this hack because it does
+	 * work some of the time (namely when you are at the top-level of a
+	 * valid repository), and would rarely make things worse (i.e., you do
+	 * not generally have a .git/config file sitting around).
+	 */
+	if (!startup_info->have_repository) {
+		struct git_config_source repo_config;
+
+		memset(&repo_config, 0, sizeof(repo_config));
+		repo_config.file = ".git/config";
+		git_config_with_options(cb, data, &repo_config, 1);
+	}
+}
+
 static void git_config_check_init(void);
 
 void git_config(config_fn_t fn, void *data)
diff --git a/pager.c b/pager.c
index ae796433630..73ca8bc3b17 100644
--- a/pager.c
+++ b/pager.c
@@ -43,37 +43,6 @@ static int core_pager_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
-static void read_early_config(config_fn_t cb, void *data)
-{
-	git_config_with_options(cb, data, NULL, 1);
-
-	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
-	 */
-	if (!startup_info->have_repository) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
-		git_config_with_options(cb, data, &repo_config, 1);
-	}
-}
-
 const char *git_pager(int stdout_is_tty)
 {
 	const char *pager;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 07/11] read_early_config(): avoid .git/config hack when unneeded

[Johannes Schindelin]

So far, we only look whether the startup_info claims to have seen a
git_dir.

However, do_git_config_sequence() (and consequently the
git_config_with_options() call used by read_early_config() asks the
have_git_dir() function whether we have a .git/ directory, which in turn
also looks at git_dir and at the environment variable GIT_DIR. And when
this is the case, the repository config is handled already, so we do not
have to do that again explicitly.

Let's just use the same function, have_git_dir(), to determine whether we
have to handle .git/config explicitly.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.c b/config.c
index 9cfbeafd04c..068fa4dcfa6 100644
--- a/config.c
+++ b/config.c
@@ -1427,14 +1427,15 @@ void read_early_config(config_fn_t cb, void *data)
 	 * core.repositoryformatversion), we'll read whatever is in .git/config
 	 * blindly. Similarly, if we _are_ in a repository, but not at the
 	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 * ../.git/config, etc), unless setup_git_directory() was already called.
+	 * See t7006 for a complete set of failures.
 	 *
 	 * However, we have historically provided this hack because it does
 	 * work some of the time (namely when you are at the top-level of a
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->have_repository) {
+	if (!have_git_dir()) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 08/11] read_early_config(): really discover .git/

[Johannes Schindelin]

Earlier, we punted and simply assumed that we are in the top-level
directory of the project, and that there is no .git file but a .git/
directory so that we can read directly from .git/config.

However, that is not necessarily true. We may be in a subdirectory. Or
.git may be a gitfile. Or the environment variable GIT_DIR may be set.

To remedy this situation, we just refactored the way
setup_git_directory() discovers the .git/ directory, to make it
reusable, and more importantly, to leave all global variables and the
current working directory alone.

Let's discover the .git/ directory correctly in read_early_config() by
using that new function.

This fixes 4 known breakages in t7006.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c         | 31 ++++++++++++-------------------
 t/t7006-pager.sh |  8 ++++----
 2 files changed, 16 insertions(+), 23 deletions(-)

diff --git a/config.c b/config.c
index 068fa4dcfa6..a88df53fdbc 100644
--- a/config.c
+++ b/config.c
@@ -1414,34 +1414,27 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 
 void read_early_config(config_fn_t cb, void *data)
 {
+	struct strbuf buf = STRBUF_INIT;
+
 	git_config_with_options(cb, data, NULL, 1);
 
 	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc), unless setup_git_directory() was already called.
-	 * See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
+	 * When setup_git_directory() was not yet asked to discover the
+	 * GIT_DIR, we ask discover_git_directory() to figure out whether there
+	 * is any repository config we should use (but unlike
+	 * setup_git_directory_gently(), no global state is changed, most
+	 * notably, the current working directory is still the same after the
+	 * call).
 	 */
-	if (!have_git_dir()) {
+	if (!have_git_dir() && discover_git_directory(&buf)) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
+		strbuf_addstr(&buf, "/config");
+		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, 1);
 	}
+	strbuf_release(&buf);
 }
 
 static void git_config_check_init(void);
diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index 304ae06c600..4f3794d415e 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -360,19 +360,19 @@ test_pager_choices                       'git aliasedlog'
 test_default_pager        expect_success 'git -p aliasedlog'
 test_PAGER_overrides      expect_success 'git -p aliasedlog'
 test_core_pager_overrides expect_success 'git -p aliasedlog'
-test_core_pager_subdir    expect_failure 'git -p aliasedlog'
+test_core_pager_subdir    expect_success 'git -p aliasedlog'
 test_GIT_PAGER_overrides  expect_success 'git -p aliasedlog'
 
 test_default_pager        expect_success 'git -p true'
 test_PAGER_overrides      expect_success 'git -p true'
 test_core_pager_overrides expect_success 'git -p true'
-test_core_pager_subdir    expect_failure 'git -p true'
+test_core_pager_subdir    expect_success 'git -p true'
 test_GIT_PAGER_overrides  expect_success 'git -p true'
 
 test_default_pager        expect_success test_must_fail 'git -p request-pull'
 test_PAGER_overrides      expect_success test_must_fail 'git -p request-pull'
 test_core_pager_overrides expect_success test_must_fail 'git -p request-pull'
-test_core_pager_subdir    expect_failure test_must_fail 'git -p request-pull'
+test_core_pager_subdir    expect_success test_must_fail 'git -p request-pull'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 
 test_default_pager        expect_success test_must_fail 'git -p'
@@ -380,7 +380,7 @@ test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
-test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+test_expect_success TTY 'core.pager in repo config works and retains cwd' '
 	sane_unset GIT_PAGER &&
 	test_config core.pager "cat >cwd-retained" &&
 	(
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v5 09/11] Test read_early_config()

[Johannes Schindelin]

So far, we had no explicit tests of that function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-config.c  | 15 +++++++++++++++
 t/t1309-early-config.sh | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 t/t1309-early-config.sh

diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 83a4f2ab869..8e3ed6a76cb 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -66,6 +66,16 @@ static int iterate_cb(const char *var, const char *value, void *data)
 	return 0;
 }
 
+static int early_config_cb(const char *var, const char *value, void *vdata)
+{
+	const char *key = vdata;
+
+	if (!strcmp(key, var))
+		printf("%s\n", value);
+
+	return 0;
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	int i, val;
@@ -73,6 +83,11 @@ int cmd_main(int argc, const char **argv)
 	const struct string_list *strptr;
 	struct config_set cs;
 
+	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		read_early_config(early_config_cb, (void *)argv[2]);
+		return 0;
+	}
+
 	setup_git_directory();
 
 	git_configset_init(&cs);
diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
new file mode 100755
index 00000000000..0c55dee514c
--- /dev/null
+++ b/t/t1309-early-config.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+test_description='Test read_early_config()'
+
+. ./test-lib.sh
+
+test_expect_success 'read early config' '
+	test_config early.config correct &&
+	test-config read_early_config early.config >output &&
+	test correct = "$(cat output)"
+'
+
+test_expect_success 'in a sub-directory' '
+	test_config early.config sub &&
+	mkdir -p sub &&
+	(
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test sub = "$(cat output)"
+'
+
+test_expect_success 'ceiling' '
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test -z "$(cat output)"
+'
+
+test_expect_success 'ceiling #2' '
+	mkdir -p xdg/git &&
+	git config -f xdg/git/config early.config xdg &&
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		XDG_CONFIG_HOME="$PWD"/xdg &&
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES XDG_CONFIG_HOME &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test xdg = "$(cat output)"
+'
+
+test_done
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v5 09/11] Test read_early_config()

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Subject: Re: [PATCH v5 09/11] Test read_early_config()

Let's retitle it to

	t1309: test read_early_config()

> So far, we had no explicit tests of that function.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  t/helper/test-config.c  | 15 +++++++++++++++
>  t/t1309-early-config.sh | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 65 insertions(+)
>  create mode 100755 t/t1309-early-config.sh
>
> diff --git a/t/helper/test-config.c b/t/helper/test-config.c
> index 83a4f2ab869..8e3ed6a76cb 100644
> --- a/t/helper/test-config.c
> +++ b/t/helper/test-config.c
> @@ -66,6 +66,16 @@ static int iterate_cb(const char *var, const char *value, void *data)
>  	return 0;
>  }
>  
> +static int early_config_cb(const char *var, const char *value, void *vdata)
> +{
> +	const char *key = vdata;
> +
> +	if (!strcmp(key, var))
> +		printf("%s\n", value);
> +
> +	return 0;
> +}
> +
>  int cmd_main(int argc, const char **argv)
>  {
>  	int i, val;
> @@ -73,6 +83,11 @@ int cmd_main(int argc, const char **argv)
>  	const struct string_list *strptr;
>  	struct config_set cs;
>  
> +	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
> +		read_early_config(early_config_cb, (void *)argv[2]);
> +		return 0;
> +	}
> +
>  	setup_git_directory();

Makes perfect sense to have this as the very beginning, before we
even do the usual setup ;-)

Re: [PATCH v5 09/11] Test read_early_config()

[Johannes Schindelin]

Hi Junio,

On Fri, 10 Mar 2017, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Subject: Re: [PATCH v5 09/11] Test read_early_config()
> 
> Let's retitle it to
> 
> 	t1309: test read_early_config()

I specifically avoided that, as it would sound as if I modified t1309. But
I *added* that file.

Ciao,
Dscho

Re: [PATCH v5 09/11] Test read_early_config()

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > Subject: Re: [PATCH v5 09/11] Test read_early_config()
>> 
>> Let's retitle it to
>> 
>> 	t1309: test read_early_config()
>
> I specifically avoided that, as it would sound as if I modified t1309. But
> I *added* that file.

Then please stop specifically avoiding that style.

For a reader of the shortlog, a title that identifies the area (and
in t/ scripts, it is common to give the test file name as the
identification) it touches gives useful information, i.e. "Thanks to
this commit, we have a test for read-early-config in t1309 now".
Doing s/in t1309 now/somewhere now/ makes the title less useful.

Thanks.

[PATCH v5 10/11] setup_git_directory_gently_1(): avoid die()ing

[Johannes Schindelin]

This function now has a new caller in addition to setup_git_directory():
the newly introduced discover_git_directory(). That function wants to
discover the current .git/ directory, and in case of a corrupted one
simply pretend that there is none to be found.

Example: if a stale .git file exists in the parent directory, and the
user calls `git -p init`, we want Git to simply *not* read any
repository config for the pager (instead of aborting with a message that
the .git file is corrupt).

Let's actually pretend that there was no GIT_DIR to be found in that case
when being called from discover_git_directory(), but keep the previous
behavior (i.e. to die()) for the setup_git_directory() case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/setup.c b/setup.c
index 43f522fa996..b0a28f609e2 100644
--- a/setup.c
+++ b/setup.c
@@ -825,7 +825,8 @@ enum discovery_result {
 	GIT_DIR_BARE,
 	/* these are errors */
 	GIT_DIR_HIT_CEILING = -1,
-	GIT_DIR_HIT_MOUNT_POINT = -2
+	GIT_DIR_HIT_MOUNT_POINT = -2,
+	GIT_DIR_INVALID_GITFILE = -3
 };
 
 /*
@@ -842,7 +843,8 @@ enum discovery_result {
  * `dir` (i.e. *not* necessarily the cwd).
  */
 static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
-							  struct strbuf *gitdir)
+							  struct strbuf *gitdir,
+							  int die_on_error)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
@@ -890,14 +892,22 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 	if (one_filesystem)
 		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		int offset = dir->len;
+		int offset = dir->len, error_code = 0;
 
 		if (offset > min_offset)
 			strbuf_addch(dir, '/');
 		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
-		gitdirenv = read_gitfile(dir->buf);
-		if (!gitdirenv && is_git_directory(dir->buf))
-			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		gitdirenv = read_gitfile_gently(dir->buf, die_on_error ?
+						NULL : &error_code);
+		if (!gitdirenv) {
+			if (die_on_error ||
+			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
+				if (is_git_directory(dir->buf))
+					gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+			} else if (error_code &&
+				   error_code != READ_GITFILE_ERR_STAT_FAILED)
+				return GIT_DIR_INVALID_GITFILE;
+		}
 		strbuf_setlen(dir, offset);
 		if (gitdirenv) {
 			strbuf_addstr(gitdir, gitdirenv);
@@ -934,7 +944,7 @@ const char *discover_git_directory(struct strbuf *gitdir)
 		return NULL;
 
 	cwd_len = dir.len;
-	if (setup_git_directory_gently_1(&dir, gitdir) <= 0) {
+	if (setup_git_directory_gently_1(&dir, gitdir, 0) <= 0) {
 		strbuf_release(&dir);
 		return NULL;
 	}
@@ -994,7 +1004,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die_errno(_("Unable to read current working directory"));
 	strbuf_addbuf(&dir, &cwd);
 
-	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
+	switch (setup_git_directory_gently_1(&dir, &gitdir, 1)) {
 	case GIT_DIR_NONE:
 		prefix = NULL;
 		break;
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v5 10/11] setup_git_directory_gently_1(): avoid die()ing

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> @@ -890,14 +892,22 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
>  	if (one_filesystem)
>  		current_device = get_device_or_die(dir->buf, NULL, 0);
>  	for (;;) {
> -		int offset = dir->len;
> +		int offset = dir->len, error_code = 0;
>  
>  		if (offset > min_offset)
>  			strbuf_addch(dir, '/');
>  		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
> -		gitdirenv = read_gitfile(dir->buf);
> -		if (!gitdirenv && is_git_directory(dir->buf))
> -			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
> +		gitdirenv = read_gitfile_gently(dir->buf, die_on_error ?
> +						NULL : &error_code);
> +		if (!gitdirenv) {

We tried to read ".git" as a regular file, but couldn't.  There are
some cases but I am having trouble to convince myself all cases are
covered correctly here, so let me follow the code aloud.

> +			if (die_on_error ||
> +			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
> +				if (is_git_directory(dir->buf))
> +					gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;

If we are told to die on error, but if it is a Git directory, then
we do not have to (and want to) die and instead report that
directory as discovered.

If we are to report the failure status instead of dying, we also do
want to recover when the read_gitfile_gentrly() failed only because
it was a real Git directory.  READ_GITFILE_ERR_NOT_A_FILE could be a
signal for that, and we recover after making sure it is a Git
directory.

Among the READ_GITFILE_ERR_* codes, ERR_NOT_A_FILE is the only one
we attempt this recovery.  All others just take the "else if" thing
below.

What happens when is_git_directory() test here does not pass,
though?  Let's say stat() in read_gitfile_gently() found a FIFO
there, it gives us ERR_NOT_A_FILE, is_git_directory() would say
"Nah", and then ...?  Don't we want the other side for this if()
statement that returns failure?

> +			} else if (error_code &&
> +				   error_code != READ_GITFILE_ERR_STAT_FAILED)
> +				return GIT_DIR_INVALID_GITFILE;
> +		}

On the other hand, if read_gitfile_gently() didn't say "we found
something that is not a regular file" we come here.  If the error
came because there wasn't ".git" in the directory we are looking at,
i.e. stat(2) in read_gitfile_gently() gave ENOENT, it is perfectly
normal and we just want to go one level up.

But shouldn't read_gitfile_gently() be inspecting errno when it sees
a stat() failure?  If there _is_ ".git" but we failed to stat it for
whatever reason (EACCES, ELOOP,...), we do not want to go up but
give the INVALID_GITFILE from here, just like other cases, no?
For that I imagine that ERR_STAT_FAILED needs to be split into two,
one for true ERR_STAT_FAILED (from which we cannot recover) and the
other for ERR_ENOENT to signal us that there is nothing there (which
allows us to go up).

By the way, is the "error_code &&" needed?  Unless the original path
given to read_gitfile_gently() is a NULL pointer, the only time its
return value is NULL is when it has non-zero error_code.


>  		strbuf_setlen(dir, offset);
>  		if (gitdirenv) {
>  			strbuf_addstr(gitdir, gitdirenv);
> @@ -934,7 +944,7 @@ const char *discover_git_directory(struct strbuf *gitdir)
>  		return NULL;
>  
>  	cwd_len = dir.len;
> -	if (setup_git_directory_gently_1(&dir, gitdir) <= 0) {
> +	if (setup_git_directory_gently_1(&dir, gitdir, 0) <= 0) {
>  		strbuf_release(&dir);
>  		return NULL;
>  	}
> @@ -994,7 +1004,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
>  		die_errno(_("Unable to read current working directory"));
>  	strbuf_addbuf(&dir, &cwd);
>  
> -	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
> +	switch (setup_git_directory_gently_1(&dir, &gitdir, 1)) {
>  	case GIT_DIR_NONE:
>  		prefix = NULL;
>  		break;

Re: [PATCH v5 10/11] setup_git_directory_gently_1(): avoid die()ing

[Johannes Schindelin]

Hi Junio,

PLEASE NOTE: the purpose of this patch series is to allow the same
function (setup_git_directory_gently_1()) to be the work horse for
setup_git_directory() as before, but also for the new
discover_git_directory() function that is introduced to fix
read_early_config() to avoid hardconfig .git/config.

The purpose is *not* to change any current behavior. Please do not ask me
to do that in this patch series.

Now to your comments:

On Fri, 10 Mar 2017, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -890,14 +892,22 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
> >  	if (one_filesystem)
> >  		current_device = get_device_or_die(dir->buf, NULL, 0);
> >  	for (;;) {
> > -		int offset = dir->len;
> > +		int offset = dir->len, error_code = 0;
> >  
> >  		if (offset > min_offset)
> >  			strbuf_addch(dir, '/');
> >  		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
> > -		gitdirenv = read_gitfile(dir->buf);
> > -		if (!gitdirenv && is_git_directory(dir->buf))
> > -			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
> > +		gitdirenv = read_gitfile_gently(dir->buf, die_on_error ?
> > +						NULL : &error_code);
> > +		if (!gitdirenv) {
> 
> We tried to read ".git" as a regular file, but couldn't.  There are
> some cases but I am having trouble to convince myself all cases are
> covered correctly here, so let me follow the code aloud.

It is subtle and finicky, I agree.

> > +			if (die_on_error ||
> > +			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
> > +				if (is_git_directory(dir->buf))
> > +					gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
> 
> If we are told to die on error, but if it is a Git directory, then
> we do not have to (and want to) die and instead report that
> directory as discovered.
> 
> If we are to report the failure status instead of dying, we also do
> want to recover when the read_gitfile_gentrly() failed only because
> it was a real Git directory.  READ_GITFILE_ERR_NOT_A_FILE could be a
> signal for that, and we recover after making sure it is a Git
> directory.
> 
> Among the READ_GITFILE_ERR_* codes, ERR_NOT_A_FILE is the only one
> we attempt this recovery.  All others just take the "else if" thing
> below.
> 
> What happens when is_git_directory() test here does not pass,
> though?  Let's say stat() in read_gitfile_gently() found a FIFO
> there, it gives us ERR_NOT_A_FILE, is_git_directory() would say
> "Nah", and then ...?  Don't we want the other side for this if()
> statement that returns failure?

The current code as per `master` detects the NOT_A_FILE condition, and as
the parameter return_error_code was passed as NULL (because read_gitfile is
actually #define'd in cache.h to call read_gitfile_gently with NULL as
second parameter), it calls read_gitfile_error_die(). That function
*ignores* the NOT_A_FILE error condition, and as a consequence returns to
read_gitfile_gently() which then returns NULL because there *was* an
error_code.

That means that setup_git_directory_gently_1() will retrieve a NULL from
the read_gitfile() call, which means it then goes on to test whether it
is looking at a .git/ directory via is_git_directory() and if that returns
false, the loop will continue to look at the *parent* directory.

That, in turn, means that what you are asking for is a change in behavior,
and as I am unwilling to introduce a change in behavior with this patch
series, my patch does the exact right thing.

> > +			} else if (error_code &&
> > +				   error_code != READ_GITFILE_ERR_STAT_FAILED)
> > +				return GIT_DIR_INVALID_GITFILE;
> > +		}
> 
> On the other hand, if read_gitfile_gently() didn't say "we found
> something that is not a regular file" we come here.  If the error
> came because there wasn't ".git" in the directory we are looking at,
> i.e. stat(2) in read_gitfile_gently() gave ENOENT, it is perfectly
> normal and we just want to go one level up.
> 
> But shouldn't read_gitfile_gently() be inspecting errno when it sees
> a stat() failure?  If there _is_ ".git" but we failed to stat it for
> whatever reason (EACCES, ELOOP,...), we do not want to go up but
> give the INVALID_GITFILE from here, just like other cases, no?
> For that I imagine that ERR_STAT_FAILED needs to be split into two,
> one for true ERR_STAT_FAILED (from which we cannot recover) and the
> other for ERR_ENOENT to signal us that there is nothing there (which
> allows us to go up).

True. But again, you are asking for a *change in behavior*, which is not
the purpose of this patch series.

> By the way, is the "error_code &&" needed?

It is not! I had the order of the if/else blocks completely differently
originally [*1*] and just overlooked that the error_code could no longer
be 0 in that condition.

Ciao,
Dscho

Footnote *1*: Yes, I wanted to construct an English sentence with three
-ly words in a row.

Re: [PATCH v5 10/11] setup_git_directory_gently_1(): avoid die()ing

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> PLEASE NOTE: the purpose of this patch series is to allow the same
> function (setup_git_directory_gently_1()) to be the work horse for
> setup_git_directory() as before, but also for the new
> discover_git_directory() function that is introduced to fix
> read_early_config() to avoid hardconfig .git/config.
>
> The purpose is *not* to change any current behavior. Please do not ask me
> to do that in this patch series.

Please note that pointing out potential problems is only asking for
confirmation ("yes that is bad") or for enlightenment ("no, you are
reading the code wrong and that bad thing does not happen
because...").  Even in the former case, unless the badness is
introduced by the change, pointing out potential problems is *NOT*
asking to change the patch to fix existing issues.

The former case may split into two.  "Yes I agree that is bad and it
is the same badness as the version without this change", in which
case we may want to leave a "NEEDSWORK" comment in-code so that
people can notice when browsing the code (but fixing the badness
would be outside the scope of the series), and "yes that is bad and
we shouldn't introduce that badness", in which case we do want to
fix it in the patch.

So do not read my (or anybody else's) reviews as _asking_ changes
and making further fixes, and you'll do better.

Re: [PATCH v5 10/11] setup_git_directory_gently_1(): avoid die()ing

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> The former case may split into two.  "Yes I agree that is bad and it
> is the same badness as the version without this change", in which
> case we may want to leave a "NEEDSWORK" comment in-code so that
> people can notice when browsing the code (but fixing the badness
> would be outside the scope of the series), and "yes that is bad and
> we shouldn't introduce that badness", in which case we do want to
> fix it in the patch.

We however need to be a bit careful here, though.

When a patch series is refactoring an existing function to be used
in different codepaths, an existing issue inherited from the
original code (e.g. perhaps an existing error checking that is
looser than ideal) may have been OK in the original context (e.g.
perhaps it died and refused to run until the user corrected the
repository), and it still is OK in the codepath that uses the
refactored building blocks to replace the original code, but it may
not be acceptable to leave the issue in the original code when a new
caller calls the refactored building block (e.g. perhaps the
refactoring made it possible for a caller to ask the function not to
die and instead act on different kinds of errors in different ways).

So "being bug-to-bug compatible with existing code" is not always a
good thing---we need to see case by case if a problem identified in
the review as inherited from the original needs to be addressed in
the series.

It would be better to address issues we identify even if it is an
old one.  After all, any change has potential to introduce new bugs,
and striving to leave known bug inherited from the old code around
would guarantee that the resulting code will be buggier than the
original ;-) 

But in order to keep bisectability, such "further fixups" should be
done as a follow-up to the series, not as part of it.

Re: [PATCH v5 10/11] setup_git_directory_gently_1(): avoid die()ing

[Johannes Schindelin]

Hi Junio,

On Mon, 13 Mar 2017, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > The former case may split into two.  "Yes I agree that is bad and it
> > is the same badness as the version without this change", in which case
> > we may want to leave a "NEEDSWORK" comment in-code so that people can
> > notice when browsing the code (but fixing the badness would be outside
> > the scope of the series), and "yes that is bad and we shouldn't
> > introduce that badness", in which case we do want to fix it in the
> > patch.
> 
> We however need to be a bit careful here, though.
> 
> When a patch series is refactoring an existing function to be used
> in different codepaths, an existing issue inherited from the
> original code (e.g. perhaps an existing error checking that is
> looser than ideal) may have been OK in the original context (e.g.
> perhaps it died and refused to run until the user corrected the
> repository), and it still is OK in the codepath that uses the
> refactored building blocks to replace the original code, but it may
> not be acceptable to leave the issue in the original code when a new
> caller calls the refactored building block (e.g. perhaps the
> refactoring made it possible for a caller to ask the function not to
> die and instead act on different kinds of errors in different ways).

In this case, discover_git_directory() is intended to use the exact same
logic as setup_git_directory() (including bugs and all) so that they do
discover the same directory.

It would not do for discover_git_directory() to detect a *different*
directory, no matter how much "more correct" one would deem it.

Ciao,
Johannes

Re: [PATCH v5 10/11] setup_git_directory_gently_1(): avoid die()ing

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Mon, 13 Mar 2017, Junio C Hamano wrote:
>
>> When a patch series is refactoring an existing function to be used
>> in different codepaths, an existing issue inherited from the
>> original code (e.g. perhaps an existing error checking that is
>> looser than ideal) may have been OK in the original context (e.g.
>> perhaps it died and refused to run until the user corrected the
>> repository), and it still is OK in the codepath that uses the
>> refactored building blocks to replace the original code, but it may
>> not be acceptable to leave the issue in the original code when a new
>> caller calls the refactored building block (e.g. perhaps the
>> refactoring made it possible for a caller to ask the function not to
>> die and instead act on different kinds of errors in different ways).
>
> In this case, ...

It becomes somewhat irritating when you get combative and defensive
unnecessarily.  We already established this case is OK two messages
ago, I think.

The above is only to make sure that people cannot take the "issues
in the original is OK to leave outside the scope of a new series" in
my message out of context and treat it as a general rule to justify
their sloppy patches in the future.  We need to see if issues
inherited from the original is necessary to fix before refactoring
even begins on case-by-case basis, seeing what the requirement of
the new code that uses the refactored code.  And the case-by-case
thing we already did for _this_ case.

[PATCH v5 11/11] t1309: document cases where we would want early config not to die()

[Johannes Schindelin]

Jeff King came up with a couple examples that demonstrate how the new
read_early_config() that looks harder for the current .git/ directory
could die() in an undesirable way.

Let's add those cases to the test script, to document what we would like
to happen when early config encounters problems.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t1309-early-config.sh | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
index 0c55dee514c..027eca63a3c 100755
--- a/t/t1309-early-config.sh
+++ b/t/t1309-early-config.sh
@@ -47,4 +47,29 @@ test_expect_success 'ceiling #2' '
 	test xdg = "$(cat output)"
 '
 
+test_with_config ()
+{
+	rm -rf throwaway &&
+	git init throwaway &&
+	(
+		cd throwaway &&
+		echo "$*" >.git/config &&
+		test-config read_early_config early.config
+	)
+}
+
+test_expect_success 'ignore .git/ with incompatible repository version' '
+	test_with_config "[core]repositoryformatversion = 999999" 2>err &&
+	grep "warning:.* Expected git repo version <= [1-9]" err
+'
+
+test_expect_failure 'ignore .git/ with invalid repository version' '
+	test_with_config "[core]repositoryformatversion = invalid"
+'
+
+
+test_expect_failure 'ignore .git/ with invalid config' '
+	test_with_config "["
+'
+
 test_done
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 00/12] Fix the early config

[Johannes Schindelin]

These patches are an attempt to make Git's startup sequence a bit less
surprising.

The idea here is to refactor setup_git_directory()'s code so that the
underlying .git/ directory discovery can be run without changing any
global state nor the current working directory, and then to use that
functionality also in read_early_config().

My own use case: in the GVFS Git fork, we need to execute pre-command
and post-command hooks before and after *every* Git command. A previous
version of the pre-command/post-command hook support was broken, as it
used run_hook() which implicitly called setup_git_directory() too early.
The discover_git_directory() function (and due to core.hooksPath also
the read_early_config() function) helped me fix this.

Notable notes:

- Even if it can cause surprising problems, `init` and `clone` are not
  special-cased. Rationale: it would introduce technical debt and
  violate the Principle Of Least Astonishment.

- The read_early_config() function does not cache Git directory
  discovery nor read values. This is left for another patch series, if
  it ever becomes necessary.

- The alias handling in git.c could possibly benefit from this work, but
  again, this is a separate topic from the current patch series.

Changes since v5:

- Reworded comment about `gitdir` being relative to `dir` so that it
  does not confuse Junio anymore

- Removed a superfluous `error_code &&` in an if() condition

- Reworded the oneline of 9/11

- Added a new patch at the end that marks the two NEEDSWORK issues that
  Junio identified (but that should not be addressed in this patch
  series, of course, because the entire idea of the early-config work is
  to *preserve* the current behavior of setup_git_directory() while making
  it possible for read_early_config() to reuse the same code paths)


Johannes Schindelin (12):
  t7006: replace dubious test
  setup_git_directory(): use is_dir_sep() helper
  Prepare setup_discovered_git_directory() the root directory
  setup_git_directory_1(): avoid changing global state
  Introduce the discover_git_directory() function
  Make read_early_config() reusable
  read_early_config(): avoid .git/config hack when unneeded
  read_early_config(): really discover .git/
  Add t1309 to test read_early_config()
  setup_git_directory_gently_1(): avoid die()ing
  t1309: document cases where we would want early config not to die()
  setup.c: mention unresolved problems

 cache.h                 |   8 ++
 config.c                |  25 +++++
 pager.c                 |  31 ------
 setup.c                 | 253 +++++++++++++++++++++++++++++++++---------------
 t/helper/test-config.c  |  15 +++
 t/t1309-early-config.sh |  75 ++++++++++++++
 t/t7006-pager.sh        |  18 +++-
 7 files changed, 314 insertions(+), 111 deletions(-)
 create mode 100755 t/t1309-early-config.sh


base-commit: d6db3f216544d05e09159756812ccbcb16861d71
Published-As: https://github.com/dscho/git/releases/tag/early-config-v6
Fetch-It-Via: git fetch https://github.com/dscho/git early-config-v6

Interdiff vs v5:

 diff --git a/setup.c b/setup.c
 index 6733ba5fe82..64f922a9378 100644
 --- a/setup.c
 +++ b/setup.c
 @@ -531,6 +531,7 @@ const char *read_gitfile_gently(const char *path, int *return_error_code)
  	ssize_t len;
  
  	if (stat(path, &st)) {
 +		/* NEEDSWORK: discern between ENOENT vs other errors */
  		error_code = READ_GITFILE_ERR_STAT_FAILED;
  		goto cleanup_return;
  	}
 @@ -839,8 +840,8 @@ enum discovery_result {
   * The directory where the search should start needs to be passed in via the
   * `dir` parameter; upon return, the `dir` buffer will contain the path of
   * the directory where the search ended, and `gitdir` will contain the path of
 - * the discovered .git/ directory, if any. This path may be relative against
 - * `dir` (i.e. *not* necessarily the cwd).
 + * the discovered .git/ directory, if any. If `gitdir` is not absolute, it
 + * is relative to `dir` (i.e. *not* necessarily the cwd).
   */
  static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
  							  struct strbuf *gitdir,
 @@ -902,10 +903,10 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
  		if (!gitdirenv) {
  			if (die_on_error ||
  			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
 +				/* NEEDSWORK: fail if .git is not file nor dir */
  				if (is_git_directory(dir->buf))
  					gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 -			} else if (error_code &&
 -				   error_code != READ_GITFILE_ERR_STAT_FAILED)
 +			} else if (error_code != READ_GITFILE_ERR_STAT_FAILED)
  				return GIT_DIR_INVALID_GITFILE;
  		}
  		strbuf_setlen(dir, offset);

-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 01/12] t7006: replace dubious test

[Johannes Schindelin]

The idea of the test case "git -p - core.pager is not used from
subdirectory" was to verify that the setup_git_directory() function had
not been called just to obtain the core.pager setting.

However, we are about to fix the early config machinery so that it
*does* work, without messing up the global state.

Once that is done, the core.pager setting *will* be used, even when
running from a subdirectory, and that is a Good Thing.

The intention of that test case, however, was to verify that the
setup_git_directory() function has not run, because it changes global
state such as the current working directory.

To keep that spirit, but fix the incorrect assumption, this patch
replaces that test case by a new one that verifies that the pager is
run in the subdirectory, i.e. that the current working directory has
not been changed at the time the pager is configured and launched, even
if the `rev-parse` command requires a .git/ directory and *will* change
the working directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t7006-pager.sh | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index c8dc665f2fd..304ae06c600 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -378,9 +378,19 @@ test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 test_default_pager        expect_success test_must_fail 'git -p'
 test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
-test_no_local_config_subdir expect_success test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
+test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+	sane_unset GIT_PAGER &&
+	test_config core.pager "cat >cwd-retained" &&
+	(
+		cd sub &&
+		rm -f cwd-retained &&
+		test_terminal git -p rev-parse HEAD &&
+		test_path_is_file cwd-retained
+	)
+'
+
 test_doesnt_paginate      expect_failure test_must_fail 'git -p nonsense'
 
 test_pager_choices                       'git shortlog'
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 02/12] setup_git_directory(): use is_dir_sep() helper

[Johannes Schindelin]

It is okay in practice to test for forward slashes in the output of
getcwd(), because we go out of our way to convert backslashes to forward
slashes in getcwd()'s output on Windows.

Still, the correct way to test for a dir separator is by using the
helper function we introduced for that very purpose. It also serves as a
good documentation what the code tries to do (not "how").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index 8f64fbdfb28..2ac891d4b9a 100644
--- a/setup.c
+++ b/setup.c
@@ -910,7 +910,9 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 			return setup_bare_git_dir(&cwd, offset, nongit_ok);
 
 		offset_parent = offset;
-		while (--offset_parent > ceil_offset && cwd.buf[offset_parent] != '/');
+		while (--offset_parent > ceil_offset &&
+		       !is_dir_sep(cwd.buf[offset_parent]))
+			; /* continue */
 		if (offset_parent <= ceil_offset)
 			return setup_nongit(cwd.buf, nongit_ok);
 		if (one_filesystem) {
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 03/12] Prepare setup_discovered_git_directory() the root directory

[Johannes Schindelin]

Currently, the offset parameter (indicating what part of the cwd
parameter corresponds to the current directory after discovering the
.git/ directory) is set to 0 when we are running in the root directory.

However, in the next patches we will avoid changing the current working
directory while searching for the .git/ directory, meaning that the
offset corresponding to the root directory will have to be 1 to reflect
that this directory is characterized by the path "/" (and not "").

So let's make sure that setup_discovered_git_directory() only tries to
append the trailing slash to non-root directories.

Note: the setup_bare_git_directory() does not need a corresponding
change, as it does not want to return a prefix.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/setup.c b/setup.c
index 2ac891d4b9a..23114cb7aa3 100644
--- a/setup.c
+++ b/setup.c
@@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 	if (offset == cwd->len)
 		return NULL;
 
-	/* Make "offset" point to past the '/', and add a '/' at the end */
-	offset++;
+	/* Make "offset" point past the '/' (already the case for root dirs) */
+	if (offset != offset_1st_component(cwd->buf))
+		offset++;
+	/* Add a '/' at the end */
 	strbuf_addch(cwd, '/');
 	return cwd->buf + offset;
 }
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v6 03/12] Prepare setup_discovered_git_directory() the root directory

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Subject: Re: [PATCH v6 03/12] Prepare setup_discovered_git_directory() the root directory

I do not think you've changed this title throughout the rerolls, but
I cannot quite parse it.  Would something like this

    setup.c: only append '/' when not at root in setup_discovered_git_dir()

give enough information to the readers while making it readable?

> Currently, the offset parameter (indicating what part of the cwd
> parameter corresponds to the current directory after discovering the
> .git/ directory) is set to 0 when we are running in the root directory.
>
> However, in the next patches we will avoid changing the current working
> directory while searching for the .git/ directory, meaning that the
> offset corresponding to the root directory will have to be 1 to reflect
> that this directory is characterized by the path "/" (and not "").

OK.  C:\ would be 3 not 1 but the same logic to compare with
offset_1st_component() covers the case, which is good.

> So let's make sure that setup_discovered_git_directory() only tries to
> append the trailing slash to non-root directories.
>
> Note: the setup_bare_git_directory() does not need a corresponding
> change, as it does not want to return a prefix.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  setup.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/setup.c b/setup.c
> index 2ac891d4b9a..23114cb7aa3 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -721,8 +721,10 @@ static const char *setup_discovered_git_dir(const char *gitdir,
>  	if (offset == cwd->len)
>  		return NULL;
>  
> -	/* Make "offset" point to past the '/', and add a '/' at the end */
> -	offset++;
> +	/* Make "offset" point past the '/' (already the case for root dirs) */
> +	if (offset != offset_1st_component(cwd->buf))
> +		offset++;
> +	/* Add a '/' at the end */
>  	strbuf_addch(cwd, '/');
>  	return cwd->buf + offset;
>  }

Re: [PATCH v6 03/12] Prepare setup_discovered_git_directory() the root directory

[Johannes Schindelin]

Hi Junio,

On Mon, 13 Mar 2017, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Subject: Re: [PATCH v6 03/12] Prepare setup_discovered_git_directory() the root directory
> 
> I do not think you've changed this title throughout the rerolls, but
> I cannot quite parse it.

That is because a "for" is missing between "the" and "root".

Ciao,
Dscho

[PATCH v6 04/12] setup_git_directory_1(): avoid changing global state

[Johannes Schindelin]

For historical reasons, Git searches for the .git/ directory (or the
.git file) by changing the working directory successively to the parent
directory of the current directory, until either anything was found or
until a ceiling or a mount point is hit.

Further global state may be changed in case a .git/ directory was found.

We do have a use case, though, where we would like to find the .git/
directory without having any global state touched, though: when we read
the early config e.g. for the pager or for alias expansion.

Let's just move all of code that changes any global state out of the
function `setup_git_directory_gently_1()` into
`setup_git_directory_gently()`.

In subsequent patches, we will use the _1() function in a new
`discover_git_directory()` function that we will then use for the early
config code.

Note: the new loop is a *little* tricky, as we have to handle the root
directory specially: we cannot simply strip away the last component
including the slash, as the root directory only has that slash. To remedy
that, we introduce the `min_offset` variable that holds the minimal length
of an absolute path, and using that to special-case the root directory,
including an early exit before trying to find the parent of the root
directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 193 +++++++++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 118 insertions(+), 75 deletions(-)

diff --git a/setup.c b/setup.c
index 23114cb7aa3..99a722ed5f9 100644
--- a/setup.c
+++ b/setup.c
@@ -818,50 +818,49 @@ static int canonicalize_ceiling_entry(struct string_list_item *item,
 	}
 }
 
+enum discovery_result {
+	GIT_DIR_NONE = 0,
+	GIT_DIR_EXPLICIT,
+	GIT_DIR_DISCOVERED,
+	GIT_DIR_BARE,
+	/* these are errors */
+	GIT_DIR_HIT_CEILING = -1,
+	GIT_DIR_HIT_MOUNT_POINT = -2
+};
+
 /*
  * We cannot decide in this function whether we are in the work tree or
  * not, since the config can only be read _after_ this function was called.
+ *
+ * Also, we avoid changing any global state (such as the current working
+ * directory) to allow early callers.
+ *
+ * The directory where the search should start needs to be passed in via the
+ * `dir` parameter; upon return, the `dir` buffer will contain the path of
+ * the directory where the search ended, and `gitdir` will contain the path of
+ * the discovered .git/ directory, if any. If `gitdir` is not absolute, it
+ * is relative to `dir` (i.e. *not* necessarily the cwd).
  */
-static const char *setup_git_directory_gently_1(int *nongit_ok)
+static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
+							  struct strbuf *gitdir)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
-	static struct strbuf cwd = STRBUF_INIT;
-	const char *gitdirenv, *ret;
-	char *gitfile;
-	int offset, offset_parent, ceil_offset = -1;
+	const char *gitdirenv;
+	int ceil_offset = -1, min_offset = has_dos_drive_prefix(dir->buf) ? 3 : 1;
 	dev_t current_device = 0;
 	int one_filesystem = 1;
 
 	/*
-	 * We may have read an incomplete configuration before
-	 * setting-up the git directory. If so, clear the cache so
-	 * that the next queries to the configuration reload complete
-	 * configuration (including the per-repo config file that we
-	 * ignored previously).
-	 */
-	git_config_clear();
-
-	/*
-	 * Let's assume that we are in a git repository.
-	 * If it turns out later that we are somewhere else, the value will be
-	 * updated accordingly.
-	 */
-	if (nongit_ok)
-		*nongit_ok = 0;
-
-	if (strbuf_getcwd(&cwd))
-		die_errno(_("Unable to read current working directory"));
-	offset = cwd.len;
-
-	/*
 	 * If GIT_DIR is set explicitly, we're not going
 	 * to do any discovery, but we still do repository
 	 * validation.
 	 */
 	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
-	if (gitdirenv)
-		return setup_explicit_git_dir(gitdirenv, &cwd, nongit_ok);
+	if (gitdirenv) {
+		strbuf_addstr(gitdir, gitdirenv);
+		return GIT_DIR_EXPLICIT;
+	}
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -869,15 +868,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 		string_list_split(&ceiling_dirs, env_ceiling_dirs, PATH_SEP, -1);
 		filter_string_list(&ceiling_dirs, 0,
 				   canonicalize_ceiling_entry, &empty_entry_found);
-		ceil_offset = longest_ancestor_length(cwd.buf, &ceiling_dirs);
+		ceil_offset = longest_ancestor_length(dir->buf, &ceiling_dirs);
 		string_list_clear(&ceiling_dirs, 0);
 	}
 
-	if (ceil_offset < 0 && has_dos_drive_prefix(cwd.buf))
-		ceil_offset = 1;
+	if (ceil_offset < 0)
+		ceil_offset = min_offset - 2;
 
 	/*
-	 * Test in the following order (relative to the cwd):
+	 * Test in the following order (relative to the dir):
 	 * - .git (file containing "gitdir: <path>")
 	 * - .git/
 	 * - ./ (bare)
@@ -889,63 +888,104 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	 */
 	one_filesystem = !git_env_bool("GIT_DISCOVERY_ACROSS_FILESYSTEM", 0);
 	if (one_filesystem)
-		current_device = get_device_or_die(".", NULL, 0);
+		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
-		if (gitfile)
-			gitdirenv = gitfile = xstrdup(gitfile);
-		else {
-			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
-				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		int offset = dir->len;
+
+		if (offset > min_offset)
+			strbuf_addch(dir, '/');
+		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
+		gitdirenv = read_gitfile(dir->buf);
+		if (!gitdirenv && is_git_directory(dir->buf))
+			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		strbuf_setlen(dir, offset);
+		if (gitdirenv) {
+			strbuf_addstr(gitdir, gitdirenv);
+			return GIT_DIR_DISCOVERED;
 		}
 
-		if (gitdirenv) {
-			ret = setup_discovered_git_dir(gitdirenv,
-						       &cwd, offset,
-						       nongit_ok);
-			free(gitfile);
-			return ret;
+		if (is_git_directory(dir->buf)) {
+			strbuf_addstr(gitdir, ".");
+			return GIT_DIR_BARE;
 		}
-		free(gitfile);
 
-		if (is_git_directory("."))
-			return setup_bare_git_dir(&cwd, offset, nongit_ok);
+		if (offset <= min_offset)
+			return GIT_DIR_HIT_CEILING;
 
-		offset_parent = offset;
-		while (--offset_parent > ceil_offset &&
-		       !is_dir_sep(cwd.buf[offset_parent]))
+		while (--offset > ceil_offset && !is_dir_sep(dir->buf[offset]))
 			; /* continue */
-		if (offset_parent <= ceil_offset)
-			return setup_nongit(cwd.buf, nongit_ok);
-		if (one_filesystem) {
-			dev_t parent_device = get_device_or_die("..", cwd.buf,
-								offset);
-			if (parent_device != current_device) {
-				if (nongit_ok) {
-					if (chdir(cwd.buf))
-						die_errno(_("Cannot come back to cwd"));
-					*nongit_ok = 1;
-					return NULL;
-				}
-				strbuf_setlen(&cwd, offset);
-				die(_("Not a git repository (or any parent up to mount point %s)\n"
-				"Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
-				    cwd.buf);
-			}
-		}
-		if (chdir("..")) {
-			strbuf_setlen(&cwd, offset);
-			die_errno(_("Cannot change to '%s/..'"), cwd.buf);
-		}
-		offset = offset_parent;
+		if (offset <= ceil_offset)
+			return GIT_DIR_HIT_CEILING;
+
+		strbuf_setlen(dir, offset > min_offset ?  offset : min_offset);
+		if (one_filesystem &&
+		    current_device != get_device_or_die(dir->buf, NULL, offset))
+			return GIT_DIR_HIT_MOUNT_POINT;
 	}
 }
 
 const char *setup_git_directory_gently(int *nongit_ok)
 {
+	static struct strbuf cwd = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT, gitdir = STRBUF_INIT;
 	const char *prefix;
 
-	prefix = setup_git_directory_gently_1(nongit_ok);
+	/*
+	 * We may have read an incomplete configuration before
+	 * setting-up the git directory. If so, clear the cache so
+	 * that the next queries to the configuration reload complete
+	 * configuration (including the per-repo config file that we
+	 * ignored previously).
+	 */
+	git_config_clear();
+
+	/*
+	 * Let's assume that we are in a git repository.
+	 * If it turns out later that we are somewhere else, the value will be
+	 * updated accordingly.
+	 */
+	if (nongit_ok)
+		*nongit_ok = 0;
+
+	if (strbuf_getcwd(&cwd))
+		die_errno(_("Unable to read current working directory"));
+	strbuf_addbuf(&dir, &cwd);
+
+	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
+	case GIT_DIR_NONE:
+		prefix = NULL;
+		break;
+	case GIT_DIR_EXPLICIT:
+		prefix = setup_explicit_git_dir(gitdir.buf, &cwd, nongit_ok);
+		break;
+	case GIT_DIR_DISCOVERED:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_discovered_git_dir(gitdir.buf, &cwd, dir.len,
+						  nongit_ok);
+		break;
+	case GIT_DIR_BARE:
+		if (dir.len < cwd.len && chdir(dir.buf))
+			die(_("Cannot change to '%s'"), dir.buf);
+		prefix = setup_bare_git_dir(&cwd, dir.len, nongit_ok);
+		break;
+	case GIT_DIR_HIT_CEILING:
+		prefix = setup_nongit(cwd.buf, nongit_ok);
+		break;
+	case GIT_DIR_HIT_MOUNT_POINT:
+		if (nongit_ok) {
+			*nongit_ok = 1;
+			strbuf_release(&cwd);
+			strbuf_release(&dir);
+			return NULL;
+		}
+		die(_("Not a git repository (or any parent up to mount point %s)\n"
+		      "Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set)."),
+		    dir.buf);
+	default:
+		die("BUG: unhandled setup_git_directory_1() result");
+	}
+
 	if (prefix)
 		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
 	else
@@ -954,6 +994,9 @@ const char *setup_git_directory_gently(int *nongit_ok)
 	startup_info->have_repository = !nongit_ok || !*nongit_ok;
 	startup_info->prefix = prefix;
 
+	strbuf_release(&dir);
+	strbuf_release(&gitdir);
+
 	return prefix;
 }
 
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 05/12] Introduce the discover_git_directory() function

[Johannes Schindelin]

We modified the setup_git_directory_gently_1() function earlier to make
it possible to discover the GIT_DIR without changing global state.

However, it is still a bit cumbersome to use if you only need to figure
out the (possibly absolute) path of the .git/ directory. Let's just
provide a convenient wrapper function with an easier signature that
*just* discovers the .git/ directory.

We will use it in a subsequent patch to fix the early config.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h |  7 +++++++
 setup.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/cache.h b/cache.h
index 8c0e6442076..086bd9fa433 100644
--- a/cache.h
+++ b/cache.h
@@ -518,6 +518,13 @@ extern void set_git_work_tree(const char *tree);
 #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES"
 
 extern void setup_work_tree(void);
+/*
+ * Find GIT_DIR of the repository that contains the current working directory,
+ * without changing the working directory or other global state. The result is
+ * appended to gitdir. The return value is either NULL if no repository was
+ * found, or pointing to the path inside gitdir's buffer.
+ */
+extern const char *discover_git_directory(struct strbuf *gitdir);
 extern const char *setup_git_directory_gently(int *);
 extern const char *setup_git_directory(void);
 extern char *prefix_path(const char *prefix, int len, const char *path);
diff --git a/setup.c b/setup.c
index 99a722ed5f9..411e8342972 100644
--- a/setup.c
+++ b/setup.c
@@ -924,6 +924,49 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 	}
 }
 
+const char *discover_git_directory(struct strbuf *gitdir)
+{
+	struct strbuf dir = STRBUF_INIT, err = STRBUF_INIT;
+	size_t gitdir_offset = gitdir->len, cwd_len;
+	struct repository_format candidate;
+
+	if (strbuf_getcwd(&dir))
+		return NULL;
+
+	cwd_len = dir.len;
+	if (setup_git_directory_gently_1(&dir, gitdir) <= 0) {
+		strbuf_release(&dir);
+		return NULL;
+	}
+
+	/*
+	 * The returned gitdir is relative to dir, and if dir does not reflect
+	 * the current working directory, we simply make the gitdir absolute.
+	 */
+	if (dir.len < cwd_len && !is_absolute_path(gitdir->buf + gitdir_offset)) {
+		/* Avoid a trailing "/." */
+		if (!strcmp(".", gitdir->buf + gitdir_offset))
+			strbuf_setlen(gitdir, gitdir_offset);
+		else
+			strbuf_addch(&dir, '/');
+		strbuf_insert(gitdir, gitdir_offset, dir.buf, dir.len);
+	}
+
+	strbuf_reset(&dir);
+	strbuf_addf(&dir, "%s/config", gitdir->buf + gitdir_offset);
+	read_repository_format(&candidate, dir.buf);
+	strbuf_release(&dir);
+
+	if (verify_repository_format(&candidate, &err) < 0) {
+		warning("ignoring git dir '%s': %s",
+			gitdir->buf + gitdir_offset, err.buf);
+		strbuf_release(&err);
+		return NULL;
+	}
+
+	return gitdir->buf + gitdir_offset;
+}
+
 const char *setup_git_directory_gently(int *nongit_ok)
 {
 	static struct strbuf cwd = STRBUF_INIT;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 06/12] Make read_early_config() reusable

[Johannes Schindelin]

The pager configuration needs to be read early, possibly before
discovering any .git/ directory.

Let's not hide this function in pager.c, but make it available to other
callers.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 cache.h  |  1 +
 config.c | 31 +++++++++++++++++++++++++++++++
 pager.c  | 31 -------------------------------
 3 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/cache.h b/cache.h
index 086bd9fa433..973cc23d6d4 100644
--- a/cache.h
+++ b/cache.h
@@ -1801,6 +1801,7 @@ extern int git_config_from_blob_sha1(config_fn_t fn, const char *name,
 				     const unsigned char *sha1, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
+extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
diff --git a/config.c b/config.c
index 0e9e1ebefc0..8808b132761 100644
--- a/config.c
+++ b/config.c
@@ -1503,6 +1503,37 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 	}
 }
 
+void read_early_config(config_fn_t cb, void *data)
+{
+	git_config_with_options(cb, data, NULL, 1);
+
+	/*
+	 * Note that this is a really dirty hack that does the wrong thing in
+	 * many cases. The crux of the problem is that we cannot run
+	 * setup_git_directory() early on in git's setup, so we have no idea if
+	 * we are in a repository or not, and therefore are not sure whether
+	 * and how to read repository-local config.
+	 *
+	 * So if we _aren't_ in a repository (or we are but we would reject its
+	 * core.repositoryformatversion), we'll read whatever is in .git/config
+	 * blindly. Similarly, if we _are_ in a repository, but not at the
+	 * root, we'll fail to find .git/config (because it's really
+	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 *
+	 * However, we have historically provided this hack because it does
+	 * work some of the time (namely when you are at the top-level of a
+	 * valid repository), and would rarely make things worse (i.e., you do
+	 * not generally have a .git/config file sitting around).
+	 */
+	if (!startup_info->have_repository) {
+		struct git_config_source repo_config;
+
+		memset(&repo_config, 0, sizeof(repo_config));
+		repo_config.file = ".git/config";
+		git_config_with_options(cb, data, &repo_config, 1);
+	}
+}
+
 static void git_config_check_init(void);
 
 void git_config(config_fn_t fn, void *data)
diff --git a/pager.c b/pager.c
index ae796433630..73ca8bc3b17 100644
--- a/pager.c
+++ b/pager.c
@@ -43,37 +43,6 @@ static int core_pager_config(const char *var, const char *value, void *data)
 	return 0;
 }
 
-static void read_early_config(config_fn_t cb, void *data)
-{
-	git_config_with_options(cb, data, NULL, 1);
-
-	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
-	 */
-	if (!startup_info->have_repository) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
-		git_config_with_options(cb, data, &repo_config, 1);
-	}
-}
-
 const char *git_pager(int stdout_is_tty)
 {
 	const char *pager;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 07/12] read_early_config(): avoid .git/config hack when unneeded

[Johannes Schindelin]

So far, we only look whether the startup_info claims to have seen a
git_dir.

However, do_git_config_sequence() (and consequently the
git_config_with_options() call used by read_early_config() asks the
have_git_dir() function whether we have a .git/ directory, which in turn
also looks at git_dir and at the environment variable GIT_DIR. And when
this is the case, the repository config is handled already, so we do not
have to do that again explicitly.

Let's just use the same function, have_git_dir(), to determine whether we
have to handle .git/config explicitly.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.c b/config.c
index 8808b132761..fc4625a71dd 100644
--- a/config.c
+++ b/config.c
@@ -1518,14 +1518,15 @@ void read_early_config(config_fn_t cb, void *data)
 	 * core.repositoryformatversion), we'll read whatever is in .git/config
 	 * blindly. Similarly, if we _are_ in a repository, but not at the
 	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc). See t7006 for a complete set of failures.
+	 * ../.git/config, etc), unless setup_git_directory() was already called.
+	 * See t7006 for a complete set of failures.
 	 *
 	 * However, we have historically provided this hack because it does
 	 * work some of the time (namely when you are at the top-level of a
 	 * valid repository), and would rarely make things worse (i.e., you do
 	 * not generally have a .git/config file sitting around).
 	 */
-	if (!startup_info->have_repository) {
+	if (!have_git_dir()) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 08/12] read_early_config(): really discover .git/

[Johannes Schindelin]

Earlier, we punted and simply assumed that we are in the top-level
directory of the project, and that there is no .git file but a .git/
directory so that we can read directly from .git/config.

However, that is not necessarily true. We may be in a subdirectory. Or
.git may be a gitfile. Or the environment variable GIT_DIR may be set.

To remedy this situation, we just refactored the way
setup_git_directory() discovers the .git/ directory, to make it
reusable, and more importantly, to leave all global variables and the
current working directory alone.

Let's discover the .git/ directory correctly in read_early_config() by
using that new function.

This fixes 4 known breakages in t7006.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 config.c         | 31 ++++++++++++-------------------
 t/t7006-pager.sh |  8 ++++----
 2 files changed, 16 insertions(+), 23 deletions(-)

diff --git a/config.c b/config.c
index fc4625a71dd..030acc50aa2 100644
--- a/config.c
+++ b/config.c
@@ -1505,34 +1505,27 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 
 void read_early_config(config_fn_t cb, void *data)
 {
+	struct strbuf buf = STRBUF_INIT;
+
 	git_config_with_options(cb, data, NULL, 1);
 
 	/*
-	 * Note that this is a really dirty hack that does the wrong thing in
-	 * many cases. The crux of the problem is that we cannot run
-	 * setup_git_directory() early on in git's setup, so we have no idea if
-	 * we are in a repository or not, and therefore are not sure whether
-	 * and how to read repository-local config.
-	 *
-	 * So if we _aren't_ in a repository (or we are but we would reject its
-	 * core.repositoryformatversion), we'll read whatever is in .git/config
-	 * blindly. Similarly, if we _are_ in a repository, but not at the
-	 * root, we'll fail to find .git/config (because it's really
-	 * ../.git/config, etc), unless setup_git_directory() was already called.
-	 * See t7006 for a complete set of failures.
-	 *
-	 * However, we have historically provided this hack because it does
-	 * work some of the time (namely when you are at the top-level of a
-	 * valid repository), and would rarely make things worse (i.e., you do
-	 * not generally have a .git/config file sitting around).
+	 * When setup_git_directory() was not yet asked to discover the
+	 * GIT_DIR, we ask discover_git_directory() to figure out whether there
+	 * is any repository config we should use (but unlike
+	 * setup_git_directory_gently(), no global state is changed, most
+	 * notably, the current working directory is still the same after the
+	 * call).
 	 */
-	if (!have_git_dir()) {
+	if (!have_git_dir() && discover_git_directory(&buf)) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		repo_config.file = ".git/config";
+		strbuf_addstr(&buf, "/config");
+		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, 1);
 	}
+	strbuf_release(&buf);
 }
 
 static void git_config_check_init(void);
diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index 304ae06c600..4f3794d415e 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -360,19 +360,19 @@ test_pager_choices                       'git aliasedlog'
 test_default_pager        expect_success 'git -p aliasedlog'
 test_PAGER_overrides      expect_success 'git -p aliasedlog'
 test_core_pager_overrides expect_success 'git -p aliasedlog'
-test_core_pager_subdir    expect_failure 'git -p aliasedlog'
+test_core_pager_subdir    expect_success 'git -p aliasedlog'
 test_GIT_PAGER_overrides  expect_success 'git -p aliasedlog'
 
 test_default_pager        expect_success 'git -p true'
 test_PAGER_overrides      expect_success 'git -p true'
 test_core_pager_overrides expect_success 'git -p true'
-test_core_pager_subdir    expect_failure 'git -p true'
+test_core_pager_subdir    expect_success 'git -p true'
 test_GIT_PAGER_overrides  expect_success 'git -p true'
 
 test_default_pager        expect_success test_must_fail 'git -p request-pull'
 test_PAGER_overrides      expect_success test_must_fail 'git -p request-pull'
 test_core_pager_overrides expect_success test_must_fail 'git -p request-pull'
-test_core_pager_subdir    expect_failure test_must_fail 'git -p request-pull'
+test_core_pager_subdir    expect_success test_must_fail 'git -p request-pull'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p request-pull'
 
 test_default_pager        expect_success test_must_fail 'git -p'
@@ -380,7 +380,7 @@ test_PAGER_overrides      expect_success test_must_fail 'git -p'
 test_local_config_ignored expect_failure test_must_fail 'git -p'
 test_GIT_PAGER_overrides  expect_success test_must_fail 'git -p'
 
-test_expect_failure TTY 'core.pager in repo config works and retains cwd' '
+test_expect_success TTY 'core.pager in repo config works and retains cwd' '
 	sane_unset GIT_PAGER &&
 	test_config core.pager "cat >cwd-retained" &&
 	(
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 09/12] Add t1309 to test read_early_config()

[Johannes Schindelin]

So far, we had no explicit tests of that function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-config.c  | 15 +++++++++++++++
 t/t1309-early-config.sh | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 t/t1309-early-config.sh

diff --git a/t/helper/test-config.c b/t/helper/test-config.c
index 83a4f2ab869..8e3ed6a76cb 100644
--- a/t/helper/test-config.c
+++ b/t/helper/test-config.c
@@ -66,6 +66,16 @@ static int iterate_cb(const char *var, const char *value, void *data)
 	return 0;
 }
 
+static int early_config_cb(const char *var, const char *value, void *vdata)
+{
+	const char *key = vdata;
+
+	if (!strcmp(key, var))
+		printf("%s\n", value);
+
+	return 0;
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	int i, val;
@@ -73,6 +83,11 @@ int cmd_main(int argc, const char **argv)
 	const struct string_list *strptr;
 	struct config_set cs;
 
+	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		read_early_config(early_config_cb, (void *)argv[2]);
+		return 0;
+	}
+
 	setup_git_directory();
 
 	git_configset_init(&cs);
diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
new file mode 100755
index 00000000000..0c55dee514c
--- /dev/null
+++ b/t/t1309-early-config.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+test_description='Test read_early_config()'
+
+. ./test-lib.sh
+
+test_expect_success 'read early config' '
+	test_config early.config correct &&
+	test-config read_early_config early.config >output &&
+	test correct = "$(cat output)"
+'
+
+test_expect_success 'in a sub-directory' '
+	test_config early.config sub &&
+	mkdir -p sub &&
+	(
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test sub = "$(cat output)"
+'
+
+test_expect_success 'ceiling' '
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test -z "$(cat output)"
+'
+
+test_expect_success 'ceiling #2' '
+	mkdir -p xdg/git &&
+	git config -f xdg/git/config early.config xdg &&
+	test_config early.config ceiling &&
+	mkdir -p sub &&
+	(
+		XDG_CONFIG_HOME="$PWD"/xdg &&
+		GIT_CEILING_DIRECTORIES="$PWD" &&
+		export GIT_CEILING_DIRECTORIES XDG_CONFIG_HOME &&
+		cd sub &&
+		test-config read_early_config early.config
+	) >output &&
+	test xdg = "$(cat output)"
+'
+
+test_done
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 10/12] setup_git_directory_gently_1(): avoid die()ing

[Johannes Schindelin]

This function now has a new caller in addition to setup_git_directory():
the newly introduced discover_git_directory(). That function wants to
discover the current .git/ directory, and in case of a corrupted one
simply pretend that there is none to be found.

Example: if a stale .git file exists in the parent directory, and the
user calls `git -p init`, we want Git to simply *not* read any
repository config for the pager (instead of aborting with a message that
the .git file is corrupt).

Let's actually pretend that there was no GIT_DIR to be found in that case
when being called from discover_git_directory(), but keep the previous
behavior (i.e. to die()) for the setup_git_directory() case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/setup.c b/setup.c
index 411e8342972..f31abf8a990 100644
--- a/setup.c
+++ b/setup.c
@@ -825,7 +825,8 @@ enum discovery_result {
 	GIT_DIR_BARE,
 	/* these are errors */
 	GIT_DIR_HIT_CEILING = -1,
-	GIT_DIR_HIT_MOUNT_POINT = -2
+	GIT_DIR_HIT_MOUNT_POINT = -2,
+	GIT_DIR_INVALID_GITFILE = -3
 };
 
 /*
@@ -842,7 +843,8 @@ enum discovery_result {
  * is relative to `dir` (i.e. *not* necessarily the cwd).
  */
 static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
-							  struct strbuf *gitdir)
+							  struct strbuf *gitdir,
+							  int die_on_error)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
@@ -890,14 +892,21 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 	if (one_filesystem)
 		current_device = get_device_or_die(dir->buf, NULL, 0);
 	for (;;) {
-		int offset = dir->len;
+		int offset = dir->len, error_code = 0;
 
 		if (offset > min_offset)
 			strbuf_addch(dir, '/');
 		strbuf_addstr(dir, DEFAULT_GIT_DIR_ENVIRONMENT);
-		gitdirenv = read_gitfile(dir->buf);
-		if (!gitdirenv && is_git_directory(dir->buf))
-			gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+		gitdirenv = read_gitfile_gently(dir->buf, die_on_error ?
+						NULL : &error_code);
+		if (!gitdirenv) {
+			if (die_on_error ||
+			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
+				if (is_git_directory(dir->buf))
+					gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+			} else if (error_code != READ_GITFILE_ERR_STAT_FAILED)
+				return GIT_DIR_INVALID_GITFILE;
+		}
 		strbuf_setlen(dir, offset);
 		if (gitdirenv) {
 			strbuf_addstr(gitdir, gitdirenv);
@@ -934,7 +943,7 @@ const char *discover_git_directory(struct strbuf *gitdir)
 		return NULL;
 
 	cwd_len = dir.len;
-	if (setup_git_directory_gently_1(&dir, gitdir) <= 0) {
+	if (setup_git_directory_gently_1(&dir, gitdir, 0) <= 0) {
 		strbuf_release(&dir);
 		return NULL;
 	}
@@ -994,7 +1003,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		die_errno(_("Unable to read current working directory"));
 	strbuf_addbuf(&dir, &cwd);
 
-	switch (setup_git_directory_gently_1(&dir, &gitdir)) {
+	switch (setup_git_directory_gently_1(&dir, &gitdir, 1)) {
 	case GIT_DIR_NONE:
 		prefix = NULL;
 		break;
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 11/12] t1309: document cases where we would want early config not to die()

[Johannes Schindelin]

Jeff King came up with a couple examples that demonstrate how the new
read_early_config() that looks harder for the current .git/ directory
could die() in an undesirable way.

Let's add those cases to the test script, to document what we would like
to happen when early config encounters problems.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t1309-early-config.sh | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/t/t1309-early-config.sh b/t/t1309-early-config.sh
index 0c55dee514c..027eca63a3c 100755
--- a/t/t1309-early-config.sh
+++ b/t/t1309-early-config.sh
@@ -47,4 +47,29 @@ test_expect_success 'ceiling #2' '
 	test xdg = "$(cat output)"
 '
 
+test_with_config ()
+{
+	rm -rf throwaway &&
+	git init throwaway &&
+	(
+		cd throwaway &&
+		echo "$*" >.git/config &&
+		test-config read_early_config early.config
+	)
+}
+
+test_expect_success 'ignore .git/ with incompatible repository version' '
+	test_with_config "[core]repositoryformatversion = 999999" 2>err &&
+	grep "warning:.* Expected git repo version <= [1-9]" err
+'
+
+test_expect_failure 'ignore .git/ with invalid repository version' '
+	test_with_config "[core]repositoryformatversion = invalid"
+'
+
+
+test_expect_failure 'ignore .git/ with invalid config' '
+	test_with_config "["
+'
+
 test_done
-- 
2.12.0.windows.1.7.g94dafc3b124

[PATCH v6 12/12] setup.c: mention unresolved problems

[Johannes Schindelin]

During the review of the `early-config` patch series, two issues have
been identified that have been with us forever.

The idea of that patch series was to fix the hard-coded (and sometimes
wrong) .git/config path when looking for the pager configurations. To
that end, the patches refactor the helper functions behind the
functionality of setup_git_directory(), to make it reusable without
changing any global state. Not to change said functionality.

So let's just mark the identified problems for later so that we do not
forget them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 setup.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/setup.c b/setup.c
index f31abf8a990..64f922a9378 100644
--- a/setup.c
+++ b/setup.c
@@ -531,6 +531,7 @@ const char *read_gitfile_gently(const char *path, int *return_error_code)
 	ssize_t len;
 
 	if (stat(path, &st)) {
+		/* NEEDSWORK: discern between ENOENT vs other errors */
 		error_code = READ_GITFILE_ERR_STAT_FAILED;
 		goto cleanup_return;
 	}
@@ -902,6 +903,7 @@ static enum discovery_result setup_git_directory_gently_1(struct strbuf *dir,
 		if (!gitdirenv) {
 			if (die_on_error ||
 			    error_code == READ_GITFILE_ERR_NOT_A_FILE) {
+				/* NEEDSWORK: fail if .git is not file nor dir */
 				if (is_git_directory(dir->buf))
 					gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 			} else if (error_code != READ_GITFILE_ERR_STAT_FAILED)
-- 
2.12.0.windows.1.7.g94dafc3b124

Re: [PATCH v6 00/12] Fix the early config

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> These patches are an attempt to make Git's startup sequence a bit less
> surprising.

I think this is ready for 'next', so let's ask reviewers to really
pay attention to this round, wait for a few days and merge it by the
end of the week at the latest.

Thanks.

Re: [PATCH v6 00/12] Fix the early config

[Jeff King]

On Mon, Mar 13, 2017 at 03:31:15PM -0700, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > These patches are an attempt to make Git's startup sequence a bit less
> > surprising.
> 
> I think this is ready for 'next', so let's ask reviewers to really
> pay attention to this round, wait for a few days and merge it by the
> end of the week at the latest.

I gave it another read-through and didn't find anything worth
mentioning. I agree it's probably ready for 'next'.

-Peff