[PATCH v7 0/8] Rewrite `git_config()` using config-set API

[PATCH v7 0/8] Rewrite `git_config()` using config-set API

[Tanay Abhra]

[Patch v7]: style nit corrected. (1/8) is Matthieu's translation patch.
	git_die_config_linenr() helper function added. Diff between v6
	and v7 appended for review.

[Patch v6]: Added _(....) to error messages.
	Diff between v6 and v4 at the bottom.

[PATCH v5]: New patch added (3/7). git_config() now returns void.

[PATCH v4]: One style nit corrected, also added key to error messages.

[PATCH V3]:All the suggestions in [3] applied. Built on top of [1].

[PATCH V2]: All the suggestions in [2] incorporated. git_config() now follows
	correct parsing order. Reordered the patches. Removed xfuncname patch
	as it was unnecssary.

This series builds on the top of topic[1] in the mailing list with name
"git config cache & special querying API utilizing the cache".

This series aims to do these three things,

* Use the config-set API to rewrite git_config().

* Solve any legacy bugs in the previous system while at it.

* To be feature complete compared to the previous git_config() implementation,
	which I think it is now. (added the line number and file name info just for
	completeness)

Also, I haven't yet checked the exact improvements but still as a teaser,
git status now only rereads the configuration files twice instead of four
times.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/254286
[2]: http://thread.gmane.org/gmane.comp.version-control.git/254101
[3]: http://thread.gmane.org/gmane.comp.version-control.git/254211

Matthieu Moy (1):
  config.c: mark error and warnings strings for translation

Tanay Abhra (7):
  config.c: fix accuracy of line number in errors
  add line number and file name info to `config_set`
  change `git_config()` return value to void
  config: add `git_die_config()` to the config-set API
  rewrite git_config() to use the config-set API
  add a test for semantic errors in config files
  add tests for `git_config_get_string_const()`

 Documentation/technical/api-config.txt |  12 +++
 branch.c                               |   5 +-
 cache.h                                |  28 +++++-
 config.c                               | 152 +++++++++++++++++++++++++++------
 t/t1308-config-set.sh                  |  21 +++++
 t/t4055-diff-context.sh                |   2 +-
 test-config.c                          |  10 +++
 7 files changed, 200 insertions(+), 30 deletions(-)

-- 
1.9.0.GIT

-- 8< --
diff --git a/Documentation/technical/api-config.txt b/Documentation/technical/api-config.txt
index e7ec7cc..d6a2c39 100644
--- a/Documentation/technical/api-config.txt
+++ b/Documentation/technical/api-config.txt
@@ -160,6 +160,13 @@ as well as retrieval for the queried variable, including:
 	Dies printing the line number and the file name of the highest
 	priority value for the configuration variable `key`.
 
+`void git_die_config_linenr(const char *key, const char *filename, int linenr)`::
+
+	Helper function which formats the die error message according to the
+	parameters entered. Used by `git_die_config()`. It can be used by callers
+	handling `git_config_get_value_multi()` to print the correct error message
+	for the desired value.
+
 See test-config.c for usage examples.
 
 Value Parsing Helpers
diff --git a/cache.h b/cache.h
index 243f618..ff17889 100644
--- a/cache.h
+++ b/cache.h
@@ -1407,6 +1407,7 @@ extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
 extern int git_config_get_maybe_bool(const char *key, int *dest);
 extern int git_config_get_pathname(const char *key, const char **dest);
 extern void git_die_config(const char *key);
+extern void git_die_config_linenr(const char *key, const char *filename, int linenr);
 
 extern int committer_ident_sufficiently_given(void);
 extern int author_ident_sufficiently_given(void);
diff --git a/config.c b/config.c
index 15fcd91..cf9124f 100644
--- a/config.c
+++ b/config.c
@@ -461,9 +461,9 @@ static int git_parse_source(config_fn_t fn, void *data)
 			break;
 	}
 	if (cf->die_on_error)
-		die("bad config file line %d in %s", cf->linenr, cf->name);
+		die(_("bad config file line %d in %s"), cf->linenr, cf->name);
 	else
-		return error("bad config file line %d in %s", cf->linenr, cf->name);
+		return error(_("bad config file line %d in %s"), cf->linenr, cf->name);
 }
 
 static int parse_unit_factor(const char *end, uintmax_t *val)
@@ -579,9 +579,9 @@ static void die_bad_number(const char *name, const char *value)
 		value = "";
 
 	if (cf && cf->name)
-		die("bad numeric config value '%s' for '%s' in %s: %s",
+		die(_("bad numeric config value '%s' for '%s' in %s: %s"),
 		    value, name, cf->name, reason);
-	die("bad numeric config value '%s' for '%s': %s", value, name, reason);
+	die(_("bad numeric config value '%s' for '%s': %s"), value, name, reason);
 }
 
 int git_config_int(const char *name, const char *value)
@@ -666,7 +666,7 @@ int git_config_pathname(const char **dest, const char *var, const char *value)
 		return config_error_nonbool(var);
 	*dest = expand_user_path(value);
 	if (!*dest)
-		die("Failed to expand user dir in: '%s'", value);
+		die(_("failed to expand user dir in: '%s'"), value);
 	return 0;
 }
 
@@ -744,7 +744,7 @@ static int git_default_core_config(const char *var, const char *value)
 		if (level == -1)
 			level = Z_DEFAULT_COMPRESSION;
 		else if (level < 0 || level > Z_BEST_COMPRESSION)
-			die("bad zlib compression level %d", level);
+			die(_("bad zlib compression level %d"), level);
 		zlib_compression_level = level;
 		zlib_compression_seen = 1;
 		return 0;
@@ -755,7 +755,7 @@ static int git_default_core_config(const char *var, const char *value)
 		if (level == -1)
 			level = Z_DEFAULT_COMPRESSION;
 		else if (level < 0 || level > Z_BEST_COMPRESSION)
-			die("bad zlib compression level %d", level);
+			die(_("bad zlib compression level %d"), level);
 		core_compression_level = level;
 		core_compression_seen = 1;
 		if (!zlib_compression_seen)
@@ -877,7 +877,7 @@ static int git_default_core_config(const char *var, const char *value)
 		else if (!strcmp(value, "link"))
 			object_creation_mode = OBJECT_CREATION_USES_HARDLINKS;
 		else
-			die("Invalid mode for object creation: %s", value);
+			die(_("invalid mode for object creation: %s"), value);
 		return 0;
 	}
 
@@ -1177,7 +1177,7 @@ int git_config_early(config_fn_t fn, void *data, const char *repo_config)
 
 	switch (git_config_from_parameters(fn, data)) {
 	case -1: /* error */
-		die("unable to parse command-line config");
+		die(_("unable to parse command-line config"));
 		break;
 	case 0: /* found nothing */
 		break;
@@ -1260,13 +1260,7 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 		values = &entry->value_list;
 		if (fn(entry->key, values->items[value_index].string, data) < 0) {
 			kv_info = values->items[value_index].util;
-			if (!kv_info->linenr)
-				die(_("unable to parse '%s' from command-line config"), entry->key);
-			else
-				die(_("bad config variable '%s' at file line %d in %s"),
-					entry->key,
-					kv_info->linenr,
-					kv_info->filename);
+			git_die_config_linenr(entry->key, kv_info->filename, kv_info->linenr);
 		}
 	}
 }
@@ -1569,20 +1563,27 @@ int git_config_get_pathname(const char *key, const char **dest)
 	return ret;
 }
 
+NORETURN
+void git_die_config_linenr(const char *key, const char *filename, int linenr)
+{
+	if (!linenr)
+		die(_("unable to parse '%s' from command-line config"), key);
+	else
+		die(_("bad config variable '%s' at file line %d in %s"),
+			key,
+			linenr,
+			filename);
+}
+
+NORETURN
 void git_die_config(const char *key)
 {
 	const struct string_list *values;
 	struct key_value_info *kv_info;
 	values = git_config_get_value_multi(key);
 	kv_info = values->items[values->nr - 1].util;
-	if (!kv_info->linenr)
-		die(_("unable to parse '%s' from command-line config"), key);
-	else
-		die(_("bad config variable '%s' at file line %d in %s"),
-			key,
-			kv_info->linenr,
-			kv_info->filename);
- }
+	git_die_config_linenr(key, kv_info->filename, kv_info->linenr);
+}
 
 /*
  * Find all the stuff for git_config_set() below.
@@ -1617,7 +1618,7 @@ static int store_aux(const char *key, const char *value, void *cb)
 	case KEY_SEEN:
 		if (matches(key, value)) {
 			if (store.seen == 1 && store.multi_replace == 0) {
-				warning("%s has multiple values", key);
+				warning(_("%s has multiple values"), key);
 			}
 
 			ALLOC_GROW(store.offset, store.seen + 1,
-- 8< --

[PATCH v7 1/8] config.c: mark error and warnings strings for translation

[Tanay Abhra]

From: Matthieu Moy <Matthieu.Moy@imag.fr>

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 config.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/config.c b/config.c
index a191328..34940fd 100644
--- a/config.c
+++ b/config.c
@@ -457,9 +457,9 @@ static int git_parse_source(config_fn_t fn, void *data)
 			break;
 	}
 	if (cf->die_on_error)
-		die("bad config file line %d in %s", cf->linenr, cf->name);
+		die(_("bad config file line %d in %s"), cf->linenr, cf->name);
 	else
-		return error("bad config file line %d in %s", cf->linenr, cf->name);
+		return error(_("bad config file line %d in %s"), cf->linenr, cf->name);
 }
 
 static int parse_unit_factor(const char *end, uintmax_t *val)
@@ -575,9 +575,9 @@ static void die_bad_number(const char *name, const char *value)
 		value = "";
 
 	if (cf && cf->name)
-		die("bad numeric config value '%s' for '%s' in %s: %s",
+		die(_("bad numeric config value '%s' for '%s' in %s: %s"),
 		    value, name, cf->name, reason);
-	die("bad numeric config value '%s' for '%s': %s", value, name, reason);
+	die(_("bad numeric config value '%s' for '%s': %s"), value, name, reason);
 }
 
 int git_config_int(const char *name, const char *value)
@@ -662,7 +662,7 @@ int git_config_pathname(const char **dest, const char *var, const char *value)
 		return config_error_nonbool(var);
 	*dest = expand_user_path(value);
 	if (!*dest)
-		die("Failed to expand user dir in: '%s'", value);
+		die(_("failed to expand user dir in: '%s'"), value);
 	return 0;
 }
 
@@ -740,7 +740,7 @@ static int git_default_core_config(const char *var, const char *value)
 		if (level == -1)
 			level = Z_DEFAULT_COMPRESSION;
 		else if (level < 0 || level > Z_BEST_COMPRESSION)
-			die("bad zlib compression level %d", level);
+			die(_("bad zlib compression level %d"), level);
 		zlib_compression_level = level;
 		zlib_compression_seen = 1;
 		return 0;
@@ -751,7 +751,7 @@ static int git_default_core_config(const char *var, const char *value)
 		if (level == -1)
 			level = Z_DEFAULT_COMPRESSION;
 		else if (level < 0 || level > Z_BEST_COMPRESSION)
-			die("bad zlib compression level %d", level);
+			die(_("bad zlib compression level %d"), level);
 		core_compression_level = level;
 		core_compression_seen = 1;
 		if (!zlib_compression_seen)
@@ -873,7 +873,7 @@ static int git_default_core_config(const char *var, const char *value)
 		else if (!strcmp(value, "link"))
 			object_creation_mode = OBJECT_CREATION_USES_HARDLINKS;
 		else
-			die("Invalid mode for object creation: %s", value);
+			die(_("invalid mode for object creation: %s"), value);
 		return 0;
 	}
 
@@ -1173,7 +1173,7 @@ int git_config_early(config_fn_t fn, void *data, const char *repo_config)
 
 	switch (git_config_from_parameters(fn, data)) {
 	case -1: /* error */
-		die("unable to parse command-line config");
+		die(_("unable to parse command-line config"));
 		break;
 	case 0: /* found nothing */
 		break;
@@ -1514,7 +1514,7 @@ static int store_aux(const char *key, const char *value, void *cb)
 	case KEY_SEEN:
 		if (matches(key, value)) {
 			if (store.seen == 1 && store.multi_replace == 0) {
-				warning("%s has multiple values", key);
+				warning(_("%s has multiple values"), key);
 			}
 
 			ALLOC_GROW(store.offset, store.seen + 1,
-- 
1.9.0.GIT

[PATCH v7 2/8] config.c: fix accuracy of line number in errors

[Tanay Abhra]

From: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>

If a callback returns a negative value to `git_config*()` family,
they call `die()` while printing the line number and the file name.
Currently the printed line number is off by one, thus printing the
wrong line number.

Make `linenr` point to the line we just parsed during the call
to callback to get accurate line number in error messages.

Commit-message-by: Tanay Abhra <tanayabh@gmail.com>
Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
---
 config.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/config.c b/config.c
index 34940fd..bb4629e 100644
--- a/config.c
+++ b/config.c
@@ -244,6 +244,7 @@ static int get_next_char(void)
 		cf->linenr++;
 	if (c == EOF) {
 		cf->eof = 1;
+		cf->linenr++;
 		c = '\n';
 	}
 	return c;
@@ -319,6 +320,7 @@ static int get_value(config_fn_t fn, void *data, struct strbuf *name)
 {
 	int c;
 	char *value;
+	int ret;
 
 	/* Get the full name */
 	for (;;) {
@@ -341,7 +343,15 @@ static int get_value(config_fn_t fn, void *data, struct strbuf *name)
 		if (!value)
 			return -1;
 	}
-	return fn(name->buf, value, data);
+	/*
+	 * We already consumed the \n, but we need linenr to point to
+	 * the line we just parsed during the call to fn to get
+	 * accurate line number in error messages.
+	 */
+	cf->linenr--;
+	ret = fn(name->buf, value, data);
+	cf->linenr++;
+	return ret;
 }
 
 static int get_extended_base_var(struct strbuf *name, int c)
-- 
1.9.0.GIT

[PATCH v7 3/8] add line number and file name info to `config_set`

[Tanay Abhra]

Store file name and line number for each key-value pair in the cache
during parsing of the configuration files.

Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
---
 config.c | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/config.c b/config.c
index bb4629e..e685b66 100644
--- a/config.c
+++ b/config.c
@@ -1230,6 +1230,11 @@ int git_config_with_options(config_fn_t fn, void *data,
 	return ret;
 }
 
+struct key_value_info {
+	const char *filename;
+	int linenr;
+};
+
 int git_config(config_fn_t fn, void *data)
 {
 	return git_config_with_options(fn, data, NULL, 1);
@@ -1260,6 +1265,9 @@ static struct config_set_element *configset_find_element(struct config_set *cs,
 static int configset_add_value(struct config_set *cs, const char *key, const char *value)
 {
 	struct config_set_element *e;
+	struct string_list_item *si;
+	struct key_value_info *kv_info = xmalloc(sizeof(*kv_info));
+
 	e = configset_find_element(cs, key);
 	/*
 	 * Since the keys are being fed by git_config*() callback mechanism, they
@@ -1272,7 +1280,16 @@ static int configset_add_value(struct config_set *cs, const char *key, const cha
 		string_list_init(&e->value_list, 1);
 		hashmap_add(&cs->config_hash, e);
 	}
-	string_list_append_nodup(&e->value_list, value ? xstrdup(value) : NULL);
+	si = string_list_append_nodup(&e->value_list, value ? xstrdup(value) : NULL);
+	if (cf) {
+		kv_info->filename = strintern(cf->name);
+		kv_info->linenr = cf->linenr;
+	} else {
+		/* for values read from `git_config_from_parameters()` */
+		kv_info->filename = strintern("parameter");
+		kv_info->linenr = 0;
+	}
+	si->util = kv_info;
 
 	return 0;
 }
@@ -1299,7 +1316,7 @@ void git_configset_clear(struct config_set *cs)
 	hashmap_iter_init(&cs->config_hash, &iter);
 	while ((entry = hashmap_iter_next(&iter))) {
 		free(entry->key);
-		string_list_clear(&entry->value_list, 0);
+		string_list_clear(&entry->value_list, 1);
 	}
 	hashmap_free(&cs->config_hash, 1);
 	cs->hash_initialized = 0;
-- 
1.9.0.GIT

[PATCH v7 4/8] change `git_config()` return value to void

[Tanay Abhra]

Currently `git_config()` returns an integer signifying an error code.
During rewrites of the function most of the code was shifted to
`git_config_with_options()`. `git_config_with_options()` normally
returns positive values if its `config_source` parameter is set as NULL,
as most errors are fatal, and non-fatal potential errors are guarded
by "if" statements that are entered only when no error is possible.

Still a negative value can be returned in case of race condition between
`access_or_die()` & `git_config_from_file()`. Also, all callers of
`git_config()` ignore the return value except for one case in branch.c.

Change `git_config()` return value to void and make it die if it receives
a negative value from `git_config_with_options()`.

Original-patch-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
---
 branch.c |  5 +----
 cache.h  |  2 +-
 config.c | 16 ++++++++++++++--
 3 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/branch.c b/branch.c
index 46e8aa8..735767d 100644
--- a/branch.c
+++ b/branch.c
@@ -161,10 +161,7 @@ int read_branch_desc(struct strbuf *buf, const char *branch_name)
 	strbuf_addf(&name, "branch.%s.description", branch_name);
 	cb.config_name = name.buf;
 	cb.value = NULL;
-	if (git_config(read_branch_desc_cb, &cb) < 0) {
-		strbuf_release(&name);
-		return -1;
-	}
+	git_config(read_branch_desc_cb, &cb);
 	if (cb.value)
 		strbuf_addstr(buf, cb.value);
 	strbuf_release(&name);
diff --git a/cache.h b/cache.h
index 7292aef..c61919d 100644
--- a/cache.h
+++ b/cache.h
@@ -1294,7 +1294,7 @@ extern int git_config_from_buf(config_fn_t fn, const char *name,
 			       const char *buf, size_t len, void *data);
 extern void git_config_push_parameter(const char *text);
 extern int git_config_from_parameters(config_fn_t fn, void *data);
-extern int git_config(config_fn_t fn, void *);
+extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
 				   int respect_includes);
diff --git a/config.c b/config.c
index e685b66..f923b1c 100644
--- a/config.c
+++ b/config.c
@@ -1235,9 +1235,21 @@ struct key_value_info {
 	int linenr;
 };
 
-int git_config(config_fn_t fn, void *data)
+void git_config(config_fn_t fn, void *data)
 {
-	return git_config_with_options(fn, data, NULL, 1);
+	if (git_config_with_options(fn, data, NULL, 1) < 0)
+		/*
+		 * git_config_with_options() normally returns only
+		 * positive values, as most errors are fatal, and
+		 * non-fatal potential errors are guarded by "if"
+		 * statements that are entered only when no error is
+		 * possible.
+		 *
+		 * If we ever encounter a non-fatal error, it means
+		 * something went really wrong and we should stop
+		 * immediately.
+		 */
+		die(_("unknown error occured while reading the configuration files"));
 }
 
 static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
-- 
1.9.0.GIT

[PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Tanay Abhra]

Add `git_die_config` that dies printing the line number and the file name
of the highest priority value for the configuration variable `key`.

It has usage in non-callback based config value retrieval where we can
raise an error and die if there is a semantic error.
For example,

	if (!git_config_get_value(key, &value)) {
		/* NULL values not allowed */
		if (!value)
			git_config_die(key);
		else
			/* do work */
	}

Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
---
 Documentation/technical/api-config.txt | 12 ++++++++++++
 cache.h                                |  2 ++
 config.c                               | 34 ++++++++++++++++++++++++++++++++--
 3 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/Documentation/technical/api-config.txt b/Documentation/technical/api-config.txt
index 815c1ee..d6a2c39 100644
--- a/Documentation/technical/api-config.txt
+++ b/Documentation/technical/api-config.txt
@@ -155,6 +155,18 @@ as well as retrieval for the queried variable, including:
 	Similar to `git_config_get_string`, but expands `~` or `~user` into
 	the user's home directory when found at the beginning of the path.
 
+`void git_die_config(const char *key)`::
+
+	Dies printing the line number and the file name of the highest
+	priority value for the configuration variable `key`.
+
+`void git_die_config_linenr(const char *key, const char *filename, int linenr)`::
+
+	Helper function which formats the die error message according to the
+	parameters entered. Used by `git_die_config()`. It can be used by callers
+	handling `git_config_get_value_multi()` to print the correct error message
+	for the desired value.
+
 See test-config.c for usage examples.
 
 Value Parsing Helpers
diff --git a/cache.h b/cache.h
index c61919d..9ec2f4e 100644
--- a/cache.h
+++ b/cache.h
@@ -1382,6 +1382,8 @@ extern int git_config_get_bool(const char *key, int *dest);
 extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
 extern int git_config_get_maybe_bool(const char *key, int *dest);
 extern int git_config_get_pathname(const char *key, const char **dest);
+extern void git_die_config(const char *key);
+extern void git_die_config_linenr(const char *key, const char *filename, int linenr);
 
 extern int committer_ident_sufficiently_given(void);
 extern int author_ident_sufficiently_given(void);
diff --git a/config.c b/config.c
index f923b1c..ddbfe4d 100644
--- a/config.c
+++ b/config.c
@@ -1474,8 +1474,12 @@ const struct string_list *git_config_get_value_multi(const char *key)
 
 int git_config_get_string_const(const char *key, const char **dest)
 {
+	int ret;
 	git_config_check_init();
-	return git_configset_get_string_const(&the_config_set, key, dest);
+	ret = git_configset_get_string_const(&the_config_set, key, dest);
+	if (ret < 0)
+		git_die_config(key);
+	return ret;
 }
 
 int git_config_get_string(const char *key, char **dest)
@@ -1516,8 +1520,34 @@ int git_config_get_maybe_bool(const char *key, int *dest)
 
 int git_config_get_pathname(const char *key, const char **dest)
 {
+	int ret;
 	git_config_check_init();
-	return git_configset_get_pathname(&the_config_set, key, dest);
+	ret = git_configset_get_pathname(&the_config_set, key, dest);
+	if (ret < 0)
+		git_die_config(key);
+	return ret;
+}
+
+NORETURN
+void git_die_config_linenr(const char *key, const char *filename, int linenr)
+{
+	if (!linenr)
+		die(_("unable to parse '%s' from command-line config"), key);
+	else
+		die(_("bad config variable '%s' at file line %d in %s"),
+			key,
+			linenr,
+			filename);
+}
+
+NORETURN
+void git_die_config(const char *key)
+{
+	const struct string_list *values;
+	struct key_value_info *kv_info;
+	values = git_config_get_value_multi(key);
+	kv_info = values->items[values->nr - 1].util;
+	git_die_config_linenr(key, kv_info->filename, kv_info->linenr);
 }
 
 /*
-- 
1.9.0.GIT

[PATCH v7 6/8] rewrite git_config() to use the config-set API

[Tanay Abhra]

Of all the functions in `git_config*()` family, `git_config()` has the
most invocations in the whole code base. Each `git_config()` invocation
causes config file rereads which can be avoided using the config-set API.

Use the config-set API to rewrite `git_config()` to use the config caching
layer to avoid config file rereads on each invocation during a git process
lifetime. First invocation constructs the cache, and after that for each
successive invocation, `git_config()` feeds values from the config cache
instead of rereading the configuration files.

Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
---
 cache.h                 | 24 +++++++++++++++++++++++
 config.c                | 51 +++++++++++++++++++++++++++++++++++++++++--------
 t/t4055-diff-context.sh |  2 +-
 3 files changed, 68 insertions(+), 9 deletions(-)

diff --git a/cache.h b/cache.h
index 9ec2f4e..ff17889 100644
--- a/cache.h
+++ b/cache.h
@@ -8,6 +8,7 @@
 #include "gettext.h"
 #include "convert.h"
 #include "trace.h"
+#include "string-list.h"
 
 #include SHA1_HEADER
 #ifndef git_SHA_CTX
@@ -1351,9 +1352,32 @@ extern int parse_config_key(const char *var,
 			    const char **subsection, int *subsection_len,
 			    const char **key);
 
+struct config_set_element {
+	struct hashmap_entry ent;
+	char *key;
+	struct string_list value_list;
+};
+
+struct configset_list_item {
+	struct config_set_element *e;
+	int value_index;
+};
+
+/*
+ * the contents of the list are ordered according to their
+ * position in the config files and order of parsing the files.
+ * (i.e. key-value pair at the last position of .git/config will
+ * be at the last item of the list)
+ */
+struct configset_list {
+	struct configset_list_item *items;
+	unsigned int nr, alloc;
+};
+
 struct config_set {
 	struct hashmap config_hash;
 	int hash_initialized;
+	struct configset_list list;
 };
 
 extern void git_configset_init(struct config_set *cs);
diff --git a/config.c b/config.c
index ddbfe4d..cf9124f 100644
--- a/config.c
+++ b/config.c
@@ -35,12 +35,6 @@ struct config_source {
 	long (*do_ftell)(struct config_source *c);
 };
 
-struct config_set_element {
-	struct hashmap_entry ent;
-	char *key;
-	struct string_list value_list;
-};
-
 static struct config_source *cf;
 
 static int zlib_compression_seen;
@@ -1235,7 +1229,7 @@ struct key_value_info {
 	int linenr;
 };
 
-void git_config(config_fn_t fn, void *data)
+static void git_config_raw(config_fn_t fn, void *data)
 {
 	if (git_config_with_options(fn, data, NULL, 1) < 0)
 		/*
@@ -1252,6 +1246,33 @@ void git_config(config_fn_t fn, void *data)
 		die(_("unknown error occured while reading the configuration files"));
 }
 
+static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
+{
+	int i, value_index;
+	struct string_list *values;
+	struct config_set_element *entry;
+	struct configset_list *list = &cs->list;
+	struct key_value_info *kv_info;
+
+	for (i = 0; i < list->nr; i++) {
+		entry = list->items[i].e;
+		value_index = list->items[i].value_index;
+		values = &entry->value_list;
+		if (fn(entry->key, values->items[value_index].string, data) < 0) {
+			kv_info = values->items[value_index].util;
+			git_die_config_linenr(entry->key, kv_info->filename, kv_info->linenr);
+		}
+	}
+}
+
+static void git_config_check_init(void);
+
+void git_config(config_fn_t fn, void *data)
+{
+	git_config_check_init();
+	configset_iter(&the_config_set, fn, data);
+}
+
 static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
 {
 	struct config_set_element k;
@@ -1278,6 +1299,7 @@ static int configset_add_value(struct config_set *cs, const char *key, const cha
 {
 	struct config_set_element *e;
 	struct string_list_item *si;
+	struct configset_list_item *l_item;
 	struct key_value_info *kv_info = xmalloc(sizeof(*kv_info));
 
 	e = configset_find_element(cs, key);
@@ -1293,6 +1315,12 @@ static int configset_add_value(struct config_set *cs, const char *key, const cha
 		hashmap_add(&cs->config_hash, e);
 	}
 	si = string_list_append_nodup(&e->value_list, value ? xstrdup(value) : NULL);
+
+	ALLOC_GROW(cs->list.items, cs->list.nr + 1, cs->list.alloc);
+	l_item = &cs->list.items[cs->list.nr++];
+	l_item->e = e;
+	l_item->value_index = e->value_list.nr - 1;
+
 	if (cf) {
 		kv_info->filename = strintern(cf->name);
 		kv_info->linenr = cf->linenr;
@@ -1316,6 +1344,9 @@ void git_configset_init(struct config_set *cs)
 {
 	hashmap_init(&cs->config_hash, (hashmap_cmp_fn)config_set_element_cmp, 0);
 	cs->hash_initialized = 1;
+	cs->list.nr = 0;
+	cs->list.alloc = 0;
+	cs->list.items = NULL;
 }
 
 void git_configset_clear(struct config_set *cs)
@@ -1332,6 +1363,10 @@ void git_configset_clear(struct config_set *cs)
 	}
 	hashmap_free(&cs->config_hash, 1);
 	cs->hash_initialized = 0;
+	free(cs->list.items);
+	cs->list.nr = 0;
+	cs->list.alloc = 0;
+	cs->list.items = NULL;
 }
 
 static int config_set_callback(const char *key, const char *value, void *cb)
@@ -1450,7 +1485,7 @@ static void git_config_check_init(void)
 	if (the_config_set.hash_initialized)
 		return;
 	git_configset_init(&the_config_set);
-	git_config(config_set_callback, &the_config_set);
+	git_config_raw(config_set_callback, &the_config_set);
 }
 
 void git_config_clear(void)
diff --git a/t/t4055-diff-context.sh b/t/t4055-diff-context.sh
index cd04543..741e080 100755
--- a/t/t4055-diff-context.sh
+++ b/t/t4055-diff-context.sh
@@ -79,7 +79,7 @@ test_expect_success 'non-integer config parsing' '
 test_expect_success 'negative integer config parsing' '
 	git config diff.context -1 &&
 	test_must_fail git diff 2>output &&
-	test_i18ngrep "bad config file" output
+	test_i18ngrep "bad config variable" output
 '
 
 test_expect_success '-U0 is valid, so is diff.context=0' '
-- 
1.9.0.GIT

[PATCH v7 7/8] add a test for semantic errors in config files

[Tanay Abhra]

Semantic errors (for example, for alias.* variables NULL values are
not allowed) in configuration files cause a die printing the line
number and file name of the offending value.

Add a test documenting that such errors cause a die printing the
accurate line number and file name.

Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
---
 t/t1308-config-set.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/t/t1308-config-set.sh b/t/t1308-config-set.sh
index 7fdf840..e2f9d0b 100755
--- a/t/t1308-config-set.sh
+++ b/t/t1308-config-set.sh
@@ -197,4 +197,15 @@ test_expect_success 'proper error on error in custom config files' '
 	test_cmp expect actual
 '
 
+test_expect_success 'check line errors for malformed values' '
+	mv .git/config .git/config.old &&
+	test_when_finished "mv .git/config.old .git/config" &&
+	cat >.git/config <<-\EOF &&
+	[alias]
+		br
+	EOF
+	test_expect_code 128 git br 2>result &&
+	grep "fatal: bad config variable '\''alias.br'\'' at file line 2 in .git/config" result
+'
+
 test_done
-- 
1.9.0.GIT

[PATCH v7 8/8] add tests for `git_config_get_string_const()`

[Tanay Abhra]

Add tests for `git_config_get_string_const()`, check whether it
dies printing the line number and the file name if a NULL
value is retrieved for the given key.

Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
---
 t/t1308-config-set.sh | 10 ++++++++++
 test-config.c         | 10 ++++++++++
 2 files changed, 20 insertions(+)

diff --git a/t/t1308-config-set.sh b/t/t1308-config-set.sh
index e2f9d0b..f012dd6 100755
--- a/t/t1308-config-set.sh
+++ b/t/t1308-config-set.sh
@@ -119,6 +119,16 @@ test_expect_success 'find integer value for a key' '
 	check_config get_int lamb.chop 65
 '
 
+test_expect_success 'find string value for a key' '
+	check_config get_string case.baz hask &&
+	check_config expect_code 1 get_string case.ba "Value not found for \"case.ba\""
+'
+
+test_expect_success 'check line error when NULL string is queried' '
+	test_expect_code 128 test-config get_string case.foo 2>result &&
+	grep "fatal: bad config variable '\''case.foo'\'' at file line 7 in .git/config" result
+'
+
 test_expect_success 'find integer if value is non parse-able' '
 	check_config expect_code 128 get_int lamb.head
 '
diff --git a/test-config.c b/test-config.c
index 9dd1b22..6a77552 100644
--- a/test-config.c
+++ b/test-config.c
@@ -16,6 +16,8 @@
  *
  * get_bool -> print bool value for the entered key or die
  *
+ * get_string -> print string value for the entered key or die
+ *
  * configset_get_value -> returns value with the highest priority for the entered key
  * 			from a config_set constructed from files entered as arguments.
  *
@@ -84,6 +86,14 @@ int main(int argc, char **argv)
 			printf("Value not found for \"%s\"\n", argv[2]);
 			goto exit1;
 		}
+	} else if (argc == 3 && !strcmp(argv[1], "get_string")) {
+		if (!git_config_get_string_const(argv[2], &v)) {
+			printf("%s\n", v);
+			goto exit0;
+		} else {
+			printf("Value not found for \"%s\"\n", argv[2]);
+			goto exit1;
+		}
 	} else if (!strcmp(argv[1], "configset_get_value")) {
 		for (i = 3; i < argc; i++) {
 			int err;
-- 
1.9.0.GIT

Re: [PATCH v7 2/8] config.c: fix accuracy of line number in errors

[Matthieu Moy]

Tanay Abhra <tanayabh@gmail.com> writes:

> From: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>

I usually use my @imag.fr, not @grenoble-inp.fr address as Git author
(even though my mailer has @grenoble-inp.fr as From: field). Both
addresses are equivalent so it's no big deal.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH v7 0/8] Rewrite `git_config()` using config-set API

[Matthieu Moy]

Tanay Abhra <tanayabh@gmail.com> writes:

> [Patch v7]: style nit corrected. (1/8) is Matthieu's translation patch.
> 	git_die_config_linenr() helper function added. Diff between v6
> 	and v7 appended for review.

This series is now

Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>

Thanks,

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Junio C Hamano]

Tanay Abhra <tanayabh@gmail.com> writes:

> Add `git_die_config` that dies printing the line number and the file name
> of the highest priority value for the configuration variable `key`.
>
> It has usage in non-callback based config value retrieval where we can
> raise an error and die if there is a semantic error.
> For example,
>
> 	if (!git_config_get_value(key, &value)) {
> 		/* NULL values not allowed */
> 		if (!value)
> 			git_config_die(key);
> 		else
> 			/* do work */
> 	}

It feels a bit unnatural at the API level that this does not take
'value'; I do understand that it is not a big deal in the error code
path to locate again the value from the configuration using the key,
but still.

It feels even more unnatural that the caller cannot say _how_ it
finds the value offending by not taking any message.  For one
particular callchain, e.g. git_config_get_string() that eventually
calls git_config_string() which will show an error message via
config_error_nonbool(), you may not want any extra message, but for
new callers that wants to make sure value falls within a supported
range, this forces it to write

	if (!git_config_get_int(key, &num)) {
        	if (!(0 < num && num < 4)) {
			error("'%s' must be between 1 and 3");
                        git_config_die(key);
		}
		/* otherwise work */
	}

and then the error message would say something like:

	error: 'core.frotz' must be between 1 and 3
	fatal: bad config variable 'core.frotz' at file line 15 in .git/config

which sounds somewhat backwards, at least to me.

> +NORETURN
> +void git_die_config_linenr(const char *key, const char *filename, int linenr)
> +{
> +	if (!linenr)
> +		die(_("unable to parse '%s' from command-line config"), key);

Do we have existing code that says "we signal that it is from the
command line by setting linenr to zero" already?  Otherwise I would
have thought filename == NULL would be a more sensible convention.

Otherwise OK.

> +	else
> +		die(_("bad config variable '%s' at file line %d in %s"),

At least, quote the last '%s'.

> +			key,
> +			linenr,
> +			filename);

Don't waste vertical real-estate line this.

Perhaps

        die(_("bad config variable '%s' in file '%s' at line %d"),
            key, linenr, filename);

Re: [PATCH v7 7/8] add a test for semantic errors in config files

[Junio C Hamano]

Tanay Abhra <tanayabh@gmail.com> writes:

> +	grep "fatal: bad config variable '\''alias.br'\'' at file line 2 in .git/config" result

This test is too tight (the full string), and also needs to know
about i18n, I think.

Re: [PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Tanay Abhra]

On 8/4/2014 11:37 PM, Junio C Hamano wrote:
> Tanay Abhra <tanayabh@gmail.com> writes:
> 
>> Add `git_die_config` that dies printing the line number and the file name
>> of the highest priority value for the configuration variable `key`.
>>
>> It has usage in non-callback based config value retrieval where we can
>> raise an error and die if there is a semantic error.
>> For example,
>>
>> 	if (!git_config_get_value(key, &value)) {
>> 		/* NULL values not allowed */
>> 		if (!value)
>> 			git_config_die(key);
>> 		else
>> 			/* do work */
>> 	}
> 
> It feels a bit unnatural at the API level that this does not take
> 'value'; I do understand that it is not a big deal in the error code
> path to locate again the value from the configuration using the key,
> but still.
>

But, we don't have a use for "value" as it is not denoted in the error
string, that is why I left it out.

> It feels even more unnatural that the caller cannot say _how_ it
> finds the value offending by not taking any message.  For one
> particular callchain, e.g. git_config_get_string() that eventually
> calls git_config_string() which will show an error message via
> config_error_nonbool(), you may not want any extra message, but for
> new callers that wants to make sure value falls within a supported
> range, this forces it to write
> 
> 	if (!git_config_get_int(key, &num)) {
>         	if (!(0 < num && num < 4)) {
> 			error("'%s' must be between 1 and 3");
>                         git_config_die(key);
> 		}
> 		/* otherwise work */
> 	}
> 
> and then the error message would say something like:
> 
> 	error: 'core.frotz' must be between 1 and 3
> 	fatal: bad config variable 'core.frotz' at file line 15 in .git/config
> 
> which sounds somewhat backwards, at least to me.
>

I was aping the old git_config() system, it also does exactly what you described
above. for example, builtin/gc.c line 91,

		if (!strcmp(var, "gc.pruneexpire")) {
		if (value && strcmp(value, "now")) {
			unsigned long now = approxidate("now");
			if (approxidate(value) >= now)
				return error(_("Invalid %s: '%s'"), var, value);
		}

would print,
 	error: Invalid gc.pruneexpire: 'value'
	fatal: bad config variable 'gc.pruneexpire' at file line 15 in .git/config

or imap-send.c line 1340,

	if (!strcmp("sslverify", key))
		server.ssl_verify = git_config_bool(key, val);
	else if (!strcmp("preformattedhtml", key))
		server.use_html = git_config_bool(key, val);
	else if (!val)
		return config_error_nonbool(key);
again would cause a error & die, message combo as above. There are many examples like that.
We can easily take a custom error message but again I was just aping the old system.

>> +NORETURN
>> +void git_die_config_linenr(const char *key, const char *filename, int linenr)
>> +{
>> +	if (!linenr)
>> +		die(_("unable to parse '%s' from command-line config"), key);
> 
> Do we have existing code that says "we signal that it is from the
> command line by setting linenr to zero" already?  Otherwise I would
> have thought filename == NULL would be a more sensible convention.
> 
> Otherwise OK.
>

Noted. Next reroll will have filename as the convention.

>> +	else
>> +		die(_("bad config variable '%s' at file line %d in %s"),
> 
> At least, quote the last '%s'.
>

Noted. Thanks.

Re: [PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Matthieu Moy]

Tanay Abhra <tanayabh@gmail.com> writes:

> I was aping the old git_config() system, it also does exactly what you described
> above. for example, builtin/gc.c line 91,
>
> 		if (!strcmp(var, "gc.pruneexpire")) {
> 		if (value && strcmp(value, "now")) {
> 			unsigned long now = approxidate("now");
> 			if (approxidate(value) >= now)
> 				return error(_("Invalid %s: '%s'"), var, value);
> 		}
>
> would print,
>  	error: Invalid gc.pruneexpire: 'value'
> 	fatal: bad config variable 'gc.pruneexpire' at file line 15 in .git/config

It's good to do at least as well as the old system, but I agree with
Junio that it's suboptimal.

Having a single API call to replace

        error("'%s' must be between 1 and 3");
        git_config_die(key);

with stg like:

        git_config_die(key, "'%s' must be between 1 and 3");

in Junio's example would allow git_config_die to format the error
message the way it likes (i.e. it can be the same as before when you
introduce it, and improve afterwards).

I've never been disturbed by the quality of Git's error messages wrt
config files (it's not a compiler!), so having good quality messages is
not a high priority IMHO, but having a good API that as a side effect
can produce good error messages is important. If changing the error
messages requires rewritting all callers later, then we've missed the
point doing the refactoring now.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Junio C Hamano]

Tanay Abhra <tanayabh@gmail.com> writes:

> On 8/4/2014 11:37 PM, Junio C Hamano wrote:
>> Tanay Abhra <tanayabh@gmail.com> writes:
>> 
>>> Add `git_die_config` that dies printing the line number and the file name
>>> of the highest priority value for the configuration variable `key`.
>>>
>>> It has usage in non-callback based config value retrieval where we can
>>> raise an error and die if there is a semantic error.
>>> For example,
>>>
>>> 	if (!git_config_get_value(key, &value)) {
>>> 		/* NULL values not allowed */
>>> 		if (!value)
>>> 			git_config_die(key);
>>> 		else
>>> 			/* do work */
>>> 	}
>> 
>> It feels a bit unnatural at the API level that this does not take
>> 'value'; I do understand that it is not a big deal in the error code
>> path to locate again the value from the configuration using the key,
>> but still.
>>
>
> But, we don't have a use for "value" as it is not denoted in the error
> string, that is why I left it out.

That is my point.  Why doesn't the error message talk about what
value the caller found was offensive, and in what way?

>>> +	else
>>> +		die(_("bad config variable '%s' at file line %d in %s"),
>> 
>> At least, quote the last '%s'.
>>
>
> Noted. Thanks.

Actually, "at file line" sounded very strange, at least to me, hence
the suggested reword in the part you did not quote.

Re: [PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Tanay Abhra]

On 8/5/2014 1:34 AM, Matthieu Moy wrote:
> Tanay Abhra <tanayabh@gmail.com> writes:
> 
>> I was aping the old git_config() system, it also does exactly what you described
>> above. for example, builtin/gc.c line 91,
>>
>> 		if (!strcmp(var, "gc.pruneexpire")) {
>> 		if (value && strcmp(value, "now")) {
>> 			unsigned long now = approxidate("now");
>> 			if (approxidate(value) >= now)
>> 				return error(_("Invalid %s: '%s'"), var, value);
>> 		}
>>
>> would print,
>>  	error: Invalid gc.pruneexpire: 'value'
>> 	fatal: bad config variable 'gc.pruneexpire' at file line 15 in .git/config
> 
> It's good to do at least as well as the old system, but I agree with
> Junio that it's suboptimal.
> 
> Having a single API call to replace
> 
>         error("'%s' must be between 1 and 3");
>         git_config_die(key);
> 
> with stg like:
> 
>         git_config_die(key, "'%s' must be between 1 and 3");
>

Matthieu, I have finished the new version, but instead of flooding the mailing list with
a series again, I wanted to confirm if the new git_config_die() is alright.

	NORETURN __attribute__((format(printf, 2, 3)))
	void git_die_config(const char *key, const char *err, ...)
	{
		const struct string_list *values;
		struct key_value_info *kv_info;

		if (err) {
			va_list params;
			va_start(params, err);
			vreportf("error: ", err, params);
			va_end(params);
		}
		values = git_config_get_value_multi(key);
		kv_info = values->items[values->nr - 1].util;
		git_die_config_linenr(key, kv_info->filename, kv_info->linenr);
	}

Currently works like the old git_config() error reporting path. If err is set to "NULL",
it would print no error message and just the die message. If given something like,

	 git_config_die(key, "value '%s' is not allowed", value);

it would print,

	error: value '3' is not allowed
	fatal: bad config variable 'core.frotz' at file line 15 in .git/config

Cheers,
Tanay Abhra.

> in Junio's example would allow git_config_die to format the error
> message the way it likes (i.e. it can be the same as before when you
> introduce it, and improve afterwards).
> 
> I've never been disturbed by the quality of Git's error messages wrt
> config files (it's not a compiler!), so having good quality messages is
> not a high priority IMHO, but having a good API that as a side effect
> can produce good error messages is important. If changing the error
> messages requires rewritting all callers later, then we've missed the
> point doing the refactoring now.
> 

Re: [PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Matthieu Moy]

Tanay Abhra <tanayabh@gmail.com> writes:

> Matthieu, I have finished the new version, but instead of flooding the mailing list with
> a series again, I wanted to confirm if the new git_config_die() is alright.
>
> 	NORETURN __attribute__((format(printf, 2, 3)))
> 	void git_die_config(const char *key, const char *err, ...)
> 	{
> 		const struct string_list *values;
> 		struct key_value_info *kv_info;
>
> 		if (err) {
> 			va_list params;
> 			va_start(params, err);
> 			vreportf("error: ", err, params);
> 			va_end(params);
> 		}
> 		values = git_config_get_value_multi(key);
> 		kv_info = values->items[values->nr - 1].util;
> 		git_die_config_linenr(key, kv_info->filename, kv_info->linenr);
> 	}
>
> Currently works like the old git_config() error reporting path. If err is set to "NULL",
> it would print no error message and just the die message. If given something like,
>
> 	 git_config_die(key, "value '%s' is not allowed", value);
>
> it would print,
>
> 	error: value '3' is not allowed
> 	fatal: bad config variable 'core.frotz' at file line 15 in .git/config

That seems to be a good step forward.

I think we would also want to improve the error message, but that
shouldn't block your series from inclusion: we can do that later without
API change.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH v7 5/8] config: add `git_die_config()` to the config-set API

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:

> Tanay Abhra <tanayabh@gmail.com> writes:
>
>> Currently works like the old git_config() error reporting path. If err is set to "NULL",
>> it would print no error message and just the die message. If given something like,
>>
>> 	 git_config_die(key, "value '%s' is not allowed", value);
>>
>> it would print,
>>
>> 	error: value '3' is not allowed
>> 	fatal: bad config variable 'core.frotz' at file line 15 in .git/config
>
> That seems to be a good step forward.
>
> I think we would also want to improve the error message, but that
> shouldn't block your series from inclusion: we can do that later without
> API change.

Yup, I agree with your assessment.

Thank you, both, for good polishing.