git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
* [PATCH] setup: warn if extensions exist on old format
@ 2020-07-13 19:20 Derrick Stolee via GitGitGadget
  2020-07-13 19:34 ` Taylor Blau
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Derrick Stolee via GitGitGadget @ 2020-07-13 19:20 UTC (permalink / raw)
  To: git; +Cc: newren, delphij, peff, Derrick Stolee, Derrick Stolee

From: Derrick Stolee <dstolee@microsoft.com>

Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions
for old repositories, 2020-06-05), Git was honoring configured
extensions, even if core.repositoryFormatVersion was 0 (or unset). This
was incorrect, and is now fixed.

The issue now is that users who relied on that previously bad behavior
will upgrade to the next version of Git and suddently be in a bad
situation. In particular, users of the 'git sparse-checkout' builting
will rely on the extensions.worktreeConfig for the core.sparseCheckout
and core.sparseCheckoutCone config options. Without that extension,
these users will suddenly have repositories stop acting like a sparse
repo.

What is worse is that a user will be confronted with the following
error if they try to run 'git sparse-checkout init' again:

	warning: unable to upgrade repository format from 0 to 1

This is because the logic in 14c7fa269e4 refuses to upgrae repos when
the version is unset but extensions exist.

While it is important to correct this improper behavior, create a
warning so users in this situation can correct themselves without too
much guesswork. By creating a warning in
check_repository_format_gently(), we can alert the user if they have a
ocnfigured extension but not a configured repository version.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
    setup: warn if extensions exist on old format
    
    This is based on xl/upgrade-repo-format.
    
    Can this be considered for v2.28.0-rc1? I think this is an important
    shift in behavior that will disrupt many users if they upgrade without
    it!
    
    If not this warning or change, then how else can we help users who are
    in this situation? How can we save those who adopted the sparse-checkout
    builtin in recent versions from terrible post-upgrade headaches?
    
    Thanks, -Stolee

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-674%2Fderrickstolee%2Fsparse-checkout-warning-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-674/derrickstolee/sparse-checkout-warning-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/674

 setup.c                            | 5 +++++
 t/t1091-sparse-checkout-builtin.sh | 9 +++++++++
 2 files changed, 14 insertions(+)

diff --git a/setup.c b/setup.c
index eb066db6d8..6ff6c54d39 100644
--- a/setup.c
+++ b/setup.c
@@ -542,6 +542,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
 		}
 	}
 
+	if (candidate->version == 0 && candidate->has_extensions) {
+		warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
+		warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
+	}
+
 	return 0;
 }
 
diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 88cdde255c..d249428f44 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -68,6 +68,15 @@ test_expect_success 'git sparse-checkout init' '
 	check_files repo a
 '
 
+test_expect_success 'warning about core.repositoryFormatVersion' '
+	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
+	git -C repo status 2>err &&
+	test_must_be_empty err &&
+	git -C repo config --local core.repositoryFormatVersion 0 &&
+	git -C repo status 2>err &&
+	test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err
+'
+
 test_expect_success 'git sparse-checkout list after init' '
 	git -C repo sparse-checkout list >actual &&
 	cat >expect <<-\EOF &&

base-commit: 14c7fa269e42df4133edd9ae7763b678ed6594cd
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: warn if extensions exist on old format
  2020-07-13 19:20 [PATCH] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
@ 2020-07-13 19:34 ` Taylor Blau
  2020-07-13 19:41   ` Derrick Stolee
  2020-07-13 19:36 ` Junio C Hamano
  2020-07-14  1:26 ` [PATCH v2 0/2] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
  2 siblings, 1 reply; 15+ messages in thread
From: Taylor Blau @ 2020-07-13 19:34 UTC (permalink / raw)
  To: Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, Derrick Stolee, Derrick Stolee

On Mon, Jul 13, 2020 at 07:20:51PM +0000, Derrick Stolee via GitGitGadget wrote:
> From: Derrick Stolee <dstolee@microsoft.com>
>
> Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions
> for old repositories, 2020-06-05), Git was honoring configured
> extensions, even if core.repositoryFormatVersion was 0 (or unset). This
> was incorrect, and is now fixed.
>
> The issue now is that users who relied on that previously bad behavior
> will upgrade to the next version of Git and suddently be in a bad
> situation. In particular, users of the 'git sparse-checkout' builting
> will rely on the extensions.worktreeConfig for the core.sparseCheckout
> and core.sparseCheckoutCone config options. Without that extension,
> these users will suddenly have repositories stop acting like a sparse
> repo.
>
> What is worse is that a user will be confronted with the following
> error if they try to run 'git sparse-checkout init' again:
>
> 	warning: unable to upgrade repository format from 0 to 1
>
> This is because the logic in 14c7fa269e4 refuses to upgrae repos when

s/upgrae/upgrade

> the version is unset but extensions exist.

I'm not sure that I want to get into a discussion about whether or not
this logic is right while -rc0 is already out, but it seems like
14c7fa269e4 could be tweaked to be a little more tolerant of this case.

On the other hand, I think that the approach here is perfectly sensible,
absent of a more invasive change to the logic in 14c7fa269e4.

> While it is important to correct this improper behavior, create a
> warning so users in this situation can correct themselves without too
> much guesswork. By creating a warning in
> check_repository_format_gently(), we can alert the user if they have a
> ocnfigured extension but not a configured repository version.

s/ocnfigured/configured

> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>     setup: warn if extensions exist on old format
>
>     This is based on xl/upgrade-repo-format.
>
>     Can this be considered for v2.28.0-rc1? I think this is an important
>     shift in behavior that will disrupt many users if they upgrade without
>     it!

I would be happy to see something like this in -rc1, unless Junio has
reservations about making this change at this point in the release cycle
(I do not have any such concerns).

>     If not this warning or change, then how else can we help users who are
>     in this situation? How can we save those who adopted the sparse-checkout
>     builtin in recent versions from terrible post-upgrade headaches?
>
>     Thanks, -Stolee
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-674%2Fderrickstolee%2Fsparse-checkout-warning-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-674/derrickstolee/sparse-checkout-warning-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/674
>
>  setup.c                            | 5 +++++
>  t/t1091-sparse-checkout-builtin.sh | 9 +++++++++
>  2 files changed, 14 insertions(+)
>
> diff --git a/setup.c b/setup.c
> index eb066db6d8..6ff6c54d39 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -542,6 +542,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
>  		}
>  	}
>
> +	if (candidate->version == 0 && candidate->has_extensions) {
> +		warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
> +		warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
> +	}
> +
>  	return 0;
>  }
>
> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
> index 88cdde255c..d249428f44 100755
> --- a/t/t1091-sparse-checkout-builtin.sh
> +++ b/t/t1091-sparse-checkout-builtin.sh
> @@ -68,6 +68,15 @@ test_expect_success 'git sparse-checkout init' '
>  	check_files repo a
>  '
>
> +test_expect_success 'warning about core.repositoryFormatVersion' '
> +	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
> +	git -C repo status 2>err &&
> +	test_must_be_empty err &&
> +	git -C repo config --local core.repositoryFormatVersion 0 &&

I don't think it's that difficult to see that this patch is correct, but
it might be worth testing this for the case that
'core.repositoryFormatVersion' is unset, too.

> +	git -C repo status 2>err &&
> +	test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err
> +'
> +
>  test_expect_success 'git sparse-checkout list after init' '
>  	git -C repo sparse-checkout list >actual &&
>  	cat >expect <<-\EOF &&
>
> base-commit: 14c7fa269e42df4133edd9ae7763b678ed6594cd
> --
> gitgitgadget

Thanks,
Taylor

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: warn if extensions exist on old format
  2020-07-13 19:20 [PATCH] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
  2020-07-13 19:34 ` Taylor Blau
@ 2020-07-13 19:36 ` Junio C Hamano
  2020-07-13 20:00   ` Junio C Hamano
  2020-07-14  1:26 ` [PATCH v2 0/2] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
  2 siblings, 1 reply; 15+ messages in thread
From: Junio C Hamano @ 2020-07-13 19:36 UTC (permalink / raw)
  To: Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, Derrick Stolee, Derrick Stolee

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Derrick Stolee <dstolee@microsoft.com>
>
> Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions
> for old repositories, 2020-06-05), Git was honoring configured
> extensions, even if core.repositoryFormatVersion was 0 (or unset). This
> was incorrect, and is now fixed.
>
> The issue now is that users who relied on that previously bad behavior
> will upgrade to the next version of Git and suddently be in a bad
> situation. In particular, users of the 'git sparse-checkout' builting
> will rely on the extensions.worktreeConfig for the core.sparseCheckout
> and core.sparseCheckoutCone config options. Without that extension,
> these users will suddenly have repositories stop acting like a sparse
> repo.

s/builting/command/ perhaps???

>
> What is worse is that a user will be confronted with the following
> error if they try to run 'git sparse-checkout init' again:
>
> 	warning: unable to upgrade repository format from 0 to 1
>
> This is because the logic in 14c7fa269e4 refuses to upgrae repos when
> the version is unset but extensions exist.
>
> While it is important to correct this improper behavior, create a
> warning so users in this situation can correct themselves without too
> much guesswork. By creating a warning in
> check_repository_format_gently(), we can alert the user if they have a
> ocnfigured extension but not a configured repository version.

s/ocn/con/

> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---

Thanks for a thoughtful analysis of the situation and coming up with
a plan to remedy.

> +	if (candidate->version == 0 && candidate->has_extensions) {
> +		warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
> +		warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
> +	}

That reads well.

An alternative may be to grandfather some extensions that were
enabled by git by mistake without updating the format version, and
we update the repository even if the repository has extensions that
should not exist, but those offending extensions are limited only to
those that we decide to special case.  That would make the end-user
experience even smoother.

Is extenions.worktreeCOnfig the only one that needs this escape
hatch?



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: warn if extensions exist on old format
  2020-07-13 19:34 ` Taylor Blau
@ 2020-07-13 19:41   ` Derrick Stolee
  0 siblings, 0 replies; 15+ messages in thread
From: Derrick Stolee @ 2020-07-13 19:41 UTC (permalink / raw)
  To: Taylor Blau, Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, Derrick Stolee, Derrick Stolee

On 7/13/2020 3:34 PM, Taylor Blau wrote:
> s/upgrae/upgrade
> s/ocnfigured/configured

Oh man, what was going on with me when I was typing this message.

Thanks for the thorough inspection.

>> diff --git a/setup.c b/setup.c
>> index eb066db6d8..6ff6c54d39 100644
>> --- a/setup.c
>> +++ b/setup.c
>> @@ -542,6 +542,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
>>  		}
>>  	}
>>
>> +	if (candidate->version == 0 && candidate->has_extensions) {
>> +		warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
>> +		warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
>> +	}
>> +
>>  	return 0;
>>  }
>>
>> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
>> index 88cdde255c..d249428f44 100755
>> --- a/t/t1091-sparse-checkout-builtin.sh
>> +++ b/t/t1091-sparse-checkout-builtin.sh
>> @@ -68,6 +68,15 @@ test_expect_success 'git sparse-checkout init' '
>>  	check_files repo a
>>  '
>>
>> +test_expect_success 'warning about core.repositoryFormatVersion' '
>> +	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
>> +	git -C repo status 2>err &&
>> +	test_must_be_empty err &&
>> +	git -C repo config --local core.repositoryFormatVersion 0 &&
> 
> I don't think it's that difficult to see that this patch is correct, but
> it might be worth testing this for the case that
> 'core.repositoryFormatVersion' is unset, too.

You were absolutely right to check this, since this diff causes
the test to fail:

diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index d249428f44..b5de141292 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -74,6 +74,9 @@ test_expect_success 'warning about core.repositoryFormatVersion' '
        test_must_be_empty err &&
        git -C repo config --local core.repositoryFormatVersion 0 &&
        git -C repo status 2>err &&
+       test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err &&
+       git -C repo config --local --unset core.repositoryFormatVersion 0 &&
+       git -C repo status 2>err &&
        test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err
 '
 

I'll investigate why the "unset" case is different than the "0" case.

Hopefully the "should we do this?" question can continue being discussed
while I work on a v2.

Thanks,
-Stolee

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: warn if extensions exist on old format
  2020-07-13 19:36 ` Junio C Hamano
@ 2020-07-13 20:00   ` Junio C Hamano
  2020-07-13 20:18     ` [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig Junio C Hamano
  0 siblings, 1 reply; 15+ messages in thread
From: Junio C Hamano @ 2020-07-13 20:00 UTC (permalink / raw)
  To: Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, Derrick Stolee, Derrick Stolee

Junio C Hamano <gitster@pobox.com> writes:

> An alternative may be to grandfather some extensions that were
> enabled by git by mistake without updating the format version, and
> we update the repository even if the repository has extensions that
> should not exist, but those offending extensions are limited only to
> those that we decide to special case.  That would make the end-user
> experience even smoother.
>
> Is extenions.worktreeCOnfig the only one that needs this escape
> hatch?

Assuming that worktreeconfig is the only thing, the change may look
like this.  With this change, we might want to drop the new warning
in hunk ll.542- to avoid encouraging people to muck with their
repository with random configuration variables that happen to share
extensions.* prefix with us.

 cache.h |  2 +-
 setup.c | 17 +++++++++++++----
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/cache.h b/cache.h
index e5885cc9ea..8ff46857f6 100644
--- a/cache.h
+++ b/cache.h
@@ -1042,8 +1042,8 @@ struct repository_format {
 	int worktree_config;
 	int is_bare;
 	int hash_algo;
-	int has_extensions;
 	char *work_tree;
+	int has_unallowed_extensions;
 	struct string_list unknown_extensions;
 };
 
diff --git a/setup.c b/setup.c
index eb066db6d8..5f4786d3b9 100644
--- a/setup.c
+++ b/setup.c
@@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 	if (strcmp(var, "core.repositoryformatversion") == 0)
 		data->version = git_config_int(var, value);
 	else if (skip_prefix(var, "extensions.", &ext)) {
-		data->has_extensions = 1;
 		/*
 		 * record any known extensions here; otherwise,
 		 * we fall through to recording it as unknown, and
 		 * check_repository_format will complain
 		 */
+		int is_unallowed_extension = 1;
+
 		if (!strcmp(ext, "noop"))
 			;
 		else if (!strcmp(ext, "preciousobjects"))
@@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 			if (!value)
 				return config_error_nonbool(var);
 			data->partial_clone = xstrdup(value);
-		} else if (!strcmp(ext, "worktreeconfig"))
+		} else if (!strcmp(ext, "worktreeconfig")) {
 			data->worktree_config = git_config_bool(var, value);
-		else
+			is_unallowed_extension = 0;
+		} else
 			string_list_append(&data->unknown_extensions, ext);
+
+		data->has_unallowed_extensions |= is_unallowed_extension;
 	}
 
 	return read_worktree_config(var, value, vdata);
@@ -542,6 +546,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
 		}
 	}
 
+	if (candidate->version == 0 && candidate->has_unallowed_extensions) {
+		warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
+		warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
+	}
+
 	return 0;
 }
 
@@ -560,7 +569,7 @@ int upgrade_repository_format(int target_version)
 		return 0;
 
 	if (verify_repository_format(&repo_fmt, &err) < 0 ||
-	    (!repo_fmt.version && repo_fmt.has_extensions)) {
+	    (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {
 		warning("unable to upgrade repository format from %d to %d: %s",
 			repo_fmt.version, target_version, err.buf);
 		strbuf_release(&err);

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
  2020-07-13 20:00   ` Junio C Hamano
@ 2020-07-13 20:18     ` Junio C Hamano
  2020-07-13 20:46       ` Derrick Stolee
  0 siblings, 1 reply; 15+ messages in thread
From: Junio C Hamano @ 2020-07-13 20:18 UTC (permalink / raw)
  To: Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, Derrick Stolee, Derrick Stolee

The "fail and warn" approach introduced in the previous step may be
with smaller impact to the codebase, but

 - it requires the end-user to read, understand and execute the
   manual upgrade

 - it encourages to follow the same procedure blindly, making the
   protection even less useful

Let's instead keep failing hard without teaching how to bypass the
repository protection, but allow upgrading even when only the
worktreeconfig extension exists in an old repository, which is
likely to be set by a broke version of Git that did not update the
repository version when setting the extension.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * This time with a log message to explain and justify the change,
   with a matching tweak to the test script, designed to be applied
   on top, but feel free to squash it in if you agree with me that
   we do not need two separate commits for this.

   Thanks.

 cache.h                            |  2 +-
 setup.c                            | 17 ++++++++---------
 t/t1091-sparse-checkout-builtin.sh |  9 ---------
 3 files changed, 9 insertions(+), 19 deletions(-)

diff --git a/cache.h b/cache.h
index e5885cc9ea..8ff46857f6 100644
--- a/cache.h
+++ b/cache.h
@@ -1042,8 +1042,8 @@ struct repository_format {
 	int worktree_config;
 	int is_bare;
 	int hash_algo;
-	int has_extensions;
 	char *work_tree;
+	int has_unallowed_extensions;
 	struct string_list unknown_extensions;
 };
 
diff --git a/setup.c b/setup.c
index 6ff6c54d39..65270440a9 100644
--- a/setup.c
+++ b/setup.c
@@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 	if (strcmp(var, "core.repositoryformatversion") == 0)
 		data->version = git_config_int(var, value);
 	else if (skip_prefix(var, "extensions.", &ext)) {
-		data->has_extensions = 1;
 		/*
 		 * record any known extensions here; otherwise,
 		 * we fall through to recording it as unknown, and
 		 * check_repository_format will complain
 		 */
+		int is_unallowed_extension = 1;
+
 		if (!strcmp(ext, "noop"))
 			;
 		else if (!strcmp(ext, "preciousobjects"))
@@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 			if (!value)
 				return config_error_nonbool(var);
 			data->partial_clone = xstrdup(value);
-		} else if (!strcmp(ext, "worktreeconfig"))
+		} else if (!strcmp(ext, "worktreeconfig")) {
 			data->worktree_config = git_config_bool(var, value);
-		else
+			is_unallowed_extension = 0;
+		} else
 			string_list_append(&data->unknown_extensions, ext);
+
+		data->has_unallowed_extensions |= is_unallowed_extension;
 	}
 
 	return read_worktree_config(var, value, vdata);
@@ -542,11 +546,6 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
 		}
 	}
 
-	if (candidate->version == 0 && candidate->has_extensions) {
-		warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
-		warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
-	}
-
 	return 0;
 }
 
@@ -565,7 +564,7 @@ int upgrade_repository_format(int target_version)
 		return 0;
 
 	if (verify_repository_format(&repo_fmt, &err) < 0 ||
-	    (!repo_fmt.version && repo_fmt.has_extensions)) {
+	    (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {
 		warning("unable to upgrade repository format from %d to %d: %s",
 			repo_fmt.version, target_version, err.buf);
 		strbuf_release(&err);
diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index d249428f44..88cdde255c 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -68,15 +68,6 @@ test_expect_success 'git sparse-checkout init' '
 	check_files repo a
 '
 
-test_expect_success 'warning about core.repositoryFormatVersion' '
-	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
-	git -C repo status 2>err &&
-	test_must_be_empty err &&
-	git -C repo config --local core.repositoryFormatVersion 0 &&
-	git -C repo status 2>err &&
-	test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err
-'
-
 test_expect_success 'git sparse-checkout list after init' '
 	git -C repo sparse-checkout list >actual &&
 	cat >expect <<-\EOF &&
-- 
2.28.0-rc0


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
  2020-07-13 20:18     ` [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig Junio C Hamano
@ 2020-07-13 20:46       ` Derrick Stolee
  2020-07-13 21:45         ` Junio C Hamano
  2020-07-14  4:06         ` Jonathan Nieder
  0 siblings, 2 replies; 15+ messages in thread
From: Derrick Stolee @ 2020-07-13 20:46 UTC (permalink / raw)
  To: Junio C Hamano, Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, Derrick Stolee, Derrick Stolee

On 7/13/2020 4:18 PM, Junio C Hamano wrote:
> The "fail and warn" approach introduced in the previous step may be
> with smaller impact to the codebase, but
> 
>  - it requires the end-user to read, understand and execute the
>    manual upgrade
> 
>  - it encourages to follow the same procedure blindly, making the
>    protection even less useful
> 
> Let's instead keep failing hard without teaching how to bypass the
> repository protection, but allow upgrading even when only the
> worktreeconfig extension exists in an old repository, which is
> likely to be set by a broke version of Git that did not update the
> repository version when setting the extension.

This is a more subtle way to handle the case. In fact, it
silently makes extensions.worktreeConfig work as it did in 2.27,
which means users will not have any troubles after upgrading.

I also like that you are actually fixing the case where a user in
the bad state _can_ get out using "git sparse-checkout init".

This can be verified by adding this test:

test_expect_success 'git sparse-checkout works if repository format is wrong' '
	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
	git -C repo sparse-checkout init &&
	git -C repo config core.repositoryFormatVersion >actual &&
	echo 1 >expect &&
	test_cmp expect actual &&
	git -C repo config core.repositoryFormatVersion 0 &&
	git -C repo sparse-checkout init &&
	git -C repo config core.repositoryFormatVersion >actual &&
	test_cmp expect actual
'

> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> 
>  * This time with a log message to explain and justify the change,
>    with a matching tweak to the test script, designed to be applied
>    on top, but feel free to squash it in if you agree with me that
>    we do not need two separate commits for this.

Since this commit removes all evidence of the previous one, I would
recommend just squashing them together.

Thanks for your fast feedback!
-Stolee

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
  2020-07-13 20:46       ` Derrick Stolee
@ 2020-07-13 21:45         ` Junio C Hamano
  2020-07-14  4:06         ` Jonathan Nieder
  1 sibling, 0 replies; 15+ messages in thread
From: Junio C Hamano @ 2020-07-13 21:45 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: Derrick Stolee via GitGitGadget, git, newren, delphij, peff,
	Derrick Stolee, Derrick Stolee

Derrick Stolee <stolee@gmail.com> writes:

> I also like that you are actually fixing the case where a user in
> the bad state _can_ get out using "git sparse-checkout init".
>
> This can be verified by adding this test:
>
> test_expect_success 'git sparse-checkout works if repository format is wrong' '
> 	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
> 	git -C repo sparse-checkout init &&
> 	git -C repo config core.repositoryFormatVersion >actual &&
> 	echo 1 >expect &&
> 	test_cmp expect actual &&
> 	git -C repo config core.repositoryFormatVersion 0 &&
> 	git -C repo sparse-checkout init &&
> 	git -C repo config core.repositoryFormatVersion >actual &&
> 	test_cmp expect actual
> '
>
>> Signed-off-by: Junio C Hamano <gitster@pobox.com>
>> ---
>> 
>>  * This time with a log message to explain and justify the change,
>>    with a matching tweak to the test script, designed to be applied
>>    on top, but feel free to squash it in if you agree with me that
>>    we do not need two separate commits for this.
>
> Since this commit removes all evidence of the previous one, I would
> recommend just squashing them together.

Alright, then care to do the honors ;-)?  Let's make sure we have it
in -rc1 to avoid nasty "regression" in the upcoming release.

Thanks for raising the issue and exploring the solution space.


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 0/2] setup: warn if extensions exist on old format
  2020-07-13 19:20 [PATCH] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
  2020-07-13 19:34 ` Taylor Blau
  2020-07-13 19:36 ` Junio C Hamano
@ 2020-07-14  1:26 ` Derrick Stolee via GitGitGadget
  2020-07-14  1:26   ` [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig Derrick Stolee via GitGitGadget
                     ` (2 more replies)
  2 siblings, 3 replies; 15+ messages in thread
From: Derrick Stolee via GitGitGadget @ 2020-07-14  1:26 UTC (permalink / raw)
  To: git; +Cc: newren, delphij, peff, johannes.schindelin, me, Derrick Stolee

This is based on xl/upgrade-repo-format.

Thanks, Taylor and Junio for jumping in with helpful review.

Updates in v2:

 1. My initial patch is essentially dropped in its entirety, with Junio's
    patch here instead. I added an extra test and kept some of my commit
    message.
    
    
 2. A second patch has joined the fray, hopefully answering the concerned
    raise by Johannes [1].
    
    

[1] 
https://lore.kernel.org/git/pull.675.git.1594677321039.gitgitgadget@gmail.com/

Thanks, -Stolee

Derrick Stolee (2):
  setup: tweak upgrade policy to grandfather worktreeconfig
  config: provide extra detail about worktree config

 builtin/config.c                   |  5 +++--
 cache.h                            |  2 +-
 setup.c                            | 12 ++++++++----
 t/t1091-sparse-checkout-builtin.sh | 12 ++++++++++++
 t/t2404-worktree-config.sh         | 12 ++++++++++++
 5 files changed, 36 insertions(+), 7 deletions(-)


base-commit: 14c7fa269e42df4133edd9ae7763b678ed6594cd
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-674%2Fderrickstolee%2Fsparse-checkout-warning-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-674/derrickstolee/sparse-checkout-warning-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/674

Range-diff vs v1:

 1:  113cb7cda2 ! 1:  1b26d9710a setup: warn if extensions exist on old format
     @@ Metadata
      Author: Derrick Stolee <dstolee@microsoft.com>
      
       ## Commit message ##
     -    setup: warn if extensions exist on old format
     +    setup: tweak upgrade policy to grandfather worktreeconfig
      
          Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions
          for old repositories, 2020-06-05), Git was honoring configured
     @@ Commit message
      
          The issue now is that users who relied on that previously bad behavior
          will upgrade to the next version of Git and suddently be in a bad
     -    situation. In particular, users of the 'git sparse-checkout' builting
     +    situation. In particular, users of the 'git sparse-checkout' command
          will rely on the extensions.worktreeConfig for the core.sparseCheckout
          and core.sparseCheckoutCone config options. Without that extension,
          these users will suddenly have repositories stop acting like a sparse
     @@ Commit message
          This is because the logic in 14c7fa269e4 refuses to upgrae repos when
          the version is unset but extensions exist.
      
     -    While it is important to correct this improper behavior, create a
     -    warning so users in this situation can correct themselves without too
     -    much guesswork. By creating a warning in
     -    check_repository_format_gently(), we can alert the user if they have a
     -    ocnfigured extension but not a configured repository version.
     +    One possible way to alert a user of this issue is to warn them when Git
     +    notices an extension exists, but core.repositoryFormatVersion is not a
     +    correct value. However,
      
     +     - it requires the end-user to read, understand and execute the
     +       manual upgrade
     +
     +     - it encourages to follow the same procedure blindly, making the
     +       protection even less useful
     +
     +    Let's instead keep failing hard without teaching how to bypass the
     +    repository protection, but allow upgrading even when only the
     +    worktreeconfig extension exists in an old repository, which is
     +    likely to be set by a broke version of Git that did not update the
     +    repository version when setting the extension.
     +
     +    This change of behavior is made visible by testing how 'git
     +    sparse-checkout init' behaves to upgrade the repository format version
     +    even if the extension.worktreeConfig is already set. This would
     +    previously fail without a clear way forward.
     +
     +    Signed-off-by: Junio C Hamano <gitster@pobox.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
     + ## cache.h ##
     +@@ cache.h: struct repository_format {
     + 	int worktree_config;
     + 	int is_bare;
     + 	int hash_algo;
     +-	int has_extensions;
     + 	char *work_tree;
     ++	int has_unallowed_extensions;
     + 	struct string_list unknown_extensions;
     + };
     + 
     +
       ## setup.c ##
     -@@ setup.c: static int check_repository_format_gently(const char *gitdir, struct repository_
     - 		}
     +@@ setup.c: static int check_repo_format(const char *var, const char *value, void *vdata)
     + 	if (strcmp(var, "core.repositoryformatversion") == 0)
     + 		data->version = git_config_int(var, value);
     + 	else if (skip_prefix(var, "extensions.", &ext)) {
     +-		data->has_extensions = 1;
     + 		/*
     + 		 * record any known extensions here; otherwise,
     + 		 * we fall through to recording it as unknown, and
     + 		 * check_repository_format will complain
     + 		 */
     ++		int is_unallowed_extension = 1;
     ++
     + 		if (!strcmp(ext, "noop"))
     + 			;
     + 		else if (!strcmp(ext, "preciousobjects"))
     +@@ setup.c: static int check_repo_format(const char *var, const char *value, void *vdata)
     + 			if (!value)
     + 				return config_error_nonbool(var);
     + 			data->partial_clone = xstrdup(value);
     +-		} else if (!strcmp(ext, "worktreeconfig"))
     ++		} else if (!strcmp(ext, "worktreeconfig")) {
     + 			data->worktree_config = git_config_bool(var, value);
     +-		else
     ++			is_unallowed_extension = 0;
     ++		} else
     + 			string_list_append(&data->unknown_extensions, ext);
     ++
     ++		data->has_unallowed_extensions |= is_unallowed_extension;
       	}
       
     -+	if (candidate->version == 0 && candidate->has_extensions) {
     -+		warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
     -+		warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
     -+	}
     -+
     - 	return 0;
     - }
     + 	return read_worktree_config(var, value, vdata);
     +@@ setup.c: int upgrade_repository_format(int target_version)
     + 		return 0;
       
     + 	if (verify_repository_format(&repo_fmt, &err) < 0 ||
     +-	    (!repo_fmt.version && repo_fmt.has_extensions)) {
     ++	    (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {
     + 		warning("unable to upgrade repository format from %d to %d: %s",
     + 			repo_fmt.version, target_version, err.buf);
     + 		strbuf_release(&err);
      
       ## t/t1091-sparse-checkout-builtin.sh ##
      @@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'git sparse-checkout init' '
       	check_files repo a
       '
       
     -+test_expect_success 'warning about core.repositoryFormatVersion' '
     ++test_expect_success 'git sparse-checkout works if repository format is wrong' '
      +	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
     -+	git -C repo status 2>err &&
     -+	test_must_be_empty err &&
     -+	git -C repo config --local core.repositoryFormatVersion 0 &&
     -+	git -C repo status 2>err &&
     -+	test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err
     ++	git -C repo config --unset core.repositoryFormatVersion &&
     ++	git -C repo sparse-checkout init &&
     ++	git -C repo config core.repositoryFormatVersion >actual &&
     ++	echo 1 >expect &&
     ++	git -C repo config core.repositoryFormatVersion 0 &&
     ++	git -C repo sparse-checkout init &&
     ++	git -C repo config core.repositoryFormatVersion >actual &&
     ++	test_cmp expect actual
      +'
      +
       test_expect_success 'git sparse-checkout list after init' '
 -:  ---------- > 2:  e11e973c6f config: provide extra detail about worktree config

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig
  2020-07-14  1:26 ` [PATCH v2 0/2] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
@ 2020-07-14  1:26   ` Derrick Stolee via GitGitGadget
  2020-07-14 20:22     ` Johannes Schindelin
  2020-07-14  1:26   ` [PATCH v2 2/2] config: provide extra detail about worktree config Derrick Stolee via GitGitGadget
  2020-07-14 20:24   ` [PATCH v2 0/2] setup: warn if extensions exist on old format Johannes Schindelin
  2 siblings, 1 reply; 15+ messages in thread
From: Derrick Stolee via GitGitGadget @ 2020-07-14  1:26 UTC (permalink / raw)
  To: git
  Cc: newren, delphij, peff, johannes.schindelin, me, Derrick Stolee,
	Derrick Stolee

From: Derrick Stolee <dstolee@microsoft.com>

Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions
for old repositories, 2020-06-05), Git was honoring configured
extensions, even if core.repositoryFormatVersion was 0 (or unset). This
was incorrect, and is now fixed.

The issue now is that users who relied on that previously bad behavior
will upgrade to the next version of Git and suddently be in a bad
situation. In particular, users of the 'git sparse-checkout' command
will rely on the extensions.worktreeConfig for the core.sparseCheckout
and core.sparseCheckoutCone config options. Without that extension,
these users will suddenly have repositories stop acting like a sparse
repo.

What is worse is that a user will be confronted with the following
error if they try to run 'git sparse-checkout init' again:

	warning: unable to upgrade repository format from 0 to 1

This is because the logic in 14c7fa269e4 refuses to upgrae repos when
the version is unset but extensions exist.

One possible way to alert a user of this issue is to warn them when Git
notices an extension exists, but core.repositoryFormatVersion is not a
correct value. However,

 - it requires the end-user to read, understand and execute the
   manual upgrade

 - it encourages to follow the same procedure blindly, making the
   protection even less useful

Let's instead keep failing hard without teaching how to bypass the
repository protection, but allow upgrading even when only the
worktreeconfig extension exists in an old repository, which is
likely to be set by a broke version of Git that did not update the
repository version when setting the extension.

This change of behavior is made visible by testing how 'git
sparse-checkout init' behaves to upgrade the repository format version
even if the extension.worktreeConfig is already set. This would
previously fail without a clear way forward.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 cache.h                            |  2 +-
 setup.c                            | 12 ++++++++----
 t/t1091-sparse-checkout-builtin.sh | 12 ++++++++++++
 3 files changed, 21 insertions(+), 5 deletions(-)

diff --git a/cache.h b/cache.h
index e5885cc9ea..8ff46857f6 100644
--- a/cache.h
+++ b/cache.h
@@ -1042,8 +1042,8 @@ struct repository_format {
 	int worktree_config;
 	int is_bare;
 	int hash_algo;
-	int has_extensions;
 	char *work_tree;
+	int has_unallowed_extensions;
 	struct string_list unknown_extensions;
 };
 
diff --git a/setup.c b/setup.c
index eb066db6d8..65270440a9 100644
--- a/setup.c
+++ b/setup.c
@@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 	if (strcmp(var, "core.repositoryformatversion") == 0)
 		data->version = git_config_int(var, value);
 	else if (skip_prefix(var, "extensions.", &ext)) {
-		data->has_extensions = 1;
 		/*
 		 * record any known extensions here; otherwise,
 		 * we fall through to recording it as unknown, and
 		 * check_repository_format will complain
 		 */
+		int is_unallowed_extension = 1;
+
 		if (!strcmp(ext, "noop"))
 			;
 		else if (!strcmp(ext, "preciousobjects"))
@@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 			if (!value)
 				return config_error_nonbool(var);
 			data->partial_clone = xstrdup(value);
-		} else if (!strcmp(ext, "worktreeconfig"))
+		} else if (!strcmp(ext, "worktreeconfig")) {
 			data->worktree_config = git_config_bool(var, value);
-		else
+			is_unallowed_extension = 0;
+		} else
 			string_list_append(&data->unknown_extensions, ext);
+
+		data->has_unallowed_extensions |= is_unallowed_extension;
 	}
 
 	return read_worktree_config(var, value, vdata);
@@ -560,7 +564,7 @@ int upgrade_repository_format(int target_version)
 		return 0;
 
 	if (verify_repository_format(&repo_fmt, &err) < 0 ||
-	    (!repo_fmt.version && repo_fmt.has_extensions)) {
+	    (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {
 		warning("unable to upgrade repository format from %d to %d: %s",
 			repo_fmt.version, target_version, err.buf);
 		strbuf_release(&err);
diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 88cdde255c..6e339c7c8e 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -68,6 +68,18 @@ test_expect_success 'git sparse-checkout init' '
 	check_files repo a
 '
 
+test_expect_success 'git sparse-checkout works if repository format is wrong' '
+	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
+	git -C repo config --unset core.repositoryFormatVersion &&
+	git -C repo sparse-checkout init &&
+	git -C repo config core.repositoryFormatVersion >actual &&
+	echo 1 >expect &&
+	git -C repo config core.repositoryFormatVersion 0 &&
+	git -C repo sparse-checkout init &&
+	git -C repo config core.repositoryFormatVersion >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git sparse-checkout list after init' '
 	git -C repo sparse-checkout list >actual &&
 	cat >expect <<-\EOF &&
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH v2 2/2] config: provide extra detail about worktree config
  2020-07-14  1:26 ` [PATCH v2 0/2] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
  2020-07-14  1:26   ` [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig Derrick Stolee via GitGitGadget
@ 2020-07-14  1:26   ` Derrick Stolee via GitGitGadget
  2020-07-14 20:24   ` [PATCH v2 0/2] setup: warn if extensions exist on old format Johannes Schindelin
  2 siblings, 0 replies; 15+ messages in thread
From: Derrick Stolee via GitGitGadget @ 2020-07-14  1:26 UTC (permalink / raw)
  To: git
  Cc: newren, delphij, peff, johannes.schindelin, me, Derrick Stolee,
	Derrick Stolee

From: Derrick Stolee <dstolee@microsoft.com>

When any `extensions.*` setting is configured, we newly ignore it unless
`core.repositoryFormatVersion` is set to a positive value.

This might be quite surprising, e.g. when calling `git config --worktree
[...]` elicits a warning that it requires
`extensions.worktreeConfig = true` when that setting _is_ configured
(but ignored because `core.repositoryFormatVersion` is unset).

Let's warn about this situation specifically, especially because there
might be already setups out there that configured a sparse worktree
using Git v2.27.0 (which does set `extensions.worktreeConfig` but not
`core.repositoryFormatVersion`) and users might want to work in those
setups with Git v2.28.0, too.

This warning is specifically placed inside an existing error message
for 'git config --worktree' that already fails if the repository format
version is not 1.

Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/config.c           |  5 +++--
 t/t2404-worktree-config.sh | 12 ++++++++++++
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/builtin/config.c b/builtin/config.c
index ee4aef6a35..2cdc293ae5 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -678,8 +678,9 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		else if (worktrees[0] && worktrees[1])
 			die(_("--worktree cannot be used with multiple "
 			      "working trees unless the config\n"
-			      "extension worktreeConfig is enabled. "
-			      "Please read \"CONFIGURATION FILE\"\n"
+			      "extension worktreeConfig is enabled "
+			      "and core.repositoryFormatVersion is at least\n"
+			      "1. Please read \"CONFIGURATION FILE\""
 			      "section in \"git help worktree\" for details"));
 		else
 			given_config_source.file = git_pathdup("config");
diff --git a/t/t2404-worktree-config.sh b/t/t2404-worktree-config.sh
index 9536d10919..303a2644bd 100755
--- a/t/t2404-worktree-config.sh
+++ b/t/t2404-worktree-config.sh
@@ -77,5 +77,17 @@ test_expect_success 'config.worktree no longer read without extension' '
 	test_cmp_config -C wt1 shared this.is &&
 	test_cmp_config -C wt2 shared this.is
 '
+test_expect_success 'show advice when extensions.* are not enabled' '
+	test_config core.repositoryformatversion 1 &&
+	test_config extensions.worktreeConfig true &&
+	git config --worktree test.one true &&
+	test_cmp_config true test.one &&
+	test_config core.repositoryformatversion 0 &&
+	test_must_fail git config --worktree test.two true 2>err &&
+	test_i18ngrep "extension worktreeConfig is enabled" err &&
+	git config --unset core.repositoryformatversion &&
+	test_must_fail git config --worktree test.three true 2>err &&
+	test_i18ngrep "core.repositoryFormatVersion is at least" err
+'
 
 test_done
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
  2020-07-13 20:46       ` Derrick Stolee
  2020-07-13 21:45         ` Junio C Hamano
@ 2020-07-14  4:06         ` Jonathan Nieder
  2020-07-14  4:27           ` Junio C Hamano
  1 sibling, 1 reply; 15+ messages in thread
From: Jonathan Nieder @ 2020-07-14  4:06 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: Junio C Hamano, Derrick Stolee via GitGitGadget, git, newren,
	delphij, peff, Derrick Stolee, Derrick Stolee

Hi,

Derrick Stolee wrote:
> On 7/13/2020 4:18 PM, Junio C Hamano wrote:

>> The "fail and warn" approach introduced in the previous step may be
>> with smaller impact to the codebase, but
>>
>>  - it requires the end-user to read, understand and execute the
>>    manual upgrade
>>
>>  - it encourages to follow the same procedure blindly, making the
>>    protection even less useful
>>
>> Let's instead keep failing hard without teaching how to bypass the
>> repository protection, but allow upgrading even when only the
>> worktreeconfig extension exists in an old repository, which is
>> likely to be set by a broke version of Git that did not update the
>> repository version when setting the extension.
>
> This is a more subtle way to handle the case. In fact, it
> silently makes extensions.worktreeConfig work as it did in 2.27,
> which means users will not have any troubles after upgrading.

I'd like to propose a different endgame:

Instead of looking at `extensions.*` settings one by one to see how
various implementations handled them with repositoryFormatVersion=0,
what if we treat repositoryFormatVersion=0 as a synonym of version=1?

That is:

1. in new repositories, set repositoryFormatVersion = 1, since (1) Git
   versions in the wild should all already know how to handle it and
   (2) as we've learned, other Git implementations need to understand
   some of extensions.* anyway

2. whenever setting any extensions.*, automatically upgrade to
   repositoryFormatVersion = 1

3. when in an existing repository with extensions.* set and
   repositoryFormatVersion = 0, act as though repositoryFormatVersion = 1

4. document this new behavior with repositoryFormatVersion = 0 in
   Documentation/technical/repository-version.txt

This way, the result is simpler than where we started.

Unfortunately, we are not even that consistent about what to do with
extensions.* settings when repositoryFormatVersion = 0 today.  So
we'll have to exercise some care and analyze them one by one to make
sure this is safe (and if it isn't, at *that* point we'd come up with
a more complex variant on (2) and (3) above).

It's too late to go that far for 2.28.  It would be tempting to try a
simple revert of 14c7fa269e4 (check_repository_format_gently(): refuse
extensions for old repositories, 2020-06-05) to get back to tried and
true behavior but that does not do enough --- it still produces an
error when trying to upgrade repository format when any extensions are
set.  So how about such a revert plus Junio's patch plus the analogous
change to Junio's patch for

  extensions.preciousObjects
  extensions.partialClone

?

Thanks,
Jonathan

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
  2020-07-14  4:06         ` Jonathan Nieder
@ 2020-07-14  4:27           ` Junio C Hamano
  0 siblings, 0 replies; 15+ messages in thread
From: Junio C Hamano @ 2020-07-14  4:27 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: Derrick Stolee, Derrick Stolee via GitGitGadget, git, newren,
	delphij, peff, Derrick Stolee, Derrick Stolee

Jonathan Nieder <jrnieder@gmail.com> writes:

> It's too late to go that far for 2.28.  It would be tempting to try a
> simple revert of 14c7fa269e4 (check_repository_format_gently(): refuse
> extensions for old repositories, 2020-06-05) to get back to tried and
> true behavior but that does not do enough --- it still produces an
> error when trying to upgrade repository format when any extensions are
> set.  So how about such a revert plus Junio's patch plus the analogous
> change to Junio's patch for
>
>   extensions.preciousObjects
>   extensions.partialClone

My illustration patch was done "assuming that worktreeconfig is the
only thing we wrote by mistake without updating the format version",
and if these two also share the same problem, I obviously is 100%
fine with covering these other ones with the same approach.

I like your "v0 and v1 are the same, but the repository is declared
to be corrupt if the extentions that are not known by today's code
is found in v0 repository", by the way.  Assuming that the two you
listed above plus worktreeconfig are the only ones known by today's
code, that is.  We seem to also know about "noop", so shouldn't it
also be grandfathered in?


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig
  2020-07-14  1:26   ` [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig Derrick Stolee via GitGitGadget
@ 2020-07-14 20:22     ` Johannes Schindelin
  0 siblings, 0 replies; 15+ messages in thread
From: Johannes Schindelin @ 2020-07-14 20:22 UTC (permalink / raw)
  To: Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, me, Derrick Stolee, Derrick Stolee

Hi Stolee,

On Tue, 14 Jul 2020, Derrick Stolee via GitGitGadget wrote:

> This is because the logic in 14c7fa269e4 refuses to upgrae repos when
> the version is unset but extensions exist.

s/upgrae/upgrade/

The rest looks good to me.

Thank you for working on this,
Dscho

> One possible way to alert a user of this issue is to warn them when Git
> notices an extension exists, but core.repositoryFormatVersion is not a
> correct value. However,
>
>  - it requires the end-user to read, understand and execute the
>    manual upgrade
>
>  - it encourages to follow the same procedure blindly, making the
>    protection even less useful
>
> Let's instead keep failing hard without teaching how to bypass the
> repository protection, but allow upgrading even when only the
> worktreeconfig extension exists in an old repository, which is
> likely to be set by a broke version of Git that did not update the
> repository version when setting the extension.
>
> This change of behavior is made visible by testing how 'git
> sparse-checkout init' behaves to upgrade the repository format version
> even if the extension.worktreeConfig is already set. This would
> previously fail without a clear way forward.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  cache.h                            |  2 +-
>  setup.c                            | 12 ++++++++----
>  t/t1091-sparse-checkout-builtin.sh | 12 ++++++++++++
>  3 files changed, 21 insertions(+), 5 deletions(-)
>
> diff --git a/cache.h b/cache.h
> index e5885cc9ea..8ff46857f6 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1042,8 +1042,8 @@ struct repository_format {
>  	int worktree_config;
>  	int is_bare;
>  	int hash_algo;
> -	int has_extensions;
>  	char *work_tree;
> +	int has_unallowed_extensions;
>  	struct string_list unknown_extensions;
>  };
>
> diff --git a/setup.c b/setup.c
> index eb066db6d8..65270440a9 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
>  	if (strcmp(var, "core.repositoryformatversion") == 0)
>  		data->version = git_config_int(var, value);
>  	else if (skip_prefix(var, "extensions.", &ext)) {
> -		data->has_extensions = 1;
>  		/*
>  		 * record any known extensions here; otherwise,
>  		 * we fall through to recording it as unknown, and
>  		 * check_repository_format will complain
>  		 */
> +		int is_unallowed_extension = 1;
> +
>  		if (!strcmp(ext, "noop"))
>  			;
>  		else if (!strcmp(ext, "preciousobjects"))
> @@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
>  			if (!value)
>  				return config_error_nonbool(var);
>  			data->partial_clone = xstrdup(value);
> -		} else if (!strcmp(ext, "worktreeconfig"))
> +		} else if (!strcmp(ext, "worktreeconfig")) {
>  			data->worktree_config = git_config_bool(var, value);
> -		else
> +			is_unallowed_extension = 0;
> +		} else
>  			string_list_append(&data->unknown_extensions, ext);
> +
> +		data->has_unallowed_extensions |= is_unallowed_extension;
>  	}
>
>  	return read_worktree_config(var, value, vdata);
> @@ -560,7 +564,7 @@ int upgrade_repository_format(int target_version)
>  		return 0;
>
>  	if (verify_repository_format(&repo_fmt, &err) < 0 ||
> -	    (!repo_fmt.version && repo_fmt.has_extensions)) {
> +	    (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {
>  		warning("unable to upgrade repository format from %d to %d: %s",
>  			repo_fmt.version, target_version, err.buf);
>  		strbuf_release(&err);
> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
> index 88cdde255c..6e339c7c8e 100755
> --- a/t/t1091-sparse-checkout-builtin.sh
> +++ b/t/t1091-sparse-checkout-builtin.sh
> @@ -68,6 +68,18 @@ test_expect_success 'git sparse-checkout init' '
>  	check_files repo a
>  '
>
> +test_expect_success 'git sparse-checkout works if repository format is wrong' '
> +	test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
> +	git -C repo config --unset core.repositoryFormatVersion &&
> +	git -C repo sparse-checkout init &&
> +	git -C repo config core.repositoryFormatVersion >actual &&
> +	echo 1 >expect &&
> +	git -C repo config core.repositoryFormatVersion 0 &&
> +	git -C repo sparse-checkout init &&
> +	git -C repo config core.repositoryFormatVersion >actual &&
> +	test_cmp expect actual
> +'
> +
>  test_expect_success 'git sparse-checkout list after init' '
>  	git -C repo sparse-checkout list >actual &&
>  	cat >expect <<-\EOF &&
> --
> gitgitgadget
>
>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH v2 0/2] setup: warn if extensions exist on old format
  2020-07-14  1:26 ` [PATCH v2 0/2] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
  2020-07-14  1:26   ` [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig Derrick Stolee via GitGitGadget
  2020-07-14  1:26   ` [PATCH v2 2/2] config: provide extra detail about worktree config Derrick Stolee via GitGitGadget
@ 2020-07-14 20:24   ` Johannes Schindelin
  2 siblings, 0 replies; 15+ messages in thread
From: Johannes Schindelin @ 2020-07-14 20:24 UTC (permalink / raw)
  To: Derrick Stolee via GitGitGadget
  Cc: git, newren, delphij, peff, me, Derrick Stolee

Hi Stolee,

On Tue, 14 Jul 2020, Derrick Stolee via GitGitGadget wrote:

>  2. A second patch has joined the fray, hopefully answering the concerned
>     raise by Johannes [1].

It answers my concern!

Thank you for providing these patches,
Dscho

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2020-07-14 20:24 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-13 19:20 [PATCH] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
2020-07-13 19:34 ` Taylor Blau
2020-07-13 19:41   ` Derrick Stolee
2020-07-13 19:36 ` Junio C Hamano
2020-07-13 20:00   ` Junio C Hamano
2020-07-13 20:18     ` [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig Junio C Hamano
2020-07-13 20:46       ` Derrick Stolee
2020-07-13 21:45         ` Junio C Hamano
2020-07-14  4:06         ` Jonathan Nieder
2020-07-14  4:27           ` Junio C Hamano
2020-07-14  1:26 ` [PATCH v2 0/2] setup: warn if extensions exist on old format Derrick Stolee via GitGitGadget
2020-07-14  1:26   ` [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig Derrick Stolee via GitGitGadget
2020-07-14 20:22     ` Johannes Schindelin
2020-07-14  1:26   ` [PATCH v2 2/2] config: provide extra detail about worktree config Derrick Stolee via GitGitGadget
2020-07-14 20:24   ` [PATCH v2 0/2] setup: warn if extensions exist on old format Johannes Schindelin

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).