From: Glen Choo <chooglen@google.com>
To: git@vger.kernel.org
Cc: Glen Choo <chooglen@google.com>,
Jonathan Tan <jonathantanmy@google.com>,
Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v4 0/3] fetch: skip unnecessary tasks when using --negotiate-only
Date: Wed, 12 Jan 2022 16:44:58 -0800 [thread overview]
Message-ID: <20220113004501.78822-1-chooglen@google.com> (raw)
In-Reply-To: <20211222001134.28933-1-chooglen@google.com>
`git fetch --negotiate-only` is used internally by push negotation and
it behaves very differently from other uses of `git fetch`, e.g. it does
not update refs or fetch objects. But because of how cmd_fetch() is
written, `git fetch --negotiate-only` performs tasks that it shouldn't.
This is results in behavior that is unnecessary at best, and incorrect
at worst:
* Submodules are updated if enabled by recurse.submodules=true, but
negotiation fetch doesn't actually update the repo, so this doesn't
make sense.
* Commit graphs will be written if enabled by
fetch.writeCommitGraph=true, but this is unnecessary because no
objects are fetched [1].
* gc is run, but according to the commit message in [2], we only do this
because we expect `git fetch` to introduce objects.
Make `git fetch --negotiate-only` handle these tasks more rigorously by
doing the following:
* Make cmd_fetch() skip irrelevant tasks if we know for certain that
objects will not be fetched
* Disable submodule recursion and die() if a user explicitly asks for it
[1] This is also confirmed by Documentation/config/fetch.txt, which
states that Git should only write commit graphs if a pack-file is
downloaded.
[2] 131b8fcbfb (fetch: run gc --auto after fetching, 2013-01-26)
Changes since v3:
* change commit message subject: builtin/fetch -> fetch --negotiate-only
* move the 'goto cleanup' to _after_ the submodule updating task because
we may want to update submodules even if objects were not fetched (as
pointed out by Junio, thanks!)
* disable submodule recursion in the patch that checks for
--negotiate-only + --recurse-submodules, so we never silently ignore
--recurse-submodules.
* incorporate some of Jonathan's suggestions (thanks!)
Changes since v2:
* added a prepatory patch that introduces a "goto cleanup"
* drop an unnecessary line move (as suggested by Junio)
* check for user-given --recurse-submodules and die() (as suggested by
Jonathan and Junio)
* update --negotiate-only's documentation
Changes since v1:
* added more context to commit message
* added a NEEDSWORK comment
Glen Choo (3):
fetch: use goto cleanup in cmd_fetch()
fetch: skip tasks related to fetching objects
fetch --negotiate-only: do not update submodules
Documentation/fetch-options.txt | 1 +
builtin/fetch.c | 40 ++++++++++++++++++++++++++++++---
t/t5516-fetch-push.sh | 12 ++++++++++
t/t5702-protocol-v2.sh | 12 ++++++++++
4 files changed, 62 insertions(+), 3 deletions(-)
Range-diff against v3:
1: 97e2cba633 ! 1: ffa1a24109 builtin/fetch: use goto cleanup in cmd_fetch()
@@ Metadata
Author: Glen Choo <chooglen@google.com>
## Commit message ##
- builtin/fetch: use goto cleanup in cmd_fetch()
+ fetch: use goto cleanup in cmd_fetch()
Replace an early return with 'goto cleanup' in cmd_fetch() so that the
- string_list is always cleared.
-
- The string_list_clear() call is purely cleanup; the string_list was not
- reused.
+ string_list is always cleared (the string_list_clear() call is purely
+ cleanup; the string_list is not reused). This makes cleanup consistent
+ so that a subsequent commit can use 'goto cleanup' to bail out early.
Signed-off-by: Glen Choo <chooglen@google.com>
@@ builtin/fetch.c
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
gtransport->smart_options->acked_commits = &acked_commits;
} else {
- warning(_("Protocol does not support --negotiate-only, exiting."));
+ warning(_("protocol does not support --negotiate-only, exiting"));
- return 1;
+ result = 1;
+ goto cleanup;
2: 3a3a04b649 < -: ---------- builtin/fetch: skip unnecessary tasks when using --negotiate-only
-: ---------- > 2: b0c73e8135 fetch: skip tasks related to fetching objects
3: 97792b5214 ! 3: 914d30866f builtin/fetch: die on --negotiate-only and --recurse-submodules
@@ Metadata
Author: Glen Choo <chooglen@google.com>
## Commit message ##
- builtin/fetch: die on --negotiate-only and --recurse-submodules
+ fetch --negotiate-only: do not update submodules
- The previous commit ignores the value of --recurse-submodules if
- --negotiate-only is given. Since non "no" values of --recurse-submodules
- are not supported with --negotiate-only, make cmd_fetch() check for
- this invalid combination and die.
+ `git fetch --negotiate-only` is an implementation detail of push
+ negotiation and, unlike most `git fetch` invocations, does not actually
+ update the main repository. Thus it should not update submodules even
+ if submodule recursion is enabled.
- This is unlikely to affect internal usage of --negotiate-only, but it
- may be helpful for users. We may also want to discourage users from
- using --negotiate-only altogether because it was not intended for them.
+ This is not just slow, it is wrong e.g. push negotiation with
+ "submodule.recurse=true" will cause submodules to be updated because it
+ invokes `git fetch --negotiate-only`.
+
+ Fix this by disabling submodule recursion if --negotiate-only was given.
+ Since this makes --negotiate-only and --recurse-submodules incompatible,
+ check for this invalid combination and die.
+
+ This does not use the "goto cleanup" introduced in the previous commit
+ because we want to recurse through submodules whenever a ref is fetched,
+ and this can happen without introducing new objects.
Signed-off-by: Glen Choo <chooglen@google.com>
@@ Documentation/fetch-options.txt: configuration variables documented in linkgit:g
ancestors of the provided `--negotiation-tip=*` arguments,
which we have in common with the server.
+
-+This is incompatible with `--recurse-submodules[=yes|on-demand]`.
++This is incompatible with `--recurse-submodules=[yes|on-demand]`.
Internally this is used to implement the `push.negotiate` option, see
linkgit:git-config[1].
@@ builtin/fetch.c: static struct transport *gtransport;
static struct transport *gsecondary;
static const char *submodule_prefix = "";
static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
-+static int recurse_submodules_explicit = RECURSE_SUBMODULES_DEFAULT;
++static int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
static int shown_url = 0;
static struct refspec refmap = REFSPEC_INIT_FETCH;
@@ builtin/fetch.c: static struct option builtin_fetch_options[] = {
OPT_BOOL('P', "prune-tags", &prune_tags,
N_("prune local tags no longer on remote and clobber changed tags")),
- OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
-+ OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_explicit, N_("on-demand"),
++ OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"),
N_("control recursive fetching of submodules"),
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+
argc = parse_options(argc, argv, prefix,
builtin_fetch_options, builtin_fetch_usage, 0);
-
-+ if (recurse_submodules_explicit != RECURSE_SUBMODULES_DEFAULT)
-+ recurse_submodules = recurse_submodules_explicit;
+
- if (negotiate_only) {
-- /*
-- * --negotiate-only should never recurse into
-- * submodules, so there is no need to read .gitmodules.
-- */
-- recurse_submodules = RECURSE_SUBMODULES_OFF;
-+ switch (recurse_submodules_explicit) {
++ if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT)
++ recurse_submodules = recurse_submodules_cli;
++
++ if (negotiate_only) {
++ switch (recurse_submodules_cli) {
+ case RECURSE_SUBMODULES_OFF:
+ case RECURSE_SUBMODULES_DEFAULT: {
+ /*
+ * --negotiate-only should never recurse into
-+ * submodules, so there is no need to read .gitmodules.
++ * submodules. Skip it by setting recurse_submodules to
++ * RECURSE_SUBMODULES_OFF.
+ */
+ recurse_submodules = RECURSE_SUBMODULES_OFF;
+ break;
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ default:
+ die(_("--negotiate-only and --recurse-submodules cannot be used together"));
+ }
- }
-
++ }
++
if (recurse_submodules != RECURSE_SUBMODULES_OFF) {
+ int *sfjc = submodule_fetch_jobs_config == -1
+ ? &submodule_fetch_jobs_config : NULL;
+
+ ## t/t5516-fetch-push.sh ##
+@@ t/t5516-fetch-push.sh: test_expect_success 'push with negotiation proceeds anyway even if negotiation f
+ test_i18ngrep "push negotiation failed" err
+ '
+
++test_expect_success 'push with negotiation does not attempt to fetch submodules' '
++ mk_empty submodule_upstream &&
++ test_commit -C submodule_upstream submodule_commit &&
++ git submodule add ./submodule_upstream submodule &&
++ mk_empty testrepo &&
++ git push testrepo $the_first_commit:refs/remotes/origin/first_commit &&
++ test_commit -C testrepo unrelated_commit &&
++ git -C testrepo config receive.hideRefs refs/remotes/origin/first_commit &&
++ git -c submodule.recurse=true -c protocol.version=2 -c push.negotiate=1 push testrepo refs/heads/main:refs/remotes/origin/main 2>err &&
++ ! grep "Fetching submodule" err
++'
++
+ test_expect_success 'push without wildcard' '
+ mk_empty testrepo &&
+
## t/t5702-protocol-v2.sh ##
@@ t/t5702-protocol-v2.sh: test_expect_success 'usage: --negotiate-only without --negotiation-tip' '
@@ t/t5702-protocol-v2.sh: test_expect_success 'usage: --negotiate-only without --n
'
+test_expect_success 'usage: --negotiate-only with --recurse-submodules' '
-+ SERVER="server" &&
-+ URI="file://$(pwd)/server" &&
-+
-+ setup_negotiate_only "$SERVER" "$URI" &&
-+
+ cat >err.expect <<-\EOF &&
+ fatal: --negotiate-only and --recurse-submodules cannot be used together
+ EOF
base-commit: 90d242d36e248acfae0033274b524bfa55a947fd
--
2.33.GIT
next prev parent reply other threads:[~2022-01-13 0:45 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-07 19:29 [PATCH] builtin/fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2021-12-09 22:12 ` Jonathan Tan
2021-12-09 22:36 ` Glen Choo
2021-12-13 22:58 ` Jonathan Tan
2021-12-16 18:11 ` Glen Choo
2021-12-17 0:02 ` [PATCH v2] " Glen Choo
2021-12-17 23:35 ` Junio C Hamano
2021-12-20 19:37 ` Glen Choo
2021-12-20 19:56 ` Junio C Hamano
2021-12-20 20:54 ` Glen Choo
2021-12-20 22:12 ` Junio C Hamano
2021-12-21 0:18 ` Glen Choo
2021-12-21 23:07 ` Glen Choo
2021-12-22 0:11 ` [PATCH v3 0/3] " Glen Choo
2021-12-22 0:11 ` [PATCH v3 1/3] builtin/fetch: use goto cleanup in cmd_fetch() Glen Choo
2021-12-22 0:11 ` [PATCH v3 2/3] builtin/fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2021-12-22 6:42 ` Junio C Hamano
2021-12-22 17:28 ` Glen Choo
2021-12-22 19:29 ` Junio C Hamano
2021-12-22 20:27 ` Glen Choo
2021-12-22 0:11 ` [PATCH v3 3/3] builtin/fetch: die on --negotiate-only and --recurse-submodules Glen Choo
2021-12-22 6:46 ` Junio C Hamano
2021-12-23 19:08 ` Jonathan Tan
2022-01-13 0:44 ` Glen Choo [this message]
2022-01-13 0:44 ` [PATCH v4 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-13 0:45 ` [PATCH v4 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-13 0:45 ` [PATCH v4 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-13 1:16 ` Junio C Hamano
2022-01-18 18:54 ` [PATCH v5 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-18 18:54 ` [PATCH v5 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-18 18:54 ` [PATCH v5 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-18 18:54 ` [PATCH v5 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-18 22:05 ` Junio C Hamano
2022-01-18 23:41 ` Glen Choo
2022-01-19 0:26 ` Junio C Hamano
2022-01-19 0:00 ` [PATCH v6 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-19 0:00 ` [PATCH v6 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-19 0:00 ` [PATCH v6 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-19 0:00 ` [PATCH v6 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-20 2:38 ` Jiang Xin
2022-01-20 17:40 ` Glen Choo
2022-01-20 17:49 ` [PATCH v7 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-20 17:49 ` [PATCH v7 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-20 17:49 ` [PATCH v7 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-20 17:49 ` [PATCH v7 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-20 23:08 ` Junio C Hamano
2022-01-20 23:16 ` Glen Choo
2022-01-20 21:58 ` Re* [PATCH v7 0/3] fetch: skip unnecessary tasks when using --negotiate-only Junio C Hamano
2022-01-20 23:15 ` Glen Choo
2022-01-21 2:17 ` Jiang Xin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220113004501.78822-1-chooglen@google.com \
--to=chooglen@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jonathantanmy@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).