git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Matheus Tavares <matheus.bernardino@usp.br>
To: git@vger.kernel.org
Cc: christian.couder@gmail.com, git@jeffhostetler.com,
	stolee@gmail.com, tboegi@web.de
Subject: [PATCH v3 3/8] checkout-index: add parallel checkout support
Date: Tue,  4 May 2021 13:27:30 -0300	[thread overview]
Message-ID: <916f391a46a1ccedee3fca52893d0339ff37864a.1620145501.git.matheus.bernardino@usp.br> (raw)
In-Reply-To: <cover.1620145501.git.matheus.bernardino@usp.br>

Allow checkout-index to use the parallel checkout framework, honoring
the checkout.workers configuration.

There are two code paths in checkout-index which call
`checkout_entry()`, and thus, can make use of parallel checkout:
`checkout_file()`, which is used to write paths explicitly given at the
command line; and `checkout_all()`, which is used to write all paths in
the index, when the `--all` option is given.

In both operation modes, checkout-index doesn't abort immediately on a
`checkout_entry()` failure. Instead, it tries to check out all remaining
paths before exiting with a non-zero exit code. To keep this behavior
when parallel checkout is being used, we must allow
`run_parallel_checkout()` to try writing the queued entries before we
exit, even if we already got an error code from a previous
`checkout_entry()` call.

However, `checkout_all()` doesn't return on errors, it calls `exit()`
with code 128. We could make it call `run_parallel_checkout()` before
exiting, but it makes the code easier to follow if we unify the exit
path for both checkout-index modes at `cmd_checkout_index()`, and let
this function take care of the interactions with the parallel checkout
API. So let's do that.

With this change, we also have to consider whether we want to keep using
128 as the error code for `git checkout-index --all`, while we use 1 for
`git checkout-index <path>` (even when the actual error is the same).
Since there is not much value in having code 128 only for `--all`, and
there is no mention about it in the docs (so it's unlikely that changing
it will break any existing script), let's make both modes exit with code
1 on `checkout_entry()` errors.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 builtin/checkout-index.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c
index c9a3c71914..e21620d964 100644
--- a/builtin/checkout-index.c
+++ b/builtin/checkout-index.c
@@ -12,6 +12,7 @@
 #include "cache-tree.h"
 #include "parse-options.h"
 #include "entry.h"
+#include "parallel-checkout.h"
 
 #define CHECKOUT_ALL 4
 static int nul_term_line;
@@ -115,7 +116,7 @@ static int checkout_file(const char *name, const char *prefix)
 	return -1;
 }
 
-static void checkout_all(const char *prefix, int prefix_length)
+static int checkout_all(const char *prefix, int prefix_length)
 {
 	int i, errs = 0;
 	struct cache_entry *last_ce = NULL;
@@ -144,11 +145,7 @@ static void checkout_all(const char *prefix, int prefix_length)
 	}
 	if (last_ce && to_tempfile)
 		write_tempfile_record(last_ce->name, prefix);
-	if (errs)
-		/* we have already done our error reporting.
-		 * exit with the same code as die().
-		 */
-		exit(128);
+	return !!errs;
 }
 
 static const char * const builtin_checkout_index_usage[] = {
@@ -184,6 +181,7 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 	int force = 0, quiet = 0, not_new = 0;
 	int index_opt = 0;
 	int err = 0;
+	int pc_workers, pc_threshold;
 	struct option builtin_checkout_index_options[] = {
 		OPT_BOOL('a', "all", &all,
 			N_("check out all files in the index")),
@@ -238,6 +236,10 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 		hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);
 	}
 
+	get_parallel_checkout_configs(&pc_workers, &pc_threshold);
+	if (pc_workers > 1)
+		init_parallel_checkout();
+
 	/* Check out named files first */
 	for (i = 0; i < argc; i++) {
 		const char *arg = argv[i];
@@ -277,12 +279,16 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 		strbuf_release(&buf);
 	}
 
+	if (all)
+		err |= checkout_all(prefix, prefix_length);
+
+	if (pc_workers > 1)
+		err |= run_parallel_checkout(&state, pc_workers, pc_threshold,
+					     NULL, NULL);
+
 	if (err)
 		return 1;
 
-	if (all)
-		checkout_all(prefix, prefix_length);
-
 	if (is_lock_file_locked(&lock_file) &&
 	    write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die("Unable to write new index file");
-- 
2.30.1


  parent reply	other threads:[~2021-05-04 16:28 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-22 15:17 [PATCH 0/7] Parallel Checkout (part 3) Matheus Tavares
2021-04-22 15:17 ` [PATCH 1/7] make_transient_cache_entry(): optionally alloc from mem_pool Matheus Tavares
2021-04-22 15:17 ` [PATCH 2/7] builtin/checkout.c: complete parallel checkout support Matheus Tavares
2021-04-23 16:19   ` Derrick Stolee
2021-04-26 21:54     ` Matheus Tavares Bernardino
2021-04-22 15:17 ` [PATCH 3/7] checkout-index: add " Matheus Tavares
2021-04-23 18:32   ` Derrick Stolee
2021-04-26 22:30     ` Matheus Tavares Bernardino
2021-04-22 15:17 ` [PATCH 4/7] parallel-checkout: add tests for basic operations Matheus Tavares
2021-04-23 19:18   ` Derrick Stolee
2021-04-27  2:30     ` Matheus Tavares Bernardino
2021-04-22 15:17 ` [PATCH 5/7] parallel-checkout: add tests related to path collisions Matheus Tavares
2021-04-22 15:17 ` [PATCH 6/7] parallel-checkout: add tests related to .gitattributes Matheus Tavares
2021-04-23 19:48   ` Derrick Stolee
2021-04-22 15:17 ` [PATCH 7/7] ci: run test round with parallel-checkout enabled Matheus Tavares
2021-04-23 19:56   ` Derrick Stolee
2021-04-30 21:40 ` [PATCH v2 0/8] Parallel Checkout (part 3) Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 1/8] make_transient_cache_entry(): optionally alloc from mem_pool Matheus Tavares
2021-05-01 17:06     ` Christian Couder
2021-05-03 14:11       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 2/8] builtin/checkout.c: complete parallel checkout support Matheus Tavares
2021-05-01 17:08     ` Christian Couder
2021-05-03 14:21       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 3/8] checkout-index: add " Matheus Tavares
2021-05-01 17:08     ` Christian Couder
2021-05-03 14:22       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 4/8] parallel-checkout: add tests for basic operations Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 5/8] parallel-checkout: add tests related to path collisions Matheus Tavares
2021-05-02  7:59     ` Torsten Bögershausen
2021-05-03 14:58       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 6/8] t0028: extract encoding helpers to lib-encoding.sh Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 7/8] parallel-checkout: add tests related to .gitattributes Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 8/8] ci: run test round with parallel-checkout enabled Matheus Tavares
2021-05-02 10:12   ` [PATCH v2 0/8] Parallel Checkout (part 3) Torsten Bögershausen
2021-05-03 15:01     ` Matheus Tavares Bernardino
2021-05-04 16:27   ` [PATCH v3 " Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 1/8] make_transient_cache_entry(): optionally alloc from mem_pool Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 2/8] builtin/checkout.c: complete parallel checkout support Matheus Tavares
2021-05-05 13:55       ` Derrick Stolee
2021-05-04 16:27     ` Matheus Tavares [this message]
2021-05-04 16:27     ` [PATCH v3 4/8] parallel-checkout: add tests for basic operations Matheus Tavares
2021-05-26 18:36       ` AIX failures on parallel checkout (new in v2.32.0-rc*) Ævar Arnfjörð Bjarmason
2021-05-26 22:01         ` Matheus Tavares Bernardino
2021-05-26 23:00           ` Junio C Hamano
2021-05-04 16:27     ` [PATCH v3 5/8] parallel-checkout: add tests related to path collisions Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 6/8] t0028: extract encoding helpers to lib-encoding.sh Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 7/8] parallel-checkout: add tests related to .gitattributes Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 8/8] ci: run test round with parallel-checkout enabled Matheus Tavares
2021-05-05 13:57     ` [PATCH v3 0/8] Parallel Checkout (part 3) Derrick Stolee
2021-05-06  0:40       ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=916f391a46a1ccedee3fca52893d0339ff37864a.1620145501.git.matheus.bernardino@usp.br \
    --to=matheus.bernardino@usp.br \
    --cc=christian.couder@gmail.com \
    --cc=git@jeffhostetler.com \
    --cc=git@vger.kernel.org \
    --cc=stolee@gmail.com \
    --cc=tboegi@web.de \
    --subject='Re: [PATCH v3 3/8] checkout-index: add parallel checkout support' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.version-control.git
	nntp://ie5yzdi7fg72h7s4sdcztq5evakq23rdt33mfyfcddc5u3ndnw24ogqd.onion/inbox.comp.version-control.git
	nntp://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git