git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Christian Couder <christian.couder@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Karsten Blees" <karsten.blees@gmail.com>,
	"Nguyen Thai Ngoc Duy" <pclouds@gmail.com>,
	"Stefan Beller" <sbeller@google.com>,
	"Eric Sunshine" <sunshine@sunshineco.com>,
	"Ramsay Jones" <ramsay@ramsayjones.plus.com>,
	"Johannes Sixt" <j6t@kdbg.org>, "René Scharfe" <l.s.r@web.de>,
	"Christian Couder" <chriscool@tuxfamily.org>
Subject: [PATCH v10 40/40] builtin/am: use apply api in run_apply()
Date: Mon,  8 Aug 2016 23:03:37 +0200	[thread overview]
Message-ID: <20160808210337.5038-41-chriscool@tuxfamily.org> (raw)
In-Reply-To: <20160808210337.5038-1-chriscool@tuxfamily.org>

This replaces run_apply() implementation with a new one that
uses the apply api that has been previously prepared in
apply.c and apply.h.

This shoud improve performance a lot in certain cases.

As the previous implementation was creating a new `git apply`
process to apply each patch, it could be slow on systems like
Windows where it is costly to create new processes.

Also the new `git apply` process had to read the index from
disk, and when the process was done the calling process
discarded its own index and read back from disk the new
index that had been created by the `git apply` process.

This could be very inefficient with big repositories that
have big index files, especially when the system decided
that it was a good idea to run the `git apply` processes on
a different processor core.

Also eliminating index reads enables further performance
improvements by using:

`git update-index --split-index`

For example here is a benchmark of a multi hundred commit
rebase on the Linux kernel on a Debian laptop with SSD:

command: git rebase --onto 1993b17 52bef0c 29dde7c

Vanilla "next" without split index:                1m54.953s
Vanilla "next" with split index:                   1m22.476s
This series on top of "next" without split index:  1m12.034s
This series on top of "next" with split index:     0m15.678s

(using branch "next" from mid April 2016.)

Benchmarked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 builtin/am.c | 65 +++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 47 insertions(+), 18 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index b77bf11..54c5728 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -28,6 +28,7 @@
 #include "rerere.h"
 #include "prompt.h"
 #include "mailinfo.h"
+#include "apply.h"
 
 /**
  * Returns 1 if the file is empty or does not exist, 0 otherwise.
@@ -1522,39 +1523,67 @@ static int parse_mail_rebase(struct am_state *state, const char *mail)
  */
 static int run_apply(const struct am_state *state, const char *index_file)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
+	struct argv_array apply_paths = ARGV_ARRAY_INIT;
+	struct argv_array apply_opts = ARGV_ARRAY_INIT;
+	struct apply_state apply_state;
+	int res, opts_left;
+	char *save_index_file;
+	static struct lock_file lock_file;
+	int force_apply = 0;
+	int options = 0;
+
+	if (index_file) {
+		save_index_file = get_index_file();
+		set_index_file((char *)index_file);
+	}
 
-	cp.git_cmd = 1;
+	if (init_apply_state(&apply_state, NULL, &lock_file))
+		die("BUG: init_apply_state() failed");
+
+	argv_array_push(&apply_opts, "apply");
+	argv_array_pushv(&apply_opts, state->git_apply_opts.argv);
+
+	opts_left = apply_parse_options(apply_opts.argc, apply_opts.argv,
+					&apply_state, &force_apply, &options,
+					NULL);
+
+	if (opts_left != 0)
+		die("unknown option passed thru to git apply");
 
 	if (index_file)
-		argv_array_pushf(&cp.env_array, "GIT_INDEX_FILE=%s", index_file);
+		apply_state.cached = 1;
+	else
+		apply_state.check_index = 1;
 
 	/*
 	 * If we are allowed to fall back on 3-way merge, don't give false
 	 * errors during the initial attempt.
 	 */
-	if (state->threeway && !index_file) {
-		cp.no_stdout = 1;
-		cp.no_stderr = 1;
-	}
+	if (state->threeway && !index_file)
+		apply_state.apply_verbosity = verbosity_silent;
 
-	argv_array_push(&cp.args, "apply");
+	if (check_apply_state(&apply_state, force_apply))
+		die("BUG: check_apply_state() failed");
 
-	argv_array_pushv(&cp.args, state->git_apply_opts.argv);
+	argv_array_push(&apply_paths, am_path(state, "patch"));
+
+	res = apply_all_patches(&apply_state, apply_paths.argc, apply_paths.argv, options);
 
 	if (index_file)
-		argv_array_push(&cp.args, "--cached");
-	else
-		argv_array_push(&cp.args, "--index");
+		set_index_file(save_index_file);
 
-	argv_array_push(&cp.args, am_path(state, "patch"));
+	argv_array_clear(&apply_paths);
+	argv_array_clear(&apply_opts);
+	clear_apply_state(&apply_state);
 
-	if (run_command(&cp))
-		return -1;
+	if (res)
+		return res;
 
-	/* Reload index as git-apply will have modified it. */
-	discard_cache();
-	read_cache_from(index_file ? index_file : get_index_file());
+	if (index_file) {
+		/* Reload index as apply_all_patches() will have modified it. */
+		discard_cache();
+		read_cache_from(index_file);
+	}
 
 	return 0;
 }
-- 
2.9.2.614.g4980f51


  parent reply	other threads:[~2016-08-08 21:05 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-08-08 21:02 [PATCH v10 00/40] libify apply and use lib in am, part 2 Christian Couder
2016-08-08 21:02 ` [PATCH v10 01/40] apply: make some names more specific Christian Couder
2016-08-09 14:51   ` stefan.naewe
2016-08-11  8:40     ` Christian Couder
2016-08-11  8:55       ` stefan.naewe
2016-08-08 21:02 ` [PATCH v10 02/40] apply: move 'struct apply_state' to apply.h Christian Couder
2016-08-08 21:03 ` [PATCH v10 03/40] builtin/apply: make apply_patch() return -1 or -128 instead of die()ing Christian Couder
2016-08-08 21:03 ` [PATCH v10 04/40] builtin/apply: read_patch_file() return -1 " Christian Couder
2016-08-08 21:03 ` [PATCH v10 05/40] builtin/apply: make find_header() return -128 " Christian Couder
2016-08-08 21:03 ` [PATCH v10 06/40] builtin/apply: make parse_chunk() return a negative integer on error Christian Couder
2016-08-08 21:03 ` [PATCH v10 07/40] builtin/apply: make parse_single_patch() return -1 " Christian Couder
2016-08-08 21:03 ` [PATCH v10 08/40] builtin/apply: make parse_whitespace_option() return -1 instead of die()ing Christian Couder
2016-08-08 21:03 ` [PATCH v10 09/40] builtin/apply: make parse_ignorewhitespace_option() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 10/40] builtin/apply: move init_apply_state() to apply.c Christian Couder
2016-08-08 21:03 ` [PATCH v10 11/40] apply: make init_apply_state() return -1 instead of exit()ing Christian Couder
2016-08-08 21:03 ` [PATCH v10 12/40] builtin/apply: make check_apply_state() return -1 instead of die()ing Christian Couder
2016-08-08 21:03 ` [PATCH v10 13/40] builtin/apply: move check_apply_state() to apply.c Christian Couder
2016-08-08 21:03 ` [PATCH v10 14/40] builtin/apply: make apply_all_patches() return 128 or 1 on error Christian Couder
2016-08-08 21:03 ` [PATCH v10 15/40] builtin/apply: make parse_traditional_patch() return -1 " Christian Couder
2016-08-08 21:03 ` [PATCH v10 16/40] builtin/apply: make gitdiff_*() return 1 at end of header Christian Couder
2016-08-08 21:03 ` [PATCH v10 17/40] builtin/apply: make gitdiff_*() return -1 on error Christian Couder
2016-08-08 21:03 ` [PATCH v10 18/40] builtin/apply: change die_on_unsafe_path() to check_unsafe_path() Christian Couder
2016-08-08 21:03 ` [PATCH v10 19/40] builtin/apply: make build_fake_ancestor() return -1 on error Christian Couder
2016-08-08 21:03 ` [PATCH v10 20/40] builtin/apply: make remove_file() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 21/40] builtin/apply: make add_conflicted_stages_file() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 22/40] builtin/apply: make add_index_file() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 23/40] builtin/apply: make create_file() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 24/40] builtin/apply: make write_out_one_result() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 25/40] builtin/apply: make write_out_results() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 26/40] builtin/apply: make try_create_file() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 27/40] builtin/apply: make create_one_file() " Christian Couder
2016-08-08 21:03 ` [PATCH v10 28/40] builtin/apply: rename option parsing functions Christian Couder
2016-08-09 14:55   ` stefan.naewe
2016-08-08 21:03 ` [PATCH v10 29/40] apply: rename and move opt constants to apply.h Christian Couder
2016-08-08 21:03 ` [PATCH v10 31/40] apply: make some parsing functions static again Christian Couder
2016-08-08 21:03 ` [PATCH v10 32/40] apply: use error_errno() where possible Christian Couder
2016-08-08 21:03 ` [PATCH v10 33/40] environment: add set_index_file() Christian Couder
2016-08-08 22:13   ` Junio C Hamano
2016-08-10 16:52     ` Christian Couder
2016-08-10 17:34       ` Junio C Hamano
2016-08-11 19:08         ` Christian Couder
2016-08-11 19:30           ` Junio C Hamano
2016-08-08 21:03 ` [PATCH v10 34/40] apply: make it possible to silently apply Christian Couder
2016-08-08 21:03 ` [PATCH v10 35/40] apply: don't print on stdout in verbosity_silent mode Christian Couder
2016-08-08 21:03 ` [PATCH v10 36/40] usage: add set_warn_routine() Christian Couder
2016-08-08 21:03 ` [PATCH v10 37/40] usage: add get_error_routine() and get_warn_routine() Christian Couder
2016-08-08 21:03 ` [PATCH v10 38/40] apply: change error_routine when silent Christian Couder
2016-08-08 21:03 ` [PATCH v10 39/40] apply: refactor `git apply` option parsing Christian Couder
2016-08-08 21:03 ` Christian Couder [this message]
2016-08-08 21:23 ` [PATCH v10 00/40] libify apply and use lib in am, part 2 Christian Couder
2016-08-08 22:16   ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160808210337.5038-41-chriscool@tuxfamily.org \
    --to=christian.couder@gmail.com \
    --cc=avarab@gmail.com \
    --cc=chriscool@tuxfamily.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=j6t@kdbg.org \
    --cc=karsten.blees@gmail.com \
    --cc=l.s.r@web.de \
    --cc=pclouds@gmail.com \
    --cc=peff@peff.net \
    --cc=ramsay@ramsayjones.plus.com \
    --cc=sbeller@google.com \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).