git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Christian Couder <christian.couder@gmail.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>,
	Ben Peart <Ben.Peart@microsoft.com>,
	Jonathan Tan <jonathantanmy@google.com>,
	Nguyen Thai Ngoc Duy <pclouds@gmail.com>,
	Mike Hommey <mh@glandium.org>,
	Lars Schneider <larsxschneider@gmail.com>,
	Eric Wong <e@80x24.org>,
	Christian Couder <chriscool@tuxfamily.org>,
	Jeff Hostetler <jeffhost@microsoft.com>
Subject: [PATCH v2 33/36] odb-helper: add have_object_process()
Date: Mon, 19 Mar 2018 14:31:44 +0100	[thread overview]
Message-ID: <20180319133147.15413-34-chriscool@tuxfamily.org> (raw)
In-Reply-To: <20180319133147.15413-1-chriscool@tuxfamily.org>

This adds the infrastructure to handle 'have' instructions in
process mode.

The answer from the helper sub-process should be like the
output in script mode, that is lines like this:

sha1 SPACE size SPACE type NEWLINE

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 odb-helper.c | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 73 insertions(+), 1 deletion(-)

diff --git a/odb-helper.c b/odb-helper.c
index 83f8cce3b1..5ec21e9d1f 100644
--- a/odb-helper.c
+++ b/odb-helper.c
@@ -649,6 +649,70 @@ static int odb_helper_object_cmp(const void *va, const void *vb)
 	return oidcmp(&a->oid, &b->oid);
 }
 
+static int send_have_packets(struct odb_helper *o,
+			     struct object_process *entry,
+			     struct strbuf *status)
+{
+	int packet_len;
+	int total_got = 0;
+	struct child_process *process = &entry->subprocess.process;
+	int err = packet_write_fmt_gently(process->in, "command=have\n");
+
+	if (err)
+		return err;
+
+	err = packet_flush_gently(process->in);
+	if (err)
+		return err;
+
+	for (;;) {
+		/* packet_read() writes a '\0' extra byte at the end */
+		char buf[LARGE_PACKET_DATA_MAX + 1];
+		char *p = buf;
+		int more;
+
+		packet_len = packet_read(process->out, NULL, NULL,
+			buf, LARGE_PACKET_DATA_MAX + 1,
+			PACKET_READ_GENTLE_ON_EOF);
+
+		if (packet_len <= 0)
+			break;
+
+		total_got += packet_len;
+
+		/* 'have' packets should end with '\n' or '\0' */
+		do {
+			char *eol = strchrnul(p, '\n');
+			more = (*eol == '\n');
+			*eol = '\0';
+			if (add_have_entry(o, p))
+				break;
+			p = eol + 1;
+		} while (more && *p);
+	}
+
+	if (packet_len < 0)
+		return packet_len;
+
+	return check_object_process_status(process->out, status);
+}
+
+static int have_object_process(struct odb_helper *o)
+{
+	int err;
+	struct object_process *entry;
+	struct strbuf status = STRBUF_INIT;
+
+	entry = launch_object_process(o, ODB_HELPER_CAP_HAVE);
+	if (!entry)
+		return -1;
+
+	err = send_have_packets(o, entry, &status);
+
+	return check_object_process_error(err, status.buf, entry, o->dealer,
+					  ODB_HELPER_CAP_HAVE);
+}
+
 static void have_object_script(struct odb_helper *o)
 {
 	struct odb_helper_cmd cmd;
@@ -670,12 +734,20 @@ static void have_object_script(struct odb_helper *o)
 
 static void odb_helper_load_have(struct odb_helper *o)
 {
+	uint64_t start;
+
 	if (o->have_valid)
 		return;
 	o->have_valid = 1;
 
+	start = getnanotime();
+
 	if (o->type == ODB_HELPER_SCRIPT_CMD)
 		have_object_script(o);
+	else if (o->type == ODB_HELPER_SUBPROCESS_CMD)
+		have_object_process(o);
+
+	trace_performance_since(start, "odb_helper_load_have");
 
 	qsort(o->have, o->have_nr, sizeof(*o->have), odb_helper_object_cmp);
 }
@@ -937,7 +1009,7 @@ int odb_helper_get_direct(struct odb_helper *o,
 		fetch_object(o->dealer, sha1);
 	else if (o->type == ODB_HELPER_SCRIPT_CMD)
 		res = get_direct_script(o, sha1);
-	else
+	else if (o->type == ODB_HELPER_SUBPROCESS_CMD)
 		res = get_object_process(o, sha1, -1);
 
 	trace_performance_since(start, "odb_helper_get_direct");
-- 
2.17.0.rc0.37.g8f476fabe9


  parent reply	other threads:[~2018-03-19 13:33 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-03-19 13:31 [PATCH v2 00/36] Promisor remotes and external ODB support Christian Couder
2018-03-19 13:31 ` [PATCH v2 01/36] Add initial external odb support Christian Couder
2018-03-19 13:31 ` [PATCH v2 02/36] sha1_file: add prepare_external_alt_odb() Christian Couder
2018-03-19 13:31 ` [PATCH v2 03/36] Add GIT_NO_EXTERNAL_ODB env variable Christian Couder
2018-03-19 13:31 ` [PATCH v2 04/36] external-odb: add has_external_odb() Christian Couder
2018-03-19 13:31 ` [PATCH v2 05/36] external-odb: implement external_odb_get_direct Christian Couder
2018-03-19 13:31 ` [PATCH v2 06/36] sha1_file: prepare for external odbs Christian Couder
2018-03-19 13:31 ` [PATCH v2 07/36] odb-helper: add 'enum odb_helper_type' Christian Couder
2018-03-19 13:31 ` [PATCH v2 08/36] external-odb: add external_odb_reinit() Christian Couder
2018-03-19 13:31 ` [PATCH v2 09/36] external-odb: add script mode support Christian Couder
2018-03-19 13:31 ` [PATCH v2 10/36] odb-helper: add 'script_mode' to 'struct odb_helper' Christian Couder
2018-03-19 13:31 ` [PATCH v2 11/36] odb-helper: add odb_helper_init() to send 'init' instruction Christian Couder
2018-03-19 13:31 ` [PATCH v2 12/36] external-odb: add external_odb_get_many_direct() Christian Couder
2018-03-19 13:31 ` [PATCH v2 13/36] Use external_odb_get_direct() and has_external_odb() Christian Couder
2018-03-19 13:31 ` [PATCH v2 14/36] Use odb.origin.partialclonefilter instead of core.partialclonefilter Christian Couder
2018-03-19 13:31 ` [PATCH v2 15/36] external-odb: add 'get_direct' support Christian Couder
2018-03-19 13:31 ` [PATCH v2 16/36] t0500: add 'put_raw_obj' instruction to odb-helper script Christian Couder
2018-03-19 13:31 ` [PATCH v2 17/36] external odb: add 'put_raw_obj' support Christian Couder
2018-03-19 13:31 ` [PATCH v2 18/36] external-odb: accept only blobs for now Christian Couder
2018-03-19 13:31 ` [PATCH v2 19/36] t0500: add test for external odb write support Christian Couder
2018-03-19 13:31 ` [PATCH v2 20/36] Add t0510 to test external ODB transfer Christian Couder
2018-03-19 13:31 ` [PATCH v2 21/36] lib-httpd: pass config file to start_httpd() Christian Couder
2018-03-19 13:31 ` [PATCH v2 22/36] lib-httpd: add upload.sh Christian Couder
2018-03-19 13:31 ` [PATCH v2 23/36] lib-httpd: add list.sh Christian Couder
2018-03-19 13:31 ` [PATCH v2 24/36] lib-httpd: add apache-e-odb.conf Christian Couder
2018-03-19 13:31 ` [PATCH v2 25/36] odb-helper: add odb_helper_get_raw_object() Christian Couder
2018-03-19 13:31 ` [PATCH v2 26/36] pack-objects: don't pack objects in external odbs Christian Couder
2018-03-19 13:31 ` [PATCH v2 27/36] Add t0520 to test transfer to HTTP external odb Christian Couder
2018-03-19 13:31 ` [PATCH v2 28/36] odb-helper: add init_object_process() Christian Couder
2018-03-19 13:31 ` [PATCH v2 29/36] Add t0550 to test 'get_direct' mechanism Christian Couder
2018-03-19 13:31 ` [PATCH v2 30/36] Add t0560 to test passing git objects Christian Couder
2018-03-19 13:31 ` [PATCH v2 31/36] odb-helper: add put_object_process() Christian Couder
2018-03-19 13:31 ` [PATCH v2 32/36] Add t0570 to test passing raw objects Christian Couder
2018-03-19 13:31 ` Christian Couder [this message]
2018-03-19 13:31 ` [PATCH v2 34/36] Add t0580 to test "have" capability and " Christian Couder
2018-03-19 13:31 ` [PATCH v2 35/36] external-odb: use 'odb=magic' attribute to mark odb blobs Christian Couder
2018-03-19 13:31 ` [PATCH v2 36/36] Add Documentation/technical/external-odb.txt Christian Couder
2018-03-19 19:13 ` [PATCH v2 00/36] Promisor remotes and external ODB support Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180319133147.15413-34-chriscool@tuxfamily.org \
    --to=christian.couder@gmail.com \
    --cc=Ben.Peart@microsoft.com \
    --cc=chriscool@tuxfamily.org \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jeffhost@microsoft.com \
    --cc=jonathantanmy@google.com \
    --cc=larsxschneider@gmail.com \
    --cc=mh@glandium.org \
    --cc=pclouds@gmail.com \
    --cc=peff@peff.net \
    --subject='Re: [PATCH v2 33/36] odb-helper: add have_object_process()' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).