From: Jeff King <peff@peff.net>
To: Jonathan Tan <jonathantanmy@google.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 2/2] index-pack: prefetch missing REF_DELTA bases
Date: Fri, 17 May 2019 00:42:52 -0400 [thread overview]
Message-ID: <20190517044252.GA12107@sigill.intra.peff.net> (raw)
In-Reply-To: <20190517043939.GA12063@sigill.intra.peff.net>
On Fri, May 17, 2019 at 12:39:39AM -0400, Jeff King wrote:
> > I think to trigger it you'd have to manually assemble an evil pack as I
> > described (e.g., using the routines in t/lib-pack.sh). I'm going offline
> > for a bit, but I may have a go at it later tonight or tomorrow.
>
> OK, doing so wasn't _too_ bad, though I did have to resurrect the
> horrible generator patch from [1]. My results are below. But more
> importantly, there is good news.
Oh, I forgot my [1]. It was this message:
https://public-inbox.org/git/20130823182409.GA30130@sigill.intra.peff.net/
which explained how I hacked up pack-objects to generate the binary goo
in lib-pack.sh. Here's my forward-ported version of it, in case we need
it again.
I do think it would be useful for debugging and experimenting to have a
mode for pack-objects that takes in a list of instructions (add object
X, make it a delta against object Y) that skips its usual ordering and
delta constraints. This isn't it, but it's enough to hackily get what
you need. ;)
---
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 9f424aabab..87955d9404 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3234,6 +3234,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
struct argv_array rp = ARGV_ARRAY_INIT;
int rev_list_unpacked = 0, rev_list_all = 0, rev_list_reflog = 0;
int rev_list_index = 0;
+ int magic = 0;
struct string_list keep_pack_list = STRING_LIST_INIT_NODUP;
struct option pack_objects_options[] = {
OPT_SET_INT('q', "quiet", &progress,
@@ -3321,6 +3322,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
N_("do not pack objects in promisor packfiles")),
OPT_BOOL(0, "delta-islands", &use_delta_islands,
N_("respect islands during delta compression")),
+ OPT_BOOL(0, "magic", &magic, "make deltas"),
OPT_END(),
};
@@ -3337,6 +3339,34 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
argc = parse_options(argc, argv, prefix, pack_objects_options,
pack_usage, 0);
+ if (magic) {
+ struct object_id oid;
+ struct delta_index *index;
+ void *src, *trg, *delta;
+ enum object_type src_type, trg_type;
+ unsigned long src_size, trg_size, delta_size, z_delta_size;
+ unsigned char header[10];
+ unsigned long header_len;
+
+ get_oid(argv[0], &oid);
+ trg = read_object_file(&oid, &trg_type, &trg_size);
+
+ get_oid(argv[1], &oid);
+ src = read_object_file(&oid, &src_type, &src_size);
+
+ index = create_delta_index(src, src_size);
+ delta = create_delta(index, trg, trg_size, &delta_size, 8192);
+
+ z_delta_size = do_compress(&delta, delta_size);
+ header_len = encode_in_pack_object_header(header, sizeof(header),
+ OBJ_REF_DELTA,
+ delta_size);
+ fwrite(header, 1, header_len, stdout);
+ fwrite(oid.hash, 1, the_hash_algo->rawsz, stdout);
+ fwrite(delta, 1, z_delta_size, stdout);
+ return 0;
+ }
+
if (argc) {
base_name = argv[0];
argc--;
next prev parent reply other threads:[~2019-05-17 4:42 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-14 21:10 [PATCH 0/2] Partial clone fix: handling received REF_DELTA Jonathan Tan
2019-05-14 21:10 ` [PATCH 1/2] t5616: refactor packfile replacement Jonathan Tan
2019-05-15 8:36 ` Johannes Schindelin
2019-05-15 18:22 ` Jonathan Tan
2019-05-14 21:10 ` [PATCH 2/2] index-pack: prefetch missing REF_DELTA bases Jonathan Tan
2019-05-15 8:46 ` Johannes Schindelin
2019-05-15 18:28 ` Jonathan Tan
2019-05-17 18:33 ` Johannes Schindelin
2019-05-15 23:16 ` Jeff King
2019-05-16 1:43 ` Junio C Hamano
2019-05-16 4:04 ` Jeff King
2019-05-16 18:26 ` Jonathan Tan
2019-05-16 21:12 ` Jeff King
2019-05-16 21:30 ` Jonathan Tan
2019-05-16 21:42 ` Jeff King
2019-05-16 23:15 ` Jonathan Tan
2019-05-17 1:09 ` Jeff King
2019-05-17 1:22 ` Jeff King
2019-05-17 4:39 ` Jeff King
2019-05-17 4:42 ` Jeff King [this message]
2019-05-17 7:20 ` Duy Nguyen
2019-05-17 8:55 ` Jeff King
2019-05-18 11:39 ` Duy Nguyen
2019-05-20 23:04 ` Nicolas Pitre
2019-05-21 21:20 ` Jeff King
2019-06-03 22:23 ` Jonathan Nieder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190517044252.GA12107@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=jonathantanmy@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).