From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-3.0 required=3.0 tests=AWL,BAYES_00,BODY_8BITS, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by dcvr.yhbt.net (Postfix) with ESMTP id 1F2171F403 for ; Mon, 17 Oct 2022 13:57:48 +0000 (UTC) Authentication-Results: dcvr.yhbt.net; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Oe+0iDG5"; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230175AbiJQN5W (ORCPT ); Mon, 17 Oct 2022 09:57:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230079AbiJQN5U (ORCPT ); Mon, 17 Oct 2022 09:57:20 -0400 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 783CC541A7 for ; Mon, 17 Oct 2022 06:57:17 -0700 (PDT) Received: by mail-ed1-x52d.google.com with SMTP id q19so16156288edd.10 for ; Mon, 17 Oct 2022 06:57:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to :content-language:subject:reply-to:user-agent:mime-version:date :message-id:from:from:to:cc:subject:date:message-id:reply-to; bh=xQy46dTGKS9lbG8/tdXN9bbjIH8vCFLxzDqbDPvpXDQ=; b=Oe+0iDG5hwSfR69eQzbszo9/sxlsQdXnFLFYFM3d0f5U3i064Fl7BJ/FHohrqivtRx 0XTmVNJa9M2yT68cIXxiei0Yn7gQBnNN4J9MKMgFR2h7nkaHlSBdf6pviX29EiHBzjKq zQl3BgDj74VI6yXmkPo4XRdH+q3NL1oBaeyxzYK3qnwyvtQnS44AjMsKSIUAV+yMJgoW 7vM5kXfguyEwe2YdHgYT6eUNRIAlNIdAOGIKk4xXTRufn8es9w2doQotJJbxVlBacBO1 r4rX/9oy1dg84fhazyHwTdenMdp8k0pLRGM55HSH7J2EIYNR5hhZyyry32oLAovwZxsB gP/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:cc:to :content-language:subject:reply-to:user-agent:mime-version:date :message-id:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xQy46dTGKS9lbG8/tdXN9bbjIH8vCFLxzDqbDPvpXDQ=; b=OkOz+lDjSeVTwc9omMv8aTT7wJuhNeL88O3ETBTBewNGrda061ADQEiSBe9StY9hB+ NTk//MxLiek3opAYu0i3c6rHRL3nQ5ovNhlsxiFwUMcTgemJ9jap2f5qWD43ajtCvZTE Osbm+J60DJm3EPUHqnxsPGPmwZLduqiyPVpeOhhoD353eX6VYsaBHi5mD8rt11VUIkZV i3w8XGRy1rCpUexf8ffWQLq2dVp6/SkoaOUEWCLQHH/805qjdE8KM+hdVUu7Dp7IBySv qzBNpameqIBn//+jQ8faU31GT+pSvWLSwg70FKPL7y2Vkg/RmjdcUykAwg99mN47Oev3 3UTQ== X-Gm-Message-State: ACrzQf2Z2oiCtQuqnkAXvhxOQfkvWmym5cf88t0kR8KdfPIu3E4/1hZf EjKstuv5969UbVD/TgPt2DsTKfKOyCw= X-Google-Smtp-Source: AMsMyM51b1aDJrCA1+y2t7ygyQXUGqKkqRYwWpKjfX1qVTPPu3kVZy2hgxlW6hAHnBsO2gb+DlPEaQ== X-Received: by 2002:a05:6402:410:b0:451:ea13:4ed7 with SMTP id q16-20020a056402041000b00451ea134ed7mr10833349edv.262.1666015035809; Mon, 17 Oct 2022 06:57:15 -0700 (PDT) Received: from [192.168.1.74] ([31.185.185.144]) by smtp.gmail.com with ESMTPSA id v3-20020a1709063bc300b0077a201f6d1esm6144376ejf.87.2022.10.17.06.57.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 17 Oct 2022 06:57:15 -0700 (PDT) From: Phillip Wood X-Google-Original-From: Phillip Wood Message-ID: <8acd4fac-491f-7ab5-88be-b7804b37d23b@dunelm.org.uk> Date: Mon, 17 Oct 2022 14:57:13 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.2 Reply-To: phillip.wood@dunelm.org.uk Subject: Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command Content-Language: en-US To: Heather Lapointe via GitGitGadget , git@vger.kernel.org Cc: =?UTF-8?Q?Ren=c3=a9_Scharfe?= , Heather Lapointe References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Hi Heather On 17/10/2022 03:23, Heather Lapointe via GitGitGadget wrote: > This makes it possible to include submodule contents in an archive command. > > The inspiration for this change comes from this Github thread, > https://github.com/dear-github/dear-github/issues/214, with at least 160 > 👍🏻 's at the time of writing. (I stumbled upon it because I wanted it as > well). > > I figured the underlying implementation wouldn't be too difficult with most > of the plumbing already in place, so I decided to add the relevant logic to > the client git-archive command. > > One of the trickier parts of this implementation involved teaching read_tree > about submodules. Some of the troublesome areas were still using the > the_repository references to look up commit or tree or oid information. I > ended up deciding that read_tree_fn_t would probably be best off having a > concrete repo reference since it allows changing the context to a subrepo > where needed (even though some of the usages did not need it specifically). > > I am open to feedback since this is all quite new to me :) I've had a quick read through and I thought this patch series was well structured and easy to follow for someone like me who is not familiar with the archive code. I've left a few mostly high-level comments on some of the individual patches. The commit messages are all pretty brief and would benefit from a bit more explaintion of why the changes are being made. From a brief read through the code changes themselves all looked pretty good. Best Wishes Phillip > Alphadelta14 (1): > tree: do not use the_repository for tree traversal methods. > > Heather Lapointe (8): > tree: update cases to use repo_ tree methods > tree: increase test coverage for tree.c > tree: handle submodule case for read_tree_at properly > tree: add repository parameter to read_tree_fn_t > archive: pass repo objects to write_archive handlers > archive: remove global repository from archive_args > archive: add --recurse-submodules to git-archive command > archive: add tests for git archive --recurse-submodules > > Documentation/git-archive.txt | 6 +- > Makefile | 1 + > archive-tar.c | 15 ++-- > archive-zip.c | 15 ++-- > archive.c | 138 ++++++++++++++++++++---------- > archive.h | 16 +++- > builtin/checkout.c | 4 +- > builtin/log.c | 4 +- > builtin/ls-files.c | 8 +- > builtin/ls-tree.c | 34 +++++--- > merge-recursive.c | 4 +- > merge.c | 4 +- > reset.c | 2 +- > revision.c | 4 +- > sequencer.c | 6 +- > sparse-index.c | 4 +- > t/helper/test-tool.c | 1 + > t/helper/test-tool.h | 1 + > t/helper/test-tree-read-tree-at.c | 41 +++++++++ > t/t1023-tree-read-tree-at.sh | 65 ++++++++++++++ > t/t5005-archive-submodules.sh | 83 ++++++++++++++++++ > tree.c | 93 ++++++++++++++------ > tree.h | 16 ++-- > wt-status.c | 4 +- > 24 files changed, 448 insertions(+), 121 deletions(-) > create mode 100644 t/helper/test-tree-read-tree-at.c > create mode 100755 t/t1023-tree-read-tree-at.sh > create mode 100755 t/t5005-archive-submodules.sh > > > base-commit: e85701b4af5b7c2a9f3a1b07858703318dce365d > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1359%2FAlphadelta14%2Farchive-recurse-submodules-v3 > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1359/Alphadelta14/archive-recurse-submodules-v3 > Pull-Request: https://github.com/git/git/pull/1359 > > Range-diff vs v2: > > -: ----------- > 1: 79959a54eb4 tree: do not use the_repository for tree traversal methods. > -: ----------- > 2: 2291e0f9b5c tree: update cases to use repo_ tree methods > -: ----------- > 3: 9a07c6932f4 tree: increase test coverage for tree.c > 2: 68f7830c6d9 ! 4: d3d1738e670 archive: fix a case of submodule in submodule traversal > @@ > ## Metadata ## > -Author: Alphadelta14 > +Author: Heather Lapointe > > ## Commit message ## > - archive: fix a case of submodule in submodule traversal > + tree: handle submodule case for read_tree_at properly > > - repo_submodule_init actually expects the path relative to submodule_prefix. > - We preform a simple strip to the correct path. > + This supports traversal into an actual submodule for read_tree_at. > + The logic is blocked on pathspec->recurse_submodules now, > + but previously hadn't been executed due to all fn() cases > + returning early for submodules. > > - Signed-off-by: Alphadelta14 > + Signed-off-by: Heather Lapointe > > ## tree.c ## > +@@ > + #include "alloc.h" > + #include "tree-walk.h" > + #include "repository.h" > ++#include "pathspec.h" > + > + const char *tree_type = "tree"; > + > @@ tree.c: int read_tree_at(struct repository *r, > - struct repository subrepo; > - struct repository* subrepo_p = &subrepo; > - struct tree* submodule_tree; > + return -1; > + } > + > +- if (S_ISDIR(entry.mode)) > ++ if (S_ISDIR(entry.mode)) { > + oidcpy(&oid, &entry.oid); > +- else if (S_ISGITLINK(entry.mode)) { > +- struct commit *commit; > + > +- commit = lookup_commit(r, &entry.oid); > ++ len = tree_entry_len(&entry); > ++ strbuf_add(base, entry.path, len); > ++ strbuf_addch(base, '/'); > ++ retval = read_tree_at(r, lookup_tree(r, &oid), > ++ base, pathspec, > ++ fn, context); > ++ strbuf_setlen(base, oldlen); > ++ if (retval) > ++ return -1; > ++ } else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) { > ++ struct commit *commit; > ++ struct repository subrepo; > ++ struct repository* subrepo_p = &subrepo; > ++ struct tree* submodule_tree; > + char *submodule_rel_path; > + int name_base_len = 0; > - > -- if (repo_submodule_init(subrepo_p, r, entry.path, null_oid())) > -- die("couldn't init submodule %s%s", base->buf, entry.path); > ++ > + len = tree_entry_len(&entry); > + strbuf_add(base, entry.path, len); > + submodule_rel_path = base->buf; > @@ tree.c: int read_tree_at(struct repository *r, > + > + if (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid())) > + die("couldn't init submodule %s", base->buf); > - > - if (repo_read_index(subrepo_p) < 0) > - die("index file corrupt"); > - > - commit = lookup_commit(subrepo_p, &entry.oid); > ++ > ++ if (repo_read_index(subrepo_p) < 0) > ++ die("index file corrupt"); > ++ > ++ commit = lookup_commit(subrepo_p, &entry.oid); > if (!commit) > - die("Commit %s in submodule path %s%s not found", > + die("Commit %s in submodule path %s not found", > oid_to_hex(&entry.oid), > - base->buf, entry.path); > -+ base->buf); > - > - if (repo_parse_commit(subrepo_p, commit)) > +- > +- // FIXME: This is the wrong repo instance (it refers to the superproject) > +- // it will always fail as is (will fix in later patch) > +- // This current codepath isn't executed by any existing callbacks > +- // so it wouldn't show up as an issue at this time. > +- if (repo_parse_commit(r, commit)) > - die("Invalid commit %s in submodule path %s%s", > ++ base->buf); > ++ > ++ if (repo_parse_commit(subrepo_p, commit)) > + die("Invalid commit %s in submodule path %s", > oid_to_hex(&entry.oid), > - base->buf, entry.path); > + base->buf); > > - submodule_tree = repo_get_commit_tree(subrepo_p, commit); > - oidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL); > +- oidcpy(&oid, get_commit_tree_oid(commit)); > +- } > +- else > +- continue; > ++ submodule_tree = repo_get_commit_tree(subrepo_p, commit); > ++ oidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL); > > -- len = tree_entry_len(&entry); > -- strbuf_add(base, entry.path, len); > - strbuf_addch(base, '/'); > +- len = tree_entry_len(&entry); > +- strbuf_add(base, entry.path, len); > +- strbuf_addch(base, '/'); > +- retval = read_tree_at(r, lookup_tree(r, &oid), > +- base, pathspec, > +- fn, context); > +- strbuf_setlen(base, oldlen); > +- if (retval) > +- return -1; > ++ strbuf_addch(base, '/'); > + > - retval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid), > - base, pathspec, > - fn, context); > - if (retval) { > -- die("failed to read tree for %s%s", base->buf, entry.path); > ++ retval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid), > ++ base, pathspec, > ++ fn, context); > ++ if (retval) > + die("failed to read tree for %s", base->buf); > - return -1; > - } > - strbuf_setlen(base, oldlen); > ++ strbuf_setlen(base, oldlen); > ++ repo_clear(subrepo_p); > ++ } > ++ // else, this is a file (or a submodule, but no pathspec->recurse_submodules) > + } > + return 0; > + } > -: ----------- > 5: 376345fdf66 tree: add repository parameter to read_tree_fn_t > 1: 41664a59029 ! 6: 1b9b049d64f archive: add --recurse-submodules to git-archive command > @@ > ## Metadata ## > -Author: Alphadelta14 > +Author: Heather Lapointe > > ## Commit message ## > - archive: add --recurse-submodules to git-archive command > + archive: pass repo objects to write_archive handlers > > - This makes it possible to include submodule contents in an archive command. > - > - This required updating the general read_tree callbacks to support sub-repos > - by not using the_repository global references where possible. > - > - archive: update streaming to use target repo > - archive: add test cases for git archive --recurse-submodules > + Use contextual repos instead of the_repository or args->repo > + to ensure that submodules will be handled correctly > + since they use multiple repo instances. > > Signed-off-by: Heather Lapointe > > @@ archive-tar.c: static unsigned long offset; > static int tar_umask = 002; > > static int write_tar_filter_archive(const struct archiver *ar, > -+ struct repository *repo, > ++ struct repository *repo, > struct archiver_args *args); > > /* > @@ archive-tar.c: static void write_extended_header(struct archiver_args *args, > } > > -static int write_tar_entry(struct archiver_args *args, > -+static int write_tar_entry(struct repository *repo, > ++static int write_tar_entry( > ++ struct repository *repo, > + struct archiver_args *args, > const struct object_id *oid, > const char *path, size_t pathlen, > @@ archive-tar.c: static void tgz_write_block(const void *data) > static const char internal_gzip_command[] = "git archive gzip"; > > static int write_tar_filter_archive(const struct archiver *ar, > -+ struct repository *repo, > ++ struct repository *repo, > struct archiver_args *args) > { > #if ZLIB_VERNUM >= 0x1221 > @@ archive-zip.c: static int entry_is_binary(struct index_state *istate, const char > #define STREAM_BUFFER_SIZE (1024 * 16) > > -static int write_zip_entry(struct archiver_args *args, > -+static int write_zip_entry(struct repository *repo, > ++static int write_zip_entry( > ++ struct repository *repo, > + struct archiver_args *args, > const struct object_id *oid, > const char *path, size_t pathlen, > @@ archive-zip.c: static int archive_zip_config(const char *var, const char *value, > } > > static int write_zip_archive(const struct archiver *ar UNUSED, > -+ struct repository *repo, > ++ struct repository *repo, > struct archiver_args *args) > { > int err; > @@ archive-zip.c: static int write_zip_archive(const struct archiver *ar UNUSED, > > > ## archive.c ## > -@@ > - #include "unpack-trees.h" > - #include "dir.h" > - #include "quote.h" > -+#include "submodule.h" > - > - static char const * const archive_usage[] = { > - N_("git archive [] [...]"), > -@@ archive.c: static void format_subst(const struct commit *commit, > - } > - > - static void *object_file_to_archive(const struct archiver_args *args, > -+ struct repository *repo, > - const char *path, > - const struct object_id *oid, > - unsigned int mode, > -@@ archive.c: static void *object_file_to_archive(const struct archiver_args *args, > - (args->tree ? &args->tree->object.oid : NULL), oid); > - > - path += args->baselen; > -- buffer = read_object_file(oid, type, sizep); > -+ buffer = repo_read_object_file(repo, oid, type, sizep); > - if (buffer && S_ISREG(mode)) { > - struct strbuf buf = STRBUF_INIT; > - size_t size = 0; > - > - strbuf_attach(&buf, buffer, *sizep, *sizep + 1); > -- convert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta); > -+ convert_to_working_tree(repo->index, path, buf.buf, buf.len, &buf, &meta); > - if (commit) > - format_subst(commit, buf.buf, buf.len, &buf, args->pretty_ctx); > - buffer = strbuf_detach(&buf, &size); > @@ archive.c: static int check_attr_export_subst(const struct attr_check *check) > return check && ATTR_TRUE(check->items[1].value); > } > > -static int write_archive_entry(const struct object_id *oid, const char *base, > -+static int write_archive_entry(struct repository *repo, const struct object_id *oid, const char *base, > ++static int write_archive_entry( > ++ struct repository *repo, > ++ const struct object_id *oid, const char *base, > int baselen, const char *filename, unsigned mode, > void *context) > { > @@ archive.c: static int write_archive_entry(const struct object_id *oid, const cha > + err = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0); > if (err) > return err; > -- return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0); > -+ return READ_TREE_RECURSIVE; > - } > - > - if (args->verbose) > + return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0); > @@ archive.c: static int write_archive_entry(const struct object_id *oid, const char *base, > > /* Stream it? */ > if (S_ISREG(mode) && !args->convert && > - oid_object_info(args->repo, oid, &size) == OBJ_BLOB && > -- size > big_file_threshold) > -- return write_entry(args, oid, path.buf, path.len, mode, NULL, size); > + oid_object_info(repo, oid, &size) == OBJ_BLOB && > -+ size > big_file_threshold) { > -+ err = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size); > -+ if (err) { > -+ die("Failed to write file %.*s", (int)path.len, path.buf); > -+ } > -+ return err; > -+ } > + size > big_file_threshold) > +- return write_entry(args, oid, path.buf, path.len, mode, NULL, size); > ++ return write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size); > > -- buffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size); > -+ buffer = object_file_to_archive(args, repo, path.buf, oid, mode, &type, &size); > + buffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size); > if (!buffer) > return error(_("cannot read '%s'"), oid_to_hex(oid)); > - err = write_entry(args, oid, path.buf, path.len, mode, buffer, size); > @@ archive.c: static void queue_directory(const struct object_id *oid, > } > > -static int write_directory(struct archiver_context *c) > -+static void queue_submodule(struct repository *superproject, > -+ const struct object_id *oid, > -+ struct strbuf *base, const char *filename, > -+ unsigned mode, struct archiver_context *c) > -+{ > -+ struct repository subrepo; > -+ > -+ if (repo_submodule_init(&subrepo, superproject, filename, null_oid())) > -+ return; > -+ > -+ if (repo_read_index(&subrepo) < 0) > -+ die("index file corrupt"); > -+ > -+ queue_directory(oid, base, filename, mode, c); > -+ > -+ repo_clear(&subrepo); > -+} > -+ > -+static int write_directory(struct repository *repo, struct archiver_context *c) > ++static int write_directory( > ++ struct repository *repo, > ++ struct archiver_context *c) > { > struct directory *d = c->bottom; > int ret; > @@ archive.c: static int write_directory(struct archiver_context *c) > + write_directory(repo, c) || > + write_archive_entry(repo, &d->oid, d->path, d->baselen, > d->path + d->baselen, d->mode, > -- c) != READ_TREE_RECURSIVE; > -+ c); > + c) != READ_TREE_RECURSIVE; > free(d); > -- return ret ? -1 : 0; > -+ if (ret == READ_TREE_RECURSIVE) > -+ return 0; > -+ return ret; > - } > - > --static int queue_or_write_archive_entry(const struct object_id *oid, > -+static int queue_or_write_archive_entry( > -+ struct repository *repo, const struct object_id *oid, > - struct strbuf *base, const char *filename, > - unsigned mode, void *context) > - { > -@@ archive.c: static int queue_or_write_archive_entry(const struct object_id *oid, > - /* Borrow base, but restore its original value when done. */ > - strbuf_addstr(base, filename); > - strbuf_addch(base, '/'); > -- check = get_archive_attrs(c->args->repo->index, base->buf); > -+ check = get_archive_attrs(repo->index, base->buf); > - strbuf_setlen(base, baselen); > - > - if (check_attr_export_ignore(check)) > - return 0; > - queue_directory(oid, base, filename, mode, c); > +@@ archive.c: static int queue_or_write_archive_entry( > return READ_TREE_RECURSIVE; > -+ } else if (c->args->recurse_submodules && S_ISGITLINK(mode)) { > -+ if (is_submodule_active(repo, filename)) { > -+ queue_submodule(repo, oid, base, filename, mode, c); > -+ return READ_TREE_RECURSIVE; > -+ } > } > > - if (write_directory(c)) > -+ if (write_directory(repo, c)) > ++ if (write_directory(r, c)) > return -1; > - return write_archive_entry(oid, base->buf, base->len, filename, mode, > -+ return write_archive_entry(repo, oid, base->buf, base->len, filename, mode, > ++ return write_archive_entry(r, oid, base->buf, base->len, filename, mode, > context); > } > > @@ archive.c: struct extra_file_info { > }; > > -int write_archive_entries(struct archiver_args *args, > -+int write_archive_entries(struct repository *repo, > ++int write_archive_entries( > ++ struct repository *repo, > + struct archiver_args *args, > write_archive_entry_fn_t write_entry) > { > @@ archive.c: int write_archive_entries(struct archiver_args *args, > len, 040777, NULL, 0); > if (err) > return err; > -@@ archive.c: int write_archive_entries(struct archiver_args *args, > - memset(&opts, 0, sizeof(opts)); > - opts.index_only = 1; > - opts.head_idx = -1; > -- opts.src_index = args->repo->index; > -- opts.dst_index = args->repo->index; > -+ opts.src_index = repo->index; > -+ opts.dst_index = repo->index; > - opts.fn = oneway_merge; > - init_tree_desc(&t, args->tree->buffer, args->tree->size); > - if (unpack_trees(1, &t, &opts)) > -@@ archive.c: int write_archive_entries(struct archiver_args *args, > - git_attr_set_direction(GIT_ATTR_INDEX); > - } > - > -- err = read_tree(args->repo, args->tree, > -+ err = read_tree(repo, args->tree, > - &args->pathspec, > - queue_or_write_archive_entry, > - &context); > @@ archive.c: int write_archive_entries(struct archiver_args *args, > if (strbuf_read_file(&content, path, info->stat.st_size) < 0) > err = error_errno(_("cannot read '%s'"), path); > @@ archive.c: int write_archive_entries(struct archiver_args *args, > path, strlen(path), > canon_mode(info->stat.st_mode), > info->content, info->stat.st_size); > -@@ archive.c: struct path_exists_context { > - struct archiver_args *args; > - }; > - > --static int reject_entry(const struct object_id *oid UNUSED, > -+static int reject_entry(struct repository *repo, const struct object_id *oid UNUSED, > - struct strbuf *base, > - const char *filename, unsigned mode, > - void *context) > -@@ archive.c: static int reject_entry(const struct object_id *oid UNUSED, > - struct strbuf sb = STRBUF_INIT; > - strbuf_addbuf(&sb, base); > - strbuf_addstr(&sb, filename); > -- if (!match_pathspec(ctx->args->repo->index, > -+ if (!match_pathspec(repo->index, > - &ctx->pathspec, > - sb.buf, sb.len, 0, NULL, 1)) > - ret = READ_TREE_RECURSIVE; > -@@ archive.c: static void parse_pathspec_arg(const char **pathspec, > - PATHSPEC_PREFER_FULL, > - "", pathspec); > - ar_args->pathspec.recursive = 1; > -+ ar_args->pathspec.recurse_submodules = ar_args->recurse_submodules; > - if (pathspec) { > - while (*pathspec) { > - if (**pathspec && !path_exists(ar_args, *pathspec)) > -@@ archive.c: static int parse_archive_args(int argc, const char **argv, > - int verbose = 0; > - int i; > - int list = 0; > -+ int recurse_submodules = 0; > - int worktree_attributes = 0; > - struct option opts[] = { > - OPT_GROUP(""), > -@@ archive.c: static int parse_archive_args(int argc, const char **argv, > - add_file_cb, (intptr_t)&base }, > - OPT_STRING('o', "output", &output, N_("file"), > - N_("write the archive to this file")), > -+ OPT_BOOL(0, "recurse-submodules", &recurse_submodules, > -+ N_("include submodules in archive")), > - OPT_BOOL(0, "worktree-attributes", &worktree_attributes, > - N_("read .gitattributes in working directory")), > - OPT__VERBOSE(&verbose, N_("report archived files on stderr")), > -@@ archive.c: static int parse_archive_args(int argc, const char **argv, > - args->base = base; > - args->baselen = strlen(base); > - args->worktree_attributes = worktree_attributes; > -+ args->recurse_submodules = recurse_submodules; > - > - return argc; > - } > @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix, > parse_treeish_arg(argv, &args, prefix, remote); > parse_pathspec_arg(argv + 1, &args); > @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix, > free(args.refname); > > ## archive.h ## > -@@ archive.h: struct archiver_args { > - timestamp_t time; > - struct pathspec pathspec; > - unsigned int verbose : 1; > -+ unsigned int recurse_submodules : 1; > - unsigned int worktree_attributes : 1; > - unsigned int convert : 1; > - int compression_level; > @@ archive.h: const char *archive_format_from_filename(const char *filename); > #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4 > struct archiver { > const char *name; > - int (*write_archive)(const struct archiver *, struct archiver_args *); > -+ int (*write_archive)(const struct archiver *, struct repository *repo, struct archiver_args *); > ++ int (*write_archive)( > ++ const struct archiver *, > ++ struct repository *, > ++ struct archiver_args *); > unsigned flags; > char *filter_command; > }; > @@ archive.h: void init_tar_archiver(void); > void init_archivers(void); > > -typedef int (*write_archive_entry_fn_t)(struct archiver_args *args, > -+typedef int (*write_archive_entry_fn_t)(struct repository *repo, > ++typedef int (*write_archive_entry_fn_t)( > ++ struct repository *repo, > + struct archiver_args *args, > const struct object_id *oid, > const char *path, size_t pathlen, > @@ archive.h: void init_tar_archiver(void); > void *buffer, unsigned long size); > > -int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry); > -+int write_archive_entries(struct repository *repo, struct archiver_args *args, write_archive_entry_fn_t write_entry); > ++int write_archive_entries( > ++ struct repository *repo, > ++ struct archiver_args *args, > ++ write_archive_entry_fn_t write_entry); > > #endif /* ARCHIVE_H */ > - > - ## builtin/checkout.c ## > -@@ builtin/checkout.c: static int post_checkout_hook(struct commit *old_commit, struct commit *new_comm > - > - } > - > --static int update_some(const struct object_id *oid, struct strbuf *base, > -+static int update_some(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, void *context UNUSED) > - { > - int len; > - > - ## builtin/log.c ## > -@@ builtin/log.c: static int show_tag_object(const struct object_id *oid, struct rev_info *rev) > - return 0; > - } > - > --static int show_tree_object(const struct object_id *oid UNUSED, > -+static int show_tree_object(struct repository *repo UNUSED, const struct object_id *oid UNUSED, > - struct strbuf *base UNUSED, > - const char *pathname, unsigned mode, > - void *context) > - > - ## builtin/ls-files.c ## > -@@ builtin/ls-files.c: static int get_common_prefix_len(const char *common_prefix) > - return common_prefix_len; > - } > - > --static int read_one_entry_opt(struct index_state *istate, > -+static int read_one_entry_opt(struct repository *repo UNUSED, struct index_state *istate, > - const struct object_id *oid, > - struct strbuf *base, > - const char *pathname, > -@@ builtin/ls-files.c: static int read_one_entry_opt(struct index_state *istate, > - return add_index_entry(istate, ce, opt); > - } > - > --static int read_one_entry(const struct object_id *oid, struct strbuf *base, > -+static int read_one_entry(struct repository *repo, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, > - void *context) > - { > - struct index_state *istate = context; > -- return read_one_entry_opt(istate, oid, base, pathname, > -+ return read_one_entry_opt(repo, istate, oid, base, pathname, > - mode, > - ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK); > - } > -@@ builtin/ls-files.c: static int read_one_entry(const struct object_id *oid, struct strbuf *base, > - * This is used when the caller knows there is no existing entries at > - * the stage that will conflict with the entry being added. > - */ > --static int read_one_entry_quick(const struct object_id *oid, struct strbuf *base, > -+static int read_one_entry_quick(struct repository *repo, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, > - void *context) > - { > - struct index_state *istate = context; > -- return read_one_entry_opt(istate, oid, base, pathname, > -+ return read_one_entry_opt(repo, istate, oid, base, pathname, > - mode, ADD_CACHE_JUST_APPEND); > - } > - > - > - ## builtin/ls-tree.c ## > -@@ builtin/ls-tree.c: static int show_recursive(const char *base, size_t baselen, const char *pathname > - return 0; > - } > - > --static int show_tree_fmt(const struct object_id *oid, struct strbuf *base, > -+static int show_tree_fmt(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, void *context UNUSED) > - { > - size_t baselen; > -@@ builtin/ls-tree.c: static void show_tree_common_default_long(struct strbuf *base, > - strbuf_setlen(base, baselen); > - } > - > --static int show_tree_default(const struct object_id *oid, struct strbuf *base, > -+static int show_tree_default(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, > - void *context UNUSED) > - { > -@@ builtin/ls-tree.c: static int show_tree_default(const struct object_id *oid, struct strbuf *base, > - return recurse; > - } > - > --static int show_tree_long(const struct object_id *oid, struct strbuf *base, > -+static int show_tree_long(struct repository *repo, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, > - void *context UNUSED) > - { > -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base, > - > - if (data.type == OBJ_BLOB) { > - unsigned long size; > -- if (oid_object_info(the_repository, data.oid, &size) == OBJ_BAD) > -+ if (oid_object_info(repo, data.oid, &size) == OBJ_BAD) > - xsnprintf(size_text, sizeof(size_text), "BAD"); > - else > - xsnprintf(size_text, sizeof(size_text), > -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base, > - } > - > - printf("%06o %s %s %7s\t", data.mode, type_name(data.type), > -- find_unique_abbrev(data.oid, abbrev), size_text); > -+ repo_find_unique_abbrev(repo, data.oid, abbrev), size_text); > - show_tree_common_default_long(base, pathname, data.base->len); > - return recurse; > - } > - > --static int show_tree_name_only(const struct object_id *oid, struct strbuf *base, > -+static int show_tree_name_only(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, > - void *context UNUSED) > - { > -@@ builtin/ls-tree.c: static int show_tree_name_only(const struct object_id *oid, struct strbuf *base, > - return recurse; > - } > - > --static int show_tree_object(const struct object_id *oid, struct strbuf *base, > -+static int show_tree_object(struct repository *repo, const struct object_id *oid, struct strbuf *base, > - const char *pathname, unsigned mode, > - void *context UNUSED) > - { > -@@ builtin/ls-tree.c: static int show_tree_object(const struct object_id *oid, struct strbuf *base, > - if (early >= 0) > - return early; > - > -- printf("%s%c", find_unique_abbrev(oid, abbrev), line_termination); > -+ printf("%s%c", repo_find_unique_abbrev(repo, oid, abbrev), line_termination); > - return recurse; > - } > - > - > - ## list-objects.c ## > -@@ list-objects.c: static void process_tree(struct traversal_context *ctx, > - !revs->include_check_obj(&tree->object, revs->include_check_data)) > - return; > - > -- failed_parse = parse_tree_gently(tree, 1); > -+ failed_parse = parse_tree_gently(revs->repo, tree, 1); > - if (failed_parse) { > - if (revs->ignore_missing_links) > - return; > - > - ## merge-recursive.c ## > -@@ merge-recursive.c: static void unpack_trees_finish(struct merge_options *opt) > - clear_unpack_trees_porcelain(&opt->priv->unpack_opts); > - } > - > --static int save_files_dirs(const struct object_id *oid UNUSED, > -+static int save_files_dirs(struct repository *repo UNUSED, const struct object_id *oid UNUSED, > - struct strbuf *base, const char *path, > - unsigned int mode, void *context) > - { > - > - ## revision.c ## > -@@ revision.c: static void mark_tree_contents_uninteresting(struct repository *r, > - struct tree_desc desc; > - struct name_entry entry; > - > -- if (parse_tree_gently(tree, 1) < 0) > -+ if (parse_tree_gently(r, tree, 1) < 0) > - return; > - > - init_tree_desc(&desc, tree->buffer, tree->size); > -@@ revision.c: static void add_children_by_path(struct repository *r, > - if (!tree) > - return; > - > -- if (parse_tree_gently(tree, 1) < 0) > -+ if (parse_tree_gently(r, tree, 1) < 0) > - return; > - > - init_tree_desc(&desc, tree->buffer, tree->size); > - > - ## sparse-index.c ## > -@@ sparse-index.c: static void set_index_entry(struct index_state *istate, int nr, struct cache_ent > - add_name_hash(istate, ce); > - } > - > --static int add_path_to_index(const struct object_id *oid, > -+static int add_path_to_index(struct repository *repo UNUSED, const struct object_id *oid, > - struct strbuf *base, const char *path, > - unsigned int mode, void *context) > - { > - > - ## t/t5005-archive-submodules.sh (new) ## > -@@ > -+#!/bin/sh > -+ > -+test_description='git archive --recurse-submodules test' > -+ > -+. ./test-lib.sh > -+. "$TEST_DIRECTORY"/lib-submodule-update.sh > -+ > -+test_expect_success 'setup' ' > -+ create_lib_submodule_repo && > -+ git -C submodule_update_repo checkout valid_sub1 && > -+ git -C submodule_update_repo submodule update > -+' > -+ > -+check_tar() { > -+ tarfile=$1.tar > -+ listfile=$1.lst > -+ dir=$1 > -+ dir_with_prefix=$dir/$2 > -+ > -+ test_expect_success ' extract tar archive' ' > -+ (mkdir $dir && cd $dir && "$TAR" xf -) <$tarfile > -+ ' > -+} > -+ > -+check_added() { > -+ dir=$1 > -+ path_in_fs=$2 > -+ path_in_archive=$3 > -+ > -+ test_expect_success " validate extra file $path_in_archive" ' > -+ test -f $dir/$path_in_archive && > -+ diff -r $path_in_fs $dir/$path_in_archive > -+ ' > -+} > -+ > -+check_not_added() { > -+ dir=$1 > -+ path_in_archive=$2 > -+ > -+ test_expect_success " validate unpresent file $path_in_archive" ' > -+ ! test -f $dir/$path_in_archive && > -+ ! test -d $dir/$path_in_archive > -+ ' > -+} > -+ > -+test_expect_success 'archive without recurse, non-init' ' > -+ reset_work_tree_to valid_sub1 && > -+ git -C submodule_update archive HEAD >b.tar > -+' > -+ > -+check_tar b > -+check_added b submodule_update/file1 file1 > -+check_not_added b sub1/file1 > -+ > -+test_expect_success 'archive with recurse, non-init' ' > -+ reset_work_tree_to valid_sub1 && > -+ ! git -C submodule_update archive --recurse-submodules HEAD >b2-err.tar > -+' > -+ > -+test_expect_success 'archive with recurse, init' ' > -+ reset_work_tree_to valid_sub1 && > -+ git -C submodule_update submodule update --init && > -+ git -C submodule_update ls-files --recurse-submodules && > -+ git -C submodule_update ls-tree HEAD && > -+ git -C submodule_update archive --recurse-submodules HEAD >b2.tar > -+' > -+ > -+check_tar b2 > -+check_added b2 submodule_update/sub1/file1 sub1/file1 > -+ > -+test_expect_success 'archive with recurse with big files' ' > -+ reset_work_tree_to valid_sub1 && > -+ test_config core.bigfilethreshold 1 && > -+ git -C submodule_update submodule update --init && > -+ git -C submodule_update ls-files --recurse-submodules && > -+ git -C submodule_update ls-tree HEAD && > -+ git -C submodule_update archive --recurse-submodules HEAD >b3.tar > -+' > -+ > -+check_tar b3 > -+check_added b3 submodule_update/sub1/file1 sub1/file1 > -+ > -+ > -+test_done > - > - ## tree.c ## > -@@ > - #include "alloc.h" > - #include "tree-walk.h" > - #include "repository.h" > -+#include "pathspec.h" > - > - const char *tree_type = "tree"; > - > -@@ tree.c: int read_tree_at(struct repository *r, > - int len, oldlen = base->len; > - enum interesting retval = entry_not_interesting; > - > -- if (parse_tree(tree)) > -- return -1; > -+ if (repo_parse_tree(r, tree)) > -+ die("Failed to parse tree"); > - > - init_tree_desc(&desc, tree->buffer, tree->size); > - > -@@ tree.c: int read_tree_at(struct repository *r, > - continue; > - } > - > -- switch (fn(&entry.oid, base, > -+ switch (fn(r, &entry.oid, base, > - entry.path, entry.mode, context)) { > - case 0: > - continue; > -@@ tree.c: int read_tree_at(struct repository *r, > - return -1; > - } > - > -- if (S_ISDIR(entry.mode)) > -+ if (S_ISDIR(entry.mode)) { > - oidcpy(&oid, &entry.oid); > -- else if (S_ISGITLINK(entry.mode)) { > -+ len = tree_entry_len(&entry); > -+ strbuf_add(base, entry.path, len); > -+ strbuf_addch(base, '/'); > -+ retval = read_tree_at(r, lookup_tree(r, &oid), > -+ base, pathspec, > -+ fn, context); > -+ strbuf_setlen(base, oldlen); > -+ if (retval) > -+ return -1; > -+ } else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) { > - struct commit *commit; > -+ struct repository subrepo; > -+ struct repository* subrepo_p = &subrepo; > -+ struct tree* submodule_tree; > - > -- commit = lookup_commit(r, &entry.oid); > -+ if (repo_submodule_init(subrepo_p, r, entry.path, null_oid())) > -+ die("couldn't init submodule %s%s", base->buf, entry.path); > -+ > -+ if (repo_read_index(subrepo_p) < 0) > -+ die("index file corrupt"); > -+ > -+ commit = lookup_commit(subrepo_p, &entry.oid); > - if (!commit) > - die("Commit %s in submodule path %s%s not found", > - oid_to_hex(&entry.oid), > - base->buf, entry.path); > - > -- if (parse_commit(commit)) > -+ if (repo_parse_commit(subrepo_p, commit)) > - die("Invalid commit %s in submodule path %s%s", > - oid_to_hex(&entry.oid), > - base->buf, entry.path); > - > -- oidcpy(&oid, get_commit_tree_oid(commit)); > -+ submodule_tree = repo_get_commit_tree(subrepo_p, commit); > -+ oidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL); > -+ > -+ len = tree_entry_len(&entry); > -+ strbuf_add(base, entry.path, len); > -+ strbuf_addch(base, '/'); > -+ retval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid), > -+ base, pathspec, > -+ fn, context); > -+ if (retval) { > -+ die("failed to read tree for %s%s", base->buf, entry.path); > -+ return -1; > -+ } > -+ strbuf_setlen(base, oldlen); > -+ repo_clear(subrepo_p); > - } > -- else > -- continue; > - > -- len = tree_entry_len(&entry); > -- strbuf_add(base, entry.path, len); > -- strbuf_addch(base, '/'); > -- retval = read_tree_at(r, lookup_tree(r, &oid), > -- base, pathspec, > -- fn, context); > -- strbuf_setlen(base, oldlen); > -- if (retval) > -- return -1; > - } > - return 0; > - } > -@@ tree.c: int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size) > - return 0; > - } > - > --int parse_tree_gently(struct tree *item, int quiet_on_missing) > -+int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing) > - { > - enum object_type type; > - void *buffer; > -@@ tree.c: int parse_tree_gently(struct tree *item, int quiet_on_missing) > - > - if (item->object.parsed) > - return 0; > -- buffer = read_object_file(&item->object.oid, &type, &size); > -+ buffer = repo_read_object_file(r, &item->object.oid, &type, &size); > - if (!buffer) > - return quiet_on_missing ? -1 : > - error("Could not read %s", > - > - ## tree.h ## > -@@ tree.h: struct tree *lookup_tree(struct repository *r, const struct object_id *oid); > - > - int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size); > - > --int parse_tree_gently(struct tree *tree, int quiet_on_missing); > --static inline int parse_tree(struct tree *tree) > -+int parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing); > -+static inline int repo_parse_tree(struct repository *r, struct tree *tree) > - { > -- return parse_tree_gently(tree, 0); > -+ return parse_tree_gently(r, tree, 0); > - } > -+#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS > -+#define parse_tree(tree) repo_parse_tree(the_repository, tree) > -+#endif > - void free_tree_buffer(struct tree *tree); > - > - /* Parses and returns the tree in the given ent, chasing tags and commits. */ > -@@ tree.h: struct tree *parse_tree_indirect(const struct object_id *oid); > - int cmp_cache_name_compare(const void *a_, const void *b_); > - > - #define READ_TREE_RECURSIVE 1 > --typedef int (*read_tree_fn_t)(const struct object_id *, struct strbuf *, const char *, unsigned int, void *); > -+typedef int (*read_tree_fn_t)(struct repository *, const struct object_id *, struct strbuf *, const char *, unsigned int, void *); > - > - int read_tree_at(struct repository *r, > - struct tree *tree, struct strbuf *base, > - > - ## wt-status.c ## > -@@ wt-status.c: static void wt_status_collect_changes_index(struct wt_status *s) > - release_revisions(&rev); > - } > - > --static int add_file_to_list(const struct object_id *oid, > -+static int add_file_to_list(struct repository *repo UNUSED, const struct object_id *oid, > - struct strbuf *base, const char *path, > - unsigned int mode, void *context) > - { > -: ----------- > 7: 2443c9b1b6e archive: remove global repository from archive_args > -: ----------- > 8: 4672e3d9586 archive: add --recurse-submodules to git-archive command > -: ----------- > 9: f88ebbaf17c archive: add tests for git archive --recurse-submodules >