Re: [PATCH v3 0/9] archive: Add --recurse-submodules to git-archive command

[Phillip Wood <phillip.wood123@gmail.com>]

Hi Heather

On 17/10/2022 03:23, Heather Lapointe via GitGitGadget wrote:
> This makes it possible to include submodule contents in an archive command.
> 
> The inspiration for this change comes from this Github thread,
> https://github.com/dear-github/dear-github/issues/214, with at least 160
> 👍🏻 's at the time of writing. (I stumbled upon it because I wanted it as
> well).
> 
> I figured the underlying implementation wouldn't be too difficult with most
> of the plumbing already in place, so I decided to add the relevant logic to
> the client git-archive command.
> 
> One of the trickier parts of this implementation involved teaching read_tree
> about submodules. Some of the troublesome areas were still using the
> the_repository references to look up commit or tree or oid information. I
> ended up deciding that read_tree_fn_t would probably be best off having a
> concrete repo reference since it allows changing the context to a subrepo
> where needed (even though some of the usages did not need it specifically).
> 
> I am open to feedback since this is all quite new to me :)

I've had a quick read through and I thought this patch series was well 
structured and easy to follow for someone like me who is not familiar 
with the archive code. I've left a few mostly high-level comments on 
some of the individual patches. The commit messages are all pretty brief 
and would benefit from a bit more explaintion of why the changes are 
being made. From a brief read through the code changes themselves all 
looked pretty good.

Best Wishes

Phillip


> Alphadelta14 (1):
>    tree: do not use the_repository for tree traversal methods.
 >
> Heather Lapointe (8):
>    tree: update cases to use repo_ tree methods
>    tree: increase test coverage for tree.c
>    tree: handle submodule case for read_tree_at properly
>    tree: add repository parameter to read_tree_fn_t
>    archive: pass repo objects to write_archive handlers
>    archive: remove global repository from archive_args
>    archive: add --recurse-submodules to git-archive command
>    archive: add tests for git archive --recurse-submodules
> 
>   Documentation/git-archive.txt     |   6 +-
>   Makefile                          |   1 +
>   archive-tar.c                     |  15 ++--
>   archive-zip.c                     |  15 ++--
>   archive.c                         | 138 ++++++++++++++++++++----------
>   archive.h                         |  16 +++-
>   builtin/checkout.c                |   4 +-
>   builtin/log.c                     |   4 +-
>   builtin/ls-files.c                |   8 +-
>   builtin/ls-tree.c                 |  34 +++++---
>   merge-recursive.c                 |   4 +-
>   merge.c                           |   4 +-
>   reset.c                           |   2 +-
>   revision.c                        |   4 +-
>   sequencer.c                       |   6 +-
>   sparse-index.c                    |   4 +-
>   t/helper/test-tool.c              |   1 +
>   t/helper/test-tool.h              |   1 +
>   t/helper/test-tree-read-tree-at.c |  41 +++++++++
>   t/t1023-tree-read-tree-at.sh      |  65 ++++++++++++++
>   t/t5005-archive-submodules.sh     |  83 ++++++++++++++++++
>   tree.c                            |  93 ++++++++++++++------
>   tree.h                            |  16 ++--
>   wt-status.c                       |   4 +-
>   24 files changed, 448 insertions(+), 121 deletions(-)
>   create mode 100644 t/helper/test-tree-read-tree-at.c
>   create mode 100755 t/t1023-tree-read-tree-at.sh
>   create mode 100755 t/t5005-archive-submodules.sh
> 
> 
> base-commit: e85701b4af5b7c2a9f3a1b07858703318dce365d
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1359%2FAlphadelta14%2Farchive-recurse-submodules-v3
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1359/Alphadelta14/archive-recurse-submodules-v3
> Pull-Request: https://github.com/git/git/pull/1359
> 
> Range-diff vs v2:
> 
>    -:  ----------- >  1:  79959a54eb4 tree: do not use the_repository for tree traversal methods.
>    -:  ----------- >  2:  2291e0f9b5c tree: update cases to use repo_ tree methods
>    -:  ----------- >  3:  9a07c6932f4 tree: increase test coverage for tree.c
>    2:  68f7830c6d9 !  4:  d3d1738e670 archive: fix a case of submodule in submodule traversal
>       @@
>         ## Metadata ##
>       -Author: Alphadelta14 <alpha@alphaservcomputing.solutions>
>       +Author: Heather Lapointe <alpha@alphaservcomputing.solutions>
>        
>         ## Commit message ##
>       -    archive: fix a case of submodule in submodule traversal
>       +    tree: handle submodule case for read_tree_at properly
>        
>       -    repo_submodule_init actually expects the path relative to submodule_prefix.
>       -    We preform a simple strip to the correct path.
>       +    This supports traversal into an actual submodule for read_tree_at.
>       +    The logic is blocked on pathspec->recurse_submodules now,
>       +    but previously hadn't been executed due to all fn() cases
>       +    returning early for submodules.
>        
>       -    Signed-off-by: Alphadelta14 <alpha@alphaservcomputing.solutions>
>       +    Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>
>        
>         ## tree.c ##
>       +@@
>       + #include "alloc.h"
>       + #include "tree-walk.h"
>       + #include "repository.h"
>       ++#include "pathspec.h"
>       +
>       + const char *tree_type = "tree";
>       +
>        @@ tree.c: int read_tree_at(struct repository *r,
>       - 			struct repository subrepo;
>       - 			struct repository* subrepo_p = &subrepo;
>       - 			struct tree* submodule_tree;
>       + 			return -1;
>       + 		}
>       +
>       +-		if (S_ISDIR(entry.mode))
>       ++		if (S_ISDIR(entry.mode)) {
>       + 			oidcpy(&oid, &entry.oid);
>       +-		else if (S_ISGITLINK(entry.mode)) {
>       +-			struct commit *commit;
>       +
>       +-			commit = lookup_commit(r, &entry.oid);
>       ++			len = tree_entry_len(&entry);
>       ++			strbuf_add(base, entry.path, len);
>       ++			strbuf_addch(base, '/');
>       ++			retval = read_tree_at(r, lookup_tree(r, &oid),
>       ++						base, pathspec,
>       ++						fn, context);
>       ++			strbuf_setlen(base, oldlen);
>       ++			if (retval)
>       ++				return -1;
>       ++		} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {
>       ++			struct commit *commit;
>       ++			struct repository subrepo;
>       ++			struct repository* subrepo_p = &subrepo;
>       ++			struct tree* submodule_tree;
>        +			char *submodule_rel_path;
>        +			int name_base_len = 0;
>       -
>       --			if (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))
>       --				die("couldn't init submodule %s%s", base->buf, entry.path);
>       ++
>        +			len = tree_entry_len(&entry);
>        +			strbuf_add(base, entry.path, len);
>        +			submodule_rel_path = base->buf;
>       @@ tree.c: int read_tree_at(struct repository *r,
>        +
>        +			if (repo_submodule_init(subrepo_p, r, submodule_rel_path, null_oid()))
>        +				die("couldn't init submodule %s", base->buf);
>       -
>       - 			if (repo_read_index(subrepo_p) < 0)
>       - 				die("index file corrupt");
>       -
>       - 			commit = lookup_commit(subrepo_p, &entry.oid);
>       ++
>       ++			if (repo_read_index(subrepo_p) < 0)
>       ++				die("index file corrupt");
>       ++
>       ++			commit = lookup_commit(subrepo_p, &entry.oid);
>         			if (!commit)
>        -				die("Commit %s in submodule path %s%s not found",
>        +				die("Commit %s in submodule path %s not found",
>         				    oid_to_hex(&entry.oid),
>        -				    base->buf, entry.path);
>       -+				    base->buf);
>       -
>       - 			if (repo_parse_commit(subrepo_p, commit))
>       +-
>       +-			// FIXME: This is the wrong repo instance (it refers to the superproject)
>       +-			// it will always fail as is (will fix in later patch)
>       +-			// This current codepath isn't executed by any existing callbacks
>       +-			// so it wouldn't show up as an issue at this time.
>       +-			if (repo_parse_commit(r, commit))
>        -				die("Invalid commit %s in submodule path %s%s",
>       ++				    base->buf);
>       ++
>       ++			if (repo_parse_commit(subrepo_p, commit))
>        +				die("Invalid commit %s in submodule path %s",
>         				    oid_to_hex(&entry.oid),
>        -				    base->buf, entry.path);
>        +				    base->buf);
>         
>       - 			submodule_tree = repo_get_commit_tree(subrepo_p, commit);
>       - 			oidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);
>       +-			oidcpy(&oid, get_commit_tree_oid(commit));
>       +-		}
>       +-		else
>       +-			continue;
>       ++			submodule_tree = repo_get_commit_tree(subrepo_p, commit);
>       ++			oidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);
>         
>       --			len = tree_entry_len(&entry);
>       --			strbuf_add(base, entry.path, len);
>       - 			strbuf_addch(base, '/');
>       +-		len = tree_entry_len(&entry);
>       +-		strbuf_add(base, entry.path, len);
>       +-		strbuf_addch(base, '/');
>       +-		retval = read_tree_at(r, lookup_tree(r, &oid),
>       +-				      base, pathspec,
>       +-				      fn, context);
>       +-		strbuf_setlen(base, oldlen);
>       +-		if (retval)
>       +-			return -1;
>       ++			strbuf_addch(base, '/');
>        +
>       - 			retval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),
>       - 						base, pathspec,
>       - 						fn, context);
>       - 			if (retval) {
>       --			    die("failed to read tree for %s%s", base->buf, entry.path);
>       ++			retval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),
>       ++						base, pathspec,
>       ++						fn, context);
>       ++			if (retval)
>        +			    die("failed to read tree for %s", base->buf);
>       - 			    return -1;
>       - 			}
>       - 			strbuf_setlen(base, oldlen);
>       ++			strbuf_setlen(base, oldlen);
>       ++			repo_clear(subrepo_p);
>       ++		}
>       ++		// else, this is a file (or a submodule, but no pathspec->recurse_submodules)
>       + 	}
>       + 	return 0;
>       + }
>    -:  ----------- >  5:  376345fdf66 tree: add repository parameter to read_tree_fn_t
>    1:  41664a59029 !  6:  1b9b049d64f archive: add --recurse-submodules to git-archive command
>       @@
>         ## Metadata ##
>       -Author: Alphadelta14 <alpha@alphaservcomputing.solutions>
>       +Author: Heather Lapointe <alpha@alphaservcomputing.solutions>
>        
>         ## Commit message ##
>       -    archive: add --recurse-submodules to git-archive command
>       +    archive: pass repo objects to write_archive handlers
>        
>       -    This makes it possible to include submodule contents in an archive command.
>       -
>       -    This required updating the general read_tree callbacks to support sub-repos
>       -    by not using the_repository global references where possible.
>       -
>       -    archive: update streaming to use target repo
>       -    archive: add test cases for git archive --recurse-submodules
>       +    Use contextual repos instead of the_repository or args->repo
>       +    to ensure that submodules will be handled correctly
>       +    since they use multiple repo instances.
>        
>            Signed-off-by: Heather Lapointe <alpha@alphaservcomputing.solutions>
>        
>       @@ archive-tar.c: static unsigned long offset;
>         static int tar_umask = 002;
>         
>         static int write_tar_filter_archive(const struct archiver *ar,
>       -+				    struct repository *repo,
>       ++					struct repository *repo,
>         				    struct archiver_args *args);
>         
>         /*
>       @@ archive-tar.c: static void write_extended_header(struct archiver_args *args,
>         }
>         
>        -static int write_tar_entry(struct archiver_args *args,
>       -+static int write_tar_entry(struct repository *repo,
>       ++static int write_tar_entry(
>       ++			   struct repository *repo,
>        +			   struct archiver_args *args,
>         			   const struct object_id *oid,
>         			   const char *path, size_t pathlen,
>       @@ archive-tar.c: static void tgz_write_block(const void *data)
>         static const char internal_gzip_command[] = "git archive gzip";
>         
>         static int write_tar_filter_archive(const struct archiver *ar,
>       -+				    struct repository *repo,
>       ++					struct repository *repo,
>         				    struct archiver_args *args)
>         {
>         #if ZLIB_VERNUM >= 0x1221
>       @@ archive-zip.c: static int entry_is_binary(struct index_state *istate, const char
>         #define STREAM_BUFFER_SIZE (1024 * 16)
>         
>        -static int write_zip_entry(struct archiver_args *args,
>       -+static int write_zip_entry(struct repository *repo,
>       ++static int write_zip_entry(
>       ++			   struct repository *repo,
>        +			   struct archiver_args *args,
>         			   const struct object_id *oid,
>         			   const char *path, size_t pathlen,
>       @@ archive-zip.c: static int archive_zip_config(const char *var, const char *value,
>         }
>         
>         static int write_zip_archive(const struct archiver *ar UNUSED,
>       -+			     struct repository *repo,
>       ++				 struct repository *repo,
>         			     struct archiver_args *args)
>         {
>         	int err;
>       @@ archive-zip.c: static int write_zip_archive(const struct archiver *ar UNUSED,
>         
>        
>         ## archive.c ##
>       -@@
>       - #include "unpack-trees.h"
>       - #include "dir.h"
>       - #include "quote.h"
>       -+#include "submodule.h"
>       -
>       - static char const * const archive_usage[] = {
>       - 	N_("git archive [<options>] <tree-ish> [<path>...]"),
>       -@@ archive.c: static void format_subst(const struct commit *commit,
>       - }
>       -
>       - static void *object_file_to_archive(const struct archiver_args *args,
>       -+				    struct repository *repo,
>       - 				    const char *path,
>       - 				    const struct object_id *oid,
>       - 				    unsigned int mode,
>       -@@ archive.c: static void *object_file_to_archive(const struct archiver_args *args,
>       - 			       (args->tree ? &args->tree->object.oid : NULL), oid);
>       -
>       - 	path += args->baselen;
>       --	buffer = read_object_file(oid, type, sizep);
>       -+	buffer = repo_read_object_file(repo, oid, type, sizep);
>       - 	if (buffer && S_ISREG(mode)) {
>       - 		struct strbuf buf = STRBUF_INIT;
>       - 		size_t size = 0;
>       -
>       - 		strbuf_attach(&buf, buffer, *sizep, *sizep + 1);
>       --		convert_to_working_tree(args->repo->index, path, buf.buf, buf.len, &buf, &meta);
>       -+		convert_to_working_tree(repo->index, path, buf.buf, buf.len, &buf, &meta);
>       - 		if (commit)
>       - 			format_subst(commit, buf.buf, buf.len, &buf, args->pretty_ctx);
>       - 		buffer = strbuf_detach(&buf, &size);
>        @@ archive.c: static int check_attr_export_subst(const struct attr_check *check)
>         	return check && ATTR_TRUE(check->items[1].value);
>         }
>         
>        -static int write_archive_entry(const struct object_id *oid, const char *base,
>       -+static int write_archive_entry(struct repository *repo, const struct object_id *oid, const char *base,
>       ++static int write_archive_entry(
>       ++		struct repository *repo,
>       ++		const struct object_id *oid, const char *base,
>         		int baselen, const char *filename, unsigned mode,
>         		void *context)
>         {
>       @@ archive.c: static int write_archive_entry(const struct object_id *oid, const cha
>        +		err = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, 0);
>         		if (err)
>         			return err;
>       --		return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);
>       -+		return READ_TREE_RECURSIVE;
>       - 	}
>       -
>       - 	if (args->verbose)
>       + 		return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);
>        @@ archive.c: static int write_archive_entry(const struct object_id *oid, const char *base,
>         
>         	/* Stream it? */
>         	if (S_ISREG(mode) && !args->convert &&
>        -	    oid_object_info(args->repo, oid, &size) == OBJ_BLOB &&
>       --	    size > big_file_threshold)
>       --		return write_entry(args, oid, path.buf, path.len, mode, NULL, size);
>        +	    oid_object_info(repo, oid, &size) == OBJ_BLOB &&
>       -+	    size > big_file_threshold) {
>       -+			err = write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);
>       -+			if (err) {
>       -+				die("Failed to write file %.*s", (int)path.len, path.buf);
>       -+			}
>       -+			return err;
>       -+		}
>       + 	    size > big_file_threshold)
>       +-		return write_entry(args, oid, path.buf, path.len, mode, NULL, size);
>       ++		return write_entry(repo, args, oid, path.buf, path.len, mode, NULL, size);
>         
>       --	buffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);
>       -+	buffer = object_file_to_archive(args, repo, path.buf, oid, mode, &type, &size);
>       + 	buffer = object_file_to_archive(args, path.buf, oid, mode, &type, &size);
>         	if (!buffer)
>         		return error(_("cannot read '%s'"), oid_to_hex(oid));
>        -	err = write_entry(args, oid, path.buf, path.len, mode, buffer, size);
>       @@ archive.c: static void queue_directory(const struct object_id *oid,
>         }
>         
>        -static int write_directory(struct archiver_context *c)
>       -+static void queue_submodule(struct repository *superproject,
>       -+		const struct object_id *oid,
>       -+		struct strbuf *base, const char *filename,
>       -+		unsigned mode, struct archiver_context *c)
>       -+{
>       -+	struct repository subrepo;
>       -+
>       -+	if (repo_submodule_init(&subrepo, superproject, filename, null_oid()))
>       -+		return;
>       -+
>       -+	if (repo_read_index(&subrepo) < 0)
>       -+		die("index file corrupt");
>       -+
>       -+    queue_directory(oid, base, filename, mode, c);
>       -+
>       -+	repo_clear(&subrepo);
>       -+}
>       -+
>       -+static int write_directory(struct repository *repo, struct archiver_context *c)
>       ++static int write_directory(
>       ++		struct repository *repo,
>       ++		struct archiver_context *c)
>         {
>         	struct directory *d = c->bottom;
>         	int ret;
>       @@ archive.c: static int write_directory(struct archiver_context *c)
>        +		write_directory(repo, c) ||
>        +		write_archive_entry(repo, &d->oid, d->path, d->baselen,
>         				    d->path + d->baselen, d->mode,
>       --				    c) != READ_TREE_RECURSIVE;
>       -+				    c);
>       + 				    c) != READ_TREE_RECURSIVE;
>         	free(d);
>       --	return ret ? -1 : 0;
>       -+	if (ret == READ_TREE_RECURSIVE)
>       -+		return 0;
>       -+	return ret;
>       - }
>       -
>       --static int queue_or_write_archive_entry(const struct object_id *oid,
>       -+static int queue_or_write_archive_entry(
>       -+		struct repository *repo, const struct object_id *oid,
>       - 		struct strbuf *base, const char *filename,
>       - 		unsigned mode, void *context)
>       - {
>       -@@ archive.c: static int queue_or_write_archive_entry(const struct object_id *oid,
>       - 		/* Borrow base, but restore its original value when done. */
>       - 		strbuf_addstr(base, filename);
>       - 		strbuf_addch(base, '/');
>       --		check = get_archive_attrs(c->args->repo->index, base->buf);
>       -+		check = get_archive_attrs(repo->index, base->buf);
>       - 		strbuf_setlen(base, baselen);
>       -
>       - 		if (check_attr_export_ignore(check))
>       - 			return 0;
>       - 		queue_directory(oid, base, filename, mode, c);
>       +@@ archive.c: static int queue_or_write_archive_entry(
>         		return READ_TREE_RECURSIVE;
>       -+	} else if (c->args->recurse_submodules && S_ISGITLINK(mode)) {
>       -+		if (is_submodule_active(repo, filename)) {
>       -+			queue_submodule(repo, oid, base, filename, mode, c);
>       -+			return READ_TREE_RECURSIVE;
>       -+		}
>         	}
>         
>        -	if (write_directory(c))
>       -+	if (write_directory(repo, c))
>       ++	if (write_directory(r, c))
>         		return -1;
>        -	return write_archive_entry(oid, base->buf, base->len, filename, mode,
>       -+	return write_archive_entry(repo, oid, base->buf, base->len, filename, mode,
>       ++	return write_archive_entry(r, oid, base->buf, base->len, filename, mode,
>         				   context);
>         }
>         
>       @@ archive.c: struct extra_file_info {
>         };
>         
>        -int write_archive_entries(struct archiver_args *args,
>       -+int write_archive_entries(struct repository *repo,
>       ++int write_archive_entries(
>       ++		struct repository *repo,
>        +		struct archiver_args *args,
>         		write_archive_entry_fn_t write_entry)
>         {
>       @@ archive.c: int write_archive_entries(struct archiver_args *args,
>         				  len, 040777, NULL, 0);
>         		if (err)
>         			return err;
>       -@@ archive.c: int write_archive_entries(struct archiver_args *args,
>       - 		memset(&opts, 0, sizeof(opts));
>       - 		opts.index_only = 1;
>       - 		opts.head_idx = -1;
>       --		opts.src_index = args->repo->index;
>       --		opts.dst_index = args->repo->index;
>       -+		opts.src_index = repo->index;
>       -+		opts.dst_index = repo->index;
>       - 		opts.fn = oneway_merge;
>       - 		init_tree_desc(&t, args->tree->buffer, args->tree->size);
>       - 		if (unpack_trees(1, &t, &opts))
>       -@@ archive.c: int write_archive_entries(struct archiver_args *args,
>       - 		git_attr_set_direction(GIT_ATTR_INDEX);
>       - 	}
>       -
>       --	err = read_tree(args->repo, args->tree,
>       -+	err = read_tree(repo, args->tree,
>       - 			&args->pathspec,
>       - 			queue_or_write_archive_entry,
>       - 			&context);
>        @@ archive.c: int write_archive_entries(struct archiver_args *args,
>         			if (strbuf_read_file(&content, path, info->stat.st_size) < 0)
>         				err = error_errno(_("cannot read '%s'"), path);
>       @@ archive.c: int write_archive_entries(struct archiver_args *args,
>         					  path, strlen(path),
>         					  canon_mode(info->stat.st_mode),
>         					  info->content, info->stat.st_size);
>       -@@ archive.c: struct path_exists_context {
>       - 	struct archiver_args *args;
>       - };
>       -
>       --static int reject_entry(const struct object_id *oid UNUSED,
>       -+static int reject_entry(struct repository *repo, const struct object_id *oid UNUSED,
>       - 			struct strbuf *base,
>       - 			const char *filename, unsigned mode,
>       - 			void *context)
>       -@@ archive.c: static int reject_entry(const struct object_id *oid UNUSED,
>       - 		struct strbuf sb = STRBUF_INIT;
>       - 		strbuf_addbuf(&sb, base);
>       - 		strbuf_addstr(&sb, filename);
>       --		if (!match_pathspec(ctx->args->repo->index,
>       -+		if (!match_pathspec(repo->index,
>       - 				    &ctx->pathspec,
>       - 				    sb.buf, sb.len, 0, NULL, 1))
>       - 			ret = READ_TREE_RECURSIVE;
>       -@@ archive.c: static void parse_pathspec_arg(const char **pathspec,
>       - 		       PATHSPEC_PREFER_FULL,
>       - 		       "", pathspec);
>       - 	ar_args->pathspec.recursive = 1;
>       -+	ar_args->pathspec.recurse_submodules = ar_args->recurse_submodules;
>       - 	if (pathspec) {
>       - 		while (*pathspec) {
>       - 			if (**pathspec && !path_exists(ar_args, *pathspec))
>       -@@ archive.c: static int parse_archive_args(int argc, const char **argv,
>       - 	int verbose = 0;
>       - 	int i;
>       - 	int list = 0;
>       -+	int recurse_submodules = 0;
>       - 	int worktree_attributes = 0;
>       - 	struct option opts[] = {
>       - 		OPT_GROUP(""),
>       -@@ archive.c: static int parse_archive_args(int argc, const char **argv,
>       - 		  add_file_cb, (intptr_t)&base },
>       - 		OPT_STRING('o', "output", &output, N_("file"),
>       - 			N_("write the archive to this file")),
>       -+		OPT_BOOL(0, "recurse-submodules", &recurse_submodules,
>       -+			N_("include submodules in archive")),
>       - 		OPT_BOOL(0, "worktree-attributes", &worktree_attributes,
>       - 			N_("read .gitattributes in working directory")),
>       - 		OPT__VERBOSE(&verbose, N_("report archived files on stderr")),
>       -@@ archive.c: static int parse_archive_args(int argc, const char **argv,
>       - 	args->base = base;
>       - 	args->baselen = strlen(base);
>       - 	args->worktree_attributes = worktree_attributes;
>       -+	args->recurse_submodules = recurse_submodules;
>       -
>       - 	return argc;
>       - }
>        @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix,
>         	parse_treeish_arg(argv, &args, prefix, remote);
>         	parse_pathspec_arg(argv + 1, &args);
>       @@ archive.c: int write_archive(int argc, const char **argv, const char *prefix,
>         	free(args.refname);
>        
>         ## archive.h ##
>       -@@ archive.h: struct archiver_args {
>       - 	timestamp_t time;
>       - 	struct pathspec pathspec;
>       - 	unsigned int verbose : 1;
>       -+	unsigned int recurse_submodules : 1;
>       - 	unsigned int worktree_attributes : 1;
>       - 	unsigned int convert : 1;
>       - 	int compression_level;
>        @@ archive.h: const char *archive_format_from_filename(const char *filename);
>         #define ARCHIVER_HIGH_COMPRESSION_LEVELS 4
>         struct archiver {
>         	const char *name;
>        -	int (*write_archive)(const struct archiver *, struct archiver_args *);
>       -+	int (*write_archive)(const struct archiver *, struct repository *repo, struct archiver_args *);
>       ++	int (*write_archive)(
>       ++		const struct archiver *,
>       ++		struct repository *,
>       ++		struct archiver_args *);
>         	unsigned flags;
>         	char *filter_command;
>         };
>       @@ archive.h: void init_tar_archiver(void);
>         void init_archivers(void);
>         
>        -typedef int (*write_archive_entry_fn_t)(struct archiver_args *args,
>       -+typedef int (*write_archive_entry_fn_t)(struct repository *repo,
>       ++typedef int (*write_archive_entry_fn_t)(
>       ++					struct repository *repo,
>        +					struct archiver_args *args,
>         					const struct object_id *oid,
>         					const char *path, size_t pathlen,
>       @@ archive.h: void init_tar_archiver(void);
>         					void *buffer, unsigned long size);
>         
>        -int write_archive_entries(struct archiver_args *args, write_archive_entry_fn_t write_entry);
>       -+int write_archive_entries(struct repository *repo, struct archiver_args *args, write_archive_entry_fn_t write_entry);
>       ++int write_archive_entries(
>       ++	struct repository *repo,
>       ++	struct archiver_args *args,
>       ++	write_archive_entry_fn_t write_entry);
>         
>         #endif	/* ARCHIVE_H */
>       -
>       - ## builtin/checkout.c ##
>       -@@ builtin/checkout.c: static int post_checkout_hook(struct commit *old_commit, struct commit *new_comm
>       -
>       - }
>       -
>       --static int update_some(const struct object_id *oid, struct strbuf *base,
>       -+static int update_some(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,
>       - 		       const char *pathname, unsigned mode, void *context UNUSED)
>       - {
>       - 	int len;
>       -
>       - ## builtin/log.c ##
>       -@@ builtin/log.c: static int show_tag_object(const struct object_id *oid, struct rev_info *rev)
>       - 	return 0;
>       - }
>       -
>       --static int show_tree_object(const struct object_id *oid UNUSED,
>       -+static int show_tree_object(struct repository *repo UNUSED, const struct object_id *oid UNUSED,
>       - 			    struct strbuf *base UNUSED,
>       - 			    const char *pathname, unsigned mode,
>       - 			    void *context)
>       -
>       - ## builtin/ls-files.c ##
>       -@@ builtin/ls-files.c: static int get_common_prefix_len(const char *common_prefix)
>       - 	return common_prefix_len;
>       - }
>       -
>       --static int read_one_entry_opt(struct index_state *istate,
>       -+static int read_one_entry_opt(struct repository *repo UNUSED, struct index_state *istate,
>       - 			      const struct object_id *oid,
>       - 			      struct strbuf *base,
>       - 			      const char *pathname,
>       -@@ builtin/ls-files.c: static int read_one_entry_opt(struct index_state *istate,
>       - 	return add_index_entry(istate, ce, opt);
>       - }
>       -
>       --static int read_one_entry(const struct object_id *oid, struct strbuf *base,
>       -+static int read_one_entry(struct repository *repo, const struct object_id *oid, struct strbuf *base,
>       - 			  const char *pathname, unsigned mode,
>       - 			  void *context)
>       - {
>       - 	struct index_state *istate = context;
>       --	return read_one_entry_opt(istate, oid, base, pathname,
>       -+	return read_one_entry_opt(repo, istate, oid, base, pathname,
>       - 				  mode,
>       - 				  ADD_CACHE_OK_TO_ADD|ADD_CACHE_SKIP_DFCHECK);
>       - }
>       -@@ builtin/ls-files.c: static int read_one_entry(const struct object_id *oid, struct strbuf *base,
>       -  * This is used when the caller knows there is no existing entries at
>       -  * the stage that will conflict with the entry being added.
>       -  */
>       --static int read_one_entry_quick(const struct object_id *oid, struct strbuf *base,
>       -+static int read_one_entry_quick(struct repository *repo, const struct object_id *oid, struct strbuf *base,
>       - 				const char *pathname, unsigned mode,
>       - 				void *context)
>       - {
>       - 	struct index_state *istate = context;
>       --	return read_one_entry_opt(istate, oid, base, pathname,
>       -+	return read_one_entry_opt(repo, istate, oid, base, pathname,
>       - 				  mode, ADD_CACHE_JUST_APPEND);
>       - }
>       -
>       -
>       - ## builtin/ls-tree.c ##
>       -@@ builtin/ls-tree.c: static int show_recursive(const char *base, size_t baselen, const char *pathname
>       - 	return 0;
>       - }
>       -
>       --static int show_tree_fmt(const struct object_id *oid, struct strbuf *base,
>       -+static int show_tree_fmt(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,
>       - 			 const char *pathname, unsigned mode, void *context UNUSED)
>       - {
>       - 	size_t baselen;
>       -@@ builtin/ls-tree.c: static void show_tree_common_default_long(struct strbuf *base,
>       - 	strbuf_setlen(base, baselen);
>       - }
>       -
>       --static int show_tree_default(const struct object_id *oid, struct strbuf *base,
>       -+static int show_tree_default(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,
>       - 			     const char *pathname, unsigned mode,
>       - 			     void *context UNUSED)
>       - {
>       -@@ builtin/ls-tree.c: static int show_tree_default(const struct object_id *oid, struct strbuf *base,
>       - 	return recurse;
>       - }
>       -
>       --static int show_tree_long(const struct object_id *oid, struct strbuf *base,
>       -+static int show_tree_long(struct repository *repo, const struct object_id *oid, struct strbuf *base,
>       - 			  const char *pathname, unsigned mode,
>       - 			  void *context UNUSED)
>       - {
>       -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base,
>       -
>       - 	if (data.type == OBJ_BLOB) {
>       - 		unsigned long size;
>       --		if (oid_object_info(the_repository, data.oid, &size) == OBJ_BAD)
>       -+		if (oid_object_info(repo, data.oid, &size) == OBJ_BAD)
>       - 			xsnprintf(size_text, sizeof(size_text), "BAD");
>       - 		else
>       - 			xsnprintf(size_text, sizeof(size_text),
>       -@@ builtin/ls-tree.c: static int show_tree_long(const struct object_id *oid, struct strbuf *base,
>       - 	}
>       -
>       - 	printf("%06o %s %s %7s\t", data.mode, type_name(data.type),
>       --	       find_unique_abbrev(data.oid, abbrev), size_text);
>       -+	       repo_find_unique_abbrev(repo, data.oid, abbrev), size_text);
>       - 	show_tree_common_default_long(base, pathname, data.base->len);
>       - 	return recurse;
>       - }
>       -
>       --static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,
>       -+static int show_tree_name_only(struct repository *repo UNUSED, const struct object_id *oid, struct strbuf *base,
>       - 			       const char *pathname, unsigned mode,
>       - 			       void *context UNUSED)
>       - {
>       -@@ builtin/ls-tree.c: static int show_tree_name_only(const struct object_id *oid, struct strbuf *base,
>       - 	return recurse;
>       - }
>       -
>       --static int show_tree_object(const struct object_id *oid, struct strbuf *base,
>       -+static int show_tree_object(struct repository *repo, const struct object_id *oid, struct strbuf *base,
>       - 			    const char *pathname, unsigned mode,
>       - 			    void *context UNUSED)
>       - {
>       -@@ builtin/ls-tree.c: static int show_tree_object(const struct object_id *oid, struct strbuf *base,
>       - 	if (early >= 0)
>       - 		return early;
>       -
>       --	printf("%s%c", find_unique_abbrev(oid, abbrev), line_termination);
>       -+	printf("%s%c", repo_find_unique_abbrev(repo, oid, abbrev), line_termination);
>       - 	return recurse;
>       - }
>       -
>       -
>       - ## list-objects.c ##
>       -@@ list-objects.c: static void process_tree(struct traversal_context *ctx,
>       - 	    !revs->include_check_obj(&tree->object, revs->include_check_data))
>       - 		return;
>       -
>       --	failed_parse = parse_tree_gently(tree, 1);
>       -+	failed_parse = parse_tree_gently(revs->repo, tree, 1);
>       - 	if (failed_parse) {
>       - 		if (revs->ignore_missing_links)
>       - 			return;
>       -
>       - ## merge-recursive.c ##
>       -@@ merge-recursive.c: static void unpack_trees_finish(struct merge_options *opt)
>       - 	clear_unpack_trees_porcelain(&opt->priv->unpack_opts);
>       - }
>       -
>       --static int save_files_dirs(const struct object_id *oid UNUSED,
>       -+static int save_files_dirs(struct repository *repo UNUSED, const struct object_id *oid UNUSED,
>       - 			   struct strbuf *base, const char *path,
>       - 			   unsigned int mode, void *context)
>       - {
>       -
>       - ## revision.c ##
>       -@@ revision.c: static void mark_tree_contents_uninteresting(struct repository *r,
>       - 	struct tree_desc desc;
>       - 	struct name_entry entry;
>       -
>       --	if (parse_tree_gently(tree, 1) < 0)
>       -+	if (parse_tree_gently(r, tree, 1) < 0)
>       - 		return;
>       -
>       - 	init_tree_desc(&desc, tree->buffer, tree->size);
>       -@@ revision.c: static void add_children_by_path(struct repository *r,
>       - 	if (!tree)
>       - 		return;
>       -
>       --	if (parse_tree_gently(tree, 1) < 0)
>       -+	if (parse_tree_gently(r, tree, 1) < 0)
>       - 		return;
>       -
>       - 	init_tree_desc(&desc, tree->buffer, tree->size);
>       -
>       - ## sparse-index.c ##
>       -@@ sparse-index.c: static void set_index_entry(struct index_state *istate, int nr, struct cache_ent
>       - 	add_name_hash(istate, ce);
>       - }
>       -
>       --static int add_path_to_index(const struct object_id *oid,
>       -+static int add_path_to_index(struct repository *repo UNUSED, const struct object_id *oid,
>       - 			     struct strbuf *base, const char *path,
>       - 			     unsigned int mode, void *context)
>       - {
>       -
>       - ## t/t5005-archive-submodules.sh (new) ##
>       -@@
>       -+#!/bin/sh
>       -+
>       -+test_description='git archive --recurse-submodules test'
>       -+
>       -+. ./test-lib.sh
>       -+. "$TEST_DIRECTORY"/lib-submodule-update.sh
>       -+
>       -+test_expect_success 'setup' '
>       -+	create_lib_submodule_repo &&
>       -+	git -C submodule_update_repo checkout valid_sub1 &&
>       -+	git -C submodule_update_repo submodule update
>       -+'
>       -+
>       -+check_tar() {
>       -+	tarfile=$1.tar
>       -+	listfile=$1.lst
>       -+	dir=$1
>       -+	dir_with_prefix=$dir/$2
>       -+
>       -+	test_expect_success ' extract tar archive' '
>       -+		(mkdir $dir && cd $dir && "$TAR" xf -) <$tarfile
>       -+	'
>       -+}
>       -+
>       -+check_added() {
>       -+	dir=$1
>       -+	path_in_fs=$2
>       -+	path_in_archive=$3
>       -+
>       -+	test_expect_success " validate extra file $path_in_archive" '
>       -+		test -f $dir/$path_in_archive &&
>       -+		diff -r $path_in_fs $dir/$path_in_archive
>       -+	'
>       -+}
>       -+
>       -+check_not_added() {
>       -+	dir=$1
>       -+	path_in_archive=$2
>       -+
>       -+	test_expect_success " validate unpresent file $path_in_archive" '
>       -+		! test -f $dir/$path_in_archive &&
>       -+		! test -d $dir/$path_in_archive
>       -+	'
>       -+}
>       -+
>       -+test_expect_success 'archive without recurse, non-init' '
>       -+	reset_work_tree_to valid_sub1 &&
>       -+	git -C submodule_update archive HEAD >b.tar
>       -+'
>       -+
>       -+check_tar b
>       -+check_added b submodule_update/file1 file1
>       -+check_not_added b sub1/file1
>       -+
>       -+test_expect_success 'archive with recurse, non-init' '
>       -+	reset_work_tree_to valid_sub1 &&
>       -+	! git -C submodule_update archive --recurse-submodules HEAD >b2-err.tar
>       -+'
>       -+
>       -+test_expect_success 'archive with recurse, init' '
>       -+	reset_work_tree_to valid_sub1 &&
>       -+	git -C submodule_update submodule update --init &&
>       -+	git -C submodule_update ls-files --recurse-submodules &&
>       -+	git -C submodule_update ls-tree HEAD &&
>       -+	git -C submodule_update archive --recurse-submodules HEAD >b2.tar
>       -+'
>       -+
>       -+check_tar b2
>       -+check_added b2 submodule_update/sub1/file1 sub1/file1
>       -+
>       -+test_expect_success 'archive with recurse with big files' '
>       -+	reset_work_tree_to valid_sub1 &&
>       -+	test_config core.bigfilethreshold 1 &&
>       -+	git -C submodule_update submodule update --init &&
>       -+	git -C submodule_update ls-files --recurse-submodules &&
>       -+	git -C submodule_update ls-tree HEAD &&
>       -+	git -C submodule_update archive --recurse-submodules HEAD >b3.tar
>       -+'
>       -+
>       -+check_tar b3
>       -+check_added b3 submodule_update/sub1/file1 sub1/file1
>       -+
>       -+
>       -+test_done
>       -
>       - ## tree.c ##
>       -@@
>       - #include "alloc.h"
>       - #include "tree-walk.h"
>       - #include "repository.h"
>       -+#include "pathspec.h"
>       -
>       - const char *tree_type = "tree";
>       -
>       -@@ tree.c: int read_tree_at(struct repository *r,
>       - 	int len, oldlen = base->len;
>       - 	enum interesting retval = entry_not_interesting;
>       -
>       --	if (parse_tree(tree))
>       --		return -1;
>       -+	if (repo_parse_tree(r, tree))
>       -+		die("Failed to parse tree");
>       -
>       - 	init_tree_desc(&desc, tree->buffer, tree->size);
>       -
>       -@@ tree.c: int read_tree_at(struct repository *r,
>       - 				continue;
>       - 		}
>       -
>       --		switch (fn(&entry.oid, base,
>       -+		switch (fn(r, &entry.oid, base,
>       - 			   entry.path, entry.mode, context)) {
>       - 		case 0:
>       - 			continue;
>       -@@ tree.c: int read_tree_at(struct repository *r,
>       - 			return -1;
>       - 		}
>       -
>       --		if (S_ISDIR(entry.mode))
>       -+		if (S_ISDIR(entry.mode)) {
>       - 			oidcpy(&oid, &entry.oid);
>       --		else if (S_ISGITLINK(entry.mode)) {
>       -+			len = tree_entry_len(&entry);
>       -+			strbuf_add(base, entry.path, len);
>       -+			strbuf_addch(base, '/');
>       -+			retval = read_tree_at(r, lookup_tree(r, &oid),
>       -+						base, pathspec,
>       -+						fn, context);
>       -+			strbuf_setlen(base, oldlen);
>       -+			if (retval)
>       -+				return -1;
>       -+		} else if (pathspec->recurse_submodules && S_ISGITLINK(entry.mode)) {
>       - 			struct commit *commit;
>       -+			struct repository subrepo;
>       -+			struct repository* subrepo_p = &subrepo;
>       -+			struct tree* submodule_tree;
>       -
>       --			commit = lookup_commit(r, &entry.oid);
>       -+			if (repo_submodule_init(subrepo_p, r, entry.path, null_oid()))
>       -+				die("couldn't init submodule %s%s", base->buf, entry.path);
>       -+
>       -+			if (repo_read_index(subrepo_p) < 0)
>       -+				die("index file corrupt");
>       -+
>       -+			commit = lookup_commit(subrepo_p, &entry.oid);
>       - 			if (!commit)
>       - 				die("Commit %s in submodule path %s%s not found",
>       - 				    oid_to_hex(&entry.oid),
>       - 				    base->buf, entry.path);
>       -
>       --			if (parse_commit(commit))
>       -+			if (repo_parse_commit(subrepo_p, commit))
>       - 				die("Invalid commit %s in submodule path %s%s",
>       - 				    oid_to_hex(&entry.oid),
>       - 				    base->buf, entry.path);
>       -
>       --			oidcpy(&oid, get_commit_tree_oid(commit));
>       -+			submodule_tree = repo_get_commit_tree(subrepo_p, commit);
>       -+			oidcpy(&oid, submodule_tree ? &submodule_tree->object.oid : NULL);
>       -+
>       -+			len = tree_entry_len(&entry);
>       -+			strbuf_add(base, entry.path, len);
>       -+			strbuf_addch(base, '/');
>       -+			retval = read_tree_at(subrepo_p, lookup_tree(subrepo_p, &oid),
>       -+						base, pathspec,
>       -+						fn, context);
>       -+			if (retval) {
>       -+			    die("failed to read tree for %s%s", base->buf, entry.path);
>       -+			    return -1;
>       -+			}
>       -+			strbuf_setlen(base, oldlen);
>       -+			repo_clear(subrepo_p);
>       - 		}
>       --		else
>       --			continue;
>       -
>       --		len = tree_entry_len(&entry);
>       --		strbuf_add(base, entry.path, len);
>       --		strbuf_addch(base, '/');
>       --		retval = read_tree_at(r, lookup_tree(r, &oid),
>       --				      base, pathspec,
>       --				      fn, context);
>       --		strbuf_setlen(base, oldlen);
>       --		if (retval)
>       --			return -1;
>       - 	}
>       - 	return 0;
>       - }
>       -@@ tree.c: int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size)
>       - 	return 0;
>       - }
>       -
>       --int parse_tree_gently(struct tree *item, int quiet_on_missing)
>       -+int parse_tree_gently(struct repository *r, struct tree *item, int quiet_on_missing)
>       - {
>       - 	 enum object_type type;
>       - 	 void *buffer;
>       -@@ tree.c: int parse_tree_gently(struct tree *item, int quiet_on_missing)
>       -
>       - 	if (item->object.parsed)
>       - 		return 0;
>       --	buffer = read_object_file(&item->object.oid, &type, &size);
>       -+	buffer = repo_read_object_file(r, &item->object.oid, &type, &size);
>       - 	if (!buffer)
>       - 		return quiet_on_missing ? -1 :
>       - 			error("Could not read %s",
>       -
>       - ## tree.h ##
>       -@@ tree.h: struct tree *lookup_tree(struct repository *r, const struct object_id *oid);
>       -
>       - int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size);
>       -
>       --int parse_tree_gently(struct tree *tree, int quiet_on_missing);
>       --static inline int parse_tree(struct tree *tree)
>       -+int parse_tree_gently(struct repository *r, struct tree *tree, int quiet_on_missing);
>       -+static inline int repo_parse_tree(struct repository *r, struct tree *tree)
>       - {
>       --	return parse_tree_gently(tree, 0);
>       -+	return parse_tree_gently(r, tree, 0);
>       - }
>       -+#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS
>       -+#define parse_tree(tree) repo_parse_tree(the_repository, tree)
>       -+#endif
>       - void free_tree_buffer(struct tree *tree);
>       -
>       - /* Parses and returns the tree in the given ent, chasing tags and commits. */
>       -@@ tree.h: struct tree *parse_tree_indirect(const struct object_id *oid);
>       - int cmp_cache_name_compare(const void *a_, const void *b_);
>       -
>       - #define READ_TREE_RECURSIVE 1
>       --typedef int (*read_tree_fn_t)(const struct object_id *, struct strbuf *, const char *, unsigned int, void *);
>       -+typedef int (*read_tree_fn_t)(struct repository *, const struct object_id *, struct strbuf *, const char *, unsigned int, void *);
>       -
>       - int read_tree_at(struct repository *r,
>       - 		 struct tree *tree, struct strbuf *base,
>       -
>       - ## wt-status.c ##
>       -@@ wt-status.c: static void wt_status_collect_changes_index(struct wt_status *s)
>       - 	release_revisions(&rev);
>       - }
>       -
>       --static int add_file_to_list(const struct object_id *oid,
>       -+static int add_file_to_list(struct repository *repo UNUSED, const struct object_id *oid,
>       - 			    struct strbuf *base, const char *path,
>       - 			    unsigned int mode, void *context)
>       - {
>    -:  ----------- >  7:  2443c9b1b6e archive: remove global repository from archive_args
>    -:  ----------- >  8:  4672e3d9586 archive: add --recurse-submodules to git-archive command
>    -:  ----------- >  9:  f88ebbaf17c archive: add tests for git archive --recurse-submodules
>