git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
* [PATCH 00/26] Moving global state into the repository object (part 1)
@ 2018-02-13  1:22 Stefan Beller
  2018-02-13  1:22 ` [PATCH 01/26] repository: introduce raw object store field Stefan Beller
                   ` (30 more replies)
  0 siblings, 31 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller

This is a real take on the first part of the recent RFC[1].

Jonathan Tan suggested[2] that "sha1_loose_object_info to handle arbitrary repositories"
might be a good breaking point for a first part at that RFC at patch 38.
This series is smaller and contains only 26 patches as the patches in the big
RFC were slightly out of order.

I developed this series partly by writing patches, but mostly by cherrypicking
from that RFC on top of current master. I noticed no external conflicts apart
from one addition to the repositories _INIT macro, which was easy to resolve.

Comments in the early range of that RFC were on 003 where Junio pointed out
that the coccinelle patch ought to be not in contrib/coccinelle, so I put it
in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.

brian had a questoin on patch 25 in the RFC, but that seemed to resolve itself
without any suggestion to include into this series[3].

Duy suggested that we shall not use the repository blindly, but should carefully
examine whether to pass on an object store or the refstore or such[4], which 
I agree with if it makes sense. This series unfortunately has an issue with that
as I would not want to pass down the `ignore_env` flag separately from the object
store, so I made all functions that only take the object store to have the raw
object store as the first parameter, and others using the full repository.

Eric Sunshine brought up memory leaks with the RFC, and I would think to
have plugged all holes.

[1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/
[2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/
[3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/
[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/

Thanks,
Stefan

Jonathan Nieder (8):
  pack: move prepare_packed_git_run_once to object store
  pack: move approximate object count to object store
  sha1_file: add repository argument to sha1_file_name
  sha1_file: add repository argument to map_sha1_file
  sha1_file: allow stat_sha1_file to handle arbitrary repositories
  sha1_file: allow open_sha1_file to handle arbitrary repositories
  sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
  sha1_file: allow sha1_loose_object_info to handle arbitrary
    repositories

Stefan Beller (18):
  repository: introduce raw object store field
  object-store: move alt_odb_list and alt_odb_tail to object store
  object-store: free alt_odb_list
  object-store: move packed_git and packed_git_mru to object store
  object-store: close all packs upon clearing the object store
  sha1_file: add raw_object_store argument to alt_odb_usable
  sha1_file: add repository argument to link_alt_odb_entry
  sha1_file: add repository argument to read_info_alternates
  sha1_file: add repository argument to link_alt_odb_entries
  sha1_file: add repository argument to prepare_alt_odb
  sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
  sha1_file: allow prepare_alt_odb to handle arbitrary repositories
  sha1_file: add repository argument to stat_sha1_file
  sha1_file: add repository argument to open_sha1_file
  sha1_file: add repository argument to map_sha1_file_1
  sha1_file: add repository argument to sha1_loose_object_info
  sha1_file: allow sha1_file_name to handle arbitrary repositories
  sha1_file: allow map_sha1_file to handle arbitrary repositories

 builtin/am.c                                    |   2 +-
 builtin/clone.c                                 |   2 +-
 builtin/count-objects.c                         |   6 +-
 builtin/fetch.c                                 |   2 +-
 builtin/fsck.c                                  |  13 ++-
 builtin/gc.c                                    |   4 +-
 builtin/grep.c                                  |   2 +-
 builtin/index-pack.c                            |   1 +
 builtin/merge.c                                 |   2 +-
 builtin/pack-objects.c                          |  21 ++--
 builtin/pack-redundant.c                        |   6 +-
 builtin/receive-pack.c                          |   3 +-
 cache.h                                         |  46 ++------
 contrib/coccinelle/refactoring/packed_git.cocci |   7 ++
 environment.c                                   |   5 +-
 fast-import.c                                   |   6 +-
 http-backend.c                                  |   6 +-
 http-push.c                                     |   1 +
 http-walker.c                                   |   4 +-
 http.c                                          |   6 +-
 mru.h                                           |   1 +
 object-store.h                                  |  75 +++++++++++++
 object.c                                        |  26 +++++
 pack-bitmap.c                                   |   4 +-
 pack-check.c                                    |   1 +
 pack-revindex.c                                 |   1 +
 packfile.c                                      |  64 +++++------
 packfile.h                                      |   2 +-
 path.c                                          |   2 +-
 reachable.c                                     |   1 +
 repository.c                                    |  17 ++-
 repository.h                                    |   7 +-
 server-info.c                                   |   6 +-
 sha1_file.c                                     | 135 +++++++++++++-----------
 sha1_name.c                                     |  11 +-
 streaming.c                                     |   5 +-
 36 files changed, 321 insertions(+), 182 deletions(-)
 create mode 100644 contrib/coccinelle/refactoring/packed_git.cocci
 create mode 100644 object-store.h

-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 01/26] repository: introduce raw object store field
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
                   ` (29 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

The raw object store field will contain any objects needed for
access to objects in a given repository.

This patch introduces the raw object store and populates it with the
`objectdir`, which used to be part of the repository struct.

As the struct gains members, we'll also populate the function to clear
the memory for these members.

In a later we'll introduce a struct object_parser, that will complement
the object parsing in a repository struct: The raw object parser is the
layer that will provide access to raw object content, while the higher
level object parser code will parse raw objects and keeps track of
parenthood and other object relationships using 'struct object'.
For now only add the lower level to the repository struct.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/grep.c |  2 +-
 environment.c  |  5 +++--
 object-store.h | 15 +++++++++++++++
 object.c       |  5 +++++
 path.c         |  2 +-
 repository.c   | 17 +++++++++++++----
 repository.h   |  7 ++++---
 7 files changed, 42 insertions(+), 11 deletions(-)
 create mode 100644 object-store.h

diff --git a/builtin/grep.c b/builtin/grep.c
index 3ca4ac80d8..0f0c195705 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objectdir);
+	add_to_alternates_memory(submodule.objects.objectdir);
 	grep_read_unlock();
 
 	if (oid) {
diff --git a/environment.c b/environment.c
index 63ac38a46f..5d7ffb3a3f 100644
--- a/environment.c
+++ b/environment.c
@@ -13,6 +13,7 @@
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -242,9 +243,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objectdir)
+	if (!the_repository->objects.objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objectdir;
+	return the_repository->objects.objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
diff --git a/object-store.h b/object-store.h
new file mode 100644
index 0000000000..cf35760ceb
--- /dev/null
+++ b/object-store.h
@@ -0,0 +1,15 @@
+#ifndef OBJECT_STORE_H
+#define OBJECT_STORE_H
+
+struct raw_object_store {
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+};
+#define RAW_OBJECT_STORE_INIT { NULL }
+
+void raw_object_store_clear(struct raw_object_store *o);
+
+#endif /* OBJECT_STORE_H */
diff --git a/object.c b/object.c
index 0afdfd19b7..9b5b65e189 100644
--- a/object.c
+++ b/object.c
@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)
 			obj->flags &= ~flags;
 	}
 }
+
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	free(o->objectdir);
+}
diff --git a/path.c b/path.c
index da8b655730..81a42d9115 100644
--- a/path.c
+++ b/path.c
@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects.objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
diff --git a/repository.c b/repository.c
index f66fcb1342..bd2ad578de 100644
--- a/repository.c
+++ b/repository.c
@@ -1,11 +1,18 @@
 #include "cache.h"
 #include "repository.h"
+#include "object-store.h"
 #include "config.h"
 #include "submodule-config.h"
 
 /* The main repository */
 static struct repository the_repo = {
-	NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
+	NULL, NULL,
+	RAW_OBJECT_STORE_INIT,
+	NULL, NULL, NULL,
+	NULL, NULL, NULL,
+	&the_index,
+	&hash_algos[GIT_HASH_SHA1],
+	0, 0
 };
 struct repository *the_repository = &the_repo;
 
@@ -42,8 +49,8 @@ static void repo_setup_env(struct repository *repo)
 						    !repo->ignore_env);
 	free(repo->commondir);
 	repo->commondir = strbuf_detach(&sb, NULL);
-	free(repo->objectdir);
-	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+	free(repo->objects.objectdir);
+	repo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
 					    "objects", !repo->ignore_env);
 	free(repo->graft_file);
 	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
@@ -209,12 +216,14 @@ void repo_clear(struct repository *repo)
 {
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
-	FREE_AND_NULL(repo->objectdir);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
+	raw_object_store_clear(&repo->objects);
+	memset(&repo->objects, 0, sizeof(repo->objects));
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
diff --git a/repository.h b/repository.h
index 0329e40c7f..1f8bc7a7cf 100644
--- a/repository.h
+++ b/repository.h
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+#include "object-store.h"
+
 struct config_set;
 struct index_state;
 struct submodule_cache;
@@ -21,10 +23,9 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Path to the repository's object store.
-	 * Cannot be NULL after initialization.
+	 * Holds any information related to the object store.
 	 */
-	char *objectdir;
+	struct raw_object_store objects;
 
 	/*
 	 * Path to the repository's graft file.
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
  2018-02-13  1:22 ` [PATCH 01/26] repository: introduce raw object store field Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13 18:51   ` Brandon Williams
  2018-02-13  1:22 ` [PATCH 03/26] object-store: free alt_odb_list Stefan Beller
                   ` (28 subsequent siblings)
  30 siblings, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

In a process with multiple repositories open, alternates should be
associated to a single repository and not shared globally. Move
alt_odb_list and alt_odb_tail into the_repository and adjust callers
to reflect this.

Now that the alternative object data base is per repository, we're
leaking its memory upon freeing a repository. The next patch plugs
this hole.

No functional change intended.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/fsck.c |  4 +++-
 cache.h        |  4 ++--
 object-store.h |  7 ++++++-
 packfile.c     |  3 ++-
 sha1_file.c    | 25 ++++++++++++-------------
 sha1_name.c    |  3 ++-
 6 files changed, 27 insertions(+), 19 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index 04846d46f9..1048255da1 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "commit.h"
 #include "tree.h"
@@ -694,7 +695,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 		fsck_object_dir(get_object_directory());
 
 		prepare_alt_odb();
-		for (alt = alt_odb_list; alt; alt = alt->next)
+		for (alt = the_repository->objects.alt_odb_list;
+				alt; alt = alt->next)
 			fsck_object_dir(alt->path);
 
 		if (check_full) {
diff --git a/cache.h b/cache.h
index d8b975a571..918c2f15b4 100644
--- a/cache.h
+++ b/cache.h
@@ -1573,7 +1573,7 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);
 extern void schedule_dir_for_removal(const char *name, int len);
 extern void remove_scheduled_dirs(void);
 
-extern struct alternate_object_database {
+struct alternate_object_database {
 	struct alternate_object_database *next;
 
 	/* see alt_scratch_buf() */
@@ -1591,7 +1591,7 @@ extern struct alternate_object_database {
 	struct oid_array loose_objects_cache;
 
 	char path[FLEX_ARRAY];
-} *alt_odb_list;
+};
 extern void prepare_alt_odb(void);
 extern char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
diff --git a/object-store.h b/object-store.h
index cf35760ceb..44b8f84753 100644
--- a/object-store.h
+++ b/object-store.h
@@ -1,14 +1,19 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
+#include "cache.h"
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
 	 * Cannot be NULL after initialization.
 	 */
 	char *objectdir;
+
+	struct alternate_object_database *alt_odb_list;
+	struct alternate_object_database **alt_odb_tail;
 };
-#define RAW_OBJECT_STORE_INIT { NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index 4a5fe7ab18..d61076faaf 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "mru.h"
 #include "pack.h"
+#include "repository.h"
 #include "dir.h"
 #include "mergesort.h"
 #include "packfile.h"
@@ -880,7 +881,7 @@ void prepare_packed_git(void)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next)
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
diff --git a/sha1_file.c b/sha1_file.c
index 3da70ac650..2826d5d6ed 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -22,6 +22,7 @@
 #include "pack-revindex.h"
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "streaming.h"
 #include "dir.h"
 #include "mru.h"
@@ -346,9 +347,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 	return buf->buf;
 }
 
-struct alternate_object_database *alt_odb_list;
-static struct alternate_object_database **alt_odb_tail;
-
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
@@ -368,7 +366,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -428,8 +426,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*alt_odb_tail = ent;
-	alt_odb_tail = &(ent->next);
+	*the_repository->objects.alt_odb_tail = ent;
+	the_repository->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
@@ -563,7 +561,7 @@ void add_to_alternates_file(const char *reference)
 		fprintf_or_die(out, "%s\n", reference);
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
-		if (alt_odb_tail)
+		if (the_repository->objects.alt_odb_tail)
 			link_alt_odb_entries(reference, '\n', NULL, 0);
 	}
 	free(alts);
@@ -661,7 +659,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	int r = 0;
 
 	prepare_alt_odb();
-	for (ent = alt_odb_list; ent; ent = ent->next) {
+	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
 			break;
@@ -673,12 +671,13 @@ void prepare_alt_odb(void)
 {
 	const char *alt;
 
-	if (alt_odb_tail)
+	if (the_repository->objects.alt_odb_tail)
 		return;
 
 	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
 
-	alt_odb_tail = &alt_odb_list;
+	the_repository->objects.alt_odb_tail =
+			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
@@ -717,7 +716,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
 			return 1;
@@ -873,7 +872,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 
 	prepare_alt_odb();
 	errno = ENOENT;
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -899,7 +898,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	most_interesting_errno = errno;
 
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
diff --git a/sha1_name.c b/sha1_name.c
index 611c7d24dd..957ce25680 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -10,6 +10,7 @@
 #include "dir.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "repository.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
 		 */
 		fakeent = alloc_alt_odb(get_object_directory());
 	}
-	fakeent->next = alt_odb_list;
+	fakeent->next = the_repository->objects.alt_odb_list;
 
 	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
 		int pos;
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 03/26] object-store: free alt_odb_list
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
  2018-02-13  1:22 ` [PATCH 01/26] repository: introduce raw object store field Stefan Beller
  2018-02-13  1:22 ` [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 04/26] object-store: move packed_git and packed_git_mru to object store Stefan Beller
                   ` (27 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller

Free the memory and reset alt_odb_{list, tail} to NULL.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/object.c b/object.c
index 9b5b65e189..d778f09717 100644
--- a/object.c
+++ b/object.c
@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)
 	}
 }
 
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+	strbuf_release(&alt->scratch);
+	oid_array_clear(&alt->loose_objects_cache);
+}
+
+static void free_alt_odbs(struct raw_object_store *o)
+{
+	while (o->alt_odb_list) {
+		free_alt_odb(o->alt_odb_list);
+		o->alt_odb_list = o->alt_odb_list->next;
+	}
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	free(o->objectdir);
+
+	free_alt_odbs(o);
+	o->alt_odb_tail = NULL;
 }
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 04/26] object-store: move packed_git and packed_git_mru to object store
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (2 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 03/26] object-store: free alt_odb_list Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 05/26] object-store: close all packs upon clearing the " Stefan Beller
                   ` (26 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

In a process with multiple repositories open, packfile accessors
should be associated to a single repository and not shared globally.
Move packed_git and packed_git_mru into the_repository and adjust
callers to reflect this.

Patch generated by

 1. Moving the struct packed_git declaration to object-store.h
    and packed_git, packed_git_mru globals to struct object_store.

 2. Applying the semantic patch
    contrib/coccinelle/refactoring/packed_git.cocci to adjust callers.
    This semantic patch is placed in a sub directory of the coccinelle
    contrib dir, as this semantic patch is not expected to be of general
    usefulness; it is only useful during developing this series and
    merging it with other topics in flight. At a later date, just
    delete that semantic patch.

 3. Applying line wrapping fixes from "make style" to break the
    resulting long lines.

 4. Adding missing #includes of repository.h and object-store.h
    where needed.

 5. As the packfiles are now owned by an objectstore/repository, which
    is ephemeral unlike globals, we introduce memory leaks. So address
    them in raw_object_store_clear().

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/count-objects.c                         |  6 ++--
 builtin/fsck.c                                  |  7 +++--
 builtin/gc.c                                    |  4 ++-
 builtin/index-pack.c                            |  1 +
 builtin/pack-objects.c                          | 21 ++++++++-----
 builtin/pack-redundant.c                        |  6 ++--
 builtin/receive-pack.c                          |  1 +
 cache.h                                         | 28 ------------------
 contrib/coccinelle/refactoring/packed_git.cocci |  7 +++++
 fast-import.c                                   |  6 ++--
 http-backend.c                                  |  6 ++--
 http-push.c                                     |  1 +
 http-walker.c                                   |  1 +
 http.c                                          |  1 +
 mru.h                                           |  1 +
 object-store.h                                  | 32 +++++++++++++++++++-
 object.c                                        |  6 ++++
 pack-bitmap.c                                   |  4 ++-
 pack-check.c                                    |  1 +
 pack-revindex.c                                 |  1 +
 packfile.c                                      | 39 +++++++++++++------------
 reachable.c                                     |  1 +
 server-info.c                                   |  6 ++--
 sha1_name.c                                     |  5 ++--
 24 files changed, 120 insertions(+), 72 deletions(-)
 create mode 100644 contrib/coccinelle/refactoring/packed_git.cocci

diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 33343818c8..9334648dcc 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -7,6 +7,8 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
+#include "object-store.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -120,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!the_repository->objects.packed_git)
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 1048255da1..d4d249c2ed 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
@@ -707,7 +708,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = the_repository->objects.packed_git; p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -715,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
diff --git a/builtin/gc.c b/builtin/gc.c
index 3c5eae0edf..6970f325e5 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -18,6 +19,7 @@
 #include "run-command.h"
 #include "sigchain.h"
 #include "argv-array.h"
+#include "object-store.h"
 #include "commit.h"
 #include "packfile.h"
 
@@ -173,7 +175,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 4c51aec81f..1902d4e096 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -12,6 +12,7 @@
 #include "exec_cmd.h"
 #include "streaming.h"
 #include "thread-utils.h"
+#include "object-store.h"
 #include "packfile.h"
 
 static const char index_pack_usage[] =
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 6b9cfc289d..2ee8b69238 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1,6 +1,8 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "attr.h"
 #include "object.h"
 #include "blob.h"
@@ -1023,7 +1025,8 @@ static int want_object_in_pack(const struct object_id *oid,
 			return want;
 	}
 
-	for (entry = packed_git_mru.head; entry; entry = entry->next) {
+	for (entry = the_repository->objects.packed_git_mru.head; entry;
+	     entry = entry->next) {
 		struct packed_git *p = entry->item;
 		off_t offset;
 
@@ -1041,7 +1044,8 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				mru_mark(&packed_git_mru, entry);
+				mru_mark(&the_repository->objects.packed_git_mru,
+					 entry);
 			if (want != -1)
 				return want;
 		}
@@ -2648,7 +2652,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2711,7 +2715,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					the_repository->objects.packed_git;
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2720,7 +2725,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = the_repository->objects.packed_git;
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2756,7 +2761,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3119,7 +3124,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = the_repository->objects.packed_git; p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3132,7 +3137,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index aaa8136322..55462bc826 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -7,6 +7,8 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 #define BLKSIZE 512
@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b7ce7c7f52..b2eac79a6e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "sideband.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "remote.h"
diff --git a/cache.h b/cache.h
index 918c2f15b4..459fd01dbb 100644
--- a/cache.h
+++ b/cache.h
@@ -1633,34 +1633,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list, which can
- * be iterated instead of packed_git (and marked via mru_mark).
- */
-extern struct mru packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
diff --git a/contrib/coccinelle/refactoring/packed_git.cocci b/contrib/coccinelle/refactoring/packed_git.cocci
new file mode 100644
index 0000000000..da317a51a9
--- /dev/null
+++ b/contrib/coccinelle/refactoring/packed_git.cocci
@@ -0,0 +1,7 @@
+@@ @@
+- packed_git
++ the_repository->objects.packed_git
+
+@@ @@
+- packed_git_mru
++ the_repository->objects.packed_git_mru
diff --git a/fast-import.c b/fast-import.c
index b70ac025e0..1685fe59a2 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -154,8 +154,10 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "object.h"
 #include "blob.h"
 #include "tree.h"
@@ -1110,7 +1112,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1305,7 +1307,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/http-backend.c b/http-backend.c
index f3dc218b2a..4d93126c15 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -1,7 +1,9 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "exec_cmd.h"
@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http-push.c b/http-push.c
index 14435ab65d..34c5d030b7 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "blob.h"
diff --git a/http-walker.c b/http-walker.c
index 1ae8363de2..a1c6f2639b 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -4,6 +4,7 @@
 #include "http.h"
 #include "list.h"
 #include "transport.h"
+#include "object-store.h"
 #include "packfile.h"
 
 struct alt_base {
diff --git a/http.c b/http.c
index 5977712712..ab989b88dd 100644
--- a/http.c
+++ b/http.c
@@ -1,6 +1,7 @@
 #include "git-compat-util.h"
 #include "http.h"
 #include "config.h"
+#include "object-store.h"
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/mru.h b/mru.h
index 42e4aeaa10..d7de81efb9 100644
--- a/mru.h
+++ b/mru.h
@@ -37,6 +37,7 @@ struct mru_entry {
 struct mru {
 	struct mru_entry *head, *tail;
 };
+#define MRU_INIT { NULL, NULL }
 
 void mru_append(struct mru *mru, void *item);
 void mru_mark(struct mru *mru, struct mru_entry *entry);
diff --git a/object-store.h b/object-store.h
index 44b8f84753..8778ab9685 100644
--- a/object-store.h
+++ b/object-store.h
@@ -2,6 +2,7 @@
 #define OBJECT_STORE_H
 
 #include "cache.h"
+#include "mru.h"
 
 struct raw_object_store {
 	/*
@@ -10,11 +11,40 @@ struct raw_object_store {
 	 */
 	char *objectdir;
 
+	struct packed_git *packed_git;
+	/*
+	 * A most-recently-used ordered version of the packed_git list, which can
+	 * be iterated instead of packed_git (and marked via mru_mark).
+	 */
+	struct mru packed_git_mru;
+
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
+struct packed_git {
+	struct packed_git *next;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
 #endif /* OBJECT_STORE_H */
diff --git a/object.c b/object.c
index d778f09717..81d083bec1 100644
--- a/object.c
+++ b/object.c
@@ -466,4 +466,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	mru_clear(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 9270983e5f..d51172b1d5 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -5,11 +5,13 @@
 #include "revision.h"
 #include "progress.h"
 #include "list-objects.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-bitmap.h"
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
diff --git a/pack-check.c b/pack-check.c
index 073c1fbd46..c80c3ed8fd 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack.h"
+#include "object-store.h"
 #include "pack-revindex.h"
 #include "progress.h"
 #include "packfile.h"
diff --git a/pack-revindex.c b/pack-revindex.c
index 1b7ebd8d7e..82c3119652 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack-revindex.h"
+#include "object-store.h"
 
 /*
  * Pack index for existing packs give us easy access to the offsets into
diff --git a/packfile.c b/packfile.c
index d61076faaf..11b0e6613c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -7,6 +7,7 @@
 #include "packfile.h"
 #include "delta.h"
 #include "list.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "sha1-lookup.h"
 
@@ -40,8 +41,6 @@ static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
 static size_t peak_pack_mapped;
 static size_t pack_mapped;
-struct packed_git *packed_git;
-struct mru packed_git_mru;
 
 #define SZ_FMT PRIuMAX
 static inline uintmax_t sz_fmt(size_t s) { return s; }
@@ -241,7 +240,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -311,7 +310,7 @@ void close_all_packs(void)
 {
 	struct packed_git *p;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
@@ -379,7 +378,7 @@ static int close_one_pack(void)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -677,8 +676,8 @@ void install_packed_git(struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = packed_git;
-	packed_git = pack;
+	pack->next = the_repository->objects.packed_git;
+	the_repository->objects.packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -760,7 +759,8 @@ static void prepare_packed_git_one(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
 				    len == base_len &&
@@ -810,7 +810,7 @@ unsigned long approximate_object_count(void)
 
 		prepare_packed_git();
 		count = 0;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -859,17 +859,18 @@ static int sort_pack(const void *a_, const void *b_)
 
 static void rearrange_packed_git(void)
 {
-	packed_git = llist_mergesort(packed_git, get_next_packed_git,
-				     set_next_packed_git, sort_pack);
+	the_repository->objects.packed_git = llist_mergesort(
+		the_repository->objects.packed_git, get_next_packed_git,
+		set_next_packed_git, sort_pack);
 }
 
 static void prepare_packed_git_mru(void)
 {
 	struct packed_git *p;
 
-	mru_clear(&packed_git_mru);
-	for (p = packed_git; p; p = p->next)
-		mru_append(&packed_git_mru, p);
+	mru_clear(&the_repository->objects.packed_git_mru);
+	for (p = the_repository->objects.packed_git; p; p = p->next)
+		mru_append(&the_repository->objects.packed_git_mru, p);
 }
 
 static int prepare_packed_git_run_once = 0;
@@ -1003,7 +1004,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)
 	struct packed_git *p;
 	unsigned i;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		for (i = 0; i < p->num_bad_objects; i++)
 			if (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))
 				return p;
@@ -1835,12 +1836,12 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 	struct mru_entry *p;
 
 	prepare_packed_git();
-	if (!packed_git)
+	if (!the_repository->objects.packed_git)
 		return 0;
 
-	for (p = packed_git_mru.head; p; p = p->next) {
+	for (p = the_repository->objects.packed_git_mru.head; p; p = p->next) {
 		if (fill_pack_entry(sha1, e, p->item)) {
-			mru_mark(&packed_git_mru, p);
+			mru_mark(&the_repository->objects.packed_git_mru, p);
 			return 1;
 		}
 	}
@@ -1887,7 +1888,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int pack_errors = 0;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if (open_pack_index(p)) {
diff --git a/reachable.c b/reachable.c
index 88d7d679da..59ed16ba76 100644
--- a/reachable.c
+++ b/reachable.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tag.h"
 #include "commit.h"
 #include "blob.h"
diff --git a/server-info.c b/server-info.c
index 26a6c20b7d..96885c45f6 100644
--- a/server-info.c
+++ b/server-info.c
@@ -1,5 +1,7 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "tag.h"
@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
diff --git a/sha1_name.c b/sha1_name.c
index 957ce25680..016c883b5c 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = the_repository->objects.packed_git; p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 05/26] object-store: close all packs upon clearing the object store
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (3 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 04/26] object-store: move packed_git and packed_git_mru to object store Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:22 ` [PATCH 06/26] pack: move prepare_packed_git_run_once to " Stefan Beller
                   ` (25 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 6 ++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index acfe9d3c8c..75e16dfec6 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
diff --git a/builtin/clone.c b/builtin/clone.c
index 284651797e..7ec08d4696 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -1200,7 +1200,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7bbcd26faf..70015ccdc7 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1407,7 +1407,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
diff --git a/builtin/merge.c b/builtin/merge.c
index 30264cfd7c..907ae44ab5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b2eac79a6e..954fc72c7c 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
diff --git a/object.c b/object.c
index 81d083bec1..79c2c447bc 100644
--- a/object.c
+++ b/object.c
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -468,8 +469,5 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->alt_odb_tail = NULL;
 
 	mru_clear(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
 }
diff --git a/packfile.c b/packfile.c
index 11b0e6613c..2ce1b2cef7 100644
--- a/packfile.c
+++ b/packfile.c
@@ -306,11 +306,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
diff --git a/packfile.h b/packfile.h
index 0cdeb54dcd..fa547226b7 100644
--- a/packfile.h
+++ b/packfile.h
@@ -61,7 +61,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 06/26] pack: move prepare_packed_git_run_once to object store
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (4 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 05/26] object-store: close all packs upon clearing the " Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:22 ` [PATCH 07/26] pack: move approximate object count " Stefan Beller
                   ` (24 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

Each repository's object store can be initialized independently, so
they must not share a run_once variable.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 8 +++++++-
 packfile.c     | 7 +++----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/object-store.h b/object-store.h
index 8778ab9685..9fd109fa9c 100644
--- a/object-store.h
+++ b/object-store.h
@@ -20,8 +20,14 @@ struct raw_object_store {
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index 2ce1b2cef7..c107fce171 100644
--- a/packfile.c
+++ b/packfile.c
@@ -873,12 +873,11 @@ static void prepare_packed_git_mru(void)
 		mru_append(&the_repository->objects.packed_git_mru, p);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -886,13 +885,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 07/26] pack: move approximate object count to object store
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (5 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 06/26] pack: move prepare_packed_git_run_once to " Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:22 ` [PATCH 08/26] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
                   ` (23 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

The approximate_object_count() function maintains a rough count of
objects in a repository to estimate how long object name abbreviates
should be.  Object names are scoped to a repository and the
appropriate length may differ by repository, so the object count
should not be global.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 10 +++++++++-
 packfile.c     | 11 +++++------
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/object-store.h b/object-store.h
index 9fd109fa9c..86f1ec2cd0 100644
--- a/object-store.h
+++ b/object-store.h
@@ -21,13 +21,21 @@ struct raw_object_store {
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
 	 */
 	unsigned packed_git_initialized : 1;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0 }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0, 0, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index c107fce171..a53706cf16 100644
--- a/packfile.c
+++ b/packfile.c
@@ -793,8 +793,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -804,8 +802,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects.approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -815,8 +813,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects.approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects.approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -890,7 +889,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 08/26] sha1_file: add raw_object_store argument to alt_odb_usable
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (6 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 07/26] pack: move approximate object count " Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 09/26] sha1_file: add repository argument to link_alt_odb_entry Stefan Beller
                   ` (22 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller

Add a raw_object_store to alt_odb_usable to be more specific about which
repository to act on. The choice of the repository is delegated to its
only caller link_alt_odb_entry.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 2826d5d6ed..d25b68a8fb 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -350,7 +350,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir)
 {
 	struct alternate_object_database *alt;
 
@@ -366,7 +368,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -418,7 +420,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 09/26] sha1_file: add repository argument to link_alt_odb_entry
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (7 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 08/26] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 10/26] sha1_file: add repository argument to read_info_alternates Stefan Beller
                   ` (21 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Add a repository argument to allow the link_alt_odb_entry caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

Since the implementation does not yet work with other repositories,
use a wrapper macro to enforce that the caller passes in
the_repository as the first argument. It would be more appealing to
use BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work
because it requires a compile-time constant and common compilers like
gcc 4.8.4 do not consider "r == the_repository" a compile-time
constant.

This and the following three patches add repository arguments to
link_alt_odb_entry, read_info_alternates, link_alt_odb_entries
and prepare_alt_odb. Three out of the four functions are found
in a recursive call chain, calling each other, and one of them
accesses the repositories `objectdir` (which was migrated; it
was an obvious choice) and `ignore_env` (which we need to keep in
the repository struct for clarify); hence we will pass through the
repository unlike just the object store object + the ignore_env flag.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index d25b68a8fb..7dc5f690e2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -394,8 +394,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * terminating NUL.
  */
 static void read_info_alternates(const char * relative_base, int depth);
-static int link_alt_odb_entry(const char *entry, const char *relative_base,
-	int depth, const char *normalized_objdir)
+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
+static int link_alt_odb_entry_the_repository(const char *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
 	struct strbuf pathbuf = STRBUF_INIT;
@@ -492,7 +493,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);
+		link_alt_odb_entry(the_repository, entry.buf,
+				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 10/26] sha1_file: add repository argument to read_info_alternates
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (8 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 09/26] sha1_file: add repository argument to link_alt_odb_entry Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 11/26] sha1_file: add repository argument to link_alt_odb_entries Stefan Beller
                   ` (20 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

See previous patch for explanation.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 7dc5f690e2..4fdfdd945a 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -393,7 +393,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void read_info_alternates(const char * relative_base, int depth);
+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth);
 #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
 static int link_alt_odb_entry_the_repository(const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
@@ -434,7 +436,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(pathbuf.buf, depth + 1);
+	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -500,7 +502,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates(const char * relative_base, int depth)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -684,7 +687,7 @@ void prepare_alt_odb(void)
 			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
-	read_info_alternates(get_object_directory(), 0);
+	read_info_alternates(the_repository, get_object_directory(), 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 11/26] sha1_file: add repository argument to link_alt_odb_entries
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (9 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 10/26] sha1_file: add repository argument to read_info_alternates Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 12/26] sha1_file: add repository argument to prepare_alt_odb Stefan Beller
                   ` (19 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

See previous patch for explanation.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 4fdfdd945a..b090f403d8 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -471,8 +471,12 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-static void link_alt_odb_entries(const char *alt, int sep,
-				 const char *relative_base, int depth)
+#define link_alt_odb_entries(r, a, s, rb, d) \
+	link_alt_odb_entries_##r(a, s, rb, d)
+static void link_alt_odb_entries_the_repository(const char *alt,
+						int sep,
+						const char *relative_base,
+						int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -515,7 +519,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
@@ -569,7 +573,8 @@ void add_to_alternates_file(const char *reference)
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
 		if (the_repository->objects.alt_odb_tail)
-			link_alt_odb_entries(reference, '\n', NULL, 0);
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
 	}
 	free(alts);
 }
@@ -582,7 +587,8 @@ void add_to_alternates_memory(const char *reference)
 	 */
 	prepare_alt_odb();
 
-	link_alt_odb_entries(reference, '\n', NULL, 0);
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
 }
 
 /*
@@ -685,7 +691,8 @@ void prepare_alt_odb(void)
 
 	the_repository->objects.alt_odb_tail =
 			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository, alt,
+			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(the_repository, get_object_directory(), 0);
 }
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 12/26] sha1_file: add repository argument to prepare_alt_odb
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (10 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 11/26] sha1_file: add repository argument to link_alt_odb_entries Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 13/26] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories Stefan Beller
                   ` (18 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

See previous patch for explanation.

While at it, move the declaration to object-store.h,
where it should be easier to find.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/fsck.c |  2 +-
 cache.h        |  1 -
 object-store.h |  3 +++
 packfile.c     |  2 +-
 sha1_file.c    | 13 +++++++------
 sha1_name.c    |  3 ++-
 6 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index d4d249c2ed..00ec8eecf0 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -695,7 +695,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	} else {
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(the_repository);
 		for (alt = the_repository->objects.alt_odb_list;
 				alt; alt = alt->next)
 			fsck_object_dir(alt->path);
diff --git a/cache.h b/cache.h
index 459fd01dbb..70518e24ce 100644
--- a/cache.h
+++ b/cache.h
@@ -1592,7 +1592,6 @@ struct alternate_object_database {
 
 	char path[FLEX_ARRAY];
 };
-extern void prepare_alt_odb(void);
 extern char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 extern int foreach_alt_odb(alt_odb_fn, void*);
diff --git a/object-store.h b/object-store.h
index 86f1ec2cd0..d96a16edd1 100644
--- a/object-store.h
+++ b/object-store.h
@@ -61,4 +61,7 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+#define prepare_alt_odb(r) prepare_alt_odb_##r()
+extern void prepare_alt_odb_the_repository(void);
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index a53706cf16..9c49b34d2b 100644
--- a/packfile.c
+++ b/packfile.c
@@ -879,7 +879,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
diff --git a/sha1_file.c b/sha1_file.c
index b090f403d8..d7f271fe6e 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "dir.h"
 #include "mru.h"
@@ -585,7 +586,7 @@ void add_to_alternates_memory(const char *reference)
 	 * Make sure alternates are initialized, or else our entry may be
 	 * overwritten when they are.
 	 */
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 
 	link_alt_odb_entries(the_repository, reference,
 			     '\n', NULL, 0);
@@ -671,7 +672,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	struct alternate_object_database *ent;
 	int r = 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
@@ -680,7 +681,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(void)
+void prepare_alt_odb_the_repository(void)
 {
 	const char *alt;
 
@@ -729,7 +730,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
@@ -884,7 +885,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	errno = ENOENT;
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
@@ -911,7 +912,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
diff --git a/sha1_name.c b/sha1_name.c
index 016c883b5c..3e490ee8f6 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -11,6 +11,7 @@
 #include "sha1-array.h"
 #include "packfile.h"
 #include "repository.h"
+#include "object-store.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -353,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,
 
 	ds->len = len;
 	ds->hex_pfx[len] = '\0';
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	return 0;
 }
 
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 13/26] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (11 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 12/26] sha1_file: add repository argument to prepare_alt_odb Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 14/26] sha1_file: allow prepare_alt_odb " Stefan Beller
                   ` (17 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Actually this also allows read_info_alternates and link_alt_odb_entry to
handle arbitrary repositories, but link_alt_odb_entries is the most
interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function
graph, i.e. the recursive call chain might look like

  -> link_alt_odb_entries
    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h     |  4 ++++
 sha1_file.c | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/cache.h b/cache.h
index 70518e24ce..09d37cb651 100644
--- a/cache.h
+++ b/cache.h
@@ -1590,6 +1590,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 extern char *compute_alternate_path(const char *path, struct strbuf *err);
diff --git a/sha1_file.c b/sha1_file.c
index d7f271fe6e..d18ce3aeba 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -394,11 +394,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -424,7 +423,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -432,12 +431,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*r->objects.alt_odb_tail = ent;
+	r->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -472,12 +471,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -491,7 +486,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -500,15 +495,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -520,7 +516,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 14/26] sha1_file: allow prepare_alt_odb to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (12 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 13/26] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-14  0:37   ` Duy Nguyen
  2018-02-13  1:22 ` [PATCH 15/26] sha1_file: add repository argument to sha1_file_name Stefan Beller
                   ` (16 subsequent siblings)
  30 siblings, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object-store.h |  3 +--
 sha1_file.c    | 21 +++++++++++----------
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/object-store.h b/object-store.h
index d96a16edd1..add1d4e27c 100644
--- a/object-store.h
+++ b/object-store.h
@@ -61,7 +61,6 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
-#define prepare_alt_odb(r) prepare_alt_odb_##r()
-extern void prepare_alt_odb_the_repository(void);
+void prepare_alt_odb(struct repository *r);
 
 #endif /* OBJECT_STORE_H */
diff --git a/sha1_file.c b/sha1_file.c
index d18ce3aeba..f046d560f8 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -677,21 +677,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
-	const char *alt;
-
-	if (the_repository->objects.alt_odb_tail)
+	if (r->objects.alt_odb_tail)
 		return;
 
-	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
+
+	if (!r->ignore_env) {
+		const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+		if (!alt)
+			alt = "";
 
-	the_repository->objects.alt_odb_tail =
-			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(the_repository, alt,
-			     PATH_SEP, NULL, 0);
+		link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);
+	}
 
-	read_info_alternates(the_repository, get_object_directory(), 0);
+	read_info_alternates(r, r->objects.objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 15/26] sha1_file: add repository argument to sha1_file_name
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (13 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 14/26] sha1_file: allow prepare_alt_odb " Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 16/26] sha1_file: add repository argument to stat_sha1_file Stefan Beller
                   ` (15 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

Add a repository argument to allow sha1_file_name callers to be more
specific about which repository to handle. This is a small mechanical
change; it doesn't change the implementation to handle repositories
other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should
be easier to find.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 cache.h        |  8 --------
 http-walker.c  |  3 ++-
 http.c         |  5 +++--
 object-store.h |  9 +++++++++
 sha1_file.c    | 11 ++++++-----
 5 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/cache.h b/cache.h
index 09d37cb651..4b01c0de10 100644
--- a/cache.h
+++ b/cache.h
@@ -956,14 +956,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Return the name of the file in the local object database that would
- * be used to store a loose object with the specified sha1.  The
- * return value is a pointer to a statically allocated buffer that is
- * overwritten each time the function is called.
- */
-extern const char *sha1_file_name(const unsigned char *sha1);
-
 /*
  * Return an abbreviated sha1 unique within this repository's object database.
  * The result will be at least `len` characters long, and will be NUL
diff --git a/http-walker.c b/http-walker.c
index a1c6f2639b..96873bdfed 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "walker.h"
 #include "http.h"
@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)
 		ret = error("File %s has bad hash", hex);
 	} else if (req->rename < 0) {
 		ret = error("unable to write sha1 filename %s",
-			    sha1_file_name(req->sha1));
+			    sha1_file_name(the_repository, req->sha1));
 	}
 
 	release_http_object_request(req);
diff --git a/http.c b/http.c
index ab989b88dd..c7d2b74ce2 100644
--- a/http.c
+++ b/http.c
@@ -2181,7 +2181,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	hashcpy(freq->sha1, sha1);
 	freq->localfile = -1;
 
-	filename = sha1_file_name(sha1);
+	filename = sha1_file_name(the_repository, sha1);
 	snprintf(freq->tmpfile, sizeof(freq->tmpfile),
 		 "%s.temp", filename);
 
@@ -2329,7 +2329,8 @@ int finish_http_object_request(struct http_object_request *freq)
 		return -1;
 	}
 	freq->rename =
-		finalize_object_file(freq->tmpfile, sha1_file_name(freq->sha1));
+		finalize_object_file(freq->tmpfile,
+				     sha1_file_name(the_repository, freq->sha1));
 
 	return freq->rename;
 }
diff --git a/object-store.h b/object-store.h
index add1d4e27c..5422e80c08 100644
--- a/object-store.h
+++ b/object-store.h
@@ -61,6 +61,15 @@ struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * Return the name of the file in a repository's local object database
+ * that would be used to store a loose object with the specified sha1.
+ * The return value is a pointer to a statically allocated buffer that
+ * is overwritten each time the function is called.
+ */
+#define sha1_file_name(r, s) sha1_file_name_##r(s)
+const char *sha1_file_name_the_repository(const unsigned char *sha1);
+
 void prepare_alt_odb(struct repository *r);
 
 #endif /* OBJECT_STORE_H */
diff --git a/sha1_file.c b/sha1_file.c
index f046d560f8..65fefdf4ac 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -323,7 +323,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-const char *sha1_file_name(const unsigned char *sha1)
+const char *sha1_file_name_the_repository(const unsigned char *sha1)
 {
 	static struct strbuf buf = STRBUF_INIT;
 
@@ -721,7 +721,8 @@ int check_and_freshen_file(const char *fn, int freshen)
 
 static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 {
-	return check_and_freshen_file(sha1_file_name(sha1), freshen);
+	return check_and_freshen_file(sha1_file_name(the_repository, sha1),
+				      freshen);
 }
 
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
@@ -878,7 +879,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 {
 	struct alternate_object_database *alt;
 
-	*path = sha1_file_name(sha1);
+	*path = sha1_file_name(the_repository, sha1);
 	if (!lstat(*path, st))
 		return 0;
 
@@ -903,7 +904,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	struct alternate_object_database *alt;
 	int most_interesting_errno;
 
-	*path = sha1_file_name(sha1);
+	*path = sha1_file_name(the_repository, sha1);
 	fd = git_open(*path);
 	if (fd >= 0)
 		return fd;
@@ -1568,7 +1569,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	git_SHA_CTX c;
 	unsigned char parano_sha1[20];
 	static struct strbuf tmp_file = STRBUF_INIT;
-	const char *filename = sha1_file_name(sha1);
+	const char *filename = sha1_file_name(the_repository, sha1);
 
 	fd = create_tmpfile(&tmp_file, filename);
 	if (fd < 0) {
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 16/26] sha1_file: add repository argument to stat_sha1_file
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (14 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 15/26] sha1_file: add repository argument to sha1_file_name Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 17/26] sha1_file: add repository argument to open_sha1_file Stefan Beller
                   ` (14 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Add a repository argument to allow the stat_sha1_file caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 65fefdf4ac..1d81bafe56 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -874,8 +874,9 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
-			  const char **path)
+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
+static int stat_sha1_file_the_repository(const unsigned char *sha1,
+					 struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 
@@ -1173,7 +1174,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(sha1, &st, &path) < 0)
+		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
@@ -1372,7 +1373,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,
 		die("replacement %s not found for %s",
 		    sha1_to_hex(repl), sha1_to_hex(sha1));
 
-	if (!stat_sha1_file(repl, &st, &path))
+	if (!stat_sha1_file(the_repository, repl, &st, &path))
 		die("loose object %s (stored in %s) is corrupt",
 		    sha1_to_hex(repl), path);
 
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 17/26] sha1_file: add repository argument to open_sha1_file
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (15 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 16/26] sha1_file: add repository argument to stat_sha1_file Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 18/26] sha1_file: add repository argument to map_sha1_file_1 Stefan Beller
                   ` (13 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Add a repository argument to allow the open_sha1_file caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 1d81bafe56..e9e01fb471 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -899,7 +899,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-static int open_sha1_file(const unsigned char *sha1, const char **path)
+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
+static int open_sha1_file_the_repository(const unsigned char *sha1,
+					 const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -938,7 +940,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(sha1, &path);
+		fd = open_sha1_file(the_repository, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 18/26] sha1_file: add repository argument to map_sha1_file_1
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (16 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 17/26] sha1_file: add repository argument to open_sha1_file Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 19/26] sha1_file: add repository argument to map_sha1_file Stefan Beller
                   ` (12 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Add a repository argument to allow the map_sha1_file_1 caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index e9e01fb471..f706526bc2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -930,9 +930,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-static void *map_sha1_file_1(const char *path,
-			     const unsigned char *sha1,
-			     unsigned long *size)
+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
+static void *map_sha1_file_1_the_repository(const char *path,
+					    const unsigned char *sha1,
+					    unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -961,7 +962,7 @@ static void *map_sha1_file_1(const char *path,
 
 void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(NULL, sha1, size);
+	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -2173,7 +2174,7 @@ int read_loose_object(const char *path,
 
 	*contents = NULL;
 
-	map = map_sha1_file_1(path, NULL, &mapsize);
+	map = map_sha1_file_1(the_repository, path, NULL, &mapsize);
 	if (!map) {
 		error_errno("unable to mmap %s", path);
 		goto out;
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 19/26] sha1_file: add repository argument to map_sha1_file
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (17 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 18/26] sha1_file: add repository argument to map_sha1_file_1 Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 20/26] sha1_file: add repository argument to sha1_loose_object_info Stefan Beller
                   ` (11 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

Add a repository argument to allow map_sha1_file callers to be more
specific about which repository to handle. This is a small mechanical
change; it doesn't change the implementation to handle repositories
other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should
be easier to find.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 cache.h        | 1 -
 object-store.h | 2 ++
 sha1_file.c    | 4 ++--
 streaming.c    | 5 ++++-
 4 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/cache.h b/cache.h
index 4b01c0de10..d0a1236dac 100644
--- a/cache.h
+++ b/cache.h
@@ -1237,7 +1237,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c
 extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
 
diff --git a/object-store.h b/object-store.h
index 5422e80c08..0a4561f476 100644
--- a/object-store.h
+++ b/object-store.h
@@ -69,6 +69,8 @@ struct packed_git {
  */
 #define sha1_file_name(r, s) sha1_file_name_##r(s)
 const char *sha1_file_name_the_repository(const unsigned char *sha1);
+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
 
 void prepare_alt_odb(struct repository *r);
 
diff --git a/sha1_file.c b/sha1_file.c
index f706526bc2..1ce4058644 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -960,7 +960,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	return map;
 }
 
-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
 {
 	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
@@ -1184,7 +1184,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 		return 0;
 	}
 
-	map = map_sha1_file(sha1, &mapsize);
+	map = map_sha1_file(the_repository, sha1, &mapsize);
 	if (!map)
 		return -1;
 
diff --git a/streaming.c b/streaming.c
index 5892b50bd8..22d27df55e 100644
--- a/streaming.c
+++ b/streaming.c
@@ -3,6 +3,8 @@
  */
 #include "cache.h"
 #include "streaming.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 enum input_source {
@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {
 
 static open_method_decl(loose)
 {
-	st->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);
+	st->u.loose.mapped = map_sha1_file(the_repository,
+					   sha1, &st->u.loose.mapsize);
 	if (!st->u.loose.mapped)
 		return -1;
 	if ((unpack_sha1_header(&st->z,
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 20/26] sha1_file: add repository argument to sha1_loose_object_info
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (18 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 19/26] sha1_file: add repository argument to map_sha1_file Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 21/26] sha1_file: allow sha1_file_name to handle arbitrary repositories Stefan Beller
                   ` (10 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Add a repository argument to allow the sha1_loose_object_info caller
to be more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 1ce4058644..d0bc09a089 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1151,9 +1151,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-static int sha1_loose_object_info(const unsigned char *sha1,
-				  struct object_info *oi,
-				  int flags)
+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
+						 struct object_info *oi,
+						 int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1266,7 +1267,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 
 	if (!find_pack_entry(real, &e)) {
 		/* Most likely it's a loose object. */
-		if (!sha1_loose_object_info(real, oi, flags))
+		if (!sha1_loose_object_info(the_repository, real, oi, flags))
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 21/26] sha1_file: allow sha1_file_name to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (19 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 20/26] sha1_file: add repository argument to sha1_loose_object_info Stefan Beller
@ 2018-02-13  1:22 ` Stefan Beller
  2018-02-13  1:22 ` [PATCH 22/26] sha1_file: allow stat_sha1_file " Stefan Beller
                   ` (9 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 3 +--
 sha1_file.c    | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/object-store.h b/object-store.h
index 0a4561f476..b4756444bb 100644
--- a/object-store.h
+++ b/object-store.h
@@ -67,8 +67,7 @@ struct packed_git {
  * The return value is a pointer to a statically allocated buffer that
  * is overwritten each time the function is called.
  */
-#define sha1_file_name(r, s) sha1_file_name_##r(s)
-const char *sha1_file_name_the_repository(const unsigned char *sha1);
+const char *sha1_file_name(struct repository *r, const unsigned char *sha1);
 #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
 void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
 
diff --git a/sha1_file.c b/sha1_file.c
index d0bc09a089..fddada5756 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -323,12 +323,12 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-const char *sha1_file_name_the_repository(const unsigned char *sha1)
+const char *sha1_file_name(struct repository *r, const unsigned char *sha1)
 {
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	strbuf_addf(&buf, "%s/", get_object_directory());
+	strbuf_addf(&buf, "%s/", r->objects.objectdir);
 
 	fill_sha1_path(&buf, sha1);
 	return buf.buf;
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 22/26] sha1_file: allow stat_sha1_file to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (20 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 21/26] sha1_file: allow sha1_file_name to handle arbitrary repositories Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:22 ` [PATCH 23/26] sha1_file: allow open_sha1_file " Stefan Beller
                   ` (8 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index fddada5756..21ddbff846 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -874,19 +874,18 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
-static int stat_sha1_file_the_repository(const unsigned char *sha1,
-					 struct stat *st, const char **path)
+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
+			  struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 
-	*path = sha1_file_name(the_repository, sha1);
+	*path = sha1_file_name(r, sha1);
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb(the_repository);
+	prepare_alt_odb(r);
 	errno = ENOENT;
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 23/26] sha1_file: allow open_sha1_file to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (21 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 22/26] sha1_file: allow stat_sha1_file " Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:22 ` [PATCH 24/26] sha1_file: allow map_sha1_file_1 " Stefan Beller
                   ` (7 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 21ddbff846..50202f0959 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -898,22 +898,21 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
-static int open_sha1_file_the_repository(const unsigned char *sha1,
-					 const char **path)
+static int open_sha1_file(struct repository *r,
+			  const unsigned char *sha1, const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
 	int most_interesting_errno;
 
-	*path = sha1_file_name(the_repository, sha1);
+	*path = sha1_file_name(r, sha1);
 	fd = git_open(*path);
 	if (fd >= 0)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	prepare_alt_odb(r);
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 24/26] sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (22 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 23/26] sha1_file: allow open_sha1_file " Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:22 ` [PATCH 25/26] sha1_file: allow map_sha1_file " Stefan Beller
                   ` (6 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 50202f0959..86b0ca7089 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -928,10 +928,8 @@ static int open_sha1_file(struct repository *r,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
-static void *map_sha1_file_1_the_repository(const char *path,
-					    const unsigned char *sha1,
-					    unsigned long *size)
+static void *map_sha1_file_1(struct repository *r, const char *path,
+			     const unsigned char *sha1, unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -939,7 +937,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(the_repository, sha1, &path);
+		fd = open_sha1_file(r, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 25/26] sha1_file: allow map_sha1_file to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (23 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 24/26] sha1_file: allow map_sha1_file_1 " Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:22 ` [PATCH 26/26] sha1_file: allow sha1_loose_object_info " Stefan Beller
                   ` (5 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Stefan Beller, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 3 +--
 sha1_file.c    | 5 +++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/object-store.h b/object-store.h
index b4756444bb..f164c4e5c9 100644
--- a/object-store.h
+++ b/object-store.h
@@ -68,8 +68,7 @@ struct packed_git {
  * is overwritten each time the function is called.
  */
 const char *sha1_file_name(struct repository *r, const unsigned char *sha1);
-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+extern void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
 void prepare_alt_odb(struct repository *r);
 
diff --git a/sha1_file.c b/sha1_file.c
index 86b0ca7089..6237d59a59 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -956,9 +956,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,
 	return map;
 }
 
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file(struct repository *r,
+		    const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(the_repository, NULL, sha1, size);
+	return map_sha1_file_1(r, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 26/26] sha1_file: allow sha1_loose_object_info to handle arbitrary repositories
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (24 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 25/26] sha1_file: allow map_sha1_file " Stefan Beller
@ 2018-02-13  1:22 ` " Stefan Beller
  2018-02-13  1:38 ` [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (4 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:22 UTC (permalink / raw)
  To: gitster; +Cc: git, pclouds, sunshine, jonathantanmy, Jonathan Nieder, Stefan Beller

From: Jonathan Nieder <jrnieder@gmail.com>

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 6237d59a59..c3f35914ce 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1148,10 +1148,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
-						 struct object_info *oi,
-						 int flags)
+static int sha1_loose_object_info(struct repository *r,
+				  const unsigned char *sha1,
+				  struct object_info *oi, int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1175,14 +1174,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
+		if (stat_sha1_file(r, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(the_repository, sha1, &mapsize);
+	map = map_sha1_file(r, sha1, &mapsize);
 	if (!map)
 		return -1;
 
-- 
2.16.1.73.ga2c3e9663f.dirty


^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (25 preceding siblings ...)
  2018-02-13  1:22 ` [PATCH 26/26] sha1_file: allow sha1_loose_object_info " Stefan Beller
@ 2018-02-13  1:38 ` Stefan Beller
  2018-02-13 11:49 ` Duy Nguyen
                   ` (3 subsequent siblings)
  30 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13  1:38 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Duy Nguyen, Eric Sunshine, Jonathan Tan, Stefan Beller

On Mon, Feb 12, 2018 at 5:22 PM, Stefan Beller <sbeller@google.com> wrote:

> I developed this series [...] on top of current master.

also available at https://github.com/stefanbeller/git/tree/object-store-part1

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (26 preceding siblings ...)
  2018-02-13  1:38 ` [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
@ 2018-02-13 11:49 ` Duy Nguyen
  2018-02-13 12:13   ` Duy Nguyen
  2018-02-13 19:26 ` Jonathan Nieder
                   ` (2 subsequent siblings)
  30 siblings, 1 reply; 100+ messages in thread
From: Duy Nguyen @ 2018-02-13 11:49 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, sunshine, jonathantanmy

On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:
> This is a real take on the first part of the recent RFC[1].
>
> ...
> 
> Duy suggested that we shall not use the repository blindly, but
> should carefully examine whether to pass on an object store or the
> refstore or such[4], which I agree with if it makes sense. This
> series unfortunately has an issue with that as I would not want to
> pass down the `ignore_env` flag separately from the object store, so
> I made all functions that only take the object store to have the raw
> object store as the first parameter, and others using the full
> repository.

Second proposal :) How about you store ignore_env in raw_object_store?
This would not be the first time an object has some configuration
passed in at construction time. And it has a "constructor" now,
raw_object_store_init() (I probably should merge _setup in it too)

The core changes look like this. I have a full commit on top of your
series [1] that keeps sha1_file.c functions take 'struct
raw_object_store' instead.

[1] https://github.com/pclouds/git/tree/object-store-part1

-- 8< --
diff --git a/object-store.h b/object-store.h
index f164c4e5c9..a8bf1738f2 100644
--- a/object-store.h
+++ b/object-store.h
@@ -34,9 +34,13 @@ struct raw_object_store {
 	 * packs.
 	 */
 	unsigned packed_git_initialized : 1;
+
+	unsigned ignore_env : 1;
 };
 #define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0, 0, 0 }
 
+void raw_object_store_init(struct raw_object_store *ros, struct repository *r);
+void raw_object_store_setup(struct raw_object_store *ros, char *);
 void raw_object_store_clear(struct raw_object_store *o);
 
 struct packed_git {
diff --git a/object.c b/object.c
index 79c2c447bc..a4534bf4c4 100644
--- a/object.c
+++ b/object.c
@@ -461,6 +461,20 @@ static void free_alt_odbs(struct raw_object_store *o)
 	}
 }
 
+void raw_object_store_init(struct raw_object_store *o,
+			   struct repository *r)
+{
+	/* FIXME: memset? */
+	o->ignore_env = r->ignore_env;
+}
+
+void raw_object_store_setup(struct raw_object_store *o,
+			    char *objectdir)
+{
+	free(o->objectdir);
+	o->objectdir = objectdir;
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	free(o->objectdir);
diff --git a/repository.c b/repository.c
index bd2ad578de..ac5863d7e1 100644
--- a/repository.c
+++ b/repository.c
@@ -49,9 +49,9 @@ static void repo_setup_env(struct repository *repo)
 						    !repo->ignore_env);
 	free(repo->commondir);
 	repo->commondir = strbuf_detach(&sb, NULL);
-	free(repo->objects.objectdir);
-	repo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
-					    "objects", !repo->ignore_env);
+	raw_object_store_setup(&repo->objects,
+			       git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+						 "objects", !repo->ignore_env));
 	free(repo->graft_file);
 	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
 					     "info/grafts", !repo->ignore_env);
@@ -142,6 +142,8 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 
 	repo->ignore_env = 1;
 
+	raw_object_store_init(&repo->objects, repo);
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
diff --git a/sha1_file.c b/sha1_file.c
index c3f35914ce..3be58651a1 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -677,14 +677,14 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(struct repository *r)
+void prepare_alt_odb(struct raw_object_store *ros)
 {
 	if (r->objects.alt_odb_tail)
 		return;
 
 	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
 
-	if (!r->ignore_env) {
+	if (!ros->ignore_env) {
 		const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);
 		if (!alt)
 			alt = "";
-- 8< --


--
Duy

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 11:49 ` Duy Nguyen
@ 2018-02-13 12:13   ` Duy Nguyen
  2018-02-13 16:52     ` Brandon Williams
  2018-02-13 17:47     ` Stefan Beller
  0 siblings, 2 replies; 100+ messages in thread
From: Duy Nguyen @ 2018-02-13 12:13 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Junio C Hamano, Git Mailing List, Eric Sunshine, Jonathan Tan

On Tue, Feb 13, 2018 at 6:49 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:
>> This is a real take on the first part of the recent RFC[1].
>>
>> ...
>>
>> Duy suggested that we shall not use the repository blindly, but
>> should carefully examine whether to pass on an object store or the
>> refstore or such[4], which I agree with if it makes sense. This
>> series unfortunately has an issue with that as I would not want to
>> pass down the `ignore_env` flag separately from the object store, so
>> I made all functions that only take the object store to have the raw
>> object store as the first parameter, and others using the full
>> repository.
>
> Second proposal :) How about you store ignore_env in raw_object_store?
> This would not be the first time an object has some configuration
> passed in at construction time. And it has a "constructor" now,
> raw_object_store_init() (I probably should merge _setup in it too)

A bit more on this configuration parameters. Down the road I think we
need something like this anyway to delete global config vars like
packed_git_window_size, delta_base_cache_limit...  Either all these
end up in raw_object_store, or raw_object_store holds a link to
"struct config_set".

The ignore_env specifically though looks to me like a stop gap
solution until everything goes through repo_init() first. At that
point we don't have to delay getenv() anymore. We can getenv() all at
repo_init() then pass them in raw_object_store and ignore_env should
be gone. So sticking it inside raw_object_store _temporarily_ does not
sound so bad.
-- 
Duy

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 12:13   ` Duy Nguyen
@ 2018-02-13 16:52     ` Brandon Williams
  2018-02-13 17:47     ` Stefan Beller
  1 sibling, 0 replies; 100+ messages in thread
From: Brandon Williams @ 2018-02-13 16:52 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Stefan Beller, Junio C Hamano, Git Mailing List, Eric Sunshine, Jonathan Tan

On 02/13, Duy Nguyen wrote:
> On Tue, Feb 13, 2018 at 6:49 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> > On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:
> >> This is a real take on the first part of the recent RFC[1].
> >>
> >> ...
> >>
> >> Duy suggested that we shall not use the repository blindly, but
> >> should carefully examine whether to pass on an object store or the
> >> refstore or such[4], which I agree with if it makes sense. This
> >> series unfortunately has an issue with that as I would not want to
> >> pass down the `ignore_env` flag separately from the object store, so
> >> I made all functions that only take the object store to have the raw
> >> object store as the first parameter, and others using the full
> >> repository.
> >
> > Second proposal :) How about you store ignore_env in raw_object_store?
> > This would not be the first time an object has some configuration
> > passed in at construction time. And it has a "constructor" now,
> > raw_object_store_init() (I probably should merge _setup in it too)
> 
> A bit more on this configuration parameters. Down the road I think we
> need something like this anyway to delete global config vars like
> packed_git_window_size, delta_base_cache_limit...  Either all these
> end up in raw_object_store, or raw_object_store holds a link to
> "struct config_set".
> 
> The ignore_env specifically though looks to me like a stop gap
> solution until everything goes through repo_init() first. At that
> point we don't have to delay getenv() anymore. We can getenv() all at
> repo_init() then pass them in raw_object_store and ignore_env should
> be gone. So sticking it inside raw_object_store _temporarily_ does not
> sound so bad.

I like this approach, I mean at the moment we are replicating a single
bit of data but that allows us to be able to limit the scope of where a
repository struct is passed, giving us a better abstraction layer.

> -- 
> Duy

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 12:13   ` Duy Nguyen
  2018-02-13 16:52     ` Brandon Williams
@ 2018-02-13 17:47     ` Stefan Beller
  2018-02-13 18:57       ` Junio C Hamano
  1 sibling, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-13 17:47 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Junio C Hamano, Git Mailing List, Eric Sunshine, Jonathan Tan

On Tue, Feb 13, 2018 at 4:13 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Tue, Feb 13, 2018 at 6:49 PM, Duy Nguyen <pclouds@gmail.com> wrote:
>> On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:
>>> This is a real take on the first part of the recent RFC[1].
>>>
>>> ...
>>>
>>> Duy suggested that we shall not use the repository blindly, but
>>> should carefully examine whether to pass on an object store or the
>>> refstore or such[4], which I agree with if it makes sense. This
>>> series unfortunately has an issue with that as I would not want to
>>> pass down the `ignore_env` flag separately from the object store, so
>>> I made all functions that only take the object store to have the raw
>>> object store as the first parameter, and others using the full
>>> repository.
>>
>> Second proposal :) How about you store ignore_env in raw_object_store?
>> This would not be the first time an object has some configuration
>> passed in at construction time. And it has a "constructor" now,
>> raw_object_store_init() (I probably should merge _setup in it too)
>
> A bit more on this configuration parameters. Down the road I think we
> need something like this anyway to delete global config vars like
> packed_git_window_size, delta_base_cache_limit...  Either all these
> end up in raw_object_store, or raw_object_store holds a link to
> "struct config_set".

That makes sense long term.

> The ignore_env specifically though looks to me like a stop gap
> solution until everything goes through repo_init() first. At that
> point we don't have to delay getenv() anymore. We can getenv() all at
> repo_init() then pass them in raw_object_store and ignore_env should
> be gone. So sticking it inside raw_object_store _temporarily_ does not
> sound so bad.

Oh, that is an interesting perspective. Here is how I arrived at the opposite
conclusion initially: Searching for 'ignore_env' shows that we care about it
as well for the index and graft paths, which are not the object store, hence
it would be better kept in the repository. (The alternative would be to
duplicate it into the raw object store, but I do not like data duplication)

But maybe it is better to duplicate this one bit instead of passing through
a larger scoped object.

I'll rework the patches.

Thanks!
Stefan

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store
  2018-02-13  1:22 ` [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
@ 2018-02-13 18:51   ` Brandon Williams
  2018-02-13 19:46     ` Stefan Beller
  0 siblings, 1 reply; 100+ messages in thread
From: Brandon Williams @ 2018-02-13 18:51 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, pclouds, sunshine, jonathantanmy, Jonathan Nieder

On 02/12, Stefan Beller wrote:
> In a process with multiple repositories open, alternates should be
> associated to a single repository and not shared globally. Move
> alt_odb_list and alt_odb_tail into the_repository and adjust callers
> to reflect this.
> 
> Now that the alternative object data base is per repository, we're
> leaking its memory upon freeing a repository. The next patch plugs
> this hole.
> 
> No functional change intended.
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
> ---
>  builtin/fsck.c |  4 +++-
>  cache.h        |  4 ++--
>  object-store.h |  7 ++++++-
>  packfile.c     |  3 ++-
>  sha1_file.c    | 25 ++++++++++++-------------
>  sha1_name.c    |  3 ++-
>  6 files changed, 27 insertions(+), 19 deletions(-)
> 
> diff --git a/builtin/fsck.c b/builtin/fsck.c
> index 04846d46f9..1048255da1 100644
> --- a/builtin/fsck.c
> +++ b/builtin/fsck.c
> @@ -1,5 +1,6 @@
>  #include "builtin.h"
>  #include "cache.h"
> +#include "repository.h"
>  #include "config.h"
>  #include "commit.h"
>  #include "tree.h"
> @@ -694,7 +695,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
>  		fsck_object_dir(get_object_directory());
>  
>  		prepare_alt_odb();
> -		for (alt = alt_odb_list; alt; alt = alt->next)
> +		for (alt = the_repository->objects.alt_odb_list;
> +				alt; alt = alt->next)
>  			fsck_object_dir(alt->path);
>  
>  		if (check_full) {
> diff --git a/cache.h b/cache.h
> index d8b975a571..918c2f15b4 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1573,7 +1573,7 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);
>  extern void schedule_dir_for_removal(const char *name, int len);
>  extern void remove_scheduled_dirs(void);
>  
> -extern struct alternate_object_database {
> +struct alternate_object_database {
>  	struct alternate_object_database *next;
>  
>  	/* see alt_scratch_buf() */
> @@ -1591,7 +1591,7 @@ extern struct alternate_object_database {
>  	struct oid_array loose_objects_cache;
>  
>  	char path[FLEX_ARRAY];
> -} *alt_odb_list;
> +};
>  extern void prepare_alt_odb(void);
>  extern char *compute_alternate_path(const char *path, struct strbuf *err);
>  typedef int alt_odb_fn(struct alternate_object_database *, void *);
> diff --git a/object-store.h b/object-store.h
> index cf35760ceb..44b8f84753 100644
> --- a/object-store.h
> +++ b/object-store.h
> @@ -1,14 +1,19 @@
>  #ifndef OBJECT_STORE_H
>  #define OBJECT_STORE_H
>  
> +#include "cache.h"
> +

Maybe we would want to move the definition of the alternate store to
this header file?  That way we don't need to include cache.h here.

>  struct raw_object_store {
>  	/*
>  	 * Path to the repository's object store.
>  	 * Cannot be NULL after initialization.
>  	 */
>  	char *objectdir;
> +
> +	struct alternate_object_database *alt_odb_list;
> +	struct alternate_object_database **alt_odb_tail;
>  };
> -#define RAW_OBJECT_STORE_INIT { NULL }
> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
>  
>  void raw_object_store_clear(struct raw_object_store *o);
>  
> diff --git a/packfile.c b/packfile.c
> index 4a5fe7ab18..d61076faaf 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -1,6 +1,7 @@
>  #include "cache.h"
>  #include "mru.h"
>  #include "pack.h"
> +#include "repository.h"
>  #include "dir.h"
>  #include "mergesort.h"
>  #include "packfile.h"
> @@ -880,7 +881,7 @@ void prepare_packed_git(void)
>  		return;
>  	prepare_packed_git_one(get_object_directory(), 1);
>  	prepare_alt_odb();
> -	for (alt = alt_odb_list; alt; alt = alt->next)
> +	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
>  		prepare_packed_git_one(alt->path, 0);
>  	rearrange_packed_git();
>  	prepare_packed_git_mru();
> diff --git a/sha1_file.c b/sha1_file.c
> index 3da70ac650..2826d5d6ed 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -22,6 +22,7 @@
>  #include "pack-revindex.h"
>  #include "sha1-lookup.h"
>  #include "bulk-checkin.h"
> +#include "repository.h"
>  #include "streaming.h"
>  #include "dir.h"
>  #include "mru.h"
> @@ -346,9 +347,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
>  	return buf->buf;
>  }
>  
> -struct alternate_object_database *alt_odb_list;
> -static struct alternate_object_database **alt_odb_tail;
> -
>  /*
>   * Return non-zero iff the path is usable as an alternate object database.
>   */
> @@ -368,7 +366,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
>  	 * Prevent the common mistake of listing the same
>  	 * thing twice, or object directory itself.
>  	 */
> -	for (alt = alt_odb_list; alt; alt = alt->next) {
> +	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
>  		if (!fspathcmp(path->buf, alt->path))
>  			return 0;
>  	}
> @@ -428,8 +426,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
>  	ent = alloc_alt_odb(pathbuf.buf);
>  
>  	/* add the alternate entry */
> -	*alt_odb_tail = ent;
> -	alt_odb_tail = &(ent->next);
> +	*the_repository->objects.alt_odb_tail = ent;
> +	the_repository->objects.alt_odb_tail = &(ent->next);
>  	ent->next = NULL;
>  
>  	/* recursively add alternates */
> @@ -563,7 +561,7 @@ void add_to_alternates_file(const char *reference)
>  		fprintf_or_die(out, "%s\n", reference);
>  		if (commit_lock_file(&lock))
>  			die_errno("unable to move new alternates file into place");
> -		if (alt_odb_tail)
> +		if (the_repository->objects.alt_odb_tail)
>  			link_alt_odb_entries(reference, '\n', NULL, 0);
>  	}
>  	free(alts);
> @@ -661,7 +659,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
>  	int r = 0;
>  
>  	prepare_alt_odb();
> -	for (ent = alt_odb_list; ent; ent = ent->next) {
> +	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
>  		r = fn(ent, cb);
>  		if (r)
>  			break;
> @@ -673,12 +671,13 @@ void prepare_alt_odb(void)
>  {
>  	const char *alt;
>  
> -	if (alt_odb_tail)
> +	if (the_repository->objects.alt_odb_tail)
>  		return;
>  
>  	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
>  
> -	alt_odb_tail = &alt_odb_list;
> +	the_repository->objects.alt_odb_tail =
> +			&the_repository->objects.alt_odb_list;
>  	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
>  
>  	read_info_alternates(get_object_directory(), 0);
> @@ -717,7 +716,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
>  {
>  	struct alternate_object_database *alt;
>  	prepare_alt_odb();
> -	for (alt = alt_odb_list; alt; alt = alt->next) {
> +	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
>  		const char *path = alt_sha1_path(alt, sha1);
>  		if (check_and_freshen_file(path, freshen))
>  			return 1;
> @@ -873,7 +872,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
>  
>  	prepare_alt_odb();
>  	errno = ENOENT;
> -	for (alt = alt_odb_list; alt; alt = alt->next) {
> +	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
>  		*path = alt_sha1_path(alt, sha1);
>  		if (!lstat(*path, st))
>  			return 0;
> @@ -899,7 +898,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
>  	most_interesting_errno = errno;
>  
>  	prepare_alt_odb();
> -	for (alt = alt_odb_list; alt; alt = alt->next) {
> +	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
>  		*path = alt_sha1_path(alt, sha1);
>  		fd = git_open(*path);
>  		if (fd >= 0)
> diff --git a/sha1_name.c b/sha1_name.c
> index 611c7d24dd..957ce25680 100644
> --- a/sha1_name.c
> +++ b/sha1_name.c
> @@ -10,6 +10,7 @@
>  #include "dir.h"
>  #include "sha1-array.h"
>  #include "packfile.h"
> +#include "repository.h"
>  
>  static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
>  
> @@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
>  		 */
>  		fakeent = alloc_alt_odb(get_object_directory());
>  	}
> -	fakeent->next = alt_odb_list;
> +	fakeent->next = the_repository->objects.alt_odb_list;
>  
>  	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
>  		int pos;
> -- 
> 2.16.1.73.ga2c3e9663f.dirty
> 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 17:47     ` Stefan Beller
@ 2018-02-13 18:57       ` Junio C Hamano
  2018-02-13 19:23         ` Stefan Beller
  0 siblings, 1 reply; 100+ messages in thread
From: Junio C Hamano @ 2018-02-13 18:57 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Duy Nguyen, Git Mailing List, Eric Sunshine, Jonathan Tan

Stefan Beller <sbeller@google.com> writes:

> Oh, that is an interesting perspective. Here is how I arrived at the opposite
> conclusion initially: Searching for 'ignore_env' shows that we care about it
> as well for the index and graft paths, which are not the object store, hence
> it would be better kept in the repository. (The alternative would be to
> duplicate it into the raw object store, but I do not like data duplication)
>
> But maybe it is better to duplicate this one bit instead of passing through
> a larger scoped object.

If a larger scoped repository refers to a smaller scoped
object-store, is there still a need to duplicate that bit, instead
of referring to the bit the smaller scoped one has when asking about
the bit in the larger scoped one?

I am not sure if these "do not look at environment variables" is an
attribute of these entities---it sounds more like an attribute for
each invocation of an operation, i.e. "I want to learn where the
index is but would ignore GIT_INDEX environment for this particular
query." and "What's the value of this ref?  Please honor the
common-dir environment during this query".

So from that point of view, it may not matter where the "bit" lives,
among repository, object-store, or ref-store.


^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 18:57       ` Junio C Hamano
@ 2018-02-13 19:23         ` Stefan Beller
  2018-02-13 19:35           ` Junio C Hamano
  0 siblings, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-13 19:23 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Duy Nguyen, Git Mailing List, Eric Sunshine, Jonathan Tan

On Tue, Feb 13, 2018 at 10:57 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> Oh, that is an interesting perspective. Here is how I arrived at the opposite
>> conclusion initially: Searching for 'ignore_env' shows that we care about it
>> as well for the index and graft paths, which are not the object store, hence
>> it would be better kept in the repository. (The alternative would be to
>> duplicate it into the raw object store, but I do not like data duplication)
>>
>> But maybe it is better to duplicate this one bit instead of passing through
>> a larger scoped object.
>
> If a larger scoped repository refers to a smaller scoped
> object-store, is there still a need to duplicate that bit, instead
> of referring to the bit the smaller scoped one has when asking about
> the bit in the larger scoped one?

No (in theory). But in practice it may be worthwhile:

    "What's the value of this ref?"

    "Oh let me check the ignore_env bit that happens
    to live in the object store first."

would be super confusing to me.

> I am not sure if these "do not look at environment variables" is an
> attribute of these entities---it sounds more like an attribute for
> each invocation of an operation, i.e. "I want to learn where the
> index is but would ignore GIT_INDEX environment for this particular
> query." and "What's the value of this ref?  Please honor the
> common-dir environment during this query".

That sounds like we want to have a configset struct eventually.

For now the ignore_env bit lives in the repository, as that helps
when working with submodules, when reading its comments.
Unfortunately 359efeffc1 (repository: introduce the repository
object, 2017-06-22) did not reason about the existence of the ignore_env
flag in its commit message.

> So from that point of view, it may not matter where the "bit" lives,
> among repository, object-store, or ref-store.

It matters on the scale of confusing the developer?

Stefan

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (27 preceding siblings ...)
  2018-02-13 11:49 ` Duy Nguyen
@ 2018-02-13 19:26 ` Jonathan Nieder
  2018-02-14  0:57   ` Duy Nguyen
  2018-02-13 19:33 ` Brandon Williams
  2018-02-15 20:42 ` Junio C Hamano
  30 siblings, 1 reply; 100+ messages in thread
From: Jonathan Nieder @ 2018-02-13 19:26 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, pclouds, sunshine, jonathantanmy

Hi,

Stefan Beller wrote:

> This is a real take on the first part of the recent RFC[1].
>
> Jonathan Tan suggested[2] that "sha1_loose_object_info to handle arbitrary repositories"
> might be a good breaking point for a first part at that RFC at patch 38.
> This series is smaller and contains only 26 patches as the patches in the big
> RFC were slightly out of order.

Thanks.  This looks like a nice reviewable series, so I'm happy to see
it broken out.

[...]
> Comments in the early range of that RFC were on 003 where Junio pointed out
> that the coccinelle patch ought to be not in contrib/coccinelle, so I put it
> in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.

Can you say a little more about this?  Was the problem that the
semantic patch wasn't idempotent, that it was too slow to run, or
something else?

If we're including the semantic patch for reference but never running
it, I think I'd prefer it to go in the commit message.  But if it's
useful to run then we should make it idempotent so it can go in
contrib/coccinelle.

[...]
> Duy suggested that we shall not use the repository blindly, but should carefully
> examine whether to pass on an object store or the refstore or such[4], which
> I agree with if it makes sense. This series unfortunately has an issue with that
> as I would not want to pass down the `ignore_env` flag separately from the object
> store, so I made all functions that only take the object store to have the raw
> object store as the first parameter, and others using the full repository.

I think I want to push back on this a little.

The advantage of a function taking e.g. an object_store as an argument
instead of a repository is that it increases its flexibility, since it
allows callers that do not have access to a repository to call it.  The
disadvantage is also that it increases the flexibility without any
callers benefitting from that:

 1. It ties us to assumptions from today.  If e.g. an object access in
    the future starts relying on some other information from the
    repository (e.g. its config) then we'd have to either add a
    back-pointer from the object store to its repository or add
    additional arguments for that additional data at that point.

    If all callers already have a repository, it is simpler to pass
    that repository as context so that we have the flexibility to make
    more use of it later.

 2. It complicates the caller.  Instead of consistently passing the
    same repository argument as context to functions that access that
    repository, the caller would have to pull out relevant fields like
    the object store from it.

 3. It prevents us from making opportunistic use of other information
    from the repository, such as its name for use in error messages.

In lower-level funcitons that need to be usable by callers without a
repository (e.g. to find packfiles in an alternate) it makes sense to
not pass a repository, but without such a use case in mind I don't
think it needs to be a general goal.

To put it another way, most callers do not *care* whether they are
working with a repository's object store, ref database, or some other
aspect of the repository.  They just know they want to e.g. read an
object from this repository.

It's similar to how FILE * works: some operations rely on the buffer
the FILE * manages and some other operations only rely on the
underlying file descriptor, but using the FILE * consistently provides
a clean abstraction that generally makes life easier.

> Eric Sunshine brought up memory leaks with the RFC, and I would think to
> have plugged all holes.

Yay, thank you!

I'll try to find time to look at the patches in detail soon, but no
promises (i.e. if someone else reviews them first, then even better
;-)).

Sincerely,
Jonathan

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (28 preceding siblings ...)
  2018-02-13 19:26 ` Jonathan Nieder
@ 2018-02-13 19:33 ` Brandon Williams
  2018-02-15 20:42 ` Junio C Hamano
  30 siblings, 0 replies; 100+ messages in thread
From: Brandon Williams @ 2018-02-13 19:33 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, pclouds, sunshine, jonathantanmy

On 02/12, Stefan Beller wrote:
> This is a real take on the first part of the recent RFC[1].
> 
> Jonathan Tan suggested[2] that "sha1_loose_object_info to handle arbitrary repositories"
> might be a good breaking point for a first part at that RFC at patch 38.
> This series is smaller and contains only 26 patches as the patches in the big
> RFC were slightly out of order.
> 
> I developed this series partly by writing patches, but mostly by cherrypicking
> from that RFC on top of current master. I noticed no external conflicts apart
> from one addition to the repositories _INIT macro, which was easy to resolve.
> 
> Comments in the early range of that RFC were on 003 where Junio pointed out
> that the coccinelle patch ought to be not in contrib/coccinelle, so I put it
> in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.
> 
> brian had a questoin on patch 25 in the RFC, but that seemed to resolve itself
> without any suggestion to include into this series[3].
> 
> Duy suggested that we shall not use the repository blindly, but should carefully
> examine whether to pass on an object store or the refstore or such[4], which 
> I agree with if it makes sense. This series unfortunately has an issue with that
> as I would not want to pass down the `ignore_env` flag separately from the object
> store, so I made all functions that only take the object store to have the raw
> object store as the first parameter, and others using the full repository.
> 
> Eric Sunshine brought up memory leaks with the RFC, and I would think to
> have plugged all holes.

I've looked through the patches and I think they look good.  At the end
of the series all the #define tricks have been eliminated so we don't
have to worry about them possibly being left and forgotten :)


Thanks for getting the ball rolling on this.

> 
> [1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/
> [2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/
> [3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/
> [4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/
> 
> Thanks,
> Stefan
> 
> Jonathan Nieder (8):
>   pack: move prepare_packed_git_run_once to object store
>   pack: move approximate object count to object store
>   sha1_file: add repository argument to sha1_file_name
>   sha1_file: add repository argument to map_sha1_file
>   sha1_file: allow stat_sha1_file to handle arbitrary repositories
>   sha1_file: allow open_sha1_file to handle arbitrary repositories
>   sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
>   sha1_file: allow sha1_loose_object_info to handle arbitrary
>     repositories
> 
> Stefan Beller (18):
>   repository: introduce raw object store field
>   object-store: move alt_odb_list and alt_odb_tail to object store
>   object-store: free alt_odb_list
>   object-store: move packed_git and packed_git_mru to object store
>   object-store: close all packs upon clearing the object store
>   sha1_file: add raw_object_store argument to alt_odb_usable
>   sha1_file: add repository argument to link_alt_odb_entry
>   sha1_file: add repository argument to read_info_alternates
>   sha1_file: add repository argument to link_alt_odb_entries
>   sha1_file: add repository argument to prepare_alt_odb
>   sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
>   sha1_file: allow prepare_alt_odb to handle arbitrary repositories
>   sha1_file: add repository argument to stat_sha1_file
>   sha1_file: add repository argument to open_sha1_file
>   sha1_file: add repository argument to map_sha1_file_1
>   sha1_file: add repository argument to sha1_loose_object_info
>   sha1_file: allow sha1_file_name to handle arbitrary repositories
>   sha1_file: allow map_sha1_file to handle arbitrary repositories
> 
>  builtin/am.c                                    |   2 +-
>  builtin/clone.c                                 |   2 +-
>  builtin/count-objects.c                         |   6 +-
>  builtin/fetch.c                                 |   2 +-
>  builtin/fsck.c                                  |  13 ++-
>  builtin/gc.c                                    |   4 +-
>  builtin/grep.c                                  |   2 +-
>  builtin/index-pack.c                            |   1 +
>  builtin/merge.c                                 |   2 +-
>  builtin/pack-objects.c                          |  21 ++--
>  builtin/pack-redundant.c                        |   6 +-
>  builtin/receive-pack.c                          |   3 +-
>  cache.h                                         |  46 ++------
>  contrib/coccinelle/refactoring/packed_git.cocci |   7 ++
>  environment.c                                   |   5 +-
>  fast-import.c                                   |   6 +-
>  http-backend.c                                  |   6 +-
>  http-push.c                                     |   1 +
>  http-walker.c                                   |   4 +-
>  http.c                                          |   6 +-
>  mru.h                                           |   1 +
>  object-store.h                                  |  75 +++++++++++++
>  object.c                                        |  26 +++++
>  pack-bitmap.c                                   |   4 +-
>  pack-check.c                                    |   1 +
>  pack-revindex.c                                 |   1 +
>  packfile.c                                      |  64 +++++------
>  packfile.h                                      |   2 +-
>  path.c                                          |   2 +-
>  reachable.c                                     |   1 +
>  repository.c                                    |  17 ++-
>  repository.h                                    |   7 +-
>  server-info.c                                   |   6 +-
>  sha1_file.c                                     | 135 +++++++++++++-----------
>  sha1_name.c                                     |  11 +-
>  streaming.c                                     |   5 +-
>  36 files changed, 321 insertions(+), 182 deletions(-)
>  create mode 100644 contrib/coccinelle/refactoring/packed_git.cocci
>  create mode 100644 object-store.h
> 
> -- 
> 2.16.1.73.ga2c3e9663f.dirty
> 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 19:23         ` Stefan Beller
@ 2018-02-13 19:35           ` Junio C Hamano
  2018-02-13 19:43             ` Stefan Beller
  0 siblings, 1 reply; 100+ messages in thread
From: Junio C Hamano @ 2018-02-13 19:35 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Duy Nguyen, Git Mailing List, Eric Sunshine, Jonathan Tan

Stefan Beller <sbeller@google.com> writes:

> For now the ignore_env bit lives in the repository, as that helps
> when working with submodules, when reading its comments.
> Unfortunately 359efeffc1 (repository: introduce the repository
> object, 2017-06-22) did not reason about the existence of the ignore_env
> flag in its commit message.
>
>> So from that point of view, it may not matter where the "bit" lives,
>> among repository, object-store, or ref-store.
>
> It matters on the scale of confusing the developer?

What I meant is that from the point of view, having the bit in the
data (not on the invocation) is confusing no matter which data
structure holds it--they are equally bad.

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 19:35           ` Junio C Hamano
@ 2018-02-13 19:43             ` Stefan Beller
  2018-02-14  0:30               ` Junio C Hamano
  0 siblings, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-13 19:43 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Duy Nguyen, Git Mailing List, Eric Sunshine, Jonathan Tan

On Tue, Feb 13, 2018 at 11:35 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> For now the ignore_env bit lives in the repository, as that helps
>> when working with submodules, when reading its comments.
>> Unfortunately 359efeffc1 (repository: introduce the repository
>> object, 2017-06-22) did not reason about the existence of the ignore_env
>> flag in its commit message.
>>
>>> So from that point of view, it may not matter where the "bit" lives,
>>> among repository, object-store, or ref-store.
>>
>> It matters on the scale of confusing the developer?
>
> What I meant is that from the point of view, having the bit in the
> data (not on the invocation) is confusing no matter which data
> structure holds it--they are equally bad.

Right.

Which is why I'd strongly consider having it only in the repository
object as that is the largest-scoped thing we'd want. e.g. submodules
should care about environment variables differently:

    GIT_WORK_TREE=~/mysuperproject git checkout \
            --recurse-submodules master

 So with such a command in mind, the environment variable would
only apply to the superproject and the nested submodules should
ignore the env, but compute their paths off the superproject, i.e.
the superproject repository, not its object store?

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store
  2018-02-13 18:51   ` Brandon Williams
@ 2018-02-13 19:46     ` Stefan Beller
  0 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-13 19:46 UTC (permalink / raw)
  To: Brandon Williams; +Cc: Junio C Hamano, git, Duy Nguyen, Eric Sunshine, Jonathan Tan, Jonathan Nieder

On Tue, Feb 13, 2018 at 10:51 AM, Brandon Williams <bmwill@google.com> wrote:

>> +#include "cache.h"
>> +
>
> Maybe we would want to move the definition of the alternate store to
> this header file?  That way we don't need to include cache.h here.
>

Sounds good. I'll take a look at that.

Stefan

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 19:43             ` Stefan Beller
@ 2018-02-14  0:30               ` Junio C Hamano
  0 siblings, 0 replies; 100+ messages in thread
From: Junio C Hamano @ 2018-02-14  0:30 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Duy Nguyen, Git Mailing List, Eric Sunshine, Jonathan Tan

Stefan Beller <sbeller@google.com> writes:

> Which is why I'd strongly consider having it only in the repository
> object as that is the largest-scoped thing we'd want. e.g. submodules
> should care about environment variables differently:
>
>     GIT_WORK_TREE=~/mysuperproject git checkout \
>             --recurse-submodules master
>
>  So with such a command in mind, the environment variable would
> only apply to the superproject and the nested submodules should
> ignore the env, but compute their paths off the superproject, i.e.
> the superproject repository, not its object store?

In the longer term endgame state, what is in the repository object
cannot just be a simple "do we honor environment variable" bit
anymore.  

It will be more like "we may (or may not) look at environment when
we create a repository object", i.e. a bit passed to repo_init()
constructor, and from then on, the repository object knows where the
object store and its alternates are, where the top of the working
tree is, where the repository (i.e. the directory that has "refs/"
in it) is, what "worktree" of the repository we are talking about by
itself.  There is no need for a bit "do we or do we not look at
environment?" that needs to be consulted at runtime, which is quite
a round-about thing.

In your example, the repository object that represents the
superproject will pay attention to GIT_WORK_TREE environment when it
is consturcted, and repository objects dynamically constructed while
the command "recurse-submodules" through them will be constructed
with their "where is the top of my working tree?" set appropriately.
They won't be storing "when figuring out where my working tree is,
do not pay attention to GIT_WORK_TREE environment variable" bit.


^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 14/26] sha1_file: allow prepare_alt_odb to handle arbitrary repositories
  2018-02-13  1:22 ` [PATCH 14/26] sha1_file: allow prepare_alt_odb " Stefan Beller
@ 2018-02-14  0:37   ` Duy Nguyen
  2018-02-14 18:08     ` Brandon Williams
  0 siblings, 1 reply; 100+ messages in thread
From: Duy Nguyen @ 2018-02-14  0:37 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Junio C Hamano, Git Mailing List, Eric Sunshine, Jonathan Tan

On Tue, Feb 13, 2018 at 8:22 AM, Stefan Beller <sbeller@google.com> wrote:
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  object-store.h |  3 +--
>  sha1_file.c    | 21 +++++++++++----------
>  2 files changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/object-store.h b/object-store.h
> index d96a16edd1..add1d4e27c 100644
> --- a/object-store.h
> +++ b/object-store.h
> @@ -61,7 +61,6 @@ struct packed_git {
>         char pack_name[FLEX_ARRAY]; /* more */
>  };
>
> -#define prepare_alt_odb(r) prepare_alt_odb_##r()
> -extern void prepare_alt_odb_the_repository(void);
> +void prepare_alt_odb(struct repository *r);
>
>  #endif /* OBJECT_STORE_H */
> diff --git a/sha1_file.c b/sha1_file.c
> index d18ce3aeba..f046d560f8 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -677,21 +677,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
>         return r;
>  }
>
> -void prepare_alt_odb_the_repository(void)
> +void prepare_alt_odb(struct repository *r)
>  {
> -       const char *alt;
> -
> -       if (the_repository->objects.alt_odb_tail)
> +       if (r->objects.alt_odb_tail)
>                 return;
>
> -       alt = getenv(ALTERNATE_DB_ENVIRONMENT);
> +       r->objects.alt_odb_tail = &r->objects.alt_odb_list;
> +
> +       if (!r->ignore_env) {
> +               const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);

If one day the majority of git moves to use 'struct repository', then
ALTERNATE_DB_ENVIRONMENT is always ignored because ignore_env is
always true. I think if you ignore_env, then you still need to get
this "alt" from  'struct raw_object_store' (or 'struct repository').

Since you get lots of getenv() in repo_setup_env(), I think this
getenv(ALTERNATE_DB_ENVIRONMENT) belongs there too. Then here, if
ignore_env is true, you read r->objects.alt or something instead of
doing getenv().

I really want to kill this getenv() in this code, which is basically
delayed initialization because we haven't done proper init on
the_repo. I realize that it cannot be done earlier, when
prepare_alt_odb() does not even have a  'struct repository *' to work
with. Would it be ok if I contributed a patch on top of your series to
basically do repo_init(&the_repo) for all builtin/external commands
(and fix all the bugs that come with it)? Then we would not need
ignore_env here anymore.

> +               if (!alt)
> +                       alt = "";
>
> -       the_repository->objects.alt_odb_tail =
> -                       &the_repository->objects.alt_odb_list;
> -       link_alt_odb_entries(the_repository, alt,
> -                            PATH_SEP, NULL, 0);
> +               link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);
> +       }
>
> -       read_info_alternates(the_repository, get_object_directory(), 0);
> +       read_info_alternates(r, r->objects.objectdir, 0);
>  }
>
>  /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
> --
> 2.16.1.73.ga2c3e9663f.dirty
>



-- 
Duy

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13 19:26 ` Jonathan Nieder
@ 2018-02-14  0:57   ` Duy Nguyen
  0 siblings, 0 replies; 100+ messages in thread
From: Duy Nguyen @ 2018-02-14  0:57 UTC (permalink / raw)
  To: Jonathan Nieder; +Cc: Stefan Beller, Junio C Hamano, Git Mailing List, Eric Sunshine, Jonathan Tan

On Wed, Feb 14, 2018 at 2:26 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> Duy suggested that we shall not use the repository blindly, but should carefully
>> examine whether to pass on an object store or the refstore or such[4], which
>> I agree with if it makes sense. This series unfortunately has an issue with that
>> as I would not want to pass down the `ignore_env` flag separately from the object
>> store, so I made all functions that only take the object store to have the raw
>> object store as the first parameter, and others using the full repository.
>
> I think I want to push back on this a little.
>
> The advantage of a function taking e.g. an object_store as an argument
> instead of a repository is that it increases its flexibility, since it
> allows callers that do not have access to a repository to call it.  The
> disadvantage is also that it increases the flexibility without any
> callers benefitting from that:
>
>  1. It ties us to assumptions from today.  If e.g. an object access in
>     the future starts relying on some other information from the
>     repository (e.g. its config) then we'd have to either add a
>     back-pointer from the object store to its repository or add
>     additional arguments for that additional data at that point.
>
>     If all callers already have a repository, it is simpler to pass
>     that repository as context so that we have the flexibility to make
>     more use of it later.

It's essentially putting all global variables in the same place again.
Only this time it's not global namespace, but "struct repository".
It's no worse than the current state though.

>
>  2. It complicates the caller.  Instead of consistently passing the
>     same repository argument as context to functions that access that
>     repository, the caller would have to pull out relevant fields like
>     the object store from it.

Well, I see that as a good point :)

>
>  3. It prevents us from making opportunistic use of other information
>     from the repository, such as its name for use in error messages.

It does not exactly prevent us. It's just more effort to pass this
around. The repository name for example, there's no reason we can't
have object store name (which could be initialized the same as repo
name).

> In lower-level funcitons that need to be usable by callers without a
> repository (e.g. to find packfiles in an alternate) it makes sense to
> not pass a repository, but without such a use case in mind

I do agree with your benefit argument. But I'd like to point out
though that having all input to object store visible from something
like "struct raw_object_store" makes it easier to reason about the
code. I know how object store works, but occasionally I'm still
surprised when it getenv (or read $GIT_DIR/index, but not in object
store code) behind the scene. Imagine how hard it is for newcomers.

I would count that as benefit, even though it's not a use case per se.
Another potential benefit is writing unit tests will be much easier
(you can configure object store through struct repository too, but
setting one piece here, one piece there to control object store
behavior is not a nice experience). It's a nice thing to have, but not
a deciding factor.

> I don't think it needs to be a general goal.

My stand is a bit more aggressive. We should try to achieve better
abstraction if possible. But if it makes Stefan's life hell, it's not
worth doing. Converting to 'struct repository' is already a step
forward. Actually if it discourages him from finishing this work, it's
already not worth doing.
-- 
Duy

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 14/26] sha1_file: allow prepare_alt_odb to handle arbitrary repositories
  2018-02-14  0:37   ` Duy Nguyen
@ 2018-02-14 18:08     ` Brandon Williams
  0 siblings, 0 replies; 100+ messages in thread
From: Brandon Williams @ 2018-02-14 18:08 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Stefan Beller, Junio C Hamano, Git Mailing List, Eric Sunshine, Jonathan Tan

On 02/14, Duy Nguyen wrote:
> On Tue, Feb 13, 2018 at 8:22 AM, Stefan Beller <sbeller@google.com> wrote:
> > Signed-off-by: Stefan Beller <sbeller@google.com>
> > ---
> >  object-store.h |  3 +--
> >  sha1_file.c    | 21 +++++++++++----------
> >  2 files changed, 12 insertions(+), 12 deletions(-)
> >
> > diff --git a/object-store.h b/object-store.h
> > index d96a16edd1..add1d4e27c 100644
> > --- a/object-store.h
> > +++ b/object-store.h
> > @@ -61,7 +61,6 @@ struct packed_git {
> >         char pack_name[FLEX_ARRAY]; /* more */
> >  };
> >
> > -#define prepare_alt_odb(r) prepare_alt_odb_##r()
> > -extern void prepare_alt_odb_the_repository(void);
> > +void prepare_alt_odb(struct repository *r);
> >
> >  #endif /* OBJECT_STORE_H */
> > diff --git a/sha1_file.c b/sha1_file.c
> > index d18ce3aeba..f046d560f8 100644
> > --- a/sha1_file.c
> > +++ b/sha1_file.c
> > @@ -677,21 +677,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
> >         return r;
> >  }
> >
> > -void prepare_alt_odb_the_repository(void)
> > +void prepare_alt_odb(struct repository *r)
> >  {
> > -       const char *alt;
> > -
> > -       if (the_repository->objects.alt_odb_tail)
> > +       if (r->objects.alt_odb_tail)
> >                 return;
> >
> > -       alt = getenv(ALTERNATE_DB_ENVIRONMENT);
> > +       r->objects.alt_odb_tail = &r->objects.alt_odb_list;
> > +
> > +       if (!r->ignore_env) {
> > +               const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);
> 
> If one day the majority of git moves to use 'struct repository', then
> ALTERNATE_DB_ENVIRONMENT is always ignored because ignore_env is
> always true. I think if you ignore_env, then you still need to get
> this "alt" from  'struct raw_object_store' (or 'struct repository').
> 
> Since you get lots of getenv() in repo_setup_env(), I think this
> getenv(ALTERNATE_DB_ENVIRONMENT) belongs there too. Then here, if
> ignore_env is true, you read r->objects.alt or something instead of
> doing getenv().
> 
> I really want to kill this getenv() in this code, which is basically
> delayed initialization because we haven't done proper init on
> the_repo. I realize that it cannot be done earlier, when
> prepare_alt_odb() does not even have a  'struct repository *' to work
> with. Would it be ok if I contributed a patch on top of your series to
> basically do repo_init(&the_repo) for all builtin/external commands
> (and fix all the bugs that come with it)? Then we would not need
> ignore_env here anymore.

At some point yes we would definitely want the setup code to fully
initialize a repository object (in this case the_repository).  And I
would even like to change the function signatures of all the builtin
commands to take a repository object so that they don't implicitly rely
on the_repository at all.

When I introduced struct repository I seem to remember there being a
couple things which were different about setup that made it difficult to
simply call repo_init() on the_repository.  If you can fix whatever
those issues with setup were (I can't remember all of them) then that
would be great :)

> 
> > +               if (!alt)
> > +                       alt = "";
> >
> > -       the_repository->objects.alt_odb_tail =
> > -                       &the_repository->objects.alt_odb_list;
> > -       link_alt_odb_entries(the_repository, alt,
> > -                            PATH_SEP, NULL, 0);
> > +               link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);
> > +       }
> >
> > -       read_info_alternates(the_repository, get_object_directory(), 0);
> > +       read_info_alternates(r, r->objects.objectdir, 0);
> >  }
> >
> >  /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
> > --
> > 2.16.1.73.ga2c3e9663f.dirty
> >
> 
> 
> 
> -- 
> Duy

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
                   ` (29 preceding siblings ...)
  2018-02-13 19:33 ` Brandon Williams
@ 2018-02-15 20:42 ` Junio C Hamano
  2018-02-15 21:09   ` Stefan Beller
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
  30 siblings, 2 replies; 100+ messages in thread
From: Junio C Hamano @ 2018-02-15 20:42 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, pclouds, sunshine, jonathantanmy

Stefan Beller <sbeller@google.com> writes:

> This is a real take on the first part of the recent RFC[1].

For the patches remaining in this series, The scope is about right
and the size is more manageable.  With topics already on 'master',
they have some interactions:

 - ot/mru-on-list & gs/retire-mru

   The packed_git MRU has been greatly simplified by using list API
   directly.

 - cc/sha1-file-name

   The function signature of sha1_file_name() has been updated.

I could certainly carry evil merge resolutions going forward (these
changes need to be made to object-store.h that has no mechanical
conflicts), but it may be less distracting for later readers of the
series if we rebase it on top of a more recent master.

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 00/26] Moving global state into the repository object (part 1)
  2018-02-15 20:42 ` Junio C Hamano
@ 2018-02-15 21:09   ` Stefan Beller
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
  1 sibling, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-15 21:09 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Duy Nguyen, Eric Sunshine, Jonathan Tan

On Thu, Feb 15, 2018 at 12:42 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> This is a real take on the first part of the recent RFC[1].
>
> For the patches remaining in this series, The scope is about right
> and the size is more manageable.

ok, thanks.

>  With topics already on 'master',
> they have some interactions:
>
>  - ot/mru-on-list & gs/retire-mru
>
>    The packed_git MRU has been greatly simplified by using list API
>    directly.
>
>  - cc/sha1-file-name
>
>    The function signature of sha1_file_name() has been updated.
>
> I could certainly carry evil merge resolutions going forward (these
> changes need to be made to object-store.h that has no mechanical
> conflicts), but it may be less distracting for later readers of the
> series if we rebase it on top of a more recent master.

I was debating when to reroll this series as I want to make the
change that Duy proposed, moving the 'ignore_env' into the
object store as well.

I'll rebase on top of the latest master or these topicc while at it.

Thanks for the heads up.
Stefan

^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-15 20:42 ` Junio C Hamano
  2018-02-15 21:09   ` Stefan Beller
@ 2018-02-16 17:46   ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 01/16] repository: introduce raw object store field Stefan Beller
                       ` (17 more replies)
  1 sibling, 18 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine

v2:
* duplicated the 'ignore_env' bit into the object store as well
* the #define trick no longer works as we do not have a "the_objectstore" global,
  which means there is just one patch per function that is converted.
  As this follows the same structure of the previous series, I am still confident
  there is no hidden dependencies to globals outside the object store in these
  converted functions.
* rebased on top of current master, resolving the merge conflicts.
  I think I used the list.h APIs right, but please double check.
  
Thanks,
Stefan

v1:
This is a real take on the first part of the recent RFC[1].

Jonathan Tan suggested[2] that "sha1_loose_object_info to handle arbitrary repositories"
might be a good breaking point for a first part at that RFC at patch 38.
This series is smaller and contains only 26 patches as the patches in the big
RFC were slightly out of order.

I developed this series partly by writing patches, but mostly by cherrypicking
from that RFC on top of current master. I noticed no external conflicts apart
from one addition to the repositories _INIT macro, which was easy to resolve.

Comments in the early range of that RFC were on 003 where Junio pointed out
that the coccinelle patch ought to be not in contrib/coccinelle, so I put it
in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.

brian had a questoin on patch 25 in the RFC, but that seemed to resolve itself
without any suggestion to include into this series[3].

Duy suggested that we shall not use the repository blindly, but should carefully
examine whether to pass on an object store or the refstore or such[4], which
I agree with if it makes sense. This series unfortunately has an issue with that
as I would not want to pass down the `ignore_env` flag separately from the object
store, so I made all functions that only take the object store to have the raw
object store as the first parameter, and others using the full repository.

Eric Sunshine brought up memory leaks with the RFC, and I would think to
have plugged all holes.

[1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/
[2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/
[3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/
[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/

Thanks,
Stefan

Stefan Beller (16):
  repository: introduce raw object store field
  object-store: move alt_odb_list and alt_odb_tail to object store
  object-store: free alt_odb_list
  object-store: move packed_git and packed_git_mru to object store
  object-store: close all packs upon clearing the object store
  pack: move prepare_packed_git_run_once to object store
  pack: move approximate object count to object store
  sha1_file: add raw_object_store argument to alt_odb_usable
  sha1_file: allow link_alt_odb_entries to handle arbitrary object
    stores
  sha1_file: allow prepare_alt_odb to handle arbitrary object stores
  sha1_file: allow sha1_file_name to handle arbitrary object stores
  sha1_file: allow stat_sha1_file to handle arbitrary object stores
  sha1_file: allow open_sha1_file to handle arbitrary object stores
  sha1_file: allow map_sha1_file_1 to handle arbitrary object stores
  sha1_file: allow map_sha1_file to handle arbitrary object stores
  sha1_file: allow sha1_loose_object_info to handle arbitrary object
    stores

 builtin/am.c             |   2 +-
 builtin/clone.c          |   2 +-
 builtin/count-objects.c  |   6 +-
 builtin/fetch.c          |   2 +-
 builtin/fsck.c           |  13 +++--
 builtin/gc.c             |   4 +-
 builtin/grep.c           |   2 +-
 builtin/index-pack.c     |   1 +
 builtin/merge.c          |   2 +-
 builtin/pack-objects.c   |  19 +++---
 builtin/pack-redundant.c |   6 +-
 builtin/receive-pack.c   |   3 +-
 cache.h                  |  45 ++------------
 environment.c            |   5 +-
 fast-import.c            |   6 +-
 http-backend.c           |   6 +-
 http-push.c              |   1 +
 http-walker.c            |   4 +-
 http.c                   |   6 +-
 object-store.h           |  80 +++++++++++++++++++++++++
 object.c                 |  27 +++++++++
 pack-bitmap.c            |   4 +-
 pack-check.c             |   1 +
 pack-revindex.c          |   1 +
 packfile.c               |  64 ++++++++++----------
 packfile.h               |   2 +-
 path.c                   |   2 +-
 reachable.c              |   1 +
 repository.c             |  22 +++++--
 repository.h             |   7 ++-
 server-info.c            |   6 +-
 sha1_file.c              | 123 +++++++++++++++++++++------------------
 sha1_name.c              |  11 ++--
 streaming.c              |   5 +-
 34 files changed, 314 insertions(+), 177 deletions(-)
 create mode 100644 object-store.h

-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 01/16] repository: introduce raw object store field
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
@ 2018-02-16 17:46     ` Stefan Beller
  2018-02-16 17:46     ` [PATCH 02/16] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
                       ` (16 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

The raw object store field will contain any objects needed for
access to objects in a given repository.

This patch introduces the raw object store and populates it with the
`objectdir`, which used to be part of the repository struct as well as
'ignore_env' which is duplicated from the repository. A later refactoring
(not in this series) will hopefully get rid of that one bit, once all
the construction of repo objects goes through repo_init(). The reason for
duplication now is that the refactoring done in this series allows for
smaller scoped objects in the functions refactored, nameny passing around
the object store instead of the repository object.

As the struct gains members, we'll also populate the function to clear
the memory for these members.

In a later we'll introduce a struct object_parser, that will complement
the object parsing in a repository struct: The raw object parser is the
layer that will provide access to raw object content, while the higher
level object parser code will parse raw objects and keeps track of
parenthood and other object relationships using 'struct object'.
For now only add the lower level to the repository struct.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/grep.c |  2 +-
 environment.c  |  5 +++--
 object-store.h | 17 +++++++++++++++++
 object.c       |  5 +++++
 path.c         |  2 +-
 repository.c   | 19 +++++++++++++++----
 repository.h   |  7 ++++---
 7 files changed, 46 insertions(+), 11 deletions(-)
 create mode 100644 object-store.h

diff --git a/builtin/grep.c b/builtin/grep.c
index 3ca4ac80d8..0f0c195705 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objectdir);
+	add_to_alternates_memory(submodule.objects.objectdir);
 	grep_read_unlock();
 
 	if (oid) {
diff --git a/environment.c b/environment.c
index de8431e01e..ec10b062e6 100644
--- a/environment.c
+++ b/environment.c
@@ -13,6 +13,7 @@
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -244,9 +245,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objectdir)
+	if (!the_repository->objects.objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objectdir;
+	return the_repository->objects.objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
diff --git a/object-store.h b/object-store.h
new file mode 100644
index 0000000000..5959d990fc
--- /dev/null
+++ b/object-store.h
@@ -0,0 +1,17 @@
+#ifndef OBJECT_STORE_H
+#define OBJECT_STORE_H
+
+struct raw_object_store {
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+
+	unsigned ignore_env : 1;
+};
+#define RAW_OBJECT_STORE_INIT { NULL, 0 }
+
+void raw_object_store_clear(struct raw_object_store *o);
+
+#endif /* OBJECT_STORE_H */
diff --git a/object.c b/object.c
index 9e6f9ff20b..11d904c033 100644
--- a/object.c
+++ b/object.c
@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)
 			obj->flags &= ~flags;
 	}
 }
+
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	free(o->objectdir);
+}
diff --git a/path.c b/path.c
index da8b655730..81a42d9115 100644
--- a/path.c
+++ b/path.c
@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects.objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
diff --git a/repository.c b/repository.c
index 4ffbe9bc94..3b8f1b91be 100644
--- a/repository.c
+++ b/repository.c
@@ -1,11 +1,18 @@
 #include "cache.h"
 #include "repository.h"
+#include "object-store.h"
 #include "config.h"
 #include "submodule-config.h"
 
 /* The main repository */
 static struct repository the_repo = {
-	NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
+	NULL, NULL,
+	RAW_OBJECT_STORE_INIT,
+	NULL, NULL, NULL,
+	NULL, NULL, NULL,
+	&the_index,
+	&hash_algos[GIT_HASH_SHA1],
+	0, 0
 };
 struct repository *the_repository = &the_repo;
 
@@ -42,9 +49,11 @@ static void repo_setup_env(struct repository *repo)
 						    !repo->ignore_env);
 	free(repo->commondir);
 	repo->commondir = strbuf_detach(&sb, NULL);
-	free(repo->objectdir);
-	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+	free(repo->objects.objectdir);
+	repo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
 					    "objects", !repo->ignore_env);
+	repo->objects.ignore_env = repo->ignore_env;
+
 	free(repo->graft_file);
 	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
 					     "info/grafts", !repo->ignore_env);
@@ -209,12 +218,14 @@ void repo_clear(struct repository *repo)
 {
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
-	FREE_AND_NULL(repo->objectdir);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
+	raw_object_store_clear(&repo->objects);
+	memset(&repo->objects, 0, sizeof(repo->objects));
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
diff --git a/repository.h b/repository.h
index 0329e40c7f..1f8bc7a7cf 100644
--- a/repository.h
+++ b/repository.h
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+#include "object-store.h"
+
 struct config_set;
 struct index_state;
 struct submodule_cache;
@@ -21,10 +23,9 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Path to the repository's object store.
-	 * Cannot be NULL after initialization.
+	 * Holds any information related to the object store.
 	 */
-	char *objectdir;
+	struct raw_object_store objects;
 
 	/*
 	 * Path to the repository's graft file.
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 02/16] object-store: move alt_odb_list and alt_odb_tail to object store
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
  2018-02-16 17:46     ` [PATCH 01/16] repository: introduce raw object store field Stefan Beller
@ 2018-02-16 17:46     ` Stefan Beller
  2018-02-16 17:46     ` [PATCH 03/16] object-store: free alt_odb_list Stefan Beller
                       ` (15 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

In a process with multiple repositories open, alternates should be
associated to a single repository and not shared globally. Move
alt_odb_list and alt_odb_tail into the_repository and adjust callers
to reflect this.

Now that the alternative object data base is per repository, we're
leaking its memory upon freeing a repository. The next patch plugs
this hole.

No functional change intended.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/fsck.c |  4 +++-
 cache.h        |  4 ++--
 object-store.h |  7 ++++++-
 packfile.c     |  3 ++-
 sha1_file.c    | 25 ++++++++++++-------------
 sha1_name.c    |  3 ++-
 6 files changed, 27 insertions(+), 19 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index 7a8a679d4f..908e4f092a 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "commit.h"
 #include "tree.h"
@@ -716,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 		fsck_object_dir(get_object_directory());
 
 		prepare_alt_odb();
-		for (alt = alt_odb_list; alt; alt = alt->next)
+		for (alt = the_repository->objects.alt_odb_list;
+				alt; alt = alt->next)
 			fsck_object_dir(alt->path);
 
 		if (check_full) {
diff --git a/cache.h b/cache.h
index 9cac7bb518..f373c30e25 100644
--- a/cache.h
+++ b/cache.h
@@ -1576,7 +1576,7 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);
 extern void schedule_dir_for_removal(const char *name, int len);
 extern void remove_scheduled_dirs(void);
 
-extern struct alternate_object_database {
+struct alternate_object_database {
 	struct alternate_object_database *next;
 
 	/* see alt_scratch_buf() */
@@ -1594,7 +1594,7 @@ extern struct alternate_object_database {
 	struct oid_array loose_objects_cache;
 
 	char path[FLEX_ARRAY];
-} *alt_odb_list;
+};
 extern void prepare_alt_odb(void);
 extern char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
diff --git a/object-store.h b/object-store.h
index 5959d990fc..a3f0d6ac15 100644
--- a/object-store.h
+++ b/object-store.h
@@ -1,6 +1,8 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
+#include "cache.h"
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
@@ -8,9 +10,12 @@ struct raw_object_store {
 	 */
 	char *objectdir;
 
+	struct alternate_object_database *alt_odb_list;
+	struct alternate_object_database **alt_odb_tail;
+
 	unsigned ignore_env : 1;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, 0 }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index 7dbe8739d1..216ea836ee 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "list.h"
 #include "pack.h"
+#include "repository.h"
 #include "dir.h"
 #include "mergesort.h"
 #include "packfile.h"
@@ -891,7 +892,7 @@ void prepare_packed_git(void)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next)
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
diff --git a/sha1_file.c b/sha1_file.c
index 831d9e7343..1348dce68f 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -22,6 +22,7 @@
 #include "pack-revindex.h"
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -342,9 +343,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 	return buf->buf;
 }
 
-struct alternate_object_database *alt_odb_list;
-static struct alternate_object_database **alt_odb_tail;
-
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
@@ -364,7 +362,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -424,8 +422,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*alt_odb_tail = ent;
-	alt_odb_tail = &(ent->next);
+	*the_repository->objects.alt_odb_tail = ent;
+	the_repository->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
@@ -559,7 +557,7 @@ void add_to_alternates_file(const char *reference)
 		fprintf_or_die(out, "%s\n", reference);
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
-		if (alt_odb_tail)
+		if (the_repository->objects.alt_odb_tail)
 			link_alt_odb_entries(reference, '\n', NULL, 0);
 	}
 	free(alts);
@@ -657,7 +655,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	int r = 0;
 
 	prepare_alt_odb();
-	for (ent = alt_odb_list; ent; ent = ent->next) {
+	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
 			break;
@@ -669,12 +667,13 @@ void prepare_alt_odb(void)
 {
 	const char *alt;
 
-	if (alt_odb_tail)
+	if (the_repository->objects.alt_odb_tail)
 		return;
 
 	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
 
-	alt_odb_tail = &alt_odb_list;
+	the_repository->objects.alt_odb_tail =
+			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
@@ -718,7 +717,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
 			return 1;
@@ -878,7 +877,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 
 	prepare_alt_odb();
 	errno = ENOENT;
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -908,7 +907,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	most_interesting_errno = errno;
 
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
diff --git a/sha1_name.c b/sha1_name.c
index 611c7d24dd..957ce25680 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -10,6 +10,7 @@
 #include "dir.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "repository.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
 		 */
 		fakeent = alloc_alt_odb(get_object_directory());
 	}
-	fakeent->next = alt_odb_list;
+	fakeent->next = the_repository->objects.alt_odb_list;
 
 	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
 		int pos;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 03/16] object-store: free alt_odb_list
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
  2018-02-16 17:46     ` [PATCH 01/16] repository: introduce raw object store field Stefan Beller
  2018-02-16 17:46     ` [PATCH 02/16] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
@ 2018-02-16 17:46     ` Stefan Beller
  2018-02-16 17:46     ` [PATCH 04/16] object-store: move packed_git and packed_git_mru to object store Stefan Beller
                       ` (14 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine

Free the memory and reset alt_odb_{list, tail} to NULL.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/object.c b/object.c
index 11d904c033..17b1da6d6b 100644
--- a/object.c
+++ b/object.c
@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)
 	}
 }
 
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+	strbuf_release(&alt->scratch);
+	oid_array_clear(&alt->loose_objects_cache);
+}
+
+static void free_alt_odbs(struct raw_object_store *o)
+{
+	while (o->alt_odb_list) {
+		free_alt_odb(o->alt_odb_list);
+		o->alt_odb_list = o->alt_odb_list->next;
+	}
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	free(o->objectdir);
+
+	free_alt_odbs(o);
+	o->alt_odb_tail = NULL;
 }
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 04/16] object-store: move packed_git and packed_git_mru to object store
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (2 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 03/16] object-store: free alt_odb_list Stefan Beller
@ 2018-02-16 17:46     ` Stefan Beller
  2018-02-16 20:27       ` Junio C Hamano
  2018-02-16 17:46     ` [PATCH 05/16] object-store: close all packs upon clearing the " Stefan Beller
                       ` (13 subsequent siblings)
  17 siblings, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

In a process with multiple repositories open, packfile accessors
should be associated to a single repository and not shared globally.
Move packed_git and packed_git_mru into the_repository and adjust
callers to reflect this.

Patch generated by

 1. Moving the struct packed_git declaration to object-store.h
    and packed_git, packed_git_mru globals to struct object_store.

 2. Applying the semantic patch to adjust callers.
  @@ @@
  - packed_git
  + the_repository->objects.packed_git

  @@ @@
  - packed_git_mru
  + the_repository->objects.packed_git_mru

 3. Applying line wrapping fixes from "make style" to break the
    resulting long lines.

 4. Adding missing #includes of repository.h and object-store.h
    where needed.

 5. As the packfiles are now owned by an objectstore/repository, which
    is ephemeral unlike globals, we introduce memory leaks. So address
    them in raw_object_store_clear().

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/count-objects.c  |  6 ++++--
 builtin/fsck.c           |  7 +++++--
 builtin/gc.c             |  4 +++-
 builtin/index-pack.c     |  1 +
 builtin/pack-objects.c   | 19 +++++++++++--------
 builtin/pack-redundant.c |  6 ++++--
 builtin/receive-pack.c   |  1 +
 cache.h                  | 29 -----------------------------
 fast-import.c            |  6 ++++--
 http-backend.c           |  6 ++++--
 http-push.c              |  1 +
 http-walker.c            |  1 +
 http.c                   |  1 +
 object-store.h           | 36 +++++++++++++++++++++++++++++++++++-
 object.c                 |  7 +++++++
 pack-bitmap.c            |  4 +++-
 pack-check.c             |  1 +
 pack-revindex.c          |  1 +
 packfile.c               | 39 ++++++++++++++++++++-------------------
 reachable.c              |  1 +
 repository.c             |  3 +++
 server-info.c            |  6 ++++--
 sha1_name.c              |  5 +++--
 23 files changed, 118 insertions(+), 73 deletions(-)

diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 33343818c8..9334648dcc 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -7,6 +7,8 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
+#include "object-store.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -120,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!the_repository->objects.packed_git)
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 908e4f092a..2e99e02128 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
@@ -729,7 +730,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = the_repository->objects.packed_git; p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -737,7 +739,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
diff --git a/builtin/gc.c b/builtin/gc.c
index 77fa720bd0..96ff790867 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -18,6 +19,7 @@
 #include "run-command.h"
 #include "sigchain.h"
 #include "argv-array.h"
+#include "object-store.h"
 #include "commit.h"
 #include "packfile.h"
 
@@ -173,7 +175,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 5ebd370c56..e60cc3ed98 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -12,6 +12,7 @@
 #include "exec_cmd.h"
 #include "streaming.h"
 #include "thread-utils.h"
+#include "object-store.h"
 #include "packfile.h"
 
 static const char index_pack_usage[] =
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 83dcbc9773..d73528b583 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1,6 +1,8 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "attr.h"
 #include "object.h"
 #include "blob.h"
@@ -1026,7 +1028,7 @@ static int want_object_in_pack(const struct object_id *oid,
 			return want;
 	}
 
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1044,7 +1046,7 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru, &the_repository->objects.packed_git_mru);
 			if (want != -1)
 				return want;
 		}
@@ -2673,7 +2675,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2736,7 +2738,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					the_repository->objects.packed_git;
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2745,7 +2748,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = the_repository->objects.packed_git;
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2781,7 +2784,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3152,7 +3155,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = the_repository->objects.packed_git; p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3165,7 +3168,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index aaa8136322..55462bc826 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -7,6 +7,8 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 #define BLKSIZE 512
@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b7ce7c7f52..b2eac79a6e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "sideband.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "remote.h"
diff --git a/cache.h b/cache.h
index f373c30e25..018a297e10 100644
--- a/cache.h
+++ b/cache.h
@@ -1636,35 +1636,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list.
- */
-extern struct list_head packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
diff --git a/fast-import.c b/fast-import.c
index b70ac025e0..1685fe59a2 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -154,8 +154,10 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "object.h"
 #include "blob.h"
 #include "tree.h"
@@ -1110,7 +1112,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1305,7 +1307,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/http-backend.c b/http-backend.c
index f3dc218b2a..4d93126c15 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -1,7 +1,9 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "exec_cmd.h"
@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http-push.c b/http-push.c
index 0913f8ab86..1ca662390f 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "blob.h"
diff --git a/http-walker.c b/http-walker.c
index 07c2b1af82..8bb5d991bb 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -4,6 +4,7 @@
 #include "http.h"
 #include "list.h"
 #include "transport.h"
+#include "object-store.h"
 #include "packfile.h"
 
 struct alt_base {
diff --git a/http.c b/http.c
index 31755023a4..a4a9e583c7 100644
--- a/http.c
+++ b/http.c
@@ -1,6 +1,7 @@
 #include "git-compat-util.h"
 #include "http.h"
 #include "config.h"
+#include "object-store.h"
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/object-store.h b/object-store.h
index a3f0d6ac15..024ccc91e9 100644
--- a/object-store.h
+++ b/object-store.h
@@ -2,6 +2,7 @@
 #define OBJECT_STORE_H
 
 #include "cache.h"
+#include "list.h"
 
 struct raw_object_store {
 	/*
@@ -10,13 +11,46 @@ struct raw_object_store {
 	 */
 	char *objectdir;
 
+	struct packed_git *packed_git;
+	/*
+	 * A most-recently-used ordered version of the packed_git list, which can
+	 * be iterated instead of packed_git (and marked via mru_mark).
+	 */
+	struct list_head packed_git_mru;
+
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
 	unsigned ignore_env : 1;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL, 0 }
+
+#define MRU_LIST_INIT {NULL, NULL}
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
+extern struct packed_git {
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+} *packed_git;
+
 #endif /* OBJECT_STORE_H */
diff --git a/object.c b/object.c
index 17b1da6d6b..c76b62572a 100644
--- a/object.c
+++ b/object.c
@@ -466,4 +466,11 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	while (!list_empty(&o->packed_git_mru))
+		list_del(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 9270983e5f..d51172b1d5 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -5,11 +5,13 @@
 #include "revision.h"
 #include "progress.h"
 #include "list-objects.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-bitmap.h"
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
diff --git a/pack-check.c b/pack-check.c
index 073c1fbd46..c80c3ed8fd 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack.h"
+#include "object-store.h"
 #include "pack-revindex.h"
 #include "progress.h"
 #include "packfile.h"
diff --git a/pack-revindex.c b/pack-revindex.c
index ff5f62c033..bb521cf7fb 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack-revindex.h"
+#include "object-store.h"
 
 /*
  * Pack index for existing packs give us easy access to the offsets into
diff --git a/packfile.c b/packfile.c
index 216ea836ee..d41e4c83d0 100644
--- a/packfile.c
+++ b/packfile.c
@@ -7,6 +7,7 @@
 #include "packfile.h"
 #include "delta.h"
 #include "list.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "sha1-lookup.h"
 #include "commit.h"
@@ -45,8 +46,6 @@ static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
 static size_t peak_pack_mapped;
 static size_t pack_mapped;
-struct packed_git *packed_git;
-LIST_HEAD(packed_git_mru);
 
 #define SZ_FMT PRIuMAX
 static inline uintmax_t sz_fmt(size_t s) { return s; }
@@ -246,7 +245,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -316,7 +315,7 @@ void close_all_packs(void)
 {
 	struct packed_git *p;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
@@ -384,7 +383,7 @@ static int close_one_pack(void)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -686,8 +685,8 @@ void install_packed_git(struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = packed_git;
-	packed_git = pack;
+	pack->next = the_repository->objects.packed_git;
+	the_repository->objects.packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -769,7 +768,8 @@ static void prepare_packed_git_one(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
 				    len == base_len &&
@@ -820,7 +820,7 @@ unsigned long approximate_object_count(void)
 
 		prepare_packed_git();
 		count = 0;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -869,18 +869,19 @@ static int sort_pack(const void *a_, const void *b_)
 
 static void rearrange_packed_git(void)
 {
-	packed_git = llist_mergesort(packed_git, get_next_packed_git,
-				     set_next_packed_git, sort_pack);
+	the_repository->objects.packed_git = llist_mergesort(
+		the_repository->objects.packed_git, get_next_packed_git,
+		set_next_packed_git, sort_pack);
 }
 
 static void prepare_packed_git_mru(void)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&packed_git_mru);
+	INIT_LIST_HEAD(&the_repository->objects.packed_git_mru);
 
-	for (p = packed_git; p; p = p->next)
-		list_add_tail(&p->mru, &packed_git_mru);
+	for (p = the_repository->objects.packed_git; p; p = p->next)
+		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
 static int prepare_packed_git_run_once = 0;
@@ -1014,7 +1015,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)
 	struct packed_git *p;
 	unsigned i;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		for (i = 0; i < p->num_bad_objects; i++)
 			if (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))
 				return p;
@@ -1845,13 +1846,13 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 	struct list_head *pos;
 
 	prepare_packed_git();
-	if (!packed_git)
+	if (!the_repository->objects.packed_git)
 		return 0;
 
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (fill_pack_entry(sha1, e, p)) {
-			list_move(&p->mru, &packed_git_mru);
+			list_move(&p->mru, &the_repository->objects.packed_git_mru);
 			return 1;
 		}
 	}
@@ -1898,7 +1899,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int pack_errors = 0;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/reachable.c b/reachable.c
index 88d7d679da..59ed16ba76 100644
--- a/reachable.c
+++ b/reachable.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tag.h"
 #include "commit.h"
 #include "blob.h"
diff --git a/repository.c b/repository.c
index 3b8f1b91be..61b23558d0 100644
--- a/repository.c
+++ b/repository.c
@@ -144,6 +144,9 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 
 	repo->ignore_env = 1;
 
+	repo->objects.ignore_env = 1;
+	INIT_LIST_HEAD(&repo->objects.packed_git_mru);
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
diff --git a/server-info.c b/server-info.c
index 26a6c20b7d..96885c45f6 100644
--- a/server-info.c
+++ b/server-info.c
@@ -1,5 +1,7 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "tag.h"
@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
diff --git a/sha1_name.c b/sha1_name.c
index 957ce25680..016c883b5c 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = the_repository->objects.packed_git; p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 05/16] object-store: close all packs upon clearing the object store
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (3 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 04/16] object-store: move packed_git and packed_git_mru to object store Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 06/16] pack: move prepare_packed_git_run_once to " Stefan Beller
                       ` (12 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 6 ++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 5bdd2d7578..4762a702e3 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
diff --git a/builtin/clone.c b/builtin/clone.c
index 101c27a593..13cfaa6488 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 8ee998ea2e..4d72efca78 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
diff --git a/builtin/merge.c b/builtin/merge.c
index 30264cfd7c..907ae44ab5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b2eac79a6e..954fc72c7c 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
diff --git a/object.c b/object.c
index c76b62572a..34daaf37b3 100644
--- a/object.c
+++ b/object.c
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -469,8 +470,5 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	while (!list_empty(&o->packed_git_mru))
 		list_del(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
 }
diff --git a/packfile.c b/packfile.c
index d41e4c83d0..511a2b0cdf 100644
--- a/packfile.c
+++ b/packfile.c
@@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
diff --git a/packfile.h b/packfile.h
index a7fca598d6..6a2c57045c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 06/16] pack: move prepare_packed_git_run_once to object store
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (4 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 05/16] object-store: close all packs upon clearing the " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 07/16] pack: move approximate object count " Stefan Beller
                       ` (11 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

Each repository's object store can be initialized independently, so
they must not share a run_once variable.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 8 +++++++-
 packfile.c     | 7 +++----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/object-store.h b/object-store.h
index 024ccc91e9..794cb0af59 100644
--- a/object-store.h
+++ b/object-store.h
@@ -22,10 +22,16 @@ struct raw_object_store {
 	struct alternate_object_database **alt_odb_tail;
 
 	unsigned ignore_env : 1;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
 
 #define MRU_LIST_INIT {NULL, NULL}
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0 }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index 511a2b0cdf..a8a2e7fe43 100644
--- a/packfile.c
+++ b/packfile.c
@@ -884,12 +884,11 @@ static void prepare_packed_git_mru(void)
 		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -897,13 +896,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 07/16] pack: move approximate object count to object store
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (5 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 06/16] pack: move prepare_packed_git_run_once to " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 08/16] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
                       ` (10 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

The approximate_object_count() function maintains a rough count of
objects in a repository to estimate how long object name abbreviates
should be.  Object names are scoped to a repository and the
appropriate length may differ by repository, so the object count
should not be global.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 10 +++++++++-
 packfile.c     | 11 +++++------
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/object-store.h b/object-store.h
index 794cb0af59..80a2ca6f75 100644
--- a/object-store.h
+++ b/object-store.h
@@ -23,6 +23,14 @@ struct raw_object_store {
 
 	unsigned ignore_env : 1;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
@@ -31,7 +39,7 @@ struct raw_object_store {
 };
 
 #define MRU_LIST_INIT {NULL, NULL}
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0, 0 }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0, 0, 0, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index a8a2e7fe43..693bafbc98 100644
--- a/packfile.c
+++ b/packfile.c
@@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -814,8 +812,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects.approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects.approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects.approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -901,7 +900,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 08/16] sha1_file: add raw_object_store argument to alt_odb_usable
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (6 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 07/16] pack: move approximate object count " Stefan Beller
@ 2018-02-16 17:46     ` Stefan Beller
  2018-02-16 17:46     ` [PATCH 09/16] sha1_file: allow link_alt_odb_entries to handle arbitrary object stores Stefan Beller
                       ` (9 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine

Add a raw_object_store to alt_odb_usable to be more specific about which
repository to act on. The choice of the repository is delegated to its
only caller link_alt_odb_entry.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 1348dce68f..a8e23bd2f8 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -346,7 +346,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir)
 {
 	struct alternate_object_database *alt;
 
@@ -362,7 +364,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -414,7 +416,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 09/16] sha1_file: allow link_alt_odb_entries to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (7 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 08/16] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
@ 2018-02-16 17:46     ` Stefan Beller
  2018-02-16 17:46     ` [PATCH 10/16] sha1_file: allow prepare_alt_odb " Stefan Beller
                       ` (8 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

Actually this also allows read_info_alternates and link_alt_odb_entry to
handle arbitrary object stores, but link_alt_odb_entries is the most
interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function
graph, i.e. the recursive call chain might look like

  -> link_alt_odb_entries
    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h     |  4 ++++
 sha1_file.c | 42 +++++++++++++++++++++++++-----------------
 2 files changed, 29 insertions(+), 17 deletions(-)

diff --git a/cache.h b/cache.h
index 018a297e10..12bb42f951 100644
--- a/cache.h
+++ b/cache.h
@@ -1593,6 +1593,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 extern void prepare_alt_odb(void);
diff --git a/sha1_file.c b/sha1_file.c
index a8e23bd2f8..c4255a2da6 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -389,9 +389,11 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void read_info_alternates(const char * relative_base, int depth);
-static int link_alt_odb_entry(const char *entry, const char *relative_base,
-	int depth, const char *normalized_objdir)
+static void read_info_alternates(struct raw_object_store *o,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct raw_object_store *o, const char *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
 	struct strbuf pathbuf = STRBUF_INIT;
@@ -416,7 +418,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(o, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -424,12 +426,12 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*o->alt_odb_tail = ent;
+	o->alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(pathbuf.buf, depth + 1);
+	read_info_alternates(o, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -464,8 +466,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-static void link_alt_odb_entries(const char *alt, int sep,
-				 const char *relative_base, int depth)
+static void link_alt_odb_entries(struct raw_object_store *o, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -479,7 +481,7 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, o->objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -488,13 +490,16 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);
+		link_alt_odb_entry(o, entry.buf,
+				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates(const char * relative_base, int depth)
+static void read_info_alternates(struct raw_object_store *o,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -506,7 +511,7 @@ static void read_info_alternates(const char * relative_base, int depth)
 		return;
 	}
 
-	link_alt_odb_entries(buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(o, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
@@ -560,7 +565,8 @@ void add_to_alternates_file(const char *reference)
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
 		if (the_repository->objects.alt_odb_tail)
-			link_alt_odb_entries(reference, '\n', NULL, 0);
+			link_alt_odb_entries(&the_repository->objects, reference,
+					     '\n', NULL, 0);
 	}
 	free(alts);
 }
@@ -573,7 +579,8 @@ void add_to_alternates_memory(const char *reference)
 	 */
 	prepare_alt_odb();
 
-	link_alt_odb_entries(reference, '\n', NULL, 0);
+	link_alt_odb_entries(&the_repository->objects, reference,
+			     '\n', NULL, 0);
 }
 
 /*
@@ -676,9 +683,10 @@ void prepare_alt_odb(void)
 
 	the_repository->objects.alt_odb_tail =
 			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(&the_repository->objects, alt,
+			     PATH_SEP, NULL, 0);
 
-	read_info_alternates(get_object_directory(), 0);
+	read_info_alternates(&the_repository->objects, get_object_directory(), 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 10/16] sha1_file: allow prepare_alt_odb to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (8 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 09/16] sha1_file: allow link_alt_odb_entries to handle arbitrary object stores Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 11/16] sha1_file: allow sha1_file_name " Stefan Beller
                       ` (7 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/fsck.c |  2 +-
 cache.h        |  1 -
 object-store.h |  2 ++
 packfile.c     |  2 +-
 sha1_file.c    | 23 +++++++++++------------
 sha1_name.c    |  3 ++-
 6 files changed, 17 insertions(+), 16 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index 2e99e02128..c0d967d6d7 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -717,7 +717,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	} else {
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(&the_repository->objects);
 		for (alt = the_repository->objects.alt_odb_list;
 				alt; alt = alt->next)
 			fsck_object_dir(alt->path);
diff --git a/cache.h b/cache.h
index 12bb42f951..d0c1a533c5 100644
--- a/cache.h
+++ b/cache.h
@@ -1599,7 +1599,6 @@ struct alternate_object_database {
 	 */
 	char path[FLEX_ARRAY];
 };
-extern void prepare_alt_odb(void);
 extern char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 extern int foreach_alt_odb(alt_odb_fn, void*);
diff --git a/object-store.h b/object-store.h
index 80a2ca6f75..280e8e16ad 100644
--- a/object-store.h
+++ b/object-store.h
@@ -67,4 +67,6 @@ extern struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 } *packed_git;
 
+void prepare_alt_odb(struct raw_object_store *o);
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 693bafbc98..83b10e55c0 100644
--- a/packfile.c
+++ b/packfile.c
@@ -890,7 +890,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(&the_repository->objects);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
diff --git a/sha1_file.c b/sha1_file.c
index c4255a2da6..04a81e29f2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -577,7 +578,7 @@ void add_to_alternates_memory(const char *reference)
 	 * Make sure alternates are initialized, or else our entry may be
 	 * overwritten when they are.
 	 */
-	prepare_alt_odb();
+	prepare_alt_odb(&the_repository->objects);
 
 	link_alt_odb_entries(&the_repository->objects, reference,
 			     '\n', NULL, 0);
@@ -663,7 +664,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	struct alternate_object_database *ent;
 	int r = 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(&the_repository->objects);
 	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
@@ -672,21 +673,19 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(void)
+void prepare_alt_odb(struct raw_object_store *o)
 {
 	const char *alt;
 
-	if (the_repository->objects.alt_odb_tail)
+	if (o->alt_odb_tail)
 		return;
 
 	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
 
-	the_repository->objects.alt_odb_tail =
-			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(&the_repository->objects, alt,
-			     PATH_SEP, NULL, 0);
+	o->alt_odb_tail = &o->alt_odb_list;
+	link_alt_odb_entries(o, alt, PATH_SEP, NULL, 0);
 
-	read_info_alternates(&the_repository->objects, get_object_directory(), 0);
+	read_info_alternates(o, o->objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
@@ -726,7 +725,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
-	prepare_alt_odb();
+	prepare_alt_odb(&the_repository->objects);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
@@ -885,7 +884,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(&the_repository->objects);
 	errno = ENOENT;
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
@@ -916,7 +915,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb();
+	prepare_alt_odb(&the_repository->objects);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
diff --git a/sha1_name.c b/sha1_name.c
index 016c883b5c..f1e6c910ba 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -11,6 +11,7 @@
 #include "sha1-array.h"
 #include "packfile.h"
 #include "repository.h"
+#include "object-store.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -353,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,
 
 	ds->len = len;
 	ds->hex_pfx[len] = '\0';
-	prepare_alt_odb();
+	prepare_alt_odb(&the_repository->objects);
 	return 0;
 }
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 11/16] sha1_file: allow sha1_file_name to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (9 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 10/16] sha1_file: allow prepare_alt_odb " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 12/16] sha1_file: allow stat_sha1_file " Stefan Beller
                       ` (6 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

While at it, move the declaration to object-store.h, where it should
be easier to find.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 cache.h        |  6 ------
 http-walker.c  |  3 ++-
 http.c         |  5 ++---
 object-store.h |  6 ++++++
 sha1_file.c    | 12 ++++++------
 5 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/cache.h b/cache.h
index d0c1a533c5..2495c7081d 100644
--- a/cache.h
+++ b/cache.h
@@ -961,12 +961,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified sha1.
- */
-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);
-
 /*
  * Return an abbreviated sha1 unique within this repository's object database.
  * The result will be at least `len` characters long, and will be NUL
diff --git a/http-walker.c b/http-walker.c
index 8bb5d991bb..59bc0417d2 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "walker.h"
 #include "http.h"
@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)
 		ret = error("File %s has bad hash", hex);
 	} else if (req->rename < 0) {
 		struct strbuf buf = STRBUF_INIT;
-		sha1_file_name(&buf, req->sha1);
+		sha1_file_name(&the_repository->objects, &buf, req->sha1);
 		ret = error("unable to write sha1 filename %s", buf.buf);
 		strbuf_release(&buf);
 	}
diff --git a/http.c b/http.c
index a4a9e583c7..058436d9d3 100644
--- a/http.c
+++ b/http.c
@@ -2247,7 +2247,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	hashcpy(freq->sha1, sha1);
 	freq->localfile = -1;
 
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(&the_repository->objects, &filename, sha1);
 	snprintf(freq->tmpfile, sizeof(freq->tmpfile),
 		 "%s.temp", filename.buf);
 
@@ -2396,8 +2396,7 @@ int finish_http_object_request(struct http_object_request *freq)
 		unlink_or_warn(freq->tmpfile);
 		return -1;
 	}
-
-	sha1_file_name(&filename, freq->sha1);
+	sha1_file_name(&the_repository->objects, &filename, freq->sha1);
 	freq->rename = finalize_object_file(freq->tmpfile, filename.buf);
 	strbuf_release(&filename);
 
diff --git a/object-store.h b/object-store.h
index 280e8e16ad..5f6beb9d91 100644
--- a/object-store.h
+++ b/object-store.h
@@ -67,6 +67,12 @@ extern struct packed_git {
 	char pack_name[FLEX_ARRAY]; /* more */
 } *packed_git;
 
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified sha1.
+ */
+void sha1_file_name(struct raw_object_store *o, struct strbuf *buf, const unsigned char *sha1);
+
 void prepare_alt_odb(struct raw_object_store *o);
 
 #endif /* OBJECT_STORE_H */
diff --git a/sha1_file.c b/sha1_file.c
index 04a81e29f2..d64cdbb494 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -323,9 +323,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct raw_object_store *o, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, get_object_directory());
+	strbuf_addstr(buf, o->objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }
@@ -717,7 +717,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(&the_repository->objects, &buf, sha1);
 
 	return check_and_freshen_file(buf.buf, freshen);
 }
@@ -878,7 +878,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(&the_repository->objects, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
@@ -907,7 +907,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(&the_repository->objects, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -1592,7 +1592,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	static struct strbuf filename = STRBUF_INIT;
 
 	strbuf_reset(&filename);
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(&the_repository->objects, &filename, sha1);
 
 	fd = create_tmpfile(&tmp_file, filename.buf);
 	if (fd < 0) {
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 12/16] sha1_file: allow stat_sha1_file to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (10 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 11/16] sha1_file: allow sha1_file_name " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 13/16] sha1_file: allow open_sha1_file " Stefan Beller
                       ` (5 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index d64cdbb494..bd8b0331f0 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -871,22 +871,23 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
-			  const char **path)
+static int stat_sha1_file(struct raw_object_store *o, const unsigned char *sha1,
+			  struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&the_repository->objects, &buf, sha1);
+	sha1_file_name(o, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb(&the_repository->objects);
+	prepare_alt_odb(o);
+
 	errno = ENOENT;
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -1178,7 +1179,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(sha1, &st, &path) < 0)
+		if (stat_sha1_file(&the_repository->objects, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
@@ -1392,7 +1393,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,
 		die("replacement %s not found for %s",
 		    sha1_to_hex(repl), sha1_to_hex(sha1));
 
-	if (!stat_sha1_file(repl, &st, &path))
+	if (!stat_sha1_file(&the_repository->objects, repl, &st, &path))
 		die("loose object %s (stored in %s) is corrupt",
 		    sha1_to_hex(repl), path);
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 13/16] sha1_file: allow open_sha1_file to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (11 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 12/16] sha1_file: allow stat_sha1_file " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 14/16] sha1_file: allow map_sha1_file_1 " Stefan Beller
                       ` (4 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index bd8b0331f0..3356f70dd2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -900,7 +900,8 @@ static int stat_sha1_file(struct raw_object_store *o, const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-static int open_sha1_file(const unsigned char *sha1, const char **path)
+static int open_sha1_file(struct raw_object_store *o,
+			  const unsigned char *sha1, const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -908,7 +909,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&the_repository->objects, &buf, sha1);
+	sha1_file_name(o, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -916,8 +917,8 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb(&the_repository->objects);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	prepare_alt_odb(o);
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
@@ -943,7 +944,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(sha1, &path);
+		fd = open_sha1_file(&the_repository->objects, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 14/16] sha1_file: allow map_sha1_file_1 to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (12 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 13/16] sha1_file: allow open_sha1_file " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 15/16] sha1_file: allow map_sha1_file " Stefan Beller
                       ` (3 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 3356f70dd2..aeb925a895 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -934,9 +934,8 @@ static int open_sha1_file(struct raw_object_store *o,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-static void *map_sha1_file_1(const char *path,
-			     const unsigned char *sha1,
-			     unsigned long *size)
+static void *map_sha1_file_1(struct raw_object_store *o, const char *path,
+			     const unsigned char *sha1, unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -944,7 +943,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(&the_repository->objects, sha1, &path);
+		fd = open_sha1_file(o, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
@@ -965,7 +964,7 @@ static void *map_sha1_file_1(const char *path,
 
 void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(NULL, sha1, size);
+	return map_sha1_file_1(&the_repository->objects, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -2195,7 +2194,7 @@ int read_loose_object(const char *path,
 
 	*contents = NULL;
 
-	map = map_sha1_file_1(path, NULL, &mapsize);
+	map = map_sha1_file_1(&the_repository->objects, path, NULL, &mapsize);
 	if (!map) {
 		error_errno("unable to mmap %s", path);
 		goto out;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 15/16] sha1_file: allow map_sha1_file to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (13 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 14/16] sha1_file: allow map_sha1_file_1 " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 17:46     ` [PATCH 16/16] sha1_file: allow sha1_loose_object_info " Stefan Beller
                       ` (2 subsequent siblings)
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 cache.h        | 1 -
 object-store.h | 2 ++
 sha1_file.c    | 7 ++++---
 streaming.c    | 5 ++++-
 4 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/cache.h b/cache.h
index 2495c7081d..df29ee72e9 100644
--- a/cache.h
+++ b/cache.h
@@ -1242,7 +1242,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c
 extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
 
diff --git a/object-store.h b/object-store.h
index 5f6beb9d91..30b4cfdbf5 100644
--- a/object-store.h
+++ b/object-store.h
@@ -73,6 +73,8 @@ extern struct packed_git {
  */
 void sha1_file_name(struct raw_object_store *o, struct strbuf *buf, const unsigned char *sha1);
 
+void *map_sha1_file(struct raw_object_store *o, const unsigned char *sha1, unsigned long *size);
+
 void prepare_alt_odb(struct raw_object_store *o);
 
 #endif /* OBJECT_STORE_H */
diff --git a/sha1_file.c b/sha1_file.c
index aeb925a895..4cbff471a2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -962,9 +962,10 @@ static void *map_sha1_file_1(struct raw_object_store *o, const char *path,
 	return map;
 }
 
-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file(struct raw_object_store *o,
+		    const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(&the_repository->objects, NULL, sha1, size);
+	return map_sha1_file_1(o, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -1186,7 +1187,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 		return 0;
 	}
 
-	map = map_sha1_file(sha1, &mapsize);
+	map = map_sha1_file(&the_repository->objects, sha1, &mapsize);
 	if (!map)
 		return -1;
 
diff --git a/streaming.c b/streaming.c
index 5892b50bd8..6888448b12 100644
--- a/streaming.c
+++ b/streaming.c
@@ -3,6 +3,8 @@
  */
 #include "cache.h"
 #include "streaming.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 enum input_source {
@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {
 
 static open_method_decl(loose)
 {
-	st->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);
+	st->u.loose.mapped = map_sha1_file(&the_repository->objects,
+					   sha1, &st->u.loose.mapsize);
 	if (!st->u.loose.mapped)
 		return -1;
 	if ((unpack_sha1_header(&st->z,
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 16/16] sha1_file: allow sha1_loose_object_info to handle arbitrary object stores
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (14 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 15/16] sha1_file: allow map_sha1_file " Stefan Beller
@ 2018-02-16 17:46     ` " Stefan Beller
  2018-02-16 22:34     ` [PATCHv2 00/16] Moving global state into the repository object (part 1) Jonathan Nieder
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
  17 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-16 17:46 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, pclouds, sbeller, sunshine

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 4cbff471a2..31be57249f 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1154,7 +1154,8 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-static int sha1_loose_object_info(const unsigned char *sha1,
+static int sha1_loose_object_info(struct raw_object_store *o,
+				  const unsigned char *sha1,
 				  struct object_info *oi,
 				  int flags)
 {
@@ -1180,14 +1181,14 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(&the_repository->objects, sha1, &st, &path) < 0)
+		if (stat_sha1_file(o, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(&the_repository->objects, sha1, &mapsize);
+	map = map_sha1_file(o, sha1, &mapsize);
 	if (!map)
 		return -1;
 
@@ -1275,7 +1276,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			break;
 
 		/* Most likely it's a loose object. */
-		if (!sha1_loose_object_info(real, oi, flags))
+		if (!sha1_loose_object_info(&the_repository->objects, real, oi, flags))
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCH 04/16] object-store: move packed_git and packed_git_mru to object store
  2018-02-16 17:46     ` [PATCH 04/16] object-store: move packed_git and packed_git_mru to object store Stefan Beller
@ 2018-02-16 20:27       ` Junio C Hamano
  0 siblings, 0 replies; 100+ messages in thread
From: Junio C Hamano @ 2018-02-16 20:27 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Stefan Beller <sbeller@google.com> writes:

>   @@ @@
>   - packed_git_mru
>   + the_repository->objects.packed_git_mru

Regarding this...

> diff --git a/object-store.h b/object-store.h
> index a3f0d6ac15..024ccc91e9 100644
> --- a/object-store.h
> +++ b/object-store.h
> @@ -2,6 +2,7 @@
>  #define OBJECT_STORE_H
>  
>  #include "cache.h"
> +#include "list.h"
>  
>  struct raw_object_store {
> ... 
> +	struct packed_git *packed_git;
> +	/*
> +	 * A most-recently-used ordered version of the packed_git list, which can
> +	 * be iterated instead of packed_git (and marked via mru_mark).
> +	 */
> +	struct list_head packed_git_mru;
> +
>  	struct alternate_object_database *alt_odb_list;
>  	struct alternate_object_database **alt_odb_tail;
>  
>  	unsigned ignore_env : 1;
>  };
> -#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL, 0 }
> +
> +#define MRU_LIST_INIT {NULL, NULL}
> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0 }
> ...
> diff --git a/packfile.c b/packfile.c
> index 216ea836ee..d41e4c83d0 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -7,6 +7,7 @@
> -...
> -LIST_HEAD(packed_git_mru);

Given that the definition of LIST_HEAD() is

    /* Define a variable with the head and tail of the list. */
    #define LIST_HEAD(name) \
            struct list_head name = { &(name), &(name) }

doesn't the updated definition of RAW_OBJECT_STORE_INIT look fishy?


^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (15 preceding siblings ...)
  2018-02-16 17:46     ` [PATCH 16/16] sha1_file: allow sha1_loose_object_info " Stefan Beller
@ 2018-02-16 22:34     ` Jonathan Nieder
  2018-02-20 18:55       ` Stefan Beller
  2018-02-20 19:03       ` Junio C Hamano
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
  17 siblings, 2 replies; 100+ messages in thread
From: Jonathan Nieder @ 2018-02-16 22:34 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, jonathantanmy, pclouds, sunshine

Hi,

Stefan Beller wrote:

> v2:
> * duplicated the 'ignore_env' bit into the object store as well
> * the #define trick no longer works as we do not have a "the_objectstore" global,
>   which means there is just one patch per function that is converted.
>   As this follows the same structure of the previous series, I am still confident
>   there is no hidden dependencies to globals outside the object store in these
>   converted functions.
> * rebased on top of current master, resolving the merge conflicts.
>   I think I used the list.h APIs right, but please double check.

For what it's worth, I think I prefer v1.  I put some comments on why
on patch 0 of v1 and would be interested in your thoughts on them
(e.g. as a reply to that).  I also think that even if we want to
switch to a style that passes around object_store separately from
repository, it is easier to do the migration in two steps: first get
rid of hidden dependencies on the_repository, then do the (simpler)
automatic migration from

 f(the_repository)

to

 f(the_repository->object_store)

*afterwards*.

Thoughts?

Thanks,
Jonathan

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-16 22:34     ` [PATCHv2 00/16] Moving global state into the repository object (part 1) Jonathan Nieder
@ 2018-02-20 18:55       ` Stefan Beller
  2018-02-20 19:00         ` Brandon Williams
  2018-02-20 19:03       ` Junio C Hamano
  1 sibling, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-20 18:55 UTC (permalink / raw)
  To: Jonathan Nieder; +Cc: Junio C Hamano, git, Jonathan Tan, Duy Nguyen, Eric Sunshine

On Fri, Feb 16, 2018 at 2:34 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi,
>
> Stefan Beller wrote:
>
>> v2:
>> * duplicated the 'ignore_env' bit into the object store as well
>> * the #define trick no longer works as we do not have a "the_objectstore" global,
>>   which means there is just one patch per function that is converted.
>>   As this follows the same structure of the previous series, I am still confident
>>   there is no hidden dependencies to globals outside the object store in these
>>   converted functions.
>> * rebased on top of current master, resolving the merge conflicts.
>>   I think I used the list.h APIs right, but please double check.
>
> For what it's worth, I think I prefer v1.  I put some comments on why
> on patch 0 of v1 and would be interested in your thoughts on them
> (e.g. as a reply to that).  I also think that even if we want to
> switch to a style that passes around object_store separately from
> repository, it is easier to do the migration in two steps: first get
> rid of hidden dependencies on the_repository, then do the (simpler)
> automatic migration from
>
>  f(the_repository)
>
> to
>
>  f(the_repository->object_store)
>
> *afterwards*.
>
> Thoughts?

I would prefer to not spend more time on these conversions.
If Duy and you would come to a conclusion to either pick this
or the previous version I would be happy.

I do not see the benefit in splitting up the series even further and
do this multistep f(repo) -> f(object store), as the cost in potential
merge conflicts is too high. Note that brian just sent another object
id conversion series, also touching sha1_file.c, which I am sure will
produce merge conflicts for Junio.

For the next part 2 and onwards I'd be happy to take either this
strategy or Duys strategy as requested.

Stefan

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-20 18:55       ` Stefan Beller
@ 2018-02-20 19:00         ` Brandon Williams
  2018-02-20 19:03           ` Stefan Beller
  0 siblings, 1 reply; 100+ messages in thread
From: Brandon Williams @ 2018-02-20 19:00 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Jonathan Nieder, Junio C Hamano, git, Jonathan Tan, Duy Nguyen, Eric Sunshine

On 02/20, Stefan Beller wrote:
> On Fri, Feb 16, 2018 at 2:34 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> > Hi,
> >
> > Stefan Beller wrote:
> >
> >> v2:
> >> * duplicated the 'ignore_env' bit into the object store as well
> >> * the #define trick no longer works as we do not have a "the_objectstore" global,
> >>   which means there is just one patch per function that is converted.
> >>   As this follows the same structure of the previous series, I am still confident
> >>   there is no hidden dependencies to globals outside the object store in these
> >>   converted functions.
> >> * rebased on top of current master, resolving the merge conflicts.
> >>   I think I used the list.h APIs right, but please double check.
> >
> > For what it's worth, I think I prefer v1.  I put some comments on why
> > on patch 0 of v1 and would be interested in your thoughts on them
> > (e.g. as a reply to that).  I also think that even if we want to
> > switch to a style that passes around object_store separately from
> > repository, it is easier to do the migration in two steps: first get
> > rid of hidden dependencies on the_repository, then do the (simpler)
> > automatic migration from
> >
> >  f(the_repository)
> >
> > to
> >
> >  f(the_repository->object_store)
> >
> > *afterwards*.
> >
> > Thoughts?
> 
> I would prefer to not spend more time on these conversions.
> If Duy and you would come to a conclusion to either pick this
> or the previous version I would be happy.
> 
> I do not see the benefit in splitting up the series even further and
> do this multistep f(repo) -> f(object store), as the cost in potential
> merge conflicts is too high. Note that brian just sent another object
> id conversion series, also touching sha1_file.c, which I am sure will
> produce merge conflicts for Junio.
> 
> For the next part 2 and onwards I'd be happy to take either this
> strategy or Duys strategy as requested.

I think Jonathan is trying to point out that converting to f(repo) maybe
easier than converting to f(repo->object_store) upfront which would make
it easier to write the patches (which most of them are already written)
and to review because you can use the #define trick to make some sort of
guarantees.

After we have successfully completed the migration to f(repo), then we
can revisit the subsystems which want to have a clearer abstraction
layer and make the jump to f(repo->object_store).

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-16 22:34     ` [PATCHv2 00/16] Moving global state into the repository object (part 1) Jonathan Nieder
  2018-02-20 18:55       ` Stefan Beller
@ 2018-02-20 19:03       ` Junio C Hamano
  2018-02-20 19:06         ` Stefan Beller
  1 sibling, 1 reply; 100+ messages in thread
From: Junio C Hamano @ 2018-02-20 19:03 UTC (permalink / raw)
  To: Jonathan Nieder; +Cc: Stefan Beller, git, jonathantanmy, pclouds, sunshine

Jonathan Nieder <jrnieder@gmail.com> writes:

> For what it's worth, I think I prefer v1.  I put some comments on why
> on patch 0 of v1 and would be interested in your thoughts on them
> (e.g. as a reply to that).  I also think that even if we want to
> switch to a style that passes around object_store separately from
> repository, it is easier to do the migration in two steps: first get
> rid of hidden dependencies on the_repository, then do the (simpler)
> automatic migration from
>
>  f(the_repository)
>
> to
>
>  f(the_repository->object_store)
>
> *afterwards*.
>
> Thoughts?

Are we envisioning the future in which one repository has more than
one object-store (I am counting an object store and its alternates
that are pointed by its $GIT_OBJECT_DIRECTORY/info/alternates as a
single logical "object store")?  If not, doing f(the_repository)
migration, stopping there without f(the_repository->object_store)
may perfectly be adequate, I would think.



^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-20 19:00         ` Brandon Williams
@ 2018-02-20 19:03           ` Stefan Beller
  0 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-20 19:03 UTC (permalink / raw)
  To: Brandon Williams; +Cc: Jonathan Nieder, Junio C Hamano, git, Jonathan Tan, Duy Nguyen, Eric Sunshine

On Tue, Feb 20, 2018 at 11:00 AM, Brandon Williams <bmwill@google.com> wrote:
> On 02/20, Stefan Beller wrote:
>> On Fri, Feb 16, 2018 at 2:34 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> > Hi,
>> >
>> > Stefan Beller wrote:
>> >
>> >> v2:
>> >> * duplicated the 'ignore_env' bit into the object store as well
>> >> * the #define trick no longer works as we do not have a "the_objectstore" global,
>> >>   which means there is just one patch per function that is converted.
>> >>   As this follows the same structure of the previous series, I am still confident
>> >>   there is no hidden dependencies to globals outside the object store in these
>> >>   converted functions.
>> >> * rebased on top of current master, resolving the merge conflicts.
>> >>   I think I used the list.h APIs right, but please double check.
>> >
>> > For what it's worth, I think I prefer v1.  I put some comments on why
>> > on patch 0 of v1 and would be interested in your thoughts on them
>> > (e.g. as a reply to that).  I also think that even if we want to
>> > switch to a style that passes around object_store separately from
>> > repository, it is easier to do the migration in two steps: first get
>> > rid of hidden dependencies on the_repository, then do the (simpler)
>> > automatic migration from
>> >
>> >  f(the_repository)
>> >
>> > to
>> >
>> >  f(the_repository->object_store)
>> >
>> > *afterwards*.
>> >
>> > Thoughts?
>>
>> I would prefer to not spend more time on these conversions.
>> If Duy and you would come to a conclusion to either pick this
>> or the previous version I would be happy.
>>
>> I do not see the benefit in splitting up the series even further and
>> do this multistep f(repo) -> f(object store), as the cost in potential
>> merge conflicts is too high. Note that brian just sent another object
>> id conversion series, also touching sha1_file.c, which I am sure will
>> produce merge conflicts for Junio.
>>
>> For the next part 2 and onwards I'd be happy to take either this
>> strategy or Duys strategy as requested.
>
> I think Jonathan is trying to point out that converting to f(repo) maybe
> easier than converting to f(repo->object_store) upfront

I agree.

> which would make
> it easier to write the patches (which most of them are already written)

true, but for this series we also have the conversion to f(object_store)
written already.

> and to review because you can use the #define trick to make some sort of
> guarantees.

That is true, so it would be a tradeoff between reviewers and maintainers time?

> After we have successfully completed the migration to f(repo), then we
> can revisit the subsystems which want to have a clearer abstraction
> layer and make the jump to f(repo->object_store).

I would think we can take this series as-is and then move on with making
f(repo) abstractions, eventually moving to f(specialized-subsystem) as those
patches are not written yet (neither direct conversions, nor the repo trick;
the patches I already have need adaption which takes enough time on its own.)


>
> --
> Brandon Williams

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-20 19:03       ` Junio C Hamano
@ 2018-02-20 19:06         ` Stefan Beller
  2018-02-20 19:55           ` Junio C Hamano
  0 siblings, 1 reply; 100+ messages in thread
From: Stefan Beller @ 2018-02-20 19:06 UTC (permalink / raw)
  To: Junio C Hamano, Christian Couder; +Cc: Jonathan Nieder, git, Jonathan Tan, Duy Nguyen, Eric Sunshine

On Tue, Feb 20, 2018 at 11:03 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> For what it's worth, I think I prefer v1.  I put some comments on why
>> on patch 0 of v1 and would be interested in your thoughts on them
>> (e.g. as a reply to that).  I also think that even if we want to
>> switch to a style that passes around object_store separately from
>> repository, it is easier to do the migration in two steps: first get
>> rid of hidden dependencies on the_repository, then do the (simpler)
>> automatic migration from
>>
>>  f(the_repository)
>>
>> to
>>
>>  f(the_repository->object_store)
>>
>> *afterwards*.
>>
>> Thoughts?
>
> Are we envisioning the future in which one repository has more than
> one object-store (I am counting an object store and its alternates
> that are pointed by its $GIT_OBJECT_DIRECTORY/info/alternates as a
> single logical "object store")?  If not, doing f(the_repository)
> migration, stopping there without f(the_repository->object_store)
> may perfectly be adequate, I would think.
>

I do not envision that, maybe Christian Couder does?
The partial clone world would be happy with just one object store
per repo, I would think.

The step to take an object store would just add expressiveness
to the code, which may help in understanding what part of the code is
related to what other part of the code, so it may be a readability gain
on its own?

^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-20 19:06         ` Stefan Beller
@ 2018-02-20 19:55           ` Junio C Hamano
  2018-02-20 20:16             ` Stefan Beller
  0 siblings, 1 reply; 100+ messages in thread
From: Junio C Hamano @ 2018-02-20 19:55 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Christian Couder, Jonathan Nieder, git, Jonathan Tan, Duy Nguyen, Eric Sunshine

Stefan Beller <sbeller@google.com> writes:

> The step to take an object store would just add expressiveness
> to the code, which may help in understanding what part of the code is
> related to what other part of the code, so it may be a readability gain
> on its own?

It certainly would allow you to have a standalone object store that
is not associated with *any* repository, but if we are not using
such a layout, I doubt it would be a readability gain to add excess
and unexercised expressiveness to the code.


^ permalink raw reply	[flat|nested] 100+ messages in thread

* Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)
  2018-02-20 19:55           ` Junio C Hamano
@ 2018-02-20 20:16             ` Stefan Beller
  0 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-20 20:16 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Christian Couder, Jonathan Nieder, git, Jonathan Tan, Duy Nguyen, Eric Sunshine

On Tue, Feb 20, 2018 at 11:55 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> The step to take an object store would just add expressiveness
>> to the code, which may help in understanding what part of the code is
>> related to what other part of the code, so it may be a readability gain
>> on its own?
>
> It certainly would allow you to have a standalone object store that
> is not associated with *any* repository, but if we are not using
> such a layout, I doubt it would be a readability gain to add excess
> and unexercised expressiveness to the code.

So you favor v1?
Duy seems to be ok with v1 too if there is consensus that it is best
(or rather "if it makes Stefan's life hell, it's not worth doing.")[1].

I will try to resend that v1 shortly[2], as the only actual concern about the
code was where one struct was defined[3]. All other discussion was
meta-level, which direction to go.

Thanks for the clarification!
Stefan

[1] https://public-inbox.org/git/CACsJy8CPKESE8atc_eWdNVknQYp9T6ebwKwCdzLHyaFKH2BnZA@mail.gmail.com/
[2] https://public-inbox.org/git/20180213012241.187007-1-sbeller@google.com/
[3] https://public-inbox.org/git/20180213185120.GA108526@google.com/

^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCHv3 00/27] Moving global state into the repository object (part 1)
  2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
                       ` (16 preceding siblings ...)
  2018-02-16 22:34     ` [PATCHv2 00/16] Moving global state into the repository object (part 1) Jonathan Nieder
@ 2018-02-21  1:54     ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 01/27] repository: introduce raw object store field Stefan Beller
                         ` (26 more replies)
  17 siblings, 27 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine

v3:
* reverted back to use the repository for most of the functions
  (including unduplicating 'ignore_env')
* rebased on master again (I lost that state when doing v2, as
  I did both rebase as well as conversion to object store in one go)
* one additional patch, that moves the alternates related things to
  object-store.h, such that inclusion of cache.h (in object-store.h)
  is not required any more.

v2:
* duplicated the 'ignore_env' bit into the object store as well
* the #define trick no longer works as we do not have a "the_objectstore" global,
  which means there is just one patch per function that is converted.
  As this follows the same structure of the previous series, I am still confident
  there is no hidden dependencies to globals outside the object store in these
  converted functions.
* rebased on top of current master, resolving the merge conflicts.
  I think I used the list.h APIs right, but please double check.

Thanks,
Stefan

v1:
This is a real take on the first part of the recent RFC[1].

Jonathan Tan suggested[2] that "sha1_loose_object_info to handle arbitrary repositories"
might be a good breaking point for a first part at that RFC at patch 38.
This series is smaller and contains only 26 patches as the patches in the big
RFC were slightly out of order.

I developed this series partly by writing patches, but mostly by cherrypicking
from that RFC on top of current master. I noticed no external conflicts apart
from one addition to the repositories _INIT macro, which was easy to resolve.

Comments in the early range of that RFC were on 003 where Junio pointed out
that the coccinelle patch ought to be not in contrib/coccinelle, so I put it
in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.

brian had a questoin on patch 25 in the RFC, but that seemed to resolve itself
without any suggestion to include into this series[3].

Duy suggested that we shall not use the repository blindly, but should carefully
examine whether to pass on an object store or the refstore or such[4], which
I agree with if it makes sense. This series unfortunately has an issue with that
as I would not want to pass down the `ignore_env` flag separately from the object
store, so I made all functions that only take the object store to have the raw
object store as the first parameter, and others using the full repository.

Eric Sunshine brought up memory leaks with the RFC, and I would think to
have plugged all holes.

[1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/
[2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/
[3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/
[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/

Thanks,
Stefan

Jonathan Nieder (2):
  sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
  sha1_file: allow sha1_loose_object_info to handle arbitrary
    repositories

Stefan Beller (25):
  repository: introduce raw object store field
  object-store: migrate alternates struct and functions from cache.h
  object-store: move alt_odb_list and alt_odb_tail to object store
  object-store: free alt_odb_list
  object-store: move packed_git and packed_git_mru to object store
  object-store: close all packs upon clearing the object store
  pack: move prepare_packed_git_run_once to object store
  pack: move approximate object count to object store
  sha1_file: add raw_object_store argument to alt_odb_usable
  sha1_file: add repository argument to link_alt_odb_entry
  sha1_file: add repository argument to read_info_alternates
  sha1_file: add repository argument to link_alt_odb_entries
  sha1_file: add repository argument to prepare_alt_odb
  sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
  sha1_file: allow prepare_alt_odb to handle arbitrary repositories
  sha1_file: add repository argument to sha1_file_name
  sha1_file: add repository argument to stat_sha1_file
  sha1_file: add repository argument to open_sha1_file
  sha1_file: add repository argument to map_sha1_file_1
  sha1_file: add repository argument to map_sha1_file
  sha1_file: add repository argument to sha1_loose_object_info
  sha1_file: allow sha1_file_name to handle arbitrary repositories
  sha1_file: allow stat_sha1_file to handle arbitrary repositories
  sha1_file: allow open_sha1_file to handle arbitrary repositories
  sha1_file: allow map_sha1_file to handle arbitrary repositories

 builtin/am.c             |   2 +-
 builtin/clone.c          |   2 +-
 builtin/count-objects.c  |   6 +-
 builtin/fetch.c          |   2 +-
 builtin/fsck.c           |  13 ++--
 builtin/gc.c             |   4 +-
 builtin/grep.c           |   2 +-
 builtin/index-pack.c     |   1 +
 builtin/merge.c          |   2 +-
 builtin/pack-objects.c   |  20 +++---
 builtin/pack-redundant.c |   6 +-
 builtin/receive-pack.c   |   3 +-
 cache.h                  |  87 -------------------------
 environment.c            |   5 +-
 fast-import.c            |   6 +-
 http-backend.c           |   6 +-
 http-push.c              |   1 +
 http-walker.c            |   4 +-
 http.c                   |   6 +-
 object-store.h           | 134 +++++++++++++++++++++++++++++++++++++++
 object.c                 |  27 ++++++++
 pack-bitmap.c            |   4 +-
 pack-check.c             |   1 +
 pack-revindex.c          |   1 +
 packfile.c               |  64 +++++++++----------
 packfile.h               |   2 +-
 path.c                   |   2 +-
 reachable.c              |   1 +
 repository.c             |  19 ++++--
 repository.h             |   7 +-
 server-info.c            |   6 +-
 sha1_file.c              | 134 +++++++++++++++++++++------------------
 sha1_name.c              |  11 ++--
 streaming.c              |   5 +-
 34 files changed, 365 insertions(+), 231 deletions(-)
 create mode 100644 object-store.h

-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 01/27] repository: introduce raw object store field
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 02/27] object-store: migrate alternates struct and functions from cache.h Stefan Beller
                         ` (25 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

The raw object store field will contain any objects needed for
access to objects in a given repository.

This patch introduces the raw object store and populates it with the
`objectdir`, which used to be part of the repository struct.

As the struct gains members, we'll also populate the function to clear
the memory for these members.

In a later we'll introduce a struct object_parser, that will complement
the object parsing in a repository struct: The raw object parser is the
layer that will provide access to raw object content, while the higher
level object parser code will parse raw objects and keeps track of
parenthood and other object relationships using 'struct object'.
For now only add the lower level to the repository struct.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/grep.c |  2 +-
 environment.c  |  5 +++--
 object-store.h | 15 +++++++++++++++
 object.c       |  5 +++++
 path.c         |  2 +-
 repository.c   | 17 +++++++++++++----
 repository.h   |  7 ++++---
 7 files changed, 42 insertions(+), 11 deletions(-)
 create mode 100644 object-store.h

diff --git a/builtin/grep.c b/builtin/grep.c
index 3ca4ac80d8..0f0c195705 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 	 * object.
 	 */
 	grep_read_lock();
-	add_to_alternates_memory(submodule.objectdir);
+	add_to_alternates_memory(submodule.objects.objectdir);
 	grep_read_unlock();
 
 	if (oid) {
diff --git a/environment.c b/environment.c
index de8431e01e..ec10b062e6 100644
--- a/environment.c
+++ b/environment.c
@@ -13,6 +13,7 @@
 #include "refs.h"
 #include "fmt-merge-msg.h"
 #include "commit.h"
+#include "object-store.h"
 
 int trust_executable_bit = 1;
 int trust_ctime = 1;
@@ -244,9 +245,9 @@ const char *get_git_work_tree(void)
 
 char *get_object_directory(void)
 {
-	if (!the_repository->objectdir)
+	if (!the_repository->objects.objectdir)
 		BUG("git environment hasn't been setup");
-	return the_repository->objectdir;
+	return the_repository->objects.objectdir;
 }
 
 int odb_mkstemp(struct strbuf *template, const char *pattern)
diff --git a/object-store.h b/object-store.h
new file mode 100644
index 0000000000..cf35760ceb
--- /dev/null
+++ b/object-store.h
@@ -0,0 +1,15 @@
+#ifndef OBJECT_STORE_H
+#define OBJECT_STORE_H
+
+struct raw_object_store {
+	/*
+	 * Path to the repository's object store.
+	 * Cannot be NULL after initialization.
+	 */
+	char *objectdir;
+};
+#define RAW_OBJECT_STORE_INIT { NULL }
+
+void raw_object_store_clear(struct raw_object_store *o);
+
+#endif /* OBJECT_STORE_H */
diff --git a/object.c b/object.c
index 9e6f9ff20b..11d904c033 100644
--- a/object.c
+++ b/object.c
@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)
 			obj->flags &= ~flags;
 	}
 }
+
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	free(o->objectdir);
+}
diff --git a/path.c b/path.c
index da8b655730..81a42d9115 100644
--- a/path.c
+++ b/path.c
@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,
 		strbuf_splice(buf, 0, buf->len,
 			      repo->index_file, strlen(repo->index_file));
 	else if (dir_prefix(base, "objects"))
-		replace_dir(buf, git_dir_len + 7, repo->objectdir);
+		replace_dir(buf, git_dir_len + 7, repo->objects.objectdir);
 	else if (git_hooks_path && dir_prefix(base, "hooks"))
 		replace_dir(buf, git_dir_len + 5, git_hooks_path);
 	else if (repo->different_commondir)
diff --git a/repository.c b/repository.c
index 4ffbe9bc94..2255ff657e 100644
--- a/repository.c
+++ b/repository.c
@@ -1,11 +1,18 @@
 #include "cache.h"
 #include "repository.h"
+#include "object-store.h"
 #include "config.h"
 #include "submodule-config.h"
 
 /* The main repository */
 static struct repository the_repo = {
-	NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0
+	NULL, NULL,
+	RAW_OBJECT_STORE_INIT,
+	NULL, NULL, NULL,
+	NULL, NULL, NULL,
+	&the_index,
+	&hash_algos[GIT_HASH_SHA1],
+	0, 0
 };
 struct repository *the_repository = &the_repo;
 
@@ -42,8 +49,8 @@ static void repo_setup_env(struct repository *repo)
 						    !repo->ignore_env);
 	free(repo->commondir);
 	repo->commondir = strbuf_detach(&sb, NULL);
-	free(repo->objectdir);
-	repo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
+	free(repo->objects.objectdir);
+	repo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,
 					    "objects", !repo->ignore_env);
 	free(repo->graft_file);
 	repo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,
@@ -209,12 +216,14 @@ void repo_clear(struct repository *repo)
 {
 	FREE_AND_NULL(repo->gitdir);
 	FREE_AND_NULL(repo->commondir);
-	FREE_AND_NULL(repo->objectdir);
 	FREE_AND_NULL(repo->graft_file);
 	FREE_AND_NULL(repo->index_file);
 	FREE_AND_NULL(repo->worktree);
 	FREE_AND_NULL(repo->submodule_prefix);
 
+	raw_object_store_clear(&repo->objects);
+	memset(&repo->objects, 0, sizeof(repo->objects));
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
diff --git a/repository.h b/repository.h
index 0329e40c7f..1f8bc7a7cf 100644
--- a/repository.h
+++ b/repository.h
@@ -1,6 +1,8 @@
 #ifndef REPOSITORY_H
 #define REPOSITORY_H
 
+#include "object-store.h"
+
 struct config_set;
 struct index_state;
 struct submodule_cache;
@@ -21,10 +23,9 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Path to the repository's object store.
-	 * Cannot be NULL after initialization.
+	 * Holds any information related to the object store.
 	 */
-	char *objectdir;
+	struct raw_object_store objects;
 
 	/*
 	 * Path to the repository's graft file.
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 02/27] object-store: migrate alternates struct and functions from cache.h
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
  2018-02-21  1:54       ` [PATCH 01/27] repository: introduce raw object store field Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 03/27] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
                         ` (24 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine

Migrate the struct alternate_object_database and all its related
functions to the object store as these functions are easier found in
that header. The migration is just a verbatim copy, no need to
include the object store header at any C file, because cache.h includes
repository.h which in turn includes the object-store.h

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h        | 51 --------------------------------------------------
 object-store.h | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 51 deletions(-)

diff --git a/cache.h b/cache.h
index 9cac7bb518..964873faf2 100644
--- a/cache.h
+++ b/cache.h
@@ -1576,57 +1576,6 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);
 extern void schedule_dir_for_removal(const char *name, int len);
 extern void remove_scheduled_dirs(void);
 
-extern struct alternate_object_database {
-	struct alternate_object_database *next;
-
-	/* see alt_scratch_buf() */
-	struct strbuf scratch;
-	size_t base_len;
-
-	/*
-	 * Used to store the results of readdir(3) calls when searching
-	 * for unique abbreviated hashes.  This cache is never
-	 * invalidated, thus it's racy and not necessarily accurate.
-	 * That's fine for its purpose; don't use it for tasks requiring
-	 * greater accuracy!
-	 */
-	char loose_objects_subdir_seen[256];
-	struct oid_array loose_objects_cache;
-
-	char path[FLEX_ARRAY];
-} *alt_odb_list;
-extern void prepare_alt_odb(void);
-extern char *compute_alternate_path(const char *path, struct strbuf *err);
-typedef int alt_odb_fn(struct alternate_object_database *, void *);
-extern int foreach_alt_odb(alt_odb_fn, void*);
-
-/*
- * Allocate a "struct alternate_object_database" but do _not_ actually
- * add it to the list of alternates.
- */
-struct alternate_object_database *alloc_alt_odb(const char *dir);
-
-/*
- * Add the directory to the on-disk alternates file; the new entry will also
- * take effect in the current process.
- */
-extern void add_to_alternates_file(const char *dir);
-
-/*
- * Add the directory to the in-memory list of alternates (along with any
- * recursive alternates it points to), but do not modify the on-disk alternates
- * file.
- */
-extern void add_to_alternates_memory(const char *dir);
-
-/*
- * Returns a scratch strbuf pre-filled with the alternate object directory,
- * including a trailing slash, which can be used to access paths in the
- * alternate. Always use this over direct access to alt->scratch, as it
- * cleans up any previous use of the scratch buffer.
- */
-extern struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
-
 struct pack_window {
 	struct pack_window *next;
 	unsigned char *base;
diff --git a/object-store.h b/object-store.h
index cf35760ceb..5678aa1136 100644
--- a/object-store.h
+++ b/object-store.h
@@ -1,6 +1,57 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
+extern struct alternate_object_database {
+	struct alternate_object_database *next;
+
+	/* see alt_scratch_buf() */
+	struct strbuf scratch;
+	size_t base_len;
+
+	/*
+	 * Used to store the results of readdir(3) calls when searching
+	 * for unique abbreviated hashes.  This cache is never
+	 * invalidated, thus it's racy and not necessarily accurate.
+	 * That's fine for its purpose; don't use it for tasks requiring
+	 * greater accuracy!
+	 */
+	char loose_objects_subdir_seen[256];
+	struct oid_array loose_objects_cache;
+
+	char path[FLEX_ARRAY];
+} *alt_odb_list;
+void prepare_alt_odb(void);
+char *compute_alternate_path(const char *path, struct strbuf *err);
+typedef int alt_odb_fn(struct alternate_object_database *, void *);
+int foreach_alt_odb(alt_odb_fn, void*);
+
+/*
+ * Allocate a "struct alternate_object_database" but do _not_ actually
+ * add it to the list of alternates.
+ */
+struct alternate_object_database *alloc_alt_odb(const char *dir);
+
+/*
+ * Add the directory to the on-disk alternates file; the new entry will also
+ * take effect in the current process.
+ */
+void add_to_alternates_file(const char *dir);
+
+/*
+ * Add the directory to the in-memory list of alternates (along with any
+ * recursive alternates it points to), but do not modify the on-disk alternates
+ * file.
+ */
+void add_to_alternates_memory(const char *dir);
+
+/*
+ * Returns a scratch strbuf pre-filled with the alternate object directory,
+ * including a trailing slash, which can be used to access paths in the
+ * alternate. Always use this over direct access to alt->scratch, as it
+ * cleans up any previous use of the scratch buffer.
+ */
+struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 03/27] object-store: move alt_odb_list and alt_odb_tail to object store
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
  2018-02-21  1:54       ` [PATCH 01/27] repository: introduce raw object store field Stefan Beller
  2018-02-21  1:54       ` [PATCH 02/27] object-store: migrate alternates struct and functions from cache.h Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 04/27] object-store: free alt_odb_list Stefan Beller
                         ` (23 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

In a process with multiple repositories open, alternates should be
associated to a single repository and not shared globally. Move
alt_odb_list and alt_odb_tail into the_repository and adjust callers
to reflect this.

Now that the alternative object data base is per repository, we're
leaking its memory upon freeing a repository. The next patch plugs
this hole.

No functional change intended.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/fsck.c |  4 +++-
 object-store.h |  9 ++++++---
 packfile.c     |  3 ++-
 sha1_file.c    | 25 ++++++++++++-------------
 sha1_name.c    |  3 ++-
 5 files changed, 25 insertions(+), 19 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index 7a8a679d4f..908e4f092a 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "commit.h"
 #include "tree.h"
@@ -716,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 		fsck_object_dir(get_object_directory());
 
 		prepare_alt_odb();
-		for (alt = alt_odb_list; alt; alt = alt->next)
+		for (alt = the_repository->objects.alt_odb_list;
+				alt; alt = alt->next)
 			fsck_object_dir(alt->path);
 
 		if (check_full) {
diff --git a/object-store.h b/object-store.h
index 5678aa1136..e78eea1dde 100644
--- a/object-store.h
+++ b/object-store.h
@@ -1,7 +1,7 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
-extern struct alternate_object_database {
+struct alternate_object_database {
 	struct alternate_object_database *next;
 
 	/* see alt_scratch_buf() */
@@ -19,7 +19,7 @@ extern struct alternate_object_database {
 	struct oid_array loose_objects_cache;
 
 	char path[FLEX_ARRAY];
-} *alt_odb_list;
+};
 void prepare_alt_odb(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
@@ -58,8 +58,11 @@ struct raw_object_store {
 	 * Cannot be NULL after initialization.
 	 */
 	char *objectdir;
+
+	struct alternate_object_database *alt_odb_list;
+	struct alternate_object_database **alt_odb_tail;
 };
-#define RAW_OBJECT_STORE_INIT { NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index 7dbe8739d1..216ea836ee 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "list.h"
 #include "pack.h"
+#include "repository.h"
 #include "dir.h"
 #include "mergesort.h"
 #include "packfile.h"
@@ -891,7 +892,7 @@ void prepare_packed_git(void)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next)
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
diff --git a/sha1_file.c b/sha1_file.c
index 831d9e7343..1348dce68f 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -22,6 +22,7 @@
 #include "pack-revindex.h"
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
+#include "repository.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -342,9 +343,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 	return buf->buf;
 }
 
-struct alternate_object_database *alt_odb_list;
-static struct alternate_object_database **alt_odb_tail;
-
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
@@ -364,7 +362,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -424,8 +422,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*alt_odb_tail = ent;
-	alt_odb_tail = &(ent->next);
+	*the_repository->objects.alt_odb_tail = ent;
+	the_repository->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
@@ -559,7 +557,7 @@ void add_to_alternates_file(const char *reference)
 		fprintf_or_die(out, "%s\n", reference);
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
-		if (alt_odb_tail)
+		if (the_repository->objects.alt_odb_tail)
 			link_alt_odb_entries(reference, '\n', NULL, 0);
 	}
 	free(alts);
@@ -657,7 +655,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	int r = 0;
 
 	prepare_alt_odb();
-	for (ent = alt_odb_list; ent; ent = ent->next) {
+	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
 			break;
@@ -669,12 +667,13 @@ void prepare_alt_odb(void)
 {
 	const char *alt;
 
-	if (alt_odb_tail)
+	if (the_repository->objects.alt_odb_tail)
 		return;
 
 	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
 
-	alt_odb_tail = &alt_odb_list;
+	the_repository->objects.alt_odb_tail =
+			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
 	read_info_alternates(get_object_directory(), 0);
@@ -718,7 +717,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
 			return 1;
@@ -878,7 +877,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 
 	prepare_alt_odb();
 	errno = ENOENT;
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
@@ -908,7 +907,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	most_interesting_errno = errno;
 
 	prepare_alt_odb();
-	for (alt = alt_odb_list; alt; alt = alt->next) {
+	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
diff --git a/sha1_name.c b/sha1_name.c
index 611c7d24dd..957ce25680 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -10,6 +10,7 @@
 #include "dir.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "repository.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)
 		 */
 		fakeent = alloc_alt_odb(get_object_directory());
 	}
-	fakeent->next = alt_odb_list;
+	fakeent->next = the_repository->objects.alt_odb_list;
 
 	for (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {
 		int pos;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 04/27] object-store: free alt_odb_list
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (2 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 03/27] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store Stefan Beller
                         ` (22 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine

Free the memory and reset alt_odb_{list, tail} to NULL.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/object.c b/object.c
index 11d904c033..17b1da6d6b 100644
--- a/object.c
+++ b/object.c
@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)
 	}
 }
 
+static void free_alt_odb(struct alternate_object_database *alt)
+{
+	strbuf_release(&alt->scratch);
+	oid_array_clear(&alt->loose_objects_cache);
+}
+
+static void free_alt_odbs(struct raw_object_store *o)
+{
+	while (o->alt_odb_list) {
+		free_alt_odb(o->alt_odb_list);
+		o->alt_odb_list = o->alt_odb_list->next;
+	}
+}
+
 void raw_object_store_clear(struct raw_object_store *o)
 {
 	free(o->objectdir);
+
+	free_alt_odbs(o);
+	o->alt_odb_tail = NULL;
 }
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (3 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 04/27] object-store: free alt_odb_list Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 06/27] object-store: close all packs upon clearing the " Stefan Beller
                         ` (21 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

In a process with multiple repositories open, packfile accessors
should be associated to a single repository and not shared globally.
Move packed_git and packed_git_mru into the_repository and adjust
callers to reflect this.

Patch generated by

 1. Moving the struct packed_git declaration to object-store.h
    and packed_git, packed_git_mru globals to struct object_store.

 2. Applying the semantic patch
    contrib/coccinelle/refactoring/packed_git.cocci to adjust callers.
    This semantic patch is placed in a sub directory of the coccinelle
    contrib dir, as this semantic patch is not expected to be of general
    usefulness; it is only useful during developing this series and
    merging it with other topics in flight. At a later date, just
    delete that semantic patch.

 3. Applying line wrapping fixes from "make style" to break the
    resulting long lines.

 4. Adding missing #includes of repository.h and object-store.h
    where needed.

 5. As the packfiles are now owned by an objectstore/repository, which
    is ephemeral unlike globals, we introduce memory leaks. So address
    them in raw_object_store_clear().

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/count-objects.c  |  6 ++++--
 builtin/fsck.c           |  7 +++++--
 builtin/gc.c             |  4 +++-
 builtin/index-pack.c     |  1 +
 builtin/pack-objects.c   | 20 +++++++++++---------
 builtin/pack-redundant.c |  6 ++++--
 builtin/receive-pack.c   |  1 +
 cache.h                  | 29 ----------------------------
 fast-import.c            |  6 ++++--
 http-backend.c           |  6 ++++--
 http-push.c              |  1 +
 http-walker.c            |  1 +
 http.c                   |  1 +
 object-store.h           | 41 +++++++++++++++++++++++++++++++++++++++-
 object.c                 |  7 +++++++
 pack-bitmap.c            |  4 +++-
 pack-check.c             |  1 +
 pack-revindex.c          |  1 +
 packfile.c               | 39 +++++++++++++++++++-------------------
 reachable.c              |  1 +
 repository.c             |  2 ++
 server-info.c            |  6 ++++--
 sha1_name.c              |  5 +++--
 23 files changed, 122 insertions(+), 74 deletions(-)

diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 33343818c8..9334648dcc 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -7,6 +7,8 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "repository.h"
+#include "object-store.h"
 #include "builtin.h"
 #include "parse-options.h"
 #include "quote.h"
@@ -120,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)
 		struct strbuf loose_buf = STRBUF_INIT;
 		struct strbuf pack_buf = STRBUF_INIT;
 		struct strbuf garbage_buf = STRBUF_INIT;
-		if (!packed_git)
+		if (!the_repository->objects.packed_git)
 			prepare_packed_git();
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local)
 				continue;
 			if (open_pack_index(p))
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 908e4f092a..2e99e02128 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
@@ -729,7 +730,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 			prepare_packed_git();
 
 			if (show_progress) {
-				for (p = packed_git; p; p = p->next) {
+				for (p = the_repository->objects.packed_git; p;
+				     p = p->next) {
 					if (open_pack_index(p))
 						continue;
 					total += p->num_objects;
@@ -737,7 +739,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 
 				progress = start_progress(_("Checking objects"), total);
 			}
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				/* verify gives error messages itself */
 				if (verify_pack(p, fsck_obj_buffer,
 						progress, count))
diff --git a/builtin/gc.c b/builtin/gc.c
index 77fa720bd0..96ff790867 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -11,6 +11,7 @@
  */
 
 #include "builtin.h"
+#include "repository.h"
 #include "config.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -18,6 +19,7 @@
 #include "run-command.h"
 #include "sigchain.h"
 #include "argv-array.h"
+#include "object-store.h"
 #include "commit.h"
 #include "packfile.h"
 
@@ -173,7 +175,7 @@ static int too_many_packs(void)
 		return 0;
 
 	prepare_packed_git();
-	for (cnt = 0, p = packed_git; p; p = p->next) {
+	for (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		if (p->pack_keep)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 5ebd370c56..e60cc3ed98 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -12,6 +12,7 @@
 #include "exec_cmd.h"
 #include "streaming.h"
 #include "thread-utils.h"
+#include "object-store.h"
 #include "packfile.h"
 
 static const char index_pack_usage[] =
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 83dcbc9773..7f376c2aef 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1,6 +1,8 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
+#include "object-store.h"
 #include "attr.h"
 #include "object.h"
 #include "blob.h"
@@ -1025,8 +1027,7 @@ static int want_object_in_pack(const struct object_id *oid,
 		if (want != -1)
 			return want;
 	}
-
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		off_t offset;
 
@@ -1044,7 +1045,7 @@ static int want_object_in_pack(const struct object_id *oid,
 			}
 			want = want_found_object(exclude, p);
 			if (!exclude && want > 0)
-				list_move(&p->mru, &packed_git_mru);
+				list_move(&p->mru, &the_repository->objects.packed_git_mru);
 			if (want != -1)
 				return want;
 		}
@@ -2673,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)
 
 	memset(&in_pack, 0, sizeof(in_pack));
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		struct object_id oid;
 		struct object *o;
 
@@ -2736,7 +2737,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 	static struct packed_git *last_found = (void *)1;
 	struct packed_git *p;
 
-	p = (last_found != (void *)1) ? last_found : packed_git;
+	p = (last_found != (void *)1) ? last_found :
+					the_repository->objects.packed_git;
 
 	while (p) {
 		if ((!p->pack_local || p->pack_keep) &&
@@ -2745,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)
 			return 1;
 		}
 		if (p == last_found)
-			p = packed_git;
+			p = the_repository->objects.packed_git;
 		else
 			p = p->next;
 		if (p == last_found)
@@ -2781,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)
 	uint32_t i;
 	struct object_id oid;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local || p->pack_keep)
 			continue;
 
@@ -3152,7 +3154,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	prepare_packed_git();
 	if (ignore_packed_keep) {
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next)
+		for (p = the_repository->objects.packed_git; p; p = p->next)
 			if (p->pack_local && p->pack_keep)
 				break;
 		if (!p) /* no keep-able packs found */
@@ -3165,7 +3167,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		 * also covers non-local objects
 		 */
 		struct packed_git *p;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (!p->pack_local) {
 				have_non_local_packs = 1;
 				break;
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index aaa8136322..55462bc826 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -7,6 +7,8 @@
 */
 
 #include "builtin.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 #define BLKSIZE 512
@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)
 
 static struct pack_list * add_pack_file(const char *filename)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	if (strlen(filename) < 40)
 		die("Bad pack filename: %s", filename);
@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)
 
 static void load_all(void)
 {
-	struct packed_git *p = packed_git;
+	struct packed_git *p = the_repository->objects.packed_git;
 
 	while (p) {
 		add_pack(p);
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b7ce7c7f52..b2eac79a6e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "sideband.h"
 #include "run-command.h"
 #include "exec_cmd.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "remote.h"
diff --git a/cache.h b/cache.h
index 964873faf2..3fd9dc52cd 100644
--- a/cache.h
+++ b/cache.h
@@ -1585,35 +1585,6 @@ struct pack_window {
 	unsigned int inuse_cnt;
 };
 
-extern struct packed_git {
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	uint32_t num_bad_objects;
-	unsigned char *bad_object_sha1;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1;
-	unsigned char sha1[20];
-	struct revindex_entry *revindex;
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-} *packed_git;
-
-/*
- * A most-recently-used ordered version of the packed_git list.
- */
-extern struct list_head packed_git_mru;
-
 struct pack_entry {
 	off_t offset;
 	unsigned char sha1[20];
diff --git a/fast-import.c b/fast-import.c
index b70ac025e0..1685fe59a2 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -154,8 +154,10 @@ Format of STDIN stream:
 
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "object.h"
 #include "blob.h"
 #include "tree.h"
@@ -1110,7 +1112,7 @@ static int store_object(
 	if (e->idx.offset) {
 		duplicate_count_by_type[type]++;
 		return 1;
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = type;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
@@ -1305,7 +1307,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		duplicate_count_by_type[OBJ_BLOB]++;
 		truncate_pack(&checkpoint);
 
-	} else if (find_sha1_pack(oid.hash, packed_git)) {
+	} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {
 		e->type = OBJ_BLOB;
 		e->pack_id = MAX_PACK_ID;
 		e->idx.offset = 1; /* just not zero! */
diff --git a/http-backend.c b/http-backend.c
index f3dc218b2a..4d93126c15 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -1,7 +1,9 @@
 #include "cache.h"
 #include "config.h"
+#include "repository.h"
 #include "refs.h"
 #include "pkt-line.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "exec_cmd.h"
@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)
 
 	select_getanyfile(hdr);
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			cnt++;
 	}
 
 	strbuf_grow(&buf, cnt * 53 + 2);
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_local)
 			strbuf_addf(&buf, "P %s\n", p->pack_name + objdirlen + 6);
 	}
diff --git a/http-push.c b/http-push.c
index 0913f8ab86..1ca662390f 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "blob.h"
diff --git a/http-walker.c b/http-walker.c
index 07c2b1af82..8bb5d991bb 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -4,6 +4,7 @@
 #include "http.h"
 #include "list.h"
 #include "transport.h"
+#include "object-store.h"
 #include "packfile.h"
 
 struct alt_base {
diff --git a/http.c b/http.c
index 31755023a4..a4a9e583c7 100644
--- a/http.c
+++ b/http.c
@@ -1,6 +1,7 @@
 #include "git-compat-util.h"
 #include "http.h"
 #include "config.h"
+#include "object-store.h"
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/object-store.h b/object-store.h
index e78eea1dde..1de9e07102 100644
--- a/object-store.h
+++ b/object-store.h
@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);
  */
 struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);
 
+struct packed_git {
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	uint32_t num_bad_objects;
+	unsigned char *bad_object_sha1;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1;
+	unsigned char sha1[20];
+	struct revindex_entry *revindex;
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
 struct raw_object_store {
 	/*
 	 * Path to the repository's object store.
@@ -59,10 +83,25 @@ struct raw_object_store {
 	 */
 	char *objectdir;
 
+	struct packed_git *packed_git;
+	/*
+	 * A most-recently-used ordered version of the packed_git list, which can
+	 * be iterated instead of packed_git (and marked via mru_mark).
+	 */
+	struct list_head packed_git_mru;
+
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 };
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }
+
+/*
+ * The mru list_head is supposed to be initialized using
+ * the LIST_HEAD macro, assigning prev/next to itself.
+ * However this doesn't work in this case as some compilers dislike
+ * that macro on member variables. Use NULL instead as that is defined
+ * and accepted, deferring the real init to prepare_packed_git_mru(). */
+#define __MRU_INIT { NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/object.c b/object.c
index 17b1da6d6b..c76b62572a 100644
--- a/object.c
+++ b/object.c
@@ -466,4 +466,11 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
+
+	while (!list_empty(&o->packed_git_mru))
+		list_del(&o->packed_git_mru);
+	/*
+	 * TODO: call close_all_packs once migrated to
+	 * take an object store argument
+	 */
 }
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 9270983e5f..d51172b1d5 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -5,11 +5,13 @@
 #include "revision.h"
 #include "progress.h"
 #include "list-objects.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-bitmap.h"
 #include "pack-revindex.h"
 #include "pack-objects.h"
 #include "packfile.h"
+#include "repository.h"
 
 /*
  * An entry on the bitmap index, representing the bitmap for a given
@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)
 	assert(!bitmap_git.map && !bitmap_git.loaded);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (open_pack_bitmap_1(p) == 0)
 			ret = 0;
 	}
diff --git a/pack-check.c b/pack-check.c
index 073c1fbd46..c80c3ed8fd 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack.h"
+#include "object-store.h"
 #include "pack-revindex.h"
 #include "progress.h"
 #include "packfile.h"
diff --git a/pack-revindex.c b/pack-revindex.c
index ff5f62c033..bb521cf7fb 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "pack-revindex.h"
+#include "object-store.h"
 
 /*
  * Pack index for existing packs give us easy access to the offsets into
diff --git a/packfile.c b/packfile.c
index 216ea836ee..d41e4c83d0 100644
--- a/packfile.c
+++ b/packfile.c
@@ -7,6 +7,7 @@
 #include "packfile.h"
 #include "delta.h"
 #include "list.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "sha1-lookup.h"
 #include "commit.h"
@@ -45,8 +46,6 @@ static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
 static size_t peak_pack_mapped;
 static size_t pack_mapped;
-struct packed_git *packed_git;
-LIST_HEAD(packed_git_mru);
 
 #define SZ_FMT PRIuMAX
 static inline uintmax_t sz_fmt(size_t s) { return s; }
@@ -246,7 +245,7 @@ static int unuse_one_window(struct packed_git *current)
 
 	if (current)
 		scan_windows(current, &lru_p, &lru_w, &lru_l);
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		scan_windows(p, &lru_p, &lru_w, &lru_l);
 	if (lru_p) {
 		munmap(lru_w->base, lru_w->len);
@@ -316,7 +315,7 @@ void close_all_packs(void)
 {
 	struct packed_git *p;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
@@ -384,7 +383,7 @@ static int close_one_pack(void)
 	struct pack_window *mru_w = NULL;
 	int accept_windows_inuse = 1;
 
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if (p->pack_fd == -1)
 			continue;
 		find_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);
@@ -686,8 +685,8 @@ void install_packed_git(struct packed_git *pack)
 	if (pack->pack_fd != -1)
 		pack_open_fds++;
 
-	pack->next = packed_git;
-	packed_git = pack;
+	pack->next = the_repository->objects.packed_git;
+	the_repository->objects.packed_git = pack;
 }
 
 void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -769,7 +768,8 @@ static void prepare_packed_git_one(char *objdir, int local)
 		base_len = path.len;
 		if (strip_suffix_mem(path.buf, &base_len, ".idx")) {
 			/* Don't reopen a pack we already have. */
-			for (p = packed_git; p; p = p->next) {
+			for (p = the_repository->objects.packed_git; p;
+			     p = p->next) {
 				size_t len;
 				if (strip_suffix(p->pack_name, ".pack", &len) &&
 				    len == base_len &&
@@ -820,7 +820,7 @@ unsigned long approximate_object_count(void)
 
 		prepare_packed_git();
 		count = 0;
-		for (p = packed_git; p; p = p->next) {
+		for (p = the_repository->objects.packed_git; p; p = p->next) {
 			if (open_pack_index(p))
 				continue;
 			count += p->num_objects;
@@ -869,18 +869,19 @@ static int sort_pack(const void *a_, const void *b_)
 
 static void rearrange_packed_git(void)
 {
-	packed_git = llist_mergesort(packed_git, get_next_packed_git,
-				     set_next_packed_git, sort_pack);
+	the_repository->objects.packed_git = llist_mergesort(
+		the_repository->objects.packed_git, get_next_packed_git,
+		set_next_packed_git, sort_pack);
 }
 
 static void prepare_packed_git_mru(void)
 {
 	struct packed_git *p;
 
-	INIT_LIST_HEAD(&packed_git_mru);
+	INIT_LIST_HEAD(&the_repository->objects.packed_git_mru);
 
-	for (p = packed_git; p; p = p->next)
-		list_add_tail(&p->mru, &packed_git_mru);
+	for (p = the_repository->objects.packed_git; p; p = p->next)
+		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
 static int prepare_packed_git_run_once = 0;
@@ -1014,7 +1015,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)
 	struct packed_git *p;
 	unsigned i;
 
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		for (i = 0; i < p->num_bad_objects; i++)
 			if (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))
 				return p;
@@ -1845,13 +1846,13 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)
 	struct list_head *pos;
 
 	prepare_packed_git();
-	if (!packed_git)
+	if (!the_repository->objects.packed_git)
 		return 0;
 
-	list_for_each(pos, &packed_git_mru) {
+	list_for_each(pos, &the_repository->objects.packed_git_mru) {
 		struct packed_git *p = list_entry(pos, struct packed_git, mru);
 		if (fill_pack_entry(sha1, e, p)) {
-			list_move(&p->mru, &packed_git_mru);
+			list_move(&p->mru, &the_repository->objects.packed_git_mru);
 			return 1;
 		}
 	}
@@ -1898,7 +1899,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 	int pack_errors = 0;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
diff --git a/reachable.c b/reachable.c
index 88d7d679da..59ed16ba76 100644
--- a/reachable.c
+++ b/reachable.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tag.h"
 #include "commit.h"
 #include "blob.h"
diff --git a/repository.c b/repository.c
index 2255ff657e..9798f4ffca 100644
--- a/repository.c
+++ b/repository.c
@@ -142,6 +142,8 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)
 
 	repo->ignore_env = 1;
 
+	INIT_LIST_HEAD(&repo->objects.packed_git_mru);
+
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
 
diff --git a/server-info.c b/server-info.c
index 26a6c20b7d..96885c45f6 100644
--- a/server-info.c
+++ b/server-info.c
@@ -1,5 +1,7 @@
 #include "cache.h"
+#include "repository.h"
 #include "refs.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "tag.h"
@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)
 	objdirlen = strlen(objdir);
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next) {
+	for (p = the_repository->objects.packed_git; p; p = p->next) {
 		/* we ignore things on alternate path since they are
 		 * not available to the pullers in general.
 		 */
@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)
 	}
 	num_pack = i;
 	info = xcalloc(num_pack, sizeof(struct pack_info *));
-	for (i = 0, p = packed_git; p; p = p->next) {
+	for (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {
 		if (!p->pack_local)
 			continue;
 		info[i] = xcalloc(1, sizeof(struct pack_info));
diff --git a/sha1_name.c b/sha1_name.c
index 957ce25680..016c883b5c 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p && !ds->ambiguous; p = p->next)
+	for (p = the_repository->objects.packed_git; p && !ds->ambiguous;
+	     p = p->next)
 		unique_in_pack(p, ds);
 }
 
@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)
 	struct packed_git *p;
 
 	prepare_packed_git();
-	for (p = packed_git; p; p = p->next)
+	for (p = the_repository->objects.packed_git; p; p = p->next)
 		find_abbrev_len_for_pack(p, mad);
 }
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 06/27] object-store: close all packs upon clearing the object store
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (4 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 07/27] pack: move prepare_packed_git_run_once to " Stefan Beller
                         ` (20 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/am.c           | 2 +-
 builtin/clone.c        | 2 +-
 builtin/fetch.c        | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 object.c               | 6 ++----
 packfile.c             | 4 ++--
 packfile.h             | 2 +-
 8 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 5bdd2d7578..4762a702e3 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)
 	 */
 	if (!state->rebasing) {
 		am_destroy(state);
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 	}
 }
diff --git a/builtin/clone.c b/builtin/clone.c
index 101c27a593..13cfaa6488 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	transport_disconnect(transport);
 
 	if (option_dissociate) {
-		close_all_packs();
+		close_all_packs(&the_repository->objects);
 		dissociate_from_references();
 	}
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 8ee998ea2e..4d72efca78 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 
 	string_list_clear(&list, 0);
 
-	close_all_packs();
+	close_all_packs(&the_repository->objects);
 
 	argv_array_pushl(&argv_gc_auto, "gc", "--auto", NULL);
 	if (verbosity < 0)
diff --git a/builtin/merge.c b/builtin/merge.c
index 30264cfd7c..907ae44ab5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,
 			 * We ignore errors in 'gc --auto', since the
 			 * user should see them.
 			 */
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
 		}
 	}
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b2eac79a6e..954fc72c7c 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 			proc.git_cmd = 1;
 			proc.argv = argv_gc_auto;
 
-			close_all_packs();
+			close_all_packs(&the_repository->objects);
 			if (!start_command(&proc)) {
 				if (use_sideband)
 					copy_to_sideband(proc.err, -1, NULL);
diff --git a/object.c b/object.c
index c76b62572a..34daaf37b3 100644
--- a/object.c
+++ b/object.c
@@ -4,6 +4,7 @@
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
+#include "packfile.h"
 
 static struct object **obj_hash;
 static int nr_objs, obj_hash_size;
@@ -469,8 +470,5 @@ void raw_object_store_clear(struct raw_object_store *o)
 
 	while (!list_empty(&o->packed_git_mru))
 		list_del(&o->packed_git_mru);
-	/*
-	 * TODO: call close_all_packs once migrated to
-	 * take an object store argument
-	 */
+	close_all_packs(o);
 }
diff --git a/packfile.c b/packfile.c
index d41e4c83d0..511a2b0cdf 100644
--- a/packfile.c
+++ b/packfile.c
@@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)
 	close_pack_index(p);
 }
 
-void close_all_packs(void)
+void close_all_packs(struct raw_object_store *o)
 {
 	struct packed_git *p;
 
-	for (p = the_repository->objects.packed_git; p; p = p->next)
+	for (p = o->packed_git; p; p = p->next)
 		if (p->do_not_close)
 			die("BUG: want to close pack marked 'do-not-close'");
 		else
diff --git a/packfile.h b/packfile.h
index a7fca598d6..6a2c57045c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
 extern void close_pack_windows(struct packed_git *);
-extern void close_all_packs(void);
+extern void close_all_packs(struct raw_object_store *o);
 extern void unuse_pack(struct pack_window **);
 extern void clear_delta_base_cache(void);
 extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 07/27] pack: move prepare_packed_git_run_once to object store
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (5 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 06/27] object-store: close all packs upon clearing the " Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 08/27] pack: move approximate object count " Stefan Beller
                         ` (19 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Each repository's object store can be initialized independently, so
they must not share a run_once variable.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 8 +++++++-
 packfile.c     | 7 +++----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/object-store.h b/object-store.h
index 1de9e07102..6cecba3951 100644
--- a/object-store.h
+++ b/object-store.h
@@ -92,6 +92,12 @@ struct raw_object_store {
 
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
 };
 
 /*
@@ -101,7 +107,7 @@ struct raw_object_store {
  * that macro on member variables. Use NULL instead as that is defined
  * and accepted, deferring the real init to prepare_packed_git_mru(). */
 #define __MRU_INIT { NULL, NULL }
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index 511a2b0cdf..a8a2e7fe43 100644
--- a/packfile.c
+++ b/packfile.c
@@ -884,12 +884,11 @@ static void prepare_packed_git_mru(void)
 		list_add_tail(&p->mru, &the_repository->objects.packed_git_mru);
 }
 
-static int prepare_packed_git_run_once = 0;
 void prepare_packed_git(void)
 {
 	struct alternate_object_database *alt;
 
-	if (prepare_packed_git_run_once)
+	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
 	prepare_alt_odb();
@@ -897,13 +896,13 @@ void prepare_packed_git(void)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
 	prepare_packed_git_mru();
-	prepare_packed_git_run_once = 1;
+	the_repository->objects.packed_git_initialized = 1;
 }
 
 void reprepare_packed_git(void)
 {
 	approximate_object_count_valid = 0;
-	prepare_packed_git_run_once = 0;
+	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 08/27] pack: move approximate object count to object store
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (6 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 07/27] pack: move prepare_packed_git_run_once to " Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 09/27] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
                         ` (18 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

The approximate_object_count() function maintains a rough count of
objects in a repository to estimate how long object name abbreviates
should be.  Object names are scoped to a repository and the
appropriate length may differ by repository, so the object count
should not be global.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 10 +++++++++-
 packfile.c     | 11 +++++------
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/object-store.h b/object-store.h
index 6cecba3951..bd1e4fcd8b 100644
--- a/object-store.h
+++ b/object-store.h
@@ -93,6 +93,14 @@ struct raw_object_store {
 	struct alternate_object_database *alt_odb_list;
 	struct alternate_object_database **alt_odb_tail;
 
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
 	/*
 	 * Whether packed_git has already been populated with this repository's
 	 * packs.
@@ -107,7 +115,7 @@ struct raw_object_store {
  * that macro on member variables. Use NULL instead as that is defined
  * and accepted, deferring the real init to prepare_packed_git_mru(). */
 #define __MRU_INIT { NULL, NULL }
-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }
+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0, 0, 0 }
 
 void raw_object_store_clear(struct raw_object_store *o);
 
diff --git a/packfile.c b/packfile.c
index a8a2e7fe43..693bafbc98 100644
--- a/packfile.c
+++ b/packfile.c
@@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)
 	strbuf_release(&path);
 }
 
-static int approximate_object_count_valid;
-
 /*
  * Give a fast, rough count of the number of objects in the repository. This
  * ignores loose objects completely. If you have a lot of them, then either
@@ -814,8 +812,8 @@ static int approximate_object_count_valid;
  */
 unsigned long approximate_object_count(void)
 {
-	static unsigned long count;
-	if (!approximate_object_count_valid) {
+	if (!the_repository->objects.approximate_object_count_valid) {
+		unsigned long count;
 		struct packed_git *p;
 
 		prepare_packed_git();
@@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)
 				continue;
 			count += p->num_objects;
 		}
+		the_repository->objects.approximate_object_count = count;
 	}
-	return count;
+	return the_repository->objects.approximate_object_count;
 }
 
 static void *get_next_packed_git(const void *p)
@@ -901,7 +900,7 @@ void prepare_packed_git(void)
 
 void reprepare_packed_git(void)
 {
-	approximate_object_count_valid = 0;
+	the_repository->objects.approximate_object_count_valid = 0;
 	the_repository->objects.packed_git_initialized = 0;
 	prepare_packed_git();
 }
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 09/27] sha1_file: add raw_object_store argument to alt_odb_usable
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (7 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 08/27] pack: move approximate object count " Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 10/27] sha1_file: add repository argument to link_alt_odb_entry Stefan Beller
                         ` (17 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine

Add a raw_object_store to alt_odb_usable to be more specific about which
repository to act on. The choice of the repository is delegated to its
only caller link_alt_odb_entry.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 1348dce68f..a8e23bd2f8 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -346,7 +346,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,
 /*
  * Return non-zero iff the path is usable as an alternate object database.
  */
-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir)
 {
 	struct alternate_object_database *alt;
 
@@ -362,7 +364,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)
 	 * Prevent the common mistake of listing the same
 	 * thing twice, or object directory itself.
 	 */
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = o->alt_odb_list; alt; alt = alt->next) {
 		if (!fspathcmp(path->buf, alt->path))
 			return 0;
 	}
@@ -414,7 +416,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 10/27] sha1_file: add repository argument to link_alt_odb_entry
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (8 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 09/27] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 11/27] sha1_file: add repository argument to read_info_alternates Stefan Beller
                         ` (16 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Add a repository argument to allow the link_alt_odb_entry caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

Since the implementation does not yet work with other repositories,
use a wrapper macro to enforce that the caller passes in
the_repository as the first argument. It would be more appealing to
use BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work
because it requires a compile-time constant and common compilers like
gcc 4.8.4 do not consider "r == the_repository" a compile-time
constant.

This and the following three patches add repository arguments to
link_alt_odb_entry, read_info_alternates, link_alt_odb_entries
and prepare_alt_odb. Three out of the four functions are found
in a recursive call chain, calling each other, and one of them
accesses the repositories `objectdir` (which was migrated; it
was an obvious choice) and `ignore_env` (which we need to keep in
the repository struct for clarify); hence we will pass through the
repository unlike just the object store object + the ignore_env flag.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index a8e23bd2f8..aaa6276211 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -390,8 +390,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * terminating NUL.
  */
 static void read_info_alternates(const char * relative_base, int depth);
-static int link_alt_odb_entry(const char *entry, const char *relative_base,
-	int depth, const char *normalized_objdir)
+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
+static int link_alt_odb_entry_the_repository(const char *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
 	struct strbuf pathbuf = STRBUF_INIT;
@@ -488,7 +489,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);
+		link_alt_odb_entry(the_repository, entry.buf,
+				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 11/27] sha1_file: add repository argument to read_info_alternates
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (9 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 10/27] sha1_file: add repository argument to link_alt_odb_entry Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 12/27] sha1_file: add repository argument to link_alt_odb_entries Stefan Beller
                         ` (15 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

See previous patch for explanation.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index aaa6276211..6edcc3d3a7 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -389,7 +389,9 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void read_info_alternates(const char * relative_base, int depth);
+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth);
 #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
 static int link_alt_odb_entry_the_repository(const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
@@ -430,7 +432,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(pathbuf.buf, depth + 1);
+	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -496,7 +498,8 @@ static void link_alt_odb_entries(const char *alt, int sep,
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates(const char * relative_base, int depth)
+static void read_info_alternates_the_repository(const char *relative_base,
+						int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -680,7 +683,7 @@ void prepare_alt_odb(void)
 			&the_repository->objects.alt_odb_list;
 	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
 
-	read_info_alternates(get_object_directory(), 0);
+	read_info_alternates(the_repository, get_object_directory(), 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 12/27] sha1_file: add repository argument to link_alt_odb_entries
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (10 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 11/27] sha1_file: add repository argument to read_info_alternates Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb Stefan Beller
                         ` (14 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

See previous patch for explanation.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 6edcc3d3a7..a0e9116318 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -467,8 +467,12 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-static void link_alt_odb_entries(const char *alt, int sep,
-				 const char *relative_base, int depth)
+#define link_alt_odb_entries(r, a, s, rb, d) \
+	link_alt_odb_entries_##r(a, s, rb, d)
+static void link_alt_odb_entries_the_repository(const char *alt,
+						int sep,
+						const char *relative_base,
+						int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -511,7 +515,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
@@ -565,7 +569,8 @@ void add_to_alternates_file(const char *reference)
 		if (commit_lock_file(&lock))
 			die_errno("unable to move new alternates file into place");
 		if (the_repository->objects.alt_odb_tail)
-			link_alt_odb_entries(reference, '\n', NULL, 0);
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
 	}
 	free(alts);
 }
@@ -578,7 +583,8 @@ void add_to_alternates_memory(const char *reference)
 	 */
 	prepare_alt_odb();
 
-	link_alt_odb_entries(reference, '\n', NULL, 0);
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
 }
 
 /*
@@ -681,7 +687,8 @@ void prepare_alt_odb(void)
 
 	the_repository->objects.alt_odb_tail =
 			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(alt, PATH_SEP, NULL, 0);
+	link_alt_odb_entries(the_repository, alt,
+			     PATH_SEP, NULL, 0);
 
 	read_info_alternates(the_repository, get_object_directory(), 0);
 }
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (11 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 12/27] sha1_file: add repository argument to link_alt_odb_entries Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories Stefan Beller
                         ` (13 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

See previous patch for explanation.

While at it, move the declaration to object-store.h,
where it should be easier to find.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 builtin/fsck.c |  2 +-
 object-store.h |  3 ++-
 packfile.c     |  2 +-
 sha1_file.c    | 13 +++++++------
 sha1_name.c    |  3 ++-
 5 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index 2e99e02128..76c94e9b5a 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -717,7 +717,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	} else {
 		fsck_object_dir(get_object_directory());
 
-		prepare_alt_odb();
+		prepare_alt_odb(the_repository);
 		for (alt = the_repository->objects.alt_odb_list;
 				alt; alt = alt->next)
 			fsck_object_dir(alt->path);
diff --git a/object-store.h b/object-store.h
index bd1e4fcd8b..514ad94287 100644
--- a/object-store.h
+++ b/object-store.h
@@ -20,7 +20,8 @@ struct alternate_object_database {
 
 	char path[FLEX_ARRAY];
 };
-void prepare_alt_odb(void);
+#define prepare_alt_odb(r) prepare_alt_odb_##r()
+void prepare_alt_odb_the_repository(void);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
diff --git a/packfile.c b/packfile.c
index 693bafbc98..ccfe084642 100644
--- a/packfile.c
+++ b/packfile.c
@@ -890,7 +890,7 @@ void prepare_packed_git(void)
 	if (the_repository->objects.packed_git_initialized)
 		return;
 	prepare_packed_git_one(get_object_directory(), 1);
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)
 		prepare_packed_git_one(alt->path, 0);
 	rearrange_packed_git();
diff --git a/sha1_file.c b/sha1_file.c
index a0e9116318..c26921df4a 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -23,6 +23,7 @@
 #include "sha1-lookup.h"
 #include "bulk-checkin.h"
 #include "repository.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "dir.h"
 #include "list.h"
@@ -581,7 +582,7 @@ void add_to_alternates_memory(const char *reference)
 	 * Make sure alternates are initialized, or else our entry may be
 	 * overwritten when they are.
 	 */
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 
 	link_alt_odb_entries(the_repository, reference,
 			     '\n', NULL, 0);
@@ -667,7 +668,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	struct alternate_object_database *ent;
 	int r = 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {
 		r = fn(ent, cb);
 		if (r)
@@ -676,7 +677,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb(void)
+void prepare_alt_odb_the_repository(void)
 {
 	const char *alt;
 
@@ -730,7 +731,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)
 {
 	struct alternate_object_database *alt;
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		const char *path = alt_sha1_path(alt, sha1);
 		if (check_and_freshen_file(path, freshen))
@@ -889,7 +890,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	errno = ENOENT;
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
@@ -920,7 +921,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
diff --git a/sha1_name.c b/sha1_name.c
index 016c883b5c..3e490ee8f6 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -11,6 +11,7 @@
 #include "sha1-array.h"
 #include "packfile.h"
 #include "repository.h"
+#include "object-store.h"
 
 static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);
 
@@ -353,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,
 
 	ds->len = len;
 	ds->hex_pfx[len] = '\0';
-	prepare_alt_odb();
+	prepare_alt_odb(the_repository);
 	return 0;
 }
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (12 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 15/27] sha1_file: allow prepare_alt_odb " Stefan Beller
                         ` (12 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Actually this also allows read_info_alternates and link_alt_odb_entry to
handle arbitrary repositories, but link_alt_odb_entries is the most
interesting function in this set of functions, hence the commit subject.

These functions span a strongly connected component in the function
graph, i.e. the recursive call chain might look like

  -> link_alt_odb_entries
    -> link_alt_odb_entry
      -> read_info_alternates
        -> link_alt_odb_entries

That is why we need to convert all these functions at the same time.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object-store.h |  4 ++++
 sha1_file.c    | 36 ++++++++++++++++--------------------
 2 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/object-store.h b/object-store.h
index 514ad94287..f283fbdba9 100644
--- a/object-store.h
+++ b/object-store.h
@@ -18,6 +18,10 @@ struct alternate_object_database {
 	char loose_objects_subdir_seen[256];
 	struct oid_array loose_objects_cache;
 
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
 	char path[FLEX_ARRAY];
 };
 #define prepare_alt_odb(r) prepare_alt_odb_##r()
diff --git a/sha1_file.c b/sha1_file.c
index c26921df4a..6e5105a252 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth);
-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)
-static int link_alt_odb_entry_the_repository(const char *entry,
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const char *entry,
 	const char *relative_base, int depth, const char *normalized_objdir)
 {
 	struct alternate_object_database *ent;
@@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
 		strbuf_setlen(&pathbuf, pathbuf.len - 1);
 
-	if (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {
+	if (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {
 		strbuf_release(&pathbuf);
 		return -1;
 	}
@@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,
 	ent = alloc_alt_odb(pathbuf.buf);
 
 	/* add the alternate entry */
-	*the_repository->objects.alt_odb_tail = ent;
-	the_repository->objects.alt_odb_tail = &(ent->next);
+	*r->objects.alt_odb_tail = ent;
+	r->objects.alt_odb_tail = &(ent->next);
 	ent->next = NULL;
 
 	/* recursively add alternates */
-	read_info_alternates(the_repository, pathbuf.buf, depth + 1);
+	read_info_alternates(r, pathbuf.buf, depth + 1);
 
 	strbuf_release(&pathbuf);
 	return 0;
@@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,
 	return end;
 }
 
-#define link_alt_odb_entries(r, a, s, rb, d) \
-	link_alt_odb_entries_##r(a, s, rb, d)
-static void link_alt_odb_entries_the_repository(const char *alt,
-						int sep,
-						const char *relative_base,
-						int depth)
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
 {
 	struct strbuf objdirbuf = STRBUF_INIT;
 	struct strbuf entry = STRBUF_INIT;
@@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		return;
 	}
 
-	strbuf_add_absolute_path(&objdirbuf, get_object_directory());
+	strbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);
 	if (strbuf_normalize_path(&objdirbuf) < 0)
 		die("unable to normalize object directory: %s",
 		    objdirbuf.buf);
@@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,
 		alt = parse_alt_odb_entry(alt, sep, &entry);
 		if (!entry.len)
 			continue;
-		link_alt_odb_entry(the_repository, entry.buf,
+		link_alt_odb_entry(r, entry.buf,
 				   relative_base, depth, objdirbuf.buf);
 	}
 	strbuf_release(&entry);
 	strbuf_release(&objdirbuf);
 }
 
-static void read_info_alternates_the_repository(const char *relative_base,
-						int depth)
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
 {
 	char *path;
 	struct strbuf buf = STRBUF_INIT;
@@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,
 		return;
 	}
 
-	link_alt_odb_entries(the_repository, buf.buf, '\n', relative_base, depth);
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
 	strbuf_release(&buf);
 	free(path);
 }
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 15/27] sha1_file: allow prepare_alt_odb to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (13 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 16/27] sha1_file: add repository argument to sha1_file_name Stefan Beller
                         ` (11 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object-store.h |  3 +--
 sha1_file.c    | 21 +++++++++++----------
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/object-store.h b/object-store.h
index f283fbdba9..873b341774 100644
--- a/object-store.h
+++ b/object-store.h
@@ -24,8 +24,7 @@ struct alternate_object_database {
 	 */
 	char path[FLEX_ARRAY];
 };
-#define prepare_alt_odb(r) prepare_alt_odb_##r()
-void prepare_alt_odb_the_repository(void);
+void prepare_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
 typedef int alt_odb_fn(struct alternate_object_database *, void *);
 int foreach_alt_odb(alt_odb_fn, void*);
diff --git a/sha1_file.c b/sha1_file.c
index 6e5105a252..9cf3fffbeb 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -673,21 +673,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)
 	return r;
 }
 
-void prepare_alt_odb_the_repository(void)
+void prepare_alt_odb(struct repository *r)
 {
-	const char *alt;
-
-	if (the_repository->objects.alt_odb_tail)
+	if (r->objects.alt_odb_tail)
 		return;
 
-	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+	r->objects.alt_odb_tail = &r->objects.alt_odb_list;
+
+	if (!r->ignore_env) {
+		const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+		if (!alt)
+			alt = "";
 
-	the_repository->objects.alt_odb_tail =
-			&the_repository->objects.alt_odb_list;
-	link_alt_odb_entries(the_repository, alt,
-			     PATH_SEP, NULL, 0);
+		link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);
+	}
 
-	read_info_alternates(the_repository, get_object_directory(), 0);
+	read_info_alternates(r, r->objects.objectdir, 0);
 }
 
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 16/27] sha1_file: add repository argument to sha1_file_name
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (14 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 15/27] sha1_file: allow prepare_alt_odb " Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 17/27] sha1_file: add repository argument to stat_sha1_file Stefan Beller
                         ` (10 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Add a repository argument to allow sha1_file_name callers to be more
specific about which repository to handle. This is a small mechanical
change; it doesn't change the implementation to handle repositories
other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should
be easier to find.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 cache.h        |  6 ------
 http-walker.c  |  3 ++-
 http.c         |  5 ++---
 object-store.h |  7 +++++++
 sha1_file.c    | 10 +++++-----
 5 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/cache.h b/cache.h
index 3fd9dc52cd..e641c23b0a 100644
--- a/cache.h
+++ b/cache.h
@@ -961,12 +961,6 @@ extern void check_repository_format(void);
 #define DATA_CHANGED    0x0020
 #define TYPE_CHANGED    0x0040
 
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified sha1.
- */
-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);
-
 /*
  * Return an abbreviated sha1 unique within this repository's object database.
  * The result will be at least `len` characters long, and will be NUL
diff --git a/http-walker.c b/http-walker.c
index 8bb5d991bb..76bfdbca76 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "walker.h"
 #include "http.h"
@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)
 		ret = error("File %s has bad hash", hex);
 	} else if (req->rename < 0) {
 		struct strbuf buf = STRBUF_INIT;
-		sha1_file_name(&buf, req->sha1);
+		sha1_file_name(the_repository, &buf, req->sha1);
 		ret = error("unable to write sha1 filename %s", buf.buf);
 		strbuf_release(&buf);
 	}
diff --git a/http.c b/http.c
index a4a9e583c7..afe2707e86 100644
--- a/http.c
+++ b/http.c
@@ -2247,7 +2247,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	hashcpy(freq->sha1, sha1);
 	freq->localfile = -1;
 
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 	snprintf(freq->tmpfile, sizeof(freq->tmpfile),
 		 "%s.temp", filename.buf);
 
@@ -2396,8 +2396,7 @@ int finish_http_object_request(struct http_object_request *freq)
 		unlink_or_warn(freq->tmpfile);
 		return -1;
 	}
-
-	sha1_file_name(&filename, freq->sha1);
+	sha1_file_name(the_repository, &filename, freq->sha1);
 	freq->rename = finalize_object_file(freq->tmpfile, filename.buf);
 	strbuf_release(&filename);
 
diff --git a/object-store.h b/object-store.h
index 873b341774..4f5060e768 100644
--- a/object-store.h
+++ b/object-store.h
@@ -123,4 +123,11 @@ struct raw_object_store {
 
 void raw_object_store_clear(struct raw_object_store *o);
 
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified sha1.
+ */
+#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+
 #endif /* OBJECT_STORE_H */
diff --git a/sha1_file.c b/sha1_file.c
index 9cf3fffbeb..811ed9e53f 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -323,7 +323,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
 {
 	strbuf_addstr(buf, get_object_directory());
 	strbuf_addch(buf, '/');
@@ -720,7 +720,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 
 	return check_and_freshen_file(buf.buf, freshen);
 }
@@ -881,7 +881,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
@@ -910,7 +910,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(&buf, sha1);
+	sha1_file_name(the_repository, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -1595,7 +1595,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	static struct strbuf filename = STRBUF_INIT;
 
 	strbuf_reset(&filename);
-	sha1_file_name(&filename, sha1);
+	sha1_file_name(the_repository, &filename, sha1);
 
 	fd = create_tmpfile(&tmp_file, filename.buf);
 	if (fd < 0) {
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 17/27] sha1_file: add repository argument to stat_sha1_file
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (15 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 16/27] sha1_file: add repository argument to sha1_file_name Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 18/27] sha1_file: add repository argument to open_sha1_file Stefan Beller
                         ` (9 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Add a repository argument to allow the stat_sha1_file caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 811ed9e53f..ed297773e5 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -874,8 +874,9 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,
-			  const char **path)
+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
+static int stat_sha1_file_the_repository(const unsigned char *sha1,
+					 struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
@@ -1181,7 +1182,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(sha1, &st, &path) < 0)
+		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
@@ -1395,7 +1396,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,
 		die("replacement %s not found for %s",
 		    sha1_to_hex(repl), sha1_to_hex(sha1));
 
-	if (!stat_sha1_file(repl, &st, &path))
+	if (!stat_sha1_file(the_repository, repl, &st, &path))
 		die("loose object %s (stored in %s) is corrupt",
 		    sha1_to_hex(repl), path);
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 18/27] sha1_file: add repository argument to open_sha1_file
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (16 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 17/27] sha1_file: add repository argument to stat_sha1_file Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 19/27] sha1_file: add repository argument to map_sha1_file_1 Stefan Beller
                         ` (8 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Add a repository argument to allow the open_sha1_file caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index ed297773e5..68aff90792 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -903,7 +903,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-static int open_sha1_file(const unsigned char *sha1, const char **path)
+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
+static int open_sha1_file_the_repository(const unsigned char *sha1,
+					 const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -946,7 +948,7 @@ static void *map_sha1_file_1(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(sha1, &path);
+		fd = open_sha1_file(the_repository, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 19/27] sha1_file: add repository argument to map_sha1_file_1
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (17 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 18/27] sha1_file: add repository argument to open_sha1_file Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 20/27] sha1_file: add repository argument to map_sha1_file Stefan Beller
                         ` (7 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Add a repository argument to allow the map_sha1_file_1 caller to be
more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 68aff90792..7fa9ed2f25 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -938,9 +938,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-static void *map_sha1_file_1(const char *path,
-			     const unsigned char *sha1,
-			     unsigned long *size)
+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
+static void *map_sha1_file_1_the_repository(const char *path,
+					    const unsigned char *sha1,
+					    unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -969,7 +970,7 @@ static void *map_sha1_file_1(const char *path,
 
 void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(NULL, sha1, size);
+	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
@@ -2199,7 +2200,7 @@ int read_loose_object(const char *path,
 
 	*contents = NULL;
 
-	map = map_sha1_file_1(path, NULL, &mapsize);
+	map = map_sha1_file_1(the_repository, path, NULL, &mapsize);
 	if (!map) {
 		error_errno("unable to mmap %s", path);
 		goto out;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 20/27] sha1_file: add repository argument to map_sha1_file
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (18 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 19/27] sha1_file: add repository argument to map_sha1_file_1 Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 21/27] sha1_file: add repository argument to sha1_loose_object_info Stefan Beller
                         ` (6 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Add a repository argument to allow map_sha1_file callers to be more
specific about which repository to handle. This is a small mechanical
change; it doesn't change the implementation to handle repositories
other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

While at it, move the declaration to object-store.h, where it should
be easier to find.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 cache.h        | 1 -
 object-store.h | 3 +++
 sha1_file.c    | 4 ++--
 streaming.c    | 5 ++++-
 4 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/cache.h b/cache.h
index e641c23b0a..d338b65d75 100644
--- a/cache.h
+++ b/cache.h
@@ -1242,7 +1242,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c
 extern int force_object_loose(const unsigned char *sha1, time_t mtime);
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
 extern int parse_sha1_header(const char *hdr, unsigned long *sizep);
 
diff --git a/object-store.h b/object-store.h
index 4f5060e768..a37d74a0c5 100644
--- a/object-store.h
+++ b/object-store.h
@@ -130,4 +130,7 @@ void raw_object_store_clear(struct raw_object_store *o);
 #define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
 void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
 
+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+
 #endif /* OBJECT_STORE_H */
diff --git a/sha1_file.c b/sha1_file.c
index 7fa9ed2f25..0d3ff99cf2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -968,7 +968,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	return map;
 }
 
-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
 {
 	return map_sha1_file_1(the_repository, NULL, sha1, size);
 }
@@ -1192,7 +1192,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 		return 0;
 	}
 
-	map = map_sha1_file(sha1, &mapsize);
+	map = map_sha1_file(the_repository, sha1, &mapsize);
 	if (!map)
 		return -1;
 
diff --git a/streaming.c b/streaming.c
index 5892b50bd8..22d27df55e 100644
--- a/streaming.c
+++ b/streaming.c
@@ -3,6 +3,8 @@
  */
 #include "cache.h"
 #include "streaming.h"
+#include "repository.h"
+#include "object-store.h"
 #include "packfile.h"
 
 enum input_source {
@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {
 
 static open_method_decl(loose)
 {
-	st->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);
+	st->u.loose.mapped = map_sha1_file(the_repository,
+					   sha1, &st->u.loose.mapsize);
 	if (!st->u.loose.mapped)
 		return -1;
 	if ((unpack_sha1_header(&st->z,
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 21/27] sha1_file: add repository argument to sha1_loose_object_info
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (19 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 20/27] sha1_file: add repository argument to map_sha1_file Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories Stefan Beller
                         ` (5 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Add a repository argument to allow the sha1_loose_object_info caller
to be more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 0d3ff99cf2..d1f77a4c80 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1159,9 +1159,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-static int sha1_loose_object_info(const unsigned char *sha1,
-				  struct object_info *oi,
-				  int flags)
+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
+						 struct object_info *oi,
+						 int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1280,7 +1281,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,
 			break;
 
 		/* Most likely it's a loose object. */
-		if (!sha1_loose_object_info(real, oi, flags))
+		if (!sha1_loose_object_info(the_repository, real, oi, flags))
 			return 0;
 
 		/* Not a loose object; someone else may have just packed it. */
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (20 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 21/27] sha1_file: add repository argument to sha1_loose_object_info Stefan Beller
@ 2018-02-21  1:54       ` Stefan Beller
  2018-02-21  1:54       ` [PATCH 23/27] sha1_file: allow stat_sha1_file " Stefan Beller
                         ` (4 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 3 +--
 sha1_file.c    | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/object-store.h b/object-store.h
index a37d74a0c5..146358fe33 100644
--- a/object-store.h
+++ b/object-store.h
@@ -127,8 +127,7 @@ void raw_object_store_clear(struct raw_object_store *o);
  * Put in `buf` the name of the file in the local object database that
  * would be used to store a loose object with the specified sha1.
  */
-#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
 #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
 void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
diff --git a/sha1_file.c b/sha1_file.c
index d1f77a4c80..8fad523d85 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -323,9 +323,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)
 	}
 }
 
-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)
+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)
 {
-	strbuf_addstr(buf, get_object_directory());
+	strbuf_addstr(buf, r->objects.objectdir);
 	strbuf_addch(buf, '/');
 	fill_sha1_path(buf, sha1);
 }
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 23/27] sha1_file: allow stat_sha1_file to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (21 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 24/27] sha1_file: allow open_sha1_file " Stefan Beller
                         ` (3 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 8fad523d85..6879af8993 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -874,23 +874,22 @@ int git_open_cloexec(const char *name, int flags)
  * Note that it may point to static storage and is only valid until another
  * call to sha1_file_name(), etc.
  */
-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)
-static int stat_sha1_file_the_repository(const unsigned char *sha1,
-					 struct stat *st, const char **path)
+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
+			  struct stat *st, const char **path)
 {
 	struct alternate_object_database *alt;
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	if (!lstat(*path, st))
 		return 0;
 
-	prepare_alt_odb(the_repository);
+	prepare_alt_odb(r);
 	errno = ENOENT;
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		if (!lstat(*path, st))
 			return 0;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 24/27] sha1_file: allow open_sha1_file to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (22 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 23/27] sha1_file: allow stat_sha1_file " Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 25/27] sha1_file: allow map_sha1_file_1 " Stefan Beller
                         ` (2 subsequent siblings)
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 6879af8993..598acc5410 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -902,9 +902,8 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,
  * Like stat_sha1_file(), but actually open the object and return the
  * descriptor. See the caveats on the "path" parameter above.
  */
-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)
-static int open_sha1_file_the_repository(const unsigned char *sha1,
-					 const char **path)
+static int open_sha1_file(struct repository *r,
+			  const unsigned char *sha1, const char **path)
 {
 	int fd;
 	struct alternate_object_database *alt;
@@ -912,7 +911,7 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 	static struct strbuf buf = STRBUF_INIT;
 
 	strbuf_reset(&buf);
-	sha1_file_name(the_repository, &buf, sha1);
+	sha1_file_name(r, &buf, sha1);
 	*path = buf.buf;
 
 	fd = git_open(*path);
@@ -920,8 +919,8 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,
 		return fd;
 	most_interesting_errno = errno;
 
-	prepare_alt_odb(the_repository);
-	for (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {
+	prepare_alt_odb(r);
+	for (alt = r->objects.alt_odb_list; alt; alt = alt->next) {
 		*path = alt_sha1_path(alt, sha1);
 		fd = git_open(*path);
 		if (fd >= 0)
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 25/27] sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (23 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 24/27] sha1_file: allow open_sha1_file " Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 26/27] sha1_file: allow map_sha1_file " Stefan Beller
  2018-02-21  1:54       ` [PATCH 27/27] sha1_file: allow sha1_loose_object_info " Stefan Beller
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

From: Jonathan Nieder <jrnieder@gmail.com>

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 598acc5410..522804d9f0 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -936,10 +936,8 @@ static int open_sha1_file(struct repository *r,
  * Map the loose object at "path" if it is not NULL, or the path found by
  * searching for a loose object named "sha1".
  */
-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)
-static void *map_sha1_file_1_the_repository(const char *path,
-					    const unsigned char *sha1,
-					    unsigned long *size)
+static void *map_sha1_file_1(struct repository *r, const char *path,
+			     const unsigned char *sha1, unsigned long *size)
 {
 	void *map;
 	int fd;
@@ -947,7 +945,7 @@ static void *map_sha1_file_1_the_repository(const char *path,
 	if (path)
 		fd = git_open(path);
 	else
-		fd = open_sha1_file(the_repository, sha1, &path);
+		fd = open_sha1_file(r, sha1, &path);
 	map = NULL;
 	if (fd >= 0) {
 		struct stat st;
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 26/27] sha1_file: allow map_sha1_file to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (24 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 25/27] sha1_file: allow map_sha1_file_1 " Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  2018-02-21  1:54       ` [PATCH 27/27] sha1_file: allow sha1_loose_object_info " Stefan Beller
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 object-store.h | 3 +--
 sha1_file.c    | 5 +++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/object-store.h b/object-store.h
index 146358fe33..5d93677783 100644
--- a/object-store.h
+++ b/object-store.h
@@ -129,7 +129,6 @@ void raw_object_store_clear(struct raw_object_store *o);
  */
 void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);
 
-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);
+void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
 #endif /* OBJECT_STORE_H */
diff --git a/sha1_file.c b/sha1_file.c
index 522804d9f0..a20f811a07 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -964,9 +964,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,
 	return map;
 }
 
-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)
+void *map_sha1_file(struct repository *r,
+		    const unsigned char *sha1, unsigned long *size)
 {
-	return map_sha1_file_1(the_repository, NULL, sha1, size);
+	return map_sha1_file_1(r, NULL, sha1, size);
 }
 
 static int unpack_sha1_short_header(git_zstream *stream,
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

* [PATCH 27/27] sha1_file: allow sha1_loose_object_info to handle arbitrary repositories
  2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
                         ` (25 preceding siblings ...)
  2018-02-21  1:54       ` [PATCH 26/27] sha1_file: allow map_sha1_file " Stefan Beller
@ 2018-02-21  1:54       ` " Stefan Beller
  26 siblings, 0 replies; 100+ messages in thread
From: Stefan Beller @ 2018-02-21  1:54 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy, pclouds, sunshine, Jonathan Nieder

From: Jonathan Nieder <jrnieder@gmail.com>

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 sha1_file.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index a20f811a07..36282acb1a 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1156,10 +1156,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)
 	return parse_sha1_header_extended(hdr, &oi, 0);
 }
 
-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)
-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
-						 struct object_info *oi,
-						 int flags)
+static int sha1_loose_object_info(struct repository *r,
+				  const unsigned char *sha1,
+				  struct object_info *oi, int flags)
 {
 	int status = 0;
 	unsigned long mapsize;
@@ -1183,14 +1182,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,
 	if (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {
 		const char *path;
 		struct stat st;
-		if (stat_sha1_file(the_repository, sha1, &st, &path) < 0)
+		if (stat_sha1_file(r, sha1, &st, &path) < 0)
 			return -1;
 		if (oi->disk_sizep)
 			*oi->disk_sizep = st.st_size;
 		return 0;
 	}
 
-	map = map_sha1_file(the_repository, sha1, &mapsize);
+	map = map_sha1_file(r, sha1, &mapsize);
 	if (!map)
 		return -1;
 
-- 
2.16.1.291.g4437f3f132-goog


^ permalink raw reply	[flat|nested] 100+ messages in thread

end of thread, back to index

Thread overview: 100+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-13  1:22 [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
2018-02-13  1:22 ` [PATCH 01/26] repository: introduce raw object store field Stefan Beller
2018-02-13  1:22 ` [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
2018-02-13 18:51   ` Brandon Williams
2018-02-13 19:46     ` Stefan Beller
2018-02-13  1:22 ` [PATCH 03/26] object-store: free alt_odb_list Stefan Beller
2018-02-13  1:22 ` [PATCH 04/26] object-store: move packed_git and packed_git_mru to object store Stefan Beller
2018-02-13  1:22 ` [PATCH 05/26] object-store: close all packs upon clearing the " Stefan Beller
2018-02-13  1:22 ` [PATCH 06/26] pack: move prepare_packed_git_run_once to " Stefan Beller
2018-02-13  1:22 ` [PATCH 07/26] pack: move approximate object count " Stefan Beller
2018-02-13  1:22 ` [PATCH 08/26] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
2018-02-13  1:22 ` [PATCH 09/26] sha1_file: add repository argument to link_alt_odb_entry Stefan Beller
2018-02-13  1:22 ` [PATCH 10/26] sha1_file: add repository argument to read_info_alternates Stefan Beller
2018-02-13  1:22 ` [PATCH 11/26] sha1_file: add repository argument to link_alt_odb_entries Stefan Beller
2018-02-13  1:22 ` [PATCH 12/26] sha1_file: add repository argument to prepare_alt_odb Stefan Beller
2018-02-13  1:22 ` [PATCH 13/26] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories Stefan Beller
2018-02-13  1:22 ` [PATCH 14/26] sha1_file: allow prepare_alt_odb " Stefan Beller
2018-02-14  0:37   ` Duy Nguyen
2018-02-14 18:08     ` Brandon Williams
2018-02-13  1:22 ` [PATCH 15/26] sha1_file: add repository argument to sha1_file_name Stefan Beller
2018-02-13  1:22 ` [PATCH 16/26] sha1_file: add repository argument to stat_sha1_file Stefan Beller
2018-02-13  1:22 ` [PATCH 17/26] sha1_file: add repository argument to open_sha1_file Stefan Beller
2018-02-13  1:22 ` [PATCH 18/26] sha1_file: add repository argument to map_sha1_file_1 Stefan Beller
2018-02-13  1:22 ` [PATCH 19/26] sha1_file: add repository argument to map_sha1_file Stefan Beller
2018-02-13  1:22 ` [PATCH 20/26] sha1_file: add repository argument to sha1_loose_object_info Stefan Beller
2018-02-13  1:22 ` [PATCH 21/26] sha1_file: allow sha1_file_name to handle arbitrary repositories Stefan Beller
2018-02-13  1:22 ` [PATCH 22/26] sha1_file: allow stat_sha1_file " Stefan Beller
2018-02-13  1:22 ` [PATCH 23/26] sha1_file: allow open_sha1_file " Stefan Beller
2018-02-13  1:22 ` [PATCH 24/26] sha1_file: allow map_sha1_file_1 " Stefan Beller
2018-02-13  1:22 ` [PATCH 25/26] sha1_file: allow map_sha1_file " Stefan Beller
2018-02-13  1:22 ` [PATCH 26/26] sha1_file: allow sha1_loose_object_info " Stefan Beller
2018-02-13  1:38 ` [PATCH 00/26] Moving global state into the repository object (part 1) Stefan Beller
2018-02-13 11:49 ` Duy Nguyen
2018-02-13 12:13   ` Duy Nguyen
2018-02-13 16:52     ` Brandon Williams
2018-02-13 17:47     ` Stefan Beller
2018-02-13 18:57       ` Junio C Hamano
2018-02-13 19:23         ` Stefan Beller
2018-02-13 19:35           ` Junio C Hamano
2018-02-13 19:43             ` Stefan Beller
2018-02-14  0:30               ` Junio C Hamano
2018-02-13 19:26 ` Jonathan Nieder
2018-02-14  0:57   ` Duy Nguyen
2018-02-13 19:33 ` Brandon Williams
2018-02-15 20:42 ` Junio C Hamano
2018-02-15 21:09   ` Stefan Beller
2018-02-16 17:46   ` [PATCHv2 00/16] " Stefan Beller
2018-02-16 17:46     ` [PATCH 01/16] repository: introduce raw object store field Stefan Beller
2018-02-16 17:46     ` [PATCH 02/16] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
2018-02-16 17:46     ` [PATCH 03/16] object-store: free alt_odb_list Stefan Beller
2018-02-16 17:46     ` [PATCH 04/16] object-store: move packed_git and packed_git_mru to object store Stefan Beller
2018-02-16 20:27       ` Junio C Hamano
2018-02-16 17:46     ` [PATCH 05/16] object-store: close all packs upon clearing the " Stefan Beller
2018-02-16 17:46     ` [PATCH 06/16] pack: move prepare_packed_git_run_once to " Stefan Beller
2018-02-16 17:46     ` [PATCH 07/16] pack: move approximate object count " Stefan Beller
2018-02-16 17:46     ` [PATCH 08/16] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
2018-02-16 17:46     ` [PATCH 09/16] sha1_file: allow link_alt_odb_entries to handle arbitrary object stores Stefan Beller
2018-02-16 17:46     ` [PATCH 10/16] sha1_file: allow prepare_alt_odb " Stefan Beller
2018-02-16 17:46     ` [PATCH 11/16] sha1_file: allow sha1_file_name " Stefan Beller
2018-02-16 17:46     ` [PATCH 12/16] sha1_file: allow stat_sha1_file " Stefan Beller
2018-02-16 17:46     ` [PATCH 13/16] sha1_file: allow open_sha1_file " Stefan Beller
2018-02-16 17:46     ` [PATCH 14/16] sha1_file: allow map_sha1_file_1 " Stefan Beller
2018-02-16 17:46     ` [PATCH 15/16] sha1_file: allow map_sha1_file " Stefan Beller
2018-02-16 17:46     ` [PATCH 16/16] sha1_file: allow sha1_loose_object_info " Stefan Beller
2018-02-16 22:34     ` [PATCHv2 00/16] Moving global state into the repository object (part 1) Jonathan Nieder
2018-02-20 18:55       ` Stefan Beller
2018-02-20 19:00         ` Brandon Williams
2018-02-20 19:03           ` Stefan Beller
2018-02-20 19:03       ` Junio C Hamano
2018-02-20 19:06         ` Stefan Beller
2018-02-20 19:55           ` Junio C Hamano
2018-02-20 20:16             ` Stefan Beller
2018-02-21  1:54     ` [PATCHv3 00/27] " Stefan Beller
2018-02-21  1:54       ` [PATCH 01/27] repository: introduce raw object store field Stefan Beller
2018-02-21  1:54       ` [PATCH 02/27] object-store: migrate alternates struct and functions from cache.h Stefan Beller
2018-02-21  1:54       ` [PATCH 03/27] object-store: move alt_odb_list and alt_odb_tail to object store Stefan Beller
2018-02-21  1:54       ` [PATCH 04/27] object-store: free alt_odb_list Stefan Beller
2018-02-21  1:54       ` [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store Stefan Beller
2018-02-21  1:54       ` [PATCH 06/27] object-store: close all packs upon clearing the " Stefan Beller
2018-02-21  1:54       ` [PATCH 07/27] pack: move prepare_packed_git_run_once to " Stefan Beller
2018-02-21  1:54       ` [PATCH 08/27] pack: move approximate object count " Stefan Beller
2018-02-21  1:54       ` [PATCH 09/27] sha1_file: add raw_object_store argument to alt_odb_usable Stefan Beller
2018-02-21  1:54       ` [PATCH 10/27] sha1_file: add repository argument to link_alt_odb_entry Stefan Beller
2018-02-21  1:54       ` [PATCH 11/27] sha1_file: add repository argument to read_info_alternates Stefan Beller
2018-02-21  1:54       ` [PATCH 12/27] sha1_file: add repository argument to link_alt_odb_entries Stefan Beller
2018-02-21  1:54       ` [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb Stefan Beller
2018-02-21  1:54       ` [PATCH 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories Stefan Beller
2018-02-21  1:54       ` [PATCH 15/27] sha1_file: allow prepare_alt_odb " Stefan Beller
2018-02-21  1:54       ` [PATCH 16/27] sha1_file: add repository argument to sha1_file_name Stefan Beller
2018-02-21  1:54       ` [PATCH 17/27] sha1_file: add repository argument to stat_sha1_file Stefan Beller
2018-02-21  1:54       ` [PATCH 18/27] sha1_file: add repository argument to open_sha1_file Stefan Beller
2018-02-21  1:54       ` [PATCH 19/27] sha1_file: add repository argument to map_sha1_file_1 Stefan Beller
2018-02-21  1:54       ` [PATCH 20/27] sha1_file: add repository argument to map_sha1_file Stefan Beller
2018-02-21  1:54       ` [PATCH 21/27] sha1_file: add repository argument to sha1_loose_object_info Stefan Beller
2018-02-21  1:54       ` [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories Stefan Beller
2018-02-21  1:54       ` [PATCH 23/27] sha1_file: allow stat_sha1_file " Stefan Beller
2018-02-21  1:54       ` [PATCH 24/27] sha1_file: allow open_sha1_file " Stefan Beller
2018-02-21  1:54       ` [PATCH 25/27] sha1_file: allow map_sha1_file_1 " Stefan Beller
2018-02-21  1:54       ` [PATCH 26/27] sha1_file: allow map_sha1_file " Stefan Beller
2018-02-21  1:54       ` [PATCH 27/27] sha1_file: allow sha1_loose_object_info " Stefan Beller

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox