* [PATCH v1 01/11] fetch-object: make functions return an error code
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 02/11] Add initial remote odb support Christian Couder
` (9 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
The callers of the fetch_object() and fetch_objects() might
be interested in knowing if these functions succeeded or not.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
fetch-object.c | 15 +++++++++------
fetch-object.h | 6 +++---
sha1-file.c | 4 ++--
3 files changed, 14 insertions(+), 11 deletions(-)
diff --git a/fetch-object.c b/fetch-object.c
index 853624f811..ccc4ea7f1a 100644
--- a/fetch-object.c
+++ b/fetch-object.c
@@ -5,11 +5,12 @@
#include "transport.h"
#include "fetch-object.h"
-static void fetch_refs(const char *remote_name, struct ref *ref)
+static int fetch_refs(const char *remote_name, struct ref *ref)
{
struct remote *remote;
struct transport *transport;
int original_fetch_if_missing = fetch_if_missing;
+ int res;
fetch_if_missing = 0;
remote = remote_get(remote_name);
@@ -19,18 +20,20 @@ static void fetch_refs(const char *remote_name, struct ref *ref)
transport_set_option(transport, TRANS_OPT_FROM_PROMISOR, "1");
transport_set_option(transport, TRANS_OPT_NO_DEPENDENTS, "1");
- transport_fetch_refs(transport, ref);
+ res = transport_fetch_refs(transport, ref);
fetch_if_missing = original_fetch_if_missing;
+
+ return res;
}
-void fetch_object(const char *remote_name, const unsigned char *sha1)
+int fetch_object(const char *remote_name, const unsigned char *sha1)
{
struct ref *ref = alloc_ref(sha1_to_hex(sha1));
hashcpy(ref->old_oid.hash, sha1);
- fetch_refs(remote_name, ref);
+ return fetch_refs(remote_name, ref);
}
-void fetch_objects(const char *remote_name, const struct oid_array *to_fetch)
+int fetch_objects(const char *remote_name, const struct oid_array *to_fetch)
{
struct ref *ref = NULL;
int i;
@@ -41,5 +44,5 @@ void fetch_objects(const char *remote_name, const struct oid_array *to_fetch)
new_ref->next = ref;
ref = new_ref;
}
- fetch_refs(remote_name, ref);
+ return fetch_refs(remote_name, ref);
}
diff --git a/fetch-object.h b/fetch-object.h
index 4b269d07ed..12e1f9ee70 100644
--- a/fetch-object.h
+++ b/fetch-object.h
@@ -3,9 +3,9 @@
#include "sha1-array.h"
-extern void fetch_object(const char *remote_name, const unsigned char *sha1);
+extern int fetch_object(const char *remote_name, const unsigned char *sha1);
-extern void fetch_objects(const char *remote_name,
- const struct oid_array *to_fetch);
+extern int fetch_objects(const char *remote_name,
+ const struct oid_array *to_fetch);
#endif
diff --git a/sha1-file.c b/sha1-file.c
index 695e5c6276..374a56d0e1 100644
--- a/sha1-file.c
+++ b/sha1-file.c
@@ -1312,8 +1312,8 @@ int oid_object_info_extended(struct repository *r, const struct object_id *oid,
if (fetch_if_missing && repository_format_partial_clone &&
!already_retried && r == the_repository) {
/*
- * TODO Investigate having fetch_object() return
- * TODO error/success and stopping the music here.
+ * TODO Investigate checking fetch_object() return
+ * TODO value and stopping on error here.
* TODO Pass a repository struct through fetch_object,
* such that arbitrary repositories work.
*/
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 02/11] Add initial remote odb support
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
2018-06-23 12:18 ` [PATCH v1 01/11] fetch-object: make functions return an error code Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 03/11] remote-odb: implement remote_odb_get_direct() Christian Couder
` (8 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
The remote-odb.{c,h} files will contain the functions
that are called by the rest of Git mostly from
"sha1-file.c" to access the objects managed by the
remote odbs.
The odb-helper.{c,h} files will contain the functions to
actually implement communication with either the internal
functions or the external scripts or processes that will
manage and provide remote git objects.
For now only infrastructure to create helpers from the
config and to check if there are remote odbs and odb
helpers is implemented.
We expect that there will not be a lot of helpers, so it
is ok to use a simple linked list to manage them.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
Makefile | 2 ++
odb-helper.c | 16 ++++++++++++
odb-helper.h | 19 ++++++++++++++
remote-odb.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++
remote-odb.h | 7 +++++
5 files changed, 116 insertions(+)
create mode 100644 odb-helper.c
create mode 100644 odb-helper.h
create mode 100644 remote-odb.c
create mode 100644 remote-odb.h
diff --git a/Makefile b/Makefile
index e4b503d259..8465e3ccd4 100644
--- a/Makefile
+++ b/Makefile
@@ -896,6 +896,8 @@ LIB_OBJS += notes-cache.o
LIB_OBJS += notes-merge.o
LIB_OBJS += notes-utils.o
LIB_OBJS += object.o
+LIB_OBJS += odb-helper.o
+LIB_OBJS += remote-odb.o
LIB_OBJS += oidmap.o
LIB_OBJS += oidset.o
LIB_OBJS += packfile.o
diff --git a/odb-helper.c b/odb-helper.c
new file mode 100644
index 0000000000..b4d403ffa9
--- /dev/null
+++ b/odb-helper.c
@@ -0,0 +1,16 @@
+#include "cache.h"
+#include "object.h"
+#include "argv-array.h"
+#include "odb-helper.h"
+#include "run-command.h"
+#include "sha1-lookup.h"
+
+struct odb_helper *odb_helper_new(const char *name, int namelen)
+{
+ struct odb_helper *o;
+
+ o = xcalloc(1, sizeof(*o));
+ o->name = xmemdupz(name, namelen);
+
+ return o;
+}
diff --git a/odb-helper.h b/odb-helper.h
new file mode 100644
index 0000000000..4b792a3896
--- /dev/null
+++ b/odb-helper.h
@@ -0,0 +1,19 @@
+#ifndef ODB_HELPER_H
+#define ODB_HELPER_H
+
+/*
+ * An odb helper is a way to access a remote odb.
+ *
+ * Information in its fields comes from the config file [odb "NAME"]
+ * entries.
+ */
+struct odb_helper {
+ const char *name; /* odb.<NAME>.* */
+ const char *remote; /* odb.<NAME>.promisorRemote */
+
+ struct odb_helper *next;
+};
+
+extern struct odb_helper *odb_helper_new(const char *name, int namelen);
+
+#endif /* ODB_HELPER_H */
diff --git a/remote-odb.c b/remote-odb.c
new file mode 100644
index 0000000000..1dc165eaf6
--- /dev/null
+++ b/remote-odb.c
@@ -0,0 +1,72 @@
+#include "cache.h"
+#include "remote-odb.h"
+#include "odb-helper.h"
+#include "config.h"
+
+static struct odb_helper *helpers;
+static struct odb_helper **helpers_tail = &helpers;
+
+static struct odb_helper *find_or_create_helper(const char *name, int len)
+{
+ struct odb_helper *o;
+
+ for (o = helpers; o; o = o->next)
+ if (!strncmp(o->name, name, len) && !o->name[len])
+ return o;
+
+ o = odb_helper_new(name, len);
+ *helpers_tail = o;
+ helpers_tail = &o->next;
+
+ return o;
+}
+
+static int remote_odb_config(const char *var, const char *value, void *data)
+{
+ struct odb_helper *o;
+ const char *name;
+ int namelen;
+ const char *subkey;
+
+ if (parse_config_key(var, "odb", &name, &namelen, &subkey) < 0)
+ return 0;
+
+ o = find_or_create_helper(name, namelen);
+
+ if (!strcmp(subkey, "promisorremote"))
+ return git_config_string(&o->remote, var, value);
+
+ return 0;
+}
+
+static void remote_odb_init(void)
+{
+ static int initialized;
+
+ if (initialized)
+ return;
+ initialized = 1;
+
+ git_config(remote_odb_config, NULL);
+}
+
+struct odb_helper *find_odb_helper(const char *remote)
+{
+ struct odb_helper *o;
+
+ remote_odb_init();
+
+ if (!remote)
+ return helpers;
+
+ for (o = helpers; o; o = o->next)
+ if (!strcmp(o->remote, remote))
+ return o;
+
+ return NULL;
+}
+
+int has_remote_odb(void)
+{
+ return !!find_odb_helper(NULL);
+}
diff --git a/remote-odb.h b/remote-odb.h
new file mode 100644
index 0000000000..4c7b13695f
--- /dev/null
+++ b/remote-odb.h
@@ -0,0 +1,7 @@
+#ifndef REMOTE_ODB_H
+#define REMOTE_ODB_H
+
+extern struct odb_helper *find_odb_helper(const char *remote);
+extern int has_remote_odb(void);
+
+#endif /* REMOTE_ODB_H */
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 03/11] remote-odb: implement remote_odb_get_direct()
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
2018-06-23 12:18 ` [PATCH v1 01/11] fetch-object: make functions return an error code Christian Couder
2018-06-23 12:18 ` [PATCH v1 02/11] Add initial remote odb support Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 04/11] remote-odb: implement remote_odb_get_many_direct() Christian Couder
` (7 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
This is implemented only in the promisor remote mode
for now by calling fetch_object().
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
odb-helper.c | 14 ++++++++++++++
odb-helper.h | 3 ++-
remote-odb.c | 17 +++++++++++++++++
remote-odb.h | 1 +
4 files changed, 34 insertions(+), 1 deletion(-)
diff --git a/odb-helper.c b/odb-helper.c
index b4d403ffa9..99b5a345e8 100644
--- a/odb-helper.c
+++ b/odb-helper.c
@@ -4,6 +4,7 @@
#include "odb-helper.h"
#include "run-command.h"
#include "sha1-lookup.h"
+#include "fetch-object.h"
struct odb_helper *odb_helper_new(const char *name, int namelen)
{
@@ -14,3 +15,16 @@ struct odb_helper *odb_helper_new(const char *name, int namelen)
return o;
}
+
+int odb_helper_get_direct(struct odb_helper *o,
+ const unsigned char *sha1)
+{
+ int res;
+ uint64_t start = getnanotime();
+
+ res = fetch_object(o->remote, sha1);
+
+ trace_performance_since(start, "odb_helper_get_direct");
+
+ return res;
+}
diff --git a/odb-helper.h b/odb-helper.h
index 4b792a3896..4c52e81ce8 100644
--- a/odb-helper.h
+++ b/odb-helper.h
@@ -15,5 +15,6 @@ struct odb_helper {
};
extern struct odb_helper *odb_helper_new(const char *name, int namelen);
-
+extern int odb_helper_get_direct(struct odb_helper *o,
+ const unsigned char *sha1);
#endif /* ODB_HELPER_H */
diff --git a/remote-odb.c b/remote-odb.c
index 1dc165eaf6..a34537c05c 100644
--- a/remote-odb.c
+++ b/remote-odb.c
@@ -70,3 +70,20 @@ int has_remote_odb(void)
{
return !!find_odb_helper(NULL);
}
+
+int remote_odb_get_direct(const unsigned char *sha1)
+{
+ struct odb_helper *o;
+
+ trace_printf("trace: remote_odb_get_direct: %s", sha1_to_hex(sha1));
+
+ remote_odb_init();
+
+ for (o = helpers; o; o = o->next) {
+ if (odb_helper_get_direct(o, sha1) < 0)
+ continue;
+ return 0;
+ }
+
+ return -1;
+}
diff --git a/remote-odb.h b/remote-odb.h
index 4c7b13695f..c5384c922d 100644
--- a/remote-odb.h
+++ b/remote-odb.h
@@ -3,5 +3,6 @@
extern struct odb_helper *find_odb_helper(const char *remote);
extern int has_remote_odb(void);
+extern int remote_odb_get_direct(const unsigned char *sha1);
#endif /* REMOTE_ODB_H */
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 04/11] remote-odb: implement remote_odb_get_many_direct()
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (2 preceding siblings ...)
2018-06-23 12:18 ` [PATCH v1 03/11] remote-odb: implement remote_odb_get_direct() Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 05/11] remote-odb: add remote_odb_reinit() Christian Couder
` (6 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
This function will be used to get many objects from a promisor
remote.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
odb-helper.c | 15 +++++++++++++++
odb-helper.h | 3 +++
remote-odb.c | 17 +++++++++++++++++
remote-odb.h | 1 +
4 files changed, 36 insertions(+)
diff --git a/odb-helper.c b/odb-helper.c
index 99b5a345e8..246ebf8f7a 100644
--- a/odb-helper.c
+++ b/odb-helper.c
@@ -28,3 +28,18 @@ int odb_helper_get_direct(struct odb_helper *o,
return res;
}
+
+int odb_helper_get_many_direct(struct odb_helper *o,
+ const struct oid_array *to_get)
+{
+ int res;
+ uint64_t start;
+
+ start = getnanotime();
+
+ res = fetch_objects(o->remote, to_get);
+
+ trace_performance_since(start, "odb_helper_get_many_direct");
+
+ return res;
+}
diff --git a/odb-helper.h b/odb-helper.h
index 4c52e81ce8..154ce4c7e4 100644
--- a/odb-helper.h
+++ b/odb-helper.h
@@ -17,4 +17,7 @@ struct odb_helper {
extern struct odb_helper *odb_helper_new(const char *name, int namelen);
extern int odb_helper_get_direct(struct odb_helper *o,
const unsigned char *sha1);
+extern int odb_helper_get_many_direct(struct odb_helper *o,
+ const struct oid_array *to_get);
+
#endif /* ODB_HELPER_H */
diff --git a/remote-odb.c b/remote-odb.c
index a34537c05c..cc7059b23a 100644
--- a/remote-odb.c
+++ b/remote-odb.c
@@ -87,3 +87,20 @@ int remote_odb_get_direct(const unsigned char *sha1)
return -1;
}
+
+int remote_odb_get_many_direct(const struct oid_array *to_get)
+{
+ struct odb_helper *o;
+
+ trace_printf("trace: remote_odb_get_many_direct: nr: %d", to_get->nr);
+
+ remote_odb_init();
+
+ for (o = helpers; o; o = o->next) {
+ if (odb_helper_get_many_direct(o, to_get) < 0)
+ continue;
+ return 0;
+ }
+
+ return -1;
+}
diff --git a/remote-odb.h b/remote-odb.h
index c5384c922d..e10a8bf855 100644
--- a/remote-odb.h
+++ b/remote-odb.h
@@ -4,5 +4,6 @@
extern struct odb_helper *find_odb_helper(const char *remote);
extern int has_remote_odb(void);
extern int remote_odb_get_direct(const unsigned char *sha1);
+extern int remote_odb_get_many_direct(const struct oid_array *to_get);
#endif /* REMOTE_ODB_H */
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 05/11] remote-odb: add remote_odb_reinit()
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (3 preceding siblings ...)
2018-06-23 12:18 ` [PATCH v1 04/11] remote-odb: implement remote_odb_get_many_direct() Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 06/11] Use remote_odb_get_direct() and has_remote_odb() Christian Couder
` (5 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
We will need to reinitialize the remote odb configuration
as we will make some changes to it in a later commit when
we will detect that a remote is also a remote odb.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
remote-odb.c | 14 ++++++++++++--
remote-odb.h | 1 +
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/remote-odb.c b/remote-odb.c
index cc7059b23a..2b93d13abd 100644
--- a/remote-odb.c
+++ b/remote-odb.c
@@ -39,17 +39,27 @@ static int remote_odb_config(const char *var, const char *value, void *data)
return 0;
}
-static void remote_odb_init(void)
+static void remote_odb_do_init(int force)
{
static int initialized;
- if (initialized)
+ if (!force && initialized)
return;
initialized = 1;
git_config(remote_odb_config, NULL);
}
+static inline void remote_odb_init(void)
+{
+ remote_odb_do_init(0);
+}
+
+inline void remote_odb_reinit(void)
+{
+ remote_odb_do_init(1);
+}
+
struct odb_helper *find_odb_helper(const char *remote)
{
struct odb_helper *o;
diff --git a/remote-odb.h b/remote-odb.h
index e10a8bf855..79803782af 100644
--- a/remote-odb.h
+++ b/remote-odb.h
@@ -1,6 +1,7 @@
#ifndef REMOTE_ODB_H
#define REMOTE_ODB_H
+extern void remote_odb_reinit(void);
extern struct odb_helper *find_odb_helper(const char *remote);
extern int has_remote_odb(void);
extern int remote_odb_get_direct(const unsigned char *sha1);
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 06/11] Use remote_odb_get_direct() and has_remote_odb()
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (4 preceding siblings ...)
2018-06-23 12:18 ` [PATCH v1 05/11] remote-odb: add remote_odb_reinit() Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 07/11] Use odb.origin.partialclonefilter instead of core.partialclonefilter Christian Couder
` (4 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
Instead of using the repository_format_partial_clone global
and fetch_object() directly, let's use has_remote_odb() and
remote_odb_get_direct().
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
builtin/cat-file.c | 5 +++--
builtin/fetch.c | 11 ++++++-----
builtin/gc.c | 3 ++-
builtin/repack.c | 3 ++-
cache.h | 2 --
connected.c | 3 ++-
environment.c | 1 -
list-objects-filter-options.c | 27 +++++++++++++++------------
packfile.c | 3 ++-
setup.c | 7 +------
sha1-file.c | 14 ++++++++------
t/t0410-partial-clone.sh | 30 +++++++++++++++---------------
t/t5500-fetch-pack.sh | 4 ++--
t/t5601-clone.sh | 2 +-
t/t5616-partial-clone.sh | 2 +-
unpack-trees.c | 6 +++---
16 files changed, 63 insertions(+), 60 deletions(-)
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 665b581949..3b24c1b9b7 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -13,6 +13,7 @@
#include "tree-walk.h"
#include "sha1-array.h"
#include "packfile.h"
+#include "remote-odb.h"
struct batch_options {
int enabled;
@@ -477,8 +478,8 @@ static int batch_objects(struct batch_options *opt)
for_each_loose_object(batch_loose_object, &sa, 0);
for_each_packed_object(batch_packed_object, &sa, 0);
- if (repository_format_partial_clone)
- warning("This repository has extensions.partialClone set. Some objects may not be loaded.");
+ if (has_remote_odb())
+ warning("This repository uses an odb. Some objects may not be loaded.");
cb.opt = opt;
cb.expand = &data;
diff --git a/builtin/fetch.c b/builtin/fetch.c
index ea5b9669ad..1fd4dfd024 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -21,6 +21,7 @@
#include "utf8.h"
#include "packfile.h"
#include "list-objects-filter-options.h"
+#include "remote-odb.h"
static const char * const builtin_fetch_usage[] = {
N_("git fetch [<options>] [<repository> [<refspec>...]]"),
@@ -1313,7 +1314,7 @@ static inline void fetch_one_setup_partial(struct remote *remote)
* If no prior partial clone/fetch and the current fetch DID NOT
* request a partial-fetch, do a normal fetch.
*/
- if (!repository_format_partial_clone && !filter_options.choice)
+ if (!has_remote_odb() && !filter_options.choice)
return;
/*
@@ -1321,7 +1322,7 @@ static inline void fetch_one_setup_partial(struct remote *remote)
* on this repo and remember the given filter-spec as the default
* for subsequent fetches to this remote.
*/
- if (!repository_format_partial_clone && filter_options.choice) {
+ if (!has_remote_odb() && filter_options.choice) {
partial_clone_register(remote->name, &filter_options);
return;
}
@@ -1330,7 +1331,7 @@ static inline void fetch_one_setup_partial(struct remote *remote)
* We are currently limited to only ONE promisor remote and only
* allow partial-fetches from the promisor remote.
*/
- if (strcmp(remote->name, repository_format_partial_clone)) {
+ if (!find_odb_helper(remote->name)) {
if (filter_options.choice)
die(_("--filter can only be used with the remote configured in core.partialClone"));
return;
@@ -1461,7 +1462,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
if (depth || deepen_since || deepen_not.nr)
deepen = 1;
- if (filter_options.choice && !repository_format_partial_clone)
+ if (filter_options.choice && !has_remote_odb())
die("--filter can only be used when extensions.partialClone is set");
if (all) {
@@ -1495,7 +1496,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
}
if (remote) {
- if (filter_options.choice || repository_format_partial_clone)
+ if (filter_options.choice || has_remote_odb())
fetch_one_setup_partial(remote);
result = fetch_one(remote, argc, argv, prune_tags_ok);
} else {
diff --git a/builtin/gc.c b/builtin/gc.c
index ccfb1ceaeb..02c783b514 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -26,6 +26,7 @@
#include "pack-objects.h"
#include "blob.h"
#include "tree.h"
+#include "remote-odb.h"
#define FAILED_RUN "failed to run %s"
@@ -619,7 +620,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
argv_array_push(&prune, prune_expire);
if (quiet)
argv_array_push(&prune, "--no-progress");
- if (repository_format_partial_clone)
+ if (has_remote_odb())
argv_array_push(&prune,
"--exclude-promisor-objects");
if (run_command_v_opt(prune.argv, RUN_GIT_CMD))
diff --git a/builtin/repack.c b/builtin/repack.c
index 6c636e159e..72b1e2d94f 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -8,6 +8,7 @@
#include "strbuf.h"
#include "string-list.h"
#include "argv-array.h"
+#include "remote-odb.h"
static int delta_base_offset = 1;
static int pack_kept_objects = -1;
@@ -249,7 +250,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
argv_array_push(&cmd.args, "--all");
argv_array_push(&cmd.args, "--reflog");
argv_array_push(&cmd.args, "--indexed-objects");
- if (repository_format_partial_clone)
+ if (has_remote_odb())
argv_array_push(&cmd.args, "--exclude-promisor-objects");
if (window)
argv_array_pushf(&cmd.args, "--window=%s", window);
diff --git a/cache.h b/cache.h
index 89a107a7f7..2855435798 100644
--- a/cache.h
+++ b/cache.h
@@ -904,13 +904,11 @@ extern int grafts_replace_parents;
#define GIT_REPO_VERSION 0
#define GIT_REPO_VERSION_READ 1
extern int repository_format_precious_objects;
-extern char *repository_format_partial_clone;
extern const char *core_partial_clone_filter_default;
struct repository_format {
int version;
int precious_objects;
- char *partial_clone; /* value of extensions.partialclone */
int is_bare;
int hash_algo;
char *work_tree;
diff --git a/connected.c b/connected.c
index 91feb78815..07154a358d 100644
--- a/connected.c
+++ b/connected.c
@@ -4,6 +4,7 @@
#include "connected.h"
#include "transport.h"
#include "packfile.h"
+#include "remote-odb.h"
/*
* If we feed all the commits we want to verify to this command
@@ -56,7 +57,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
argv_array_push(&rev_list.args,"rev-list");
argv_array_push(&rev_list.args, "--objects");
argv_array_push(&rev_list.args, "--stdin");
- if (repository_format_partial_clone)
+ if (has_remote_odb())
argv_array_push(&rev_list.args, "--exclude-promisor-objects");
argv_array_push(&rev_list.args, "--not");
argv_array_push(&rev_list.args, "--all");
diff --git a/environment.c b/environment.c
index 2a6de2330b..f8048f4a5d 100644
--- a/environment.c
+++ b/environment.c
@@ -30,7 +30,6 @@ int warn_ambiguous_refs = 1;
int warn_on_object_refname_ambiguity = 1;
int ref_paranoia = -1;
int repository_format_precious_objects;
-char *repository_format_partial_clone;
const char *core_partial_clone_filter_default;
const char *git_commit_encoding;
const char *git_log_output_encoding;
diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c
index 6a3cc985c4..8e7b9ae776 100644
--- a/list-objects-filter-options.c
+++ b/list-objects-filter-options.c
@@ -6,6 +6,7 @@
#include "list-objects.h"
#include "list-objects-filter.h"
#include "list-objects-filter-options.h"
+#include "remote-odb.h"
/*
* Parse value of the argument to the "filter" keyword.
@@ -114,30 +115,32 @@ void partial_clone_register(
const char *remote,
const struct list_objects_filter_options *filter_options)
{
- /*
- * Record the name of the partial clone remote in the
- * config and in the global variable -- the latter is
- * used throughout to indicate that partial clone is
- * enabled and to expect missing objects.
- */
- if (repository_format_partial_clone &&
- *repository_format_partial_clone &&
- strcmp(remote, repository_format_partial_clone))
- die(_("cannot change partial clone promisor remote"));
+ char *cfg_name;
+
+ /* Check if it is already registered */
+ if (find_odb_helper(remote))
+ return;
git_config_set("core.repositoryformatversion", "1");
- git_config_set("extensions.partialclone", remote);
- repository_format_partial_clone = xstrdup(remote);
+ /* Add odb config for the remote */
+ cfg_name = xstrfmt("odb.%s.promisorRemote", remote);
+ git_config_set(cfg_name, remote);
+ free(cfg_name);
/*
* Record the initial filter-spec in the config as
* the default for subsequent fetches from this remote.
+ *
+ * TODO: move core.partialclonefilter into odb.<name>
*/
core_partial_clone_filter_default =
xstrdup(filter_options->filter_spec);
git_config_set("core.partialclonefilter",
core_partial_clone_filter_default);
+
+ /* Make sure the config info are reset */
+ remote_odb_reinit();
}
void partial_clone_get_default_filter_spec(
diff --git a/packfile.c b/packfile.c
index 7cd45aa4b2..9cbc974612 100644
--- a/packfile.c
+++ b/packfile.c
@@ -15,6 +15,7 @@
#include "tree-walk.h"
#include "tree.h"
#include "object-store.h"
+#include "remote-odb.h"
char *odb_pack_name(struct strbuf *buf,
const unsigned char *sha1,
@@ -1976,7 +1977,7 @@ int is_promisor_object(const struct object_id *oid)
static int promisor_objects_prepared;
if (!promisor_objects_prepared) {
- if (repository_format_partial_clone) {
+ if (has_remote_odb()) {
for_each_packed_object(add_promisor_object,
&promisor_objects,
FOR_EACH_OBJECT_PROMISOR_ONLY);
diff --git a/setup.c b/setup.c
index b24c811c1c..4eeb7fe073 100644
--- a/setup.c
+++ b/setup.c
@@ -419,11 +419,7 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
;
else if (!strcmp(ext, "preciousobjects"))
data->precious_objects = git_config_bool(var, value);
- else if (!strcmp(ext, "partialclone")) {
- if (!value)
- return config_error_nonbool(var);
- data->partial_clone = xstrdup(value);
- } else
+ else
string_list_append(&data->unknown_extensions, ext);
} else if (strcmp(var, "core.bare") == 0) {
data->is_bare = git_config_bool(var, value);
@@ -465,7 +461,6 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
}
repository_format_precious_objects = candidate->precious_objects;
- repository_format_partial_clone = candidate->partial_clone;
string_list_clear(&candidate->unknown_extensions, 0);
if (!has_common) {
if (candidate->is_bare != -1) {
diff --git a/sha1-file.c b/sha1-file.c
index 374a56d0e1..1fb95790de 100644
--- a/sha1-file.c
+++ b/sha1-file.c
@@ -32,6 +32,7 @@
#include "packfile.h"
#include "fetch-object.h"
#include "object-store.h"
+#include "remote-odb.h"
/* The maximum size for an object header. */
#define MAX_HEADER_LEN 32
@@ -1309,15 +1310,16 @@ int oid_object_info_extended(struct repository *r, const struct object_id *oid,
}
/* Check if it is a missing object */
- if (fetch_if_missing && repository_format_partial_clone &&
+ if (fetch_if_missing && has_remote_odb() &&
!already_retried && r == the_repository) {
/*
- * TODO Investigate checking fetch_object() return
- * TODO value and stopping on error here.
- * TODO Pass a repository struct through fetch_object,
- * such that arbitrary repositories work.
+ * TODO Investigate checking remote_odb_get_direct()
+ * TODO return value and stopping on error here.
+ * TODO Pass a repository struct through
+ * remote_odb_get_direct(), such that arbitrary
+ * repositories work.
*/
- fetch_object(repository_format_partial_clone, real->hash);
+ remote_odb_get_direct(real->hash);
already_retried = 1;
continue;
}
diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index cc18b75c03..6af4712da8 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -43,7 +43,7 @@ test_expect_success 'missing reflog object, but promised by a commit, passes fsc
# But with the extension, it succeeds
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo fsck
'
@@ -66,7 +66,7 @@ test_expect_success 'missing reflog object, but promised by a tag, passes fsck'
printf "$T\n" | pack_as_from_promisor &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo fsck
'
@@ -84,7 +84,7 @@ test_expect_success 'missing reflog object alone fails fsck, even with extension
delete_object repo "$A" &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
test_must_fail git -C repo fsck
'
@@ -100,7 +100,7 @@ test_expect_success 'missing ref object, but promised, passes fsck' '
promise_and_delete "$A" &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo fsck
'
@@ -123,7 +123,7 @@ test_expect_success 'missing object, but promised, passes fsck' '
promise_and_delete "$AT" &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo fsck
'
@@ -136,7 +136,7 @@ test_expect_success 'missing CLI object, but promised, passes fsck' '
promise_and_delete "$A" &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo fsck "$A"
'
@@ -151,7 +151,7 @@ test_expect_success 'fetching of missing objects' '
rm -rf repo/.git/objects/* &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "origin" &&
+ git -C repo config odb.magic.promisorRemote "origin" &&
git -C repo cat-file -p "$HASH" &&
# Ensure that the .promisor file is written, and check that its
@@ -172,7 +172,7 @@ test_expect_success 'rev-list stops traversal at missing and promised commit' '
promise_and_delete "$FOO" &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo rev-list --exclude-promisor-objects --objects bar >out &&
grep $(git -C repo rev-parse bar) out &&
! grep $FOO out
@@ -197,7 +197,7 @@ test_expect_success 'rev-list stops traversal at missing and promised tree' '
promise_and_delete "$TREE2" &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo rev-list --exclude-promisor-objects --objects HEAD >out &&
grep $(git -C repo rev-parse foo) out &&
! grep $TREE out &&
@@ -216,7 +216,7 @@ test_expect_success 'rev-list stops traversal at missing and promised blob' '
promise_and_delete "$BLOB" &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo rev-list --exclude-promisor-objects --objects HEAD >out &&
grep $(git -C repo rev-parse HEAD) out &&
! grep $BLOB out
@@ -235,7 +235,7 @@ test_expect_success 'rev-list stops traversal at promisor commit, tree, and blob
printf "%s\n%s\n%s\n" $COMMIT $TREE $BLOB | pack_as_from_promisor &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo rev-list --exclude-promisor-objects --objects HEAD >out &&
! grep $COMMIT out &&
! grep $TREE out &&
@@ -259,7 +259,7 @@ test_expect_success 'rev-list accepts missing and promised objects on command li
promise_and_delete $BLOB &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo rev-list --exclude-promisor-objects --objects "$COMMIT" "$TREE" "$BLOB"
'
@@ -272,7 +272,7 @@ test_expect_success 'gc does not repack promisor objects' '
HASH=$(printf "$TREE_HASH\n" | pack_as_from_promisor) &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo gc &&
# Ensure that the promisor packfile still exists, and remove it
@@ -296,7 +296,7 @@ test_expect_success 'gc stops traversal when a missing but promised object is re
HASH=$(promise_and_delete $TREE_HASH) &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "arbitrary string" &&
+ git -C repo config odb.magic.promisorRemote "arbitrary string" &&
git -C repo gc &&
# Ensure that the promisor packfile still exists, and remove it
@@ -327,7 +327,7 @@ test_expect_success 'fetching of missing objects from an HTTP server' '
rm -rf repo/.git/objects/* &&
git -C repo config core.repositoryformatversion 1 &&
- git -C repo config extensions.partialclone "origin" &&
+ git -C repo config odb.magic.promisorRemote "origin" &&
git -C repo cat-file -p "$HASH" &&
# Ensure that the .promisor file is written, and check that its
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index d4f435155f..d57fa2dbe4 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -792,14 +792,14 @@ fetch_filter_blob_limit_zero () {
test_config -C "$SERVER" uploadpack.allowfilter 1 &&
git clone "$URL" client &&
- test_config -C client extensions.partialclone origin &&
+ test_config -C client odb.magic.promisorRemote origin &&
test_commit -C "$SERVER" two &&
git -C client fetch --filter=blob:limit=0 origin HEAD:somewhere &&
# Ensure that commit is fetched, but blob is not
- test_config -C client extensions.partialclone "arbitrary string" &&
+ test_config -C client odb.magic.promisorRemote "arbitrary string" &&
git -C client cat-file -e $(git -C "$SERVER" rev-parse two) &&
test_must_fail git -C client cat-file -e $(git hash-object "$SERVER/two.t")
}
diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
index 0b62037744..022c65a7ae 100755
--- a/t/t5601-clone.sh
+++ b/t/t5601-clone.sh
@@ -647,7 +647,7 @@ partial_clone () {
git -C client fsck &&
# Ensure that unneeded blobs are not inadvertently fetched.
- test_config -C client extensions.partialclone "not a remote" &&
+ test_config -C client odb.origin.promisorRemote "not a remote" &&
test_must_fail git -C client cat-file -e "$HASH1" &&
# But this blob was fetched, because clone performs an initial checkout
diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index cee5565367..5ddd1e011c 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -40,7 +40,7 @@ test_expect_success 'do partial clone 1' '
| sort >observed.oids &&
test_cmp expect_1.oids observed.oids &&
test "$(git -C pc1 config --local core.repositoryformatversion)" = "1" &&
- test "$(git -C pc1 config --local extensions.partialclone)" = "origin" &&
+ test "$(git -C pc1 config --local odb.origin.promisorRemote)" = "origin" &&
test "$(git -C pc1 config --local core.partialclonefilter)" = "blob:none"
'
diff --git a/unpack-trees.c b/unpack-trees.c
index 3a85a02a77..e6ac3e6d10 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -17,6 +17,7 @@
#include "submodule-config.h"
#include "fsmonitor.h"
#include "fetch-object.h"
+#include "remote-odb.h"
/*
* Error messages expected by scripts out of plumbing commands such as
@@ -382,7 +383,7 @@ static int check_updates(struct unpack_trees_options *o)
load_gitmodules_file(index, &state);
enable_delayed_checkout(&state);
- if (repository_format_partial_clone && o->update && !o->dry_run) {
+ if (has_remote_odb() && o->update && !o->dry_run) {
/*
* Prefetch the objects that are to be checked out in the loop
* below.
@@ -399,8 +400,7 @@ static int check_updates(struct unpack_trees_options *o)
}
}
if (to_fetch.nr)
- fetch_objects(repository_format_partial_clone,
- &to_fetch);
+ remote_odb_get_many_direct(&to_fetch);
fetch_if_missing = fetch_if_missing_store;
oid_array_clear(&to_fetch);
}
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 07/11] Use odb.origin.partialclonefilter instead of core.partialclonefilter
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (5 preceding siblings ...)
2018-06-23 12:18 ` [PATCH v1 06/11] Use remote_odb_get_direct() and has_remote_odb() Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 08/11] t0410: test fetching from many promisor remotes Christian Couder
` (3 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
Let's make the partial clone filter specific to one odb
instead of general to all the odbs.
This makes it possible to have different partial clone
filters for different odbs.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
builtin/fetch.c | 2 +-
list-objects-filter-options.c | 26 ++++++++++++++++----------
list-objects-filter-options.h | 3 ++-
odb-helper.h | 1 +
remote-odb.c | 2 ++
t/t5616-partial-clone.sh | 2 +-
6 files changed, 23 insertions(+), 13 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 1fd4dfd024..52ec2fd200 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1343,7 +1343,7 @@ static inline void fetch_one_setup_partial(struct remote *remote)
* the config.
*/
if (!filter_options.choice)
- partial_clone_get_default_filter_spec(&filter_options);
+ partial_clone_get_default_filter_spec(&filter_options, remote->name);
return;
}
diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c
index 8e7b9ae776..755a793664 100644
--- a/list-objects-filter-options.c
+++ b/list-objects-filter-options.c
@@ -7,6 +7,7 @@
#include "list-objects-filter.h"
#include "list-objects-filter-options.h"
#include "remote-odb.h"
+#include "odb-helper.h"
/*
* Parse value of the argument to the "filter" keyword.
@@ -29,6 +30,9 @@ static int gently_parse_list_objects_filter(
{
const char *v0;
+ if (!arg)
+ return 0;
+
if (filter_options->choice) {
if (errbuf) {
strbuf_init(errbuf, 0);
@@ -116,6 +120,7 @@ void partial_clone_register(
const struct list_objects_filter_options *filter_options)
{
char *cfg_name;
+ char *filter_name;
/* Check if it is already registered */
if (find_odb_helper(remote))
@@ -131,25 +136,26 @@ void partial_clone_register(
/*
* Record the initial filter-spec in the config as
* the default for subsequent fetches from this remote.
- *
- * TODO: move core.partialclonefilter into odb.<name>
*/
- core_partial_clone_filter_default =
- xstrdup(filter_options->filter_spec);
- git_config_set("core.partialclonefilter",
- core_partial_clone_filter_default);
+ filter_name = xstrfmt("odb.%s.partialclonefilter", remote);
+ git_config_set(filter_name, filter_options->filter_spec);
+ free(filter_name);
/* Make sure the config info are reset */
remote_odb_reinit();
}
void partial_clone_get_default_filter_spec(
- struct list_objects_filter_options *filter_options)
+ struct list_objects_filter_options *filter_options,
+ const char *remote)
{
+ struct odb_helper *helper = find_odb_helper(remote);
+
/*
* Parse default value, but silently ignore it if it is invalid.
*/
- gently_parse_list_objects_filter(filter_options,
- core_partial_clone_filter_default,
- NULL);
+ if (helper)
+ gently_parse_list_objects_filter(filter_options,
+ helper->partial_clone_filter,
+ NULL);
}
diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h
index 0000a61f82..12ceef3230 100644
--- a/list-objects-filter-options.h
+++ b/list-objects-filter-options.h
@@ -74,6 +74,7 @@ void partial_clone_register(
const char *remote,
const struct list_objects_filter_options *filter_options);
void partial_clone_get_default_filter_spec(
- struct list_objects_filter_options *filter_options);
+ struct list_objects_filter_options *filter_options,
+ const char *remote);
#endif /* LIST_OBJECTS_FILTER_OPTIONS_H */
diff --git a/odb-helper.h b/odb-helper.h
index 154ce4c7e4..4af75ef55c 100644
--- a/odb-helper.h
+++ b/odb-helper.h
@@ -10,6 +10,7 @@
struct odb_helper {
const char *name; /* odb.<NAME>.* */
const char *remote; /* odb.<NAME>.promisorRemote */
+ const char *partial_clone_filter; /* odb.<NAME>.partialCloneFilter */
struct odb_helper *next;
};
diff --git a/remote-odb.c b/remote-odb.c
index 2b93d13abd..5b9d1930b5 100644
--- a/remote-odb.c
+++ b/remote-odb.c
@@ -35,6 +35,8 @@ static int remote_odb_config(const char *var, const char *value, void *data)
if (!strcmp(subkey, "promisorremote"))
return git_config_string(&o->remote, var, value);
+ if (!strcmp(subkey, "partialclonefilter"))
+ return git_config_string(&o->partial_clone_filter, var, value);
return 0;
}
diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index 5ddd1e011c..4231121181 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -41,7 +41,7 @@ test_expect_success 'do partial clone 1' '
test_cmp expect_1.oids observed.oids &&
test "$(git -C pc1 config --local core.repositoryformatversion)" = "1" &&
test "$(git -C pc1 config --local odb.origin.promisorRemote)" = "origin" &&
- test "$(git -C pc1 config --local core.partialclonefilter)" = "blob:none"
+ test "$(git -C pc1 config --local odb.origin.partialclonefilter)" = "blob:none"
'
# checkout master to force dynamic object fetch of blobs at HEAD.
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 08/11] t0410: test fetching from many promisor remotes
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (6 preceding siblings ...)
2018-06-23 12:18 ` [PATCH v1 07/11] Use odb.origin.partialclonefilter instead of core.partialclonefilter Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:18 ` [PATCH v1 09/11] Documentation/config: add odb.<name>.promisorRemote Christian Couder
` (2 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/t0410-partial-clone.sh | 24 +++++++++++++++++++++++-
1 file changed, 23 insertions(+), 1 deletion(-)
diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index 6af4712da8..4a7a662512 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -162,8 +162,30 @@ test_expect_success 'fetching of missing objects' '
git verify-pack --verbose "$IDX" | grep "$HASH"
'
+test_expect_success 'fetching of missing objects from another odb remote' '
+ git clone "file://$(pwd)/server" server2 &&
+ test_commit -C server2 bar &&
+ git -C server2 repack -a -d --write-bitmap-index &&
+ HASH2=$(git -C server2 rev-parse bar) &&
+
+ git -C repo remote add server2 "file://$(pwd)/server2" &&
+ git -C repo config odb.magic2.promisorRemote server2 &&
+ git -C repo cat-file -p "$HASH2" &&
+
+ git -C repo fetch server2 &&
+ rm -rf repo/.git/objects/* &&
+ git -C repo cat-file -p "$HASH2" &&
+
+ # Ensure that the .promisor file is written, and check that its
+ # associated packfile contains the object
+ ls repo/.git/objects/pack/pack-*.promisor >promisorlist &&
+ test_line_count = 1 promisorlist &&
+ IDX=$(cat promisorlist | sed "s/promisor$/idx/") &&
+ git verify-pack --verbose "$IDX" | grep "$HASH2"
+'
+
test_expect_success 'rev-list stops traversal at missing and promised commit' '
- rm -rf repo &&
+ rm -rf repo server server2 &&
test_create_repo repo &&
test_commit -C repo foo &&
test_commit -C repo bar &&
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 09/11] Documentation/config: add odb.<name>.promisorRemote
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (7 preceding siblings ...)
2018-06-23 12:18 ` [PATCH v1 08/11] t0410: test fetching from many promisor remotes Christian Couder
@ 2018-06-23 12:18 ` Christian Couder
2018-06-23 12:29 ` [PATCH v1 0/9] Introducing remote ODBs Christian Couder
2018-06-25 21:49 ` Junio C Hamano
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:18 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
---
Documentation/config.txt | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index ab641bf5a9..8df0c7177e 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2504,6 +2504,11 @@ This setting can be overridden with the `GIT_NOTES_REWRITE_REF`
environment variable, which must be a colon separated list of refs or
globs.
+odb.<name>.promisorRemote::
+ The name of a promisor remote. For now promisor remotes are
+ the only kind of remote object database (odb) that is
+ supported.
+
pack.window::
The size of the window used by linkgit:git-pack-objects[1] when no
window size is given on the command line. Defaults to 10.
--
2.17.0.762.g886f54ea46
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v1 0/9] Introducing remote ODBs
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (8 preceding siblings ...)
2018-06-23 12:18 ` [PATCH v1 09/11] Documentation/config: add odb.<name>.promisorRemote Christian Couder
@ 2018-06-23 12:29 ` Christian Couder
2018-06-25 21:49 ` Junio C Hamano
10 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-23 12:29 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Ben Peart, Jonathan Tan,
Nguyen Thai Ngoc Duy, Mike Hommey, Lars Schneider, Eric Wong,
Christian Couder, Jeff Hostetler
On Sat, Jun 23, 2018 at 2:18 PM, Christian Couder
<christian.couder@gmail.com> wrote:
> High level overview of this patch series
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> - Patch 1/9:
Sorry the patches are numbered X/11 but they should be numbered X/9 as
only the first 9 should be in the series and have been sent.
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v1 0/9] Introducing remote ODBs
2018-06-23 12:18 [PATCH v1 0/9] Introducing remote ODBs Christian Couder
` (9 preceding siblings ...)
2018-06-23 12:29 ` [PATCH v1 0/9] Introducing remote ODBs Christian Couder
@ 2018-06-25 21:49 ` Junio C Hamano
2018-06-26 0:37 ` Eric Sunshine
2018-06-28 13:04 ` Christian Couder
10 siblings, 2 replies; 15+ messages in thread
From: Junio C Hamano @ 2018-06-25 21:49 UTC (permalink / raw)
To: Christian Couder
Cc: git, Jeff King, Ben Peart, Jonathan Tan, Nguyen Thai Ngoc Duy,
Mike Hommey, Lars Schneider, Eric Wong, Christian Couder,
Jeff Hostetler
Christian Couder <christian.couder@gmail.com> writes:
> This is a follow up from the patch series called "odb remote" that I
> ...
Just an early warning, as I haven't even complained on patch titles
of these patches in the series ;-)
5702.20 and 5702.21 seems to fail in standalone test, when these are
directly queued on top of Git v2.18.0; I haven't looked into the
failure myself (yet).
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v1 0/9] Introducing remote ODBs
2018-06-25 21:49 ` Junio C Hamano
@ 2018-06-26 0:37 ` Eric Sunshine
2018-06-28 15:17 ` Christian Couder
2018-06-28 13:04 ` Christian Couder
1 sibling, 1 reply; 15+ messages in thread
From: Eric Sunshine @ 2018-06-26 0:37 UTC (permalink / raw)
To: Junio C Hamano
Cc: Christian Couder, Git List, Jeff King, Ben Peart, Jonathan Tan,
Nguyễn Thái Ngọc Duy, Mike Hommey,
Lars Schneider, Eric Wong, Christian Couder, Jeff Hostetler
On Mon, Jun 25, 2018 at 5:49 PM Junio C Hamano <gitster@pobox.com> wrote:
> Christian Couder <christian.couder@gmail.com> writes:
> > This is a follow up from the patch series called "odb remote" [...]
>
> 5702.20 and 5702.21 seems to fail in standalone test, when these are
> directly queued on top of Git v2.18.0; I haven't looked into the
> failure myself (yet).
In addition to the t5702 failures, I'm also seeing failures of
t0410.1, t5616.6 and t5616.7 at the tip of 'pu' as of [1], all of
which seem to be related to these changes.
[1]: https://public-inbox.org/git/xmqqin66mql6.fsf@gitster-ct.c.googlers.com/
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 0/9] Introducing remote ODBs
2018-06-26 0:37 ` Eric Sunshine
@ 2018-06-28 15:17 ` Christian Couder
0 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-28 15:17 UTC (permalink / raw)
To: Eric Sunshine
Cc: Junio C Hamano, Git List, Jeff King, Ben Peart, Jonathan Tan,
Nguyễn Thái Ngọc Duy, Mike Hommey,
Lars Schneider, Eric Wong, Christian Couder, Jeff Hostetler
On Tue, Jun 26, 2018 at 2:37 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
> In addition to the t5702 failures, I'm also seeing failures of
> t0410.1, t5616.6 and t5616.7 at the tip of 'pu' as of [1], all of
> which seem to be related to these changes.
Yeah but only s/core.partialclonefilter/odb.origin.partialclonefilter/
is needed on top of the fix for for the master branch.
Thanks for the report!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 0/9] Introducing remote ODBs
2018-06-25 21:49 ` Junio C Hamano
2018-06-26 0:37 ` Eric Sunshine
@ 2018-06-28 13:04 ` Christian Couder
1 sibling, 0 replies; 15+ messages in thread
From: Christian Couder @ 2018-06-28 13:04 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Jeff King, Ben Peart, Jonathan Tan, Nguyen Thai Ngoc Duy,
Mike Hommey, Lars Schneider, Eric Wong, Christian Couder,
Jeff Hostetler
On Mon, Jun 25, 2018 at 11:49 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Just an early warning, as I haven't even complained on patch titles
> of these patches in the series ;-)
>
> 5702.20 and 5702.21 seems to fail in standalone test, when these are
> directly queued on top of Git v2.18.0; I haven't looked into the
> failure myself (yet).
Yeah sorry about that.
These tests can be fixed with only
s/extensions.partialClone/odb.magic.promisorRemote/ though.
^ permalink raw reply [flat|nested] 15+ messages in thread