From: Jonathan Tan <jonathantanmy@google.com>
To: git@vger.kernel.org
Cc: Jonathan Tan <jonathantanmy@google.com>, gitster@pobox.com
Subject: [PATCH v2 1/4] sha1-file: introduce no-lazy-fetch has_object()
Date: Wed, 5 Aug 2020 16:06:49 -0700 [thread overview]
Message-ID: <f413b34574a330030e08a6eb93ad812975959d7d.1596668156.git.jonathantanmy@google.com> (raw)
In-Reply-To: <cover.1596668156.git.jonathantanmy@google.com>
There have been a few bugs wherein Git fetches missing objects whenever
the existence of an object is checked, even though it does not need to
perform such a fetch. To resolve these bugs, we could look at all the
places that has_object_file() (or a similar function) is used. As a
first step, introduce a new function has_object() that checks for the
existence of an object, with a default behavior of not fetching if the
object is missing and the repository is a partial clone. As we verify
each has_object_file() (or similar) usage, we can replace it with
has_object(), and we will know that we are done when we can delete
has_object_file() (and the other similar functions).
Also, the new function has_object() has more appropriate defaults:
besides not fetching, it also does not recheck packed storage.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
object-store.h | 25 +++++++++++++++++++++++--
sha1-file.c | 12 ++++++++++++
2 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/object-store.h b/object-store.h
index f439d47af8..c4fc9dd74e 100644
--- a/object-store.h
+++ b/object-store.h
@@ -239,12 +239,33 @@ int read_loose_object(const char *path,
unsigned long *size,
void **contents);
+/* Retry packed storage after checking packed and loose storage */
+#define HAS_OBJECT_RECHECK_PACKED 1
+
+/*
+ * Returns 1 if the object exists. This function will not lazily fetch objects
+ * in a partial clone.
+ */
+int has_object(struct repository *r, const struct object_id *oid,
+ unsigned flags);
+
+/*
+ * These macros and functions are deprecated. If checking existence for an
+ * object that is likely to be missing and/or whose absence is relatively
+ * inconsequential (or is consequential but the caller is prepared to handle
+ * it), use has_object(), which has better defaults (no lazy fetch in a partial
+ * clone and no rechecking of packed storage). In the unlikely event that a
+ * caller needs to assert existence of an object that it fully expects to
+ * exist, and wants to trigger a lazy fetch in a partial clone, use
+ * oid_object_info_extended() with a NULL struct object_info.
+ *
+ * These functions can be removed once all callers have migrated to
+ * has_object() and/or oid_object_info_extended().
+ */
#ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS
#define has_sha1_file_with_flags(sha1, flags) repo_has_sha1_file_with_flags(the_repository, sha1, flags)
#define has_sha1_file(sha1) repo_has_sha1_file(the_repository, sha1)
#endif
-
-/* Same as the above, except for struct object_id. */
int repo_has_object_file(struct repository *r, const struct object_id *oid);
int repo_has_object_file_with_flags(struct repository *r,
const struct object_id *oid, int flags);
diff --git a/sha1-file.c b/sha1-file.c
index ccd34dd9e8..ff444d7abb 100644
--- a/sha1-file.c
+++ b/sha1-file.c
@@ -1988,6 +1988,18 @@ int force_object_loose(const struct object_id *oid, time_t mtime)
return ret;
}
+int has_object(struct repository *r, const struct object_id *oid,
+ unsigned flags)
+{
+ int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
+ unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
+ (quick ? OBJECT_INFO_QUICK : 0);
+
+ if (!startup_info->have_repository)
+ return 0;
+ return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
+}
+
int repo_has_object_file_with_flags(struct repository *r,
const struct object_id *oid, int flags)
{
--
2.28.0.236.gb10cc79966-goog
next prev parent reply other threads:[~2020-08-05 23:07 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-28 1:04 [PATCH] apply: do not fetch when checking object existence Jonathan Tan
2020-07-28 1:19 ` Junio C Hamano
2020-07-28 18:23 ` Jonathan Tan
2020-08-05 23:06 ` [PATCH v2 0/4] No-lazy-fetch has_object() and some fixes Jonathan Tan
2020-08-05 23:06 ` Jonathan Tan [this message]
2020-08-05 23:06 ` [PATCH v2 2/4] apply: do not lazy fetch when applying binary Jonathan Tan
2020-08-05 23:06 ` [PATCH v2 3/4] pack-objects: no fetch when allow-{any,promisor} Jonathan Tan
2020-08-05 23:06 ` [PATCH v2 4/4] fsck: do not lazy fetch known non-promisor object Jonathan Tan
2020-08-06 20:00 ` [PATCH v2 0/4] No-lazy-fetch has_object() and some fixes Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f413b34574a330030e08a6eb93ad812975959d7d.1596668156.git.jonathantanmy@google.com \
--to=jonathantanmy@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).