From: Jeff King <peff@peff.net> To: git@vger.kernel.org Subject: [PATCH 08/13] pack-bitmap: basic noop bitmap filter infrastructure Date: Wed, 12 Feb 2020 21:20:58 -0500 [thread overview] Message-ID: <20200213022058.GH1126038@coredump.intra.peff.net> (raw) In-Reply-To: <20200213021506.GA1124607@coredump.intra.peff.net> Currently you can't use object filters with bitmaps, but we plan to support at least some filters with bitmaps. Let's introduce some infrastructure that will help us do that: - prepare_bitmap_walk() now accepts a list_objects_filter_options parameter (which can be NULL for no filtering; all the current callers pass this) - we'll bail early if the filter is incompatible with bitmaps (just as we would if there were no bitmaps at all). Currently all filters are incompatible. - we'll filter the resulting bitmap; since there are no supported filters yet, this is always a noop. There should be no behavior change yet, but we'll support some actual filters in a future patch. Signed-off-by: Jeff King <peff@peff.net> --- builtin/pack-objects.c | 2 +- builtin/rev-list.c | 4 ++-- pack-bitmap.c | 30 ++++++++++++++++++++++++++++-- pack-bitmap.h | 4 +++- reachable.c | 2 +- 5 files changed, 35 insertions(+), 7 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 393c20a2d7..60c943e42b 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3040,7 +3040,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs))) + if (!(bitmap_git = prepare_bitmap_walk(revs, NULL))) return -1; if (pack_options_allow_reuse() && diff --git a/builtin/rev-list.c b/builtin/rev-list.c index ce1cfc7da0..1ef180469f 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -407,7 +407,7 @@ static int try_bitmap_count(struct rev_info *revs) */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs); + bitmap_git = prepare_bitmap_walk(revs, NULL); if (!bitmap_git) return -1; @@ -441,7 +441,7 @@ static int try_bitmap_traversal(struct rev_info *revs) if (!revs->tag_objects || !revs->tree_objects || !revs->blob_objects) return -1; - bitmap_git = prepare_bitmap_walk(revs); + bitmap_git = prepare_bitmap_walk(revs, NULL); if (!bitmap_git) return -1; diff --git a/pack-bitmap.c b/pack-bitmap.c index 6c06099dc7..eb8888bb5e 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -12,6 +12,7 @@ #include "packfile.h" #include "repository.h" #include "object-store.h" +#include "list-objects-filter-options.h" /* * An entry on the bitmap index, representing the bitmap for a given @@ -705,7 +706,25 @@ static int in_bitmapped_pack(struct bitmap_index *bitmap_git, return 0; } -struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs) +static int filter_bitmap(struct bitmap_index *bitmap_git, + struct object_list *tip_objects, + struct bitmap *to_filter, + struct list_objects_filter_options *filter) +{ + if (!filter || filter->choice == LOFC_DISABLED) + return 0; + + /* filter choice not handled */ + return -1; +} + +static int can_filter_bitmap(struct list_objects_filter_options *filter) +{ + return !filter_bitmap(NULL, NULL, NULL, filter); +} + +struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, + struct list_objects_filter_options *filter) { unsigned int i; @@ -715,9 +734,14 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs) struct bitmap *wants_bitmap = NULL; struct bitmap *haves_bitmap = NULL; - struct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git)); + struct bitmap_index *bitmap_git; + + if (!can_filter_bitmap(filter)) + return NULL; + /* try to open a bitmapped pack, but don't parse it yet * because we may not need to use it */ + bitmap_git = xcalloc(1, sizeof(*bitmap_git)); if (open_pack_bitmap(revs->repo, bitmap_git) < 0) goto cleanup; @@ -784,6 +808,8 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs) if (haves_bitmap) bitmap_and_not(wants_bitmap, haves_bitmap); + filter_bitmap(bitmap_git, wants, wants_bitmap, filter); + bitmap_git->result = wants_bitmap; bitmap_git->haves = haves_bitmap; diff --git a/pack-bitmap.h b/pack-bitmap.h index 466c5afa09..04bdb20c3c 100644 --- a/pack-bitmap.h +++ b/pack-bitmap.h @@ -8,6 +8,7 @@ struct commit; struct repository; struct rev_info; +struct list_objects_filter_options; static const char BITMAP_IDX_SIGNATURE[] = {'B', 'I', 'T', 'M'}; @@ -46,7 +47,8 @@ void count_bitmap_commit_list(struct bitmap_index *, uint32_t *commits, void traverse_bitmap_commit_list(struct bitmap_index *, show_reachable_fn show_reachable); void test_bitmap_walk(struct rev_info *revs); -struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs); +struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, + struct list_objects_filter_options *filter); int reuse_partial_packfile_from_bitmap(struct bitmap_index *, struct packed_git **packfile, uint32_t *entries, off_t *up_to); diff --git a/reachable.c b/reachable.c index 8f50235b28..45cde57cab 100644 --- a/reachable.c +++ b/reachable.c @@ -223,7 +223,7 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog, cp.progress = progress; cp.count = 0; - bitmap_git = prepare_bitmap_walk(revs); + bitmap_git = prepare_bitmap_walk(revs, NULL); if (bitmap_git) { traverse_bitmap_commit_list(bitmap_git, mark_object_seen); free_bitmap_index(bitmap_git); -- 2.25.0.785.g49bcbe7794
next prev parent reply other threads:[~2020-02-13 2:21 UTC|newest] Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-02-13 2:15 [PATCH 0/13] combining object filters and bitmaps Jeff King 2020-02-13 2:16 ` [PATCH 01/13] pack-bitmap: factor out type iterator initialization Jeff King 2020-02-13 17:45 ` Junio C Hamano 2020-02-13 2:16 ` [PATCH 02/13] pack-bitmap: fix leak of haves/wants object lists Jeff King 2020-02-13 18:12 ` Junio C Hamano 2020-02-13 2:17 ` [PATCH 03/13] rev-list: fallback to non-bitmap traversal when filtering Jeff King 2020-02-13 18:19 ` Junio C Hamano 2020-02-13 18:40 ` Jeff King 2020-02-13 2:17 ` [PATCH 04/13] rev-list: consolidate bitmap-disabling options Jeff King 2020-02-13 2:18 ` [PATCH 05/13] rev-list: factor out bitmap-optimized routines Jeff King 2020-02-13 18:34 ` Junio C Hamano 2020-02-13 2:19 ` [PATCH 06/13] rev-list: make --count work with --objects Jeff King 2020-02-13 19:14 ` Junio C Hamano 2020-02-13 20:27 ` Jeff King 2020-02-13 2:20 ` [PATCH 07/13] rev-list: allow bitmaps when counting objects Jeff King 2020-02-13 21:47 ` Junio C Hamano 2020-02-13 22:27 ` Jeff King 2020-02-13 2:20 ` Jeff King [this message] 2020-02-13 2:21 ` [PATCH 09/13] rev-list: use bitmap filters for traversal Jeff King 2020-02-13 22:22 ` Junio C Hamano 2020-02-13 22:34 ` Jeff King 2020-02-13 2:21 ` [PATCH 10/13] bitmap: add bitmap_unset() function Jeff King 2020-02-13 2:23 ` [PATCH 11/13] pack-bitmap: implement BLOB_NONE filtering Jeff King 2020-02-13 2:25 ` [PATCH 12/13] pack-bitmap: implement BLOB_LIMIT filtering Jeff King 2020-02-13 23:17 ` Junio C Hamano 2020-02-13 2:25 ` [PATCH 13/13] pack-objects: support filters with bitmaps Jeff King 2020-02-14 18:21 ` [PATCH v2 0/15] combining object filters and bitmaps Jeff King 2020-02-14 18:22 ` [PATCH v2 01/15] pack-bitmap: factor out type iterator initialization Jeff King 2020-02-15 0:10 ` Taylor Blau 2020-02-14 18:22 ` [PATCH v2 02/15] pack-bitmap: fix leak of haves/wants object lists Jeff King 2020-02-15 0:15 ` Taylor Blau 2020-02-15 6:46 ` Jeff King 2020-02-18 17:58 ` Derrick Stolee 2020-02-18 20:02 ` Jeff King 2020-02-14 18:22 ` [PATCH v2 03/15] rev-list: fallback to non-bitmap traversal when filtering Jeff King 2020-02-15 0:22 ` Taylor Blau 2020-02-14 18:22 ` [PATCH v2 04/15] pack-bitmap: refuse to do a bitmap traversal with pathspecs Jeff King 2020-02-14 19:03 ` Junio C Hamano 2020-02-14 20:51 ` Jeff King 2020-02-14 18:22 ` [PATCH v2 05/15] rev-list: factor out bitmap-optimized routines Jeff King 2020-02-15 0:35 ` Taylor Blau 2020-02-14 18:22 ` [PATCH v2 06/15] rev-list: make --count work with --objects Jeff King 2020-02-15 0:42 ` Taylor Blau 2020-02-15 6:48 ` Jeff King 2020-02-16 23:34 ` Junio C Hamano 2020-02-18 5:24 ` Jeff King 2020-02-18 17:28 ` Junio C Hamano 2020-02-18 19:55 ` Jeff King 2020-02-18 21:19 ` Junio C Hamano 2020-02-18 21:23 ` Jeff King 2020-02-18 18:05 ` Derrick Stolee 2020-02-18 19:59 ` Jeff King 2020-02-14 18:22 ` [PATCH v2 07/15] rev-list: allow bitmaps when counting objects Jeff King 2020-02-15 0:45 ` Taylor Blau 2020-02-15 6:55 ` Jeff King 2020-02-16 23:36 ` Junio C Hamano 2020-02-14 18:22 ` [PATCH v2 08/15] t5310: factor out bitmap traversal comparison Jeff King 2020-02-15 2:14 ` Taylor Blau 2020-02-15 7:00 ` Jeff King 2020-02-14 18:22 ` [PATCH v2 09/15] rev-list: allow commit-only bitmap traversals Jeff King 2020-02-18 18:18 ` Derrick Stolee 2020-02-18 20:05 ` Jeff King 2020-02-18 20:11 ` Derrick Stolee 2020-02-14 18:22 ` [PATCH v2 10/15] pack-bitmap: basic noop bitmap filter infrastructure Jeff King 2020-02-14 18:22 ` [PATCH v2 11/15] rev-list: use bitmap filters for traversal Jeff King 2020-02-14 18:22 ` [PATCH v2 12/15] bitmap: add bitmap_unset() function Jeff King 2020-02-14 18:22 ` [PATCH v2 13/15] pack-bitmap: implement BLOB_NONE filtering Jeff King 2020-02-18 19:26 ` Derrick Stolee 2020-02-18 19:36 ` Derrick Stolee 2020-02-18 20:30 ` Jeff King 2020-02-18 20:24 ` Jeff King 2020-02-14 18:22 ` [PATCH v2 14/15] pack-bitmap: implement BLOB_LIMIT filtering Jeff King 2020-02-14 18:22 ` [PATCH v2 15/15] pack-objects: support filters with bitmaps Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200213022058.GH1126038@coredump.intra.peff.net \ --to=peff@peff.net \ --cc=git@vger.kernel.org \ --subject='Re: [PATCH 08/13] pack-bitmap: basic noop bitmap filter infrastructure' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).