From: Jeff King <peff@peff.net>
To: Taylor Blau <me@ttaylorr.com>
Cc: git@vger.kernel.org, jrnieder@gmail.com
Subject: Re: [PATCH 01/20] pack-revindex: introduce a new API
Date: Tue, 12 Jan 2021 03:41:28 -0500 [thread overview]
Message-ID: <X/1guCOGWybOzIS7@coredump.intra.peff.net> (raw)
In-Reply-To: <fa6b8309088fd04410ca7276c5cf14db0fb82fb2.1610129796.git.me@ttaylorr.com>
On Fri, Jan 08, 2021 at 01:16:43PM -0500, Taylor Blau wrote:
> In the next several patches, we will prepare for loading a reverse index
> either in memory, or from a yet-to-be-introduced on-disk format. To do
> that, we'll introduce an API that avoids the caller explicitly indexing
> the revindex pointer in the packed_git structure.
This API looks good to me. Here are a few extra thoughts:
> There are four ways to interact with the reverse index. Accordingly,
> four functions will be exported from 'pack-revindex.h' by the time that
> the existing API is removed. A caller may:
This tells us what the new API functions do. That's useful, but should
it be in the header file itself, documenting each function?
Likewise, I think we'd want to define the concepts in that
documentation. Something like:
/*
* A revindex allows converting efficiently between three properties
* of an object within a pack:
*
* - index position: the numeric position within the list of
* sorted object ids found in the .idx file
*
* - pack position: the numeric position within the list of objects
* in their order within the actual .pack file (i.e., 0 is the
* first object in the .pack, 1 is the second, and so on)
*
* - offset: the byte offset within the .pack file at which the
* object contents can be found
*/
And then above each function we can just say that it converts X to Y
(like you have in the commit message). It may also be worth indicating
the run-time of each (some of them are constant-time once you have a
revindex, and some are log(n)).
> +int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos)
The types here make sense. off_t is clearly needed for a pack offset,
and uint32_t is correct for the position fields, because packs have a
4-byte object count.
Separating the error return from the out-parameter makes the interface
slightly more awkward, but is needed to use the properly-sized types.
Makes sense.
> +int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos)
> +{
> + int ret;
> +
> + if (load_pack_revindex(p) < 0)
> + return -1;
This one lazy-loads the revindex for us, which seems handy...
> +uint32_t pack_pos_to_index(struct packed_git *p, uint32_t pos)
> +{
> + if (!p->revindex)
> + BUG("pack_pos_to_index: reverse index not yet loaded");
> + if (pos >= p->num_objects)
> + BUG("pack_pos_to_index: out-of-bounds object at %"PRIu32, pos);
> + return p->revindex[pos].nr;
> +}
But these ones don't. I'm glad we at least catch it with a BUG(), but it
makes the API a little funny. Returning an error here would require a
similarly awkward out-parameter, I guess.
-Peff
next prev parent reply other threads:[~2021-01-12 8:43 UTC|newest]
Thread overview: 121+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-08 18:16 [PATCH 00/20] pack-revindex: prepare for on-disk reverse index Taylor Blau
2021-01-08 18:16 ` [PATCH 01/20] pack-revindex: introduce a new API Taylor Blau
2021-01-12 8:41 ` Jeff King [this message]
2021-01-12 9:41 ` Jeff King
2021-01-12 16:27 ` Taylor Blau
2021-01-13 8:06 ` Junio C Hamano
2021-01-13 8:54 ` Junio C Hamano
2021-01-13 13:17 ` Jeff King
2021-01-13 16:23 ` Taylor Blau
2021-01-08 18:16 ` [PATCH 02/20] write_reuse_object(): convert to new revindex API Taylor Blau
2021-01-12 8:47 ` Jeff King
2021-01-12 16:31 ` Taylor Blau
2021-01-13 13:02 ` Jeff King
2021-01-08 18:16 ` [PATCH 03/20] write_reused_pack_one(): " Taylor Blau
2021-01-12 8:50 ` Jeff King
2021-01-12 16:34 ` Taylor Blau
2021-01-08 18:16 ` [PATCH 04/20] write_reused_pack_verbatim(): " Taylor Blau
2021-01-12 8:50 ` Jeff King
2021-01-08 18:17 ` [PATCH 05/20] check_object(): " Taylor Blau
2021-01-11 11:43 ` Derrick Stolee
2021-01-11 16:15 ` Taylor Blau
2021-01-12 8:54 ` Jeff King
2021-01-12 8:51 ` Jeff King
2021-01-08 18:17 ` [PATCH 06/20] bitmap_position_packfile(): " Taylor Blau
2021-01-08 18:17 ` [PATCH 07/20] show_objects_for_type(): " Taylor Blau
2021-01-12 8:57 ` Jeff King
2021-01-12 16:35 ` Taylor Blau
2021-01-08 18:17 ` [PATCH 08/20] get_size_by_pos(): " Taylor Blau
2021-01-08 18:17 ` [PATCH 09/20] try_partial_reuse(): " Taylor Blau
2021-01-12 9:06 ` Jeff King
2021-01-12 16:47 ` Taylor Blau
2021-01-08 18:17 ` [PATCH 10/20] rebuild_existing_bitmaps(): " Taylor Blau
2021-01-08 18:17 ` [PATCH 11/20] get_delta_base_oid(): " Taylor Blau
2021-01-08 18:17 ` [PATCH 12/20] retry_bad_packed_offset(): " Taylor Blau
2021-01-08 18:17 ` [PATCH 13/20] packed_object_info(): " Taylor Blau
2021-01-12 9:11 ` Jeff King
2021-01-12 16:51 ` Taylor Blau
2021-01-08 18:17 ` [PATCH 14/20] unpack_entry(): " Taylor Blau
2021-01-12 9:22 ` Jeff King
2021-01-12 16:56 ` Taylor Blau
2021-01-08 18:17 ` [PATCH 15/20] for_each_object_in_pack(): " Taylor Blau
2021-01-08 18:17 ` [PATCH 16/20] builtin/gc.c: guess the size of the revindex Taylor Blau
2021-01-11 11:52 ` Derrick Stolee
2021-01-11 16:23 ` Taylor Blau
2021-01-11 17:09 ` Derrick Stolee
2021-01-12 9:28 ` Jeff King
2021-01-08 18:17 ` [PATCH 17/20] pack-revindex: remove unused 'find_pack_revindex()' Taylor Blau
2021-01-08 18:17 ` [PATCH 18/20] pack-revindex: remove unused 'find_revindex_position()' Taylor Blau
2021-01-11 11:57 ` Derrick Stolee
2021-01-11 16:27 ` Taylor Blau
2021-01-11 17:11 ` Derrick Stolee
2021-01-12 9:32 ` Jeff King
2021-01-12 16:59 ` Taylor Blau
2021-01-13 13:05 ` Jeff King
2021-01-08 18:18 ` [PATCH 19/20] pack-revindex: hide the definition of 'revindex_entry' Taylor Blau
2021-01-11 11:57 ` Derrick Stolee
2021-01-12 9:34 ` Jeff King
2021-01-08 18:18 ` [PATCH 20/20] pack-revindex.c: avoid direct revindex access in 'offset_to_pack_pos()' Taylor Blau
2021-01-12 9:37 ` Jeff King
2021-01-12 17:02 ` Taylor Blau
2021-01-11 12:07 ` [PATCH 00/20] pack-revindex: prepare for on-disk reverse index Derrick Stolee
2021-01-11 16:30 ` Taylor Blau
2021-01-11 17:15 ` Derrick Stolee
2021-01-11 17:29 ` Taylor Blau
2021-01-11 18:40 ` Junio C Hamano
2021-01-12 9:45 ` Jeff King
2021-01-12 17:07 ` Taylor Blau
2021-01-13 0:23 ` Junio C Hamano
2021-01-13 0:52 ` Taylor Blau
2021-01-13 2:15 ` Junio C Hamano
2021-01-13 3:23 ` Taylor Blau
2021-01-13 8:21 ` Junio C Hamano
2021-01-13 13:13 ` Jeff King
2021-01-13 15:34 ` Taylor Blau
2021-01-13 20:06 ` Junio C Hamano
2021-01-13 20:13 ` Taylor Blau
2021-01-13 20:22 ` Jeff King
2021-01-13 22:23 ` [PATCH v2 " Taylor Blau
2021-01-13 22:23 ` [PATCH v2 01/20] pack-revindex: introduce a new API Taylor Blau
2021-01-14 6:46 ` Junio C Hamano
2021-01-14 12:00 ` Derrick Stolee
2021-01-14 17:06 ` Taylor Blau
2021-01-14 19:19 ` Jeff King
2021-01-14 20:47 ` Junio C Hamano
2021-01-13 22:23 ` [PATCH v2 02/20] write_reuse_object(): convert to new revindex API Taylor Blau
2021-01-13 22:23 ` [PATCH v2 03/20] write_reused_pack_one(): " Taylor Blau
2021-01-13 22:23 ` [PATCH v2 04/20] write_reused_pack_verbatim(): " Taylor Blau
2021-01-13 22:23 ` [PATCH v2 05/20] check_object(): " Taylor Blau
2021-01-13 22:23 ` [PATCH v2 06/20] bitmap_position_packfile(): " Taylor Blau
2021-01-13 22:23 ` [PATCH v2 07/20] show_objects_for_type(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 08/20] get_size_by_pos(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 09/20] try_partial_reuse(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 10/20] rebuild_existing_bitmaps(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 11/20] get_delta_base_oid(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 12/20] retry_bad_packed_offset(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 13/20] packed_object_info(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 14/20] unpack_entry(): " Taylor Blau
2021-01-13 22:24 ` [PATCH v2 15/20] for_each_object_in_pack(): " Taylor Blau
2021-01-14 6:43 ` Junio C Hamano
2021-01-14 17:00 ` Taylor Blau
2021-01-14 19:33 ` Jeff King
2021-01-14 20:11 ` Jeff King
2021-01-14 20:15 ` Taylor Blau
2021-01-15 2:22 ` Junio C Hamano
2021-01-15 2:29 ` Taylor Blau
2021-01-14 20:51 ` Junio C Hamano
2021-01-13 22:24 ` [PATCH v2 16/20] builtin/gc.c: guess the size of the revindex Taylor Blau
2021-01-14 6:33 ` Junio C Hamano
2021-01-14 16:53 ` Taylor Blau
2021-01-14 19:43 ` Jeff King
2021-01-13 22:24 ` [PATCH v2 17/20] pack-revindex: remove unused 'find_pack_revindex()' Taylor Blau
2021-01-13 22:25 ` [PATCH v2 18/20] pack-revindex: remove unused 'find_revindex_position()' Taylor Blau
2021-01-14 6:42 ` Junio C Hamano
2021-01-13 22:25 ` [PATCH v2 19/20] pack-revindex: hide the definition of 'revindex_entry' Taylor Blau
2021-01-13 22:25 ` [PATCH v2 20/20] pack-revindex.c: avoid direct revindex access in 'offset_to_pack_pos()' Taylor Blau
2021-01-14 6:42 ` Junio C Hamano
2021-01-14 16:56 ` Taylor Blau
2021-01-14 19:51 ` [PATCH v2 00/20] pack-revindex: prepare for on-disk reverse index Jeff King
2021-01-14 20:53 ` Junio C Hamano
2021-01-15 9:32 ` Jeff King
2021-01-15 9:33 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=X/1guCOGWybOzIS7@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=jrnieder@gmail.com \
--cc=me@ttaylorr.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).