From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Han-Wen Nienhuys <hanwenn@gmail.com>,
Karthik Nayak <karthik.188@gmail.com>,
Justin Tobler <jltobler@gmail.com>
Subject: [PATCH v2 00/10] reftable: optimize table and block iterators
Date: Mon, 8 Apr 2024 14:16:22 +0200 [thread overview]
Message-ID: <cover.1712578376.git.ps@pks.im> (raw)
In-Reply-To: <cover.1711519925.git.ps@pks.im>
[-- Attachment #1: Type: text/plain, Size: 4982 bytes --]
Hi,
this is the second version of my patch series that aims to optimize
the reftable table and block iterators.
Changes compared to v1:
- The series now deepends on ps/reftable-binsearch-update at
d51d8cc368 (reftable/block: avoid decoding keys when searching
restart points, 2024-04-03). This is to resolve a merge conflict
with that other series which has landed in "next" already.
- Rewrote a comment to be single-line to fit into the style of other
comments better.
- A new patch on top that avoids copying block iterators altogether
for another speedup.
Thanks!
Patrick
Patrick Steinhardt (10):
reftable/block: rename `block_reader_start()`
reftable/block: merge `block_iter_seek()` and `block_reader_seek()`
reftable/block: better grouping of functions
reftable/block: introduce `block_reader_release()`
reftable/block: move ownership of block reader into `struct
table_iter`
reftable/reader: iterate to next block in place
reftable/block: reuse uncompressed blocks
reftable/block: open-code call to `uncompress2()`
reftable/block: reuse `zstream` state on inflation
reftable/block: avoid copying block iterators on seek
reftable/block.c | 176 +++++++++++++++++++++++++-----------------
reftable/block.h | 47 ++++++-----
reftable/block_test.c | 6 +-
reftable/iter.c | 2 +-
reftable/reader.c | 176 ++++++++++++++++++++++--------------------
5 files changed, 229 insertions(+), 178 deletions(-)
Range-diff against v1:
1: 24b0dda29e = 1: eb487557a8 reftable/block: rename `block_reader_start()`
2: a2b7f0f559 ! 2: d0b318b8ee reftable/block: merge `block_iter_seek()` and `block_reader_seek()`
@@ reftable/block.c: int block_reader_first_key(struct block_reader *br, struct str
+int block_iter_seek_key(struct block_iter *it, struct block_reader *br,
+ struct strbuf *want)
{
- struct restart_find_args args = {
- .key = *want,
+ struct restart_needle_less_args args = {
+ .needle = *want,
## reftable/block.h ##
@@ reftable/block.h: int block_reader_init(struct block_reader *br, struct reftable_block *bl,
3: 88a705b3e2 = 3: c3f928d1e9 reftable/block: better grouping of functions
4: 9a1253649a = 4: 35f1bf5072 reftable/block: introduce `block_reader_release()`
5: f10882a084 ! 5: e8e8bbae62 reftable/block: move ownership of block reader into `struct table_iter`
@@ reftable/block.c: int block_reader_first_key(struct block_reader *br, struct str
it->next_off = br->header_off + 4;
}
@@ reftable/block.c: void block_iter_seek_start(struct block_iter *it, struct block_reader *br)
- struct restart_find_args {
+ struct restart_needle_less_args {
int error;
- struct strbuf key;
-- struct block_reader *r;
-+ const struct block_reader *r;
+ struct strbuf needle;
+- struct block_reader *reader;
++ const struct block_reader *reader;
};
- static int restart_key_less(size_t idx, void *args)
-@@ reftable/block.c: static int restart_key_less(size_t idx, void *args)
- return result < 0;
+ static int restart_needle_less(size_t idx, void *_args)
+@@ reftable/block.c: static int restart_needle_less(size_t idx, void *_args)
+ return args->needle.len < suffix_len;
}
-void block_iter_copy_from(struct block_iter *dest, struct block_iter *src)
@@ reftable/block.c: int block_iter_next(struct block_iter *it, struct reftable_rec
+int block_iter_seek_key(struct block_iter *it, const struct block_reader *br,
struct strbuf *want)
{
- struct restart_find_args args = {
+ struct restart_needle_less_args args = {
@@ reftable/block.c: int block_iter_seek_key(struct block_iter *it, struct block_reader *br,
it->next_off = block_reader_restart_offset(br, i - 1);
else
@@ reftable/block.h: struct block_iter {
/* return < 0 for error, 0 for OK, > 0 for EOF. */
int block_iter_next(struct block_iter *it, struct reftable_record *rec);
-+/*
-+ * Reset the block iterator to pristine state without releasing its memory.
-+ */
++/* Reset the block iterator to pristine state without releasing its memory. */
+void block_iter_reset(struct block_iter *it);
+
/* deallocate memory for `it`. The block reader and its block is left intact. */
6: ae359cb714 = 6: 685f0a40bc reftable/reader: iterate to next block in place
7: 1e4eba7e9b = 7: a7906a3383 reftable/block: reuse uncompressed blocks
8: bf4c1ab797 = 8: 6635c7b986 reftable/block: open-code call to `uncompress2()`
9: 43e6538968 = 9: 587b5601c0 reftable/block: reuse `zstream` state on inflation
-: ---------- > 10: cc5ff0d598 reftable/block: avoid copying block iterators on seek
--
2.44.GIT
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2024-04-08 12:16 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-27 6:36 [PATCH 0/9] reftable: optimize table and block iterators Patrick Steinhardt
2024-03-27 6:36 ` [PATCH 1/9] reftable/block: rename `block_reader_start()` Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 2/9] reftable/block: merge `block_iter_seek()` and `block_reader_seek()` Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 3/9] reftable/block: better grouping of functions Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 4/9] reftable/block: introduce `block_reader_release()` Patrick Steinhardt
2024-04-03 13:16 ` Karthik Nayak
2024-04-08 12:10 ` Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 5/9] reftable/block: move ownership of block reader into `struct table_iter` Patrick Steinhardt
2024-04-03 4:52 ` Justin Tobler
2024-04-03 13:10 ` Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 6/9] reftable/reader: iterate to next block in place Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 7/9] reftable/block: reuse uncompressed blocks Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 8/9] reftable/block: open-code call to `uncompress2()` Patrick Steinhardt
2024-03-27 6:37 ` [PATCH 9/9] reftable/block: reuse `zstream` state on inflation Patrick Steinhardt
2024-04-03 13:33 ` [PATCH 0/9] reftable: optimize table and block iterators Karthik Nayak
2024-04-08 12:16 ` Patrick Steinhardt [this message]
2024-04-08 12:16 ` [PATCH v2 01/10] reftable/block: rename `block_reader_start()` Patrick Steinhardt
2024-04-08 12:16 ` [PATCH v2 02/10] reftable/block: merge `block_iter_seek()` and `block_reader_seek()` Patrick Steinhardt
2024-04-08 12:16 ` [PATCH v2 03/10] reftable/block: better grouping of functions Patrick Steinhardt
2024-04-08 12:16 ` [PATCH v2 04/10] reftable/block: introduce `block_reader_release()` Patrick Steinhardt
2024-04-08 12:16 ` [PATCH v2 05/10] reftable/block: move ownership of block reader into `struct table_iter` Patrick Steinhardt
2024-04-08 12:16 ` [PATCH v2 06/10] reftable/reader: iterate to next block in place Patrick Steinhardt
2024-04-08 12:16 ` [PATCH v2 07/10] reftable/block: reuse uncompressed blocks Patrick Steinhardt
2024-04-08 12:16 ` [PATCH v2 08/10] reftable/block: open-code call to `uncompress2()` Patrick Steinhardt
2024-04-08 12:17 ` [PATCH v2 09/10] reftable/block: reuse `zstream` state on inflation Patrick Steinhardt
2024-04-10 10:15 ` Karthik Nayak
2024-04-08 12:17 ` [PATCH v2 10/10] reftable/block: avoid copying block iterators on seek Patrick Steinhardt
2024-04-09 1:29 ` Justin Tobler
2024-04-09 3:18 ` Patrick Steinhardt
2024-04-09 1:32 ` [PATCH v2 00/10] reftable: optimize table and block iterators Justin Tobler
2024-04-10 11:35 ` Karthik Nayak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1712578376.git.ps@pks.im \
--to=ps@pks.im \
--cc=git@vger.kernel.org \
--cc=hanwenn@gmail.com \
--cc=jltobler@gmail.com \
--cc=karthik.188@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).