From: Josh Steadmon <steadmon@google.com>
To: Patrick Steinhardt <ps@pks.im>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 6/7] reftable/record: use scratch buffer when decoding records
Date: Mon, 11 Mar 2024 12:31:57 -0700 [thread overview]
Message-ID: <Ze9cLWCZoBqcfr2B@google.com> (raw)
In-Reply-To: <e0a9057593f761917a3e6fb9a77045400e2b9d1c.1709640322.git.ps@pks.im>
On 2024.03.05 13:11, Patrick Steinhardt wrote:
> When decoding log records we need a temporary buffer to decode the
> reflog entry's name, mail address and message. As this buffer is local
> to the function we thus have to reallocate it for every single log
> record which we're about to decode, which is inefficient.
>
> Refactor the code such that callers need to pass in a scratch buffer,
> which allows us to reuse it for multiple decodes. This reduces the
> number of allocations when iterating through reflogs. Before:
>
> HEAP SUMMARY:
> in use at exit: 13,473 bytes in 122 blocks
> total heap usage: 2,068,487 allocs, 2,068,365 frees, 305,122,946 bytes allocated
>
> After:
>
> HEAP SUMMARY:
> in use at exit: 13,473 bytes in 122 blocks
> total heap usage: 1,068,485 allocs, 1,068,363 frees, 281,122,886 bytes allocated
>
> Note that this commit also drop some redundant calls to `strbuf_reset()`
> right before calling `decode_string()`. The latter already knows to
> reset the buffer, so there is no need for these.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> reftable/block.c | 4 ++-
> reftable/block.h | 2 ++
> reftable/record.c | 52 ++++++++++++++++++--------------------
> reftable/record.h | 5 ++--
> reftable/record_test.c | 57 ++++++++++++++++++++++++++----------------
> 5 files changed, 68 insertions(+), 52 deletions(-)
[snip]
> diff --git a/reftable/record.c b/reftable/record.c
> index 7c86877586..060244337f 100644
> --- a/reftable/record.c
> +++ b/reftable/record.c
> @@ -374,7 +374,7 @@ static int reftable_ref_record_encode(const void *rec, struct string_view s,
>
> static int reftable_ref_record_decode(void *rec, struct strbuf key,
> uint8_t val_type, struct string_view in,
> - int hash_size)
> + int hash_size, struct strbuf *scratch)
> {
> struct reftable_ref_record *r = rec;
> struct string_view start = in;
> @@ -425,13 +425,12 @@ static int reftable_ref_record_decode(void *rec, struct strbuf key,
> break;
>
> case REFTABLE_REF_SYMREF: {
> - struct strbuf dest = STRBUF_INIT;
> - int n = decode_string(&dest, in);
> + int n = decode_string(scratch, in);
> if (n < 0) {
> return -1;
> }
> string_view_consume(&in, n);
> - r->value.symref = dest.buf;
> + r->value.symref = strbuf_detach(scratch, NULL);
> } break;
I had to dig into this to convince myself that we aren't leaking memory
here, but IIUC this gets cleaned up eventually by
reftable_ref_record_release(), right?
next prev parent reply other threads:[~2024-03-11 19:32 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-05 12:10 [PATCH 0/7] reftable: memory optimizations for reflog iteration Patrick Steinhardt
2024-03-05 12:10 ` [PATCH 1/7] refs/reftable: reload correct stack when creating reflog iter Patrick Steinhardt
2024-03-06 16:13 ` Karthik Nayak
2024-03-06 17:49 ` Junio C Hamano
2024-03-07 6:00 ` Patrick Steinhardt
2024-03-11 18:34 ` Josh Steadmon
2024-03-11 23:24 ` Patrick Steinhardt
2024-03-05 12:10 ` [PATCH 2/7] reftable/record: convert old and new object IDs to arrays Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 3/7] reftable/record: avoid copying author info Patrick Steinhardt
2024-03-13 1:09 ` James Liu
2024-03-21 13:10 ` Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 4/7] reftable/record: reuse refnames when decoding log records Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 5/7] reftable/record: reuse message " Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 6/7] reftable/record: use scratch buffer when decoding records Patrick Steinhardt
2024-03-11 19:31 ` Josh Steadmon [this message]
2024-03-11 23:25 ` Patrick Steinhardt
2024-03-11 19:40 ` Josh Steadmon
2024-03-05 12:11 ` [PATCH 7/7] refs/reftable: track last log record name via strbuf Patrick Steinhardt
2024-03-11 19:41 ` [PATCH 0/7] reftable: memory optimizations for reflog iteration Josh Steadmon
2024-03-11 23:25 ` Patrick Steinhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Ze9cLWCZoBqcfr2B@google.com \
--to=steadmon@google.com \
--cc=git@vger.kernel.org \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).