git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Subject: [PATCH 0/7] reftable: memory optimizations for reflog iteration
Date: Tue, 5 Mar 2024 13:10:51 +0100	[thread overview]
Message-ID: <cover.1709640322.git.ps@pks.im> (raw)

[-- Attachment #1: Type: text/plain, Size: 1826 bytes --]

Hi,

this patch series does the same as all the preceding patch series that
optimized how the reftable library iterates through refs, but for
reflogs instead.

The goal of this patch series is to arrive at a constant number of
allocations when iterating refs. This is achieved in mostly the same way
we did it for ref iteration, namely by reusing already-allocated memory.
Overall, this brings us down from 8 allocations per reflog record to
essentially 0 allocations per reflog. Iterating through 1 million
reflogs with `git reflog list` thus goes down from 8.068m allocations to
only around 68.5k.

This series is built on top of "master" at b387623c12 (The third batch,
2024-03-01) with Junio's "ps/reftable-iteration-perf-part2" at
43f70eaea0 (refs/reftable: precompute prefix length, 2024-03-04) merged
into it.

Patrick

Patrick Steinhardt (7):
  refs/reftable: reload correct stack when creating reflog iter
  reftable/record: convert old and new object IDs to arrays
  reftable/record: avoid copying author info
  reftable/record: reuse refnames when decoding log records
  reftable/record: reuse message when decoding log records
  reftable/record: use scratch buffer when decoding records
  refs/reftable: track last log record name via strbuf

 refs/reftable-backend.c    |  52 +++++----------
 reftable/block.c           |   4 +-
 reftable/block.h           |   2 +
 reftable/merged_test.c     |  11 ++--
 reftable/readwrite_test.c  |  62 +++++++-----------
 reftable/record.c          | 129 ++++++++++++++-----------------------
 reftable/record.h          |   5 +-
 reftable/record_test.c     |  68 ++++++++++---------
 reftable/reftable-record.h |   6 +-
 reftable/stack_test.c      |  26 ++++----
 10 files changed, 154 insertions(+), 211 deletions(-)

-- 
2.44.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

             reply	other threads:[~2024-03-05 12:11 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-03-05 12:10 Patrick Steinhardt [this message]
2024-03-05 12:10 ` [PATCH 1/7] refs/reftable: reload correct stack when creating reflog iter Patrick Steinhardt
2024-03-06 16:13   ` Karthik Nayak
2024-03-06 17:49     ` Junio C Hamano
2024-03-07  6:00     ` Patrick Steinhardt
2024-03-11 18:34   ` Josh Steadmon
2024-03-11 23:24     ` Patrick Steinhardt
2024-03-05 12:10 ` [PATCH 2/7] reftable/record: convert old and new object IDs to arrays Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 3/7] reftable/record: avoid copying author info Patrick Steinhardt
2024-03-13  1:09   ` James Liu
2024-03-21 13:10     ` Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 4/7] reftable/record: reuse refnames when decoding log records Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 5/7] reftable/record: reuse message " Patrick Steinhardt
2024-03-05 12:11 ` [PATCH 6/7] reftable/record: use scratch buffer when decoding records Patrick Steinhardt
2024-03-11 19:31   ` Josh Steadmon
2024-03-11 23:25     ` Patrick Steinhardt
2024-03-11 19:40   ` Josh Steadmon
2024-03-05 12:11 ` [PATCH 7/7] refs/reftable: track last log record name via strbuf Patrick Steinhardt
2024-03-11 19:41 ` [PATCH 0/7] reftable: memory optimizations for reflog iteration Josh Steadmon
2024-03-11 23:25   ` Patrick Steinhardt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1709640322.git.ps@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).