From: Junio C Hamano <gitster@pobox.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: git@vger.kernel.org, Jeff King <peff@peff.net>
Subject: Re: [PATCH 2/3] strbuf.h API users: don't hardcode 8192, use STRBUF_HINT_SIZE
Date: Wed, 07 Jul 2021 16:09:44 -0700 [thread overview]
Message-ID: <xmqqim1ld8lj.fsf@gitster.g> (raw)
In-Reply-To: <xmqqo8bdda2j.fsf@gitster.g> (Junio C. Hamano's message of "Wed, 07 Jul 2021 15:37:56 -0700")
Junio C Hamano <gitster@pobox.com> writes:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> Change a couple of users of strbuf_init() that pass a hint of 8192 to
>> pass STRBUF_HINT_SIZE instead.
>>
>> Both of these hardcoded occurrences pre-date the use of the strbuf
>> API. See 5242bcbb638 (Use strbuf API in cache-tree.c, 2007-09-06) and
>> af6eb82262e (Use strbuf API in apply, blame, commit-tree and diff,
>> 2007-09-06).
>>
>> In both cases the exact choice of 8192 is rather arbitrary, e.g. for
>> commit buffers I think 1024 or 2048 would probably be a better
>> default (this commit message is getting this commit close to the
>> former, but I daresay it's already way above the average for git
>> commits).
>
> Yes, they are arbitrary within the context of these callers.
>
> I do not think using STRBUF_HINT_SIZE macro in them is the right
> thing to do at all, as there is no reason to think that the best
> value for the write chunk sizes in these codepath has any linkage to
> the best value for the read chunk sizes used by strbuf_read() at
> all. When benchmarking reveals that the best default size for
> strbuf_read() is 16k, you'd update STRBUF_HINT_SIZE to 16k, but how
> do you tell that it also happens to be the best write buffer size
> for the cache-tree writeout codepath (answer: you don't)?
Having said all that, I wouldn't be so opposed to an approach that
- declares that we need only one "default I/O buffer size";
- declares that the appropriate size for it is 8192;
- #define DEFAULT_IO_SIZE 8192;
- does something like your [PATCH 1/3], but not limited to strbuf
API, and
- covers also the writeout codepath of cache-tree, etc. that uses
hardcoded I/O buffer size.
The biggest trouble I had with the posted patches, especially the
[PATCH 2/3], was that I am quite sure that you wouldn't have used
STRBUF_HINT_SIZE in the cache-tree writeout codepath or commit-tree
codepath if they didn't use strbuf as a convenient way to get an
elastic buffer. The more relevant commonality across codepaths that
use 8192 is that the constant is used for sizing the I/O buffer, and
I got an impression that the 3-patch series posted did an incomplete
job of touching some that happen to use strbuf.
An approach that concentrated on the "right" commonality, i.e. we
have hardcoded magic constants for I/O buffer sizing, would have
covered copy.c, diff-delta.c, http-backend.c etc. that do not use
strbuf API where they have hardcoded 8k constants.
Thanks.
next prev parent reply other threads:[~2021-07-07 23:09 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-07 10:38 [PATCH 0/3] strbuf.[ch]: add STRBUF_HINT_SIZE, don't hardcode 8192 Ævar Arnfjörð Bjarmason
2021-07-07 10:38 ` [PATCH 1/3] strbuf.[ch]: add STRBUF_HINT_SIZE macro = 8192 Ævar Arnfjörð Bjarmason
2021-07-07 22:33 ` Junio C Hamano
2021-07-07 10:38 ` [PATCH 2/3] strbuf.h API users: don't hardcode 8192, use STRBUF_HINT_SIZE Ævar Arnfjörð Bjarmason
2021-07-07 20:10 ` Eric Sunshine
2021-07-07 22:37 ` Junio C Hamano
2021-07-07 23:09 ` Junio C Hamano [this message]
2021-07-07 23:22 ` Randall S. Becker
2021-07-12 17:58 ` Jeff King
2021-07-07 10:38 ` [PATCH 3/3] strbuf.[ch]: make strbuf_fread() take hint, not size Ævar Arnfjörð Bjarmason
2021-07-07 11:47 ` Han-Wen Nienhuys
2021-07-07 21:27 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqim1ld8lj.fsf@gitster.g \
--to=gitster@pobox.com \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).