From: Stefan Beller <sbeller@google.com>
To: Jeff King <peff@peff.net>
Cc: "git@vger.kernel.org" <git@vger.kernel.org>
Subject: Re: [PATCH 12/12] receive-pack: send keepalives during quiet periods
Date: Tue, 19 Jul 2016 09:05:14 -0700 [thread overview]
Message-ID: <CAGZ79kZ7m94T5+SHjeWcY_cDLYSP9X74eDOOjfmZ1GwAb-C-7g@mail.gmail.com> (raw)
In-Reply-To: <20160719100730.GA5193@sigill.intra.peff.net>
On Tue, Jul 19, 2016 at 3:07 AM, Jeff King <peff@peff.net> wrote:
> On Mon, Jul 18, 2016 at 10:28:25PM -0700, Stefan Beller wrote:
>
>> On Sat, Jul 16, 2016 at 12:56 AM, Jeff King <peff@peff.net> wrote:
>> >> > + if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
>> >> > + const char *p = memchr(data, '\0', sz);
>> >> > + if (p) {
>> >> > + /*
>> >> > + * The NUL tells us to start sending keepalives. Make
>> >> > + * sure we send any other data we read along
>> >> > + * with it.
>> >> > + */
>> >> > + keepalive_active = 1;
>> >> > + send_sideband(1, 2, data, p - data, use_sideband);
>> >> > + send_sideband(1, 2, p + 1, sz - (p - data + 1), use_sideband);
>> >> > + continue;
>> >>
>> >> Oh, I see why the turn_on_keepalive_on_NUL doesn't work as well as I thought.
>> >> I wonder if we can use a better read function, that would stop reading at a NUL,
>> >> and return early instead?
>> >
>> > How would you do that, if not by read()ing a byte at a time (which is
>> > inefficient)? Otherwise you have to deal with the leftovers (after the
>> > NUL) in your buffer. It's one of the reasons I went with a single-byte
>> > signal, because otherwise it gets rather complicated to do robustly.
>>
>> I do not question the concept of a single NUL byte, but rather the
>> implementation, i.e. if we had an xread_until_nul you would not need
>> to have a double send_sideband here?
>
> What would xread_until_nul() look like?
>
> The only reasonable implementation I can think of is:
>
> ssize_t xread_until_nul(int fd, char *out, size_t len)
> {
> ssize_t total = 0;
> while (total < len) {
> ssize_t ret = xread(fd, out + total, 1);
> if (ret < 0) {
> /* Oops, anything in out[0..total] is lost, but
> * we have no way of signaling both a partial
> * read and an error at the end; fixable by
> * changing the interface, but doesn't really
> * matter in practice for this application. */
> return -1;
> }
> if (ret == 0)
> break; /* EOF, stop reading */
> if (out[total] == '\0')
> break; /* got our NUL, stop reading */
> total++;
> }
> return total;
> }
>
> There are two problems with this function:
>
> 1. Until we see the NUL, we're making an excessive number of read()
> syscalls looking for it. You could make larger reads, but then what
> do you do with the residual bytes (i.e., the ones after the NUL in
> the buffer you read)? You'd have to somehow save it to return at
> the next xread_until_nul(). Which implie some kind of internal
> buffering.
>
> The reason there are two send_sidebands is to cover the case where
> we have some real data, then the signal byte, then some more data.
> So instead of buffering, we just handle the data immediately.
>
> 2. How does it know when to return?
>
> We want to send the data as soon as it is available, in as large a
> chunk as possible. Using a single xread() as we do now, we get
> whatever the OS has for us, up to our buffer size.
>
> But after each 1-byte read above, we would not want to return;
> there might be more data. So it keeps reading until NUL or we fill
> our buffer. But that may mean the xread() blocks, even though we
> have data that _could_ be shipped over the sideband.
>
> To fix that, you'd have to poll() for each xread(), and return when
> it says nothing's ready.
>
> I know that writing the function myself and then critiquing is the very
> definition of a strawman. :) But it's the best I could think of. Maybe
> you had something more clever in mind?
Actually no, I had not. I was hoping you could come up with a clever thing.
My original point was the perceived added complexity to a simple
seemingly simple function (copy_to_sideband in your original patch),
so my gut reaction was to shove the complexity away into a helper function.
But no matter how it is done, there is always this one function that looks
complex for this problem. So I think your original approach is fine then?
Thanks,
Stefan
>
> -Peff
next prev parent reply other threads:[~2016-07-19 16:05 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-15 10:25 [PATCH 0/12] push progress reporting and keepalives Jeff King
2016-07-15 10:26 ` [PATCH 01/12] check_everything_connected: always pass --quiet to rev-list Jeff King
2016-07-15 10:28 ` [PATCH 02/12] rev-list: add optional progress reporting Jeff King
2016-07-15 18:00 ` Junio C Hamano
2016-07-16 1:23 ` Jeff King
2016-07-15 10:28 ` [PATCH 03/12] check_everything_connected: convert to argv_array Jeff King
2016-07-15 10:30 ` [PATCH 04/12] check_everything_connected: use a struct with named options Jeff King
2016-07-15 18:13 ` Junio C Hamano
2016-07-16 1:24 ` Jeff King
2016-07-15 10:32 ` [PATCH 05/12] check_connected: relay errors to alternate descriptor Jeff King
2016-07-15 18:19 ` Junio C Hamano
2016-07-16 1:27 ` Jeff King
2016-07-15 10:32 ` [PATCH 06/12] check_connected: add progress flag Jeff King
2016-07-15 10:33 ` [PATCH 07/12] clone: use a real progress meter for connectivity check Jeff King
2016-07-15 10:34 ` [PATCH 08/12] index-pack: add flag for showing delta-resolution progress Jeff King
2016-07-15 10:35 ` [PATCH 09/12] receive-pack: turn on index-pack resolving progress Jeff King
2016-07-15 10:36 ` [PATCH 10/12] receive-pack: relay connectivity errors to sideband Jeff King
2016-07-15 10:36 ` [PATCH 11/12] receive-pack: turn on connectivity progress Jeff King
2016-07-15 10:43 ` [PATCH 12/12] receive-pack: send keepalives during quiet periods Jeff King
2016-07-15 17:24 ` Stefan Beller
2016-07-16 7:56 ` Jeff King
2016-07-19 5:28 ` Stefan Beller
2016-07-19 10:07 ` Jeff King
2016-07-19 16:05 ` Stefan Beller [this message]
2016-07-20 13:28 ` Jeff King
2016-07-15 19:18 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAGZ79kZ7m94T5+SHjeWcY_cDLYSP9X74eDOOjfmZ1GwAb-C-7g@mail.gmail.com \
--to=sbeller@google.com \
--cc=git@vger.kernel.org \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).