git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: "René Scharfe" <l.s.r@web.de>
Cc: Junio C Hamano <gitster@pobox.com>,
	git@vger.kernel.org,
	Johannes Schindelin <Johannes.Schindelin@gmx.de>
Subject: [PATCH v2 4/6] pipe_command(): avoid xwrite() for writing to pipe
Date: Wed, 17 Aug 2022 02:08:06 -0400	[thread overview]
Message-ID: <YvyFxry9+2x05Di3@coredump.intra.peff.net> (raw)
In-Reply-To: <YvyE1RDFVAg4uQo1@coredump.intra.peff.net>

If xwrite() sees an EAGAIN response, it will loop forever until the
write succeeds (or encounters a real error). This is due to ef1cf0167a
(xwrite: poll on non-blocking FDs, 2016-06-26), with the idea that we
won't be surprised by a descriptor unexpectedly set as non-blocking.

But that will make things awkward when we do want a non-blocking
descriptor, and a future patch will switch pipe_command() to using one.
In that case, looping on EAGAIN is bad, because the process on the other
end of the pipe may be waiting on us before doing another read() on the
pipe, which would mean we deadlock.

In practice we're not supposed to ever see EAGAIN here, since poll()
will have just told us the descriptor is ready for writing. But our
Windows emulation of poll() will always return "ready" for writing to a
pipe descriptor! This is due to 94f4d01932 (mingw: workaround for hangs
when sending STDIN, 2020-02-17).

Our best bet in that case is to keep handling other descriptors, as any
read() we do may allow the child command to make forward progress (i.e.,
its write() finishes, and then it read()s from its stdin, freeing up
space in the pipe buffer). This means we might busy-loop between poll()
and write() on Windows if the child command is slow to read our input,
but it's much better than the alternative of deadlocking.

In practice, this busy-looping should be rare:

  - for small inputs, we'll just write the whole thing in a single
    write() anyway, non-blocking or not

  - for larger inputs where the child reads input and then processes it
    before writing (e.g., gpg verifying a signature), we may make a few
    extra write() calls that get EAGAIN during the initial write, but
    once it has taken in the whole input, we'll correctly block waiting
    to read back the data.

  - for larger inputs where the child process is streaming output back
    (like a diff filter), we'll likewise see some extra EAGAINs, but
    most of them will be followed immediately by a read(), which will
    let the child command make forward progress.

Of course it won't happen at all for now, since we don't yet use a
non-blocking pipe. This is just preparation for when we do.

Helped-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Jeff King <peff@peff.net>
---
 run-command.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/run-command.c b/run-command.c
index 14f17830f5..e078c3046f 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1364,12 +1364,24 @@ static int pump_io_round(struct io_pump *slots, int nr, struct pollfd *pfd)
 			continue;
 
 		if (io->type == POLLOUT) {
-			ssize_t len = xwrite(io->fd,
-					     io->u.out.buf, io->u.out.len);
+			ssize_t len;
+
+			/*
+			 * Don't use xwrite() here. It loops forever on EAGAIN,
+			 * and we're in our own poll() loop here.
+			 *
+			 * Note that we lose xwrite()'s handling of MAX_IO_SIZE
+			 * and EINTR, so we have to implement those ourselves.
+			 */
+			len = write(io->fd, io->u.out.buf,
+				    io->u.out.len <= MAX_IO_SIZE ?
+				    io->u.out.len : MAX_IO_SIZE);
 			if (len < 0) {
-				io->error = errno;
-				close(io->fd);
-				io->fd = -1;
+				if (errno != EINTR && errno != EAGAIN) {
+					io->error = errno;
+					close(io->fd);
+					io->fd = -1;
+				}
 			} else {
 				io->u.out.buf += len;
 				io->u.out.len -= len;
-- 
2.37.2.881.gb57357660c


  parent reply	other threads:[~2022-08-17  6:09 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-02  4:13 [RFC/PATCH] pipe_command(): mark stdin descriptor as non-blocking Jeff King
2022-08-02 15:04 ` Junio C Hamano
2022-08-02 15:39 ` Jeff King
2022-08-02 16:16   ` Junio C Hamano
2022-08-03  3:53     ` [PATCH v2] " Jeff King
2022-08-03 16:45       ` René Scharfe
2022-08-03 17:20         ` Jeff King
2022-08-03 21:56           ` René Scharfe
2022-08-05 15:36             ` Jeff King
2022-08-05 21:13               ` René Scharfe
2022-08-07 10:15                 ` René Scharfe
2022-08-07 17:41                   ` Jeff King
2022-08-10  5:39                     ` René Scharfe
2022-08-10 19:53                       ` Jeff King
2022-08-10 22:35                         ` René Scharfe
2022-08-11  8:52                           ` Jeff King
2022-08-10  5:39                     ` [PATCH] mingw: handle writes to non-blocking pipe René Scharfe
2022-08-10  9:07                       ` Johannes Schindelin
2022-08-10 20:02                       ` Jeff King
2022-08-10 22:34                         ` René Scharfe
2022-08-11  8:47                           ` Jeff King
2022-08-11 17:35                             ` René Scharfe
2022-08-11 18:20                               ` Jeff King
2022-08-14 15:37                                 ` René Scharfe
2022-08-17  5:39                                   ` Jeff King
2022-08-17  6:04                                     ` [PATCH v2 0/6] fix pipe_command() deadlock Jeff King
2022-08-17  6:04                                       ` [PATCH v2 1/6] compat: add function to enable nonblocking pipes Jeff King
2022-08-17 20:23                                         ` Junio C Hamano
2022-08-18  5:41                                           ` Jeff King
2022-08-17  6:05                                       ` [PATCH v2 2/6] nonblock: support Windows Jeff King
2022-08-17  6:06                                       ` [PATCH v2 3/6] git-compat-util: make MAX_IO_SIZE define globally available Jeff King
2022-08-17  6:08                                       ` Jeff King [this message]
2022-08-17  6:09                                       ` [PATCH v2 5/6] pipe_command(): handle ENOSPC when writing to a pipe Jeff King
2022-08-17 18:57                                         ` Junio C Hamano
2022-08-18  5:38                                           ` Jeff King
2022-08-17  6:10                                       ` [PATCH v2 6/6] pipe_command(): mark stdin descriptor as non-blocking Jeff King
2022-08-17  6:20                                       ` [PATCH v2 0/6] fix pipe_command() deadlock Jeff King
2022-08-19 21:19                                       ` René Scharfe
2022-08-20  7:04                                         ` Jeff King
2022-08-07 10:14               ` [PATCH v2] pipe_command(): mark stdin descriptor as non-blocking René Scharfe
2022-08-08 12:55                 ` Johannes Schindelin
2022-08-08 12:59       ` Johannes Schindelin
2022-08-09 13:04         ` Jeff King
2022-08-09 22:10           ` Johannes Schindelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YvyFxry9+2x05Di3@coredump.intra.peff.net \
    --to=peff@peff.net \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=l.s.r@web.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).