From: Denton Liu <liu.denton@gmail.com>
To: Git Mailing List <git@vger.kernel.org>
Cc: Jeff King <peff@peff.net>, Eric Sunshine <sunshine@sunshineco.com>
Subject: [PATCH v2 0/7] remote-curl: fix deadlocks when remote server disconnects
Date: Mon, 18 May 2020 11:47:17 -0400 [thread overview]
Message-ID: <cover.1589816718.git.liu.denton@gmail.com> (raw)
In-Reply-To: <cover.1589393036.git.liu.denton@gmail.com>
The following command hangs forever:
$ git -c protocol.version=2 clone https://github.com/git/git.git --shallow-since=20151012
Cloning into 'git'...
This occurs because the --shallow-since arg is incorrect and the server
dies early. However, remote-curl does not realise that the server
errored out and just faithfully forwards the packets to fetch-pack
before waiting on more input from fetch-pack. Meanwhile, fetch-pack
keeps reading as it still expects more input. As a result, the processes
deadlock. Original analysis by Peff:
https://lore.kernel.org/git/20200328154936.GA1217052@coredump.intra.peff.net/
Changes since v1:
* Remove fallthrough in switch in favour of just extracting the common
call out of the switch in patch 3
* Add more detail in function comment and use `const char linelen[4]` in
patch 4
* Implement most of Peff's suggestions[0] in patch 5
* Only operate on stateless_connect() in patch 5
* Add tests in patch 5
* Drop "remote-curl: ensure last packet is a flush" in favour of
"stateless-connect: send response end packet"
[0]: https://lore.kernel.org/git/20200515213844.GD115445@coredump.intra.peff.net/
Denton Liu (7):
remote-curl: fix typo
remote-curl: remove label indentation
transport: extract common fetch_pack() call
pkt-line: extern packet_length()
remote-curl: error on incomplete packet
pkt-line: PACKET_READ_RESPONSE_END
stateless-connect: send response end packet
Documentation/gitremote-helpers.txt | 4 +-
Documentation/technical/protocol-v2.txt | 2 +
builtin/fetch-pack.c | 2 +-
connect.c | 12 +++-
fetch-pack.c | 12 ++++
pkt-line.c | 13 +++-
pkt-line.h | 11 +++
remote-curl.c | 72 +++++++++++++++++--
remote.h | 3 +-
serve.c | 2 +
t/helper/test-pkt-line.c | 4 ++
t/lib-httpd.sh | 2 +
t/lib-httpd/apache.conf | 8 +++
.../incomplete-body-upload-pack-v2-http.sh | 3 +
.../incomplete-length-upload-pack-v2-http.sh | 3 +
t/t5702-protocol-v2.sh | 47 ++++++++++++
transport.c | 16 ++---
17 files changed, 197 insertions(+), 19 deletions(-)
create mode 100644 t/lib-httpd/incomplete-body-upload-pack-v2-http.sh
create mode 100644 t/lib-httpd/incomplete-length-upload-pack-v2-http.sh
Range-diff against v1:
1: b390875f87 = 1: b390875f87 remote-curl: fix typo
2: a2b28c0b28 = 2: a2b28c0b28 remote-curl: remove label indentation
3: c89c184100 < -: ---------- transport: combine common cases with a fallthrough
-: ---------- > 3: 3a42575bd5 transport: extract common fetch_pack() call
4: 891a39c853 ! 4: c2b9d033bb pkt-line: extern packet_length()
@@ Commit message
need to access the length header. In order to simplify this, extern
packet_length() so that the logic can be reused.
+ Change the function parameter from a `const char *` to
+ `const char linelen[4]`. Even though these two types behave identically
+ as function parameters, use the array notation to semantically indicate
+ exactly what this function is expecting as an argument.
+
## pkt-line.c ##
@@ pkt-line.c: static int get_packet_data(int fd, char **src_buf, size_t *src_size,
return ret;
}
-static int packet_length(const char *linelen)
-+int packet_length(const char *linelen)
++int packet_length(const char linelen[4])
{
int val = hex2chr(linelen);
return (val < 0) ? val : (val << 8) | hex2chr(linelen + 2);
@@ pkt-line.h: int write_packetized_from_buf(const char *src_in, size_t len, int fd
*buffer, unsigned size, int options);
+/*
-+ * Reads a packetized line and returns the length header of the packet.
++ * Convert a four hex digit packet line length header into its numeric
++ * representation. linelen should not be null-terminated.
++ *
++ * If linelen contains non-hex characters, return -1. Otherwise, return the
++ * numeric value of the length header.
+ */
-+int packet_length(const char *linelen);
++int packet_length(const char linelen[4]);
+
/*
* Read a packetized line into a buffer like the 'packet_read()' function but
5: 3ed7cf87aa < -: ---------- remote-curl: error on incomplete packet
6: 7a689da2bb < -: ---------- remote-curl: ensure last packet is a flush
-: ---------- > 5: 52ce5fdffd remote-curl: error on incomplete packet
-: ---------- > 6: 744b078324 pkt-line: PACKET_READ_RESPONSE_END
-: ---------- > 7: 4b079bcd83 stateless-connect: send response end packet
--
2.26.2.706.g87896c9627
next prev parent reply other threads:[~2020-05-18 15:47 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-13 18:04 [PATCH 0/6] remote-curl: partial fix for a deadlock with stateless rpc Denton Liu
2020-05-13 18:04 ` [PATCH 1/6] remote-curl: fix typo Denton Liu
2020-05-13 18:04 ` [PATCH 2/6] remote-curl: remove label indentation Denton Liu
2020-05-13 18:04 ` [PATCH 3/6] transport: combine common cases with a fallthrough Denton Liu
2020-05-13 23:14 ` Eric Sunshine
2020-05-18 9:18 ` Denton Liu
2020-05-18 17:43 ` Eric Sunshine
2020-05-13 18:04 ` [PATCH 4/6] pkt-line: extern packet_length() Denton Liu
2020-05-13 23:23 ` Eric Sunshine
2020-05-15 20:56 ` Jeff King
2020-05-15 20:57 ` Jeff King
2020-05-13 18:04 ` [PATCH 5/6] remote-curl: error on incomplete packet Denton Liu
2020-05-15 21:38 ` Jeff King
2020-05-18 9:08 ` Denton Liu
2020-05-18 15:49 ` Jeff King
2020-05-13 18:04 ` [PATCH 6/6] remote-curl: ensure last packet is a flush Denton Liu
2020-05-15 21:02 ` Denton Liu
2020-05-15 21:41 ` Jeff King
2020-05-18 16:34 ` Junio C Hamano
2020-05-18 16:52 ` Jeff King
2020-05-18 21:00 ` Jeff King
2020-05-18 15:47 ` Denton Liu [this message]
2020-05-18 15:47 ` [PATCH v2 1/7] remote-curl: fix typo Denton Liu
2020-05-18 15:47 ` [PATCH v2 2/7] remote-curl: remove label indentation Denton Liu
2020-05-18 18:37 ` Junio C Hamano
2020-05-18 15:47 ` [PATCH v2 3/7] transport: extract common fetch_pack() call Denton Liu
2020-05-18 18:40 ` Junio C Hamano
2020-05-18 15:47 ` [PATCH v2 4/7] pkt-line: extern packet_length() Denton Liu
2020-05-18 16:04 ` Jeff King
2020-05-18 17:50 ` Eric Sunshine
2020-05-18 20:08 ` Jeff King
2020-05-18 18:44 ` Junio C Hamano
2020-05-18 15:47 ` [PATCH v2 5/7] remote-curl: error on incomplete packet Denton Liu
2020-05-18 16:22 ` Jeff King
2020-05-18 16:51 ` Denton Liu
2020-05-18 15:47 ` [PATCH v2 6/7] pkt-line: PACKET_READ_RESPONSE_END Denton Liu
2020-05-18 15:47 ` [PATCH v2 7/7] stateless-connect: send response end packet Denton Liu
2020-05-18 16:43 ` Jeff King
2020-05-18 17:12 ` Denton Liu
2020-05-18 17:26 ` Jeff King
2020-05-18 16:50 ` [PATCH v2 0/7] remote-curl: fix deadlocks when remote server disconnects Jeff King
2020-05-18 17:36 ` Denton Liu
2020-05-18 20:58 ` Jeff King
2020-05-18 22:52 ` Junio C Hamano
2020-05-19 2:38 ` Jeff King
2020-05-18 19:36 ` Junio C Hamano
2020-05-19 10:53 ` [PATCH v3 " Denton Liu
2020-05-19 10:53 ` [PATCH v3 1/7] remote-curl: fix typo Denton Liu
2020-05-19 10:53 ` [PATCH v3 2/7] remote-curl: remove label indentation Denton Liu
2020-05-19 10:53 ` [PATCH v3 3/7] transport: extract common fetch_pack() call Denton Liu
2020-05-19 10:53 ` [PATCH v3 4/7] pkt-line: extern packet_length() Denton Liu
2020-05-19 16:23 ` Eric Sunshine
2020-05-19 10:53 ` [PATCH v3 5/7] remote-curl: error on incomplete packet Denton Liu
2020-05-19 10:53 ` [PATCH v3 6/7] pkt-line: define PACKET_READ_RESPONSE_END Denton Liu
2020-05-19 10:54 ` [PATCH v3 7/7] stateless-connect: send response end packet Denton Liu
2020-05-19 18:40 ` [PATCH v3 0/7] remote-curl: fix deadlocks when remote server disconnects Jeff King
2020-05-19 21:14 ` Denton Liu
2020-05-19 20:51 ` [PATCH v3 8/7] fixup! pkt-line: extern packet_length() Denton Liu
2020-05-22 13:33 ` [PATCH v3 9/9] fixup! remote-curl: error on incomplete packet Denton Liu
2020-05-22 15:54 ` Jeff King
2020-05-22 16:05 ` Denton Liu
2020-05-22 16:31 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1589816718.git.liu.denton@gmail.com \
--to=liu.denton@gmail.com \
--cc=git@vger.kernel.org \
--cc=peff@peff.net \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).