From: Daniel Duvall <dan@mutual.io>
To: git@vger.kernel.org
Cc: Daniel Duvall <dan@mutual.io>
Subject: [PATCH v3] upload-pack: allow stateless client EOF just prior to haves
Date: Fri, 30 Oct 2020 19:39:02 -0700 [thread overview]
Message-ID: <20201031023901.48193-1-dan@mutual.io> (raw)
In-Reply-To: <1604022059-18527-1-git-send-email-dan@mutual.io>
During stateless packfile negotiation where a depth is given, stateless
RPC clients (e.g. git-remote-curl) will send multiple upload-pack
requests with the first containing only the
wants/shallows/deepens/filters and the subsequent containing haves/done.
When upload-pack handles such requests, entering get_common_commits
without checking whether the client has hung up can result in unexpected
EOF during the negotiation loop and a die() with message "fatal: the
remote end hung up unexpectedly".
Real world effects include:
- A client speaking to git-http-backend via a server that doesn't check
the exit codes of CGIs (e.g. mod_cgi) doesn't know and doesn't care
about the fatal. It continues to process the response body as normal.
- A client speaking to a server that does check the exit code and
returns an errant HTTP status as a result will fail with the message
"error: RPC failed; HTTP 500 curl 22 The requested URL returned error:
500."
- Admins running servers that surface the failure must workaround it by
patching code that handles execution of git-http-backend to ignore exit
codes or take other heuristic approaches.
- Admins may have to deal with "hung up unexpectedly" log spam related
to the failures even in cases where the exit code isn't surfaced as an
HTTP server-side error status.
To avoid these EOF related fatals, have upload-pack gently peek for an
EOF between the sending of shallow/unshallow lines (followed by flush)
and the reading of client haves. If the client has hung up at this
point, exit normally.
Signed-off-by: Daniel Duvall <dan@mutual.io>
---
Changes in v2:
- Replaced unconditional flipping (XOR) of PACKET_READ_GENTLE_ON_EOF bit w/
`&= ~` to flip it back off (as it was when reader was initialized in
previous clause)
- Renamed test filename to group with other upload-pack related tests
- Refactored test using packetize helper
- Clarified in commit message that file descriptor is still valid but client
hangup/EOF is the core issue
- Added possible real-world effects of bug to commit message as suggested
Changes in v3:
- Moved test into existing t/t5530-upload-pack-error.sh
---
t/t5530-upload-pack-error.sh | 17 +++++++++++++++++
upload-pack.c | 13 ++++++++++++-
2 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/t/t5530-upload-pack-error.sh b/t/t5530-upload-pack-error.sh
index 205a2631e7..9dd2d2457a 100755
--- a/t/t5530-upload-pack-error.sh
+++ b/t/t5530-upload-pack-error.sh
@@ -88,6 +88,23 @@ test_expect_success 'upload-pack fails due to error in pack-objects enumeration'
grep "pack-objects died" output.err
'
+test_expect_success 'upload-pack tolerates EOF just after stateless client wants' '
+ test_commit initial &&
+ head=$(git rev-parse HEAD) &&
+
+ {
+ packetize "want $head" &&
+ packetize "shallow $head" &&
+ packetize "deepen 1" &&
+ printf "0000"
+ } >request &&
+
+ printf "0000" >expect &&
+
+ git upload-pack --stateless-rpc . <request >actual &&
+ test_cmp expect actual
+'
+
test_expect_success 'create empty repository' '
mkdir foo &&
diff --git a/upload-pack.c b/upload-pack.c
index 3b858eb457..5dc8e1f844 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1344,7 +1344,18 @@ void upload_pack(struct upload_pack_options *options)
PACKET_READ_DIE_ON_ERR_PACKET);
receive_needs(&data, &reader);
- if (data.want_obj.nr) {
+
+ /*
+ * An EOF at this exact point in negotiation should be
+ * acceptable from stateless clients as they will consume the
+ * shallow list before doing subsequent rpc with haves/etc.
+ */
+ if (data.stateless_rpc)
+ reader.options |= PACKET_READ_GENTLE_ON_EOF;
+
+ if (data.want_obj.nr &&
+ packet_reader_peek(&reader) != PACKET_READ_EOF) {
+ reader.options &= ~PACKET_READ_GENTLE_ON_EOF;
get_common_commits(&data, &reader);
create_pack_file(&data, NULL);
}
--
2.29.2
prev parent reply other threads:[~2020-10-31 2:39 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-30 1:40 [PATCH] upload-pack: allow stateless client EOF just prior to haves Daniel Duvall
2020-10-30 2:59 ` Eric Sunshine
2020-10-30 3:31 ` Daniel Duvall
2020-10-30 3:43 ` Daniel Duvall
2020-10-30 18:42 ` Junio C Hamano
2020-10-30 22:38 ` Daniel Duvall
2020-10-30 4:40 ` Jeff King
2020-10-30 7:47 ` Daniel Duvall
2020-10-30 9:09 ` Jeff King
2020-11-03 21:10 ` Junio C Hamano
2020-11-04 13:33 ` Jeff King
2020-11-04 14:06 ` Daniel Duvall
2020-11-04 18:25 ` Junio C Hamano
2020-10-30 22:35 ` [PATCH v2] " Daniel Duvall
2020-10-30 23:45 ` Junio C Hamano
2020-10-31 2:42 ` Daniel Duvall
2020-10-31 4:17 ` Junio C Hamano
2020-10-31 2:39 ` Daniel Duvall [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201031023901.48193-1-dan@mutual.io \
--to=dan@mutual.io \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).