error: git-remote-https died of signal 13

error: git-remote-https died of signal 13

[Greg M]

Hi,

Using git version 1.9.2 I am getting this error:

[normal@laptop tmp]$ git clone https://github.com/mozilla/rust.git
Cloning into 'rust'...
remote: Reusing existing pack: 296648, done.
remote: Counting objects: 80, done.
remote: Compressing objects: 100% (77/77), done.
remote: Total 296728 (delta 22), reused 9 (delta 3)
Receiving objects: 100% (296728/296728), 110.68 MiB | 190.00 KiB/s, done.
Resolving deltas: 100% (238828/238828), done.
Checking connectivity... done.
error: git-remote-https died of signal 13

The repository appears to be cloned fine, I can clone other repositories
without error.

I appear to have the exact same symptoms as Stefan in November 2013 as
archived here:
http://thread.gmane.org/gmane.comp.version-control.git/238242/focus=238311.
In this case the cause was a curl error that was fixed in version curl
version 7.34...

I have curl version 7.36 though, in case some of the other output matters:

[normal@laptop tmp]$ curl --version
curl 7.36.0 (x86_64-unknown-linux-gnu) libcurl/7.36.0 OpenSSL/1.0.1g
zlib/1.2.8 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp
scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz
TLS-SRP

The curl command to reproduce the since fixed bug (curl --limit-rate 250k
-L http://github.com/raspberrypi/firmware/archive/4ade27942e.tar.gz >
/dev/null) runs without error and exits with a value of 0.

The first response to Stefan's bug report requested the output of the
command with GIT_TRANSPORT_HELPER_DEBUG=1 and strace -f, so I've included
that:

GIT_TRANSPORT...:
https://drive.google.com/file/d/0BxdiDTQp3MYXQWJJaVdydUstbWs/edit?usp=sharing
strace (33M download, 330M uncompressed):
https://drive.google.com/file/d/0BxdiDTQp3MYXb1VKWWtFUDdvbjQ/edit?usp=sharing

Thanks,
Greg

Re: error: git-remote-https died of signal 13

[Jeff King]

On Sun, Apr 20, 2014 at 08:42:15PM -0400, Greg M wrote:

> Using git version 1.9.2 I am getting this error:
> 
> [normal@laptop tmp]$ git clone https://github.com/mozilla/rust.git
> Cloning into 'rust'...
> remote: Reusing existing pack: 296648, done.
> remote: Counting objects: 80, done.
> remote: Compressing objects: 100% (77/77), done.
> remote: Total 296728 (delta 22), reused 9 (delta 3)
> Receiving objects: 100% (296728/296728), 110.68 MiB | 190.00 KiB/s, done.
> Resolving deltas: 100% (238828/238828), done.
> Checking connectivity... done.
> error: git-remote-https died of signal 13

Thanks for a thorough bug report. I looked through your strace output,
and this really does look like another case of OpenSSL getting SIGPIPE
while closing the connection.

It's odd, though, as your curl version has my patches, along with
similar extra fixes in 854aca5 (multi: ignore sigpipe internally,
2014-02-17). But I guess there may be a code path that needs similar
treatment.

The easiest way to find it is probably to attach a debugger to the
running git-remote-https, and get a backtrace when it dies from SIGPIPE.
You'll probably want to install your system's debug packages for curl,
too.

> I have curl version 7.36 though, in case some of the other output matters:
> 
> [normal@laptop tmp]$ curl --version
> curl 7.36.0 (x86_64-unknown-linux-gnu) libcurl/7.36.0 OpenSSL/1.0.1g
> zlib/1.2.8 libssh2/1.4.3

Another possibility is that your curl binary is up-to-date, but you are
linking against an older version of libcurl that does not have the
SIGPIPE workarounds.

I'm not sure of the best way to check that, but a hacky way under Linux
is:

  $ ldd $(git --exec-path)/git-remote-https | grep libcurl
          libcurl.so.4 => /usr/lib/x86_64-linux-gnu/libcurl.so.4
  $ strings  /usr/lib/x86_64-linux-gnu/libcurl.so.4 | grep '7\.'
  CLIENT libcurl 7.36.0

-Peff

Re: error: git-remote-https died of signal 13

[Greg M]

On Wed, Apr 23, 2014 at 2:59 AM, Jeff King <peff@peff.net> wrote:
> On Sun, Apr 20, 2014 at 08:42:15PM -0400, Greg M wrote:
>
>> Using git version 1.9.2 I am getting this error:
>>
>> [normal@laptop tmp]$ git clone https://github.com/mozilla/rust.git
>> Cloning into 'rust'...
>> remote: Reusing existing pack: 296648, done.
>> remote: Counting objects: 80, done.
>> remote: Compressing objects: 100% (77/77), done.
>> remote: Total 296728 (delta 22), reused 9 (delta 3)
>> Receiving objects: 100% (296728/296728), 110.68 MiB | 190.00 KiB/s, done.
>> Resolving deltas: 100% (238828/238828), done.
>> Checking connectivity... done.
>> error: git-remote-https died of signal 13
>
> Thanks for a thorough bug report. I looked through your strace output,
> and this really does look like another case of OpenSSL getting SIGPIPE
> while closing the connection.
>
> It's odd, though, as your curl version has my patches, along with
> similar extra fixes in 854aca5 (multi: ignore sigpipe internally,
> 2014-02-17). But I guess there may be a code path that needs similar
> treatment.
>
> The easiest way to find it is probably to attach a debugger to the
> running git-remote-https, and get a backtrace when it dies from SIGPIPE.
> You'll probably want to install your system's debug packages for curl,
> too.
>

The backtrace:

Program received signal SIGPIPE, Broken pipe.
0x00007fdcfd511a2d in write () from /usr/lib/libpthread.so.0
(gdb) bt
#0  0x00007fdcfd511a2d in write () from /usr/lib/libpthread.so.0
#1  0x00007fdcfd81a0f6 in sock_write () from /usr/lib/libcrypto.so.1.0.0
#2  0x00007fdcfd817edb in BIO_write () from /usr/lib/libcrypto.so.1.0.0
#3  0x00007fdcfc662902 in ssl3_write_pending () from /usr/lib/libssl.so.1.0.0
#4  0x00007fdcfc664b77 in ssl3_dispatch_alert () from /usr/lib/libssl.so.1.0.0
#5  0x00007fdcfc660822 in ssl3_shutdown () from /usr/lib/libssl.so.1.0.0
#6  0x00007fdcfd2e944e in Curl_ossl_close () from /usr/lib/libcurl.so.4
#7  0x00007fdcfd2b6459 in Curl_disconnect () from /usr/lib/libcurl.so.4
#8  0x00007fdcfd2c8131 in curl_multi_cleanup () from /usr/lib/libcurl.so.4
#9  0x000000000040764b in ?? ()
#10 0x0000000000404e4d in ?? ()
#11 0x00007fdcfcf0fb05 in __libc_start_main () from /usr/lib/libc.so.6
#12 0x000000000040552e in ?? ()

> Another possibility is that your curl binary is up-to-date, but you are
> linking against an older version of libcurl that does not have the
> SIGPIPE workarounds.
>
> I'm not sure of the best way to check that, but a hacky way under Linux
> is:
>
>   $ ldd $(git --exec-path)/git-remote-https | grep libcurl
>           libcurl.so.4 => /usr/lib/x86_64-linux-gnu/libcurl.so.4
>   $ strings  /usr/lib/x86_64-linux-gnu/libcurl.so.4 | grep '7\.'
>   CLIENT libcurl 7.36.0
>
> -Peff

Seems to actually be that version of libcurl:

[normal@laptop tmp]$ ldd $(git --exec-path)/git-remote-https | grep libcurl
    libcurl.so.4 => /usr/lib/libcurl.so.4 (0x00007ff1cce5e000)
[normal@laptop tmp]$ strings /usr/lib/libcurl.so.4 | grep '7\.'
7.36f
CLIENT libcurl 7.36.0
CLIENT libcurl 7.36.0
CLIENT libcurl 7.36.0
7.36.0
[normal@laptop tmp]$

Also I don't think I have any other versions to link against:

[normal@laptop tmp]$ ls -l /usr/lib/libcurl*
lrwxrwxrwx 1 root root     16 Mar 26 10:12 /usr/lib/libcurl.so ->
libcurl.so.4.3.0
lrwxrwxrwx 1 root root     16 Mar 26 10:12 /usr/lib/libcurl.so.4 ->
libcurl.so.4.3.0
-rwxr-xr-x 1 root root 443488 Mar 26 10:12 /usr/lib/libcurl.so.4.3.0

Greg

Re: error: git-remote-https died of signal 13

[Jeff King]

On Wed, Apr 23, 2014 at 07:49:09AM -0400, Greg M wrote:

> > The easiest way to find it is probably to attach a debugger to the
> > running git-remote-https, and get a backtrace when it dies from SIGPIPE.
> > You'll probably want to install your system's debug packages for curl,
> > too.
> >
> 
> The backtrace:
> 
> Program received signal SIGPIPE, Broken pipe.
> 0x00007fdcfd511a2d in write () from /usr/lib/libpthread.so.0
> (gdb) bt
> #0  0x00007fdcfd511a2d in write () from /usr/lib/libpthread.so.0
> #1  0x00007fdcfd81a0f6 in sock_write () from /usr/lib/libcrypto.so.1.0.0
> #2  0x00007fdcfd817edb in BIO_write () from /usr/lib/libcrypto.so.1.0.0
> #3  0x00007fdcfc662902 in ssl3_write_pending () from /usr/lib/libssl.so.1.0.0
> #4  0x00007fdcfc664b77 in ssl3_dispatch_alert () from /usr/lib/libssl.so.1.0.0
> #5  0x00007fdcfc660822 in ssl3_shutdown () from /usr/lib/libssl.so.1.0.0
> #6  0x00007fdcfd2e944e in Curl_ossl_close () from /usr/lib/libcurl.so.4
> #7  0x00007fdcfd2b6459 in Curl_disconnect () from /usr/lib/libcurl.so.4
> #8  0x00007fdcfd2c8131 in curl_multi_cleanup () from /usr/lib/libcurl.so.4
> #9  0x000000000040764b in ?? ()
> #10 0x0000000000404e4d in ?? ()
> #11 0x00007fdcfcf0fb05 in __libc_start_main () from /usr/lib/libc.so.6
> #12 0x000000000040552e in ?? ()

Thanks, that's very helpful. I wasn't able to reproduce your problem
locally, but I suspect the curl patch below may fix it:

diff --git a/lib/multi.c b/lib/multi.c
index bc93264..72e4825 100644
--- a/lib/multi.c
+++ b/lib/multi.c
@@ -1804,10 +1804,13 @@ static void close_all_connections(struct Curl_multi *multi)
 
   conn = Curl_conncache_find_first_connection(multi->conn_cache);
   while(conn) {
+    SIGPIPE_VARIABLE(pipe_st);
     conn->data = multi->closure_handle;
 
+    sigpipe_ignore(conn->data, &pipe_st);
     /* This will remove the connection from the cache */
     (void)Curl_disconnect(conn, FALSE);
+    sigpipe_restore(&pipe_st);
 
     conn = Curl_conncache_find_first_connection(multi->conn_cache);
   }

Let me know if you need any pointers on getting curl built or tested
with git.

Daniel, I think the similar fix to curl_multi_cleanup in commit a900d45
missed this code path, and we need something like the above patch. I
know you were trying to keep the SIGPIPE mess at the entrance-points to
the library, and this works against that. But we need a SessionHandle to
pass to sigpipe_ignore to look at its "no_signal" flag, and of course in
the case of multi we may have several such handles. If there's a similar
flag we can check on the multi handle, we could just cover all of
curl_multi_cleanup with a single ignore/reset pair.

-Peff

Re: error: git-remote-https died of signal 13

[Greg M]

On Thu, Apr 24, 2014 at 12:15 AM, Jeff King <peff@peff.net> wrote:
> I suspect the curl patch below may fix it:
>
> diff --git a/lib/multi.c b/lib/multi.c
> index bc93264..72e4825 100644
> --- a/lib/multi.c
> +++ b/lib/multi.c
> @@ -1804,10 +1804,13 @@ static void close_all_connections(struct Curl_multi *multi)
>
>    conn = Curl_conncache_find_first_connection(multi->conn_cache);
>    while(conn) {
> +    SIGPIPE_VARIABLE(pipe_st);
>      conn->data = multi->closure_handle;
>
> +    sigpipe_ignore(conn->data, &pipe_st);
>      /* This will remove the connection from the cache */
>      (void)Curl_disconnect(conn, FALSE);
> +    sigpipe_restore(&pipe_st);
>
>      conn = Curl_conncache_find_first_connection(multi->conn_cache);
>    }

The patch fixes the problem,

Greg

Re: error: git-remote-https died of signal 13

[Daniel Stenberg]

On Thu, 24 Apr 2014, Jeff King wrote:

> Thanks, that's very helpful. I wasn't able to reproduce your problem 
> locally, but I suspect the curl patch below may fix it:

...

> Daniel, I think the similar fix to curl_multi_cleanup in commit a900d45 
> missed this code path, and we need something like the above patch. I know 
> you were trying to keep the SIGPIPE mess at the entrance-points to the 
> library, and this works against that. But we need a SessionHandle to pass to 
> sigpipe_ignore to look at its "no_signal" flag, and of course in the case of 
> multi we may have several such handles. If there's a similar flag we can 
> check on the multi handle, we could just cover all of curl_multi_cleanup 
> with a single ignore/reset pair.

Thanks a lot for this! I've taken this issue over to the libcurl mailing list 
and we'll work out the best way to address it over there... At least we know 
the patch works as intended.

-- 

  / daniel.haxx.se

error: git-remote-https died of signal 13

[Stefan Beller]

Hi,

Using the latest git version (1.8.5.rc3), I get this
this warning/error:

sb@sb:/tmp$ git clone https://github.com/Bertram25/ValyriaTear.git
Cloning into 'ValyriaTear'...
remote: Counting objects: 21346, done.
remote: Compressing objects: 100% (6307/6307), done.
remote: Total 21346 (delta 16463), reused 19820 (delta 15000)
Receiving objects: 100% (21346/21346), 176.39 MiB | 445.00 KiB/s, done.
Resolving deltas: 100% (16463/16463), done.
Checking connectivity... done.
error: git-remote-https died of signal 13

However the repository seems to be cloned fine.
I can clone different repos from github and they are working fine
without the error.
Would that be an issue on my side or on githubs side?

Stefan

Re: error: git-remote-https died of signal 13

[Jeff King]

On Sat, Nov 23, 2013 at 05:36:36PM +0100, Stefan Beller wrote:

> sb@sb:/tmp$ git clone https://github.com/Bertram25/ValyriaTear.git
> Cloning into 'ValyriaTear'...
> remote: Counting objects: 21346, done.
> remote: Compressing objects: 100% (6307/6307), done.
> remote: Total 21346 (delta 16463), reused 19820 (delta 15000)
> Receiving objects: 100% (21346/21346), 176.39 MiB | 445.00 KiB/s, done.
> Resolving deltas: 100% (16463/16463), done.
> Checking connectivity... done.
> error: git-remote-https died of signal 13
> 
> However the repository seems to be cloned fine.
> I can clone different repos from github and they are working fine
> without the error.
> Would that be an issue on my side or on githubs side?

Almost certainly on your side. 13 is SIGPIPE, so git-remote-https is
trying to write something but the other side of the pipe has hung up.
This might be a race condition in the transport-helper protocol, where
we've had similar problems before.

It doesn't reproduce here for me. Can you reproduce it consistently on
this repo? I would not be at all surprised if it is intermittent.

If you can reproduce, it would be interesting to see the output with
GIT_TRANSPORT_HELPER_DEBUG=1, or even with "strace -f". That could at
least tell us what it was trying to write (and to where) when it got the
SIGPIPE.

-Peff

Re: error: git-remote-https died of signal 13

[Stefan Beller]

On 24.11.2013 07:54, Jeff King wrote:
> On Sat, Nov 23, 2013 at 05:36:36PM +0100, Stefan Beller wrote:
> 
>> sb@sb:/tmp$ git clone https://github.com/Bertram25/ValyriaTear.git
>> Cloning into 'ValyriaTear'...
>> remote: Counting objects: 21346, done.
>> remote: Compressing objects: 100% (6307/6307), done.
>> remote: Total 21346 (delta 16463), reused 19820 (delta 15000)
>> Receiving objects: 100% (21346/21346), 176.39 MiB | 445.00 KiB/s, done.
>> Resolving deltas: 100% (16463/16463), done.
>> Checking connectivity... done.
>> error: git-remote-https died of signal 13
>>
>> However the repository seems to be cloned fine.
>> I can clone different repos from github and they are working fine
>> without the error.
>> Would that be an issue on my side or on githubs side?
> 
> Almost certainly on your side. 13 is SIGPIPE, so git-remote-https is
> trying to write something but the other side of the pipe has hung up.
> This might be a race condition in the transport-helper protocol, where
> we've had similar problems before.
> 
> It doesn't reproduce here for me. Can you reproduce it consistently on
> this repo? I would not be at all surprised if it is intermittent.
>

I tested 3 times before sending the mail and I always got the error.
Testing now doesn't always trigger this problem. So yeah it's kind of intermittent.
 
> If you can reproduce, it would be interesting to see the output with
> GIT_TRANSPORT_HELPER_DEBUG=1, or even with "strace -f". That could at
> least tell us what it was trying to write (and to where) when it got the
> SIGPIPE.
> 
> -Peff
> 

Here is the output of 
sb@sb:/tmp$ GIT_TRANSPORT_HELPER_DEBUG=1 git clone https://github.com/Bertram25/ValyriaTear.git tmp
Cloning into 'tmp'...
Debug: Remote helper: -> capabilities
Debug: Remote helper: Waiting...
Debug: Remote helper: <- fetch
Debug: Got cap fetch
Debug: Remote helper: Waiting...
Debug: Remote helper: <- option
Debug: Got cap option
Debug: Remote helper: Waiting...
Debug: Remote helper: <- push
Debug: Got cap push
Debug: Remote helper: Waiting...
Debug: Remote helper: <- check-connectivity
Debug: Got cap check-connectivity
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 
Debug: Capabilities complete.
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 064275ea9c91688442ab215b5b641ac93fdd77b0 HEAD
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a32429792c9b393732bd026d42374f226c928559 refs/heads/0.6
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 6153fdaf34d9d3eec5fc1541b0373db6ef0eac25 refs/heads/HEI-Release
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 9d452017c18389120c662bfe5a3379a0c15458e7 refs/heads/HeroOfAllacrost
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 064275ea9c91688442ab215b5b641ac93fdd77b0 refs/heads/master
Debug: Remote helper: Waiting...
Debug: Remote helper: <- f5a3727f88d5c48f3c28af85bd32567772314c1a refs/heads/sdl2
Debug: Remote helper: Waiting...
Debug: Remote helper: <- aa2cc4a7961e5c85db8c971f8ca15febeea84ac9 refs/pull/101/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b707dc07da480829b46b31b114fa3335ac10066f refs/pull/101/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 995d48764fd0485e1fd2286768926d33f436b983 refs/pull/103/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 40c6020443fabf86b9516567caf4aa5b4478022d refs/pull/103/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 0fc5e0abd6e5070df90b155a54c80ec07aabb1f0 refs/pull/111/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- dd855fe5f09f9984dad933b9512305b9ab539c55 refs/pull/111/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- de0347d77025ad8b87da616c487c650b28834182 refs/pull/114/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 224b18b50c2a0a58c6566972b7d85a5451f5c3aa refs/pull/114/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- fae33cbba4cf3c0a786f23ae6ef1808c77e4d79f refs/pull/115/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 1659dffb9828946b77fc41ef175a91f94d908e55 refs/pull/115/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ee99c3b5c2e31482c89826c7821a8feadda0ec56 refs/pull/116/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ab9e412654cc8a3f50fcf63e73aa6232e35bd7a9 refs/pull/116/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a04786140b697f3d732caae99984d94c66d8da57 refs/pull/117/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b07e621e4bce831daf2881eb128b1ebef2b83414 refs/pull/117/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- c303913334cadcaefecb7ef8df51b9437a0d9d65 refs/pull/118/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- fd074efa3286be31b92c6380a85ce15799f3ce7d refs/pull/118/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 5eb3c686e430c8337652e56313a23a394482b552 refs/pull/119/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a69fdef2ab67447024133a84ce3835519809ef3a refs/pull/119/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- d217eaf9ee0758e5589a52ca03dde3e741ed139e refs/pull/120/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 2646a7baa37e7abecc32ef64fdc1a8d8390b0078 refs/pull/120/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 67d29a5a5880da9be97ce1b7986a42ca2728a535 refs/pull/133/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 3aa0ca69b8aeaf584da8214fb8659268ea0861fd refs/pull/133/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b72e8ca2f6983f814a230da6c97efac0092709db refs/pull/135/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- fb7bd1c2c190e2802924a04f9b9138680dc7feb6 refs/pull/135/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ecd8db92684f8b4b306ba22a3d031586867712eb refs/pull/136/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 6f3255b619407401d5e917414b9290b49d29c45e refs/pull/136/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- c818fb005b57df1b6a1702a17c426031f9a7d4f5 refs/pull/137/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 2f20102c596087ee993315ee63668aa0946e4ac7 refs/pull/137/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 8717cfb9f689b12420ed6c1d99b7287809aa08c1 refs/pull/139/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- c4342ccd4157722fc04a26a202205c7e1b8bf77e refs/pull/139/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b7b92e6a0997d8e564b058b25bc4c82426ca4e12 refs/pull/140/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 4035dd121da5218fe45044c1d4a5974e2724a9b6 refs/pull/140/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 47f4834f015392c588e0ee7204552a9c0f648a78 refs/pull/144/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 9a90bfa25c4d448227815b182d78d206a3785be9 refs/pull/145/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 07ff69b42879a1f4ae1d4c747e70a49c4d7f3b6c refs/pull/145/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 6fa31b5145cc9105523bfe358b215d9cb1022290 refs/pull/146/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 0d0ca67bced99463f0bd969b7cd5f6fe1d9dc4dc refs/pull/146/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- e3a4d29fd4c5b33ff7e3359dc0640ae5fcfcdba6 refs/pull/151/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b4e2bda0ba11d2225f4fbbf778a2d2666d9b10b9 refs/pull/151/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- f8a2e833944f1e07e603ea67cd2d2d18ebf52661 refs/pull/152/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b6ecd4822f403b9c7d6f376ab516856ad3b71ecd refs/pull/152/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- f93cde603cc4d5b566507495a9bfa6b7e33870e2 refs/pull/154/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- e52682db2a98cd5aff36531d2cac72797fdfc113 refs/pull/154/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b35e6ef9c42bd58e7dc93a625a6d38369e51eb45 refs/pull/16/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a4b3ec6df618e39076c98e9de9c7d3590cbd6948 refs/pull/16/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 77433715ae16062650125c3ca9c6e3cf4781f556 refs/pull/165/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- d344ffb0823d0b6eb783faa61362ace28ce3d25d refs/pull/165/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 807d1212c20a90b3272df065e2d760534de325e7 refs/pull/173/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ddfb5ab3b908fb742eb9600620f062461da96112 refs/pull/173/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 6cea8518d3d8f8c4712dceee2830f391c2ea1386 refs/pull/175/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- c6942e7329f65d7e16d3e2c460ed1b90af5204ef refs/pull/175/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 0b74db27b2b1db1dcfdd1b238fd5dcfc4725b3a8 refs/pull/176/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 487fb35d2558a62440487ca5466af53a1cfd1c3c refs/pull/176/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a2453e58058e57d9f8d55c12cafb42865981df93 refs/pull/177/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b9cd7ec469bf1470317106f4b3e4fa357c16f17c refs/pull/177/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 4ee3fc1bfaee3a9f80f672f11fb9fa7ee7de5ca4 refs/pull/180/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 12dd091a4f2979a1eceb7b775d803a503ca8eacd refs/pull/180/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 7fc474ca95fbd60c4ee2d61d5cfbdb9a681697a5 refs/pull/181/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 7fc03a7250a6a5fb6d29ad7175cb34ae5d897b85 refs/pull/181/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 1acdb17064604518c973e386d80d460a020f14c5 refs/pull/182/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 413e75591bce8c7e2797eabcc41ae3af46765bbd refs/pull/182/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 0009b73972bd3dcfc227df5b5cd9fe3033b7667a refs/pull/183/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 478ad00eeb9a0b0042d8ace521b6963fa79461a0 refs/pull/183/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 2b304b528b76000aa3eb2acef7d3ff56753174dc refs/pull/184/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 6844a78056eaa79cba3d5ffe4b8daaecbd2c05fe refs/pull/184/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- e99f34aaef9e070f2620f336447e0d6b284d9855 refs/pull/185/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 59e47dba9e3b1e80b30d2e88beab5b16983c8db3 refs/pull/185/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 4232e258a00614102b6476d2b79b37a1edc1366c refs/pull/187/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 7a436aa4f88e013f10c3de25bc482d0b8ad1aeeb refs/pull/187/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 11af4677088521ed0c5170baf5bbaceae07b9d45 refs/pull/188/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- fdf737cc4887d4ca1941dc82bf1554c522376e61 refs/pull/188/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- c72285165e716550c2b9df5fa1b759dc1d0c24ef refs/pull/189/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- debe335a4a68c902b3d2b157454619f259a1b551 refs/pull/189/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- e95006ba47f1fea8164a154dbea59d6f0e5ad475 refs/pull/190/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 29e47864f058438b4f3a44d837d9124a64171182 refs/pull/190/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- dd2640db2289b36cd3af001d296d04bf420c2f30 refs/pull/191/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 888b9fedef90129fac9caf24fc67aa3dce48163f refs/pull/191/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- bc3ec56780376ffe01786baf09945e98107446b2 refs/pull/192/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- cb40ac81e4a19a61dff274d6ff56bc731c8f9971 refs/pull/192/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 3a5d3c3af3a695c605daf93d583e1e02cd5c9a24 refs/pull/193/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- d703ebb170d59fb41541c6523e3b83af51ab07b0 refs/pull/193/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- d37e2d4360034128379f3083d54fc9769b26debb refs/pull/194/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 7378a71cac536206fafb5eb2258e2f9e98671f45 refs/pull/194/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 5b7d90a9fa1e06f5612cded44a247d75905be90c refs/pull/195/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 47bad1ecbf6fd00bb6e2ede9ed65b5ecc21b85f3 refs/pull/195/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- abe8a4a2a04abf33b41e0d3a5a11bdb8a1922978 refs/pull/196/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- def0e6f2650bdfa8d9dcdb8b6fdbcfda4f82eb84 refs/pull/196/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a0ac4fe552872927f5c93c326eca7be53aaa788c refs/pull/197/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 1a398e0cff4849840f2f4bd14eca0579cb1f7ed6 refs/pull/197/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 65c48a7d691840df90ebe260b25d83e2d969a69b refs/pull/200/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 38ef1e4e2cb68b32766facffdbdc74a43df748de refs/pull/200/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 0188e20da2e289862f93bbb093ca4522fb308de1 refs/pull/205/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 70d3eb9ce5788cc66d9ced7a1e3a78c4a09eccf3 refs/pull/205/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 65ee01faedf5c4e53d963277c32509007f91986f refs/pull/211/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b82b7dcecca61fe252f257efb524c0d7c5ab5c0f refs/pull/211/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 92ab6ecc3036ecfb2d8816b71abef06be6e507b5 refs/pull/213/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ebeac4813732feec180ffa3bc58d503cd5928455 refs/pull/213/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- e2bef652b8054406d13dafb7bae8bf6840c51a2f refs/pull/219/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- c5a947558c15a58e649b0799385c0832c61be603 refs/pull/219/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 3de31950417820b88ce181cff1e0741e9a50b785 refs/pull/225/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 615373767843d9b519691c63f1e22ffadea19b6a refs/pull/225/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b1136a8dbf9f85fb16bf74b2dca70c5f251b5e32 refs/pull/244/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b1d84a951623221dafc9a928cfcbdf8d2c9fee5a refs/pull/244/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 6b53813f0e5ac1fe7e005791d5b78163a6c0d8be refs/pull/249/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- cd3acb3a216aaf06506fa534cf431683628bc007 refs/pull/249/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 9006bfd8333a599b4a0354cf9f47898a652ee0e6 refs/pull/27/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b2d1abb763c02d2a46093a855cdba65f2a78a80c refs/pull/27/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 7536a0ce010b4667522fa3c30f470d13c3cb32da refs/pull/28/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 31a511e20dd4e51a09cba26495551481a548e9cd refs/pull/28/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 1ebb44d3baca29026a96e2741a8bc742bd82d648 refs/pull/29/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b49b3e90d731cbfe198e7af3367d693ac68fb4ce refs/pull/29/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b1b038a854fea47676ccf8331f33602a8c507a05 refs/pull/32/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 7ff9bf883561f18d31a27e2b3049d4e846175805 refs/pull/32/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 8951b125c207e4bf8a8432bab067072a33c83f18 refs/pull/34/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 83b7a22622618b545bfe184a6dcd169240c57914 refs/pull/34/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- aa7c6c13177b05eaeab97fdf69d19fa60b1c5b54 refs/pull/35/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- d319d6558ab545e84fe9e1c54921200c6a03b0c7 refs/pull/35/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a434292b75b12ae2a805ef5dbca29f396ac3b68f refs/pull/36/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 0a81bf44197ae941e77bc80a0e80c55b9c42b20e refs/pull/36/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 8634ae7c74fab76bb4ed8369786971bc502bdeaf refs/pull/40/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- f2833395e3900cb6eba591fc5e8037af1ca658c0 refs/pull/40/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- f050618b1bfa150e33a2027ad238542d0204ddbf refs/pull/47/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 949d74de7aac670b6a2016466571d283790d3cc7 refs/pull/47/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 87667df4232d81275b0b252ddfbdc85a0c4b1b87 refs/pull/50/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 94b850a29c4c1cb3c159d85d67139e4a364f0fb2 refs/pull/50/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ffdb4cd9e9f772a85799b0e2333d652c979d8f78 refs/pull/52/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 446e1359e752d94cdd7c9bea77099a588ef2c43c refs/pull/52/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- bf530b81110339279ebcd6546ef73efc957b7a5b refs/pull/59/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 80b665b9daf3e6eb9b7461c61f3854a6f4c99c20 refs/pull/59/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 495bc60651bea4b695bc270d6ae53d6041aa7c92 refs/pull/61/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 0d3fe6839f6cf2038484a8e76236b68c2d6a6f63 refs/pull/61/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 21557db0c13b091dbe81e28dd45ce29e852bd02a refs/pull/69/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 383ac7786539eca3d15e286fd18ac045947acacd refs/pull/69/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 832dea19d6b7cc55e7fde5bffb9f718cc5127e02 refs/pull/70/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 55a6649bef659871c667f043c062b617b55dcf1e refs/pull/70/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- f5837ecffc50b2eed1d208af697cba135b044229 refs/pull/71/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 38ec58be3f554fdd10e4291ec77785dcd260b8a3 refs/pull/71/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- d45d305f03611a4fdcddc517760e88aec6d67109 refs/pull/73/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 78dfbaeabc0fe2026ccba3f825b2cab0eb443834 refs/pull/73/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- d63d8ad0f3140f9e73dad8bd56c2ab53f535a399 refs/pull/76/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 60028c36110cd26418c0fb31efa392f6e0823179 refs/pull/76/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 9c66224187bd8d55b0cf9b2c0d15a886c5559222 refs/pull/77/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 4b1f63112800fb0a31160d6ea80bb91465771548 refs/pull/77/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 58462ec625d22c0f562acf4865d97e3082d89b1d refs/pull/78/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- e607ed1342bf7e38931019baee6d64fd41c98980 refs/pull/78/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 9ad448ffb3e49920b20d36c47e410a2d3d0fda12 refs/pull/79/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 34706ec386c721933a4f2d366ee2c951d77cf2d2 refs/pull/79/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 763549817b5af82da1e37ec2711a0f948ba83ef5 refs/pull/86/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 73463b18143a2e4393d567de41c681f7c329f347 refs/pull/86/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a388e5772e411107af684cfa1eac63875310a332 refs/pull/88/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- afbb8c2f38f6fbbf26dc8bd3d6bff925ce30d9b7 refs/pull/88/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 751ef19b2643b931db0811a89ae40cf34f60460f refs/pull/91/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- bc829ea8805a9eb766c389bc629d06c9c0b5ce80 refs/pull/91/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- fde3682d22a71e1848fbcbc7346e621518ff49e8 refs/pull/95/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 6bdad763038145e90c12ebac87629a84e95b0197 refs/pull/95/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- e13368413bd5c68f444164e1e944ab52b9487c13 refs/pull/96/head
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b4f62a2f45f9be4c7f8aa16d5f9c1d15853681d4 refs/pull/96/merge
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 3af3e0632b5475427646e87b8eb4285582f431d9 refs/tags/0.5.0
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 3cece960f3f62af3b0414a1e09013b68db28a186 refs/tags/0.5.1
Debug: Remote helper: Waiting...
Debug: Remote helper: <- a32429792c9b393732bd026d42374f226c928559 refs/tags/0.6.0
Debug: Remote helper: Waiting...
Debug: Remote helper: <- f805e6aeb4e837900ec365e3bcc659d8ca5a0e5b refs/tags/0.6.0-rc1
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 2945a5023280b0f407dc04e08a6efb72ea24cc7c refs/tags/Christmas-Pre-Release
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 4a419dd6c423d085466fbfd5d5adf1d996f74e04 refs/tags/Christmas-Pre-Release^{}
Debug: Remote helper: Waiting...
Debug: Remote helper: <- b452068830d19113fb3fdb51ee52232c0534ffbb refs/tags/HEI-RC2
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 93233be952de0dc656c9c8440147aaf4a3969567 refs/tags/HEI-RC2^{}
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 
Debug: Read ref listing.
Debug: Remote helper: -> option progress true
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ok
Debug: Remote helper: -> option verbosity 1
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ok
Debug: Remote helper: -> option check-connectivity true
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ok
Debug: Remote helper: -> fetch 064275ea9c91688442ab215b5b641ac93fdd77b0 HEAD
fetch a32429792c9b393732bd026d42374f226c928559 refs/heads/0.6
fetch 6153fdaf34d9d3eec5fc1541b0373db6ef0eac25 refs/heads/HEI-Release
fetch 9d452017c18389120c662bfe5a3379a0c15458e7 refs/heads/HeroOfAllacrost
fetch 064275ea9c91688442ab215b5b641ac93fdd77b0 refs/heads/master
fetch f5a3727f88d5c48f3c28af85bd32567772314c1a refs/heads/sdl2
fetch 3af3e0632b5475427646e87b8eb4285582f431d9 refs/tags/0.5.0
fetch 3cece960f3f62af3b0414a1e09013b68db28a186 refs/tags/0.5.1
fetch a32429792c9b393732bd026d42374f226c928559 refs/tags/0.6.0
fetch f805e6aeb4e837900ec365e3bcc659d8ca5a0e5b refs/tags/0.6.0-rc1
fetch 2945a5023280b0f407dc04e08a6efb72ea24cc7c refs/tags/Christmas-Pre-Release
fetch b452068830d19113fb3fdb51ee52232c0534ffbb refs/tags/HEI-RC2

Debug: Remote helper: Waiting...
remote: Counting objects: 21346, done.
remote: Compressing objects: 100% (6307/6307), done.
remote: Total 21346 (delta 16463), reused 19820 (delta 15000)
Receiving objects: 100% (21346/21346), 176.39 MiB | 619.00 KiB/s, done.
Resolving deltas: 100% (16463/16463), done.
Debug: Remote helper: <- lock /tmp/tmp/.git/objects/pack/pack-0fbfc20b972e818e42e3c83a68d80bf4bf9f80ef.keep
Debug: Remote helper: Waiting...
Debug: Remote helper: <- connectivity-ok
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 
Checking connectivity... done.
Debug: Disconnecting.
error: git-remote-https died of signal 13
sb@sb:/tmp$ 



The output of "strace -f git clone ..." is 100M, so attaching and sending to the list is no option,
but the error is also in there
	$ cat strace_err |grep died
	write(2, "error: git-remote-https died of "..., 42error: git-remote-https died of signal 13

If you're interested in peaking around that file, obtain it with
	wget http://www.stefanbeller.de/tmp/strace_err
 
Thanks,
Stefan

Re: error: git-remote-https died of signal 13

[Jeff King]

On Sun, Nov 24, 2013 at 01:54:34PM +0100, Stefan Beller wrote:

> Here is the output of 
> sb@sb:/tmp$ GIT_TRANSPORT_HELPER_DEBUG=1 git clone https://github.com/Bertram25/ValyriaTear.git tmp

Thanks. I think I see what is going on.

We finish the helper conversation here:

> Checking connectivity... done.
> Debug: Disconnecting.
> error: git-remote-https died of signal 13
> sb@sb:/tmp$ 

which means that remote-https is trying to exit, and is cleaning up any
curl connections. The actual SIGPIPE in the strace is here:

[pid 28319] write(3, "\25\3\2\0...[binary goo]...", 27) = -1 EPIPE (Broken pipe)

and if you walk backwards, fd 3 is:

  [pid 28319] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
  ...
  [pid 28319] connect(3, {sa_family=AF_INET,
                          sin_port=htons(443),
                          sin_addr=inet_addr("192.30.252.131")}, 16
                          ) = -1 EINPROGRESS (Operation now in progress)

So it's sending binary junk to the https socket while trying to exit,
which makes me guess that it's something to do with terminating the SSL
session, but the server has already hung up. Which would make it a curl
problem.

Googling "curl sigpipe" seems to come up with a report of this exact
case:

  http://curl.haxx.se/mail/archive-2013-01/0003.html

with a bug opened here:

  http://sourceforge.net/p/curl/bugs/1180/

Looks like the fix went into curl 7.32.0. I have 7.33.0, which seems
fine. Can you confirm that your libcurl is a bit older?

-Peff

Re: error: git-remote-https died of signal 13

[Stefan Beller]

On 24.11.2013 14:33, Jeff King wrote:
> On Sun, Nov 24, 2013 at 01:54:34PM +0100, Stefan Beller wrote:
> 
>> Here is the output of 
>> sb@sb:/tmp$ GIT_TRANSPORT_HELPER_DEBUG=1 git clone https://github.com/Bertram25/ValyriaTear.git tmp
> 
> Thanks. I think I see what is going on.
> 
> We finish the helper conversation here:
> 
>> Checking connectivity... done.
>> Debug: Disconnecting.
>> error: git-remote-https died of signal 13
>> sb@sb:/tmp$ 
> 
> which means that remote-https is trying to exit, and is cleaning up any
> curl connections. The actual SIGPIPE in the strace is here:
> 
> [pid 28319] write(3, "\25\3\2\0...[binary goo]...", 27) = -1 EPIPE (Broken pipe)
> 
> and if you walk backwards, fd 3 is:
> 
>   [pid 28319] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
>   ...
>   [pid 28319] connect(3, {sa_family=AF_INET,
>                           sin_port=htons(443),
>                           sin_addr=inet_addr("192.30.252.131")}, 16
>                           ) = -1 EINPROGRESS (Operation now in progress)
> 
> So it's sending binary junk to the https socket while trying to exit,
> which makes me guess that it's something to do with terminating the SSL
> session, but the server has already hung up. Which would make it a curl
> problem.
> 
> Googling "curl sigpipe" seems to come up with a report of this exact
> case:
> 
>   http://curl.haxx.se/mail/archive-2013-01/0003.html

I cannot reproduce the error using the curl command from that site.
curl returns with 0.

> 
> with a bug opened here:
> 
>   http://sourceforge.net/p/curl/bugs/1180/
> 
> Looks like the fix went into curl 7.32.0. I have 7.33.0, which seems
> fine. Can you confirm that your libcurl is a bit older?
> 

dpkg -l |grep curl
ii  curl                                      7.32.0-1ubuntu1                            amd64        command line tool for transferring data with URL syntax
ii  libcurl3:amd64                            7.32.0-1ubuntu1                            amd64        easy-to-use client-side URL transfer library (OpenSSL flavour)
ii  libcurl3-gnutls:amd64                     7.32.0-1ubuntu1                            amd64        easy-to-use client-side URL transfer library (GnuTLS flavour)
ii  libcurl4-openssl-dev                      7.32.0-1ubuntu1                            amd64        development files and documentation for libcurl (OpenSSL flavour)
ii  python-pycurl                             7.19.0-5ubuntu8                            amd64        Python bindings to libcurl

Thanks,
Stefan

Re: error: git-remote-https died of signal 13

[Jeff King]

[+cc Daniel, who worked on the curl fix]

On Sun, Nov 24, 2013 at 04:01:43PM +0100, Stefan Beller wrote:

> On 24.11.2013 14:33, Jeff King wrote:
> > On Sun, Nov 24, 2013 at 01:54:34PM +0100, Stefan Beller wrote:
> > 
> >> Here is the output of 
> >> sb@sb:/tmp$ GIT_TRANSPORT_HELPER_DEBUG=1 git clone https://github.com/Bertram25/ValyriaTear.git tmp
> > 
> > Thanks. I think I see what is going on.
> > 
> > We finish the helper conversation here:
> > 
> >> Checking connectivity... done.
> >> Debug: Disconnecting.
> >> error: git-remote-https died of signal 13
> >> sb@sb:/tmp$ 
> > 
> > which means that remote-https is trying to exit, and is cleaning up any
> > curl connections. The actual SIGPIPE in the strace is here:
> > 
> > [pid 28319] write(3, "\25\3\2\0...[binary goo]...", 27) = -1 EPIPE (Broken pipe)
> > 
> > and if you walk backwards, fd 3 is:
> > 
> >   [pid 28319] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
> >   ...
> >   [pid 28319] connect(3, {sa_family=AF_INET,
> >                           sin_port=htons(443),
> >                           sin_addr=inet_addr("192.30.252.131")}, 16
> >                           ) = -1 EINPROGRESS (Operation now in progress)
> > 
> > So it's sending binary junk to the https socket while trying to exit,
> > which makes me guess that it's something to do with terminating the SSL
> > session, but the server has already hung up. Which would make it a curl
> > problem.
> > 
> > Googling "curl sigpipe" seems to come up with a report of this exact
> > case:
> > 
> >   http://curl.haxx.se/mail/archive-2013-01/0003.html
> 
> I cannot reproduce the error using the curl command from that site.
> curl returns with 0.
> 
> > 
> > with a bug opened here:
> > 
> >   http://sourceforge.net/p/curl/bugs/1180/
> > 
> > Looks like the fix went into curl 7.32.0. I have 7.33.0, which seems
> > fine. Can you confirm that your libcurl is a bit older?
> > 
> 
> dpkg -l |grep curl
> ii  curl                                      7.32.0-1ubuntu1                            amd64        command line tool for transferring data with URL syntax
> ii  libcurl3:amd64                            7.32.0-1ubuntu1                            amd64        easy-to-use client-side URL transfer library (OpenSSL flavour)
> ii  libcurl3-gnutls:amd64                     7.32.0-1ubuntu1                            amd64        easy-to-use client-side URL transfer library (GnuTLS flavour)
> ii  libcurl4-openssl-dev                      7.32.0-1ubuntu1                            amd64        development files and documentation for libcurl (OpenSSL flavour)
> ii  python-pycurl                             7.19.0-5ubuntu8                            amd64        Python bindings to libcurl

Hmm. The fix in curl's 7d80ed64e435155 seems to involve strategically
placed calls to ignore SIGPIPE. I wonder if there is another spot that
needs similar treatment. It looks like curl_easy_cleanup is covered,
though, and that's where I would expect problem to come.

It would be interesting to see a backtrace from remote-curl when we get
the SIGPIPE. Doing so would be slightly tricky; instrumenting with the
patch below may be enough.

Another thought is that the curl fix seems to only kick in when built
with openssl support.  I'm not sure I understand how ubuntu's packaging
of curl uses gnutls versus openssl for the shared library. That may be
related.

-Peff

---
diff --git a/http.c b/http.c
index bcf54aa..ac709cc 100644
--- a/http.c
+++ b/http.c
@@ -473,13 +473,17 @@ void http_cleanup(void)
 {
 	struct active_request_slot *slot = active_queue_head;
 
+	warning("in http_cleanup");
 	while (slot != NULL) {
 		struct active_request_slot *next = slot->next;
 		if (slot->curl != NULL) {
 #ifdef USE_CURL_MULTI
+			warning("calling curl_multi_remove_handle");
 			curl_multi_remove_handle(curlm, slot->curl);
 #endif
+			warning("calling curl_easy_cleanup on slot");
 			curl_easy_cleanup(slot->curl);
+			warning("curl_easy_cleanup done");
 		}
 		free(slot);
 		slot = next;
@@ -487,13 +491,19 @@ void http_cleanup(void)
 	active_queue_head = NULL;
 
 #ifndef NO_CURL_EASY_DUPHANDLE
+	warning("calling curl_easy_cleanup on default");
 	curl_easy_cleanup(curl_default);
+	warning("curl_easy_cleanup done");
 #endif
 
 #ifdef USE_CURL_MULTI
+	warning("calling curl_multi_cleanup");
 	curl_multi_cleanup(curlm);
+	warning("curl_multi_cleanup done");
 #endif
+	warning("calling curl_global_cleanup");
 	curl_global_cleanup();
+	warning("curl_global_cleanup done");
 
 	curl_slist_free_all(pragma_header);
 	pragma_header = NULL;

Re: error: git-remote-https died of signal 13

[Stefan Beller]

On 24.11.2013 16:54, Jeff King wrote:
> Hmm. The fix in curl's 7d80ed64e435155 seems to involve strategically
> placed calls to ignore SIGPIPE. I wonder if there is another spot that
> needs similar treatment. It looks like curl_easy_cleanup is covered,
> though, and that's where I would expect problem to come.
> 
> It would be interesting to see a backtrace from remote-curl when we get
> the SIGPIPE. Doing so would be slightly tricky; instrumenting with the
> patch below may be enough.

Ok I'll test that now.

> 
> Another thought is that the curl fix seems to only kick in when built
> with openssl support.  I'm not sure I understand how ubuntu's packaging
> of curl uses gnutls versus openssl for the shared library. That may be
> related.
> 

A better information would be the --version from curl then:
curl --version
curl 7.32.0 (x86_64-pc-linux-gnu) libcurl/7.32.0 OpenSSL/1.0.1e zlib/1.2.8 libidn/1.28 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp 
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

Re: error: git-remote-https died of signal 13

[Stefan Beller]

On 24.11.2013 16:54, Jeff King wrote:
> [+cc Daniel, who worked on the curl fix]
> 
> On Sun, Nov 24, 2013 at 04:01:43PM +0100, Stefan Beller wrote:
> 
>> On 24.11.2013 14:33, Jeff King wrote:
>>> On Sun, Nov 24, 2013 at 01:54:34PM +0100, Stefan Beller wrote:
>>>
>>>> Here is the output of 
>>>> sb@sb:/tmp$ GIT_TRANSPORT_HELPER_DEBUG=1 git clone https://github.com/Bertram25/ValyriaTear.git tmp
>>>
>>> Thanks. I think I see what is going on.
>>>
>>> We finish the helper conversation here:
>>>
>>>> Checking connectivity... done.
>>>> Debug: Disconnecting.
>>>> error: git-remote-https died of signal 13
>>>> sb@sb:/tmp$ 
>>>
>>> which means that remote-https is trying to exit, and is cleaning up any
>>> curl connections. The actual SIGPIPE in the strace is here:
>>>
>>> [pid 28319] write(3, "\25\3\2\0...[binary goo]...", 27) = -1 EPIPE (Broken pipe)
>>>
>>> and if you walk backwards, fd 3 is:
>>>
>>>   [pid 28319] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
>>>   ...
>>>   [pid 28319] connect(3, {sa_family=AF_INET,
>>>                           sin_port=htons(443),
>>>                           sin_addr=inet_addr("192.30.252.131")}, 16
>>>                           ) = -1 EINPROGRESS (Operation now in progress)
>>>
>>> So it's sending binary junk to the https socket while trying to exit,
>>> which makes me guess that it's something to do with terminating the SSL
>>> session, but the server has already hung up. Which would make it a curl
>>> problem.
>>>
>>> Googling "curl sigpipe" seems to come up with a report of this exact
>>> case:
>>>
>>>   http://curl.haxx.se/mail/archive-2013-01/0003.html
>>
>> I cannot reproduce the error using the curl command from that site.
>> curl returns with 0.
>>
>>>
>>> with a bug opened here:
>>>
>>>   http://sourceforge.net/p/curl/bugs/1180/
>>>
>>> Looks like the fix went into curl 7.32.0. I have 7.33.0, which seems
>>> fine. Can you confirm that your libcurl is a bit older?
>>>
>>
>> dpkg -l |grep curl
>> ii  curl                                      7.32.0-1ubuntu1                            amd64        command line tool for transferring data with URL syntax
>> ii  libcurl3:amd64                            7.32.0-1ubuntu1                            amd64        easy-to-use client-side URL transfer library (OpenSSL flavour)
>> ii  libcurl3-gnutls:amd64                     7.32.0-1ubuntu1                            amd64        easy-to-use client-side URL transfer library (GnuTLS flavour)
>> ii  libcurl4-openssl-dev                      7.32.0-1ubuntu1                            amd64        development files and documentation for libcurl (OpenSSL flavour)
>> ii  python-pycurl                             7.19.0-5ubuntu8                            amd64        Python bindings to libcurl
> 
> Hmm. The fix in curl's 7d80ed64e435155 seems to involve strategically
> placed calls to ignore SIGPIPE. I wonder if there is another spot that
> needs similar treatment. It looks like curl_easy_cleanup is covered,
> though, and that's where I would expect problem to come.
> 
> It would be interesting to see a backtrace from remote-curl when we get
> the SIGPIPE. Doing so would be slightly tricky; instrumenting with the
> patch below may be enough.
> 

GIT_TRANSPORT_HELPER_DEBUG=1 git clone https://github.com/Bertram25/ValyriaTear.git
now ends with:

Debug: Remote helper: Waiting...
remote: Counting objects: 21354, done.
remote: Compressing objects: 100% (6249/6249), done.
remote: Total 21354 (delta 16466), reused 19888 (delta 15066)
Receiving objects: 100% (21354/21354), 176.42 MiB | 1.22 MiB/s, done.
Resolving deltas: 100% (16466/16466), done.
Debug: Remote helper: <- lock /tmp/ValyriaTear/.git/objects/pack/pack-b6f360ab28b5078a9aefafe1c4144e6c7782c317.keep
Debug: Remote helper: Waiting...
Debug: Remote helper: <- connectivity-ok
Debug: Remote helper: Waiting...
Debug: Remote helper: <- 
Checking connectivity... done.
Debug: Disconnecting.
warning: in http_cleanup
warning: calling curl_multi_remove_handle
warning: calling curl_easy_cleanup on slot
warning: curl_easy_cleanup done
warning: calling curl_easy_cleanup on default
warning: curl_easy_cleanup done
warning: calling curl_multi_cleanup
error: git-remote-https died of signal 13

Thanks,
Stefan

Re: error: git-remote-https died of signal 13

[Jeff King]

On Sun, Nov 24, 2013 at 05:32:34PM +0100, Stefan Beller wrote:

> GIT_TRANSPORT_HELPER_DEBUG=1 git clone https://github.com/Bertram25/ValyriaTear.git
> now ends with:
> 
> Debug: Remote helper: Waiting...
> remote: Counting objects: 21354, done.
> remote: Compressing objects: 100% (6249/6249), done.
> remote: Total 21354 (delta 16466), reused 19888 (delta 15066)
> Receiving objects: 100% (21354/21354), 176.42 MiB | 1.22 MiB/s, done.
> Resolving deltas: 100% (16466/16466), done.
> Debug: Remote helper: <- lock /tmp/ValyriaTear/.git/objects/pack/pack-b6f360ab28b5078a9aefafe1c4144e6c7782c317.keep
> Debug: Remote helper: Waiting...
> Debug: Remote helper: <- connectivity-ok
> Debug: Remote helper: Waiting...
> Debug: Remote helper: <- 
> Checking connectivity... done.
> Debug: Disconnecting.
> warning: in http_cleanup
> warning: calling curl_multi_remove_handle
> warning: calling curl_easy_cleanup on slot
> warning: curl_easy_cleanup done
> warning: calling curl_easy_cleanup on default
> warning: curl_easy_cleanup done
> warning: calling curl_multi_cleanup
> error: git-remote-https died of signal 13

Thanks. I'm having trouble reproducing the SIGPIPE locally, but I am
able to see via strace the write we make in curl_multi_cleanup. The
call stack is:

  curl_multi_cleanup
    -> close_all_connections
      -> Curl_disconnect
        -> Curl_ossl_close
          ...

Daniel, does the call to Curl_disconnect need to be wrapped with
sigpipe_ignore/reset, similar to 7d80ed64e435155?

-Peff

Re: error: git-remote-https died of signal 13

[Daniel Stenberg]

On Mon, 25 Nov 2013, Jeff King wrote:

> Thanks. I'm having trouble reproducing the SIGPIPE locally, but I am able to 
> see via strace the write we make in curl_multi_cleanup. The call stack is:
>
>  curl_multi_cleanup
>    -> close_all_connections
>      -> Curl_disconnect
>        -> Curl_ossl_close
>          ...
>
> Daniel, does the call to Curl_disconnect need to be wrapped with 
> sigpipe_ignore/reset, similar to 7d80ed64e435155?

Yes. It very much looks like that. The SSL "closing" is what was the problem I 
had to adress.

But I then decided that if a 3rd library has one way to generate SIGPIPE it 
may very well have another in a separate spot so I decided to do the wrap at 
the top level immediately in the entry point when getting called by the 
application. Following that, the SIGPIPE ignore/restore should rather be made 
in curl_multi_cleanup.

-- 

  / daniel.haxx.se

Re: error: git-remote-https died of signal 13

[Jeff King]

On Mon, Nov 25, 2013 at 08:20:18AM +0100, Daniel Stenberg wrote:

> >Daniel, does the call to Curl_disconnect need to be wrapped with
> >sigpipe_ignore/reset, similar to 7d80ed64e435155?
> 
> Yes. It very much looks like that. The SSL "closing" is what was the
> problem I had to adress.
> 
> But I then decided that if a 3rd library has one way to generate
> SIGPIPE it may very well have another in a separate spot so I decided
> to do the wrap at the top level immediately in the entry point when
> getting called by the application. Following that, the SIGPIPE
> ignore/restore should rather be made in curl_multi_cleanup.

Unfortunately, we need an actual SessionHandle to know whether it is OK
to reset signals at all. There may be a more elegant way of checking
that, but here's the patch series I came up with. It does turn off
SIGPIPE for the specific case I'm seeing (again, I'm having trouble
actually getting EPIPE in the first place, but from Stefan's strace, I
think this would fix his problem).

  [1/2]: factor out sigpipe_reset from easy.c
  [2/2]: ignore SIGPIPE during curl_multi_cleanup

-Peff

Re: error: git-remote-https died of signal 13

[Jeff King]

On Mon, Nov 25, 2013 at 09:32:13AM -0500, Jeff King wrote:

> > But I then decided that if a 3rd library has one way to generate
> > SIGPIPE it may very well have another in a separate spot so I decided
> > to do the wrap at the top level immediately in the entry point when
> > getting called by the application. Following that, the SIGPIPE
> > ignore/restore should rather be made in curl_multi_cleanup.
> 
> Unfortunately, we need an actual SessionHandle to know whether it is OK
> to reset signals at all. There may be a more elegant way of checking
> that, but here's the patch series I came up with.

Scratch that.  I had originally written something like:

  if (conn->data)
    sigpipe_ignore(conn->data, &pipe);
  Curl_disconnect(conn, ...);
  sigpipe_restore(&pipe);

but while sending it out, I realized that the "data" we attach to each
connection when we close it is just the multi->closure_handle. So I was
able to hoist the check out to curl_multi_cleanup (and that's what I
just sent).

-Peff

Re: error: git-remote-https died of signal 13

[Daniel Stenberg]

On Sun, 24 Nov 2013, Jeff King wrote:

> Hmm. The fix in curl's 7d80ed64e435155 seems to involve strategically placed 
> calls to ignore SIGPIPE. I wonder if there is another spot that needs 
> similar treatment. It looks like curl_easy_cleanup is covered, though, and 
> that's where I would expect problem to come.

Sounds like a plausible reason.

> It would be interesting to see a backtrace from remote-curl when we get the 
> SIGPIPE. Doing so would be slightly tricky; instrumenting with the patch 
> below may be enough.
>
> Another thought is that the curl fix seems to only kick in when built with 
> openssl support.  I'm not sure I understand how ubuntu's packaging of curl 
> uses gnutls versus openssl for the shared library. That may be related.

I'm only aware of a SIGPIPE problem with openssl that can make it write to the 
socket in some situations when the remote end is no longer there - something 
we can't prevent it from doing.

I *believe* the problem doesn't exist in the similar way when built to use 
gnutls, but I may of course be wrong.

-- 

  / daniel.haxx.se

Re: error: git-remote-https died of signal 13

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1119 bytes --]

On Sun, Nov 24, 2013 at 10:54:39AM -0500, Jeff King wrote:
> Another thought is that the curl fix seems to only kick in when built
> with openssl support.  I'm not sure I understand how ubuntu's packaging
> of curl uses gnutls versus openssl for the shared library. That may be
> related.

Debian (and presumably Ubuntu) build the curl source three times: once
each for OpenSSL, GnuTLS, and NSS.  Each shared library is named
differently (libcurl-openssl.so.3, etc.) and in its own package
(libcurl3-openssl).  The corresponding -dev package for each version
sets up the symlinks and install headers to point to the proper version,
so you always compile and link as you expect.

The reason for this is that Debian cannot distribute GPLv2-only programs
(like git) linked against OpenSSL, so GnuTLS becomes necessary.  On
Debian and Ubuntu, git is by default linked against libcurl3-gnutls.

-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]