SNI (SSL virtual hosts)

SNI (SSL virtual hosts)

[Janusz Harkot]

I was trying to to a push some repo over https and after few unsuccessful tries I've managed to find a problem - multiple virtual SSL servers on one IP address…

Strange was, that initial communication was OK (http GET), but when there was http POST - git reported error (incorrect certificate).
The only workaround was to disable certificate verification.

My question is: does git support SNI on the https? If so - are there (undocumented) options to make it work?



Thanks!
Janusz  

Re: SNI (SSL virtual hosts)

[Daniel Stenberg]

On Tue, 4 Jun 2013, Janusz Harkot wrote:

> Strange was, that initial communication was OK (http GET), but when there 
> was http POST - git reported error (incorrect certificate). The only 
> workaround was to disable certificate verification.
>
> My question is: does git support SNI on the https? If so - are there 
> (undocumented) options to make it work?

It does. git uses libcurl for the HTTPS parts and it has support SNI for a 
long time, assuming you built libcurl with a TLS library that handles it.

Which libcurl version and SSL backend is this? (curl -V usually tells)

If you made it working by disabling certificate verification then it sounds as 
if SNI might still have worked and the problem was rahter something else, as 
without SNI you can't do name-based virtual hosting over HTTPS - but perhaps 
you wanted to communicate with the "default" server on that IP?

-- 

  / daniel.haxx.se

Re: SNI (SSL virtual hosts)

[Janusz Harkot]

> It does. git uses libcurl for the HTTPS parts and it has support SNI for a 
> long time, assuming you built libcurl with a TLS library that handles it.
> 
> Which libcurl version and SSL backend is this? (curl -V usually tells)
$ curl -V
curl 7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp 
Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz 

$ otool -L /usr/local/bin/git
/usr/local/bin/git:
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)



> If you made it working by disabling certificate verification then it sounds as 
> if SNI might still have worked and the problem was rahter something else, as 
> without SNI you can't do name-based virtual hosting over HTTPS - but perhaps 
> you wanted to communicate with the "default" server on that IP?

here is a log (with GIT_CURL_VERBOSE=1)

https://gist.github.com/anonymous/8f6533a755ae5c710c75 

Initial connection is correct (line 10 - shows that it reads correct certificate),
 but then subsequent call to the server (line 68) shows that the defat server certificate is used.

It looks like the second call was without hostname (?).

Thanks!
Janusz

Re: SNI (SSL virtual hosts)

[Daniel Stenberg]

On Tue, 4 Jun 2013, Janusz Harkot wrote:

>> Which libcurl version and SSL backend is this? (curl -V usually tells)
> $ curl -V
> curl 7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r 
> zlib/1.2.5

>From what I can tell, that OpenSSL version supports SNI fine and libcurl has 
supported it since 7.18.1.

> here is a log (with GIT_CURL_VERBOSE=1)
>
> https://gist.github.com/anonymous/8f6533a755ae5c710c75
>
> Initial connection is correct (line 10 - shows that it reads correct 
> certificate), but then subsequent call to the server (line 68) shows that 
> the defat server certificate is used.
>
> It looks like the second call was without hostname (?).

What makes you suggest that's what's happening? Sure, if it would've sent no 
or the wrong host name it would probably have that effect.

Any chance you can snoop on the network and the SSL handshake to see who's to 
blame? I can't but to think that is is a very common use case!

-- 

  / daniel.haxx.se

Re: SNI (SSL virtual hosts)

[Janusz Harkot]

> What makes you suggest that's what's happening? Sure, if it would've sent no  
> or the wrong host name it would probably have that effect.

line:

[36] * Re-using existing connection! (#0) with host (nil)  
> Any chance you can snoop on the network and the SSL handshake to see who's to  
> blame? I can't but to think that is is a very common use case!


I was trying to verify this via command line curl, and GET/POST is working fine.

There is one thing that make me suspicious - this is that message about SSL session expiration:
[64] * SSL re-using session ID
[65] * SSL connection using RC4-SHA
[66] * old SSL session ID is stale, removing



So, I've spent last few hours playing with different settings/builds etc.  
I was using sources of curl (7.30.0) and git (1.8.2.3)

curl & git bult with default os-x's openssl (0.9.8) - failed
curl & git bult with '--with-darwin-ssl' + default openssl for git - failed

curl & git both bult with newest openssl (1.0.1e):
error: SSL certificate problem: self signed certificate in certificate chain while accessing https://....

so it looks promising, I've set GIT_SSL_CAPATH (because bundle config is not supported in git - this is a good feature request)

and.. t looks like it is working…
first and second attempt was to correct SNI host!

So, the question is still why it is not working with openssl 0.9.8r - this version supports SNI by default.
This looks like an error in openssl (maybe: Only allow one SGC handshake restart for SSL/TLS.)

Now is the question, shall this be handled by curl or left alone?
(handling older version of openssl, and force new ssl session?)


kind regards,
Janusz


p.s.
I've tested cur built with polarssl - works and gnutls - works too...

Re: SNI (SSL virtual hosts)

[Daniel Stenberg]

On Tue, 4 Jun 2013, Janusz Harkot wrote:

>> What makes you suggest that's what's happening? Sure, if it would've sent no
>> or the wrong host name it would probably have that effect.
>
> line:
>
> [36] * Re-using existing connection! (#0) with host (nil)

Ah that. Yes, that's a stupid line to show (that bug has been fixed since). 
But if you look further down your log you see that the connection which is 
re-used according to that log line gets closed anyway.

> it looks like it is working

Awesome!

> So, the question is still why it is not working with openssl 0.9.8r - this 
> version supports SNI by default. This looks like an error in openssl (maybe: 
> Only allow one SGC handshake restart for SSL/TLS.)

Right. As you can see in the libcurl code it activates SNI for OpenSSL the 
exact same way independently of what version that's used.

> Now is the question, shall this be handled by curl or left alone? (handling 
> older version of openssl, and force new ssl session?)

I'm not even completely convinced this is "just" an old-OpenSSL-problem. If 
that version you're using is the one Apple has provided, there's the risk that 
the problem is rather caused by their changes!

I'm reluctant to globally switch off session-id caching for OpenSSL 0.9.8 
users since that feature has been used for over 8 years in the code and you're 
the first to have a problem with it! =-/

-- 

  / daniel.haxx.se

Re: SNI (SSL virtual hosts)

[Janusz Harkot]

valid point, but from what you can find on the web, the only solution provided everywhere was to
disable certificate checking… so maybe that's not me, but this is first time someone spent
some time to check whats going on :)

at least there will be something, maybe this will help someone…

thanks Daniel!


best!
Janusz








On Tuesday, 4 June 2013 at 23:18, Daniel Stenberg wrote:

> On Tue, 4 Jun 2013, Janusz Harkot wrote:
>  
> > > What makes you suggest that's what's happening? Sure, if it would've sent no
> > > or the wrong host name it would probably have that effect.
> >  
> >  
> >  
> > line:
> >  
> > [36] * Re-using existing connection! (#0) with host (nil)
>  
> Ah that. Yes, that's a stupid line to show (that bug has been fixed since).  
> But if you look further down your log you see that the connection which is  
> re-used according to that log line gets closed anyway.
>  
> > it looks like it is working
>  
> Awesome!
>  
> > So, the question is still why it is not working with openssl 0.9.8r - this  
> > version supports SNI by default. This looks like an error in openssl (maybe:  
> > Only allow one SGC handshake restart for SSL/TLS.)
>  
>  
>  
> Right. As you can see in the libcurl code it activates SNI for OpenSSL the  
> exact same way independently of what version that's used.
>  
> > Now is the question, shall this be handled by curl or left alone? (handling  
> > older version of openssl, and force new ssl session?)
>  
>  
>  
> I'm not even completely convinced this is "just" an old-OpenSSL-problem. If  
> that version you're using is the one Apple has provided, there's the risk that  
> the problem is rather caused by their changes!
>  
> I'm reluctant to globally switch off session-id caching for OpenSSL 0.9.8  
> users since that feature has been used for over 8 years in the code and you're  
> the first to have a problem with it! =-/
>  
> --  
>  
> / daniel.haxx.se (http://daniel.haxx.se)
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org (mailto:majordomo@vger.kernel.org)
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: SNI (SSL virtual hosts)

[Daniel Stenberg]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 816 bytes --]

On Tue, 4 Jun 2013, Janusz Harkot wrote:

> valid point, but from what you can find on the web, the only solution 
> provided everywhere was to disable certificate checking… so maybe that's not 
> me, but this is first time someone spent some time to check whats going on 
> :)

I don't disagree with that. You may be right.

But I am the maintainer of libcurl and I have *never* gotten a report about 
this before, and I rather base my actions and assumptions on true reports from 
actual developers with whom I can discuss and delve into details with (like 
you and me right now). Basing decisions on vague statements posted elsewhere 
by unknown people is for sure a road into sadness.

Anyway, now I'm off topic. I'm glad you could fix the problem. Thanks for 
flying git + libcurl! =)

-- 

  / daniel.haxx.se