* [ruby-core:99381] [Ruby master Bug#15773] Net::HTTP doesn't try next IP address in case of timeout
[not found] <redmine.issue-15773.20190416231201.10577@ruby-lang.org>
@ 2020-07-28 23:59 ` oded
2020-07-31 18:46 ` [ruby-core:99423] " ben
` (2 subsequent siblings)
3 siblings, 0 replies; 4+ messages in thread
From: oded @ 2020-07-28 23:59 UTC (permalink / raw)
To: ruby-core
Issue #15773 has been updated by guss77 (Oded Arbel).
I've encountered this issue as well. My workaround - in case someone else has the same problem:
```ruby
require 'net/http'
require 'resolv'
# ...
url = URI.parse(url) unless url.is_a?(URI)
addrs = Resolv::DNS.new.getaddresses(url.host).sort { |a,b| b.class.to_s <=> a.class.to_s }.collect { |ip| ip.to_s }
begin
response = Net::HTTP.start(url.host, url.port, :use_ssl => url.scheme == 'https', open_timeout: 3, ipaddr: addrs.shift) do |http|
http.request(Net::HTTP::Get.new(url.request_uri))
end
rescue Timeout::Error => e
retry unless addrs.empty?
raise e
end
```
----------------------------------------
Bug #15773: Net::HTTP doesn't try next IP address in case of timeout
https://bugs.ruby-lang.org/issues/15773#change-86785
* Author: nicolasnoble (Nicolas Noble)
* Status: Rejected
* Priority: Normal
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
This example requires to have a working IPv6 address. Since IPv6 is used in first priority, I am using it to demonstrate the problem, but it exists with plain IPv4, which will be more round-robin-style, so less deterministic to show a reproduction case. I have made two URLs that have both IPv4 and IPv6 addresses: http://ipv6.grumpycoder.net/ and http://bad-ipv6.grumpycoder.net/ - both URLs should work in an IPv6-enabled web browser, as well as curl or wget for instance. The difference is that the bad-ipv6 subdomain doesn't have an IPv6 that will actually connect. Therefore, browsers, curl and wget will fallback to using the IPv4 when the initial IPv6 connection attempt failed:
```
$ wget -T1 http://bad-ipv6.grumpycoder.net -O - # demonstrating using wget because its output is more clear than curl's verbose
--2019-04-16 15:56:52-- http://bad-ipv6.grumpycoder.net/
Resolving bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)... 2001:bc8:3690:200::2, 62.210.214.144
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|2001:bc8:3690:200::2|:80... failed: Connection timed out.
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|62.210.214.144|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45 [text/html]
Saving to: ‘STDOUT’
- 0%[ ] 0 --.-KB/s <html><body><h1>It works!</h1></body></html>
- 100%[=====================================================================>] 45 --.-KB/s in 0s
2019-04-16 15:56:53 (6.55 MB/s) - written to stdout [45/45]
```
However, in Ruby, using Net::HTTP (or OpenURI), that's not going to be the case:
```
$ ruby bad-ipv6.rb
http://ipv6.grumpycoder.net
<html><body><h1>It works!</h1></body></html>
http://bad-ipv6.grumpycoder.net
Net::OpenTimeout: execution expired
```
Contents of my test file:
```ruby
require 'open-uri'
['ipv6', 'bad-ipv6'].each do |s|
url = 'http://%s.grumpycoder.net' %[s]
puts url
begin
puts open(url).read
rescue StandardError => e
puts "#{e.class}: #{e.message}"
next
end
end
```
The proper behavior should be to retry the next IP address and exhaust all of the IPs in the DNS resolution results before throwing out an error.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread* [ruby-core:99423] [Ruby master Bug#15773] Net::HTTP doesn't try next IP address in case of timeout
[not found] <redmine.issue-15773.20190416231201.10577@ruby-lang.org>
2020-07-28 23:59 ` [ruby-core:99381] [Ruby master Bug#15773] Net::HTTP doesn't try next IP address in case of timeout oded
@ 2020-07-31 18:46 ` ben
2020-07-31 22:51 ` [ruby-core:99424] " matthew
2020-08-03 16:23 ` [ruby-core:99456] " ben
3 siblings, 0 replies; 4+ messages in thread
From: ben @ 2020-07-31 18:46 UTC (permalink / raw)
To: ruby-core
Issue #15773 has been updated by benlangfeld (Ben Langfeld).
naruse (Yui NARUSE) wrote in #note-1:
> In general resolving DNS is done by libc (getaddrinfo) and Ruby just uses their result.
> Therefore it is not a bug and won't be a small feature.
RFC3484 (https://www.ietf.org/rfc/rfc3484.txt) very clearly states:
```
Well-behaved applications SHOULD iterate through the list of
addresses returned from getaddrinfo() until they find a working
address.
```
That Ruby does not do this should be considered a bug, and I would ask that you please re-open this ticket on that basis. Ruby is in contravention of the RFC and the documentation of `getaddrinfo(3)`. This is not a missing feature; Ruby is broken. Note that I am not asking that anyone fix this; I will prepare a fix. All I ask is that this ticket be acknowledged as valid and not rejected based on misunderstanding it.
----------------------------------------
Bug #15773: Net::HTTP doesn't try next IP address in case of timeout
https://bugs.ruby-lang.org/issues/15773#change-86876
* Author: nicolasnoble (Nicolas Noble)
* Status: Rejected
* Priority: Normal
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
This example requires to have a working IPv6 address. Since IPv6 is used in first priority, I am using it to demonstrate the problem, but it exists with plain IPv4, which will be more round-robin-style, so less deterministic to show a reproduction case. I have made two URLs that have both IPv4 and IPv6 addresses: http://ipv6.grumpycoder.net/ and http://bad-ipv6.grumpycoder.net/ - both URLs should work in an IPv6-enabled web browser, as well as curl or wget for instance. The difference is that the bad-ipv6 subdomain doesn't have an IPv6 that will actually connect. Therefore, browsers, curl and wget will fallback to using the IPv4 when the initial IPv6 connection attempt failed:
```
$ wget -T1 http://bad-ipv6.grumpycoder.net -O - # demonstrating using wget because its output is more clear than curl's verbose
--2019-04-16 15:56:52-- http://bad-ipv6.grumpycoder.net/
Resolving bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)... 2001:bc8:3690:200::2, 62.210.214.144
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|2001:bc8:3690:200::2|:80... failed: Connection timed out.
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|62.210.214.144|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45 [text/html]
Saving to: ‘STDOUT’
- 0%[ ] 0 --.-KB/s <html><body><h1>It works!</h1></body></html>
- 100%[=====================================================================>] 45 --.-KB/s in 0s
2019-04-16 15:56:53 (6.55 MB/s) - written to stdout [45/45]
```
However, in Ruby, using Net::HTTP (or OpenURI), that's not going to be the case:
```
$ ruby bad-ipv6.rb
http://ipv6.grumpycoder.net
<html><body><h1>It works!</h1></body></html>
http://bad-ipv6.grumpycoder.net
Net::OpenTimeout: execution expired
```
Contents of my test file:
```ruby
require 'open-uri'
['ipv6', 'bad-ipv6'].each do |s|
url = 'http://%s.grumpycoder.net' %[s]
puts url
begin
puts open(url).read
rescue StandardError => e
puts "#{e.class}: #{e.message}"
next
end
end
```
The proper behavior should be to retry the next IP address and exhaust all of the IPs in the DNS resolution results before throwing out an error.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread* [ruby-core:99424] [Ruby master Bug#15773] Net::HTTP doesn't try next IP address in case of timeout
[not found] <redmine.issue-15773.20190416231201.10577@ruby-lang.org>
2020-07-28 23:59 ` [ruby-core:99381] [Ruby master Bug#15773] Net::HTTP doesn't try next IP address in case of timeout oded
2020-07-31 18:46 ` [ruby-core:99423] " ben
@ 2020-07-31 22:51 ` matthew
2020-08-03 16:23 ` [ruby-core:99456] " ben
3 siblings, 0 replies; 4+ messages in thread
From: matthew @ 2020-07-31 22:51 UTC (permalink / raw)
To: ruby-core
Issue #15773 has been updated by phluid61 (Matthew Kerwin).
benlangfeld (Ben Langfeld) wrote in #note-4:
>
> RFC3484 (https://www.ietf.org/rfc/rfc3484.txt) very clearly states:
>
Minor point, I don't think it affects much, but RFC 3484 is obsolete and is replaced by RFC 6724 https://tools.ietf.org/html/rfc6724
Cheers
----------------------------------------
Bug #15773: Net::HTTP doesn't try next IP address in case of timeout
https://bugs.ruby-lang.org/issues/15773#change-86877
* Author: nicolasnoble (Nicolas Noble)
* Status: Rejected
* Priority: Normal
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
This example requires to have a working IPv6 address. Since IPv6 is used in first priority, I am using it to demonstrate the problem, but it exists with plain IPv4, which will be more round-robin-style, so less deterministic to show a reproduction case. I have made two URLs that have both IPv4 and IPv6 addresses: http://ipv6.grumpycoder.net/ and http://bad-ipv6.grumpycoder.net/ - both URLs should work in an IPv6-enabled web browser, as well as curl or wget for instance. The difference is that the bad-ipv6 subdomain doesn't have an IPv6 that will actually connect. Therefore, browsers, curl and wget will fallback to using the IPv4 when the initial IPv6 connection attempt failed:
```
$ wget -T1 http://bad-ipv6.grumpycoder.net -O - # demonstrating using wget because its output is more clear than curl's verbose
--2019-04-16 15:56:52-- http://bad-ipv6.grumpycoder.net/
Resolving bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)... 2001:bc8:3690:200::2, 62.210.214.144
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|2001:bc8:3690:200::2|:80... failed: Connection timed out.
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|62.210.214.144|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45 [text/html]
Saving to: ‘STDOUT’
- 0%[ ] 0 --.-KB/s <html><body><h1>It works!</h1></body></html>
- 100%[=====================================================================>] 45 --.-KB/s in 0s
2019-04-16 15:56:53 (6.55 MB/s) - written to stdout [45/45]
```
However, in Ruby, using Net::HTTP (or OpenURI), that's not going to be the case:
```
$ ruby bad-ipv6.rb
http://ipv6.grumpycoder.net
<html><body><h1>It works!</h1></body></html>
http://bad-ipv6.grumpycoder.net
Net::OpenTimeout: execution expired
```
Contents of my test file:
```ruby
require 'open-uri'
['ipv6', 'bad-ipv6'].each do |s|
url = 'http://%s.grumpycoder.net' %[s]
puts url
begin
puts open(url).read
rescue StandardError => e
puts "#{e.class}: #{e.message}"
next
end
end
```
The proper behavior should be to retry the next IP address and exhaust all of the IPs in the DNS resolution results before throwing out an error.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread* [ruby-core:99456] [Ruby master Bug#15773] Net::HTTP doesn't try next IP address in case of timeout
[not found] <redmine.issue-15773.20190416231201.10577@ruby-lang.org>
` (2 preceding siblings ...)
2020-07-31 22:51 ` [ruby-core:99424] " matthew
@ 2020-08-03 16:23 ` ben
3 siblings, 0 replies; 4+ messages in thread
From: ben @ 2020-08-03 16:23 UTC (permalink / raw)
To: ruby-core
Issue #15773 has been updated by benlangfeld (Ben Langfeld).
phluid61 (Matthew Kerwin) wrote in #note-5:
> benlangfeld (Ben Langfeld) wrote in #note-4:
> >
> > RFC3484 (https://www.ietf.org/rfc/rfc3484.txt) very clearly states:
> >
>
> Minor point, I don't think it affects much, but RFC 3484 is obsolete and is replaced by RFC 6724 https://tools.ietf.org/html/rfc6724
>
> Cheers
Thank you for pointing that out. RFC 6724 is even more explicit:
```
Well-behaved applications SHOULD NOT simply use the first address
returned from an API such as getaddrinfo() and then give up if it
fails. For many applications, it is appropriate to iterate through
the list of addresses returned from getaddrinfo() until a working
address is found. For other applications, it might be appropriate to
try multiple addresses in parallel (e.g., with some small delay in
between) and use the first one to succeed.
```
----------------------------------------
Bug #15773: Net::HTTP doesn't try next IP address in case of timeout
https://bugs.ruby-lang.org/issues/15773#change-86913
* Author: nicolasnoble (Nicolas Noble)
* Status: Rejected
* Priority: Normal
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
This example requires to have a working IPv6 address. Since IPv6 is used in first priority, I am using it to demonstrate the problem, but it exists with plain IPv4, which will be more round-robin-style, so less deterministic to show a reproduction case. I have made two URLs that have both IPv4 and IPv6 addresses: http://ipv6.grumpycoder.net/ and http://bad-ipv6.grumpycoder.net/ - both URLs should work in an IPv6-enabled web browser, as well as curl or wget for instance. The difference is that the bad-ipv6 subdomain doesn't have an IPv6 that will actually connect. Therefore, browsers, curl and wget will fallback to using the IPv4 when the initial IPv6 connection attempt failed:
```
$ wget -T1 http://bad-ipv6.grumpycoder.net -O - # demonstrating using wget because its output is more clear than curl's verbose
--2019-04-16 15:56:52-- http://bad-ipv6.grumpycoder.net/
Resolving bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)... 2001:bc8:3690:200::2, 62.210.214.144
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|2001:bc8:3690:200::2|:80... failed: Connection timed out.
Connecting to bad-ipv6.grumpycoder.net (bad-ipv6.grumpycoder.net)|62.210.214.144|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45 [text/html]
Saving to: ‘STDOUT’
- 0%[ ] 0 --.-KB/s <html><body><h1>It works!</h1></body></html>
- 100%[=====================================================================>] 45 --.-KB/s in 0s
2019-04-16 15:56:53 (6.55 MB/s) - written to stdout [45/45]
```
However, in Ruby, using Net::HTTP (or OpenURI), that's not going to be the case:
```
$ ruby bad-ipv6.rb
http://ipv6.grumpycoder.net
<html><body><h1>It works!</h1></body></html>
http://bad-ipv6.grumpycoder.net
Net::OpenTimeout: execution expired
```
Contents of my test file:
```ruby
require 'open-uri'
['ipv6', 'bad-ipv6'].each do |s|
url = 'http://%s.grumpycoder.net' %[s]
puts url
begin
puts open(url).read
rescue StandardError => e
puts "#{e.class}: #{e.message}"
next
end
end
```
The proper behavior should be to retry the next IP address and exhaust all of the IPs in the DNS resolution results before throwing out an error.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread