[ruby-core:63032] [ruby-trunk - Bug #9925] [Open] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[ruby-core:63032] [ruby-trunk - Bug #9925] [Open] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[aaron]

Issue #9925 has been reported by Aaron Stone.

----------------------------------------
Bug #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume
https://bugs.ruby-lang.org/issues/9925

* Author: Aaron Stone
* Status: Open
* Priority: Normal
* Assignee: 
* Category: 
* Target version: 
* ruby -v: 2.0.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN
----------------------------------------
In ext/socket/raddrinfo.c, the function rsock_addrinfo() always uses AF_UNSPEC for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. rsock_addrinfo() is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will _always_ try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c
378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }

For example, an environment variable named something like RUBY_GAI could be set to "INET" or "INET6" to switch the hints.ai_family away from AF_UNSPEC.



-- 
https://bugs.ruby-lang.org/

[ruby-core:63033] [ruby-trunk - Bug #9925] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[aaron]

Issue #9925 has been updated by Aaron Stone.


Suggested code change: https://github.com/ruby/ruby/pull/636

----------------------------------------
Bug #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume
https://bugs.ruby-lang.org/issues/9925#change-47115

* Author: Aaron Stone
* Status: Open
* Priority: Normal
* Assignee: 
* Category: 
* Target version: 
* ruby -v: 2.0.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN
----------------------------------------
In ext/socket/raddrinfo.c, the function rsock_addrinfo() always uses AF_UNSPEC for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. rsock_addrinfo() is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will _always_ try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c
378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }

For example, an environment variable named something like RUBY_GAI could be set to "INET" or "INET6" to switch the hints.ai_family away from AF_UNSPEC.



-- 
https://bugs.ruby-lang.org/

[ruby-core:63034] [ruby-trunk - Feature #9925] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[nobu]

Issue #9925 has been updated by Nobuyoshi Nakada.

Tracker changed from Bug to Feature
Description updated

It doesn't feel a good idea to change a behavior in a library by an environment variable implicitly.

----------------------------------------
Feature #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume
https://bugs.ruby-lang.org/issues/9925#change-47116

* Author: Aaron Stone
* Status: Open
* Priority: Normal
* Assignee: 
* Category: 
* Target version: 
----------------------------------------
In ext/socket/raddrinfo.c, the function `rsock_addrinfo()` always uses `AF_UNSPEC` for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. `rsock_addrinfo()` is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will _always_ try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c

~~~c
378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }
~~~

For example, an environment variable named something like `RUBY_GAI` could be set to "INET" or "INET6" to switch the `hints.ai_family` away from `AF_UNSPEC`.



-- 
https://bugs.ruby-lang.org/

[ruby-core:63036] [ruby-trunk - Feature #9925] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[aaron]

Issue #9925 has been updated by Aaron Stone.


Similar configuration is available in the "/etc/gai.conf" system (it does not apply in this use case, however). It is exactly the global nature of the setting that helps me the most. In many cases, Ruby is hardcoded to a particular lookup style and that is causing me significant system load for the nonexistent DNS queries.

There is also a compile-time Ruby flag for LOOKUP_ORDER_HACK_INET and LOOKUP_ORDER_HACK_INET6. Perhaps those could become an environment variable instead of compile-time flags?

Another approach could be a module variable in Socket that I could toggle globally or per-instance. Something like this:

Socket::DEFAULT_DNS_LOOKUP = :UNSPEC

I could either globally set Socket::DEFAULT_DNS_LOOKUP = :INET or on a per-class basis, Socket.dns_lookup = :INET.

----------------------------------------
Feature #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume
https://bugs.ruby-lang.org/issues/9925#change-47118

* Author: Aaron Stone
* Status: Open
* Priority: Normal
* Assignee: 
* Category: 
* Target version: 
----------------------------------------
In ext/socket/raddrinfo.c, the function `rsock_addrinfo()` always uses `AF_UNSPEC` for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. `rsock_addrinfo()` is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will _always_ try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c

~~~c
378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }
~~~

For example, an environment variable named something like `RUBY_GAI` could be set to "INET" or "INET6" to switch the `hints.ai_family` away from `AF_UNSPEC`.



-- 
https://bugs.ruby-lang.org/

[ruby-core:66584] [ruby-trunk - Feature #9925] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[aaron]

Issue #9925 has been updated by Aaron Stone.


Ping. I was hoping someone from the Ruby core would provide feedback and direction regarding my suggestions:

> There is also a compile-time Ruby flag for LOOKUP_ORDER_HACK_INET and LOOKUP_ORDER_HACK_INET6. Perhaps those could become an environment variable instead of compile-time flags?

and

> Another approach could be a module variable in Socket that I could toggle globally or per-instance. Something like this:
> 
> Socket::DEFAULT_DNS_LOOKUP = :UNSPEC
> 
> I could either globally set Socket::DEFAULT_DNS_LOOKUP = :INET or on a per-instance basis, e.g.
> 
> my_socket = TCPSocket.new()
> my_socket.dns_lookup = :INET

----------------------------------------
Feature #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume
https://bugs.ruby-lang.org/issues/9925#change-50199

* Author: Aaron Stone
* Status: Open
* Priority: Normal
* Assignee: 
* Category: 
* Target version: 
----------------------------------------
In ext/socket/raddrinfo.c, the function `rsock_addrinfo()` always uses `AF_UNSPEC` for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. `rsock_addrinfo()` is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will _always_ try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c

~~~c
378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }
~~~

For example, an environment variable named something like `RUBY_GAI` could be set to "INET" or "INET6" to switch the `hints.ai_family` away from `AF_UNSPEC`.



-- 
https://bugs.ruby-lang.org/

[ruby-core:71064] [Ruby trunk - Feature #9925] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[craig65535]

Issue #9925 has been updated by Craig Davison.


I have a similar issue.

While running strace, I noticed that a ruby script sending a UDP datagram to localhost would call sendto twice. The first sendto used an IPv6 address, and would fail, and the second sendto used an IPv4 address and would succeed.

The issue seems to be that sockets are created with a domain of PF_INET by default, but hostname lookups (getaddrinfo) are done with no address family hint. I have attempted to fix this in https://github.com/ruby/ruby/pull/1052 by reading the socket's address family in rsock_addrinfo(), and passing it as a hint to getaddrinfo().

Here is a short script that illustrates the problem.

~~~
require 'socket'
socket = UDPSocket.new
socket.send("123", 0, "localhost", 5556)
~~~

Here is the strace output, before my patch:

~~~
$ strace ~/ruby-master/bin/ruby ~/udp.rb 2>&1 | grep INET | egrep '(send|socket)'
socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 7
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 8
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 8
sendto(7, "123", 3, 0, {sa_family=AF_INET6, sin6_port=htons(5556), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EAFNOSUPPORT (Address family not supported by protocol)
sendto(7, "123", 3, 0, {sa_family=AF_INET, sin_port=htons(5556), sin_addr=inet_addr("127.0.0.1")}, 16) = 3
~~~

And after my patch:

~~~
$ strace ~/ruby-fix-getaddrinfo/bin/ruby ~/udp.rb 2>&1 | grep INET | egrep '(send|socket)'
socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 7
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 8
sendto(7, "123", 3, 0, {sa_family=AF_INET, sin_port=htons(5556), sin_addr=inet_addr("127.0.0.1")}, 1
~~~


----------------------------------------
Feature #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume
https://bugs.ruby-lang.org/issues/9925#change-54432

* Author: Aaron Stone
* Status: Open
* Priority: Normal
* Assignee: 
----------------------------------------
In ext/socket/raddrinfo.c, the function `rsock_addrinfo()` always uses `AF_UNSPEC` for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. `rsock_addrinfo()` is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will _always_ try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c

~~~c
378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }
~~~

For example, an environment variable named something like `RUBY_GAI` could be set to "INET" or "INET6" to switch the `hints.ai_family` away from `AF_UNSPEC`.



-- 
https://bugs.ruby-lang.org/

[ruby-core:94401] [Ruby master Feature#9925] rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume

[aaron]

Issue #9925 has been updated by aaron@serendipity.cx (Aaron Stone).


Thank you Craig! Since your patch was accepted, this ticket is now resolved for me. I'll also mark my Ruby Github PR as resolved by your PR.

----------------------------------------
Feature #9925: rsock_addrinfo uses DNS family AF_UNSPEC for lookup causing high IPv6 AAAA volume
https://bugs.ruby-lang.org/issues/9925#change-80822

* Author: aaron@serendipity.cx (Aaron Stone)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
In ext/socket/raddrinfo.c, the function `rsock_addrinfo()` always uses `AF_UNSPEC` for DNS queries. This is causing me a very high volume of IPv6 DNS lookups. `rsock_addrinfo()` is used by TCPSocket (and all other Socket base classes, e.g. Socket and UDPSocket), and TCPSocket is used by Net::HTTP.

Remember that DNS does not do negative caching - if a hostname does not have a AAAA record, then DNS will _always_ try to look up that record again!

I propose that the following code should have some way to force IPv4 or IPv6 lookups:

http://rxr.whitequark.org/mri/source/ext/socket/raddrinfo.c

~~~c
378 struct addrinfo*
379 rsock_addrinfo(VALUE host, VALUE port, int socktype, int flags)
380 {
381     struct addrinfo hints;
382 
383     MEMZERO(&hints, struct addrinfo, 1);
384     hints.ai_family = AF_UNSPEC;
385     hints.ai_socktype = socktype;
386     hints.ai_flags = flags;
387     return rsock_getaddrinfo(host, port, &hints, 1);
388 }
~~~

For example, an environment variable named something like `RUBY_GAI` could be set to "INET" or "INET6" to switch the `hints.ai_family` away from `AF_UNSPEC`.



-- 
https://bugs.ruby-lang.org/