* [ruby-core:96988] [Ruby master Bug#16559] Net::HTTP#request does not properly close TCP socket if #started? is false
[not found] <redmine.issue-16559.20200124035042@ruby-lang.org>
@ 2020-01-24 3:50 ` me
2020-01-24 15:20 ` [ruby-core:96989] " me
1 sibling, 0 replies; 2+ messages in thread
From: me @ 2020-01-24 3:50 UTC (permalink / raw
To: ruby-core
Issue #16559 has been reported by f3ndot (Justin Bull).
----------------------------------------
Bug #16559: Net::HTTP#request does not properly close TCP socket if #started? is false
https://bugs.ruby-lang.org/issues/16559
* Author: f3ndot (Justin Bull)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: 2.7.0-preview
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
Hello,
There appears to be a bug in Net::HTTP#request (and thus #get, #post, etc.) on an instance that isn't explicitly started by the programmer (by invoking #start first, or by executing #request inside a block passed to #start).
Inspecting the source code, it reveals #request will recursively call itself inside a #start block if #started? is false. This is great and as I'd expect.
However in production and in a test setup I'm observing TCP socket connections on the server-side in the "TIME_WAIT" state, indicating the socket was never properly closed. Conversely, explicitly running #request inside a #start block yields no such behaviour.
Consider the following setup, assuming you have docker:
```
docker run --rm -it -p 8080:80/tcp --user root ubuntu
apt-get update && apt-get install net-tools watch nginx
service nginx start
watch 'netstat -tunapl'
```
Running this on your host machine:
``` ruby
net = Net::HTTP.new('localhost', 8080)
50.times { net.get('/') } # is bad
```
Will spawn 50 TCP connections on the server, and will all have on TIME_WAIT for 60 seconds (different *nix OSes have different times):
```
Every 2.0s: netstat -tunapl
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 791/nginx: master p
tcp 0 0 172.17.0.2:80 172.17.0.1:60772 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60732 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60812 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60778 TIME_WAIT -
...
```
However running any of these incantations have no such result:
``` ruby
50.times { Net::HTTP.get(URI('http://localhost:8080/')) } # is OK
```
``` ruby
net = Net::HTTP.new('localhost', 8080)
net.start
50.times { net.get('/') } # is OK
net.finish
```
``` ruby
net = Net::HTTP.new('localhost', 8080)
50.times { net.start { net.get('/') } } # is OK
```
These TIME_WAIT connections matter because a server receiving many HTTP requests from clients using Net::HTTP in this fashion (as Faraday does[1]) the server will begin to oversaturate and timeout past a particular scale.
I've tested and reproduced this in 2.7 and 2.6.
[1]: https://github.com/lostisland/faraday/pull/1117
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 2+ messages in thread
* [ruby-core:96989] [Ruby master Bug#16559] Net::HTTP#request does not properly close TCP socket if #started? is false
[not found] <redmine.issue-16559.20200124035042@ruby-lang.org>
2020-01-24 3:50 ` [ruby-core:96988] [Ruby master Bug#16559] Net::HTTP#request does not properly close TCP socket if #started? is false me
@ 2020-01-24 15:20 ` me
1 sibling, 0 replies; 2+ messages in thread
From: me @ 2020-01-24 15:20 UTC (permalink / raw
To: ruby-core
Issue #16559 has been updated by f3ndot (Justin Bull).
ruby -v changed from 2.7.0-preview to 2.8.0-dev, 2.7.0, 2.6.5
File dont-default-connection-close.patch added
I have chased this down to an opportunistic setting of 'Connection: close' header if-and-only-if #request is called when #started? is false.
Attached is a patchfile where I remove this header setting, with the rationale laid out in the description. Hopefully you agree :-)
----------------------------------------
Bug #16559: Net::HTTP#request does not properly close TCP socket if #started? is false
https://bugs.ruby-lang.org/issues/16559#change-84047
* Author: f3ndot (Justin Bull)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: 2.8.0-dev, 2.7.0, 2.6.5
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
Hello,
There appears to be a bug in Net::HTTP#request (and thus #get, #post, etc.) on an instance that isn't explicitly started by the programmer (by invoking #start first, or by executing #request inside a block passed to #start).
Inspecting the source code, it reveals #request will recursively call itself inside a #start block if #started? is false. This is great and as I'd expect.
However in production and in a test setup I'm observing TCP socket connections on the server-side in the "TIME_WAIT" state, indicating the socket was never properly closed. Conversely, explicitly running #request inside a #start block yields no such behaviour.
Consider the following setup, assuming you have docker:
```
docker run --rm -it -p 8080:80/tcp --user root ubuntu
apt-get update && apt-get install net-tools watch nginx
service nginx start
watch 'netstat -tunapl'
```
Running this on your host machine:
``` ruby
net = Net::HTTP.new('localhost', 8080)
50.times { net.get('/') } # is bad
```
Will spawn 50 TCP connections on the server, and will all have on TIME_WAIT for 60 seconds (different *nix OSes have different times):
```
Every 2.0s: netstat -tunapl
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 791/nginx: master p
tcp 0 0 172.17.0.2:80 172.17.0.1:60772 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60732 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60812 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60778 TIME_WAIT -
...
```
However running any of these incantations have no such result:
``` ruby
50.times { Net::HTTP.get(URI('http://localhost:8080/')) } # is OK
```
``` ruby
net = Net::HTTP.new('localhost', 8080)
net.start
50.times { net.get('/') } # is OK
net.finish
```
``` ruby
net = Net::HTTP.new('localhost', 8080)
50.times { net.start { net.get('/') } } # is OK
```
These TIME_WAIT connections matter because a server receiving many HTTP requests from clients using Net::HTTP in this fashion (as Faraday does[1]) the server will begin to oversaturate and timeout past a particular scale.
I've tested and reproduced this in 2.7 and 2.6.
[1]: https://github.com/lostisland/faraday/pull/1117
---Files--------------------------------
dont-default-connection-close.patch (3.77 KB)
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2020-01-24 15:20 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <redmine.issue-16559.20200124035042@ruby-lang.org>
2020-01-24 3:50 ` [ruby-core:96988] [Ruby master Bug#16559] Net::HTTP#request does not properly close TCP socket if #started? is false me
2020-01-24 15:20 ` [ruby-core:96989] " me
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).