From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.0 required=3.0 tests=AWL,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=no autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id D83761F97E for ; Sat, 24 Nov 2018 17:11:48 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 31212121B5B; Sun, 25 Nov 2018 02:11:47 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id EA7D1121B5B for ; Sun, 25 Nov 2018 02:11:44 +0900 (JST) Received: by filter0163p3mdw1.sendgrid.net with SMTP id filter0163p3mdw1-16536-5BF9864D-2B 2018-11-24 17:11:41.836659959 +0000 UTC m=+765372.000191888 Received: from herokuapp.com (ec2-54-82-99-231.compute-1.amazonaws.com [54.82.99.231]) by ismtpd0051p1mdw1.sendgrid.net (SG) with ESMTP id i-LsZSTvQw6ZEBPv_C8Urw Sat, 24 Nov 2018 17:11:41.677 +0000 (UTC) Date: Sat, 24 Nov 2018 17:11:42 +0000 (UTC) From: Greg.mpls@gmail.com To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 65438 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 14968 X-Redmine-Issue-Author: normalperson X-Redmine-Sender: MSP-Greg X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS4InJHlLq01Fquqat6nF1VCq8AQAh4Db9EcGv 9QymG886baMsMeetU2KfJC3kaG13VKtS0or0Pr94hMGyz9iVFd7eDtfaXirRAMCXjJtxLf+m930bEy CXwj5IjIfODmac/slHnRHD3EB0LOjNKwDbpCsRyDYJVF/0GZAQFarK94gw== X-ML-Name: ruby-core X-Mail-Count: 90042 Subject: [ruby-core:90042] [Ruby trunk Bug#14968] [PATCH] io.c: make all pipes nonblocking by default X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #14968 has been updated by MSP-Greg (Greg L). @normalperson Eric, > Do you have any more details on the slowdown (e.g. profiling info?) I pointed it out thinking it might help with the issues. I concentrated more on determining what's failing in test-all. > Otherwise, I think we can accept win32 and POSIX-like platforms are too different and deserve different defaults (Same as r65929 for socket): That maybe the outcome. You've taken a lot of time with all the code & testing you've done, so thank you. JFYI, there are currently several tests in test_io.rb & test_readpartial.rb that are freezing. Summary below: ``` test-all failure, errors, & stops ruby 2.6.0dev (2018-11-24 trunk 65950) [x64-mingw32] The following tests froze test/ruby/test_io.rb TestIO S test_cross_thread_close_fd S test_read_buffer_error S test_read_unlocktmp_ensure S test_readpartial_buffer_error S test_readpartial_locktmp S test_readpartial_unlocktmp_ensure S test_recycled_fd_close test/ruby/test_readpartial.rb TestReadPartial S test_open_pipe S test_with_stdio After commenting out the above tests, here are the errors in three test files. I believe the ruby test-all folder passes otherwise. 1) Failure: TestThreadQueue#test_thr_kill [ruby/test/ruby2/test_thread_queue.rb:153]: only 0/250 done in 60 seconds. 2) Error: TestIO#test_race_closed_stream: Timeout::Error: execution of assert_separately expired timeout (10 sec) pid 8996 exit 0 | ruby/test/ruby2/test_io.rb:3591:in `test_race_closed_stream' 3) Error: TestIO#test_cross_thread_close_stdio: Timeout::Error: execution of assert_separately expired timeout (10 sec) pid 6044 exit 0 | ruby/test/ruby2/test_io.rb:2946:in `test_cross_thread_close_stdio' 4) Error: TestIO#test_closed_stream_in_rescue: Timeout::Error: execution of assert_separately expired timeout (10 sec) pid 7816 exit 0 | ruby/test/ruby2/test_io.rb:3685:in `test_closed_stream_in_rescue' ``` ---------------------------------------- Bug #14968: [PATCH] io.c: make all pipes nonblocking by default https://bugs.ruby-lang.org/issues/14968#change-75151 * Author: normalperson (Eric Wong) * Status: Closed * Priority: Normal * Assignee: * Target version: * ruby -v: * Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN ---------------------------------------- Crap, I always planned to have something like this for [Feature #13618] (auto-Fiber); but introducing a race condition for Timer-thread elimination in [Misc #14937] forced me to introduce this early. Anyways, I might have to revert and reintroduce timer-thread if this change is unnacceptable :< (I HATE timer thread) ``` io.c: make all pipes nonblocking by default All normal Ruby IO methods (IO#read, IO#gets, IO#write, ...) are all capable of appearing to be "blocking" when presented with a file description with the O_NONBLOCK flag set; so there is little risk of incompatibility within Ruby-using programs. The biggest compatibility risk is when spawning external programs. As a result, stdin, stdout, and stderr are now always made blocking before exec-family calls. Timer-thread elimination in https://bugs.ruby-lang.org/issues/14937 introduced a race condition in signal handling. It is possible to receive a signal inside BLOCKING_REGION right before read/write syscalls. If this patch cannot be accepted, I will have to revert to reintroduce timer-thread and increase resource use (which led to other failures in the past). The race condition introduced for [Misc #14937] led to rare CI failures on a few tests: - test/ruby/test_thread.rb (test_thread_timer_and_interrupt): http://www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20180805T080500Z.fail.html.gz - test/ruby/test_io.rb (test_race_gets_and_close): http://ci.rvm.jp/results/trunk@P895/1190369 This change is ALSO necessary to take advantage of (proposed lightweight concurrency (aka "auto-Fiber") or any similar proposal: https://bugs.ruby-lang.org/issues/13618 TODO: all sockets and FIFOs non-blocking by default, too ``` ---Files-------------------------------- 0001-io.c-make-all-pipes-nonblocking-by-default.patch (12.1 KB) 14968_28_failures_errors.log (23.5 KB) -- https://bugs.ruby-lang.org/