From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=AWL,BAYES_00, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 2B0931F86C for ; Fri, 27 Nov 2020 18:20:19 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 6CA801209A7; Sat, 28 Nov 2020 03:19:30 +0900 (JST) Received: from o1678948x4.outbound-mail.sendgrid.net (o1678948x4.outbound-mail.sendgrid.net [167.89.48.4]) by neon.ruby-lang.org (Postfix) with ESMTPS id 177D31209A1 for ; Sat, 28 Nov 2020 03:19:27 +0900 (JST) Received: by filterdrecv-p3las1-6665f6549b-52x5p with SMTP id filterdrecv-p3las1-6665f6549b-52x5p-19-5FC14358-91 2020-11-27 18:20:09.010611414 +0000 UTC m=+942560.300862820 Received: from herokuapp.com (unknown) by ismtpd0122p1mdw1.sendgrid.net (SG) with ESMTP id sHDad51qS7yfTTRs5D3-yQ for ; Fri, 27 Nov 2020 18:20:08.857 +0000 (UTC) Date: Fri, 27 Nov 2020 18:20:09 +0000 (UTC) From: mame@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 77009 X-Redmine-Project: ruby-master X-Redmine-Issue-Tracker: Bug X-Redmine-Issue-Id: 17350 X-Redmine-Issue-Author: ttokutake X-Redmine-Sender: mame X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?EJh2gqwnyqXtd++xo=2FinyA1V0bXouTB4FkWnzNiKb489ctaDOE3RLTkmk+RvC3?= =?us-ascii?Q?PsKYpakWFH3y+j4S1thOkrTeqhecN9myar7Tc7J?= =?us-ascii?Q?iIR5wRrXHyfO0V+7iHHp6FOA+UsiZ9FLl0g2J4t?= =?us-ascii?Q?3=2FJpyL6ewmpolOpiQyKMLJSM4AVnOjI1dLvSGlT?= =?us-ascii?Q?L3+VVtNpISnVJC4OXdjpq9GT=2FL49zkWWFac7kEc?= =?us-ascii?Q?qyMcxcv=2FJL27OE4Ms=3D?= To: ruby-core@ruby-lang.org X-Entity-ID: b/2+PoftWZ6GuOu3b0IycA== X-ML-Name: ruby-core X-Mail-Count: 101134 Subject: [ruby-core:101134] [Ruby master Bug#17350] mongo-ruby-driver causes the segfault at program's exit X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #17350 has been updated by mame (Yusuke Endoh). Thank you for the report. I can reproduce and investigated the issue. I think this is the known (but unfixed) issue #16288. When Ruby process ends, it terminates all threads, run all finalizers, and then destruct everything. However, Mongo gem attempts to create a new thread in a finalizer. In fact, I've observed `native_thread_create` is called after `rb_thread_terminate_all` returned, and seen the following backtrace by `rb_backtrace()` with gdb. ``` from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/cluster.rb:422:in `block in finalize' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/session/session_pool.rb:118:in `end_sessions' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/op_msg_or_command.rb:27:in `execute' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:38:in `execute' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:26:in `do_execute' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/response_handling.rb:73:in `unpin_maybe' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:27:in `block in do_execute' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/response_handling.rb:43:in `add_error_labels' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:28:in `block (2 levels) in do_execute' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/response_handling.rb:87:in `add_server_diagnostics' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:29:in `block (3 levels) in do_execute' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:50:in `get_result' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:55:in `dispatch_message' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server.rb:410:in `with_connection' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection_pool.rb:556:in `with_connection' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection_pool.rb:357:in `check_out' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection_pool.rb:731:in `connect_connection' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection.rb:171:in `connect!' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection.rb:184:in `do_connect' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:162:in `socket' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:202:in `create_resolver' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:202:in `each' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:205:in `block in create_resolver' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address/ipv4.rb:93:in `socket' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address/ipv4.rb:93:in `new' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/socket/tcp.rb:67:in `initialize' from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/socket/tcp.rb:41:in `connect!' from /usr/local/lib/ruby/2.6.0/timeout.rb:103:in `timeout' from /usr/local/lib/ruby/2.6.0/timeout.rb:83:in `block in timeout' from /usr/local/lib/ruby/2.6.0/timeout.rb:83:in `start' ``` @ko1 and @nobu, what do you think? ---------------------------------------- Bug #17350: mongo-ruby-driver causes the segfault at program's exit https://bugs.ruby-lang.org/issues/17350#change-88817 * Author: ttokutake (Tadatoshi Tokutake) * Status: Open * Priority: Normal * ruby -v: Ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- ## Overview mongo-ruby-driver 2.11.0 has caused the segfault at program's exit when connecting a certain MongoDB cluster. I also reported this issue to mongo-ruby-driver. https://jira.mongodb.org/browse/RUBY-2453 This is the reproduction example. Please check it out. https://github.com/ttokutake/mongo-ruby-driver-segfault You can see full result by using the above example, and also debug it by GDB. ``` Fetching gem metadata from https://rubygems.org/........ Resolving dependencies... Fetching bson 4.11.1 Installing bson 4.11.1 with native extensions Using bundler 1.17.2 Fetching mongo 2.11.0 Installing mongo 2.11.0 D, [2020-11-27T00:20:31.035863 #11] DEBUG -- : MONGODB | EVENT: # ... (snip) D, [2020-11-27T00:20:31.145112 #11] DEBUG -- : MONGODB | [14] mongodb-sharded.cluster.local:27017 | admin.endSessions | SUCCEEDED | 0.000s [BUG] Segmentation fault at 0x0000000000000050 ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0001 p:---- s:0003 e:000002 (none) [FINISH] -- Machine register context ------------------------------------------------ RIP: 0x00007f52358f4078 RBP: 0x00007f5230b4fdd0 RSP: 0x00007f5230b4fc90 RAX: 0x0000000000000000 RBX: 0x0000563b4b746370 RCX: 0x0000563b44ad4158 RDX: 0x0000563b44ad4570 RDI: 0x0000563b4b8bd318 RSI: 0x0000563b44ad4020 R8: 0x0000000000000011 R9: 0x0000000000000000 R10: 0x0000563b4a8dc050 R11: 0x0000000000000003 R12: 0x0000000000000009 R13: 0x0000563b44b7f630 R14: 0x0000563b46ce6ac0 R15: 0x0000563b498440d8 EFL: 0x0000000000010297 -- C level backtrace information ------------------------------------------- corrupted double-linked list Aborted ``` ## Consideration - https://bugs.ruby-lang.org/issues/16288 may be a similar issue. - [a similar result](https://github.com/ruby-concurrency/concurrent-ruby/issues/808#issuecomment-534944201). - [This commit](https://github.com/mongodb/mongo-ruby-driver/commit/485ee7b24a1d34b3ea52b998dfd4dcc25454b6a5) has caused the segfault. - It does not happen if [this line](https://github.com/mongodb/mongo-ruby-driver/commit/485ee7b24a1d34b3ea52b998dfd4dcc25454b6a5#diff-10c9bc0ac2ab9e61b772cab992027054809ec5e71268a32541e374a7e7b5af9aR70) is commented out. ## Ruby versions - Ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux] - Ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux] -- https://bugs.ruby-lang.org/