From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_PASS,T_DKIM_INVALID, T_RP_MATCHES_RCVD shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id B734820D0A for ; Mon, 29 May 2017 18:56:16 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 9C9D71207A3; Tue, 30 May 2017 03:56:13 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 1AA351207A2 for ; Tue, 30 May 2017 03:56:10 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=sendgrid.me; h=from:to:references:subject:mime-version:content-type:content-transfer-encoding:list-id; s=smtpapi; bh=fny+hxiQbQWTBCS0vHGSTNlG1N0=; b=BNNH1o8/PEAr6DiJi5 CTcHCkvfvRbmApjaVXS0DHMXluWH9JYAg70ku72hemvOP6HvrvVQzpqY3ETpMqku nmK7yY72kAe8QJ5yYFk8LYZ2m3YEIJXP5iZYVrqDqoTZZlSO/jN1Pl4WZakMCXD6 liIp9z74v1ch+OE35ySTOcvio= Received: by filter1149p1mdw1.sendgrid.net with SMTP id filter1149p1mdw1-32630-592C6EC6-1C 2017-05-29 18:56:06.485654577 +0000 UTC Received: from herokuapp.com (ec2-54-167-29-171.compute-1.amazonaws.com [54.167.29.171]) by ismtpd0002p1iad1.sendgrid.net (SG) with ESMTP id nF_GCgIZTJ6xwxxEMtdAQA Mon, 29 May 2017 18:56:06.439 +0000 (UTC) Date: Mon, 29 May 2017 18:56:06 +0000 From: burke@libbey.me To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 56486 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 13378 X-Redmine-Issue-Author: burke X-Redmine-Sender: burke X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS5Imx9iHYWItWO2UdI7V0tOtFzQJc9mMYNQ1J k+DqJDiceahVhuSKeTv5IzYgUfK2QILP7Ygoe7LYnMEtLCg4Be4AwPkbbeS/00C+bP/2MIyNt7ijOk zxfdNVVAjekzyrKvqvfiluxHQTe7U73z3Ua8 X-ML-Name: ruby-core X-Mail-Count: 81460 Subject: [ruby-core:81460] [Ruby trunk Feature#13378] Eliminate 4 of 8 syscalls when requiring file by absolute path X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #13378 has been updated by burke (Burke Libbey). File 0001-reduce-syscalls-on-require-v2.patch added Thank you for the feedback! I've attached an updated patch to address the issues. As for testing it, I haven't been able to think of a reasonable method to verify the behaviour without using dtrace/strace, since the only observable effect without system-level instrumentation should be a slight reduction in runtime. I *could* add a case to `DTrace::TestRequire`, but it feels wrong, since this file is concerned with testing the dtrace probes implemented by ruby, not testing ruby using built-in dtrace probes. Suggestions? ---------------------------------------- Feature #13378: Eliminate 4 of 8 syscalls when requiring file by absolute path https://bugs.ruby-lang.org/issues/13378#change-65163 * Author: burke (Burke Libbey) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- Don't open file twice when specified by absolute path. When invoking `require '/a.rb'` (i.e. via an absolute path), ruby generates this sequence of syscalls: open /a.rb fstat64 /a.rb close /a.rb open /a.rb fstat64 /a.rb fstat64 /a.rb read /a.rb close /a.rb It is apparent that the only inherently necessary members of this sequence are: open /a.rb fstat64 /a.rb read /a.rb close /a.rb (the fstat64 isn't *obviously* necessary, but it does serve a purpose and probably shouldn't be removed). The first open/fstat64/close is used to check whether the file is loadable. This is important when scanning the `$LOAD_PATH`, since it is used to determine when a file has been found. However, when we've already unambiguously identified a file before invoking `require`, this serves no inherent purpose, since we can move whatever work is happening as a result of that `fstat64` into the second open/close sequence. This change bypasses the first open/fstat64/close in the case of an absolute path to `require`. It also removes one of the doubled-up `fstat64` calls later in the sequence. As a result, the number of syscalls to require a file changes: * From 8 to 4 when specified by absolute path; * From 5+3n to 4+3n otherwise *(where n is the number of `$LOAD_PATH` items scanned)*. In future work, it would be possible to re-use the file descriptor opened while searching the `$LOAD_PATH` without the close/open sequence, but this would cause some ugly layering issues. --- *We intend to use this in conjunction with something like https://github.com/shopify/bootscale, which pre-resolves required features to absolute paths before calling `require`. This change reduces our total number of filesystem accesses by 13% during application boot.* *Various notes and rationale at http://notes.burke.libbey.me/ruby-require-optimization* ---Files-------------------------------- 0001-reduce-syscalls-on-require.patch (7.56 KB) 0001-reduce-syscalls-on-require-fixed.patch (6.94 KB) 0001-reduce-syscalls-on-require-v2.patch (6.44 KB) -- https://bugs.ruby-lang.org/