From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-3.0 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DNSWL_HI, SPF_HELO_PASS,SPF_PASS,UNPARSEABLE_RELAY shortcircuit=no autolearn=no autolearn_force=no version=3.4.2 Received: from nue.mailmanlists.eu (nue.mailmanlists.eu [IPv6:2a01:4f8:1c0c:6b10::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 3347F1F910 for ; Fri, 25 Nov 2022 21:55:06 +0000 (UTC) Authentication-Results: dcvr.yhbt.net; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ruby-lang.org header.i=@ruby-lang.org header.b="eMsOf/Ut"; dkim-atps=neutral Received: from nue.mailmanlists.eu (localhost [127.0.0.1]) by nue.mailmanlists.eu (Postfix) with ESMTP id BCF307E582; Fri, 25 Nov 2022 21:54:42 +0000 (UTC) Authentication-Results: nue.mailmanlists.eu; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ruby-lang.org header.i=@ruby-lang.org header.a=rsa-sha256 header.s=s1 header.b=eMsOf/Ut; dkim-atps=neutral Received: from xtrwkhkc.outbound-mail.sendgrid.net (xtrwkhkc.outbound-mail.sendgrid.net [167.89.16.28]) by nue.mailmanlists.eu (Postfix) with ESMTPS id 26C247E561 for ; Fri, 25 Nov 2022 10:41:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ruby-lang.org; h=from:references:subject:mime-version:content-type: content-transfer-encoding:list-id:to:cc; s=s1; bh=2okJsTIk1QmCgMSPTMOoWcIb+CIH8KJrHeecaTHNDZQ=; b=eMsOf/UtGLtxL1Mdu5CcLxVtUbnidBrsWb77UxfbxLUe1EW5vqKqoxbCb8dSTe1ZqvMa 3SVc0dYTcUKEkrEN9apVWruNCvorsRLfeRUhb/SgiLHWT8Ut8cSi+Agm4rJ0lqOJ9cRdei oUzyIrpYpjqJGQwTMZFE2XPbF9Ar6xc4vNTzbBnu0Hw4xchwsdqOM3TimyZjxmSdIcbbsm SoEFawDrE/uAGC70KTtng2piBNuBFpvjXSabHvw/n0ZKMmOfP5+sRJl47567DLHunuQdpg dDrtfah9nNoZxOIDJYqsk+2wJ1+mrUAp8BVysP6j3n2GdrXYd3t31QZMpMtoS6Xw== Received: by filterdrecv-6b9548745c-dzhl2 with SMTP id filterdrecv-6b9548745c-dzhl2-1-63809BBC-18 2022-11-25 10:41:00.535242031 +0000 UTC m=+646363.769130473 Received: from herokuapp.com (unknown) by geopod-ismtpd-3-3 (SG) with ESMTP id TRlOiNEOT5emmKvUoBt9ag for ; Fri, 25 Nov 2022 10:41:00.478 +0000 (UTC) Date: Fri, 25 Nov 2022 10:41:00 +0000 (UTC) From: "jaruga (Jun Aruga)" Message-ID: References: Mime-Version: 1.0 X-Redmine-Project: ruby-master X-Redmine-Issue-Tracker: Misc X-Redmine-Issue-Id: 19149 X-Redmine-Issue-Author: jaruga X-Redmine-Sender: jaruga X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-Redmine-MailingListIntegration-Message-Ids: 87353 X-SG-EID: =?us-ascii?Q?8MFEQCfxuhzj2xN4bOwU97LZLvLMvdKraBWyqL2uDEHgSji2Jvow9iwWHnUrVV?= =?us-ascii?Q?k5pTGRuxM8KqD8RSTmRcwLIUcz7lBFRuKuMCkLM?= =?us-ascii?Q?71j9hyDhHNQ=2FgsWfcnPhuRPCWldtFtX7BbA1gGh?= =?us-ascii?Q?OpvJZ7cmjI6Fqn2Z3t2fqyKS36cdlUMSYaxC9QO?= =?us-ascii?Q?LBDYrpWYIPqxMLZNaae3sq75JmBRf2JvFn7oOEB?= =?us-ascii?Q?KGs2V0bi7RyGHnfZhUKJtWRGa1f9o=2FPW6zpujJl?= =?us-ascii?Q?mTvmfbbUp1IxTNZpeHJVQ=3D=3D?= To: ruby-core@ml.ruby-lang.org X-Entity-ID: b/2+PoftWZ6GuOu3b0IycA== Message-ID-Hash: D6XPLC2ROJDGIOV56BW35H4QY25PA53Q X-Message-ID-Hash: D6XPLC2ROJDGIOV56BW35H4QY25PA53Q X-MailFrom: bounces+313651-b711-ruby-core=ml.ruby-lang.org@em5188.ruby-lang.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list Reply-To: Ruby developers Subject: [ruby-core:111008] [Ruby master Misc#19149] Minimal covered tests with the --enable-yjit case? List-Id: Ruby developers Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Issue #19149 has been updated by jaruga (Jun Aruga). maximecb (Maxime Chevalier-Boisvert) wrote in #note-1: > If you want to thoroughly test everything, you could run make check two times, once with YJIT, and once without. > > If you just run: > ``` > $ make check RUN_OPTS="--yjit" > ``` > > This will run YJIT with its default options, which is what most people would use when running YJIT. If you specify `--yjit-call-threshold=1 --yjit-verify-ctx`, it will make YJIT compile every single method and run extra verifications. This is what we do on the CI to be extra thorough, but it will be slower. > > If you just want to quickly verify that YJIT works on the host platform, you could just run `make btest` with `--yjit` for some quick minimal verification. We do already test YJIT pretty well on the GitHub CI and on `rubyci.org`. Thank you for clarifying it. I understood it. I am still trying to find a good balance of running the YJIT tests between the running time and the coverage. In the case of "MJIT", while the MJIT is supported as a default. And we don't run the tests with RUN_OPTS="--mjit" in the Feodra Ruby's RPM recipe file, while I notice there are tests with `RUN_OPTS="--mjit"` now at . Because we can run the the following MJIT tests within the `make check` without `RUN_OPTS="--mjit"`. Here are the test results on the yesterday's latest master `66e5200ba435361624caa3e23db7962d906b70db`. ``` $ grep MJIT_SUPPORT rbconfig.rb CONFIG["MJIT_SUPPORT"] = "yes" $ make test-all TESTS=test/ruby/test_mjit.rb revision.h updated Run options: --seed=16704 "--ruby=./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/ # Running tests: Finished tests in 132.081114s, 0.7950 tests/s, 4.2020 assertions/s. 105 tests, 555 assertions, 0 failures, 0 errors, 4 skips ruby -v: ruby 3.2.0dev (2022-11-24T06:13:00Z master 66e5200ba4) [x86_64-linux] ``` However, we cannot run the following YJIT tests without `RUN_OPTS="--yjit"`. The tests were skipped. ``` $ grep YJIT_SUPPORT rbconfig.rb CONFIG["YJIT_SUPPORT"] = "dev" $ make test-all TESTS=test/ruby/test_yjit.rb revision.h updated Run options: --seed=38677 "--ruby=./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/ # Running tests: Finished tests in 0.017483s, 0.0000 tests/s, 0.0000 assertions/s. 0 tests, 0 assertions, 0 failures, 0 errors, 0 skips ruby -v: ruby 3.2.0dev (2022-11-24T06:13:00Z master 66e5200ba4) [x86_64-linux] ``` What does the difference of the ideas come from? ---------------------------------------- Misc #19149: Minimal covered tests with the --enable-yjit case? https://bugs.ruby-lang.org/issues/19149#change-100258 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal ---------------------------------------- In the [Fedora Ruby's RPM recipe file](https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec), we were running the commands below. ``` $ ./autogen.sh $ ./configure ... $ make $ make check ``` ## What is the minimal covered tests? Now the we want to add the `--enable-yjit` option. Could you tell me what is the minimal covered test commands in the `--enable-yjit` case? ``` $ ./autogen.sh $ ./configure --enable-yjit ... $ make $ ?? ``` Then do we need to run the `make check` 2 times with both with yjit and without yjit as follows? The yjit command options `--yjit-call-threshold=1 --yjit-verify-ctx` comes from the . ``` $ make check $ make check RUN_OPTS="--yjit-call-threshold=1 --yjit-verify-ctx" ``` Or is it good enough to run the `make check` and the specific tests with the yjit options as follows? ``` $ make check $ make test-all RUN_OPTS="--yjit-call-threshold=1 --yjit-verify-ctx" TESTS="test/ruby/{test_yjit_exit_locations.rb,test_yjit.rb}" ``` Or is it good enough to run the `make check` with the YJIT options as follows? ``` $ make check RUN_OPTS="--yjit-call-threshold=1 --yjit-verify-ctx" ``` ## YJIT command options Could you explain why the command options `--yjit-call-threshold=1 --yjit-verify-ctx` above is better to test the YJIT cases rather than just `--yjit`? ## Ideal situation I want to see the just running `make check` covers necessary cases in YJIT. Because it is convenience, and I think users tend to be satisfied with only running the `make check`. What do you think? ``` $ ./autogen.sh $ ./configure --enable-yjit ... $ make $ make check ``` I tried to inject the YJIT command options in a test file for that. Perhaps it might be like this. But so far I am not succeeded. ``` diff --git a/test/lib/jit_support.rb b/test/lib/jit_support.rb index 26f8542dc2..3fce402e32 100644 --- a/test/lib/jit_support.rb +++ b/test/lib/jit_support.rb @@ -69,8 +69,10 @@ def supported? end def yjit_supported? + return @yjit_supported if defined?(@yjit_supported) # e.g. x86_64-linux, x64-mswin64_140, x64-mingw32, x64-mingw-ucrt - RUBY_PLATFORM.match?(/^(x86_64|x64|arm64|aarch64)-/) + @yjit_supported = RbConfig::CONFIG["YJIT_SUPPORT"] != 'no' && + RUBY_PLATFORM.match?(/^(x86_64|x64|arm64|aarch64)-/) end def remove_mjit_logs(stderr) diff --git a/test/ruby/test_yjit.rb b/test/ruby/test_yjit.rb index 9ab058d97b..10c8e3b891 100644 --- a/test/ruby/test_yjit.rb +++ b/test/ruby/test_yjit.rb @@ -8,7 +8,7 @@ require 'tmpdir' require_relative '../lib/jit_support' -return unless defined?(RubyVM::YJIT) && RubyVM::YJIT.enabled? +return unless JITSupport.yjit_supported? # Tests for YJIT with assertions on compilation and side exits # insipired by the MJIT tests in test/ruby/test_mjit.rb ``` -- https://bugs.ruby-lang.org/