From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 3BA9A1F5AE for ; Mon, 15 Jun 2020 17:24:37 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id C6B6F120996; Tue, 16 Jun 2020 02:24:03 +0900 (JST) Received: from o1678948x4.outbound-mail.sendgrid.net (o1678948x4.outbound-mail.sendgrid.net [167.89.48.4]) by neon.ruby-lang.org (Postfix) with ESMTPS id F0F6B120924 for ; Tue, 16 Jun 2020 02:24:01 +0900 (JST) Received: by filterdrecv-p3iad2-784dbb6bd8-6snds with SMTP id filterdrecv-p3iad2-784dbb6bd8-6snds-19-5EE7AEC4-9C 2020-06-15 17:24:20.867180433 +0000 UTC m=+1027052.297167520 Received: from herokuapp.com (unknown) by ismtpd0003p1iad1.sendgrid.net (SG) with ESMTP id GIRttHIZQ1Gxz67kLPQXig for ; Mon, 15 Jun 2020 17:24:20.783 +0000 (UTC) Date: Mon, 15 Jun 2020 17:24:20 +0000 (UTC) From: jaruga@redhat.com Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 74602 X-Redmine-Project: ruby-master X-Redmine-Issue-Tracker: Bug X-Redmine-Issue-Id: 16936 X-Redmine-Issue-Author: jaruga X-Redmine-Sender: jaruga X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?yXpFNqRr1dEY0snEQ6vUpjORBmm3WV3CBkaa8tOjsKnIXsuP7QDSa8MpWLGpMg?= =?us-ascii?Q?hUPA4Y9uPs+qrV7tIJy7c1fSlJg9WSiYH+WFyc1?= =?us-ascii?Q?z3s3Pc287tTSDgf30e20shFt2IOcgug9GTJLs=2F4?= =?us-ascii?Q?I4uwqnAHb1JAg+zbtsVLtx3HZSMzDIUEQN5YUJc?= =?us-ascii?Q?8lquhCpwfsYW5Tyr6LpB=2FWr96iR+gZYeFY9VKCc?= =?us-ascii?Q?vM1v18NJzFNTcjFfA=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 98811 Subject: [ruby-core:98811] [Ruby master Bug#16936] `make test-all TESTS="-n !/Foo#method/"` not skipping the test case X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #16936 has been updated by jaruga (Jun Aruga). I noticed the test/unit negative filter was implemented as Ruby project's original code in 2016. https://github.com/ruby/ruby/commit/83e36bb5a6e02b747d10c1baf5e1b7ff2b4c49fe ---------------------------------------- Bug #16936: `make test-all TESTS="-n !/Foo#method/"` not skipping the test case https://bugs.ruby-lang.org/issues/16936#change-86173 * Author: jaruga (Jun Aruga) * Status: Closed * Priority: Normal * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- On the current latest master `cf1adf985ab78507250db0e217a0fdd779e2c6e6`. ``` $ autoconf $ ./configure --prefix=$(pwd)/dest $ make ``` The following `make test-all` works as I mentioned at #16935. ``` $ make test-all TESTS="-v -n /\^TestBugReporter#test_bug_reporter_add\$$/ -n /\^TestProcess#test_status_quit\$$/" ... [1/0] TestBugReporter#test_bug_reporter_add = 0.41 s [2/0] TestProcess#test_status_quit = 0.35 s Finished tests in 9.392046s, 0.2129 tests/s, 1.5971 assertions/s. 2 tests, 15 assertions, 0 failures, 0 errors, 0 skips ... ``` But it seems that the following `TestBugReporter#test_bug_reporter_add` and `TestProcess#test_status_quit` are not actually skipped. Is it a bug? ``` $ make test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/\^TestBugReporter#test_bug_reporter_add\$$/ -n \!/\^TestProcess#test_status_quit\$$/" 2>&1 | tee make.log Run options:- --seed=21367 "--ruby=./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/ -v -n "!/^TestBugReporter#test_bug_reporter_add$/" -n "!/^TestProcess#test_status_quit$/" # Running tests: [ 1/142] TestBugReporter#test_bug_reporter_add = 0.49 s ... [128/142] TestProcess#test_status_quit = 0.30 s ... Finished tests in 25.978847s, 5.4660 tests/s, 39.2627 assertions/s. 142 tests, 1020 assertions, 0 failures, 0 errors, 0 skips ``` I want to skip the specific test cases by `make test-all TESTS="..."` like this. I want to specify both the testing class (ex. `TestBugReporter`) and method (ex. `test_bug_reporter_add`) with the regular expression perfect matching to avoid unintended test cases are skipped. ``` $ make test-all TESTS="-v -n \!/\^TestBugReporter#test_bug_reporter_add\$$/ -n \!/\^TestProcess#test_status_quit\$$/" ``` The following `make test` works skipping `test_bug_reporter_add` and `test_status_quit` methods. ``` $ make test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/test_bug_reporter_add\$$/ -n \!/test_status_quit\$$/" 2>&1 | tee make2.log ... 140 tests, 1005 assertions, 0 failures, 0 errors, 0 skips ``` -- https://bugs.ruby-lang.org/