From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 23EDC1F454 for ; Thu, 7 Nov 2019 20:43:59 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 4C0FC120B67; Fri, 8 Nov 2019 05:43:48 +0900 (JST) Received: from o1678948x4.outbound-mail.sendgrid.net (o1678948x4.outbound-mail.sendgrid.net [167.89.48.4]) by neon.ruby-lang.org (Postfix) with ESMTPS id C97EF120B54 for ; Fri, 8 Nov 2019 05:43:45 +0900 (JST) Received: by filter0079p3mdw1.sendgrid.net with SMTP id filter0079p3mdw1-1991-5DC48205-25 2019-11-07 20:43:49.470670516 +0000 UTC m=+180233.299526263 Received: from herokuapp.com (unknown [34.230.5.86]) by ismtpd0029p1mdw1.sendgrid.net (SG) with ESMTP id q0oGWmhuRjCah8fLoQZygw for ; Thu, 07 Nov 2019 20:43:49.397 +0000 (UTC) Date: Thu, 07 Nov 2019 20:43:49 +0000 (UTC) From: jaruga@redhat.com Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 71373 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 16234 X-Redmine-Issue-Author: jaruga X-Redmine-Sender: jaruga X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?yXpFNqRr1dEY0snEQ6vUpjORBmm3WV3CBkaa8tOjsKmGb9ub+ESnpMgoRfHchH?= =?us-ascii?Q?nq5=2FPADgnHxpckViSRShjIETDCpEsYJ2XKnBkrM?= =?us-ascii?Q?wmvoKaiXTM22kJspk4Dzn4YGgPGoIfy9teXvpNA?= =?us-ascii?Q?pG9aBCSESRt2NWxdi+zpZlzAIGR5VN4fXwfjc2d?= =?us-ascii?Q?t8n0wTOnXzYeoCGVUCy03gZ6bQlCQdjs=2FoA=3D=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 95748 Subject: [ruby-core:95748] [Ruby master Misc#16234] Enabling ARM 64/32-bit cases by Drone CI X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #16234 has been updated by jaruga (Jun Aruga). jaruga (Jun Aruga) wrote: > Shall we add debug code to investigate a random "make install" error on arm64 to `.travis.yml`? > At least we can remove current existing debugging code for Mac OSX after "make install" in `.travis.yml`. > > This pull-request is the suggestion. > https://github.com/ruby/ruby/pull/2642 For the permission issue `cp: cannot create regular file '../../.ext/common/json.rb': Permission denied` in `make install`, I found a clue. I did run following commands in both Travis x86_64 and arm32. The issue is happening in `ext/*/Makefile` files. `COPY` and `V` argument are used in the `Makefile` files. ``` ls -l --full-time -t $(pwd)/.ext/.timestamp/.RUBYCOMMONDIR.time $(pwd)/.ext/common/*.rb || true make --debug --trace V=1 COPY='pwd; (ls -l --full-time -t ../../.ext/.timestamp/.RUBYCOMMONDIR.time ../../.ext/common/*.rb || true); cp -v' install ``` Then here is the result in Travis x86_64. The `.ext/.timestamp/.RUBYCOMMONDIR.time` file has to be older than other `.ext/common/*.rb` files. Because when `.RUBYCOMMONDIR.time` file is newer than a `.ext/common/foo.rb` file, `COPY` (= `cp` as default) from `ext/json/lib/json.rb` to `.ext/common/foo.rb` is executed in `ext/foo/Makefile`. But the destination `.ext/common/foo.rb` file permission is `444` (read only `-r--r--r--`), the copy is failed. When `COPY='cp -f'`, it is succeeded. ``` $ ls -l --full-time -t $(pwd)/.ext/.timestamp/.RUBYCOMMONDIR.time $(pwd)/.ext/common/*.rb Thu Nov 7 19:56:50 UTC 2019 -r--r--r-- 1 travis travis 2894 2019-11-07 19:56:48.937972000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/digest.rb -r--r--r-- 1 travis travis 16562 2019-11-07 19:56:48.901990000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/pathname.rb -r--r--r-- 1 travis travis 2494 2019-11-07 19:56:46.523179999 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/ripper.rb -r--r--r-- 1 travis travis 44702 2019-11-07 19:56:46.399242000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/socket.rb -r--r--r-- 1 travis travis 1722 2019-11-07 19:56:46.323280000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/fiddle.rb -r--r--r-- 1 travis travis 6706 2019-11-07 19:56:46.095393999 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/monitor.rb -r--r--r-- 1 travis travis 1036 2019-11-07 19:56:46.027427999 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/date.rb -r--r--r-- 1 travis travis 24 2019-11-07 19:56:45.807538000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/bigdecimal.rb -r--r--r-- 1 travis travis 368 2019-11-07 19:56:45.507688000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/coverage.rb -r--r--r-- 1 travis travis 1809 2019-11-07 19:56:45.491696000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/json.rb -r--r--r-- 1 travis travis 469 2019-11-07 19:56:45.247817999 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/openssl.rb -r--r--r-- 1 travis travis 5861 2019-11-07 19:56:44.947967999 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/kconv.rb -r--r--r-- 1 travis travis 21533 2019-11-07 19:56:44.947967999 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/psych.rb -r--r--r-- 1 travis travis 2217 2019-11-07 19:56:44.452216000 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/expect.rb -rw-rw-r-- 1 travis travis 0 2019-11-07 19:56:44.364259999 +0000 /home/travis/build/junaruga/ruby/build/.ext/.timestamp/.RUBYCOMMONDIR.time ``` In Travis arm64, Some `.ext/common/*.rb` files are newer than `.ext/.timestamp/.RUBYCOMMONDIR.time` file. That is the reason of the error. But the problem is why some `.ext/common/*.rb` files are newer than `.ext/.timestamp/.RUBYCOMMONDIR.time` file. This situation does not happen at the 1st CI build by a commit changing the `.travis.yml`. This happens after 2nd CI build by a commit without changing `.travis.yml`. ``` $ ls -l --full-time -t $(pwd)/.ext/.timestamp/.RUBYCOMMONDIR.time $(pwd)/.ext/common/*.rb Thu Nov 7 19:59:01 UTC 2019 -r--r--r-- 1 travis travis 5861 2019-11-07 19:58:46.029993853 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/kconv.rb -r--r--r-- 1 travis travis 16562 2019-11-07 19:58:45.737996933 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/pathname.rb -r--r--r-- 1 travis travis 469 2019-11-07 19:58:44.746007400 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/openssl.rb -r--r--r-- 1 travis travis 44702 2019-11-07 19:58:43.606019428 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/socket.rb -r--r--r-- 1 travis travis 2494 2019-11-07 19:58:43.530020230 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/ripper.rb -r--r--r-- 1 travis travis 21533 2019-11-07 19:58:43.470020863 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/psych.rb -r--r--r-- 1 travis travis 1036 2019-11-07 19:58:43.294022720 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/date.rb -r--r--r-- 1 travis travis 1722 2019-11-07 19:58:42.994025885 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/fiddle.rb -r--r--r-- 1 travis travis 24 2019-11-07 19:58:42.422031920 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/bigdecimal.rb -r--r--r-- 1 travis travis 2894 2019-11-07 19:58:42.370032469 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/digest.rb -rw-r--r-- 1 travis travis 0 2019-11-07 19:58:42.326032933 +0000 /home/travis/build/junaruga/ruby/build/.ext/.timestamp/.RUBYCOMMONDIR.time -r--r--r-- 1 travis travis 2217 2019-11-07 19:58:42.174034537 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/expect.rb -r--r--r-- 1 travis travis 6706 2019-11-07 19:58:42.130035001 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/monitor.rb -r--r--r-- 1 travis travis 368 2019-11-07 19:58:41.638040192 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/coverage.rb -r--r--r-- 1 travis travis 1809 2019-11-07 19:58:41.266044117 +0000 /home/travis/build/junaruga/ruby/build/.ext/common/json.rb ``` * 1st CI build (arm64): ok: https://travis-ci.org/ruby/ruby/jobs/608915893#L2361 * 2nd CI build (arm64) with empty commit: error: https://travis-ci.org/ruby/ruby/jobs/608925381#L2351 ---------------------------------------- Misc #16234: Enabling ARM 64/32-bit cases by Drone CI https://bugs.ruby-lang.org/issues/16234#change-82565 * Author: jaruga (Jun Aruga) * Status: Closed * Priority: Normal * Assignee: ---------------------------------------- Currently ruby project has 4 CIs on GitHub. 1. Travis CI: linux cases with flags and compilers. 2. GitHub Actions: macros, windows, ubuntu 3. Wercker: Ruby JIT cases 4. Appveyor: windows I like to suggest 5th CI: Drone CI for ARM 64/32-bit cases. Drone CI supports native the ARM 64/32 bit environments. Have you used Drone CI? I tried to use both Drone CI and Shippable CI supporting ARM. My impression for Drone CI is quite good. Great user experience and user interface. Shippable CI was not so good for some reasons. Drone CI have not only linux ARM 64/32 bit environments on DockerRunner mode (= using container for CI like Wercker), but also freebsd, netbsd, openbsd, dragonfly (?) and solaris environments on ExecRunner (= maybe running commands directly without container) mode according to the following documents. * https://exec-runner.docs.drone.io/configuration/platform/ * https://docker-runner.docs.drone.io/configuration/platform/ Is it exciting isn't it? We can check ARM issue at a pull-request timing. Here is the example. The content is almost same with wercker.yml except JIT option. "ruby/3" is failed on the latest master branch, but "ruby/2" arm64 case is succeeded on old master branch. https://cloud.drone.io/junaruga/ruby/3 https://github.com/junaruga/ruby/blob/feature/ci-arm/.drone.yml https://cloud.drone.io/junaruga/ruby/2 Here is the pull-request as an example. https://github.com/ruby/ruby/pull/2520 .drone.yml is the file to manage the CI cases. But when you see most of the YAML parts between ARM 64-bit and 32-bit cases in .drone.yml is same. In case of .traivs.yml, we are using YAML anchor (&) and reference (*) feature effectively. But in case of .drone.yml I am not sure we can still use it beyond the "---" separator. Luckily Drone CI started providing the alternative .drone.star file by Starlark language. https://docs.drone.io/starlark/overview/ https://blog.drone.io/create-pipelines-using-starlark/ Enabling Drone CI is quite simple. Just go to https://drone.io/ , then register and enable target repository. UI is quite good. Pros * We can check ARM 64/32-bit cases, and possibly freebsd and solaris cases too. * It's for free. * Each developer can debug ARM cases on their forked repository. * Customize easily. I see .travis.yml is used effectively. Cons * Have to manage additonal file .drone.yml or .drone.star. But first, I want to ask you. Are you interested in using Drone CI for Ruby project? -- https://bugs.ruby-lang.org/