From: email@example.com To: firstname.lastname@example.org Subject: [ruby-core:95763] [Ruby master Misc#16234] Enabling ARM 64/32-bit cases by Drone CI Date: Fri, 08 Nov 2019 18:09:35 +0000 (UTC) [thread overview] Message-ID: <email@example.com> (raw) In-Reply-To: <firstname.lastname@example.org> Issue #16234 has been updated by jaruga (Jun Aruga). > Some `.ext/common/*.rb` files are newer than `.ext/.timestamp/.RUBYCOMMONDIR.time` file. > That is the reason of the error. Sorry above is typo. To be correct, some `.ext/common/*.rb` files can be "older" than `.ext/.timestamp/.RUBYCOMMONDIR.time` file. That is the reason of the error. I investigated more about this. I might find the root cause of the error. First, `.ext/common/*.rb` and `ext/.timestamp/.RUBYCOMMONDIR.time` files are created by the command `$SETARCH make -s $JOBS`. I checked the behavior by `$SETARCH make --debug --trace V=1 $JOBS`. It runs `exts.mk` (maybe generated from `template/exts.mk.tmpl`?). And it executes `extensions` to build `ext/*/Makefile` in `exts.mk` in parallel with given `$JOBS`. The default value = -j3 for Travis x86_64, and -j33 for Travis arm64. Yes it is "-j33" on arm64. exts.mk ``` extensions = ext/psych/. ext/monitor/. ext/rbconfig/sizeof/. ext/pty/. \ ext/objspace/. ext/etc/. ext/openssl/. ext/ripper/. \ ... all static: ruby ... ruby: $(extensions:/.=/all) ``` Following each `Makefile` is executed in parallel without a exclusive lock. As a result, it seems `touch .RUBYCOMMONDIR.time` command is executed more than 2 times. `ext/*/Makefile` ``` $(TIMESTAMP_DIR)/.RUBYCOMMONDIR.time: $(Q) $(MAKEDIRS) $(@D) $(RUBYLIBDIR) $(Q) $(TOUCH) $@ ``` Following steps can happen. 1. `touch .ext/.timestamp/.RUBYCOMMONDIR.time` in `ext/foo/Makefile` 2. Create `.ext/common/foo.rb` in `ext/foo/Makefile` 3. `touch .ext/.timestamp/.RUBYCOMMONDIR.time` in `ext/bar/Makefile` 4. Create `.ext/common/bar.rb` in `ext/bar/Makefile` As a result, some `.ext/common/*.rb` files (In this case `.ext/common/foo.rb`) are older than `.ext/.timestamp/.RUBYCOMMONDIR.time` file. That is the reason of the error. ## Possible solution 1. Add an exclusive lock logic to `ext/*/Makefile` `$(TIMESTAMP_DIR)/.RUBYCOMMONDIR.time` task. 2. Run `ruby: $(extensions:/.=/all)` in `exts.mk` without parallel. But the performance is slower. Right now I am working to run with JOBS= (empty) for Travis arm64 case here. https://github.com/ruby/ruby/pull/2642 How do you think? I think the solution "1." is better, if we can implement it. ---------------------------------------- Misc #16234: Enabling ARM 64/32-bit cases by Drone CI https://bugs.ruby-lang.org/issues/16234#change-82580 * Author: jaruga (Jun Aruga) * Status: Closed * Priority: Normal * Assignee: ---------------------------------------- Currently ruby project has 4 CIs on GitHub. 1. Travis CI: linux cases with flags and compilers. 2. GitHub Actions: macros, windows, ubuntu 3. Wercker: Ruby JIT cases 4. Appveyor: windows I like to suggest 5th CI: Drone CI for ARM 64/32-bit cases. Drone CI supports native the ARM 64/32 bit environments. Have you used Drone CI? I tried to use both Drone CI and Shippable CI supporting ARM. My impression for Drone CI is quite good. Great user experience and user interface. Shippable CI was not so good for some reasons. Drone CI have not only linux ARM 64/32 bit environments on DockerRunner mode (= using container for CI like Wercker), but also freebsd, netbsd, openbsd, dragonfly (?) and solaris environments on ExecRunner (= maybe running commands directly without container) mode according to the following documents. * https://exec-runner.docs.drone.io/configuration/platform/ * https://docker-runner.docs.drone.io/configuration/platform/ Is it exciting isn't it? We can check ARM issue at a pull-request timing. Here is the example. The content is almost same with wercker.yml except JIT option. "ruby/3" is failed on the latest master branch, but "ruby/2" arm64 case is succeeded on old master branch. https://cloud.drone.io/junaruga/ruby/3 https://github.com/junaruga/ruby/blob/feature/ci-arm/.drone.yml https://cloud.drone.io/junaruga/ruby/2 Here is the pull-request as an example. https://github.com/ruby/ruby/pull/2520 .drone.yml is the file to manage the CI cases. But when you see most of the YAML parts between ARM 64-bit and 32-bit cases in .drone.yml is same. In case of .traivs.yml, we are using YAML anchor (&) and reference (*) feature effectively. But in case of .drone.yml I am not sure we can still use it beyond the "---" separator. Luckily Drone CI started providing the alternative .drone.star file by Starlark language. https://docs.drone.io/starlark/overview/ https://blog.drone.io/create-pipelines-using-starlark/ Enabling Drone CI is quite simple. Just go to https://drone.io/ , then register and enable target repository. UI is quite good. Pros * We can check ARM 64/32-bit cases, and possibly freebsd and solaris cases too. * It's for free. * Each developer can debug ARM cases on their forked repository. * Customize easily. I see .travis.yml is used effectively. Cons * Have to manage additonal file .drone.yml or .drone.star. But first, I want to ask you. Are you interested in using Drone CI for Ruby project? -- https://bugs.ruby-lang.org/
next prev parent reply other threads:[~2019-11-08 18:09 UTC|newest] Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top [not found] <email@example.com> 2019-10-03 18:41 ` [ruby-core:95201] " jaruga 2019-10-03 19:20 ` [ruby-core:95203] " jaruga 2019-10-03 19:24 ` [ruby-core:95204] " takashikkbn 2019-10-03 21:14 ` [ruby-core:95206] " takashikkbn 2019-10-03 22:48 ` [ruby-core:95209] " jaruga 2019-10-03 23:56 ` [ruby-core:95212] " mame 2019-10-07 22:30 ` [ruby-core:95270] " jaruga 2019-10-07 22:41 ` [ruby-core:95271] " jaruga 2019-10-08 2:17 ` [ruby-core:95275] " takashikkbn 2019-10-08 11:25 ` [ruby-core:95277] " jaruga 2019-10-08 13:33 ` [ruby-core:95278] " takashikkbn 2019-10-08 14:56 ` [ruby-core:95279] " jaruga 2019-10-15 20:16 ` [ruby-core:95346] " jaruga 2019-11-04 14:31 ` [ruby-core:95667] " jaruga 2019-11-06 17:11 ` [ruby-core:95725] " jaruga 2019-11-06 19:09 ` [ruby-core:95726] " jaruga 2019-11-06 19:26 ` [ruby-core:95727] " jaruga 2019-11-06 22:07 ` [ruby-core:95729] " eregontp 2019-11-06 22:08 ` [ruby-core:95730] " eregontp 2019-11-06 22:12 ` [ruby-core:95731] " eregontp 2019-11-07 14:37 ` [ruby-core:95742] " jaruga 2019-11-07 20:43 ` [ruby-core:95748] " jaruga 2019-11-08 18:09 ` jaruga [this message] 2019-11-11 17:31 ` [ruby-core:95796] " jaruga 2019-11-12 18:05 ` [ruby-core:95818] " jaruga 2019-11-21 15:52 ` [ruby-core:95906] " jaruga
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-list from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://www.ruby-lang.org/en/community/mailing-lists/ * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [ruby-core:95763] [Ruby master Misc#16234] Enabling ARM 64/32-bit cases by Drone CI' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://80x24.org/mirrors/ruby.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).