Re: 32bit or LTO movemail build fails on macOS

Re: 32bit or LTO movemail build fails on macOS

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]

Thanks for the bug report. Does the attached patch to getopt-pfx-core.h fix the 
bug for you?

I'll CC: this to bug-gnulib and to Zack Weinberg, who wrote that file. For those 
new to this thread, please see <https://bugs.gnu.org/40205>.

[-- Attachment #2: getopt-apple.diff --]
[-- Type: text/x-patch, Size: 771 bytes --]

diff --git a/lib/getopt-pfx-core.h b/lib/getopt-pfx-core.h
index 31a747d67..a4891bc80 100644
--- a/lib/getopt-pfx-core.h
+++ b/lib/getopt-pfx-core.h
@@ -48,6 +48,14 @@
 # define optind __GETOPT_ID (optind)
 # define optopt __GETOPT_ID (optopt)
 
+/* Work around a a problem on macOS, which declares getopt with a
+   trailing __DARWIN_ALIAS(getopt) that would expand to something like
+   __asm("_" "rpl_getopt" "$UNIX2003") were it not for the following
+   hack to suppress the macOS declaration <https://bugs.gnu.org/40205>.  */
+# ifdef __APPLE__
+#  define _GETOPT
+# endif
+
 /* The system's getopt.h may have already included getopt-core.h to
    declare the unprefixed identifiers.  Undef _GETOPT_CORE_H so that
    getopt-core.h declares them with prefixes.  */

Re: 32bit or LTO movemail build fails on macOS

[YAMAMOTO Mitsuharu]

On Wed, 25 Mar 2020 11:58:19 +0900,
Paul Eggert wrote:
> 
> Thanks for the bug report. Does the attached patch to
> getopt-pfx-core.h fix the bug for you?

Yes, this works for both -arch i386 and -flto cases.  Thanks.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: 32bit or LTO movemail build fails on macOS

[Jeffrey Walton]

On Tue, Mar 24, 2020 at 10:58 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> Thanks for the bug report. Does the attached patch to getopt-pfx-core.h fix the
> bug for you?
>
> I'll CC: this to bug-gnulib and to Zack Weinberg, who wrote that file. For those
> new to this thread, please see <https://bugs.gnu.org/40205>.

CI tests should be catching these mistakes. (And problems like
_NoReturn on OS X).

Is there any reasons services like Travis or Cirrus are not being used
to proactively detect problems on Linux, OS X and FreeBSD?

Daniel Stenberg has one of the best engineering processes in place
(that I have observed in the wild). You can find the curl testing
configurations in curl's GitHub at https://github.com/curl/curl. The
files of interest include .cirrus.yml, .travis.yml, and .lgtm.yml.

Jeff

Re: 32bit or LTO movemail build fails on macOS

[Zack Weinberg]

On Tue, Mar 24, 2020 at 10:58 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> Thanks for the bug report. Does the attached patch to getopt-pfx-core.h fix the
> bug for you?
>
> I'll CC: this to bug-gnulib and to Zack Weinberg, who wrote that file. For those
> new to this thread, please see <https://bugs.gnu.org/40205>.

Patch seems plausible enough to me.  I didn't think very hard about
the ways a system declaration of getopt could interfere when I wrote
this file.

zw

Re: 32bit or LTO movemail build fails on macOS

[Paul Eggert]

On 3/25/20 3:14 AM, Jeffrey Walton wrote:
> Is there any reasons services like Travis or Cirrus are not being used
> to proactively detect problems on Linux, OS X and FreeBSD?

Not enough resources, I assume. I'm a volunteer and lack time to mess with macOS 
and FreeBSD, much less all the GNU/Linux versions out there (plus all the 
programs that use Gnulib...).

Re: 32bit or LTO movemail build fails on macOS

[Jeffrey Walton]

On Wed, Mar 25, 2020 at 3:10 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
>
> On 3/25/20 3:14 AM, Jeffrey Walton wrote:
> > Is there any reasons services like Travis or Cirrus are not being used
> > to proactively detect problems on Linux, OS X and FreeBSD?
>
> Not enough resources, I assume. I'm a volunteer and lack time to mess with macOS
> and FreeBSD, much less all the GNU/Linux versions out there (plus all the
> programs that use Gnulib...).

CI is a force multiplier. Setting up CI tests now will save you
exponentially in the future. It is one of the best investments you can
make with your time.

Here's some additional projects that make heavy use of CI to detect
problems before users experience them. Crypto++, LDNS and Unbound even
perform Android and iOS cross-compiles to ensure they work for users,
too.

* https://github.com/randombit/botan
* https://github.com/weidai11/cryptopp
* https://github.com/NLnetLabs/unbound
* https://github.com/NLnetLabs/ldns
* https://github.com/NLnetLabs/nsd

I'm happy to help. This is the kind of monkey work a senior engineer
should offload.

What do you want, and what do you need?

Jeff

Re: 32bit or LTO movemail build fails on macOS

[Paul Eggert]

On 3/25/20 1:26 PM, Jeffrey Walton wrote:
> I'm happy to help. This is the kind of monkey work a senior engineer
> should offload.
> 
> What do you want, and what do you need?

Ideally, someone would take on the project of doing CI for Gnulib and/or some of 
its downstream projects, to maintain the corresponding .yml (or whatever) files 
that would be maintained as part of Gnulib etc., and to make sure that this is 
all done with free software. This should require minimal work by the current 
volunteers and should be easily partitionable.

Re: 32bit or LTO movemail build fails on macOS

[Paul Eggert]

On 3/24/20 9:07 PM, YAMAMOTO Mitsuharu wrote:

> Yes, this works for both -arch i386 and -flto cases.

Thanks for checking. I installed the patch into Gnulib and propagated the fix 
into Emacs master, and am marking this Emacs bug as done.

Re: continuous integration

[Bruno Haible]

Jeffrey Walton wrote:
> CI tests should be catching these mistakes. (And problems like
> _NoReturn on OS X).

Yes, CI can catch some mistakes. Like, just last week, this one: [1].

Tim and I maintain a continuous integration for gnulib at [2].

More effort could be put in, in two directions:

* Like Paul says, instead of only building testdirs, it could build
  some packages that use gnulib. I would estimate that this would catch
  3x as many bugs as the current CI with just testdirs.

* Like you suggest, it would also be useful to test macOS, FreeBSD,
  Cygwin, and mingw builds.

> Is there any reasons services like Travis or Cirrus are not being used
> to proactively detect problems on Linux, OS X and FreeBSD?

For my part:

* I have only limited time to work on this; that's why I limit
  myself to CI integrations for a couple of packages on gitlab.

* I had not heard of Cirrus CI. Coverage of FreeBSD, additionally to
  Windows and macOS, sounds interesting. [3]

* Travis and Cirrus CI are most easily used on Github [4][5]. I don't
  much like to work on Github, because it tends to become a closed
  environment. E.g.
    - You can fork someone else's repository only if you stay on Github.
    - Many developers' email addresses are not published, which prevents
      you from reporting issues by email. You have to use Github "issues"
      instead.

But if someone wants to set it up and maintain it, I'm all for it!

Bruno

[1] https://lists.gnu.org/archive/html/bug-gnulib/2020-03/msg00041.html
[2] https://gitlab.com/gnulib/gnulib-ci
[3] https://cirrus-ci.org/features/#comparison-with-popular-ciaas
[4] https://en.wikipedia.org/wiki/Travis_CI
[5] https://cirrus-ci.org/faq/#only-github-support

Re: continuous integration

[Tim Rühsen]

[-- Attachment #1.1: Type: text/plain, Size: 2824 bytes --]



On 26.03.20 00:46, Bruno Haible wrote:
> Jeffrey Walton wrote:
>> CI tests should be catching these mistakes. (And problems like
>> _NoReturn on OS X).
> 
> Yes, CI can catch some mistakes. Like, just last week, this one: [1].
> 
> Tim and I maintain a continuous integration for gnulib at [2].
> 
> More effort could be put in, in two directions:
> 
> * Like Paul says, instead of only building testdirs, it could build
>   some packages that use gnulib. I would estimate that this would catch
>   3x as many bugs as the current CI with just testdirs.
> 
> * Like you suggest, it would also be useful to test macOS, FreeBSD,
>   Cygwin, and mingw builds.
> 
>> Is there any reasons services like Travis or Cirrus are not being used
>> to proactively detect problems on Linux, OS X and FreeBSD?
> 
> For my part:
> 
> * I have only limited time to work on this; that's why I limit
>   myself to CI integrations for a couple of packages on gitlab.

Same here. I really wish we could had more time to put into CI runners.

I would like to point out that debugging using a CI like Travis is
absolutely tedious and might take a lot of time. Docker-based CI (Linux
only :-| ) are so much easier to debug as you can run the test
environment (images + build scripts) locally.

So while some errors are obvious and easy to fix, others are a nightmare
as you can't 'log in' and just use a debugger. At least I don't have VMs
with OSX or Windows for this purpose.

Did anyone think about using the gcc build platform for automated
testing ? I made up some scripts a while ago for Wget but then lost
focus... if someone likes to take that up.

> * I had not heard of Cirrus CI. Coverage of FreeBSD, additionally to
>   Windows and macOS, sounds interesting. [3]
> 
> * Travis and Cirrus CI are most easily used on Github [4][5]. I don't
>   much like to work on Github, because it tends to become a closed
>   environment. E.g.
>     - You can fork someone else's repository only if you stay on Github.
>     - Many developers' email addresses are not published, which prevents
>       you from reporting issues by email. You have to use Github "issues"
>       instead.

We just need a mirror / fork on Github that we push to (sync) from time
to time. If someone cares for the initial Travis and/or Cirrus setup
with OSX / FreeBSD / Windows in mind, that would be great !

> But if someone wants to set it up and maintain it, I'm all for it!
> 
> Bruno
> 
> [1] https://lists.gnu.org/archive/html/bug-gnulib/2020-03/msg00041.html
> [2] https://gitlab.com/gnulib/gnulib-ci
> [3] https://cirrus-ci.org/features/#comparison-with-popular-ciaas
> [4] https://en.wikipedia.org/wiki/Travis_CI
> [5] https://cirrus-ci.org/faq/#only-github-support
> 
> 

Regards, Tim


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: continuous integration

[Ed Maste]

On Thu, 26 Mar 2020 at 05:51, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
>
> We just need a mirror / fork on Github that we push to (sync) from time
> to time. If someone cares for the initial Travis and/or Cirrus setup
> with OSX / FreeBSD / Windows in mind, that would be great !

If such a fork/mirror is going to show up I'll add a .cirrus.yml to
get at least FreeBSD coverage (and will add others if I manage to get
them going easily).

Re: continuous integration

[Tim Rühsen]

[-- Attachment #1.1: Type: text/plain, Size: 992 bytes --]

Hi Ed,

On 11.06.20 21:23, Ed Maste wrote:
> On Thu, 26 Mar 2020 at 05:51, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
>>
>> We just need a mirror / fork on Github that we push to (sync) from time
>> to time. If someone cares for the initial Travis and/or Cirrus setup
>> with OSX / FreeBSD / Windows in mind, that would be great !
> 
> If such a fork/mirror is going to show up I'll add a .cirrus.yml to
> get at least FreeBSD coverage (and will add others if I manage to get
> them going easily).

You can mirror gnulib on your own Github account to add/test Cirrus CI.
I am not sure that gnulib will have an official account on Github, as
Github is very proprietary.

Another option is to get access to a FreeBSD machine and install the
Gitlab runner [1]. With that (plus a token from our Gitlab account), the
machine should appear on our list of CI runners and we can integrate it
into the .gitlab-ci.yml.

Regards, Tim

[1] https://docs.gitlab.com/runner/install/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]