Shared Code Changes for the RISC-V Glibc Port

Shared Code Changes for the RISC-V Glibc Port

[Palmer Dabbelt]

Sorry this took a while to get to, but things got a bit busy here when everyone
came back from vacation.  I believe I've managed to pull out all the shared
changes and addresses the feedback to them from our v3 submission.  I've
included ChangeLog entries for all of them, which I wouldn't usually do but I'm
doing here just to make sure they're OK -- I managed to screw the ChangeLog
entries up a handful of times in binutils and GCC, so I figure because it's a
slightly different set of people here it's worth sending them out as part of
the patches.

I've run these through build-many-glibcs.py, and while they're not 100% clean I
think the failures are just problems in my build environment.  I'm getting two
failures during the compilers stage, which trigger a bunch of UNRESOLVEDs
during the glibcs stage.  The m68k compiler fails with an ICE

    0x638eb5 emit_library_call_value_1
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/calls.c:4974
    0x63cfbe emit_library_call_value(rtx_def*, rtx_def*, libcall_type, machine_mode, int, ...)
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/calls.c:5168
    0x8ce358 expand_atomic_compare_and_swap(rtx_def**, rtx_def**, rtx_def*, rtx_def*, rtx_def*, bool, memmodel, memmodel)
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/optabs.c:6245
    0x627c64 expand_builtin_compare_and_swap
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/builtins.c:5527
    0x62f301 expand_builtin(tree_node*, rtx_def*, rtx_def*, machine_mode, int)
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/builtins.c:7093
    0x725c9c expand_expr_real_1(tree_node*, rtx_def*, machine_mode, expand_modifier, rtx_def**, bool)
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/expr.c:10850
    0x72e650 store_expr_with_bounds(tree_node*, rtx_def*, int, bool, bool, tree_node*)
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/expr.c:5569
    0x72f39f expand_assignment(tree_node*, tree_node*, bool)
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/expr.c:5338
    0x647b3a expand_call_stmt
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:2656
    0x647b3a expand_gimple_stmt_1
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:3571
    0x647b3a expand_gimple_stmt
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:3737
    0x649530 expand_gimple_basic_block
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:5744
    0x64db4e execute
            /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:6357
    Please submit a full bug report,
    with preprocessed source if appropriate.
    Please include the complete backtrace with any bug report.
    See <https://gcc.gnu.org/bugs/> for instructions.
    /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/libgcc/static-object.mk:17: recipe for target 'linux-atomic.o' failed
    make[3]: *** [linux-atomic.o] Error 1

and the sparc compiler fails with a #error that actually comes from a glibc
header file

    In file included from /scratch/palmer/work/20171220-glibc_upstream/topdir/install/compilers/sparc64-linux-gnu/sysroot/usr/include/bits/libio.h:520:0,
                     from /scratch/palmer/work/20171220-glibc_upstream/topdir/install/compilers/sparc64-linux-gnu/sysroot/usr/include/stdio.h:41,
                     from /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/libgcc/../gcc/tsystem.h:87,
                     from /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/libgcc/libgcc2.c:27:
    /scratch/palmer/work/20171220-glibc_upstream/topdir/install/compilers/sparc64-linux-gnu/sysroot/usr/include/bits/libio-ldbl.h:20:3: error: #error "Never include <bits/libio-ldbl.h> directly; use <libio.h>
     instead."
     # error "Never include <bits/libio-ldbl.h> directly; use <libio.h> instead."
       ^~~~~
    Makefile:491: recipe for target '_negdi2.o' failed
    make[5]: *** [_negdi2.o] Error 1

This SPARC one might actually be an glibc bug, but I can't figure out why it
would work for other people and not for me -- looking at stdio.h, it does
appear to directly include bits/libio.h

This is happening even without my patches, so I don't think they're related.
Here's the error log from build-many-glibcs.py:

    $ cat compilers-master.log | grep -v ^PASS
    FAIL: compilers-m68k-linux-gnu-coldfire gcc-first build
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc-first install
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire rm
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire mkdir
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire copy-rm
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire copy-mkdir
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire copy
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire configure
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire build
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire install
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire mkdir-lib
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc rm
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc mkdir
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc configure
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc build
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc install
    UNRESOLVED: compilers-m68k-linux-gnu-coldfire done
    FAIL: compilers-sparc64-linux-gnu gcc build
    UNRESOLVED: compilers-sparc64-linux-gnu gcc install
    UNRESOLVED: compilers-sparc64-linux-gnu done

If someone has a known-good environment in which build-many-glibcs.py I can try
to replicate it, but I don't think either of these patches could caused these
failures.  

These are running through build-many-glibcs.py again (with "--strip") just to
be sure, but assuming the failures above aren't a problem I'm OK to commit them
this weekend.

Sorry, again, this is all so last minute!

[PATCH 1/4] Add RISC-V dynamic relocations to elf.h
[PATCH 2/4] Allow make-link-multidir to make subdirectories
[PATCH 3/4] Add linux-4.15 VDSO hash for RISC-V
[PATCH 4/4] Strip shared objects in subdirectories of lib

[PATCH 1/4] Add RISC-V dynamic relocations to elf.h

[Palmer Dabbelt]

These relocations can appear in shared objects on RISC-V ELF systems.
---
 ChangeLog | 15 +++++++++++++++
 elf/elf.h | 14 ++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/ChangeLog b/ChangeLog
index 8833b1da335f..d0e02b0b1f43 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2018-01-05  Palmer Dabbelt  <palmer@sifive.com>
+
+	* elf/elf.h (R_RISCV_NONE): New define.
+	(R_RISCV_32): Likewise.
+	(R_RISCV_64): Likewise.
+	(R_RISCV_RELATIVE): Likewise.
+	(R_RISCV_COPY): Likewise.
+	(R_RISCV_JUMP_SLOT): Likewise.
+	(R_RISCV_TLS_DTPMOD32): Likewise.
+	(R_RISCV_TLS_DTPMOD64): Likewise.
+	(R_RISCV_TLS_DTPREL32): Likewise.
+	(R_RISCV_TLS_DTPREL64): Likewise.
+	(R_RISCV_TLS_TPREL32): Likewise.
+	(R_RISCV_TLS_TPREL64): Likewise.
+
 2018-01-06  Samuel Thibault  <samuel.thibault@ens-lyon.org>
 
 	* sysdeps/mach/hurd/i386/jmp_buf-macros.h: New file.
diff --git a/elf/elf.h b/elf/elf.h
index 01d794601085..031850377bb6 100644
--- a/elf/elf.h
+++ b/elf/elf.h
@@ -3762,6 +3762,20 @@ enum
 
 #define R_TILEGX_NUM		130
 
+/* RISC-V relocations.  */
+#define R_RISCV_NONE          0
+#define R_RISCV_32            1
+#define R_RISCV_64            2
+#define R_RISCV_RELATIVE      3
+#define R_RISCV_COPY          4
+#define R_RISCV_JUMP_SLOT     5
+#define R_RISCV_TLS_DTPMOD32  6
+#define R_RISCV_TLS_DTPMOD64  7
+#define R_RISCV_TLS_DTPREL32  8
+#define R_RISCV_TLS_DTPREL64  9
+#define R_RISCV_TLS_TPREL32  10
+#define R_RISCV_TLS_TPREL64  11
+
 /* BPF specific declarations.  */
 
 #define R_BPF_NONE		0	/* No reloc */
-- 
2.13.6

[PATCH 2/4] Allow make-link-multidir to make subdirectories

[Palmer Dabbelt]

The RISC-V Linux ABI doesn't define any libraries that go directly in
lib, instead they go into lib/ilp32 or lib/lp64.  This casuse
make-link-multidir to fail when attempting to make library directories
when building a static libc on multilib RISC-V systems.

This patch uses scripts/mkinstalldirs to make the base directory of the
target symlink of make-link-multidir.
---
 ChangeLog | 4 +++-
 Makerules | 1 +
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/ChangeLog b/ChangeLog
index d0e02b0b1f43..aeab82520a1f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,4 +1,4 @@
-2018-01-05  Palmer Dabbelt  <palmer@sifive.com>
+2018-01-06  Palmer Dabbelt  <palmer@sifive.com>
 
 	* elf/elf.h (R_RISCV_NONE): New define.
 	(R_RISCV_32): Likewise.
@@ -12,6 +12,8 @@
 	(R_RISCV_TLS_DTPREL64): Likewise.
 	(R_RISCV_TLS_TPREL32): Likewise.
 	(R_RISCV_TLS_TPREL64): Likewise.
+	* Makerules (make-link-multidir): Make directories before linking into
+	them.
 
 2018-01-06  Samuel Thibault  <samuel.thibault@ens-lyon.org>
 
diff --git a/Makerules b/Makerules
index d94e4ca0c18f..ef6abeac6d9d 100644
--- a/Makerules
+++ b/Makerules
@@ -1081,6 +1081,7 @@ mv -f $@.new $@
 endef
 define make-link-multidir
 $(patsubst %/,cd %,$(objpfx)); \
+  $(addprefix $(abspath $(..)scripts/mkinstalldirs) ,$(dir $(multidir))); \
   $(LN_S) . $(multidir) 2> /dev/null; \
   test -L $(multidir)
 endef
-- 
2.13.6

[PATCH 3/4] Add linux-4.15 VDSO hash for RISC-V

[Palmer Dabbelt]

The RISC-V Linux port defines VDSO symbols
---
 ChangeLog                         | 3 +++
 sysdeps/unix/sysv/linux/dl-vdso.h | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/ChangeLog b/ChangeLog
index aeab82520a1f..246d2326ab90 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -14,6 +14,9 @@
 	(R_RISCV_TLS_TPREL64): Likewise.
 	* Makerules (make-link-multidir): Make directories before linking into
 	them.
+	* sysdeps/unix/sysv/linux/dl-vdso.h (VDSO_NAME_LINUX_4_15): New
+	define.
+	(VDSO_HASH_LINUX_4_15): Likewise.
 
 2018-01-06  Samuel Thibault  <samuel.thibault@ens-lyon.org>
 
diff --git a/sysdeps/unix/sysv/linux/dl-vdso.h b/sysdeps/unix/sysv/linux/dl-vdso.h
index 913e4e027d74..7b668d0862e2 100644
--- a/sysdeps/unix/sysv/linux/dl-vdso.h
+++ b/sysdeps/unix/sysv/linux/dl-vdso.h
@@ -44,6 +44,8 @@
 #define VDSO_HASH_LINUX_2_6_15	123718565
 #define VDSO_NAME_LINUX_2_6_29	"LINUX_2.6.29"
 #define VDSO_HASH_LINUX_2_6_29	123718585
+#define VDSO_NAME_LINUX_4_15	"LINUX_4.15"
+#define VDSO_HASH_LINUX_4_15	182943605
 
 /* Functions for resolving symbols in the VDSO link map.  */
 extern void *_dl_vdso_vsym (const char *name,
-- 
2.13.6

[PATCH 4/4] Strip shared objects in subdirectories of lib

[Palmer Dabbelt]

The RISC-V port will have libraries in subdirectories of lib, like
"lib64/lp64d".  This adds support for stripping these installed
libraries.
---
 ChangeLog                    | 2 ++
 scripts/build-many-glibcs.py | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/ChangeLog b/ChangeLog
index 246d2326ab90..f9dd36033bc7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -17,6 +17,8 @@
 	* sysdeps/unix/sysv/linux/dl-vdso.h (VDSO_NAME_LINUX_4_15): New
 	define.
 	(VDSO_HASH_LINUX_4_15): Likewise.
+	* scripts/build-many-glibcs.py (class Glibc): Strip shared objects
+	in subdirectories of lib.
 
 2018-01-06  Samuel Thibault  <samuel.thibault@ens-lyon.org>
 
diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py
index f358307424a6..75a920a1611e 100755
--- a/scripts/build-many-glibcs.py
+++ b/scripts/build-many-glibcs.py
@@ -1377,7 +1377,7 @@ class Glibc(object):
             if self.ctx.strip:
                 cmdlist.add_command('strip',
                                     ['sh', '-c',
-                                     ('%s %s/lib*/*.so' %
+                                     ('%s $(find %s/lib* -name "*.so")' %
                                       (self.tool_name('strip'), installdir))])
             cmdlist.add_command('check', ['make', 'check'])
             cmdlist.add_command('save-logs', [self.ctx.save_logs],
-- 
2.13.6

Re: Shared Code Changes for the RISC-V Glibc Port

[Zack Weinberg]

On Sat, Jan 6, 2018 at 2:32 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
> Sorry this took a while to get to, but things got a bit busy here when everyone
> came back from vacation.  I believe I've managed to pull out all the shared
> changes and addresses the feedback to them from our v3 submission.  I've
> included ChangeLog entries for all of them, which I wouldn't usually do but I'm
> doing here just to make sure they're OK -- I managed to screw the ChangeLog
> entries up a handful of times in binutils and GCC, so I figure because it's a
> slightly different set of people here it's worth sending them out as part of
> the patches.

We (glibc reviewers) want to see ChangeLog entries with every patch
submission (as part of the commit message, *not* as modifications to
the actual ChangeLog file, which inevitably cause merge conflicts).

> I've run these through build-many-glibcs.py, and while they're not 100% clean I
> think the failures are just problems in my build environment.  I'm getting two
> failures during the compilers stage, which trigger a bunch of UNRESOLVEDs
> during the glibcs stage.  The m68k compiler fails with an ICE
>
>     0x638eb5 emit_library_call_value_1
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/calls.c:4974
>     0x63cfbe emit_library_call_value(rtx_def*, rtx_def*, libcall_type, machine_mode, int, ...)

You cut off the top of this error message, but if this is the
m68k-linux-gnu-coldfire configuration, it's a known and long-standing
bug in GCC which you are not expected to do anything about.  Just
ignore that configuration.  (I don't know why we still include it in
testing, frankly, it's been broken for more than a year now.)

The plain m68k-linux-gnu configuration, on the other hand, should work.

> and the sparc compiler fails with a #error that actually comes from a glibc
> header file
...
>      # error "Never include <bits/libio-ldbl.h> directly; use <libio.h> instead."

This was indeed an actual glibc bug, corrected by
4f820792a6217027744d38fc223257914847bbcc; update your source tree and
re-run the "compilers" phase of build-many-glibcs to get working SPARC
compilers.  (It "works for other people" because it only affects SPARC
configurations.)

The other expected unexpected failures from a build-many-glibcs run,
if you see what I mean, are "elf/check-exec-stack" failures for
hppa-linux-gnu, microblazeel-linux-gnu, and microblaze-linux-gnu.
This is again a GCC bug (or rather missing feature) and Joseph didn't
want us to mark them as expected failures for reasons which I no
longer remember.

zw

Re: [PATCH 2/4] Allow make-link-multidir to make subdirectories

[Zack Weinberg]

On Sat, Jan 6, 2018 at 2:32 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
> The RISC-V Linux ABI doesn't define any libraries that go directly in
> lib, instead they go into lib/ilp32 or lib/lp64.

Why did you invent a new convention for this, instead of using the
existing lib{32,64} or lib/<target-triple> conventions?

zw

Re: Shared Code Changes for the RISC-V Glibc Port

[Aurelien Jarno]

On 2018-01-06 10:02, Zack Weinberg wrote:
> On Sat, Jan 6, 2018 at 2:32 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
> > Sorry this took a while to get to, but things got a bit busy here when everyone
> > came back from vacation.  I believe I've managed to pull out all the shared
> > changes and addresses the feedback to them from our v3 submission.  I've
> > included ChangeLog entries for all of them, which I wouldn't usually do but I'm
> > doing here just to make sure they're OK -- I managed to screw the ChangeLog
> > entries up a handful of times in binutils and GCC, so I figure because it's a
> > slightly different set of people here it's worth sending them out as part of
> > the patches.
> 
> We (glibc reviewers) want to see ChangeLog entries with every patch
> submission (as part of the commit message, *not* as modifications to
> the actual ChangeLog file, which inevitably cause merge conflicts).

In practice the merge conflicts are handled automatically if
git-merge-changelog is installed on your system. That said it doesn't
prevent having the entries in both the ChangeLog file and the commit
message.

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: Shared Code Changes for the RISC-V Glibc Port

[Palmer Dabbelt]

On Sat, 06 Jan 2018 07:02:35 PST (-0800), zackw@panix.com wrote:
> On Sat, Jan 6, 2018 at 2:32 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> Sorry this took a while to get to, but things got a bit busy here when everyone
>> came back from vacation.  I believe I've managed to pull out all the shared
>> changes and addresses the feedback to them from our v3 submission.  I've
>> included ChangeLog entries for all of them, which I wouldn't usually do but I'm
>> doing here just to make sure they're OK -- I managed to screw the ChangeLog
>> entries up a handful of times in binutils and GCC, so I figure because it's a
>> slightly different set of people here it's worth sending them out as part of
>> the patches.
>
> We (glibc reviewers) want to see ChangeLog entries with every patch
> submission (as part of the commit message, *not* as modifications to
> the actual ChangeLog file, which inevitably cause merge conflicts).

OK, that makes sense.  That's how I've been doing it for binutils and GCC as 
well, so it's easy to remember.  I'll do that for all my patches from now on.

>> I've run these through build-many-glibcs.py, and while they're not 100% clean I
>> think the failures are just problems in my build environment.  I'm getting two
>> failures during the compilers stage, which trigger a bunch of UNRESOLVEDs
>> during the glibcs stage.  The m68k compiler fails with an ICE
>>
>>     0x638eb5 emit_library_call_value_1
>>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/calls.c:4974
>>     0x63cfbe emit_library_call_value(rtx_def*, rtx_def*, libcall_type, machine_mode, int, ...)
>
> You cut off the top of this error message, but if this is the
> m68k-linux-gnu-coldfire configuration, it's a known and long-standing
> bug in GCC which you are not expected to do anything about.  Just
> ignore that configuration.  (I don't know why we still include it in
> testing, frankly, it's been broken for more than a year now.)
>
> The plain m68k-linux-gnu configuration, on the other hand, should work.
>
>> and the sparc compiler fails with a #error that actually comes from a glibc
>> header file
> ...
>>      # error "Never include <bits/libio-ldbl.h> directly; use <libio.h> instead."
>
> This was indeed an actual glibc bug, corrected by
> 4f820792a6217027744d38fc223257914847bbcc; update your source tree and
> re-run the "compilers" phase of build-many-glibcs to get working SPARC
> compilers.  (It "works for other people" because it only affects SPARC
> configurations.)
>
> The other expected unexpected failures from a build-many-glibcs run,
> if you see what I mean, are "elf/check-exec-stack" failures for
> hppa-linux-gnu, microblazeel-linux-gnu, and microblaze-linux-gnu.
> This is again a GCC bug (or rather missing feature) and Joseph didn't
> want us to mark them as expected failures for reasons which I no
> longer remember.

Thanks!

Re: [PATCH 2/4] Allow make-link-multidir to make subdirectories

[Palmer Dabbelt]

On Sat, 06 Jan 2018 07:04:45 PST (-0800), zackw@panix.com wrote:
> On Sat, Jan 6, 2018 at 2:32 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> The RISC-V Linux ABI doesn't define any libraries that go directly in
>> lib, instead they go into lib/ilp32 or lib/lp64.
>
> Why did you invent a new convention for this, instead of using the
> existing lib{32,64} or lib/<target-triple> conventions?

Sorry, the message was a bit truncated.  We currently install libraries in the 
following directories:

* lib32/ilp32: RV32 (the 32-bit ISA), with the 32-bit soft float ABI
* lib32/ilp32d: RV32, with the 32-bit hard float ABI
* lib64/lp64: RV64 (the 64-bit ISA), with the 64-bit soft float ABI
* lib64/lp64d: RV64, 64-bit hard float

we plan on eventually also supporting lib64/ilp32 and lib64/ilp32d (x32-style 
ABIs).

The convention is slightly different because we wanted thing regular: we could 
have picked something like "lib32" and "lib32/softfloat" (IIRC ARM does this), 
but it seemed better to make thing orthogonal.

Re: Shared Code Changes for the RISC-V Glibc Port

[Palmer Dabbelt]

On Sat, 06 Jan 2018 09:54:26 PST (-0800), aurelien@aurel32.net wrote:
> On 2018-01-06 10:02, Zack Weinberg wrote:
>> On Sat, Jan 6, 2018 at 2:32 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> > Sorry this took a while to get to, but things got a bit busy here when everyone
>> > came back from vacation.  I believe I've managed to pull out all the shared
>> > changes and addresses the feedback to them from our v3 submission.  I've
>> > included ChangeLog entries for all of them, which I wouldn't usually do but I'm
>> > doing here just to make sure they're OK -- I managed to screw the ChangeLog
>> > entries up a handful of times in binutils and GCC, so I figure because it's a
>> > slightly different set of people here it's worth sending them out as part of
>> > the patches.
>>
>> We (glibc reviewers) want to see ChangeLog entries with every patch
>> submission (as part of the commit message, *not* as modifications to
>> the actual ChangeLog file, which inevitably cause merge conflicts).
>
> In practice the merge conflicts are handled automatically if
> git-merge-changelog is installed on your system. That said it doesn't
> prevent having the entries in both the ChangeLog file and the commit
> message.

I used that for a while in binutils land, but people seemed happier with the 
entries in the commit log.  I've gotten used to the commit log way, so I don't 
mind doing it (it's just an extra step).

Re: Shared Code Changes for the RISC-V Glibc Port

[Palmer Dabbelt]

On Fri, 05 Jan 2018 23:32:27 PST (-0800), Palmer Dabbelt wrote:
> Sorry this took a while to get to, but things got a bit busy here when everyone
> came back from vacation.  I believe I've managed to pull out all the shared
> changes and addresses the feedback to them from our v3 submission.  I've
> included ChangeLog entries for all of them, which I wouldn't usually do but I'm
> doing here just to make sure they're OK -- I managed to screw the ChangeLog
> entries up a handful of times in binutils and GCC, so I figure because it's a
> slightly different set of people here it's worth sending them out as part of
> the patches.
>
> I've run these through build-many-glibcs.py, and while they're not 100% clean I
> think the failures are just problems in my build environment.  I'm getting two
> failures during the compilers stage, which trigger a bunch of UNRESOLVEDs
> during the glibcs stage.  The m68k compiler fails with an ICE
>
>     0x638eb5 emit_library_call_value_1
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/calls.c:4974
>     0x63cfbe emit_library_call_value(rtx_def*, rtx_def*, libcall_type, machine_mode, int, ...)
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/calls.c:5168
>     0x8ce358 expand_atomic_compare_and_swap(rtx_def**, rtx_def**, rtx_def*, rtx_def*, rtx_def*, bool, memmodel, memmodel)
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/optabs.c:6245
>     0x627c64 expand_builtin_compare_and_swap
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/builtins.c:5527
>     0x62f301 expand_builtin(tree_node*, rtx_def*, rtx_def*, machine_mode, int)
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/builtins.c:7093
>     0x725c9c expand_expr_real_1(tree_node*, rtx_def*, machine_mode, expand_modifier, rtx_def**, bool)
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/expr.c:10850
>     0x72e650 store_expr_with_bounds(tree_node*, rtx_def*, int, bool, bool, tree_node*)
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/expr.c:5569
>     0x72f39f expand_assignment(tree_node*, tree_node*, bool)
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/expr.c:5338
>     0x647b3a expand_call_stmt
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:2656
>     0x647b3a expand_gimple_stmt_1
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:3571
>     0x647b3a expand_gimple_stmt
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:3737
>     0x649530 expand_gimple_basic_block
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:5744
>     0x64db4e execute
>             /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/gcc/cfgexpand.c:6357
>     Please submit a full bug report,
>     with preprocessed source if appropriate.
>     Please include the complete backtrace with any bug report.
>     See <https://gcc.gnu.org/bugs/> for instructions.
>     /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/libgcc/static-object.mk:17: recipe for target 'linux-atomic.o' failed
>     make[3]: *** [linux-atomic.o] Error 1
>
> and the sparc compiler fails with a #error that actually comes from a glibc
> header file
>
>     In file included from /scratch/palmer/work/20171220-glibc_upstream/topdir/install/compilers/sparc64-linux-gnu/sysroot/usr/include/bits/libio.h:520:0,
>                      from /scratch/palmer/work/20171220-glibc_upstream/topdir/install/compilers/sparc64-linux-gnu/sysroot/usr/include/stdio.h:41,
>                      from /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/libgcc/../gcc/tsystem.h:87,
>                      from /scratch/palmer/work/20171220-glibc_upstream/topdir/src/gcc/libgcc/libgcc2.c:27:
>     /scratch/palmer/work/20171220-glibc_upstream/topdir/install/compilers/sparc64-linux-gnu/sysroot/usr/include/bits/libio-ldbl.h:20:3: error: #error "Never include <bits/libio-ldbl.h> directly; use <libio.h>
>      instead."
>      # error "Never include <bits/libio-ldbl.h> directly; use <libio.h> instead."
>        ^~~~~
>     Makefile:491: recipe for target '_negdi2.o' failed
>     make[5]: *** [_negdi2.o] Error 1
>
> This SPARC one might actually be an glibc bug, but I can't figure out why it
> would work for other people and not for me -- looking at stdio.h, it does
> appear to directly include bits/libio.h
>
> This is happening even without my patches, so I don't think they're related.
> Here's the error log from build-many-glibcs.py:
>
>     $ cat compilers-master.log | grep -v ^PASS
>     FAIL: compilers-m68k-linux-gnu-coldfire gcc-first build
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc-first install
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire rm
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire mkdir
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire copy-rm
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire copy-mkdir
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire copy
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire configure
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire build
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire install
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire glibc m68k-linux-gnu-coldfire mkdir-lib
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc rm
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc mkdir
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc configure
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc build
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire gcc install
>     UNRESOLVED: compilers-m68k-linux-gnu-coldfire done
>     FAIL: compilers-sparc64-linux-gnu gcc build
>     UNRESOLVED: compilers-sparc64-linux-gnu gcc install
>     UNRESOLVED: compilers-sparc64-linux-gnu done
>
> If someone has a known-good environment in which build-many-glibcs.py I can try
> to replicate it, but I don't think either of these patches could caused these
> failures.
>
> These are running through build-many-glibcs.py again (with "--strip") just to
> be sure, but assuming the failures above aren't a problem I'm OK to commit them
> this weekend.
>
> Sorry, again, this is all so last minute!
>
> [PATCH 1/4] Add RISC-V dynamic relocations to elf.h
> [PATCH 2/4] Allow make-link-multidir to make subdirectories
> [PATCH 3/4] Add linux-4.15 VDSO hash for RISC-V
> [PATCH 4/4] Strip shared objects in subdirectories of lib

I went ahead and committed these.  I'm still getting some failures related to 
getrlimit64 after my rebase, but it looks like things are changing there right 
now and I don't think I could have broken that.

Darius has been going through lots of the v3 feedback, and while I'm a bit 
behind I think we should be able to get through all of it soon.

Thanks for all the help!

Re: Shared Code Changes for the RISC-V Glibc Port

[Joseph Myers]

On Fri, 5 Jan 2018, Palmer Dabbelt wrote:

> I've run these through build-many-glibcs.py, and while they're not 100% clean I
> think the failures are just problems in my build environment.  I'm getting two

I should reiterate that the vast bulk of changes do not need a full 
build-many-glibcs.py run.  Just because something is changing 
system-independent code does not make it require such a run - such a run 
rather tends to be indicated if it's changing code in a way that looks 
likely to have system-specific issues (and even in that case, a few 
particular configurations may be sufficient to test).

Of course each version of the RISC-V port submission should be verified to 
pass build-many-glibcs.py *for RISC-V configurations added to 
build-many-glibcs.py* (given suitable kernel sources used), but you can 
name particular configurations to test on the build-many-glibcs.py command 
line rather than testing the full set.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Shared Code Changes for the RISC-V Glibc Port

[Joseph Myers]

On Sat, 6 Jan 2018, Zack Weinberg wrote:

> You cut off the top of this error message, but if this is the
> m68k-linux-gnu-coldfire configuration, it's a known and long-standing
> bug in GCC which you are not expected to do anything about.  Just
> ignore that configuration.  (I don't know why we still include it in
> testing, frankly, it's been broken for more than a year now.)

(This is <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68467>.)

build-many-glibcs.py is supposed to test, at least, every ABI supported in 
glibc - meaning it's correct for it to show failures when such a 
configuration is broken.  Similarly, it would be appropriate for a Hurd 
configuration to be added to build-many-glibcs.py, and for it to fail 
building until all the required support is upstreamed.

> The other expected unexpected failures from a build-many-glibcs run,
> if you see what I mean, are "elf/check-exec-stack" failures for
> hppa-linux-gnu, microblazeel-linux-gnu, and microblaze-linux-gnu.
> This is again a GCC bug (or rather missing feature) and Joseph didn't
> want us to mark them as expected failures for reasons which I no
> longer remember.

The correct handling of a failure should be determined case-by-case.  In 
these cases, I fixed those failures for GCC 8.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Shared Code Changes for the RISC-V Glibc Port

[Zack Weinberg]

On Tue, Jan 9, 2018 at 11:16 AM, Joseph Myers <joseph@codesourcery.com> wrote:
> On Sat, 6 Jan 2018, Zack Weinberg wrote:
>
>> You cut off the top of this error message, but if this is the
>> m68k-linux-gnu-coldfire configuration, it's a known and long-standing
>> bug in GCC which you are not expected to do anything about.  Just
>> ignore that configuration.  (I don't know why we still include it in
>> testing, frankly, it's been broken for more than a year now.)
>
> (This is <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68467>.)
>
> build-many-glibcs.py is supposed to test, at least, every ABI supported in
> glibc - meaning it's correct for it to show failures when such a
> configuration is broken.

In view of the complete lack of progress on that bug for over a year,
I think it is inappropriate to describe the -coldfire configuration as
"supported" anymore; my recommendation would be to disable it and to
add the Coldfire-specific code in glibc to a deathwatch list, such
that if there is still no progress after, I dunno, another release or
two, it will be removed.

>  Similarly, it would be appropriate for a Hurd
> configuration to be added to build-many-glibcs.py, and for it to fail
> building until all the required support is upstreamed.

I would rather see the Hurd configuration added to build-many-glibcs
as the final step in upstreaming the required support.

>> The other expected unexpected failures from a build-many-glibcs run,
>> if you see what I mean, are "elf/check-exec-stack" failures for
>> hppa-linux-gnu, microblazeel-linux-gnu, and microblaze-linux-gnu.
>> This is again a GCC bug (or rather missing feature) and Joseph didn't
>> want us to mark them as expected failures for reasons which I no
>> longer remember.
>
> The correct handling of a failure should be determined case-by-case.  In
> these cases, I fixed those failures for GCC 8.

Then they should be tagged either UNSUPPORTED or XFAIL for GCC 7 and
below.  It doesn't do anyone any good to have to carry around the fact
that this is an expected failure (with older toolchains) in their
head.

zw

Re: [PATCH 2/4] Allow make-link-multidir to make subdirectories

[Joseph Myers]

On Fri, 5 Jan 2018, Palmer Dabbelt wrote:

> diff --git a/Makerules b/Makerules
> index d94e4ca0c18f..ef6abeac6d9d 100644
> --- a/Makerules
> +++ b/Makerules
> @@ -1081,6 +1081,7 @@ mv -f $@.new $@
>  endef
>  define make-link-multidir
>  $(patsubst %/,cd %,$(objpfx)); \
> +  $(addprefix $(abspath $(..)scripts/mkinstalldirs) ,$(dir $(multidir))); \
>    $(LN_S) . $(multidir) 2> /dev/null; \
>    test -L $(multidir)
>  endef

This doesn't actually achieve the desired effect.

That is, it stops the build from failing.  But the point of HJ's commit 
abcb584d0eae7270b35e1b3fed1f9661e26b8be0 was that multidir ends up as a 
symlink pointing to the csu object directory containing crt*.o.  When 
there are multiple subdirectory levels involved, you're still creating a 
symlink to ".", which is just the subdirectory one level up; you need a 
symlink that actually points to the csu object directory, however many 
levels up it might be.  So presumably the "wrong" (previously built) 
crt*.o could get used in some cases for building shared libraries.

(I think the case of no ln -s support gets this right, though I haven't 
tested that case.)

-- 
Joseph S. Myers
joseph@codesourcery.com