* [PATCH v10 0/4] Extend rseq support
@ 2024-03-25 18:29 Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 1/4] nptl: fix potential merge of __rseq_* relro symbols Michael Jeanson
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Michael Jeanson @ 2024-03-25 18:29 UTC (permalink / raw)
To: libc-alpha
Cc: Michael Jeanson, Florian Weimer, Carlos O'Donell, DJ Delorie,
Mathieu Desnoyers
This series rebases the standalone "Add rseq extensible ABI" patch on
current master and addresses many review comments documented in each
patch commit message.
The accelerated getcpu() implementation using the rseq extensible ABI
was split in a different patch series while we figure out the best way
to interface with the librseq project as an upstream.
Cc: Florian Weimer <fweimer@redhat.com>
Cc: Carlos O'Donell <carlos@redhat.com>
Cc: DJ Delorie <dj@redhat.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
Changes since v9:
- Rebased on current master
Changes since v8:
- Address review comments
- Split the accelerated getcpu() implementation in a separate patch set
Changes since v7:
- Fix sorting of symbols in abilist files
Michael Jeanson (4):
nptl: fix potential merge of __rseq_* relro symbols
Add rseq extensible ABI support
nptl: Add public __rseq_feature_size symbol
nptl: Add features to internal 'struct rseq_area'
csu/Makefile | 2 +-
csu/libc-tls.c | 84 +++++++++++++++++--
csu/rseq-sizes.sym | 11 +++
elf/Makefile | 1 +
elf/dl-rseq-symbols.S | 72 ++++++++++++++++
elf/dl-tls.c | 78 +++++++++++++++++
elf/rtld_static_init.c | 12 +++
manual/threads.texi | 9 ++
nptl/descr.h | 20 +----
nptl/pthread_create.c | 2 +-
sysdeps/generic/dl-rseq.h | 26 ++++++
sysdeps/generic/ldsodefs.h | 12 +++
sysdeps/i386/nptl/tcb-access.h | 56 +++++++++++++
sysdeps/nptl/dl-tls_init_tp.c | 22 +++--
sysdeps/nptl/tcb-access.h | 5 ++
sysdeps/unix/sysv/linux/Makefile | 10 +++
sysdeps/unix/sysv/linux/Versions | 3 +
sysdeps/unix/sysv/linux/aarch64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/alpha/ld.abilist | 1 +
sysdeps/unix/sysv/linux/arc/ld.abilist | 1 +
sysdeps/unix/sysv/linux/arm/be/ld.abilist | 1 +
sysdeps/unix/sysv/linux/arm/le/ld.abilist | 1 +
sysdeps/unix/sysv/linux/csky/ld.abilist | 1 +
sysdeps/unix/sysv/linux/dl-parse_auxv.h | 6 ++
sysdeps/unix/sysv/linux/hppa/ld.abilist | 1 +
sysdeps/unix/sysv/linux/i386/ld.abilist | 1 +
.../unix/sysv/linux/loongarch/lp64/ld.abilist | 1 +
.../unix/sysv/linux/m68k/coldfire/ld.abilist | 1 +
.../unix/sysv/linux/m68k/m680x0/ld.abilist | 1 +
sysdeps/unix/sysv/linux/microblaze/ld.abilist | 1 +
.../unix/sysv/linux/mips/mips32/ld.abilist | 1 +
.../sysv/linux/mips/mips64/n32/ld.abilist | 1 +
.../sysv/linux/mips/mips64/n64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/nios2/ld.abilist | 1 +
sysdeps/unix/sysv/linux/or1k/ld.abilist | 1 +
.../sysv/linux/powerpc/powerpc32/ld.abilist | 1 +
.../linux/powerpc/powerpc64/be/ld.abilist | 1 +
.../linux/powerpc/powerpc64/le/ld.abilist | 1 +
sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist | 1 +
sysdeps/unix/sysv/linux/riscv/rv64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/rseq-internal.h | 33 +++++++-
.../unix/sysv/linux/s390/s390-32/ld.abilist | 1 +
.../unix/sysv/linux/s390/s390-64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/sched_getcpu.c | 3 +-
sysdeps/unix/sysv/linux/sh/be/ld.abilist | 1 +
sysdeps/unix/sysv/linux/sh/le/ld.abilist | 1 +
.../unix/sysv/linux/sparc/sparc32/ld.abilist | 1 +
.../unix/sysv/linux/sparc/sparc64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/sys/rseq.h | 4 +
.../unix/sysv/linux/tst-rseq-disable-static.c | 1 +
sysdeps/unix/sysv/linux/tst-rseq-disable.c | 18 ++--
.../unix/sysv/linux/tst-rseq-nptl-static.c | 1 +
sysdeps/unix/sysv/linux/tst-rseq-static.c | 1 +
sysdeps/unix/sysv/linux/tst-rseq.c | 24 +++++-
sysdeps/unix/sysv/linux/tst-rseq.h | 9 +-
sysdeps/unix/sysv/linux/x86_64/64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist | 1 +
sysdeps/x86_64/nptl/tcb-access.h | 56 +++++++++++++
58 files changed, 557 insertions(+), 54 deletions(-)
create mode 100644 csu/rseq-sizes.sym
create mode 100644 elf/dl-rseq-symbols.S
create mode 100644 sysdeps/generic/dl-rseq.h
create mode 100644 sysdeps/unix/sysv/linux/tst-rseq-disable-static.c
create mode 100644 sysdeps/unix/sysv/linux/tst-rseq-nptl-static.c
create mode 100644 sysdeps/unix/sysv/linux/tst-rseq-static.c
--
2.34.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v10 1/4] nptl: fix potential merge of __rseq_* relro symbols
2024-03-25 18:29 [PATCH v10 0/4] Extend rseq support Michael Jeanson
@ 2024-03-25 18:29 ` Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 2/4] Add rseq extensible ABI support Michael Jeanson
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Michael Jeanson @ 2024-03-25 18:29 UTC (permalink / raw)
To: libc-alpha
Cc: Michael Jeanson, Florian Weimer, Carlos O'Donell, DJ Delorie,
Mathieu Desnoyers
While working on a patch to add support for the extensible rseq ABI, we
came across an issue where a new 'const' variable would be merged with
the existing '__rseq_size' variable. We tracked this to the use of
'-fmerge-all-constants' which allows the compiler to merge identical
constant variables. This means that all 'const' variables in a compile
unit that are of the same size and are initialized to the same value can
be merged.
In this specific case, on 32 bit systems 'unsigned int' and 'ptrdiff_t'
are both 4 bytes and initialized to 0 which should trigger the merge.
However for reasons we haven't delved into when the attribute 'section
(".data.rel.ro")' is added to the mix, only variables of the same exact
types are merged. As far as we know this behavior is not specified
anywhere and could change with a new compiler version, hence this patch.
Move the definitions of these variables into an assembler file and add
hidden writable aliases for internal use. This has the added bonus of
removing the asm workaround to set the values on rseq registration.
Tested on Debian 12 with GCC 12.2.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
Changes sinve v8:
- Remove superfluous attributes on externs
---
csu/Makefile | 2 +-
csu/rseq-sizes.sym | 8 +++++
elf/Makefile | 1 +
elf/dl-rseq-symbols.S | 55 +++++++++++++++++++++++++++++++++++
sysdeps/nptl/dl-tls_init_tp.c | 14 ++++-----
5 files changed, 71 insertions(+), 9 deletions(-)
create mode 100644 csu/rseq-sizes.sym
create mode 100644 elf/dl-rseq-symbols.S
diff --git a/csu/Makefile b/csu/Makefile
index 946fd91031..948b1aa79e 100644
--- a/csu/Makefile
+++ b/csu/Makefile
@@ -136,7 +136,7 @@ before-compile += $(objpfx)abi-tag.h
generated += abi-tag.h
# Put it here to generate it earlier.
-gen-as-const-headers += rtld-sizes.sym
+gen-as-const-headers += rtld-sizes.sym rseq-sizes.sym
# These are the special initializer/finalizer files. They are always the
# first and last file in the link. crti.o ... crtn.o define the global
diff --git a/csu/rseq-sizes.sym b/csu/rseq-sizes.sym
new file mode 100644
index 0000000000..c959758ff0
--- /dev/null
+++ b/csu/rseq-sizes.sym
@@ -0,0 +1,8 @@
+#include <stddef.h>
+
+--
+RSEQ_SIZE_SIZE sizeof (unsigned int)
+RSEQ_SIZE_ALIGN __alignof (unsigned int)
+
+RSEQ_OFFSET_SIZE sizeof (ptrdiff_t)
+RSEQ_OFFSET_ALIGN __alignof (ptrdiff_t)
diff --git a/elf/Makefile b/elf/Makefile
index 4f1903391a..1019e3e2dd 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -73,6 +73,7 @@ dl-routines = \
dl-origin \
dl-printf \
dl-reloc \
+ dl-rseq-symbols \
dl-runtime \
dl-scope \
dl-setup_hash \
diff --git a/elf/dl-rseq-symbols.S b/elf/dl-rseq-symbols.S
new file mode 100644
index 0000000000..2d8e88367f
--- /dev/null
+++ b/elf/dl-rseq-symbols.S
@@ -0,0 +1,55 @@
+/* Define symbols used by rseq.
+ Copyright (C) 2024 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#include <rseq-sizes.h>
+#include <sysdep.h>
+
+/* Some targets define a macro to denote the zero register. */
+#undef zero
+
+/* Define 2 symbols, __rseq_size is public const and _rseq_size, which is an
+ alias of __rseq_size, but hidden and writable for internal use. */
+
+ .globl __rseq_size
+ .type __rseq_size, %object
+ .size __rseq_size, RSEQ_SIZE_SIZE
+ .hidden _rseq_size
+ .globl _rseq_size
+ .type _rseq_size, %object
+ .size _rseq_size, RSEQ_SIZE_SIZE
+ .section .data.rel.ro
+ .balign RSEQ_SIZE_ALIGN
+__rseq_size:
+_rseq_size:
+ .zero RSEQ_SIZE_SIZE
+
+/* Define 2 symbols, __rseq_offset is public const and _rseq_offset, which is an
+ alias of __rseq_offset, but hidden and writable for internal use. */
+
+ .globl __rseq_offset
+ .type __rseq_offset, %object
+ .size __rseq_offset, RSEQ_OFFSET_SIZE
+ .hidden _rseq_offset
+ .globl _rseq_offset
+ .type _rseq_offset, %object
+ .size _rseq_offset, RSEQ_OFFSET_SIZE
+ .section .data.rel.ro
+ .balign RSEQ_OFFSET_ALIGN
+__rseq_offset:
+_rseq_offset:
+ .zero RSEQ_OFFSET_SIZE
diff --git a/sysdeps/nptl/dl-tls_init_tp.c b/sysdeps/nptl/dl-tls_init_tp.c
index 092c274f36..2f9750c50b 100644
--- a/sysdeps/nptl/dl-tls_init_tp.c
+++ b/sysdeps/nptl/dl-tls_init_tp.c
@@ -45,8 +45,10 @@ rtld_mutex_dummy (pthread_mutex_t *lock)
#endif
const unsigned int __rseq_flags;
-const unsigned int __rseq_size attribute_relro;
-const ptrdiff_t __rseq_offset attribute_relro;
+
+/* The variables are in .data.relro but are not yet write-protected. */
+extern unsigned int _rseq_size;
+extern ptrdiff_t _rseq_offset;
void
__tls_pre_init_tp (void)
@@ -105,10 +107,7 @@ __tls_init_tp (void)
do_rseq = TUNABLE_GET (rseq, int, NULL);
if (rseq_register_current_thread (pd, do_rseq))
{
- /* We need a writable view of the variables. They are in
- .data.relro and are not yet write-protected. */
- extern unsigned int size __asm__ ("__rseq_size");
- size = sizeof (pd->rseq_area);
+ _rseq_size = sizeof (pd->rseq_area);
}
#ifdef RSEQ_SIG
@@ -117,8 +116,7 @@ __tls_init_tp (void)
all targets support __thread_pointer, so set __rseq_offset only
if the rseq registration may have happened because RSEQ_SIG is
defined. */
- extern ptrdiff_t offset __asm__ ("__rseq_offset");
- offset = (char *) &pd->rseq_area - (char *) __thread_pointer ();
+ _rseq_offset = (char *) &pd->rseq_area - (char *) __thread_pointer ();
#endif
}
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH v10 2/4] Add rseq extensible ABI support
2024-03-25 18:29 [PATCH v10 0/4] Extend rseq support Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 1/4] nptl: fix potential merge of __rseq_* relro symbols Michael Jeanson
@ 2024-03-25 18:29 ` Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 3/4] nptl: Add public __rseq_feature_size symbol Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 4/4] nptl: Add features to internal 'struct rseq_area' Michael Jeanson
3 siblings, 0 replies; 5+ messages in thread
From: Michael Jeanson @ 2024-03-25 18:29 UTC (permalink / raw)
To: libc-alpha
Cc: Michael Jeanson, Florian Weimer, Carlos O'Donell, DJ Delorie,
Mathieu Desnoyers
Introduced in Linux v6.3 the rseq extensible ABI [1] will allow adding
rseq features past the initial 32 bytes of the original ABI.
While the rseq features in the latest kernel still fit within the
original ABI size, there are currently only 4 bytes left. It would thus
be a good time to add support for the extensible ABI so that when new
features are added, they are immediately available to GNU libc users.
We use the ELF auxiliary vectors to query the kernel for the size and
alignment of the rseq area, if this fails we default to the original
fixed size and alignment of '32' which the kernel will accept as a
compatibility mode with the original ABI.
This makes the size of the rseq area variable and thus requires to
relocate it out of 'struct pthread'. We chose to move it after (in block
allocation order) the last TLS block inside the static TLS block
allocation. Tt required a fairly small modification to the TLS block
allocator and did not interfere with the main executable TLS block which
must always be the first block relative to the thread pointer.
[1] https://lore.kernel.org/all/20221122203932.231377-4-mathieu.desnoyers@efficios.com/
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Co-Authored-By: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-By: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
Changes since RFC v1:
- Insert the rseq area after the last TLS block
- Add proper support for TLS_TCB_AT_TP variant
Changes since RFC v2:
- Set __rseq_size even when the registration fails
- Adjust rseq tests to the new ABI
- Added support for statically linked executables
Changes since RFC v3:
- Fix RSEQ_SETMEM for rseq disabled
- Replace sys/auxv.h usage with dl-parse_auxv.h
- Fix offset for TLS_TCB_AT_TP with statically linked executables
- Zero the rseq area before registration
Changes since RFC v4:
- Move dynamic linker defines to a header file
- Fix alignment when tls block align is smaller than rseq align with
statically linked executables
- Add statically linked rseq tests
- Revert: Set __rseq_size even when the registration fails
- Use minimum size when rseq is disabled by tunable
Changes since v5:
- Fix TLS_DTV_AT_TP rseq offset with statically linked executables
Changes since v6:
- Fix tst-rseq for feature size over 32 bytes
- Rebased on 'nptl: fix potential merge of __rseq_* relro symbols'
Changes since v8:
- Fix copyright year in sysdeps/generic/dl-rseq.h
- Clarify the tcb math comments
- Add a comment to clarify what enforces the aligment requirements of a
pointer calculated from the rseq_offset
- Remove nonsensical test in tst-rseq-disable
- Add comments to clarify why the rseq size is 0 when registration fails
or is disabled
- Add comments to explain why we allocate an rseq area block even when
the registration is disabled by tunable
- Rename 'rseq_size' -> 'rseq_alloc_size' and 'dl_tls_rseq_size' ->
'dl_tls_rseq_alloc_size' to clarify the distinction between the
allocated rseq size and the size reported to application code in
'__rseq_size'
---
csu/libc-tls.c | 84 +++++++++++++++++--
elf/dl-tls.c | 78 +++++++++++++++++
elf/rtld_static_init.c | 12 +++
nptl/descr.h | 20 +----
nptl/pthread_create.c | 2 +-
sysdeps/generic/dl-rseq.h | 26 ++++++
sysdeps/generic/ldsodefs.h | 12 +++
sysdeps/i386/nptl/tcb-access.h | 56 +++++++++++++
sysdeps/nptl/dl-tls_init_tp.c | 10 ++-
sysdeps/nptl/tcb-access.h | 5 ++
sysdeps/unix/sysv/linux/Makefile | 10 +++
sysdeps/unix/sysv/linux/dl-parse_auxv.h | 6 ++
sysdeps/unix/sysv/linux/rseq-internal.h | 29 ++++++-
sysdeps/unix/sysv/linux/sched_getcpu.c | 3 +-
.../unix/sysv/linux/tst-rseq-disable-static.c | 1 +
sysdeps/unix/sysv/linux/tst-rseq-disable.c | 17 ++--
.../unix/sysv/linux/tst-rseq-nptl-static.c | 1 +
sysdeps/unix/sysv/linux/tst-rseq-static.c | 1 +
sysdeps/unix/sysv/linux/tst-rseq.c | 22 ++++-
sysdeps/unix/sysv/linux/tst-rseq.h | 9 +-
sysdeps/x86_64/nptl/tcb-access.h | 56 +++++++++++++
21 files changed, 413 insertions(+), 47 deletions(-)
create mode 100644 sysdeps/generic/dl-rseq.h
create mode 100644 sysdeps/unix/sysv/linux/tst-rseq-disable-static.c
create mode 100644 sysdeps/unix/sysv/linux/tst-rseq-nptl-static.c
create mode 100644 sysdeps/unix/sysv/linux/tst-rseq-static.c
diff --git a/csu/libc-tls.c b/csu/libc-tls.c
index b7682bdf43..f73d0e1c52 100644
--- a/csu/libc-tls.c
+++ b/csu/libc-tls.c
@@ -26,6 +26,8 @@
#include <array_length.h>
#include <pthreadP.h>
#include <dl-call_tls_init_tp.h>
+#include <dl-rseq.h>
+#include <elf/dl-tunables.h>
#ifdef SHARED
#error makefile bug, this file is for static only
@@ -62,6 +64,18 @@ size_t _dl_tls_static_surplus;
dynamic TLS access (e.g. with TLSDESC). */
size_t _dl_tls_static_optional;
+/* Size of the features present in the rseq area. */
+size_t _dl_tls_rseq_feature_size;
+
+/* Alignment requirement of the rseq area. */
+size_t _dl_tls_rseq_align;
+
+/* Size of the rseq area allocated in the static TLS block. */
+size_t _dl_tls_rseq_alloc_size;
+
+/* Offset of the rseq area from the thread pointer. */
+ptrdiff_t _dl_tls_rseq_offset;
+
/* Generation counter for the dtv. */
size_t _dl_tls_generation;
@@ -110,6 +124,7 @@ __libc_setup_tls (void)
size_t filesz = 0;
void *initimage = NULL;
size_t align = 0;
+ size_t tls_blocks_size = 0;
size_t max_align = TCB_ALIGNMENT;
size_t tcb_offset;
const ElfW(Phdr) *phdr;
@@ -135,22 +150,79 @@ __libc_setup_tls (void)
/* Calculate the size of the static TLS surplus, with 0 auditors. */
_dl_tls_static_surplus_init (0);
+ /* Even when disabled by tunable, an rseq area will be allocated to allow
+ application code to test the registration status with 'rseq->cpud_id >= 0'.
+ Default to the rseq ABI minimum size and alignment, this will ensure we
+ don't use more TLS than necessary. */
+ size_t rseq_alloc_size = TLS_DL_RSEQ_MIN_SIZE;
+ size_t rseq_align = TLS_DL_RSEQ_MIN_ALIGN;
+ bool do_rseq = true;
+ do_rseq = TUNABLE_GET_FULL (glibc, pthread, rseq, int, NULL);
+ if (do_rseq)
+ {
+ rseq_align = GLRO(dl_tls_rseq_align);
+ /* Make sure the rseq area size is at least the minimum ABI size and a
+ multiple of the requested aligment. */
+ rseq_alloc_size = roundup (MAX (GLRO(dl_tls_rseq_feature_size),
+ TLS_DL_RSEQ_MIN_SIZE), rseq_align);
+ }
+
+ /* Increase the maximum alignment with the rseq alignment requirements if
+ necessary. */
+ max_align = MAX (max_align, rseq_align);
+
+ /* Record the rseq_area block size. */
+ GLRO (dl_tls_rseq_alloc_size) = rseq_alloc_size;
+
/* We have to set up the TCB block which also (possibly) contains
'errno'. Therefore we avoid 'malloc' which might touch 'errno'.
Instead we use 'sbrk' which would only uses 'errno' if it fails.
In this case we are right away out of memory and the user gets
what she/he deserves. */
#if TLS_TCB_AT_TP
+ /* Before the the thread pointer, add the aligned tls block size and then
+ align the rseq area block on top. */
+ tls_blocks_size = roundup (roundup (memsz, align ?: 1) + rseq_alloc_size, rseq_align);
+
+ /* Record the rseq_area offset.
+
+ With TLS_TCB_AT_TP the TLS blocks are allocated before the thread pointer
+ in reverse order. Our block is added last which results in it being the
+ first in the static TLS block, thus record the most negative offset.
+
+ The alignment requirements of the pointer resulting from this offset and
+ the thread pointer are enforced by 'max_align' which is used to align the
+ tcb_offset. */
+ GLRO (dl_tls_rseq_offset) = -tls_blocks_size;
+
/* Align the TCB offset to the maximum alignment, as
_dl_allocate_tls_storage (in elf/dl-tls.c) does using __libc_memalign
and dl_tls_static_align. */
- tcb_offset = roundup (memsz + GLRO(dl_tls_static_surplus), max_align);
+ tcb_offset = roundup (tls_blocks_size + GLRO(dl_tls_static_surplus), max_align);
tlsblock = _dl_early_allocate (tcb_offset + TLS_INIT_TCB_SIZE + max_align);
if (tlsblock == NULL)
_startup_fatal_tls_error ();
#elif TLS_DTV_AT_TP
+ /* Align memsz on top of the initial tcb. */
tcb_offset = roundup (TLS_INIT_TCB_SIZE, align ?: 1);
- tlsblock = _dl_early_allocate (tcb_offset + memsz + max_align
+
+ /* After the thread pointer, add the initial tcb plus the tls block size and
+ then align the rseq area block on top. */
+ tls_blocks_size = roundup (tcb_offset + memsz + rseq_alloc_size, rseq_align);
+
+ /* Record the rseq_area offset.
+
+ With TLS_DTV_AT_TP the TLS blocks are allocated after the thread pointer in
+ order. Our block is added last which results in it being the last in the
+ static TLS block, thus record the offset as the size of the static TLS
+ block minus the size of our block. The resulting offset will be positive.
+
+ The alignment requirements of the pointer resulting from this offset and
+ the thread pointer are enforced by 'max_align' which is used to align the
+ tcb_offset. */
+ GLRO (dl_tls_rseq_offset) = tls_blocks_size - rseq_alloc_size;
+
+ tlsblock = _dl_early_allocate (tls_blocks_size + max_align
+ TLS_PRE_TCB_SIZE
+ GLRO(dl_tls_static_surplus));
if (tlsblock == NULL)
@@ -209,11 +281,5 @@ __libc_setup_tls (void)
/* static_slotinfo.slotinfo[1].gen = 0; -- Already zero. */
static_slotinfo.slotinfo[1].map = main_map;
- memsz = roundup (memsz, align ?: 1);
-
-#if TLS_DTV_AT_TP
- memsz += tcb_offset;
-#endif
-
- init_static_tls (memsz, MAX (TCB_ALIGNMENT, max_align));
+ init_static_tls (tls_blocks_size, MAX (TCB_ALIGNMENT, max_align));
}
diff --git a/elf/dl-tls.c b/elf/dl-tls.c
index 7b3dd9ab60..7a22a804a9 100644
--- a/elf/dl-tls.c
+++ b/elf/dl-tls.c
@@ -27,6 +27,7 @@
#include <tls.h>
#include <dl-tls.h>
+#include <dl-rseq.h>
#include <ldsodefs.h>
#if PTHREAD_IN_LIBC
@@ -298,6 +299,44 @@ _dl_determine_tlsoffset (void)
slotinfo[cnt].map->l_tls_offset = off;
}
+ /* Insert the rseq area block after the last TLS block. */
+
+ /* Even when disabled by tunable, an rseq area will be allocated to allow
+ application code to test the registration status with 'rseq->cpud_id >= 0'.
+ Default to the rseq ABI minimum size and aligment, this will ensure we
+ don't use more TLS than necessary. */
+ size_t rseq_alloc_size = TLS_DL_RSEQ_MIN_SIZE;
+ size_t rseq_align = TLS_DL_RSEQ_MIN_ALIGN;
+ bool do_rseq = true;
+ do_rseq = TUNABLE_GET_FULL (glibc, pthread, rseq, int, NULL);
+ if (do_rseq)
+ {
+ rseq_align = GLRO(dl_tls_rseq_align);
+ /* Make sure the rseq area size is at least the minimum ABI size and a
+ multiple of the requested aligment. */
+ rseq_alloc_size = roundup (MAX (GLRO(dl_tls_rseq_feature_size),
+ TLS_DL_RSEQ_MIN_SIZE), rseq_align);
+ }
+
+ /* Add the rseq area block to the global offset. */
+ offset = roundup (offset, rseq_align) + rseq_alloc_size;
+
+ /* Increase the maximum alignment with the rseq alignment requirements if
+ necessary. */
+ max_align = MAX (max_align, rseq_align);
+
+ /* Record the rseq_area offset.
+
+ With TLS_TCB_AT_TP the TLS blocks are allocated before the thread pointer
+ in reverse order. Our block is added last which results in it being the
+ first in the static TLS block, thus record the most negative offset.
+
+ The alignment requirements of the pointer resulting from this offset and
+ the thread pointer are enforced by 'max_align' which is used to align the
+ tcb_offset. */
+ GLRO (dl_tls_rseq_offset) = -offset;
+ GLRO (dl_tls_rseq_alloc_size) = rseq_alloc_size;
+
GL(dl_tls_static_used) = offset;
GLRO (dl_tls_static_size) = (roundup (offset + GLRO(dl_tls_static_surplus),
max_align)
@@ -343,6 +382,45 @@ _dl_determine_tlsoffset (void)
offset = off + slotinfo[cnt].map->l_tls_blocksize - firstbyte;
}
+ /* Insert the rseq area block after the last TLS block. */
+
+ /* Default to the rseq ABI minimum sizes, this will reduce TLS usage to 32
+ bytes when rseq is disabled by tunables. */
+ size_t rseq_alloc_size = TLS_DL_RSEQ_MIN_SIZE;
+ size_t rseq_align = TLS_DL_RSEQ_MIN_ALIGN;
+ bool do_rseq = true;
+ do_rseq = TUNABLE_GET_FULL (glibc, pthread, rseq, int, NULL);
+ if (do_rseq)
+ {
+ rseq_align = GLRO(dl_tls_rseq_align);
+ /* Make sure the rseq area size is at least the minimum ABI size and a
+ multiple of the requested aligment. */
+ rseq_alloc_size = roundup (MAX (GLRO(dl_tls_rseq_feature_size),
+ TLS_DL_RSEQ_MIN_SIZE), rseq_align);
+ }
+
+ /* Align the global offset to the beginning of the rseq area. */
+ offset = roundup (offset, rseq_align);
+
+ /* Record the rseq_area offset.
+
+ With TLS_DTV_AT_TP the TLS blocks are allocated after the thread pointer in
+ order. Our block is added last which results in it being the last in the
+ static TLS block, thus record the offset as the size of the static TLS
+ block minus the size of our block. The resulting offset will be positive.
+
+ The alignment requirements of the pointer resulting from this offset and
+ the thread pointer are enforced by 'max_align' which is used to align the
+ tcb_offset. */
+ GLRO (dl_tls_rseq_alloc_size) = rseq_alloc_size;
+ GLRO (dl_tls_rseq_offset) = offset;
+
+ /* Add the rseq area block to the global offset. */
+ offset += rseq_alloc_size;
+
+ /* Increase the max_align if necessary. */
+ max_align = MAX (max_align, rseq_align);
+
GL(dl_tls_static_used) = offset;
GLRO (dl_tls_static_size) = roundup (offset + GLRO(dl_tls_static_surplus),
TCB_ALIGNMENT);
diff --git a/elf/rtld_static_init.c b/elf/rtld_static_init.c
index e918e4ebdf..293d078201 100644
--- a/elf/rtld_static_init.c
+++ b/elf/rtld_static_init.c
@@ -78,6 +78,18 @@ __rtld_static_init (struct link_map *map)
extern __typeof (dl->_dl_tls_static_size) _dl_tls_static_size
attribute_hidden;
dl->_dl_tls_static_size = _dl_tls_static_size;
+ extern __typeof (dl->_dl_tls_rseq_feature_size) _dl_tls_rseq_feature_size
+ attribute_hidden;
+ dl->_dl_tls_rseq_feature_size = _dl_tls_rseq_feature_size;
+ extern __typeof (dl->_dl_tls_rseq_align) _dl_tls_rseq_align
+ attribute_hidden;
+ dl->_dl_tls_rseq_align = _dl_tls_rseq_align;
+ extern __typeof (dl->_dl_tls_rseq_alloc_size) _dl_tls_rseq_alloc_size
+ attribute_hidden;
+ dl->_dl_tls_rseq_alloc_size = _dl_tls_rseq_alloc_size;
+ extern __typeof (dl->_dl_tls_rseq_offset) _dl_tls_rseq_offset
+ attribute_hidden;
+ dl->_dl_tls_rseq_offset = _dl_tls_rseq_offset;
dl->_dl_find_object = _dl_find_object;
__rtld_static_init_arch (map, dl);
diff --git a/nptl/descr.h b/nptl/descr.h
index 8cef95810c..cdc3c82d9a 100644
--- a/nptl/descr.h
+++ b/nptl/descr.h
@@ -404,25 +404,11 @@ struct pthread
/* Used on strsignal. */
struct tls_internal_t tls_state;
- /* rseq area registered with the kernel. Use a custom definition
- here to isolate from kernel struct rseq changes. The
- implementation of sched_getcpu needs acccess to the cpu_id field;
- the other fields are unused and not included here. */
- union
- {
- struct
- {
- uint32_t cpu_id_start;
- uint32_t cpu_id;
- };
- char pad[32]; /* Original rseq area size. */
- } rseq_area __attribute__ ((aligned (32)));
-
/* Amount of end padding, if any, in this structure.
- This definition relies on rseq_area being last. */
+ This definition relies on tls_state being last. */
#define PTHREAD_STRUCT_END_PADDING \
- (sizeof (struct pthread) - offsetof (struct pthread, rseq_area) \
- + sizeof ((struct pthread) {}.rseq_area))
+ (sizeof (struct pthread) - offsetof (struct pthread, tls_state) \
+ + sizeof ((struct pthread) {}.tls_state))
} __attribute ((aligned (TCB_ALIGNMENT)));
static inline bool
diff --git a/nptl/pthread_create.c b/nptl/pthread_create.c
index 1d3665d5ed..9b49ee7121 100644
--- a/nptl/pthread_create.c
+++ b/nptl/pthread_create.c
@@ -691,7 +691,7 @@ __pthread_create_2_1 (pthread_t *newthread, const pthread_attr_t *attr,
/* Inherit rseq registration state. Without seccomp filters, rseq
registration will either always fail or always succeed. */
- if ((int) THREAD_GETMEM_VOLATILE (self, rseq_area.cpu_id) >= 0)
+ if ((int) RSEQ_GETMEM_VOLATILE (rseq_get_area(), cpu_id) >= 0)
pd->flags |= ATTR_FLAG_DO_RSEQ;
/* Initialize the field for the ID of the thread which is waiting
diff --git a/sysdeps/generic/dl-rseq.h b/sysdeps/generic/dl-rseq.h
new file mode 100644
index 0000000000..c967f99f33
--- /dev/null
+++ b/sysdeps/generic/dl-rseq.h
@@ -0,0 +1,26 @@
+/* RSEQ defines for the dynamic linker. Generic version.
+ Copyright (C) 2024 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+/* Minimum size of the rseq area. */
+#define TLS_DL_RSEQ_MIN_SIZE 32
+
+/* Minimum feature size of the rseq area. */
+#define TLS_DL_RSEQ_MIN_FEATURE_SIZE 20
+
+/* Minimum size of the rseq area alignment. */
+#define TLS_DL_RSEQ_MIN_ALIGN 32
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index 50f58a60e3..c8bd39ddcf 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -610,6 +610,18 @@ struct rtld_global_ro
See comments in elf/dl-tls.c where it is initialized. */
EXTERN size_t _dl_tls_static_surplus;
+ /* Size of the features present in the rseq area. */
+ EXTERN size_t _dl_tls_rseq_feature_size;
+
+ /* Alignment requirement of the rseq area. */
+ EXTERN size_t _dl_tls_rseq_align;
+
+ /* Size of the rseq area allocated in the static TLS block. */
+ EXTERN size_t _dl_tls_rseq_alloc_size;
+
+ /* Offset of the rseq area from the thread pointer. */
+ EXTERN ptrdiff_t _dl_tls_rseq_offset;
+
/* Name of the shared object to be profiled (if any). */
EXTERN const char *_dl_profile;
/* Filename of the output file. */
diff --git a/sysdeps/i386/nptl/tcb-access.h b/sysdeps/i386/nptl/tcb-access.h
index 4b6221e103..e6988186d0 100644
--- a/sysdeps/i386/nptl/tcb-access.h
+++ b/sysdeps/i386/nptl/tcb-access.h
@@ -123,3 +123,59 @@
"i" (offsetof (struct pthread, member)), \
"r" (idx)); \
}})
+
+
+/* Read member of the RSEQ area directly. */
+#define RSEQ_GETMEM_VOLATILE(descr, member) \
+ ({ __typeof (descr->member) __value; \
+ ptrdiff_t _rseq_offset = GLRO (dl_tls_rseq_offset); \
+ _Static_assert (sizeof (__value) == 1 \
+ || sizeof (__value) == 4 \
+ || sizeof (__value) == 8, \
+ "size of per-thread data"); \
+ if (sizeof (__value) == 1) \
+ asm volatile ("movb %%gs:%P2(%3),%b0" \
+ : "=q" (__value) \
+ : "0" (0), "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else if (sizeof (__value) == 4) \
+ asm volatile ("movl %%gs:%P1(%2),%0" \
+ : "=r" (__value) \
+ : "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else /* 8 */ \
+ { \
+ asm volatile ("movl %%gs:%P1(%2),%%eax\n\t" \
+ "movl %%gs:4+%P1(%2),%%edx" \
+ : "=&A" (__value) \
+ : "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ } \
+ __value; })
+
+/* Set member of the RSEQ area directly. */
+#define RSEQ_SETMEM(descr, member, value) \
+ ({ \
+ ptrdiff_t _rseq_offset = GLRO (dl_tls_rseq_offset); \
+ _Static_assert (sizeof (descr->member) == 1 \
+ || sizeof (descr->member) == 4 \
+ || sizeof (descr->member) == 8, \
+ "size of per-thread data"); \
+ if (sizeof (descr->member) == 1) \
+ asm volatile ("movb %b0,%%gs:%P1(%2)" : \
+ : "iq" (value), \
+ "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else if (sizeof (descr->member) == 4) \
+ asm volatile ("movl %0,%%gs:%P1(%2)" : \
+ : "ir" (value), \
+ "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else /* 8 */ \
+ { \
+ asm volatile ("movl %%eax,%%gs:%P1(%2)\n\t" \
+ "movl %%edx,%%gs:4+%P1(%2)" : \
+ : "A" ((uint64_t) cast_to_integer (value)), \
+ "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ }})
diff --git a/sysdeps/nptl/dl-tls_init_tp.c b/sysdeps/nptl/dl-tls_init_tp.c
index 2f9750c50b..1cfaf38cf8 100644
--- a/sysdeps/nptl/dl-tls_init_tp.c
+++ b/sysdeps/nptl/dl-tls_init_tp.c
@@ -107,16 +107,22 @@ __tls_init_tp (void)
do_rseq = TUNABLE_GET (rseq, int, NULL);
if (rseq_register_current_thread (pd, do_rseq))
{
- _rseq_size = sizeof (pd->rseq_area);
+ _rseq_size = GLRO (dl_tls_rseq_alloc_size);
}
+ /* If the registration fails or is disabled by tunable, the public rseq
+ size will be '0' regardless of the size of the allocated rseq area. An
+ rseq area of at least 32 bytes is always allocated since application
+ code is allowed to test the status of the rseq registration with
+ 'rseq->cpu_id >= 0'. */
+
#ifdef RSEQ_SIG
/* This should be a compile-time constant, but the current
infrastructure makes it difficult to determine its value. Not
all targets support __thread_pointer, so set __rseq_offset only
if the rseq registration may have happened because RSEQ_SIG is
defined. */
- _rseq_offset = (char *) &pd->rseq_area - (char *) __thread_pointer ();
+ _rseq_offset = GLRO (dl_tls_rseq_offset);
#endif
}
diff --git a/sysdeps/nptl/tcb-access.h b/sysdeps/nptl/tcb-access.h
index 600433766f..9532f30022 100644
--- a/sysdeps/nptl/tcb-access.h
+++ b/sysdeps/nptl/tcb-access.h
@@ -30,3 +30,8 @@
descr->member = (value)
#define THREAD_SETMEM_NC(descr, member, idx, value) \
descr->member[idx] = (value)
+
+#define RSEQ_GETMEM_VOLATILE(descr, member) \
+ THREAD_GETMEM_VOLATILE(descr, member)
+#define RSEQ_SETMEM(descr, member, value) \
+ THREAD_SETMEM(descr, member, value)
diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
index 415aa1f14d..6bcf81461b 100644
--- a/sysdeps/unix/sysv/linux/Makefile
+++ b/sysdeps/unix/sysv/linux/Makefile
@@ -261,6 +261,11 @@ tests-internal += \
tst-rseq-disable \
# tests-internal
+tests-static += \
+ tst-rseq-disable-static \
+ tst-rseq-static \
+ # tests-static
+
tests-time64 += \
tst-adjtimex-time64 \
tst-clock_adjtime-time64 \
@@ -394,6 +399,7 @@ $(objpfx)tst-mount-compile.out: ../sysdeps/unix/sysv/linux/tst-mount-compile.py
$(objpfx)tst-mount-compile.out: $(sysdeps-linux-python-deps)
tst-rseq-disable-ENV = GLIBC_TUNABLES=glibc.pthread.rseq=0
+tst-rseq-disable-static-ENV = GLIBC_TUNABLES=glibc.pthread.rseq=0
endif # $(subdir) == misc
@@ -655,4 +661,8 @@ tests += \
tests-internal += \
tst-rseq-nptl \
# tests-internal
+
+tests-static += \
+ tst-rseq-nptl-static \
+ # tests-static
endif
diff --git a/sysdeps/unix/sysv/linux/dl-parse_auxv.h b/sysdeps/unix/sysv/linux/dl-parse_auxv.h
index ea2a58ecb1..51943dd483 100644
--- a/sysdeps/unix/sysv/linux/dl-parse_auxv.h
+++ b/sysdeps/unix/sysv/linux/dl-parse_auxv.h
@@ -21,6 +21,7 @@
#include <fpu_control.h>
#include <ldsodefs.h>
#include <link.h>
+#include <dl-rseq.h>
typedef ElfW(Addr) dl_parse_auxv_t[AT_MINSIGSTKSZ + 1];
@@ -59,5 +60,10 @@ void _dl_parse_auxv (ElfW(auxv_t) *av, dl_parse_auxv_t auxv_values)
GLRO(dl_sysinfo) = auxv_values[AT_SYSINFO];
#endif
+ GLRO(dl_tls_rseq_feature_size) = MAX (auxv_values[AT_RSEQ_FEATURE_SIZE],
+ TLS_DL_RSEQ_MIN_FEATURE_SIZE);
+ GLRO(dl_tls_rseq_align) = MAX (auxv_values[AT_RSEQ_ALIGN],
+ TLS_DL_RSEQ_MIN_ALIGN);
+
DL_PLATFORM_AUXV
}
diff --git a/sysdeps/unix/sysv/linux/rseq-internal.h b/sysdeps/unix/sysv/linux/rseq-internal.h
index 48eebc1e16..4123072274 100644
--- a/sysdeps/unix/sysv/linux/rseq-internal.h
+++ b/sysdeps/unix/sysv/linux/rseq-internal.h
@@ -24,6 +24,24 @@
#include <stdbool.h>
#include <stdio.h>
#include <sys/rseq.h>
+#include <thread_pointer.h>
+#include <ldsodefs.h>
+
+/* rseq area registered with the kernel. Use a custom definition
+ here to isolate from kernel struct rseq changes. The
+ implementation of sched_getcpu needs acccess to the cpu_id field;
+ the other fields are unused and not included here. */
+struct rseq_area
+{
+ uint32_t cpu_id_start;
+ uint32_t cpu_id;
+};
+
+static inline struct rseq_area *
+rseq_get_area(void)
+{
+ return (struct rseq_area *) ((char *) __thread_pointer() + GLRO (dl_tls_rseq_offset));
+}
#ifdef RSEQ_SIG
static inline bool
@@ -31,20 +49,23 @@ rseq_register_current_thread (struct pthread *self, bool do_rseq)
{
if (do_rseq)
{
- int ret = INTERNAL_SYSCALL_CALL (rseq, &self->rseq_area,
- sizeof (self->rseq_area),
+ /* The kernel expects 'rseq_area->rseq_cs == NULL' on registration, zero
+ the whole rseq area. */
+ memset(rseq_get_area(), 0, GLRO (dl_tls_rseq_alloc_size));
+ int ret = INTERNAL_SYSCALL_CALL (rseq, rseq_get_area(),
+ GLRO (dl_tls_rseq_alloc_size),
0, RSEQ_SIG);
if (!INTERNAL_SYSCALL_ERROR_P (ret))
return true;
}
- THREAD_SETMEM (self, rseq_area.cpu_id, RSEQ_CPU_ID_REGISTRATION_FAILED);
+ RSEQ_SETMEM (rseq_get_area(), cpu_id, RSEQ_CPU_ID_REGISTRATION_FAILED);
return false;
}
#else /* RSEQ_SIG */
static inline bool
rseq_register_current_thread (struct pthread *self, bool do_rseq)
{
- THREAD_SETMEM (self, rseq_area.cpu_id, RSEQ_CPU_ID_REGISTRATION_FAILED);
+ RSEQ_SETMEM (rseq_get_area(), cpu_id, RSEQ_CPU_ID_REGISTRATION_FAILED);
return false;
}
#endif /* RSEQ_SIG */
diff --git a/sysdeps/unix/sysv/linux/sched_getcpu.c b/sysdeps/unix/sysv/linux/sched_getcpu.c
index 72a3360550..3cdf854316 100644
--- a/sysdeps/unix/sysv/linux/sched_getcpu.c
+++ b/sysdeps/unix/sysv/linux/sched_getcpu.c
@@ -19,6 +19,7 @@
#include <sched.h>
#include <sysdep.h>
#include <sysdep-vdso.h>
+#include <rseq-internal.h>
static int
vsyscall_sched_getcpu (void)
@@ -36,6 +37,6 @@ vsyscall_sched_getcpu (void)
int
sched_getcpu (void)
{
- int cpu_id = THREAD_GETMEM_VOLATILE (THREAD_SELF, rseq_area.cpu_id);
+ int cpu_id = RSEQ_GETMEM_VOLATILE (rseq_get_area(), cpu_id);
return __glibc_likely (cpu_id >= 0) ? cpu_id : vsyscall_sched_getcpu ();
}
diff --git a/sysdeps/unix/sysv/linux/tst-rseq-disable-static.c b/sysdeps/unix/sysv/linux/tst-rseq-disable-static.c
new file mode 100644
index 0000000000..2687d13d3d
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/tst-rseq-disable-static.c
@@ -0,0 +1 @@
+#include "tst-rseq-disable.c"
diff --git a/sysdeps/unix/sysv/linux/tst-rseq-disable.c b/sysdeps/unix/sysv/linux/tst-rseq-disable.c
index bbc655bec4..cd28f1ccfd 100644
--- a/sysdeps/unix/sysv/linux/tst-rseq-disable.c
+++ b/sysdeps/unix/sysv/linux/tst-rseq-disable.c
@@ -26,27 +26,28 @@
#include <unistd.h>
#ifdef RSEQ_SIG
+# include <sys/auxv.h>
+# include "tst-rseq.h"
+
+static __thread struct rseq local_rseq;
/* Check that rseq can be registered and has not been taken by glibc. */
static void
check_rseq_disabled (void)
{
- struct pthread *pd = THREAD_SELF;
+ struct rseq *rseq_area = (struct rseq *) ((char *) __thread_pointer () + __rseq_offset);
TEST_COMPARE (__rseq_flags, 0);
- TEST_VERIFY ((char *) __thread_pointer () + __rseq_offset
- == (char *) &pd->rseq_area);
TEST_COMPARE (__rseq_size, 0);
- TEST_COMPARE ((int) pd->rseq_area.cpu_id, RSEQ_CPU_ID_REGISTRATION_FAILED);
+ TEST_COMPARE ((int) rseq_area->cpu_id, RSEQ_CPU_ID_REGISTRATION_FAILED);
- int ret = syscall (__NR_rseq, &pd->rseq_area, sizeof (pd->rseq_area),
- 0, RSEQ_SIG);
+ int ret = syscall (__NR_rseq, &local_rseq, RSEQ_TEST_MIN_SIZE, 0, RSEQ_SIG);
if (ret == 0)
{
- ret = syscall (__NR_rseq, &pd->rseq_area, sizeof (pd->rseq_area),
+ ret = syscall (__NR_rseq, &local_rseq, RSEQ_TEST_MIN_SIZE,
RSEQ_FLAG_UNREGISTER, RSEQ_SIG);
TEST_COMPARE (ret, 0);
- pd->rseq_area.cpu_id = RSEQ_CPU_ID_REGISTRATION_FAILED;
+ rseq_area->cpu_id = RSEQ_CPU_ID_REGISTRATION_FAILED;
}
else
{
diff --git a/sysdeps/unix/sysv/linux/tst-rseq-nptl-static.c b/sysdeps/unix/sysv/linux/tst-rseq-nptl-static.c
new file mode 100644
index 0000000000..6e2c923bb9
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/tst-rseq-nptl-static.c
@@ -0,0 +1 @@
+#include "tst-rseq-nptl.c"
diff --git a/sysdeps/unix/sysv/linux/tst-rseq-static.c b/sysdeps/unix/sysv/linux/tst-rseq-static.c
new file mode 100644
index 0000000000..1d97f3bd3d
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/tst-rseq-static.c
@@ -0,0 +1 @@
+#include "tst-rseq.c"
diff --git a/sysdeps/unix/sysv/linux/tst-rseq.c b/sysdeps/unix/sysv/linux/tst-rseq.c
index 2c90409ba0..c8c0518a5d 100644
--- a/sysdeps/unix/sysv/linux/tst-rseq.c
+++ b/sysdeps/unix/sysv/linux/tst-rseq.c
@@ -31,18 +31,32 @@
# include <syscall.h>
# include <thread_pointer.h>
# include <tls.h>
+# include <sys/auxv.h>
# include "tst-rseq.h"
static void
do_rseq_main_test (void)
{
- struct pthread *pd = THREAD_SELF;
+ size_t rseq_align = MAX (getauxval (AT_RSEQ_ALIGN), RSEQ_TEST_MIN_ALIGN);
+ size_t rseq_size = roundup (MAX (getauxval (AT_RSEQ_FEATURE_SIZE), RSEQ_TEST_MIN_SIZE), rseq_align);
+ struct rseq *rseq = __thread_pointer () + __rseq_offset;
TEST_VERIFY_EXIT (rseq_thread_registered ());
TEST_COMPARE (__rseq_flags, 0);
- TEST_VERIFY ((char *) __thread_pointer () + __rseq_offset
- == (char *) &pd->rseq_area);
- TEST_COMPARE (__rseq_size, sizeof (pd->rseq_area));
+ TEST_COMPARE (__rseq_size, rseq_size);
+ /* The size of the rseq area must be a multiple of the alignment. */
+ TEST_VERIFY ((__rseq_size % rseq_align) == 0);
+ /* The rseq area address must be aligned. */
+ TEST_VERIFY (((unsigned long) rseq % rseq_align) == 0);
+#if TLS_TCB_AT_TP
+ /* The rseq area block should come before the thread pointer and be at least 32 bytes. */
+ TEST_VERIFY (__rseq_offset <= RSEQ_TEST_MIN_SIZE);
+#elif TLS_DTV_AT_TP
+ /* The rseq area block should come after the thread pointer. */
+ TEST_VERIFY (__rseq_offset >= 0);
+#else
+# error "Either TLS_TCB_AT_TP or TLS_DTV_AT_TP must be defined"
+#endif
}
static void
diff --git a/sysdeps/unix/sysv/linux/tst-rseq.h b/sysdeps/unix/sysv/linux/tst-rseq.h
index dc603327d3..4931aa3d14 100644
--- a/sysdeps/unix/sysv/linux/tst-rseq.h
+++ b/sysdeps/unix/sysv/linux/tst-rseq.h
@@ -23,11 +23,18 @@
#include <syscall.h>
#include <sys/rseq.h>
#include <tls.h>
+#include <rseq-internal.h>
+
+#define RSEQ_TEST_MIN_SIZE 32
+#define RSEQ_TEST_MIN_FEATURE_SIZE 20
+#define RSEQ_TEST_MIN_ALIGN 32
static inline bool
rseq_thread_registered (void)
{
- return THREAD_GETMEM_VOLATILE (THREAD_SELF, rseq_area.cpu_id) >= 0;
+ struct rseq_area *rseq = (struct rseq_area *) ((char *) __thread_pointer () + __rseq_offset);
+
+ return __atomic_load_n (&rseq->cpu_id, __ATOMIC_RELAXED) >= 0;
}
static inline int
diff --git a/sysdeps/x86_64/nptl/tcb-access.h b/sysdeps/x86_64/nptl/tcb-access.h
index d35948f111..75ba4b3ce9 100644
--- a/sysdeps/x86_64/nptl/tcb-access.h
+++ b/sysdeps/x86_64/nptl/tcb-access.h
@@ -130,3 +130,59 @@
"i" (offsetof (struct pthread, member[0])), \
"r" (idx)); \
}})
+
+/* Read member of the RSEQ area directly. */
+# define RSEQ_GETMEM_VOLATILE(descr, member) \
+ ({ __typeof (descr->member) __value; \
+ ptrdiff_t _rseq_offset = GLRO (dl_tls_rseq_offset); \
+ _Static_assert (sizeof (__value) == 1 \
+ || sizeof (__value) == 4 \
+ || sizeof (__value) == 8, \
+ "size of per-thread data"); \
+ if (sizeof (__value) == 1) \
+ asm volatile ("movb %%fs:%P2(%q3),%b0" \
+ : "=q" (__value) \
+ : "0" (0), "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else if (sizeof (__value) == 4) \
+ asm volatile ("movl %%fs:%P1(%q2),%0" \
+ : "=r" (__value) \
+ : "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else /* 8 */ \
+ { \
+ asm volatile ("movq %%fs:%P1(%q2),%q0" \
+ : "=r" (__value) \
+ : "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ } \
+ __value; })
+
+/* Set member of the RSEQ area directly. */
+# define RSEQ_SETMEM(descr, member, value) \
+ ({ \
+ ptrdiff_t _rseq_offset = GLRO (dl_tls_rseq_offset); \
+ _Static_assert (sizeof (descr->member) == 1 \
+ || sizeof (descr->member) == 4 \
+ || sizeof (descr->member) == 8, \
+ "size of per-thread data"); \
+ if (sizeof (descr->member) == 1) \
+ asm volatile ("movb %b0,%%fs:%P1(%q2)" : \
+ : "iq" (value), \
+ "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else if (sizeof (descr->member) == 4) \
+ asm volatile ("movl %0,%%fs:%P1(%q2)" : \
+ : IMM_MODE (value), \
+ "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ else /* 8 */ \
+ { \
+ /* Since movq takes a signed 32-bit immediate or a register source \
+ operand, use "er" constraint for 32-bit signed integer constant \
+ or register. */ \
+ asm volatile ("movq %q0,%%fs:%P1(%q2)" : \
+ : "er" ((uint64_t) cast_to_integer (value)), \
+ "i" (offsetof (struct rseq_area, member)), \
+ "r" (_rseq_offset)); \
+ }})
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH v10 3/4] nptl: Add public __rseq_feature_size symbol
2024-03-25 18:29 [PATCH v10 0/4] Extend rseq support Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 1/4] nptl: fix potential merge of __rseq_* relro symbols Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 2/4] Add rseq extensible ABI support Michael Jeanson
@ 2024-03-25 18:29 ` Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 4/4] nptl: Add features to internal 'struct rseq_area' Michael Jeanson
3 siblings, 0 replies; 5+ messages in thread
From: Michael Jeanson @ 2024-03-25 18:29 UTC (permalink / raw)
To: libc-alpha
Cc: Michael Jeanson, Florian Weimer, Carlos O'Donell, DJ Delorie,
Mathieu Desnoyers
Exposing this symbol allows applications wishing to use rseq features
which are part of the extensible rseq ABI like 'node_id' and 'mm_cid' to
test the two following conditions in a single load / conditional branch:
- rseq is registered
- the specific rseq feature is available
This is useful as rseq is expected to be used in hot paths.
This variable is either zero (if restartable sequence registration
failed or has been disabled) or the size of the available restartable
sequence features.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
---
Changes since v7:
- Fix sorting of symbols in abilist files
Changes since v8:
- Remove superfluous attributes on externs
- Clarify where the magic '20' initial feature size comes from
---
csu/rseq-sizes.sym | 3 +++
elf/dl-rseq-symbols.S | 17 +++++++++++++++++
manual/threads.texi | 9 +++++++++
sysdeps/nptl/dl-tls_init_tp.c | 2 ++
sysdeps/unix/sysv/linux/Versions | 3 +++
sysdeps/unix/sysv/linux/aarch64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/alpha/ld.abilist | 1 +
sysdeps/unix/sysv/linux/arc/ld.abilist | 1 +
sysdeps/unix/sysv/linux/arm/be/ld.abilist | 1 +
sysdeps/unix/sysv/linux/arm/le/ld.abilist | 1 +
sysdeps/unix/sysv/linux/csky/ld.abilist | 1 +
sysdeps/unix/sysv/linux/hppa/ld.abilist | 1 +
sysdeps/unix/sysv/linux/i386/ld.abilist | 1 +
.../unix/sysv/linux/loongarch/lp64/ld.abilist | 1 +
.../unix/sysv/linux/m68k/coldfire/ld.abilist | 1 +
sysdeps/unix/sysv/linux/m68k/m680x0/ld.abilist | 1 +
sysdeps/unix/sysv/linux/microblaze/ld.abilist | 1 +
sysdeps/unix/sysv/linux/mips/mips32/ld.abilist | 1 +
.../unix/sysv/linux/mips/mips64/n32/ld.abilist | 1 +
.../unix/sysv/linux/mips/mips64/n64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/nios2/ld.abilist | 1 +
sysdeps/unix/sysv/linux/or1k/ld.abilist | 1 +
.../sysv/linux/powerpc/powerpc32/ld.abilist | 1 +
.../sysv/linux/powerpc/powerpc64/be/ld.abilist | 1 +
.../sysv/linux/powerpc/powerpc64/le/ld.abilist | 1 +
sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist | 1 +
sysdeps/unix/sysv/linux/riscv/rv64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/s390/s390-32/ld.abilist | 1 +
sysdeps/unix/sysv/linux/s390/s390-64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/sh/be/ld.abilist | 1 +
sysdeps/unix/sysv/linux/sh/le/ld.abilist | 1 +
.../unix/sysv/linux/sparc/sparc32/ld.abilist | 1 +
.../unix/sysv/linux/sparc/sparc64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/sys/rseq.h | 4 ++++
sysdeps/unix/sysv/linux/tst-rseq-disable.c | 1 +
sysdeps/unix/sysv/linux/tst-rseq.c | 4 +++-
sysdeps/unix/sysv/linux/x86_64/64/ld.abilist | 1 +
sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist | 1 +
38 files changed, 72 insertions(+), 1 deletion(-)
diff --git a/csu/rseq-sizes.sym b/csu/rseq-sizes.sym
index c959758ff0..dde53bfa60 100644
--- a/csu/rseq-sizes.sym
+++ b/csu/rseq-sizes.sym
@@ -4,5 +4,8 @@
RSEQ_SIZE_SIZE sizeof (unsigned int)
RSEQ_SIZE_ALIGN __alignof (unsigned int)
+RSEQ_FEATURE_SIZE_SIZE sizeof (unsigned int)
+RSEQ_FEATURE_SIZE_ALIGN __alignof (unsigned int)
+
RSEQ_OFFSET_SIZE sizeof (ptrdiff_t)
RSEQ_OFFSET_ALIGN __alignof (ptrdiff_t)
diff --git a/elf/dl-rseq-symbols.S b/elf/dl-rseq-symbols.S
index 2d8e88367f..709188ae22 100644
--- a/elf/dl-rseq-symbols.S
+++ b/elf/dl-rseq-symbols.S
@@ -38,6 +38,23 @@ __rseq_size:
_rseq_size:
.zero RSEQ_SIZE_SIZE
+/* Define 2 symbols, __rseq_feature_size is public const and
+ _rseq_feature_size, which is an alias of __rseq_feature_size, but hidden and
+ writable for internal use. */
+
+ .globl __rseq_feature_size
+ .type __rseq_feature_size, %object
+ .size __rseq_feature_size, RSEQ_FEATURE_SIZE_SIZE
+ .hidden _rseq_feature_size
+ .globl _rseq_feature_size
+ .type _rseq_feature_size, %object
+ .size _rseq_feature_size, RSEQ_FEATURE_SIZE_SIZE
+ .section .data.rel.ro
+ .balign RSEQ_FEATURE_SIZE_ALIGN
+__rseq_feature_size:
+_rseq_feature_size:
+ .zero RSEQ_FEATURE_SIZE_SIZE
+
/* Define 2 symbols, __rseq_offset is public const and _rseq_offset, which is an
alias of __rseq_offset, but hidden and writable for internal use. */
diff --git a/manual/threads.texi b/manual/threads.texi
index e5544ff3da..186982246d 100644
--- a/manual/threads.texi
+++ b/manual/threads.texi
@@ -1011,6 +1011,15 @@ registration is successful, @code{__rseq_size} is at least 32 (the
initial size of @code{struct rseq}).
@end deftypevar
+@deftypevar {unsigned int} __rseq_feature_size
+@standards{Linux, sys/rseq.h}
+This variable is either zero (if restartable sequence registration
+failed or has been disabled) or the size of the restartable sequence
+features supported by the running kernel. If registration is
+successful, @code{__rseq_feature_size} is at least 20 (the feature size
+of @code{struct rseq} in the initial rseq kernel implementation).
+@end deftypevar
+
@deftypevar {unsigned int} __rseq_flags
@standards{Linux, sys/rseq.h}
The flags used during restartable sequence registration with the kernel.
diff --git a/sysdeps/nptl/dl-tls_init_tp.c b/sysdeps/nptl/dl-tls_init_tp.c
index 1cfaf38cf8..2e94268a96 100644
--- a/sysdeps/nptl/dl-tls_init_tp.c
+++ b/sysdeps/nptl/dl-tls_init_tp.c
@@ -48,6 +48,7 @@ const unsigned int __rseq_flags;
/* The variables are in .data.relro but are not yet write-protected. */
extern unsigned int _rseq_size;
+extern unsigned int _rseq_feature_size;
extern ptrdiff_t _rseq_offset;
void
@@ -108,6 +109,7 @@ __tls_init_tp (void)
if (rseq_register_current_thread (pd, do_rseq))
{
_rseq_size = GLRO (dl_tls_rseq_alloc_size);
+ _rseq_feature_size = GLRO (dl_tls_rseq_feature_size);
}
/* If the registration fails or is disabled by tunable, the public rseq
diff --git a/sysdeps/unix/sysv/linux/Versions b/sysdeps/unix/sysv/linux/Versions
index 268ba1b6ac..055be26dde 100644
--- a/sysdeps/unix/sysv/linux/Versions
+++ b/sysdeps/unix/sysv/linux/Versions
@@ -356,6 +356,9 @@ ld {
__rseq_offset;
__rseq_size;
}
+ GLIBC_2.40 {
+ __rseq_feature_size;
+ }
GLIBC_PRIVATE {
__nptl_change_stack_perm;
}
diff --git a/sysdeps/unix/sysv/linux/aarch64/ld.abilist b/sysdeps/unix/sysv/linux/aarch64/ld.abilist
index 5151c0781d..93039b756d 100644
--- a/sysdeps/unix/sysv/linux/aarch64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/aarch64/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/alpha/ld.abilist b/sysdeps/unix/sysv/linux/alpha/ld.abilist
index 3e296c5473..fc67e31293 100644
--- a/sysdeps/unix/sysv/linux/alpha/ld.abilist
+++ b/sysdeps/unix/sysv/linux/alpha/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x8
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/arc/ld.abilist b/sysdeps/unix/sysv/linux/arc/ld.abilist
index 55f0c2ab9c..31156b6ee1 100644
--- a/sysdeps/unix/sysv/linux/arc/ld.abilist
+++ b/sysdeps/unix/sysv/linux/arc/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/arm/be/ld.abilist b/sysdeps/unix/sysv/linux/arm/be/ld.abilist
index f1da2c636d..3d16fa60db 100644
--- a/sysdeps/unix/sysv/linux/arm/be/ld.abilist
+++ b/sysdeps/unix/sysv/linux/arm/be/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.4 __stack_chk_guard D 0x4
GLIBC_2.4 __tls_get_addr F
GLIBC_2.4 _dl_mcount F
GLIBC_2.4 _r_debug D 0x14
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/arm/le/ld.abilist b/sysdeps/unix/sysv/linux/arm/le/ld.abilist
index f1da2c636d..3d16fa60db 100644
--- a/sysdeps/unix/sysv/linux/arm/le/ld.abilist
+++ b/sysdeps/unix/sysv/linux/arm/le/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.4 __stack_chk_guard D 0x4
GLIBC_2.4 __tls_get_addr F
GLIBC_2.4 _dl_mcount F
GLIBC_2.4 _r_debug D 0x14
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/csky/ld.abilist b/sysdeps/unix/sysv/linux/csky/ld.abilist
index 7f482276ed..088f000c4e 100644
--- a/sysdeps/unix/sysv/linux/csky/ld.abilist
+++ b/sysdeps/unix/sysv/linux/csky/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/hppa/ld.abilist b/sysdeps/unix/sysv/linux/hppa/ld.abilist
index 7f5527fb30..8ae26c46c1 100644
--- a/sysdeps/unix/sysv/linux/hppa/ld.abilist
+++ b/sysdeps/unix/sysv/linux/hppa/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/i386/ld.abilist b/sysdeps/unix/sysv/linux/i386/ld.abilist
index 9c4a45d8dc..d9761c34e3 100644
--- a/sysdeps/unix/sysv/linux/i386/ld.abilist
+++ b/sysdeps/unix/sysv/linux/i386/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/loongarch/lp64/ld.abilist b/sysdeps/unix/sysv/linux/loongarch/lp64/ld.abilist
index 93fcd64eee..f5dbb17ec9 100644
--- a/sysdeps/unix/sysv/linux/loongarch/lp64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/loongarch/lp64/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.36 __stack_chk_guard D 0x8
GLIBC_2.36 __tls_get_addr F
GLIBC_2.36 _dl_mcount F
GLIBC_2.36 _r_debug D 0x28
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/m68k/coldfire/ld.abilist b/sysdeps/unix/sysv/linux/m68k/coldfire/ld.abilist
index f1da2c636d..3d16fa60db 100644
--- a/sysdeps/unix/sysv/linux/m68k/coldfire/ld.abilist
+++ b/sysdeps/unix/sysv/linux/m68k/coldfire/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.4 __stack_chk_guard D 0x4
GLIBC_2.4 __tls_get_addr F
GLIBC_2.4 _dl_mcount F
GLIBC_2.4 _r_debug D 0x14
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/m68k/m680x0/ld.abilist b/sysdeps/unix/sysv/linux/m68k/m680x0/ld.abilist
index dadbf852d0..3888e39812 100644
--- a/sysdeps/unix/sysv/linux/m68k/m680x0/ld.abilist
+++ b/sysdeps/unix/sysv/linux/m68k/m680x0/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/microblaze/ld.abilist b/sysdeps/unix/sysv/linux/microblaze/ld.abilist
index 89a0b7e4fd..c83b62dce7 100644
--- a/sysdeps/unix/sysv/linux/microblaze/ld.abilist
+++ b/sysdeps/unix/sysv/linux/microblaze/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/mips/mips32/ld.abilist b/sysdeps/unix/sysv/linux/mips/mips32/ld.abilist
index e304d1bb46..9710fdb941 100644
--- a/sysdeps/unix/sysv/linux/mips/mips32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/mips/mips32/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/mips/mips64/n32/ld.abilist b/sysdeps/unix/sysv/linux/mips/mips64/n32/ld.abilist
index e304d1bb46..9710fdb941 100644
--- a/sysdeps/unix/sysv/linux/mips/mips64/n32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/mips/mips64/n32/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/mips/mips64/n64/ld.abilist b/sysdeps/unix/sysv/linux/mips/mips64/n64/ld.abilist
index f26e594a13..ec1bdfd965 100644
--- a/sysdeps/unix/sysv/linux/mips/mips64/n64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/mips/mips64/n64/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x8
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/nios2/ld.abilist b/sysdeps/unix/sysv/linux/nios2/ld.abilist
index 811ae9da2f..d2b742ec25 100644
--- a/sysdeps/unix/sysv/linux/nios2/ld.abilist
+++ b/sysdeps/unix/sysv/linux/nios2/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/or1k/ld.abilist b/sysdeps/unix/sysv/linux/or1k/ld.abilist
index cff2ffd23b..eb225ca4c5 100644
--- a/sysdeps/unix/sysv/linux/or1k/ld.abilist
+++ b/sysdeps/unix/sysv/linux/or1k/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.35 __stack_chk_guard D 0x4
GLIBC_2.35 __tls_get_addr F
GLIBC_2.35 _dl_mcount F
GLIBC_2.35 _r_debug D 0x14
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc32/ld.abilist b/sysdeps/unix/sysv/linux/powerpc/powerpc32/ld.abilist
index b1073f0942..7a94751723 100644
--- a/sysdeps/unix/sysv/linux/powerpc/powerpc32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/powerpc/powerpc32/ld.abilist
@@ -9,3 +9,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.39 __parse_hwcap_3_4_and_convert_at_platform F
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/ld.abilist b/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/ld.abilist
index 40942a2cc6..f6bdd89083 100644
--- a/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/ld.abilist
+++ b/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/ld.abilist
@@ -9,3 +9,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.39 __parse_hwcap_3_4_and_convert_at_platform F
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/ld.abilist b/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/ld.abilist
index 01f2694a4d..672bade9ba 100644
--- a/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/ld.abilist
+++ b/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/ld.abilist
@@ -9,3 +9,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.39 __parse_hwcap_3_4_and_convert_at_platform F
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
index 068368878e..d3be9236c2 100644
--- a/sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/riscv/rv64/ld.abilist b/sysdeps/unix/sysv/linux/riscv/rv64/ld.abilist
index a7758a0e52..46b22238ce 100644
--- a/sysdeps/unix/sysv/linux/riscv/rv64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/riscv/rv64/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/s390/s390-32/ld.abilist b/sysdeps/unix/sysv/linux/s390/s390-32/ld.abilist
index c15288394a..f438808c08 100644
--- a/sysdeps/unix/sysv/linux/s390/s390-32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/s390/s390-32/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/s390/s390-64/ld.abilist b/sysdeps/unix/sysv/linux/s390/s390-64/ld.abilist
index 78d071600b..f5fbc6de43 100644
--- a/sysdeps/unix/sysv/linux/s390/s390-64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/s390/s390-64/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/sh/be/ld.abilist b/sysdeps/unix/sysv/linux/sh/be/ld.abilist
index 7f5527fb30..8ae26c46c1 100644
--- a/sysdeps/unix/sysv/linux/sh/be/ld.abilist
+++ b/sysdeps/unix/sysv/linux/sh/be/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/sh/le/ld.abilist b/sysdeps/unix/sysv/linux/sh/le/ld.abilist
index 7f5527fb30..8ae26c46c1 100644
--- a/sysdeps/unix/sysv/linux/sh/le/ld.abilist
+++ b/sysdeps/unix/sysv/linux/sh/le/ld.abilist
@@ -7,3 +7,4 @@ GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
GLIBC_2.4 __stack_chk_guard D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/sparc/sparc32/ld.abilist b/sysdeps/unix/sysv/linux/sparc/sparc32/ld.abilist
index 3aac73f3df..df3f7fefb3 100644
--- a/sysdeps/unix/sysv/linux/sparc/sparc32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/sparc/sparc32/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/sparc/sparc64/ld.abilist b/sysdeps/unix/sysv/linux/sparc/sparc64/ld.abilist
index 5471b24d59..3bf61e1210 100644
--- a/sysdeps/unix/sysv/linux/sparc/sparc64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/sparc/sparc64/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/sys/rseq.h b/sysdeps/unix/sysv/linux/sys/rseq.h
index b8afab8945..a048ba8958 100644
--- a/sysdeps/unix/sysv/linux/sys/rseq.h
+++ b/sysdeps/unix/sysv/linux/sys/rseq.h
@@ -163,6 +163,10 @@ extern const ptrdiff_t __rseq_offset;
unsuccessful. */
extern const unsigned int __rseq_size;
+/* Size of the registered rseq features. 0 if the registration was
+ unsuccessful. */
+extern const unsigned int __rseq_feature_size;
+
/* Flags used during rseq registration. */
extern const unsigned int __rseq_flags;
diff --git a/sysdeps/unix/sysv/linux/tst-rseq-disable.c b/sysdeps/unix/sysv/linux/tst-rseq-disable.c
index cd28f1ccfd..a3ea924daf 100644
--- a/sysdeps/unix/sysv/linux/tst-rseq-disable.c
+++ b/sysdeps/unix/sysv/linux/tst-rseq-disable.c
@@ -39,6 +39,7 @@ check_rseq_disabled (void)
TEST_COMPARE (__rseq_flags, 0);
TEST_COMPARE (__rseq_size, 0);
+ TEST_COMPARE (__rseq_feature_size, 0);
TEST_COMPARE ((int) rseq_area->cpu_id, RSEQ_CPU_ID_REGISTRATION_FAILED);
int ret = syscall (__NR_rseq, &local_rseq, RSEQ_TEST_MIN_SIZE, 0, RSEQ_SIG);
diff --git a/sysdeps/unix/sysv/linux/tst-rseq.c b/sysdeps/unix/sysv/linux/tst-rseq.c
index c8c0518a5d..c5d9afbb0a 100644
--- a/sysdeps/unix/sysv/linux/tst-rseq.c
+++ b/sysdeps/unix/sysv/linux/tst-rseq.c
@@ -38,12 +38,14 @@ static void
do_rseq_main_test (void)
{
size_t rseq_align = MAX (getauxval (AT_RSEQ_ALIGN), RSEQ_TEST_MIN_ALIGN);
- size_t rseq_size = roundup (MAX (getauxval (AT_RSEQ_FEATURE_SIZE), RSEQ_TEST_MIN_SIZE), rseq_align);
+ size_t rseq_feature_size = MAX (getauxval (AT_RSEQ_FEATURE_SIZE), RSEQ_TEST_MIN_FEATURE_SIZE);
+ size_t rseq_size = roundup (MAX (rseq_feature_size, RSEQ_TEST_MIN_SIZE), rseq_align);
struct rseq *rseq = __thread_pointer () + __rseq_offset;
TEST_VERIFY_EXIT (rseq_thread_registered ());
TEST_COMPARE (__rseq_flags, 0);
TEST_COMPARE (__rseq_size, rseq_size);
+ TEST_COMPARE (__rseq_feature_size, rseq_feature_size);
/* The size of the rseq area must be a multiple of the alignment. */
TEST_VERIFY ((__rseq_size % rseq_align) == 0);
/* The rseq area address must be aligned. */
diff --git a/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist b/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
index 5a8bd322cd..25c2153c21 100644
--- a/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x8
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
diff --git a/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist b/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
index e17496d124..05327004e2 100644
--- a/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
@@ -6,3 +6,4 @@ GLIBC_2.34 __rtld_version_placeholder F
GLIBC_2.35 __rseq_flags D 0x4
GLIBC_2.35 __rseq_offset D 0x4
GLIBC_2.35 __rseq_size D 0x4
+GLIBC_2.40 __rseq_feature_size D 0x4
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH v10 4/4] nptl: Add features to internal 'struct rseq_area'
2024-03-25 18:29 [PATCH v10 0/4] Extend rseq support Michael Jeanson
` (2 preceding siblings ...)
2024-03-25 18:29 ` [PATCH v10 3/4] nptl: Add public __rseq_feature_size symbol Michael Jeanson
@ 2024-03-25 18:29 ` Michael Jeanson
3 siblings, 0 replies; 5+ messages in thread
From: Michael Jeanson @ 2024-03-25 18:29 UTC (permalink / raw)
To: libc-alpha
Cc: Michael Jeanson, Florian Weimer, Carlos O'Donell, DJ Delorie,
Mathieu Desnoyers
The 'struct rseq_area' is used to define the offset of the various
features available in the rseq ABI. Add fields that follow cpu_id
(rseq_cs, flags, node_id, mm_cid) in preparation for their use.
Access to features following the original rseq ABI 20 bytes (after
'flags') starting with 'node_id' must be gated by an rseq feature size
test.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
---
sysdeps/unix/sysv/linux/rseq-internal.h | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/sysdeps/unix/sysv/linux/rseq-internal.h b/sysdeps/unix/sysv/linux/rseq-internal.h
index 4123072274..da98e44dad 100644
--- a/sysdeps/unix/sysv/linux/rseq-internal.h
+++ b/sysdeps/unix/sysv/linux/rseq-internal.h
@@ -28,13 +28,17 @@
#include <ldsodefs.h>
/* rseq area registered with the kernel. Use a custom definition
- here to isolate from kernel struct rseq changes. The
- implementation of sched_getcpu needs acccess to the cpu_id field;
- the other fields are unused and not included here. */
+ here to isolate from kernel struct rseq changes. Access to fields
+ beyond the 20 bytes of the original ABI (after 'flags') must be gated
+ by a check of the feature size. */
struct rseq_area
{
uint32_t cpu_id_start;
uint32_t cpu_id;
+ uint64_t rseq_cs;
+ uint32_t flags;
+ uint32_t node_id;
+ uint32_t mm_cid;
};
static inline struct rseq_area *
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-03-25 18:31 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-25 18:29 [PATCH v10 0/4] Extend rseq support Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 1/4] nptl: fix potential merge of __rseq_* relro symbols Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 2/4] Add rseq extensible ABI support Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 3/4] nptl: Add public __rseq_feature_size symbol Michael Jeanson
2024-03-25 18:29 ` [PATCH v10 4/4] nptl: Add features to internal 'struct rseq_area' Michael Jeanson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).