[PATCH htl v3 0/5] Rewrite THREAD_GSCOPE

[PATCH htl v3 0/5] Rewrite THREAD_GSCOPE

[Sergey Bugaev via Libc-alpha]

Hello,

this is yet another revision of the same patchset to stop issuing many
redundant gsync_wake () calls. Following the brief feedback I got the last
time, it switches most users of THREAD_GSCOPE_IN_TCB to PTHREAD_IN_LIBC, then
actually introduces the new GSCOPE implementation, and then finally drops the
remaining uses of THREAD_GSCOPE_IN_TCB entirely, since it's not always on.

Things still seem to build and work somewhat (tested on GNU/Hurd i686 and
GNU/Linux x86_64). rpctrace uname still shows that there are no more
gsync_wake () calls.

Apparently dlopen (RTLD_GLOBAL) is a good way to trigger THREAD_GSCOPE_WAIT ().
I have written a test program that spawns 100 threads, then each of the threads
dlopen's 100 shared objects, then dlclose's them back. On GNU/Linux, I have
verified with GDB that it hits NPTL's __thread_gscope_wait (). I haven't been
able to verify the same on the Hurd due to the general flakiness of GDB there,
but I have ran the program with the patched glibc multiple times and it doesn't
crash, so I assume it's working fine.

Sergey Bugaev (5):
  elf: Replace most uses of THREAD_GSCOPE_IN_TCB
  htl: Move thread table to ld.so
  htl: Reimplement GSCOPE
  elf: Remove THREAD_GSCOPE_IN_TCB
  testrun.sh: Add support for --tool=rpctrace

 Makefile                                 |  9 +++-
 elf/dl-reloc.c                           |  4 +-
 elf/dl-support.c                         |  3 +-
 elf/dl-tls.c                             |  6 +--
 elf/dl-tls_init_tp.c                     |  2 +-
 htl/Versions                             |  2 -
 htl/pt-alloc.c                           | 50 +++++++++------------
 htl/pt-create.c                          | 11 ++---
 htl/pt-internal.h                        | 23 +++-------
 sysdeps/aarch64/nptl/tls.h               |  1 -
 sysdeps/alpha/nptl/tls.h                 |  1 -
 sysdeps/arc/nptl/tls.h                   |  1 -
 sysdeps/arm/nptl/tls.h                   |  1 -
 sysdeps/csky/nptl/tls.h                  |  1 -
 sysdeps/generic/ldsodefs.h               | 18 +++++---
 sysdeps/generic/tls.h                    |  6 ---
 sysdeps/hppa/nptl/tls.h                  |  1 -
 sysdeps/htl/dl-support.c                 | 23 ++++++++++
 sysdeps/htl/dl-thread_gscope_wait.c      | 55 ++++++++++++++++++++++++
 sysdeps/htl/pt-key-delete.c              |  8 ++--
 sysdeps/htl/pthreadP.h                   |  2 -
 sysdeps/htl/raise.c                      |  8 +++-
 sysdeps/htl/thrd_current.c               |  7 ++-
 sysdeps/i386/nptl/tls.h                  |  1 -
 sysdeps/ia64/nptl/tls.h                  |  1 -
 sysdeps/m68k/nptl/tls.h                  |  1 -
 sysdeps/mach/hurd/htl/pt-sigstate-init.c |  2 +-
 sysdeps/mach/hurd/htl/pt-sysdep.c        |  2 +-
 sysdeps/mach/hurd/htl/pt-sysdep.h        |  2 +-
 sysdeps/mach/hurd/i386/tls.h             | 19 ++++++++
 sysdeps/mach/hurd/tls.h                  | 20 ---------
 sysdeps/microblaze/nptl/tls.h            |  1 -
 sysdeps/mips/nptl/tls.h                  |  1 -
 sysdeps/nios2/nptl/tls.h                 |  1 -
 sysdeps/powerpc/nptl/tls.h               |  1 -
 sysdeps/riscv/nptl/tls.h                 |  1 -
 sysdeps/s390/nptl/tls.h                  |  1 -
 sysdeps/sh/nptl/tls.h                    |  1 -
 sysdeps/sparc/nptl/tls.h                 |  1 -
 sysdeps/x86_64/nptl/tls.h                |  1 -
 40 files changed, 172 insertions(+), 128 deletions(-)
 create mode 100644 sysdeps/htl/dl-support.c
 create mode 100644 sysdeps/htl/dl-thread_gscope_wait.c

-- 
2.31.1

[PATCH htl v3 1/5] elf: Replace most uses of THREAD_GSCOPE_IN_TCB

[Sergey Bugaev via Libc-alpha]

While originally this definition was indeed used to distinguish between
the cases where the GSCOPE flag was stored in TCB or not, it has since
become used as a general way to distinguish between HTL and NPTL.

THREAD_GSCOPE_IN_TCB will be removed in the following commits, as HTL,
which currently is the only port that does not put the flag into TCB,
will get ported to put the GSCOPE flag into the TCB as well. To prepare
for that change, migrate all code that wants to distinguish between HTL
and NPTL to use PTHREAD_IN_LIBC instead, which is a better choice since
the distinction mostly has to do with whether libc has access to the
list of thread structures and therefore can initialize thread-local
storage.

The parts of code that actually depend on whether the GSCOPE flag is in
TCB are left unchanged.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 elf/dl-reloc.c             |  4 ++--
 elf/dl-support.c           |  6 ++++--
 elf/dl-tls.c               |  6 +++---
 elf/dl-tls_init_tp.c       |  2 +-
 sysdeps/generic/ldsodefs.h | 11 ++++++-----
 5 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c
index e13a672ade..6c957456b8 100644
--- a/elf/dl-reloc.c
+++ b/elf/dl-reloc.c
@@ -141,7 +141,7 @@ cannot allocate memory in static TLS block"));
     }
 }
 
-#if !THREAD_GSCOPE_IN_TCB
+#if !PTHREAD_IN_LIBC
 /* Initialize static TLS area and DTV for current (only) thread.
    libpthread implementations should provide their own hook
    to handle all threads.  */
@@ -160,7 +160,7 @@ _dl_nothread_init_static_tls (struct link_map *map)
   memset (__mempcpy (dest, map->l_tls_initimage, map->l_tls_initimage_size),
 	  '\0', map->l_tls_blocksize - map->l_tls_initimage_size);
 }
-#endif /* !THREAD_GSCOPE_IN_TCB */
+#endif /* !PTHREAD_IN_LIBC */
 
 void
 _dl_relocate_object (struct link_map *l, struct r_scope_elem *scope[],
diff --git a/elf/dl-support.c b/elf/dl-support.c
index 0155718175..a317459fae 100644
--- a/elf/dl-support.c
+++ b/elf/dl-support.c
@@ -183,7 +183,7 @@ uint64_t _dl_hwcap_mask;
  * executable but this isn't true for all platforms.  */
 ElfW(Word) _dl_stack_flags = DEFAULT_STACK_PERMS;
 
-#if THREAD_GSCOPE_IN_TCB
+#if PTHREAD_IN_LIBC
 list_t _dl_stack_used;
 list_t _dl_stack_user;
 list_t _dl_stack_cache;
@@ -195,9 +195,11 @@ int _dl_stack_cache_lock;
    when it was not, we do it by calling this function.
    It returns an errno code or zero on success.  */
 int (*_dl_make_stack_executable_hook) (void **) = _dl_make_stack_executable;
-int _dl_thread_gscope_count;
 void (*_dl_init_static_tls) (struct link_map *) = &_dl_nothread_init_static_tls;
 #endif
+#if !THREAD_GSCOPE_IN_TCB
+int _dl_thread_gscope_count;
+#endif
 struct dl_scope_free_list *_dl_scope_free_list;
 
 #ifdef NEED_DL_SYSINFO
diff --git a/elf/dl-tls.c b/elf/dl-tls.c
index 423e380f7c..d554ae4497 100644
--- a/elf/dl-tls.c
+++ b/elf/dl-tls.c
@@ -29,7 +29,7 @@
 #include <dl-tls.h>
 #include <ldsodefs.h>
 
-#if THREAD_GSCOPE_IN_TCB
+#if PTHREAD_IN_LIBC
 # include <list.h>
 #endif
 
@@ -1058,7 +1058,7 @@ cannot create TLS data structures"));
     }
 }
 
-#if THREAD_GSCOPE_IN_TCB
+#if PTHREAD_IN_LIBC
 static inline void __attribute__((always_inline))
 init_one_static_tls (struct pthread *curp, struct link_map *map)
 {
@@ -1091,4 +1091,4 @@ _dl_init_static_tls (struct link_map *map)
 
   lll_unlock (GL (dl_stack_cache_lock), LLL_PRIVATE);
 }
-#endif /* THREAD_GSCOPE_IN_TCB */
+#endif /* PTHREAD_IN_LIBC */
diff --git a/elf/dl-tls_init_tp.c b/elf/dl-tls_init_tp.c
index d84adc992c..e482f3cfa9 100644
--- a/elf/dl-tls_init_tp.c
+++ b/elf/dl-tls_init_tp.c
@@ -36,7 +36,7 @@ rtld_lock_default_unlock_recursive (void *lock)
 void
 __tls_pre_init_tp (void)
 {
-#if !THREAD_GSCOPE_IN_TCB
+#if !PTHREAD_IN_LIBC
   GL(dl_init_static_tls) = &_dl_nothread_init_static_tls;
 #endif
 
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index 9c15259236..02bd579ef8 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -456,7 +456,7 @@ struct rtld_global
   /* Generation counter for the dtv.  */
   EXTERN size_t _dl_tls_generation;
 
-#if !THREAD_GSCOPE_IN_TCB
+#if !PTHREAD_IN_LIBC
   EXTERN void (*_dl_init_static_tls) (struct link_map *);
 #endif
 
@@ -466,7 +466,7 @@ struct rtld_global
     size_t count;
     void *list[50];
   } *_dl_scope_free_list;
-#if THREAD_GSCOPE_IN_TCB
+#if PTHREAD_IN_LIBC
   /* List of active thread stacks, with memory managed by glibc.  */
   EXTERN list_t _dl_stack_used;
 
@@ -486,7 +486,8 @@ struct rtld_global
 
   /* Mutex protecting the stack lists.  */
   EXTERN int _dl_stack_cache_lock;
-#else
+#endif
+#if !THREAD_GSCOPE_IN_TCB
   EXTERN int _dl_thread_gscope_count;
 #endif
 #ifdef SHARED
@@ -1297,13 +1298,13 @@ extern void _dl_aux_init (ElfW(auxv_t) *av)
 
 /* Initialize the static TLS space for the link map in all existing
    threads. */
-#if THREAD_GSCOPE_IN_TCB
+#if PTHREAD_IN_LIBC
 void _dl_init_static_tls (struct link_map *map) attribute_hidden;
 #endif
 static inline void
 dl_init_static_tls (struct link_map *map)
 {
-#if THREAD_GSCOPE_IN_TCB
+#if PTHREAD_IN_LIBC
   /* The stack list is available to ld.so, so the initialization can
      be handled within ld.so directly.  */
   _dl_init_static_tls (map);
-- 
2.31.1

[PATCH htl v3 2/5] htl: Move thread table to ld.so

[Sergey Bugaev via Libc-alpha]

The next commit is going to introduce a new implementation of
THREAD_GSCOPE_WAIT which needs to access the list of threads.
Since it must be usable from the dynamic laoder, we have to move
the symbols for the list of threads into the loader.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 htl/Versions                             |  2 -
 htl/pt-alloc.c                           | 50 ++++++++++--------------
 htl/pt-create.c                          | 11 ++----
 htl/pt-internal.h                        | 23 ++++-------
 sysdeps/generic/ldsodefs.h               |  9 +++++
 sysdeps/htl/dl-support.c                 | 23 +++++++++++
 sysdeps/htl/pt-key-delete.c              |  8 ++--
 sysdeps/htl/pthreadP.h                   |  2 -
 sysdeps/htl/raise.c                      |  8 +++-
 sysdeps/htl/thrd_current.c               |  7 +++-
 sysdeps/mach/hurd/htl/pt-sigstate-init.c |  2 +-
 sysdeps/mach/hurd/htl/pt-sysdep.c        |  2 +-
 sysdeps/mach/hurd/htl/pt-sysdep.h        |  2 +-
 13 files changed, 82 insertions(+), 67 deletions(-)
 create mode 100644 sysdeps/htl/dl-support.c

diff --git a/htl/Versions b/htl/Versions
index 4aea321016..4e0ebac285 100644
--- a/htl/Versions
+++ b/htl/Versions
@@ -168,8 +168,6 @@ libpthread {
   GLIBC_PRIVATE {
     __pthread_initialize_minimal;
 
-    __pthread_threads;
-
     __cthread_detach;
     __cthread_fork;
     __pthread_detach;
diff --git a/htl/pt-alloc.c b/htl/pt-alloc.c
index acc67f2711..f6708a60bf 100644
--- a/htl/pt-alloc.c
+++ b/htl/pt-alloc.c
@@ -28,19 +28,9 @@
    of the threads functions "shall fail" if "No thread could be found
    corresponding to that specified by the given thread ID."  */
 
-/* Thread ID lookup table.  */
-struct __pthread **__pthread_threads;
-
 /* The size of the thread ID lookup table.  */
 int __pthread_max_threads;
 
-/* The total number of thread IDs currently in use, or on the list of
-   available thread IDs.  */
-int __pthread_num_threads;
-
-/* A lock for the table, and the other variables above.  */
-pthread_rwlock_t __pthread_threads_lock;
-
 /* List of thread structures corresponding to free thread IDs.  */
 struct __pthread *__pthread_free_threads;
 pthread_mutex_t __pthread_free_threads_lock;
@@ -132,25 +122,25 @@ __pthread_alloc (struct __pthread **pthread)
     }
 
 retry:
-  __pthread_rwlock_wrlock (&__pthread_threads_lock);
+  lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
-  if (__pthread_num_threads < __pthread_max_threads)
+  if (GL (dl_pthread_num_threads) < __pthread_max_threads)
     {
       /* We have a free slot.  Use the slot number plus one as the
          thread ID for the new thread.  */
-      new->thread = 1 + __pthread_num_threads++;
-      __pthread_threads[new->thread - 1] = NULL;
+      new->thread = 1 + GL (dl_pthread_num_threads)++;
+      GL (dl_pthread_threads)[new->thread - 1] = NULL;
 
-      __pthread_rwlock_unlock (&__pthread_threads_lock);
+      lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
       *pthread = new;
       return 0;
     }
 #ifdef PTHREAD_THREADS_MAX
-  else if (__pthread_num_threads >= PTHREAD_THREADS_MAX)
+  else if (GL (dl_pthread_num_threads) >= PTHREAD_THREADS_MAX)
     {
       /* We have reached the limit on the number of threads per process.  */
-      __pthread_rwlock_unlock (&__pthread_threads_lock);
+      lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
       free (new);
       return EAGAIN;
@@ -162,7 +152,7 @@ retry:
      memory allocation, since that's a potentially blocking operation.  */
   max_threads = __pthread_max_threads;
 
-  __pthread_rwlock_unlock (&__pthread_threads_lock);
+  lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
   /* Allocate a new lookup table that's twice as large.  */
   new_max_threads
@@ -174,13 +164,13 @@ retry:
       return ENOMEM;
     }
 
-  __pthread_rwlock_wrlock (&__pthread_threads_lock);
+  lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
   /* Check if nobody else has already enlarged the table.  */
   if (max_threads != __pthread_max_threads)
     {
       /* Yep, they did.  */
-      __pthread_rwlock_unlock (&__pthread_threads_lock);
+      lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
       /* Free the newly allocated table and try again to allocate a slot.  */
       free (threads);
@@ -188,22 +178,22 @@ retry:
     }
 
   /* Copy over the contents of the old table.  */
-  memcpy (threads, __pthread_threads,
+  memcpy (threads, GL (dl_pthread_threads),
 	  __pthread_max_threads * sizeof (struct __pthread *));
 
   /* Save the location of the old table.  We want to deallocate its
      storage after we released the lock.  */
-  old_threads = __pthread_threads;
+  old_threads = GL (dl_pthread_threads);
 
   /* Replace the table with the new one.  */
   __pthread_max_threads = new_max_threads;
-  __pthread_threads = threads;
+  GL (dl_pthread_threads) = threads;
 
   /* And allocate ourselves one of the newly created slots.  */
-  new->thread = 1 + __pthread_num_threads++;
-  __pthread_threads[new->thread - 1] = NULL;
+  new->thread = 1 + GL (dl_pthread_num_threads)++;
+  GL (dl_pthread_threads)[new->thread - 1] = NULL;
 
-  __pthread_rwlock_unlock (&__pthread_threads_lock);
+  lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
   free (old_threads);
 
@@ -217,10 +207,10 @@ __pthread_init_static_tls (struct link_map *map)
 {
   int i;
 
-  __pthread_rwlock_wrlock (&__pthread_threads_lock);
-  for (i = 0; i < __pthread_num_threads; ++i)
+  lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
+  for (i = 0; i < GL (dl_pthread_num_threads); ++i)
     {
-      struct __pthread *t = __pthread_threads[i];
+      struct __pthread *t = GL (dl_pthread_threads)[i];
 
       if (t == NULL)
 	continue;
@@ -237,5 +227,5 @@ __pthread_init_static_tls (struct link_map *map)
       memset (__mempcpy (dest, map->l_tls_initimage, map->l_tls_initimage_size),
 	      '\0', map->l_tls_blocksize - map->l_tls_initimage_size);
     }
-  __pthread_rwlock_unlock (&__pthread_threads_lock);
+  lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 }
diff --git a/htl/pt-create.c b/htl/pt-create.c
index f0a1f1ab09..4e16cbbdc9 100644
--- a/htl/pt-create.c
+++ b/htl/pt-create.c
@@ -207,7 +207,7 @@ __pthread_create_internal (struct __pthread **thread,
      creating thread.  The set of signals pending for the new thread
      shall be empty."  If the currnet thread is not a pthread then we
      just inherit the process' sigmask.  */
-  if (__pthread_num_threads == 1)
+  if (GL (dl_pthread_num_threads) == 1)
     err = __sigprocmask (0, 0, &pthread->init_sigset);
   else
     err = __pthread_sigstate (_pthread_self (), 0, 0, &pthread->init_sigset, 0);
@@ -227,13 +227,8 @@ __pthread_create_internal (struct __pthread **thread,
      new thread runs.  */
   atomic_increment (&__pthread_total);
 
-  /* Store a pointer to this thread in the thread ID lookup table.  We
-     could use __thread_setid, however, we only lock for reading as no
-     other thread should be using this entry (we also assume that the
-     store is atomic).  */
-  __pthread_rwlock_rdlock (&__pthread_threads_lock);
-  __pthread_threads[pthread->thread - 1] = pthread;
-  __pthread_rwlock_unlock (&__pthread_threads_lock);
+  /* Store a pointer to this thread in the thread ID lookup table.  */
+  __pthread_setid (pthread->thread, pthread);
 
   /* At this point it is possible to guess our pthread ID.  We have to
      make sure that all functions taking a pthread_t argument can
diff --git a/htl/pt-internal.h b/htl/pt-internal.h
index 34e6da338e..7c16f8744d 100644
--- a/htl/pt-internal.h
+++ b/htl/pt-internal.h
@@ -166,33 +166,24 @@ __pthread_dequeue (struct __pthread *thread)
 /* The total number of threads currently active.  */
 extern unsigned int __pthread_total;
 
-/* The total number of thread IDs currently in use, or on the list of
-   available thread IDs.  */
-extern int __pthread_num_threads;
-
 /* Concurrency hint.  */
 extern int __pthread_concurrency;
 
-/* Array of __pthread structures and its lock.  Indexed by the pthread
-   id minus one.  (Why not just use the pthread id?  Because some
-   brain-dead users of the pthread interface incorrectly assume that 0
-   is an invalid pthread id.)  */
-extern struct __pthread **__pthread_threads;
+/* The size of the thread ID lookup table.  */
 extern int __pthread_max_threads;
-extern pthread_rwlock_t __pthread_threads_lock;
 
 #define __pthread_getid(thread) \
   ({ struct __pthread *__t = NULL;                                           \
-     __pthread_rwlock_rdlock (&__pthread_threads_lock);                      \
+     lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);                   \
      if (thread <= __pthread_max_threads)                                    \
-       __t = __pthread_threads[thread - 1];                                  \
-     __pthread_rwlock_unlock (&__pthread_threads_lock);                      \
+       __t = GL (dl_pthread_threads)[thread - 1];                            \
+     lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);                 \
      __t; })
 
 #define __pthread_setid(thread, pthread) \
-  __pthread_rwlock_wrlock (&__pthread_threads_lock);                         \
-  __pthread_threads[thread - 1] = pthread;                                   \
-  __pthread_rwlock_unlock (&__pthread_threads_lock);
+  lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);                      \
+  GL (dl_pthread_threads)[thread - 1] = pthread;                             \
+  lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
 
 /* Similar to pthread_self, but returns the thread descriptor instead
    of the thread ID.  */
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index 02bd579ef8..3092bbc924 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -486,7 +486,16 @@ struct rtld_global
 
   /* Mutex protecting the stack lists.  */
   EXTERN int _dl_stack_cache_lock;
+#else
+  /* The total number of thread IDs currently in use, or on the list of
+     available thread IDs.  */
+  EXTERN int _dl_pthread_num_threads;
+
+  /* Array of __pthread structures and its lock.  */
+  EXTERN struct __pthread **_dl_pthread_threads;
+  EXTERN int _dl_pthread_threads_lock;
 #endif
+
 #if !THREAD_GSCOPE_IN_TCB
   EXTERN int _dl_thread_gscope_count;
 #endif
diff --git a/sysdeps/htl/dl-support.c b/sysdeps/htl/dl-support.c
new file mode 100644
index 0000000000..21a391a59f
--- /dev/null
+++ b/sysdeps/htl/dl-support.c
@@ -0,0 +1,23 @@
+/* Support for dynamic linking code in static libc.
+   Copyright (C) 2007-2021 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <elf/dl-support.c>
+
+int _dl_pthread_num_threads;
+struct __pthread **_dl_pthread_threads;
+int _dl_pthread_threads_lock;
diff --git a/sysdeps/htl/pt-key-delete.c b/sysdeps/htl/pt-key-delete.c
index 4e77ef7521..4acd8dd755 100644
--- a/sysdeps/htl/pt-key-delete.c
+++ b/sysdeps/htl/pt-key-delete.c
@@ -39,12 +39,12 @@ __pthread_key_delete (pthread_key_t key)
       __pthread_key_destructors[key] = PTHREAD_KEY_INVALID;
       __pthread_key_invalid_count++;
 
-      __pthread_rwlock_rdlock (&__pthread_threads_lock);
-      for (i = 0; i < __pthread_num_threads; ++i)
+      lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
+      for (i = 0; i < GL (dl_pthread_num_threads); ++i)
 	{
 	  struct __pthread *t;
 
-	  t = __pthread_threads[i];
+	  t = GL (dl_pthread_threads)[i];
 
 	  if (t == NULL)
 	    continue;
@@ -54,7 +54,7 @@ __pthread_key_delete (pthread_key_t key)
 	  if (key < t->thread_specifics_size)
 	    t->thread_specifics[key] = 0;
 	}
-      __pthread_rwlock_unlock (&__pthread_threads_lock);
+      lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
     }
 
   __pthread_mutex_unlock (&__pthread_key_lock);
diff --git a/sysdeps/htl/pthreadP.h b/sysdeps/htl/pthreadP.h
index 3fd774269f..8b02bef355 100644
--- a/sysdeps/htl/pthreadP.h
+++ b/sysdeps/htl/pthreadP.h
@@ -31,8 +31,6 @@ extern void __pthread_init_static_tls (struct link_map *) attribute_hidden;
 
 /* These represent the interface used by glibc itself.  */
 
-extern struct __pthread **__pthread_threads;
-
 extern int __pthread_mutex_init (pthread_mutex_t *__mutex, const pthread_mutexattr_t *__attr);
 extern int __pthread_mutex_destroy (pthread_mutex_t *__mutex);
 extern int __pthread_mutex_lock (pthread_mutex_t *__mutex);
diff --git a/sysdeps/htl/raise.c b/sysdeps/htl/raise.c
index 5a0f37d3b8..8c17db6f65 100644
--- a/sysdeps/htl/raise.c
+++ b/sysdeps/htl/raise.c
@@ -17,6 +17,7 @@
    License along with this program.  If not, see
    <https://www.gnu.org/licenses/>.  */
 
+#include <ldsodefs.h>
 #include <pthreadP.h>
 #include <signal.h>
 #include <unistd.h>
@@ -24,6 +25,11 @@
 #pragma weak __pthread_kill
 #pragma weak __pthread_self
 #pragma weak __pthread_threads
+
+#ifndef SHARED
+#pragma weak _dl_pthread_threads
+#endif
+
 int
 raise (int signo)
 {
@@ -31,7 +37,7 @@ raise (int signo)
      "the effect of the raise() function shall be equivalent to
      calling: pthread_kill(pthread_self(), sig);"  */
 
-  if (__pthread_kill != NULL && __pthread_threads != NULL)
+  if (__pthread_kill != NULL && GL (dl_pthread_threads) != NULL)
     {
       int err;
       err = __pthread_kill (__pthread_self (), signo);
diff --git a/sysdeps/htl/thrd_current.c b/sysdeps/htl/thrd_current.c
index eecb86af85..87359677d6 100644
--- a/sysdeps/htl/thrd_current.c
+++ b/sysdeps/htl/thrd_current.c
@@ -17,14 +17,19 @@
    <https://www.gnu.org/licenses/>.  */
 
 #include "thrd_priv.h"
+#include <ldsodefs.h>
 
 #pragma weak __pthread_self
 #pragma weak __pthread_threads
 
+#ifndef SHARED
+#pragma weak _dl_pthread_threads
+#endif
+
 thrd_t
 thrd_current (void)
 {
-  if (__pthread_threads)
+  if (GL (dl_pthread_threads))
     return (thrd_t) __pthread_self ();
 
   return (thrd_t) 0;
diff --git a/sysdeps/mach/hurd/htl/pt-sigstate-init.c b/sysdeps/mach/hurd/htl/pt-sigstate-init.c
index 399068ca78..6f43ade653 100644
--- a/sysdeps/mach/hurd/htl/pt-sigstate-init.c
+++ b/sysdeps/mach/hurd/htl/pt-sigstate-init.c
@@ -37,7 +37,7 @@ __pthread_sigstate_init (struct __pthread *thread)
       struct hurd_sigstate *ss = _hurd_thread_sigstate (thread->kernel_thread);
       _hurd_sigstate_set_global_rcv (ss);
     }
-  else if (__pthread_num_threads >= 2)
+  else if (GL (dl_pthread_num_threads) >= 2)
     do_init_global = 1;
 
   return 0;
diff --git a/sysdeps/mach/hurd/htl/pt-sysdep.c b/sysdeps/mach/hurd/htl/pt-sysdep.c
index 03ff8079ba..2fd0a8d6e7 100644
--- a/sysdeps/mach/hurd/htl/pt-sysdep.c
+++ b/sysdeps/mach/hurd/htl/pt-sysdep.c
@@ -45,7 +45,7 @@ _init_routine (void *stack)
   int err;
   pthread_attr_t attr, *attrp = 0;
 
-  if (__pthread_threads != NULL)
+  if (GL (dl_pthread_threads) != NULL)
     /* Already initialized */
     return;
 
diff --git a/sysdeps/mach/hurd/htl/pt-sysdep.h b/sysdeps/mach/hurd/htl/pt-sysdep.h
index acd4fce9c4..d90563af0c 100644
--- a/sysdeps/mach/hurd/htl/pt-sysdep.h
+++ b/sysdeps/mach/hurd/htl/pt-sysdep.h
@@ -37,7 +37,7 @@ extern __thread struct __pthread *___pthread_self;
 	({                                                         \
 	  struct __pthread *thread;                                \
 	                                                           \
-	  assert (__pthread_threads);                              \
+	  assert (GL (dl_pthread_threads));                        \
 	  thread = ___pthread_self;                                \
 	                                                           \
 	  assert (thread);                                         \
-- 
2.31.1

[PATCH htl v3 3/5] htl: Reimplement GSCOPE

[Sergey Bugaev via Libc-alpha]

This is a new implementation of GSCOPE which largely mirrors its NPTL
counterpart. Same as in NPTL, instead of a global flag shared between
threads, there is now a per-thread GSCOPE flag stored in each thread's
TCB. This makes entering and exiting a GSCOPE faster at the expense of
making THREAD_GSCOPE_WAIT () slower.

The largest win is the elimination of many redundant gsync_wake () RPC
calls; previously, even simplest programs would make dozens of fully
redundant gsync_wake () calls.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 sysdeps/htl/dl-thread_gscope_wait.c | 55 +++++++++++++++++++++++++++++
 sysdeps/mach/hurd/i386/tls.h        | 21 +++++++++++
 sysdeps/mach/hurd/tls.h             | 20 -----------
 3 files changed, 76 insertions(+), 20 deletions(-)
 create mode 100644 sysdeps/htl/dl-thread_gscope_wait.c

diff --git a/sysdeps/htl/dl-thread_gscope_wait.c b/sysdeps/htl/dl-thread_gscope_wait.c
new file mode 100644
index 0000000000..b277217b8e
--- /dev/null
+++ b/sysdeps/htl/dl-thread_gscope_wait.c
@@ -0,0 +1,55 @@
+/* Out-of-line notification function for the GSCOPE locking mechanism.
+   Copyright (C) 2007-2021 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <ldsodefs.h>
+#include <pthread.h>
+#include <htl/pt-internal.h>
+
+void
+__thread_gscope_wait (void)
+{
+  size_t i;
+  struct __pthread *t;
+  int *gscope_flagp;
+
+  lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
+
+  /* Iterate over the list of threads.  */
+  for (i = 0; i < GL (dl_pthread_num_threads); ++i)
+    {
+      t = GL (dl_pthread_threads[i]);
+      if (t == NULL || t->tcb->gscope_flag == THREAD_GSCOPE_FLAG_UNUSED)
+        continue;
+
+      gscope_flagp = &t->tcb->gscope_flag;
+
+      /* We have to wait until this thread is done with the global
+         scope.  First tell the thread that we are waiting and
+         possibly have to be woken.  */
+      if (atomic_compare_and_exchange_bool_acq (gscope_flagp,
+                                                THREAD_GSCOPE_FLAG_WAIT,
+                                                THREAD_GSCOPE_FLAG_USED))
+        continue;
+
+      do
+        lll_wait (gscope_flagp, THREAD_GSCOPE_FLAG_WAIT, LLL_PRIVATE);
+      while (*gscope_flagp == THREAD_GSCOPE_FLAG_WAIT);
+    }
+
+  lll_unlock (GL (dl_pthread_threads_lock), LLL_PRIVATE);
+}
diff --git a/sysdeps/mach/hurd/i386/tls.h b/sysdeps/mach/hurd/i386/tls.h
index 057b2613f3..2ac65df921 100644
--- a/sysdeps/mach/hurd/i386/tls.h
+++ b/sysdeps/mach/hurd/i386/tls.h
@@ -369,6 +369,27 @@ _hurd_tls_new (thread_t child, struct i386_thread_state *state, tcbhead_t *tcb)
   return err;
 }
 
+/* Global scope switch support.  */
+# define THREAD_GSCOPE_IN_TCB      1
+
+# define THREAD_GSCOPE_FLAG_UNUSED 0
+# define THREAD_GSCOPE_FLAG_USED   1
+# define THREAD_GSCOPE_FLAG_WAIT   2
+
+# define THREAD_GSCOPE_SET_FLAG() \
+  THREAD_SETMEM (THREAD_SELF, gscope_flag, THREAD_GSCOPE_FLAG_USED)
+
+# define THREAD_GSCOPE_RESET_FLAG() \
+  ({                                                                         \
+    int __flag;                                                              \
+    asm volatile ("xchgl %0, %%gs:%P1"                                       \
+                  : "=r" (__flag)                                            \
+                  : "i" (offsetof (tcbhead_t, gscope_flag)),                 \
+                    "0" (THREAD_GSCOPE_FLAG_UNUSED));                        \
+    if (__flag == THREAD_GSCOPE_FLAG_WAIT)                                   \
+      lll_wake (THREAD_SELF->gscope_flag, LLL_PRIVATE);                      \
+  })
+
 #endif	/* !__ASSEMBLER__ */
 
 #endif	/* i386/tls.h */
diff --git a/sysdeps/mach/hurd/tls.h b/sysdeps/mach/hurd/tls.h
index f83956d3d7..8e66d5ff53 100644
--- a/sysdeps/mach/hurd/tls.h
+++ b/sysdeps/mach/hurd/tls.h
@@ -52,26 +52,6 @@
 # define GET_DTV(descr) \
   (((tcbhead_t *) (descr))->dtv)
 
-/* Global scope switch support.  */
-#define THREAD_GSCOPE_IN_TCB      0
-#define THREAD_GSCOPE_GLOBAL
-#define THREAD_GSCOPE_SET_FLAG() \
-  atomic_exchange_and_add_acq (&GL(dl_thread_gscope_count), 1)
-#define THREAD_GSCOPE_RESET_FLAG() \
-  do 									      \
-    if (atomic_exchange_and_add_rel (&GL(dl_thread_gscope_count), -1) == 1)   \
-      lll_wake (GL(dl_thread_gscope_count), 0);				      \
-  while (0)
-#define THREAD_GSCOPE_WAIT() \
-  do 									      \
-    {									      \
-      int count;							      \
-      atomic_write_barrier ();						      \
-      while ((count = GL(dl_thread_gscope_count)))			      \
-        lll_wait (GL(dl_thread_gscope_count), count, 0);		      \
-    }									      \
-  while (0)
-
 #endif /* !ASSEMBLER */
 
 
-- 
2.31.1

[PATCH htl v3 4/5] elf: Remove THREAD_GSCOPE_IN_TCB

[Sergey Bugaev via Libc-alpha]

All the ports now have THREAD_GSCOPE_IN_TCB set to 1. Remove all
support for !THREAD_GSCOPE_IN_TCB, along with the definition itself.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 elf/dl-support.c              | 3 ---
 sysdeps/aarch64/nptl/tls.h    | 1 -
 sysdeps/alpha/nptl/tls.h      | 1 -
 sysdeps/arc/nptl/tls.h        | 1 -
 sysdeps/arm/nptl/tls.h        | 1 -
 sysdeps/csky/nptl/tls.h       | 1 -
 sysdeps/generic/ldsodefs.h    | 6 ------
 sysdeps/generic/tls.h         | 6 ------
 sysdeps/hppa/nptl/tls.h       | 1 -
 sysdeps/i386/nptl/tls.h       | 1 -
 sysdeps/ia64/nptl/tls.h       | 1 -
 sysdeps/m68k/nptl/tls.h       | 1 -
 sysdeps/mach/hurd/i386/tls.h  | 2 --
 sysdeps/microblaze/nptl/tls.h | 1 -
 sysdeps/mips/nptl/tls.h       | 1 -
 sysdeps/nios2/nptl/tls.h      | 1 -
 sysdeps/powerpc/nptl/tls.h    | 1 -
 sysdeps/riscv/nptl/tls.h      | 1 -
 sysdeps/s390/nptl/tls.h       | 1 -
 sysdeps/sh/nptl/tls.h         | 1 -
 sysdeps/sparc/nptl/tls.h      | 1 -
 sysdeps/x86_64/nptl/tls.h     | 1 -
 22 files changed, 35 deletions(-)

diff --git a/elf/dl-support.c b/elf/dl-support.c
index a317459fae..02e2ed72f5 100644
--- a/elf/dl-support.c
+++ b/elf/dl-support.c
@@ -197,9 +197,6 @@ int _dl_stack_cache_lock;
 int (*_dl_make_stack_executable_hook) (void **) = _dl_make_stack_executable;
 void (*_dl_init_static_tls) (struct link_map *) = &_dl_nothread_init_static_tls;
 #endif
-#if !THREAD_GSCOPE_IN_TCB
-int _dl_thread_gscope_count;
-#endif
 struct dl_scope_free_list *_dl_scope_free_list;
 
 #ifdef NEED_DL_SYSINFO
diff --git a/sysdeps/aarch64/nptl/tls.h b/sysdeps/aarch64/nptl/tls.h
index 6e896207a6..72f22dc718 100644
--- a/sysdeps/aarch64/nptl/tls.h
+++ b/sysdeps/aarch64/nptl/tls.h
@@ -109,7 +109,6 @@ typedef struct
   descr->member[idx] = (value)
 
 /* Get and set the global scope generation counter in struct pthread.  */
-# define THREAD_GSCOPE_IN_TCB      1
 # define THREAD_GSCOPE_FLAG_UNUSED 0
 # define THREAD_GSCOPE_FLAG_USED   1
 # define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/alpha/nptl/tls.h b/sysdeps/alpha/nptl/tls.h
index 4dbccc5249..6328112135 100644
--- a/sysdeps/alpha/nptl/tls.h
+++ b/sysdeps/alpha/nptl/tls.h
@@ -103,7 +103,6 @@ typedef struct
   descr->member[idx] = (value)
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/arc/nptl/tls.h b/sysdeps/arc/nptl/tls.h
index 95300fdd21..e269c0a7a5 100644
--- a/sysdeps/arc/nptl/tls.h
+++ b/sysdeps/arc/nptl/tls.h
@@ -111,7 +111,6 @@ typedef struct
   descr->member[idx] = (value)
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/arm/nptl/tls.h b/sysdeps/arm/nptl/tls.h
index 1bd11307ce..699c16acfb 100644
--- a/sysdeps/arm/nptl/tls.h
+++ b/sysdeps/arm/nptl/tls.h
@@ -100,7 +100,6 @@ typedef struct
   descr->member[idx] = (value)
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/csky/nptl/tls.h b/sysdeps/csky/nptl/tls.h
index 7a234041ed..b210dfcb76 100644
--- a/sysdeps/csky/nptl/tls.h
+++ b/sysdeps/csky/nptl/tls.h
@@ -127,7 +127,6 @@ typedef struct
   descr->member[idx] = (value)
 
 /* Get and set the global scope generation counter in struct pthread.  */
-# define THREAD_GSCOPE_IN_TCB      1
 # define THREAD_GSCOPE_FLAG_UNUSED 0
 # define THREAD_GSCOPE_FLAG_USED   1
 # define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index 3092bbc924..b251fd1733 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -495,10 +495,6 @@ struct rtld_global
   EXTERN struct __pthread **_dl_pthread_threads;
   EXTERN int _dl_pthread_threads_lock;
 #endif
-
-#if !THREAD_GSCOPE_IN_TCB
-  EXTERN int _dl_thread_gscope_count;
-#endif
 #ifdef SHARED
 };
 # define __rtld_global_attribute__
@@ -1381,10 +1377,8 @@ __rtld_mutex_init (void)
 }
 #endif /* !PTHREAD_IN_LIBC */
 
-#if THREAD_GSCOPE_IN_TCB
 void __thread_gscope_wait (void) attribute_hidden;
 # define THREAD_GSCOPE_WAIT() __thread_gscope_wait ()
-#endif
 
 __END_DECLS
 
diff --git a/sysdeps/generic/tls.h b/sysdeps/generic/tls.h
index e86d70e6ce..f581c9a992 100644
--- a/sysdeps/generic/tls.h
+++ b/sysdeps/generic/tls.h
@@ -71,10 +71,4 @@
      This macro returns the address of the DTV of the current thread.
      This normally is done using the thread register which points
      to the dtv or the TCB (from which the DTV can found).
-
-
-     THREAD_GSCOPE_IN_TCB
-
-     This should be set to 1 if the global scope flag is stored within the TCB.
-     When set to 0, GL(_dl_thread_gscope_count) will be defined to store it.
   */
diff --git a/sysdeps/hppa/nptl/tls.h b/sysdeps/hppa/nptl/tls.h
index 857003a7d3..55559eb327 100644
--- a/sysdeps/hppa/nptl/tls.h
+++ b/sysdeps/hppa/nptl/tls.h
@@ -134,7 +134,6 @@ static inline void __set_cr27(struct pthread *cr27)
 }
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/i386/nptl/tls.h b/sysdeps/i386/nptl/tls.h
index 86ee1ef302..cfb27f5ccd 100644
--- a/sysdeps/i386/nptl/tls.h
+++ b/sysdeps/i386/nptl/tls.h
@@ -375,7 +375,6 @@ tls_fill_user_desc (union user_desc_init *desc,
 
 
 /* Get and set the global scope generation counter in the TCB head.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/ia64/nptl/tls.h b/sysdeps/ia64/nptl/tls.h
index 66d9bf3189..8c26728859 100644
--- a/sysdeps/ia64/nptl/tls.h
+++ b/sysdeps/ia64/nptl/tls.h
@@ -155,7 +155,6 @@ register struct pthread *__thread_self __asm__("r13");
    = THREAD_GET_POINTER_GUARD ())
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/m68k/nptl/tls.h b/sysdeps/m68k/nptl/tls.h
index dd526cdc2e..34906b1c13 100644
--- a/sysdeps/m68k/nptl/tls.h
+++ b/sysdeps/m68k/nptl/tls.h
@@ -132,7 +132,6 @@ extern void * __m68k_read_tp (void);
 # define NO_TLS_OFFSET		-1
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/mach/hurd/i386/tls.h b/sysdeps/mach/hurd/i386/tls.h
index 2ac65df921..c70ea73a81 100644
--- a/sysdeps/mach/hurd/i386/tls.h
+++ b/sysdeps/mach/hurd/i386/tls.h
@@ -370,8 +370,6 @@ _hurd_tls_new (thread_t child, struct i386_thread_state *state, tcbhead_t *tcb)
 }
 
 /* Global scope switch support.  */
-# define THREAD_GSCOPE_IN_TCB      1
-
 # define THREAD_GSCOPE_FLAG_UNUSED 0
 # define THREAD_GSCOPE_FLAG_USED   1
 # define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/microblaze/nptl/tls.h b/sysdeps/microblaze/nptl/tls.h
index c93d90b11b..0ca67a777d 100644
--- a/sysdeps/microblaze/nptl/tls.h
+++ b/sysdeps/microblaze/nptl/tls.h
@@ -116,7 +116,6 @@ typedef struct
   (descr->member[idx] = (value))
 
 /* Get and set the global scope generation counter in struct pthread.  */
-# define THREAD_GSCOPE_IN_TCB      1
 # define THREAD_GSCOPE_FLAG_UNUSED 0
 # define THREAD_GSCOPE_FLAG_USED   1
 # define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/mips/nptl/tls.h b/sysdeps/mips/nptl/tls.h
index c09f49071c..04e823b4c7 100644
--- a/sysdeps/mips/nptl/tls.h
+++ b/sysdeps/mips/nptl/tls.h
@@ -158,7 +158,6 @@ typedef struct
 # define NO_TLS_OFFSET		-1
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/nios2/nptl/tls.h b/sysdeps/nios2/nptl/tls.h
index 02a05b4e74..fd484135f4 100644
--- a/sysdeps/nios2/nptl/tls.h
+++ b/sysdeps/nios2/nptl/tls.h
@@ -137,7 +137,6 @@ register struct pthread *__thread_self __asm__("r23");
 # define NO_TLS_OFFSET		-1
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/powerpc/nptl/tls.h b/sysdeps/powerpc/nptl/tls.h
index 6c779b6609..cc93c44964 100644
--- a/sysdeps/powerpc/nptl/tls.h
+++ b/sysdeps/powerpc/nptl/tls.h
@@ -231,7 +231,6 @@ typedef struct
 # define NO_TLS_OFFSET		-1
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/riscv/nptl/tls.h b/sysdeps/riscv/nptl/tls.h
index 5350bcc049..e4bd736feb 100644
--- a/sysdeps/riscv/nptl/tls.h
+++ b/sysdeps/riscv/nptl/tls.h
@@ -119,7 +119,6 @@ typedef struct
 # define NO_TLS_OFFSET		-1
 
 /* Get and set the global scope generation counter in struct pthread.  */
-# define THREAD_GSCOPE_IN_TCB      1
 # define THREAD_GSCOPE_FLAG_UNUSED 0
 # define THREAD_GSCOPE_FLAG_USED   1
 # define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/s390/nptl/tls.h b/sysdeps/s390/nptl/tls.h
index efb52515e0..804486dfdd 100644
--- a/sysdeps/s390/nptl/tls.h
+++ b/sysdeps/s390/nptl/tls.h
@@ -165,7 +165,6 @@ typedef struct
 #define THREAD_COPY_POINTER_GUARD(descr)
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/sh/nptl/tls.h b/sysdeps/sh/nptl/tls.h
index ac3c9a9e85..2a9ee1def1 100644
--- a/sysdeps/sh/nptl/tls.h
+++ b/sysdeps/sh/nptl/tls.h
@@ -141,7 +141,6 @@ typedef struct
      ((tcbhead_t *) (descr + 1))->pointer_guard	= __tcbp->pointer_guard;})
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/sparc/nptl/tls.h b/sysdeps/sparc/nptl/tls.h
index dd1eb82a59..55955f376a 100644
--- a/sysdeps/sparc/nptl/tls.h
+++ b/sysdeps/sparc/nptl/tls.h
@@ -138,7 +138,6 @@ register struct pthread *__thread_self __asm__("%g7");
   ((descr)->header.pointer_guard = THREAD_GET_POINTER_GUARD ())
 
 /* Get and set the global scope generation counter in struct pthread.  */
-#define THREAD_GSCOPE_IN_TCB      1
 #define THREAD_GSCOPE_FLAG_UNUSED 0
 #define THREAD_GSCOPE_FLAG_USED   1
 #define THREAD_GSCOPE_FLAG_WAIT   2
diff --git a/sysdeps/x86_64/nptl/tls.h b/sysdeps/x86_64/nptl/tls.h
index a78c4f4d01..b0d044353b 100644
--- a/sysdeps/x86_64/nptl/tls.h
+++ b/sysdeps/x86_64/nptl/tls.h
@@ -326,7 +326,6 @@ _Static_assert (offsetof (tcbhead_t, __glibc_unused2) == 0x80,
 
 
 /* Get and set the global scope generation counter in the TCB head.  */
-# define THREAD_GSCOPE_IN_TCB      1
 # define THREAD_GSCOPE_FLAG_UNUSED 0
 # define THREAD_GSCOPE_FLAG_USED   1
 # define THREAD_GSCOPE_FLAG_WAIT   2
-- 
2.31.1

[PATCH htl v3 5/5] testrun.sh: Add support for --tool=rpctrace

[Sergey Bugaev via Libc-alpha]

rpctrace(1) is a Hurd RPC tracer tool, which is used similar to how
strace(1) is used on GNU/Linux.

Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
---
 Makefile | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index f98d5a9e67..a49870d3d1 100644
--- a/Makefile
+++ b/Makefile
@@ -141,8 +141,9 @@ usage () {
 cat << EOF
 Usage: $$0 [OPTIONS] <program> [ARGUMENTS...]
 
-  --tool=TOOL  Run with the specified TOOL. It can be strace, valgrind or
-               container. The container will run within support/test-container.
+  --tool=TOOL  Run with the specified TOOL. It can be strace, rpctrace,
+               valgrind or container. The container will run within
+               support/test-container.
 EOF
 
   exit 1
@@ -177,6 +178,10 @@ case "$$toolname" in
     exec strace $(patsubst %, -E%, $(run-program-env)) \
       $(test-via-rtld-prefix) $${1+"$$@"}
     ;;
+  rpctrace)
+    exec rpctrace $(patsubst %, -E%, $(run-program-env)) \
+      $(test-via-rtld-prefix) $${1+"$$@"}
+    ;;
   valgrind)
     exec env $(run-program-env) valgrind $(test-via-rtld-prefix) $${1+"$$@"}
     ;;
-- 
2.31.1

Re: [PATCH htl v3 0/5] Rewrite THREAD_GSCOPE

[Adhemerval Zanella via Libc-alpha]

On 07/09/2021 10:33, Sergey Bugaev via Libc-alpha wrote:
> Hello,
> 
> this is yet another revision of the same patchset to stop issuing many
> redundant gsync_wake () calls. Following the brief feedback I got the last
> time, it switches most users of THREAD_GSCOPE_IN_TCB to PTHREAD_IN_LIBC, then
> actually introduces the new GSCOPE implementation, and then finally drops the
> remaining uses of THREAD_GSCOPE_IN_TCB entirely, since it's not always on.
> 
> Things still seem to build and work somewhat (tested on GNU/Hurd i686 and
> GNU/Linux x86_64). rpctrace uname still shows that there are no more
> gsync_wake () calls.
> 
> Apparently dlopen (RTLD_GLOBAL) is a good way to trigger THREAD_GSCOPE_WAIT ().
> I have written a test program that spawns 100 threads, then each of the threads
> dlopen's 100 shared objects, then dlclose's them back. On GNU/Linux, I have
> verified with GDB that it hits NPTL's __thread_gscope_wait (). I haven't been
> able to verify the same on the Hurd due to the general flakiness of GDB there,
> but I have ran the program with the patched glibc multiple times and it doesn't
> crash, so I assume it's working fine.

I can't voucher for Hurd changes, but the generic ones that might affect
Linux seems ok (the THREAD_GSCOPE_IN_TCB define is a nice cleanup).

Re: [PATCH htl v3 5/5] testrun.sh: Add support for --tool=rpctrace

[Florian Weimer via Libc-alpha]

* Sergey Bugaev:

> rpctrace(1) is a Hurd RPC tracer tool, which is used similar to how
> strace(1) is used on GNU/Linux.
>
> Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
> ---
>  Makefile | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index f98d5a9e67..a49870d3d1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -141,8 +141,9 @@ usage () {
>  cat << EOF
>  Usage: $$0 [OPTIONS] <program> [ARGUMENTS...]
>  
> -  --tool=TOOL  Run with the specified TOOL. It can be strace, valgrind or
> -               container. The container will run within support/test-container.
> +  --tool=TOOL  Run with the specified TOOL. It can be strace, rpctrace,
> +               valgrind or container. The container will run within
> +               support/test-container.
>  EOF
>  
>    exit 1
> @@ -177,6 +178,10 @@ case "$$toolname" in
>      exec strace $(patsubst %, -E%, $(run-program-env)) \
>        $(test-via-rtld-prefix) $${1+"$$@"}
>      ;;
> +  rpctrace)
> +    exec rpctrace $(patsubst %, -E%, $(run-program-env)) \
> +      $(test-via-rtld-prefix) $${1+"$$@"}
> +    ;;
>    valgrind)
>      exec env $(run-program-env) valgrind $(test-via-rtld-prefix) $${1+"$$@"}
>      ;;

Looks okay.  This can go in separately.

Thanks,
Florian

Re: [PATCH htl v3 5/5] testrun.sh: Add support for --tool=rpctrace

[Samuel Thibault via Libc-alpha]

Applied this patch, thanks!

Samuel

Florian Weimer, le mer. 08 sept. 2021 08:57:43 +0200, a ecrit:
> * Sergey Bugaev:
> 
> > rpctrace(1) is a Hurd RPC tracer tool, which is used similar to how
> > strace(1) is used on GNU/Linux.
> >
> > Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
> > ---
> >  Makefile | 9 +++++++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index f98d5a9e67..a49870d3d1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -141,8 +141,9 @@ usage () {
> >  cat << EOF
> >  Usage: $$0 [OPTIONS] <program> [ARGUMENTS...]
> >  
> > -  --tool=TOOL  Run with the specified TOOL. It can be strace, valgrind or
> > -               container. The container will run within support/test-container.
> > +  --tool=TOOL  Run with the specified TOOL. It can be strace, rpctrace,
> > +               valgrind or container. The container will run within
> > +               support/test-container.
> >  EOF
> >  
> >    exit 1
> > @@ -177,6 +178,10 @@ case "$$toolname" in
> >      exec strace $(patsubst %, -E%, $(run-program-env)) \
> >        $(test-via-rtld-prefix) $${1+"$$@"}
> >      ;;
> > +  rpctrace)
> > +    exec rpctrace $(patsubst %, -E%, $(run-program-env)) \
> > +      $(test-via-rtld-prefix) $${1+"$$@"}
> > +    ;;
> >    valgrind)
> >      exec env $(run-program-env) valgrind $(test-via-rtld-prefix) $${1+"$$@"}
> >      ;;
> 
> Looks okay.  This can go in separately.
> 
> Thanks,
> Florian

Re: [PATCH htl v3 2/5] htl: Move thread table to ld.so

[Samuel Thibault via Libc-alpha]

Hello,

Sergey Bugaev, le mar. 07 sept. 2021 16:33:22 +0300, a ecrit:
> diff --git a/htl/Versions b/htl/Versions
> index 4aea321016..4e0ebac285 100644
> --- a/htl/Versions
> +++ b/htl/Versions
> @@ -132,25 +122,25 @@ __pthread_alloc (struct __pthread **pthread)
>      }
>  
>  retry:
> -  __pthread_rwlock_wrlock (&__pthread_threads_lock);
> +  lll_lock (GL (dl_pthread_threads_lock), LLL_PRIVATE);

Please rather use __libc_rwlock_wrlock etc.
Even if for now they are actually locks and not rwlocks, the day we fix
that the rwlocking of dl_pthread_threads_lock will get fixed alongside.

> -  /* Store a pointer to this thread in the thread ID lookup table.  We
> -     could use __thread_setid, however, we only lock for reading as no
> -     other thread should be using this entry (we also assume that the
> -     store is atomic).  */
> -  __pthread_rwlock_rdlock (&__pthread_threads_lock);
> -  __pthread_threads[pthread->thread - 1] = pthread;
> -  __pthread_rwlock_unlock (&__pthread_threads_lock);
> +  /* Store a pointer to this thread in the thread ID lookup table.  */
> +  __pthread_setid (pthread->thread, pthread);

We still want to optimize for read-lock.

Apart from this it looks good, thanks!

Samuel

Re: [PATCH htl v3 3/5] htl: Reimplement GSCOPE

[Samuel Thibault via Libc-alpha]

Sergey Bugaev, le mar. 07 sept. 2021 16:33:23 +0300, a ecrit:
> This is a new implementation of GSCOPE which largely mirrors its NPTL
> counterpart. Same as in NPTL, instead of a global flag shared between
> threads, there is now a per-thread GSCOPE flag stored in each thread's
> TCB. This makes entering and exiting a GSCOPE faster at the expense of
> making THREAD_GSCOPE_WAIT () slower.
> 
> The largest win is the elimination of many redundant gsync_wake () RPC
> calls; previously, even simplest programs would make dozens of fully
> redundant gsync_wake () calls.

That looks good, thanks!

Samuel

Re: [PATCH htl v3 4/5] elf: Remove THREAD_GSCOPE_IN_TCB

[Samuel Thibault via Libc-alpha]

Sergey Bugaev, le mar. 07 sept. 2021 16:33:24 +0300, a ecrit:
> All the ports now have THREAD_GSCOPE_IN_TCB set to 1. Remove all
> support for !THREAD_GSCOPE_IN_TCB, along with the definition itself.
> 
> Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>

And thus a nice cleanup, thanks!

> ---
>  elf/dl-support.c              | 3 ---
>  sysdeps/aarch64/nptl/tls.h    | 1 -
>  sysdeps/alpha/nptl/tls.h      | 1 -
>  sysdeps/arc/nptl/tls.h        | 1 -
>  sysdeps/arm/nptl/tls.h        | 1 -
>  sysdeps/csky/nptl/tls.h       | 1 -
>  sysdeps/generic/ldsodefs.h    | 6 ------
>  sysdeps/generic/tls.h         | 6 ------
>  sysdeps/hppa/nptl/tls.h       | 1 -
>  sysdeps/i386/nptl/tls.h       | 1 -
>  sysdeps/ia64/nptl/tls.h       | 1 -
>  sysdeps/m68k/nptl/tls.h       | 1 -
>  sysdeps/mach/hurd/i386/tls.h  | 2 --
>  sysdeps/microblaze/nptl/tls.h | 1 -
>  sysdeps/mips/nptl/tls.h       | 1 -
>  sysdeps/nios2/nptl/tls.h      | 1 -
>  sysdeps/powerpc/nptl/tls.h    | 1 -
>  sysdeps/riscv/nptl/tls.h      | 1 -
>  sysdeps/s390/nptl/tls.h       | 1 -
>  sysdeps/sh/nptl/tls.h         | 1 -
>  sysdeps/sparc/nptl/tls.h      | 1 -
>  sysdeps/x86_64/nptl/tls.h     | 1 -
>  22 files changed, 35 deletions(-)
> 
> diff --git a/elf/dl-support.c b/elf/dl-support.c
> index a317459fae..02e2ed72f5 100644
> --- a/elf/dl-support.c
> +++ b/elf/dl-support.c
> @@ -197,9 +197,6 @@ int _dl_stack_cache_lock;
>  int (*_dl_make_stack_executable_hook) (void **) = _dl_make_stack_executable;
>  void (*_dl_init_static_tls) (struct link_map *) = &_dl_nothread_init_static_tls;
>  #endif
> -#if !THREAD_GSCOPE_IN_TCB
> -int _dl_thread_gscope_count;
> -#endif
>  struct dl_scope_free_list *_dl_scope_free_list;
>  
>  #ifdef NEED_DL_SYSINFO
> diff --git a/sysdeps/aarch64/nptl/tls.h b/sysdeps/aarch64/nptl/tls.h
> index 6e896207a6..72f22dc718 100644
> --- a/sysdeps/aarch64/nptl/tls.h
> +++ b/sysdeps/aarch64/nptl/tls.h
> @@ -109,7 +109,6 @@ typedef struct
>    descr->member[idx] = (value)
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -# define THREAD_GSCOPE_IN_TCB      1
>  # define THREAD_GSCOPE_FLAG_UNUSED 0
>  # define THREAD_GSCOPE_FLAG_USED   1
>  # define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/alpha/nptl/tls.h b/sysdeps/alpha/nptl/tls.h
> index 4dbccc5249..6328112135 100644
> --- a/sysdeps/alpha/nptl/tls.h
> +++ b/sysdeps/alpha/nptl/tls.h
> @@ -103,7 +103,6 @@ typedef struct
>    descr->member[idx] = (value)
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/arc/nptl/tls.h b/sysdeps/arc/nptl/tls.h
> index 95300fdd21..e269c0a7a5 100644
> --- a/sysdeps/arc/nptl/tls.h
> +++ b/sysdeps/arc/nptl/tls.h
> @@ -111,7 +111,6 @@ typedef struct
>    descr->member[idx] = (value)
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/arm/nptl/tls.h b/sysdeps/arm/nptl/tls.h
> index 1bd11307ce..699c16acfb 100644
> --- a/sysdeps/arm/nptl/tls.h
> +++ b/sysdeps/arm/nptl/tls.h
> @@ -100,7 +100,6 @@ typedef struct
>    descr->member[idx] = (value)
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/csky/nptl/tls.h b/sysdeps/csky/nptl/tls.h
> index 7a234041ed..b210dfcb76 100644
> --- a/sysdeps/csky/nptl/tls.h
> +++ b/sysdeps/csky/nptl/tls.h
> @@ -127,7 +127,6 @@ typedef struct
>    descr->member[idx] = (value)
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -# define THREAD_GSCOPE_IN_TCB      1
>  # define THREAD_GSCOPE_FLAG_UNUSED 0
>  # define THREAD_GSCOPE_FLAG_USED   1
>  # define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
> index 3092bbc924..b251fd1733 100644
> --- a/sysdeps/generic/ldsodefs.h
> +++ b/sysdeps/generic/ldsodefs.h
> @@ -495,10 +495,6 @@ struct rtld_global
>    EXTERN struct __pthread **_dl_pthread_threads;
>    EXTERN int _dl_pthread_threads_lock;
>  #endif
> -
> -#if !THREAD_GSCOPE_IN_TCB
> -  EXTERN int _dl_thread_gscope_count;
> -#endif
>  #ifdef SHARED
>  };
>  # define __rtld_global_attribute__
> @@ -1381,10 +1377,8 @@ __rtld_mutex_init (void)
>  }
>  #endif /* !PTHREAD_IN_LIBC */
>  
> -#if THREAD_GSCOPE_IN_TCB
>  void __thread_gscope_wait (void) attribute_hidden;
>  # define THREAD_GSCOPE_WAIT() __thread_gscope_wait ()
> -#endif
>  
>  __END_DECLS
>  
> diff --git a/sysdeps/generic/tls.h b/sysdeps/generic/tls.h
> index e86d70e6ce..f581c9a992 100644
> --- a/sysdeps/generic/tls.h
> +++ b/sysdeps/generic/tls.h
> @@ -71,10 +71,4 @@
>       This macro returns the address of the DTV of the current thread.
>       This normally is done using the thread register which points
>       to the dtv or the TCB (from which the DTV can found).
> -
> -
> -     THREAD_GSCOPE_IN_TCB
> -
> -     This should be set to 1 if the global scope flag is stored within the TCB.
> -     When set to 0, GL(_dl_thread_gscope_count) will be defined to store it.
>    */
> diff --git a/sysdeps/hppa/nptl/tls.h b/sysdeps/hppa/nptl/tls.h
> index 857003a7d3..55559eb327 100644
> --- a/sysdeps/hppa/nptl/tls.h
> +++ b/sysdeps/hppa/nptl/tls.h
> @@ -134,7 +134,6 @@ static inline void __set_cr27(struct pthread *cr27)
>  }
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/i386/nptl/tls.h b/sysdeps/i386/nptl/tls.h
> index 86ee1ef302..cfb27f5ccd 100644
> --- a/sysdeps/i386/nptl/tls.h
> +++ b/sysdeps/i386/nptl/tls.h
> @@ -375,7 +375,6 @@ tls_fill_user_desc (union user_desc_init *desc,
>  
>  
>  /* Get and set the global scope generation counter in the TCB head.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/ia64/nptl/tls.h b/sysdeps/ia64/nptl/tls.h
> index 66d9bf3189..8c26728859 100644
> --- a/sysdeps/ia64/nptl/tls.h
> +++ b/sysdeps/ia64/nptl/tls.h
> @@ -155,7 +155,6 @@ register struct pthread *__thread_self __asm__("r13");
>     = THREAD_GET_POINTER_GUARD ())
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/m68k/nptl/tls.h b/sysdeps/m68k/nptl/tls.h
> index dd526cdc2e..34906b1c13 100644
> --- a/sysdeps/m68k/nptl/tls.h
> +++ b/sysdeps/m68k/nptl/tls.h
> @@ -132,7 +132,6 @@ extern void * __m68k_read_tp (void);
>  # define NO_TLS_OFFSET		-1
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/mach/hurd/i386/tls.h b/sysdeps/mach/hurd/i386/tls.h
> index 2ac65df921..c70ea73a81 100644
> --- a/sysdeps/mach/hurd/i386/tls.h
> +++ b/sysdeps/mach/hurd/i386/tls.h
> @@ -370,8 +370,6 @@ _hurd_tls_new (thread_t child, struct i386_thread_state *state, tcbhead_t *tcb)
>  }
>  
>  /* Global scope switch support.  */
> -# define THREAD_GSCOPE_IN_TCB      1
> -
>  # define THREAD_GSCOPE_FLAG_UNUSED 0
>  # define THREAD_GSCOPE_FLAG_USED   1
>  # define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/microblaze/nptl/tls.h b/sysdeps/microblaze/nptl/tls.h
> index c93d90b11b..0ca67a777d 100644
> --- a/sysdeps/microblaze/nptl/tls.h
> +++ b/sysdeps/microblaze/nptl/tls.h
> @@ -116,7 +116,6 @@ typedef struct
>    (descr->member[idx] = (value))
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -# define THREAD_GSCOPE_IN_TCB      1
>  # define THREAD_GSCOPE_FLAG_UNUSED 0
>  # define THREAD_GSCOPE_FLAG_USED   1
>  # define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/mips/nptl/tls.h b/sysdeps/mips/nptl/tls.h
> index c09f49071c..04e823b4c7 100644
> --- a/sysdeps/mips/nptl/tls.h
> +++ b/sysdeps/mips/nptl/tls.h
> @@ -158,7 +158,6 @@ typedef struct
>  # define NO_TLS_OFFSET		-1
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/nios2/nptl/tls.h b/sysdeps/nios2/nptl/tls.h
> index 02a05b4e74..fd484135f4 100644
> --- a/sysdeps/nios2/nptl/tls.h
> +++ b/sysdeps/nios2/nptl/tls.h
> @@ -137,7 +137,6 @@ register struct pthread *__thread_self __asm__("r23");
>  # define NO_TLS_OFFSET		-1
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/powerpc/nptl/tls.h b/sysdeps/powerpc/nptl/tls.h
> index 6c779b6609..cc93c44964 100644
> --- a/sysdeps/powerpc/nptl/tls.h
> +++ b/sysdeps/powerpc/nptl/tls.h
> @@ -231,7 +231,6 @@ typedef struct
>  # define NO_TLS_OFFSET		-1
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/riscv/nptl/tls.h b/sysdeps/riscv/nptl/tls.h
> index 5350bcc049..e4bd736feb 100644
> --- a/sysdeps/riscv/nptl/tls.h
> +++ b/sysdeps/riscv/nptl/tls.h
> @@ -119,7 +119,6 @@ typedef struct
>  # define NO_TLS_OFFSET		-1
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -# define THREAD_GSCOPE_IN_TCB      1
>  # define THREAD_GSCOPE_FLAG_UNUSED 0
>  # define THREAD_GSCOPE_FLAG_USED   1
>  # define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/s390/nptl/tls.h b/sysdeps/s390/nptl/tls.h
> index efb52515e0..804486dfdd 100644
> --- a/sysdeps/s390/nptl/tls.h
> +++ b/sysdeps/s390/nptl/tls.h
> @@ -165,7 +165,6 @@ typedef struct
>  #define THREAD_COPY_POINTER_GUARD(descr)
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/sh/nptl/tls.h b/sysdeps/sh/nptl/tls.h
> index ac3c9a9e85..2a9ee1def1 100644
> --- a/sysdeps/sh/nptl/tls.h
> +++ b/sysdeps/sh/nptl/tls.h
> @@ -141,7 +141,6 @@ typedef struct
>       ((tcbhead_t *) (descr + 1))->pointer_guard	= __tcbp->pointer_guard;})
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/sparc/nptl/tls.h b/sysdeps/sparc/nptl/tls.h
> index dd1eb82a59..55955f376a 100644
> --- a/sysdeps/sparc/nptl/tls.h
> +++ b/sysdeps/sparc/nptl/tls.h
> @@ -138,7 +138,6 @@ register struct pthread *__thread_self __asm__("%g7");
>    ((descr)->header.pointer_guard = THREAD_GET_POINTER_GUARD ())
>  
>  /* Get and set the global scope generation counter in struct pthread.  */
> -#define THREAD_GSCOPE_IN_TCB      1
>  #define THREAD_GSCOPE_FLAG_UNUSED 0
>  #define THREAD_GSCOPE_FLAG_USED   1
>  #define THREAD_GSCOPE_FLAG_WAIT   2
> diff --git a/sysdeps/x86_64/nptl/tls.h b/sysdeps/x86_64/nptl/tls.h
> index a78c4f4d01..b0d044353b 100644
> --- a/sysdeps/x86_64/nptl/tls.h
> +++ b/sysdeps/x86_64/nptl/tls.h
> @@ -326,7 +326,6 @@ _Static_assert (offsetof (tcbhead_t, __glibc_unused2) == 0x80,
>  
>  
>  /* Get and set the global scope generation counter in the TCB head.  */
> -# define THREAD_GSCOPE_IN_TCB      1
>  # define THREAD_GSCOPE_FLAG_UNUSED 0
>  # define THREAD_GSCOPE_FLAG_USED   1
>  # define THREAD_GSCOPE_FLAG_WAIT   2
> -- 
> 2.31.1
> 

-- 
Samuel
In mutt, type cthis
Dans mutt, taper cceci

Re: [PATCH htl v3 1/5] elf: Replace most uses of THREAD_GSCOPE_IN_TCB

[Samuel Thibault via Libc-alpha]

Sergey Bugaev, le mar. 07 sept. 2021 16:33:21 +0300, a ecrit:
> While originally this definition was indeed used to distinguish between
> the cases where the GSCOPE flag was stored in TCB or not, it has since
> become used as a general way to distinguish between HTL and NPTL.
> 
> THREAD_GSCOPE_IN_TCB will be removed in the following commits, as HTL,
> which currently is the only port that does not put the flag into TCB,
> will get ported to put the GSCOPE flag into the TCB as well. To prepare
> for that change, migrate all code that wants to distinguish between HTL
> and NPTL to use PTHREAD_IN_LIBC instead, which is a better choice since
> the distinction mostly has to do with whether libc has access to the
> list of thread structures and therefore can initialize thread-local
> storage.
> 
> The parts of code that actually depend on whether the GSCOPE flag is in
> TCB are left unchanged.
> 
> Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>

Applied, thanks!

> ---
>  elf/dl-reloc.c             |  4 ++--
>  elf/dl-support.c           |  6 ++++--
>  elf/dl-tls.c               |  6 +++---
>  elf/dl-tls_init_tp.c       |  2 +-
>  sysdeps/generic/ldsodefs.h | 11 ++++++-----
>  5 files changed, 16 insertions(+), 13 deletions(-)
> 
> diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c
> index e13a672ade..6c957456b8 100644
> --- a/elf/dl-reloc.c
> +++ b/elf/dl-reloc.c
> @@ -141,7 +141,7 @@ cannot allocate memory in static TLS block"));
>      }
>  }
>  
> -#if !THREAD_GSCOPE_IN_TCB
> +#if !PTHREAD_IN_LIBC
>  /* Initialize static TLS area and DTV for current (only) thread.
>     libpthread implementations should provide their own hook
>     to handle all threads.  */
> @@ -160,7 +160,7 @@ _dl_nothread_init_static_tls (struct link_map *map)
>    memset (__mempcpy (dest, map->l_tls_initimage, map->l_tls_initimage_size),
>  	  '\0', map->l_tls_blocksize - map->l_tls_initimage_size);
>  }
> -#endif /* !THREAD_GSCOPE_IN_TCB */
> +#endif /* !PTHREAD_IN_LIBC */
>  
>  void
>  _dl_relocate_object (struct link_map *l, struct r_scope_elem *scope[],
> diff --git a/elf/dl-support.c b/elf/dl-support.c
> index 0155718175..a317459fae 100644
> --- a/elf/dl-support.c
> +++ b/elf/dl-support.c
> @@ -183,7 +183,7 @@ uint64_t _dl_hwcap_mask;
>   * executable but this isn't true for all platforms.  */
>  ElfW(Word) _dl_stack_flags = DEFAULT_STACK_PERMS;
>  
> -#if THREAD_GSCOPE_IN_TCB
> +#if PTHREAD_IN_LIBC
>  list_t _dl_stack_used;
>  list_t _dl_stack_user;
>  list_t _dl_stack_cache;
> @@ -195,9 +195,11 @@ int _dl_stack_cache_lock;
>     when it was not, we do it by calling this function.
>     It returns an errno code or zero on success.  */
>  int (*_dl_make_stack_executable_hook) (void **) = _dl_make_stack_executable;
> -int _dl_thread_gscope_count;
>  void (*_dl_init_static_tls) (struct link_map *) = &_dl_nothread_init_static_tls;
>  #endif
> +#if !THREAD_GSCOPE_IN_TCB
> +int _dl_thread_gscope_count;
> +#endif
>  struct dl_scope_free_list *_dl_scope_free_list;
>  
>  #ifdef NEED_DL_SYSINFO
> diff --git a/elf/dl-tls.c b/elf/dl-tls.c
> index 423e380f7c..d554ae4497 100644
> --- a/elf/dl-tls.c
> +++ b/elf/dl-tls.c
> @@ -29,7 +29,7 @@
>  #include <dl-tls.h>
>  #include <ldsodefs.h>
>  
> -#if THREAD_GSCOPE_IN_TCB
> +#if PTHREAD_IN_LIBC
>  # include <list.h>
>  #endif
>  
> @@ -1058,7 +1058,7 @@ cannot create TLS data structures"));
>      }
>  }
>  
> -#if THREAD_GSCOPE_IN_TCB
> +#if PTHREAD_IN_LIBC
>  static inline void __attribute__((always_inline))
>  init_one_static_tls (struct pthread *curp, struct link_map *map)
>  {
> @@ -1091,4 +1091,4 @@ _dl_init_static_tls (struct link_map *map)
>  
>    lll_unlock (GL (dl_stack_cache_lock), LLL_PRIVATE);
>  }
> -#endif /* THREAD_GSCOPE_IN_TCB */
> +#endif /* PTHREAD_IN_LIBC */
> diff --git a/elf/dl-tls_init_tp.c b/elf/dl-tls_init_tp.c
> index d84adc992c..e482f3cfa9 100644
> --- a/elf/dl-tls_init_tp.c
> +++ b/elf/dl-tls_init_tp.c
> @@ -36,7 +36,7 @@ rtld_lock_default_unlock_recursive (void *lock)
>  void
>  __tls_pre_init_tp (void)
>  {
> -#if !THREAD_GSCOPE_IN_TCB
> +#if !PTHREAD_IN_LIBC
>    GL(dl_init_static_tls) = &_dl_nothread_init_static_tls;
>  #endif
>  
> diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
> index 9c15259236..02bd579ef8 100644
> --- a/sysdeps/generic/ldsodefs.h
> +++ b/sysdeps/generic/ldsodefs.h
> @@ -456,7 +456,7 @@ struct rtld_global
>    /* Generation counter for the dtv.  */
>    EXTERN size_t _dl_tls_generation;
>  
> -#if !THREAD_GSCOPE_IN_TCB
> +#if !PTHREAD_IN_LIBC
>    EXTERN void (*_dl_init_static_tls) (struct link_map *);
>  #endif
>  
> @@ -466,7 +466,7 @@ struct rtld_global
>      size_t count;
>      void *list[50];
>    } *_dl_scope_free_list;
> -#if THREAD_GSCOPE_IN_TCB
> +#if PTHREAD_IN_LIBC
>    /* List of active thread stacks, with memory managed by glibc.  */
>    EXTERN list_t _dl_stack_used;
>  
> @@ -486,7 +486,8 @@ struct rtld_global
>  
>    /* Mutex protecting the stack lists.  */
>    EXTERN int _dl_stack_cache_lock;
> -#else
> +#endif
> +#if !THREAD_GSCOPE_IN_TCB
>    EXTERN int _dl_thread_gscope_count;
>  #endif
>  #ifdef SHARED
> @@ -1297,13 +1298,13 @@ extern void _dl_aux_init (ElfW(auxv_t) *av)
>  
>  /* Initialize the static TLS space for the link map in all existing
>     threads. */
> -#if THREAD_GSCOPE_IN_TCB
> +#if PTHREAD_IN_LIBC
>  void _dl_init_static_tls (struct link_map *map) attribute_hidden;
>  #endif
>  static inline void
>  dl_init_static_tls (struct link_map *map)
>  {
> -#if THREAD_GSCOPE_IN_TCB
> +#if PTHREAD_IN_LIBC
>    /* The stack list is available to ld.so, so the initialization can
>       be handled within ld.so directly.  */
>    _dl_init_static_tls (map);
> -- 
> 2.31.1
> 

-- 
Samuel
Now, it we had this sort of thing:
  yield -a     for yield to all traffic
  yield -t     for yield to trucks
  yield -f     for yield to people walking (yield foot)
  yield -d t*  for yield on days starting with t
...you'd have a lot of dead people at intersections, and traffic jams you
wouldn't believe...
(Discussion in comp.os.linux.misc on the intuitiveness of commands.)

Re: [PATCH htl v3 2/5] htl: Move thread table to ld.so

[Sergey Bugaev via Libc-alpha]

On Wed, Sep 15, 2021 at 2:27 AM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> Please rather use __libc_rwlock_wrlock etc.

Thanks, I did not realize there were actually more internal
synchronization primitives implemented on top of lll. I will send out
a new version shortly.

> Even if for now they are actually locks and not rwlocks, the day we fix
> that the rwlocking of dl_pthread_threads_lock will get fixed alongside.

May I ask, why are rwlocks unimplemented? Is it just that nobody has
done the work, or are there some unobvious complications?

I'm asking because I happen to have an implementation of rwlocks [0]
that works (passes the test) on the Hurd. It makes use of futex
bitsets (which don't seem to be supported by gsync), but seems to work
just fine without them. And HTL of course has its own rwlocks
implementation. Would it make sense to me to spend some time to try
and write a __libc_rwlock based on them, or are there some obstacles?

[0]: https://github.com/bugaevc/lets-write-sync-primitives/blob/master/src/rwlock.cpp

Sergey

Re: [PATCH htl v3 2/5] htl: Move thread table to ld.so

[Samuel Thibault via Libc-alpha]

Sergey Bugaev, le mer. 15 sept. 2021 17:14:00 +0300, a ecrit:
> > Even if for now they are actually locks and not rwlocks, the day we fix
> > that the rwlocking of dl_pthread_threads_lock will get fixed alongside.
> 
> May I ask, why are rwlocks unimplemented? Is it just that nobody has
> done the work,

This, yes, as usual :)

> And HTL of course has its own rwlocks implementation. Would it make
> sense to me to spend some time to try and write a __libc_rwlock based
> on them, or are there some obstacles?

Well, at some point we'll probably want to just do like nptl and
simply #define __libc_rwlock pthread_rwlock, so it's probably not
worth spending time on making a separate __libc_rwlock implementation,
and rather spend it on making pthread_rwlock use gsync, like was done
for pthread_mutex and sem, and then we can just #define __libc_rwlock
pthread_rwlock.

Samuel

Re: [PATCH htl v3 2/5] htl: Move thread table to ld.so

[Samuel Thibault via Libc-alpha]

Samuel Thibault, le mer. 15 sept. 2021 16:34:27 +0200, a ecrit:
> making pthread_rwlock use gsync, like was done for pthread_mutex and
> sem, and then we can just #define __libc_rwlock pthread_rwlock.

I forgot an important point: that will also depend on moving pthread_*
from libpthread.so to libc.so, which we'll also want sooner or later.

Samuel

Re: [PATCH htl v3 2/5] htl: Move thread table to ld.so

[Sergey Bugaev via Libc-alpha]

On Wed, Sep 15, 2021 at 5:34 PM Samuel Thibault <samuel.thibault@gnu.org> wrote:
> so it's probably not
> worth spending time on making a separate __libc_rwlock implementation,
> and rather spend it on making pthread_rwlock use gsync, like was done
> for pthread_mutex and sem

Oh, it currently doesn't? That's horrifying. Then I'll look into that next.

Sergey

P.S. There's this other thing that is way more urgent than either
eliminating gsync_wake () calls or optimizing rwlocks. I sent you an
email a week ago. Please take a look.

Re: [PATCH htl v3 2/5] htl: Move thread table to ld.so

[Samuel Thibault via Libc-alpha]

Sergey Bugaev, le mer. 15 sept. 2021 18:13:15 +0300, a ecrit:
> P.S. There's this other thing that is way more urgent than either
> eliminating gsync_wake () calls or optimizing rwlocks. I sent you an
> email a week ago. Please take a look.

I know.

My mbox is still 700 mails long.

Samuel