From: Eric Wong <normalperson@yhbt.net>
To: Carlos O'Donell <carlos@redhat.com>
Cc: libc-alpha@sourceware.org
Subject: Re: [RFC/PoC] malloc: use wfcqueue to speed up remote frees
Date: Wed, 1 Aug 2018 09:26:26 +0000 [thread overview]
Message-ID: <20180801092626.jrwyrojfye4avcis@whir> (raw)
In-Reply-To: <aa9b36d4-02a1-e3cc-30e8-eca9f0d8b6eb@redhat.com>
Carlos O'Donell <carlos@redhat.com> wrote:
> On 08/01/2018 02:23 AM, Eric Wong wrote:
> > Carlos O'Donell <carlos@redhat.com> wrote:
> >> On 07/31/2018 07:18 PM, Eric Wong wrote:
> >>> Also, if I spawn a bunch of threads and get a bunch of
> >>> arenas early in the program lifetime; and then only have few
> >>> threads later, there can be a lot of idle arenas.
> >>
> >> Yes. That is true. We don't coalesce arenas to match the thread
> >> demand.
> >
> > Eep :< If contention can be avoided (which tcache seems to
> > work well for), limiting arenas to CPU count seems desirable and
> > worth trying.
>
> Agreed.
>
> In general it is not as bad as you think.
>
> An arena is made up of a chain of heaps, each an mmap'd block, and
> if we can manage to free an entire heap then we unmap the heap,
> and if we're lucky we can manage to free down the entire arena
> (_int_free -> large chunk / consolidate -> heap_trim -> shrink_heap).
>
> So we might just end up with a large number of arena's that don't
> have very much allocated at all, but are all on the arena free list
> waiting for a thread to attach to them to reduce overall contention.
>
> I agree that it would be *better* if we had one arena per CPU and
> each thread could easily determine the CPU it was on (via a
> restartable sequence) and then allocate CPU-local memory to work
> with (the best you can do; ignoring NUMA effects).
Thanks for the info on arenas. One problem for Ruby is we get
many threads[1], and they create allocations of varying
lifetimes. All this while malloc contention is rarely a
problem in Ruby because of the global VM lock (GVL).
Even without restartable sequences, I was wondering if lfstack
(also in urcu) could even be used for sharing/distributing
arenas between threads. This would require tcache to avoid
retries on lfstack pop/push.
Much less straighforward than using wfcqueue for frees with
this patch, though :)
[1] we only had green-threads back in Ruby 1.8, and I guess many
Rubyists got used to the idea that they could have many
threads cheaply. Ruby 1.9+ moved to 100% native threads,
so I'm also trying to reintroduce green threads as an option
back into Ruby (but still keeping native threads)
> > OK, I noticed my patch fails conformance tests because
> > (despite my use of __cds_wfcq_splice_nonblocking) it references
> > poll(), despite poll() being in an impossible code path:
> >
> > __cds_wfcq_splice_nonblocking -> ___cds_wfcq_splice
> > -> ___cds_wfcq_busy_wait -> poll
> >
> > The poll call is impossible because the `blocking' parameter is 0;
> > but I guess the linker doesn't know that?
>
> Correct. We can fix that easily at a later date. Don't worry about it.
Heh, a bit dirty, but #define-ing poll away seems to work :)
diff --git a/malloc/malloc.c b/malloc/malloc.c
index 40d61e45db..89e675c7a0 100644
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -247,6 +247,11 @@
/* For SINGLE_THREAD_P. */
#include <sysdep-cancel.h>
+/* prevent wfcqueue.h from including poll.h and linking to it */
+#include <poll.h>
+#undef poll
+#define poll(a,b,c) assert(0 && "should not be called")
+
#define _LGPL_SOURCE /* allows inlines */
#include <urcu/wfcqueue.h>
next prev parent reply other threads:[~2018-08-01 9:26 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-31 8:49 [RFC/PoC] malloc: use wfcqueue to speed up remote frees Eric Wong
2018-07-31 12:16 ` Carlos O'Donell
2018-07-31 23:18 ` Eric Wong
2018-08-01 4:41 ` Carlos O'Donell
2018-08-01 6:23 ` Eric Wong
2018-08-01 7:01 ` Carlos O'Donell
2018-08-01 9:26 ` Eric Wong [this message]
2018-08-02 21:38 ` Carlos O'Donell
2023-01-17 6:42 ` Eric Wong via Libc-alpha
2023-01-17 19:05 ` Mathieu Desnoyers via Libc-alpha
2023-01-18 15:48 ` Mathieu Desnoyers via Libc-alpha
2023-01-18 19:12 ` Eric Wong via Libc-alpha
2023-01-18 19:17 ` Mathieu Desnoyers via Libc-alpha
2023-01-18 20:05 ` Eric Wong via Libc-alpha
2023-01-18 14:53 ` Mathieu Desnoyers via Libc-alpha
2023-01-18 14:58 ` Mathieu Desnoyers via Libc-alpha
2018-08-08 10:40 ` Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/libc/involved.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180801092626.jrwyrojfye4avcis@whir \
--to=normalperson@yhbt.net \
--cc=carlos@redhat.com \
--cc=libc-alpha@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).