unofficial mirror of libc-alpha@sourceware.org
 help / color / mirror / Atom feed
From: Carlos O'Donell <carlos@redhat.com>
To: Eric Wong <normalperson@yhbt.net>
Cc: libc-alpha@sourceware.org
Subject: Re: [RFC/PoC] malloc: use wfcqueue to speed up remote frees
Date: Wed, 1 Aug 2018 00:41:01 -0400	[thread overview]
Message-ID: <c061de55-cc2a-88fe-564b-2ea9c4a7e632@redhat.com> (raw)
In-Reply-To: <20180731231819.57xsqvdfdyfxrzy5@whir>

On 07/31/2018 07:18 PM, Eric Wong wrote:
>> - Can you explain the RSS reduction given this patch? You
>> might think that just adding the frees to a queue wouldn't
>> result in any RSS gains.
> 
> At least two reasons I can see:
> 
> 1) With lock contention, the freeing thread can lose to the
>    allocating thread.  This makes the allocating thread hit
>    sysmalloc since it prevented the freeing thread from doing
>    its job.  sysmalloc is the slow path, so the lock gets held
>    even longer and the problem compounds from there.

How does this impact RSS? It would only block the remote thread
from freeing in a timely fashion, but it would eventually make
progress.

> 2) thread caching - memory ends up in the wrong thread and
>    could never get used in some cases.  Fortunately this is
>    bounded, but still a waste.

We can't have memory end up in the wrong thread. The remote thread
computes the arena from the chunk it has, and then frees back to
the appropriate arena, even if it's not the arena that the thread
is attached to.

> I'm still new to the code, but it looks like threads are pinned
> to the arena and the memory used for arenas never gets released.
> Is that correct?

Threads are pinned to their arenas, but they can move in the event
of allocation failures, particularly to the main arena to attempt
sbrk to get more memory.

> I was wondering if there was another possibility: the allocating
> thread gives up the arena and creates a new one because the
> freeing thread locked it, but I don't think that's the case.

No.

> Also, if I spawn a bunch of threads and get a bunch of
> arenas early in the program lifetime; and then only have few
> threads later, there can be a lot of idle arenas.
 
Yes. That is true. We don't coalesce arenas to match the thread
demand.

>> However, you are calling _int_free a lot in row and that
>> deinterleaving may help (you really want vector free API here
>> so you don't walk all the lists so many times, tcache had the
>> same problem but in reverse for finding chunks). 
> 
> Maybe...  I think in the ideal case, the number of allocations
> and frees is close 1:1, so the loop is kept short.
> 
> What may be worth trying is to bypass _int_free for cases where
> a chunk can fulfill the allocation which triggers it.  Delaying
> or avoiding consolidation could worsen fragmentation, though. 

Right.

>> - Adding urcu as a build-time dependency is not acceptable for
>> bootstrap, instead we would bundle a copy of urcu and keep it
>> in sync with upstream. Would that make your work easier?
> 
> Yes, bundling that sounds great.  I assume it's something for
> you or one of the regular contributors to work on (build systems
> scare me :x)

Yes, that is something we'd have to do.

>> - What problems are you having with `make -j4 check?' Try
>> master and report back.  We are about to release 2.28 so it
>> should build and pass.
> 
> My fault.  It seems like tests aren't automatically rerun when I
> change the code; so some of my broken work-in-progress changes
> ended up being false positives :x.  When working on this, I made
> the mistake of doing remote_free_step inside malloc_consolidate,
> which could recurse into _int_free or _int_malloc

This depends a bit on what you touch.

> I guess I should remove the *.test-result files before rerunning
> tests?

Yes, that will definitely force the test to be re-run.

> I still get:
> 
> FAIL: nptl/tst-sched1
> 
> 	"Create failed"
> 
> 	I guess my system was overloaded.  pthread_create
> 	failures seemed to happen a lot for me when working
> 	on Ruby, too, and POSIX forcing EAGAIN makes it
> 	hard to diagnose :< (ulimit -u 47999 and 12GB RAM)
> 
> 	Removing the test-result and retrying seems OK.

OK. This one is new. There are a few tests where pthread_create
fails with EAGAIN because the kernel can't reap the children
fast enough.

> 
> FAIL: resolv/tst-resolv-ai_idn
> FAIL: resolv/tst-resolv-ai_idn-latin1
> 
> 	Not root, so no CLONE_NEWUTS
> 
> So I think that's expected...
> 

Agreed.

-- 
Cheers,
Carlos.

  reply	other threads:[~2018-08-01  4:41 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-31  8:49 [RFC/PoC] malloc: use wfcqueue to speed up remote frees Eric Wong
2018-07-31 12:16 ` Carlos O'Donell
2018-07-31 23:18   ` Eric Wong
2018-08-01  4:41     ` Carlos O'Donell [this message]
2018-08-01  6:23       ` Eric Wong
2018-08-01  7:01         ` Carlos O'Donell
2018-08-01  9:26           ` Eric Wong
2018-08-02 21:38             ` Carlos O'Donell
2023-01-17  6:42       ` Eric Wong via Libc-alpha
2023-01-17 19:05         ` Mathieu Desnoyers via Libc-alpha
2023-01-18 15:48           ` Mathieu Desnoyers via Libc-alpha
2023-01-18 19:12             ` Eric Wong via Libc-alpha
2023-01-18 19:17               ` Mathieu Desnoyers via Libc-alpha
2023-01-18 20:05                 ` Eric Wong via Libc-alpha
2023-01-18 14:53         ` Mathieu Desnoyers via Libc-alpha
2023-01-18 14:58           ` Mathieu Desnoyers via Libc-alpha
2018-08-08 10:40   ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/libc/involved.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c061de55-cc2a-88fe-564b-2ea9c4a7e632@redhat.com \
    --to=carlos@redhat.com \
    --cc=libc-alpha@sourceware.org \
    --cc=normalperson@yhbt.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).