Re: [PATCH] benchtests: Add memset zero fill benchmark tests

[naohirot--- via Libc-alpha <libc-alpha@sourceware.org>]

Hi Lucas, Wilco, Noah,

Thanks for the comments.

> From: Noah Goldstein <goldstein.w.n@gmail.com>
> Sent: Wednesday, 14 July 2021 03:47
> 
> On Tue, Jul 13, 2021 at 12:15 PM Wilco Dijkstra via Libc-alpha <libc-alpha@sourceware.org<mailto:libc-alpha@sourceware.org>> wrote:
> > Hi,
> > 
> > > I like the idea of a benchmark specific for 0 on memset. However having two
> > > implementations seems too much. I would rather see just one
> > > bench-memset-zerofill.c. What I guess would be even better is to have this
> > > performance test inside bench-memset.c and bench-memset-large.c.

Yes, that's one way. In this case, benchtests/scripts/plot_strings.py
cannot be used as is to visualize zero-fill performance.
We need to extract the zero fill data from the JSON bench results.

> > I agree just copying the files is not a good idea. Currently bench-memset and
> > bench-memset-walk already test zero memsets. Bench-memset-large could just
> > test zero since that is the most common, especially for large sizes. Reducing the
> > number of non-zero tests in bench-memset would make it more representative -
> > you could do the main set of tests with zero only and then have a small selection
> > where it alternates between zero and non-zero.

Yes, this is one way too.
However, we also need to extract the zero fill data from JSON bench result.

> 
> I'm in favor of a seperate file. On some x86_64 systems writing zeros to a cacheline
> that has not been modified can leave the cacheline in an unmodified state[1] which
> affects memory bandwidth on the writeback to DRAM for larger regions. I can imagine
> we might want to test memset zero on unmodified vs modified region which will require
> unique setup that I think justifies a separate file (at least for memset-large-zero).
> 
> [1] https://travisdowns.github.io/blog/2020/05/13/intel-zero-opt.html

I throughly read the blog, it's very interesting.
It seems that Skylake-S microarchitcture has some zero-over-zero
optimization in the size range of L3 and RAM.

So why don't we proceed like this?
- create a new benchtest file, ex. memset-zerofill.c.
  it measures zero-over-zero and zero-over-one from 32KB(L1), through
  L2 and L3, to 64MB(RAM)
- create a script to extract some part of data such as zero-over-zero
  or zero-over-one from JSON bench result to CSV in the first place in
  order to create graph in spreadsheet
- update benchtests/scripts/plot_strings.py later so that it can draw
  zero-over-zero and zero-over-one separately

> 
> > 
> > > Quoting Naohiro Tamura via Libc-alpha (2021-07-13 05:22:14)
> > >> Memset takes 0 as the second parameter in most cases.
> > >> More than 95% of memset takes 0 as the second parameter in case of
> > >> Linux Kernel source code.
> > > The Linux Kernel does not use glibc, it has his own memset implementation.
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/string.c#n784
> > > Therefore IMO this argument is not suited for this commit.
> > 
> > The argument is true in general - you could simply state that almost all memset
> > calls are zeroing without mentioning the Linux kernel. In some old stats from
> > SPEC I saw about 1.8% non-zero memsets.

I'll fix that.

Thanks.
Naohiro