Re: [PATCH v3 5/6] bitmap-lookup-table: add performance tests for lookup table

[Taylor Blau <me@ttaylorr.com>]

On Mon, Jul 04, 2022 at 08:46:15AM +0000, Abhradeep Chakraborty via GitGitGadget wrote:
> From: Abhradeep Chakraborty <chakrabortyabhradeep79@gmail.com>
>
> Add performance tests to verify the performance of lookup table with
> `pack.writeReverseIndex` enabled. This is to check the performance
> when the above configuration is set.
>
> Lookup table makes Git run faster in most of the cases. Below is the
> result of `t/perf/p5310-pack-bitmaps.sh`.`perf/p5326-multi-pack-bitmaps.sh`
> gives similar result. The repository used in the test is linux kernel.
>
> Test                                                      this tree
> ---------------------------------------------------------------------------
> 5310.4: repack to disk (lookup=false)                   296.55(256.53+14.52)

Having "lookup=false" in this test definitely helps visually
differentiate which tests have a bitmap with and without the lookup
table.

I think we should take a slightly different approach for these
performance tests. I think the first change to the t/perf tests in this
series should only enable `pack.writeReverseIndex`. That patch would be
a good place to highlight the benefit of enabling the on-disk reverse
index by showing a before and after of running p5310 before and after
that commit.

Then the patch after that should look like this one, which runs the
suite with and without the lookup table. That should give us a sense of:

  - bitmaps without a lookup table or reverse index
  - bitmaps without a lookup table, but with a reverse index
  - bitamps with a reverse index and a lookup table

...which I think are the most interesting combinations (I wouldn't
expect many or any users to have lookup tables enabled without reverse
indexes).

I think that would allow us to drop the last patch in this version of
the series. But I'm definitely open to other testing strategies for the
performance tests (including this one!) if you have different thoughts
about what the best way to go about this is.

Thanks,
Taylor