Re: move some test-tools to 'unstable plumbing' built-ins (was: [PATCH] test_bitmap_hashes(): handle repository without bitmaps)

[Taylor Blau <me@ttaylorr.com>]

On Sat, Nov 06, 2021 at 05:08:25AM +0100, Ævar Arnfjörð Bjarmason wrote:
> > On the test-tool vs. plumbing thing: I think there are some compelling
> > reasons in either direction. There's no *good* home for these in our
> > current set of plumbing tools. E.g., the closest example we have is `git
> > rev-list --test-bitmap <rev>`, which is kind of ugly. When we needed
> > these new inspection tools for some of the newer bitmap-related tests,
> > adding them via the test-helper suite was a conscious choice to not
> > build on the ugliness of `--test-bitmap`.
> >
> > But on occasion these test-tool things are useful to have "in the
> > field", as you say. It's rare enough that I usually just clone a copy of
> > our fork as needed and build it when I do find myself reaching for
> > test-helpers.
>
> As part of the proposed integration for "scalar" I added a category to
> the command-list.txt called "optionalcontrib", which we'll list on its
> own in "man git" as (paraphrasing) super-duper-experimental.
>
> I really don't see why we shouldn't do so very lightly with some of
> these remotely-useful test-tool tools.
>
> It's pretty much the same amount of work to create a new built-in as a
> new test-tool, and as long as we make it clear in our documentation that
> these aren't in the same "plumbing" category I don't see why we
> shouldn't add those quite freely.

I'm not sure that I agree completely.

Just because it's easy to create a new builtin (or as easy as it is to
make a new test helper) does not alone make it a good reason to do so.
I think that your idea to make a distinction between these and normal
plumbing tools is a good one.

But...

> It seems to me that we've ended up with the current status quo of not
> adding "new plumbing" because we'll need to support it forever out of
> some self-imposed constraint that we couldn't add new categories to the
> "git" manual page.
>
> But if we just prominently list them as being unstable helpers aimed at
> git experts, and note the same thing prominently in their manual page
> (trivially done via an include) everyone should be on the same page
> about their stability, and we'll be able to use stuff like "test-tool
> pkt-line" "in the field".

...just because we say that these tools are unstable does not mean that
users will listen to us (or even read the relevant documentation saying
so).

In my experience I *rarely* rely on test-helpers when debugging wedged
repositories, and much more often end up either in gdb, or in an
anonymized copy of the repository on a different server. I would imagine
that others have similar experiences.

So unless we had a much more compelling reason to have the test helpers
more readily available, I do not think that the risk our users will
begin to depend on these unstable tools is worth taking.

Thanks,
Taylor