Re: [PATCH 2/2] t5516/t5601: be less strict about the number of credential warnings

[Jeff King <peff@peff.net>]

On Mon, Oct 31, 2022 at 09:54:23PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > It is unclear as to _why_, but under certain circumstances the warning
> > about credentials being passed as part of the URL seems to be swallowed
> > by the `git remote-https` helper in the Windows jobs of Git's CI builds.
> >
> > Since it is not actually important how many times Git prints the
> > warning/error message, as long as it prints it at least once, ...
> 
> Sorry, but I do not quite see the value of keeping the test to
> expect success in a weakend form.  If we think we are emitting three
> warnings, whether we do so by mistake or by design, and some of them
> are lost and not shown for an unknown reason, is there a guarantee
> that at least one would survive?  And when all three are lost, even
> the test in the weakened form would fail and stop the CI builds, no?

Without understanding the cause of the loss, I agree that things are a
little hand-wavy. But the assumption _does_ seem to hold that we
consistently produce at least one (presumably from the parent
clone/fetch/push process). And if we can rely on that, there's value in
the tests asserting that the message was shown to the user at least
once.

> If we do not know why we are losing some messages, and if we do not
> have resources to track down why, that is perfectly fine.  Fixes can
> be prioritised.  But wouldn't test_expect_failure be a better way to
> express "we think we ought to get 3 but for some reason we may not
> get all of them and we haven't figured it out"?

Marking it as expect_failure throws out the main point of the test,
though, which is that the user sees _some_ message (and that the "die"
form aborts the operation).

It might make sense to add a separate test in the meantime that
documents the "oops, we get the wrong number" sometimes state (and
eventually, if fixed, that could be folded back into the main test for
efficiency / simplicity).

-Peff