Re: [PATCH v3 10/10] progress.c: add & assert a "global_progress" variable

[Emily Shaffer <emilyshaffer@google.com>]

On Thu, Oct 14, 2021 at 12:28:26AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> The progress.c code makes a hard assumption that only one progress bar
> be active at a time (see [1] for a bug where this wasn't the
> case). Add a BUG() that'll trigger if we ever regress on that promise
> and have two progress bars active at the same time.
> 
> There was an alternative test-only approach to doing the same
> thing[2], but by doing this outside of a GIT_TEST_* mode we'll know
> we've put a hard stop to this particular API misuse.
> 
> It will also establish scaffolding to address current fundamental
> limitations in the progress output: The current output must be
> "driven" by calls to the likes of display_progress(). Once we have a
> global current progress object we'll be able to update that object via
> SIGALRM. See [3] for early code to do that.
> 
> It's conceivable that this change will hit the BUG() condition in some
> scenario that we don't currently have tests for, this would be very
> bad. If that happened we'd die just because we couldn't emit some
> pretty output.
> 
> See [4] for a discussion of why our test coverage is lacking; our
> progress display is hidden behind isatty(2) checks in many cases, so
> the test suite doesn't cover it unless individual tests are run in
> "--verbose" mode, we might also have multi-threaded use of the API, so
> two progress bars stopping and starting would only be visible due to a
> race condition.
> 
> Despite that, I think that this change won't introduce such
> regressions, because:
> 
>  1. I've read all the code using the progress API (and have modified a
>     large part of it in some WIP code I have). Almost all of it is really
>     simple, the parts that aren't[5] are complex in the display_progress() part,
>     not in starting or stopping the progress bar.
> 
>  2. The entire test suite passes when instrumented with an ad-hoc
>     Linux-specific mode (it uses gettid()) to die if progress bars are
>     ever started or stopped on anything but the main thread[6].
> 
>     Extending that to die if display_progress() is called in a thread
>     reveals that we have exactly two users of the progress bar under
>     threaded conditions, "git index-pack" and "git pack-objects". Both
>     uses are straightforward, and they don't start/stop the progress
>     bar when threads are active.
> 
>  3. I've likewise done an ad-hoc test to force progress bars to be
>     displayed with:
> 
>         perl -pi -e 's[isatty\(2\)][1]g' $(git grep -l -F 'isatty(2)')
> 
>     I.e. to replace all checks (not just for progress) of checking
>     whether STDERR is connected to a TTY, and then monkeypatching
>     is_foreground_fd() in progress.c to always "return 1". Running the
>     tests with those applied, interactively and under -V reveals via:
> 
>         $ grep -e set_progress_signal -e clear_progress_signal test-results/*out
> 
>     That nothing our tests cover hits the BUG conditions added here,
>     except the expected "BUG: start two concurrent progress bars" test
>     being added here.
> 
>     That isn't entirely true since we won't be getting 100% coverage
>     due to cascading failures from tests that expected no progress
>     output on stderr. To make sure I covered 100% I also tried making
>     the display() function in progress.c a NOOP on top of that (it's
>     the calls to start_progress_delay() and stop_progress()) that
>     matter.
> 
>     That doesn't hit the BUG() either. Some tests fail in that mode
>     due to a combination of the overzealous isatty(2) munging noted
>     above, and the tests that are testing that the progress output
>     itself is present (but for testing I'd made display() a NOOP).
> 
> Between those three points I think it's safe to go ahead with this
> change.
> 
> 1. 6f9d5f2fda1 (commit-graph: fix progress of reachable commits, 2020-07-09)
> 2. https://lore.kernel.org/git/20210620200303.2328957-3-szeder.dev@gmail.com
> 3. https://lore.kernel.org/git/patch-18.25-e21fc66623f-20210623T155626Z-avarab@gmail.com/
> 4. https://lore.kernel.org/git/cover-00.25-00000000000-20210623T155626Z-avarab@gmail.com/
> 5. b50c37aa44d (Merge branch 'ab/progress-users-adjust-counters' into
>    next, 2021-09-10)
> 6. https://lore.kernel.org/git/877dffg37n.fsf@evledraar.gmail.com/
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

I find it much nicer to understand now, thanks. The BUG() change in
particular is excellent.

Reviewed-by: Emily Shaffer <emilyshaffer@google.com>