git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: "René Scharfe" <l.s.r@web.de>
Cc: "SZEDER Gábor" <szeder.dev@gmail.com>, git@vger.kernel.org
Subject: Re: [PATCH 4/7] commit-graph: fix bogus counter in "Scanning merged commits" progress line
Date: Sat, 26 Jun 2021 16:11:31 +0200	[thread overview]
Message-ID: <87a6ncy9aq.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <c1c5316a-6a43-a377-69d5-531d226463c8@web.de>


On Sat, Jun 26 2021, René Scharfe wrote:

> Am 21.06.21 um 22:08 schrieb Ævar Arnfjörð Bjarmason:
>>
>> On Mon, Jun 21 2021, René Scharfe wrote:
>>
>>> Am 21.06.21 um 00:13 schrieb Ævar Arnfjörð Bjarmason:
>>>>
>>>> On Sun, Jun 20 2021, SZEDER Gábor wrote:
>>>>
>>>>> The final value of the counter of the "Scanning merged commits"
>>>>> progress line is always one less than its expected total, e.g.:
>>>>>
>>>>>   Scanning merged commits:  83% (5/6), done.
>>>>>
>>>>> This happens because while iterating over an array the loop variable
>>>>> is passed to display_progress() as-is, but while C arrays (and thus
>>>>> the loop variable) start at 0 and end at N-1, the progress counter
>>>>> must end at N.  This causes the failures of the tests
>>>>> 'fetch.writeCommitGraph' and 'fetch.writeCommitGraph with submodules'
>>>>> in 't5510-fetch.sh' when run with GIT_TEST_CHECK_PROGRESS=1.
>>>>>
>>>>> Fix this by passing 'i + 1' to display_progress(), like most other
>>>>> callsites do.
>>>>>
>>>>> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
>>>>> ---
>>>>>  commit-graph.c | 2 +-
>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/commit-graph.c b/commit-graph.c
>>>>> index 2bcb4e0f89..3181906368 100644
>>>>> --- a/commit-graph.c
>>>>> +++ b/commit-graph.c
>>>>> @@ -2096,7 +2096,7 @@ static void sort_and_scan_merged_commits(struct write_commit_graph_context *ctx)
>>>>>
>>>>>  	ctx->num_extra_edges = 0;
>>>>>  	for (i = 0; i < ctx->commits.nr; i++) {
>>>>> -		display_progress(ctx->progress, i);
>>>>> +		display_progress(ctx->progress, i + 1);
>>>>>
>>>>>  		if (i && oideq(&ctx->commits.list[i - 1]->object.oid,
>>>>>  			  &ctx->commits.list[i]->object.oid)) {
>>>>
>>>> I think this fix makes sense, but FWIW there's a large thread starting
>>>> at [1] where René disagrees with me, and thinks the fix for this sort of
>>>> thing would be to display_progress(..., i + 1) at the end of that
>>>> for-loop, or just before the stop_progress().
>>>>
>>>> I don't agree, but just noting the disagreement, and that if that
>>>> argument wins then a patch like this would involve changing the other
>>>> 20-some calls to display_progress() in commit-graph.c to work
>>>> differently (and to be more complex, we'd need to deal with loop
>>>> break/continue etc.).
>>>>
>>>> 1. https://lore.kernel.org/git/patch-2.2-042f598826-20210607T144206Z-avarab@gmail.com/
>>>
>>> *sigh*  (And sorry, Ævar.)
>>>
>>> Before an item is done, it should be reported as not done.  After an
>>> item is done, it should be reported as done.  One loop iteration
>>> finishes one item.  Thus the number of items to report at the bottom of
>>> the loop is one higher than at the top.  i is the correct number to
>>> report at the top of a zero-based loop, i+1 at the bottom.
>
>> Anyone with more time than sense can go and read over our linked back &
>> forth thread where we're disagreeing on that point :). I think the pattern
>> in commit-graph.c makes sense, you don't.
>
> Thanks for this comment, I think I got it now: Work doesn't count in the
> commit-graph.c model of measuring progress, literally.  I.e. progress is
> the same before and after one item of work.

The progress isn't the same, we update the count. Or do you mean in the
time it takes us to go from the end of the for-loop & jump to the start
of it and update the count?

> Instead it counts the number of loop iterations.  The model I describe
> above counts finished work items instead.  The results of the two
> models differ by at most one despite their inverted axiom regarding
> the value of work.
>
> Phew, that took me a while.

For what it's worth I had some extensive examples in our initial
thread[1][2] (search for "apple" and "throughput", respectively), that
you cut out when replying to the relevant E-Mails. I'd think we could
probably have gotten here earlier :)

I'm a bit confused about this "value of work" comment.

If you pick up a copy of say a video game like Mario Kart you'll find
that for a 3-lap race you start at 1/3, and still have an entire lap to
go when the count is at 3/3.

So it's just a question of whether you report progress on item N or work
finished on item N, not whether laps in a race have more or less
value.

To reference my earlier E-Mail[1] are you eating the first apple or the
zeroeth apple? I don't think one is more or less right in the
mathematical sense, I just think for UX aimed at people counting "laps"
makes more sense than counting completed items.

>> Anyway, aside from that. I think, and I really would be advocating this
>> too, even if our respective positions were reversed, that *in this case*
>> it makes sense to just take something like SZEDER's patch here
>> as-is. Because in that file there's some dozen occurrences of that exact
>> pattern.
>
> The code without the patch either forgets to report the last work item
> in the count-work-items model or is one short in the count-iterations
> model, so a fix is needed either way.

It won't be one short, for a loop of 2 items we'll go from:

     0/2
     1/2
     1/2, done

To:

     1/2
     2/2
     2/2, done

Just like the rest of the uses of the progress API in that file.

Which is one of the two reasons I prefer this pattern, i.e. this is less
verbose:

    start_progress()
    for i in (0..X-1):
        display_progress(i+1)
        work()
    stop_progress()

Than one of these, which AFAICT would be your recommendation:

    # Simplest, but stalls on work()
    start_progress()
    for i in (0..X-1):
        work()
        display_progress(i+1)
    stop_progress()

    # More verbose, but doesn't:
    start_progress()
    for i in (0..X-1):
        display_progress(i)
        work()
        display_progress(i+1)
    stop_progress()

    # Ditto:
    start_progress()
    display_progress(0)
    for i in (0..X-1):
        work()
        display_progress(i+1)
    stop_progress()

And of course if your loop continues or whatever you'll need a last
"display_progress(X)" before the "stop_progress()".

The other is that if you count laps you can have your progress bar
optionally show progress on that item. E.g. we could if we stall show
seconds spend that we're hung on that item, or '3/3 ETA 40s". I have a
patch[3] that takes an initial step towards that, with some more queued
locally.

> The number of the other occurrences wouldn't matter if they were
> buggy, but in this case they indicate that Stolee consistently used
> the count-iterations model.  Thus using it in the patch as well makes
> sense.

>> Let's just bring this one case in line with the rest, if we then want to
>> argue that one or the other use of the progress.c API is wrong as a
>> general thing, I think it makes more sense to discuss that as some
>> follow-up series that changes these various API uses en-masse than
>> holding back isolated fixes that leave the state of the progress bar it
>> != 100%.
>
> Agreed.

Sorry to go on about this again :)

1. https://lore.kernel.org/git/87lf7k2bem.fsf@evledraar.gmail.com/
2. https://lore.kernel.org/git/87o8c8z105.fsf@evledraar.gmail.com/
3. https://lore.kernel.org/git/patch-18.25-e21fc66623f-20210623T155626Z-avarab@gmail.com/

  reply	other threads:[~2021-06-26 14:45 UTC|newest]

Thread overview: 117+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-20 20:02 [PATCH 0/7] progress: verify progress counters in the test suite SZEDER Gábor
2021-06-20 20:02 ` [PATCH 1/7] progress: introduce GIT_TEST_CHECK_PROGRESS to verify progress counters SZEDER Gábor
2021-06-21  7:09   ` Ævar Arnfjörð Bjarmason
2021-06-22 15:55   ` Taylor Blau
2021-06-20 20:02 ` [PATCH 2/7] progress: catch nested/overlapping progresses with GIT_TEST_CHECK_PROGRESS SZEDER Gábor
2021-06-22 16:00   ` Taylor Blau
2021-08-30 21:15     ` SZEDER Gábor
2021-06-20 20:02 ` [PATCH 3/7] progress: catch backwards counting " SZEDER Gábor
2021-06-20 20:03 ` [PATCH 4/7] commit-graph: fix bogus counter in "Scanning merged commits" progress line SZEDER Gábor
2021-06-20 22:13   ` Ævar Arnfjörð Bjarmason
2021-06-21 18:32     ` René Scharfe
2021-06-21 20:08       ` Ævar Arnfjörð Bjarmason
2021-06-26  8:27         ` René Scharfe
2021-06-26 14:11           ` Ævar Arnfjörð Bjarmason [this message]
2021-06-26 20:22             ` René Scharfe
2021-06-26 21:38               ` Ævar Arnfjörð Bjarmason
2021-07-04 12:15                 ` René Scharfe
2021-07-05 14:09                   ` Junio C Hamano
2021-07-05 23:28                   ` Ævar Arnfjörð Bjarmason
2021-07-06 16:02                     ` René Scharfe
2021-06-27 17:31               ` Felipe Contreras
2021-06-20 20:03 ` [PATCH 5/7] entry: show finer-grained counter in "Filtering content" " SZEDER Gábor
2021-06-20 20:03 ` [PATCH 6/7] [RFC] entry: don't show "Filtering content: ... done." line in case of errors SZEDER Gábor
2021-06-21 18:32   ` René Scharfe
2021-06-23  1:52     ` Taylor Blau
2021-08-30 21:17       ` SZEDER Gábor
2021-06-20 20:03 ` [PATCH 7/7] test-lib: enable GIT_TEST_CHECK_PROGRESS by default SZEDER Gábor
2021-06-21  0:59 ` [PATCH 0/7] progress: verify progress counters in the test suite Ævar Arnfjörð Bjarmason
2021-06-23  2:04   ` Taylor Blau
2021-06-23 17:48     ` [PATCH 00/25] progress.c: various fixes + SZEDER's RFC code Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 01/25] progress.c tests: fix breakage with COLUMNS != 80 Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 02/25] progress.c tests: make start/stop verbs on stdin Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 03/25] progress.c tests: test some invalid usage Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 04/25] progress.c tests: add a "signal" verb Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 05/25] progress.c: move signal handler functions lower Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 06/25] progress.c: call progress_interval() from progress_test_force_update() Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 07/25] progress.c: stop eagerly fflush(stderr) when not a terminal Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 08/25] progress.c: add temporary variable from progress struct Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 09/25] midx perf: add a perf test for multi-pack-index Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 10/25] progress.c: remove the "sparse" mode nano-optimization Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 11/25] pack-bitmap-write.c: add a missing stop_progress() Ævar Arnfjörð Bjarmason
2021-09-17  5:14         ` SZEDER Gábor
2021-09-17  5:56           ` Ævar Arnfjörð Bjarmason
2021-09-17 21:38             ` SZEDER Gábor
2021-06-23 17:48       ` [PATCH 12/25] progress.c: add & assert a "global_progress" variable Ævar Arnfjörð Bjarmason
2021-09-16 18:31         ` SZEDER Gábor
2021-06-23 17:48       ` [PATCH 13/25] progress.[ch]: move the "struct progress" to the header Ævar Arnfjörð Bjarmason
2021-09-16 19:42         ` SZEDER Gábor
2021-06-23 17:48       ` [PATCH 14/25] progress.[ch]: move test-only code away from "extern" variables Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 15/25] progress.c: pass "is done?" (again) to display() Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 16/25] progress.[ch]: convert "title" to "struct strbuf" Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 17/25] progress.c: refactor display() for less confusion, and fix bug Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 18/25] progress.c: emit progress on first signal, show "stalled" Ævar Arnfjörð Bjarmason
2021-09-16 18:37         ` SZEDER Gábor
2021-06-23 17:48       ` [PATCH 19/25] commit-graph: fix bogus counter in "Scanning merged commits" progress line Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 20/25] midx: don't provide a total for QSORT() progress Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 21/25] entry: show finer-grained counter in "Filtering content" progress line Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [PATCH 22/25] progress.c: add a stop_progress_early() function Ævar Arnfjörð Bjarmason
2021-06-24 10:35         ` Ævar Arnfjörð Bjarmason
2021-06-25  1:24         ` Andrei Rybak
2021-06-23 17:48       ` [PATCH 23/25] entry: deal with unexpected "Filtering content" total Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [RFC/PATCH 24/25] progress: assert last update in stop_progress() Ævar Arnfjörð Bjarmason
2021-06-23 17:48       ` [RFC/PATCH 25/25] progress: assert counting upwards in display() Ævar Arnfjörð Bjarmason
2021-06-23 17:59       ` [PATCH 00/25] progress.c: various fixes + SZEDER's RFC code Randall S. Becker
2021-06-23 20:01         ` Ævar Arnfjörð Bjarmason
2021-06-23 20:25           ` Randall S. Becker
2021-06-23 21:57 ` [PATCH 0/4] WIP/POC check isatty(2)-protected progress lines SZEDER Gábor
2021-06-23 21:57   ` [PATCH 1/4] WIP progress, isatty(2), hidden progress lnies for GIT_TEST_CHECK_PROGRESS SZEDER Gábor
2021-06-23 21:57   ` [PATCH 2/4] blame: fix progress total with line ranges SZEDER Gábor
2021-06-23 21:57   ` [PATCH 3/4] read-cache: avoid overlapping progress lines SZEDER Gábor
2021-06-23 21:57   ` [PATCH 4/4] preload-index: fix "Refreshing index" progress line SZEDER Gábor
2021-06-23 22:11   ` [PATCH 0/4] WIP/POC check isatty(2)-protected progress lines SZEDER Gábor
2021-06-24 10:43     ` Ævar Arnfjörð Bjarmason
2021-06-24 10:45   ` Ævar Arnfjörð Bjarmason
2021-07-22 12:20 ` [PATCH 0/3] progress.c API users: fix bogus counting Ævar Arnfjörð Bjarmason
2021-07-22 12:20   ` [PATCH 1/3] commit-graph: fix bogus counter in "Scanning merged commits" progress line Ævar Arnfjörð Bjarmason
2021-07-23 21:55     ` Junio C Hamano
2021-08-02 21:07     ` SZEDER Gábor
2021-07-22 12:20   ` [PATCH 2/3] midx: don't provide a total for QSORT() progress Ævar Arnfjörð Bjarmason
2021-07-23 21:56     ` Junio C Hamano
2021-08-05 15:07     ` Phillip Wood
2021-08-05 19:07       ` Ævar Arnfjörð Bjarmason
2021-07-22 12:20   ` [PATCH 3/3] entry: show finer-grained counter in "Filtering content" progress line Ævar Arnfjörð Bjarmason
2021-07-23 22:01     ` Junio C Hamano
2021-08-02 22:05       ` SZEDER Gábor
2021-08-02 21:48     ` SZEDER Gábor
2021-08-05 11:01   ` [PATCH v2 0/3] progress.c API users: fix bogus counting Ævar Arnfjörð Bjarmason
2021-08-05 11:01     ` [PATCH v2 1/3] commit-graph: fix bogus counter in "Scanning merged commits" progress line Ævar Arnfjörð Bjarmason
2021-08-05 11:01     ` [PATCH v2 2/3] midx: don't provide a total for QSORT() progress Ævar Arnfjörð Bjarmason
2021-08-05 11:01     ` [PATCH v2 3/3] entry: show finer-grained counter in "Filtering content" progress line Ævar Arnfjörð Bjarmason
2021-08-23 10:29     ` [PATCH v3 0/2] progress.c API users: fix bogus counting Ævar Arnfjörð Bjarmason
2021-08-23 10:29       ` [PATCH v3 1/2] commit-graph: fix bogus counter in "Scanning merged commits" progress line Ævar Arnfjörð Bjarmason
2021-08-23 10:29       ` [PATCH v3 2/2] entry: show finer-grained counter in "Filtering content" " Ævar Arnfjörð Bjarmason
2021-09-09  1:10       ` [PATCH v4 0/2] progress.c API users: fix bogus counting Ævar Arnfjörð Bjarmason
2021-09-09  1:10         ` [PATCH v4 1/2] commit-graph: fix bogus counter in "Scanning merged commits" progress line Ævar Arnfjörð Bjarmason
2021-09-09  1:10         ` [PATCH v4 2/2] entry: show finer-grained counter in "Filtering content" " Ævar Arnfjörð Bjarmason
2021-09-09 20:02         ` [PATCH v4 0/2] progress.c API users: fix bogus counting Junio C Hamano
2021-07-22 12:54 ` [PATCH 0/8] progress: assert "global_progress" + test fixes / cleanup Ævar Arnfjörð Bjarmason
2021-07-22 12:54   ` [PATCH 1/8] progress.c tests: make start/stop verbs on stdin Ævar Arnfjörð Bjarmason
2021-07-22 12:55   ` [PATCH 2/8] progress.c tests: test some invalid usage Ævar Arnfjörð Bjarmason
2021-07-22 12:55   ` [PATCH 3/8] progress.c: move signal handler functions lower Ævar Arnfjörð Bjarmason
2021-07-22 12:55   ` [PATCH 4/8] progress.c: call progress_interval() from progress_test_force_update() Ævar Arnfjörð Bjarmason
2021-07-22 12:55   ` [PATCH 5/8] progress.c: stop eagerly fflush(stderr) when not a terminal Ævar Arnfjörð Bjarmason
2021-07-22 12:55   ` [PATCH 6/8] progress.c: add temporary variable from progress struct Ævar Arnfjörð Bjarmason
2021-07-22 12:55   ` [PATCH 7/8] pack-bitmap-write.c: add a missing stop_progress() Ævar Arnfjörð Bjarmason
2021-07-22 12:55   ` [PATCH 8/8] progress.c: add & assert a "global_progress" variable Ævar Arnfjörð Bjarmason
2021-09-16 21:34     ` [PATCH 12/25] " Ævar Arnfjörð Bjarmason
2021-07-23 22:02   ` [PATCH 0/8] progress: assert "global_progress" + test fixes / cleanup Junio C Hamano
2021-09-20 23:09   ` [PATCH v2 " Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 1/8] progress.c tests: make start/stop verbs on stdin Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 2/8] progress.c tests: test some invalid usage Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 3/8] progress.c: move signal handler functions lower Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 4/8] progress.c: call progress_interval() from progress_test_force_update() Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 5/8] progress.c: stop eagerly fflush(stderr) when not a terminal Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 6/8] progress.c: add temporary variable from progress struct Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 7/8] pack-bitmap-write.c: add a missing stop_progress() Ævar Arnfjörð Bjarmason
2021-09-20 23:09     ` [PATCH v2 8/8] progress.c: add & assert a "global_progress" variable Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87a6ncy9aq.fsf@evledraar.gmail.com \
    --to=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=l.s.r@web.de \
    --cc=szeder.dev@gmail.com \
    --subject='Re: [PATCH 4/7] commit-graph: fix bogus counter in "Scanning merged commits" progress line' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).