[PATCH 0/2] Add progress to format-patch and rebase

[PATCH 0/2] Add progress to format-patch and rebase

[Kevin Willford]

This patch series is to add progress for the user when generating
patches in format-patch.  This is to aid the user when performing 
large rebases and have some indication to the user that progress
is being made.  

These patches just add the normal progress indication when
generating patches and makes it the default when running rebase.
It will also respect the -q[uiet] flag and not show when it
is present.

Kevin Willford (2):
  format-patch: have progress option while generating patches
  rebase: turn on progress option by default for format-patch

 Documentation/git-format-patch.txt |  8 ++++++++
 builtin/log.c                      | 10 ++++++++++
 git-rebase--am.sh                  |  5 +++--
 git-rebase.sh                      |  2 ++
 4 files changed, 23 insertions(+), 2 deletions(-)

-- 
2.13.0.92.g73a4ce6a77

[PATCH 1/2] format-patch: have progress option while generating patches

[Kevin Willford]

When generating patches for the rebase command if the user does
not realize the branch they are rebasing onto is thousands of
commits different there is no progress indication after initial
rewinding message.

This patch allows a progress option to be passed to format-patch
so that the user can be informed the progress of generating the
patch.  This option will then be used by the rebase command when
calling format-patch.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 Documentation/git-format-patch.txt |  8 ++++++++
 builtin/log.c                      | 10 ++++++++++
 2 files changed, 18 insertions(+)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index c890328b02..ee5f99f606 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -23,6 +23,7 @@ SYNOPSIS
 		   [(--reroll-count|-v) <n>]
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
+		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
 
@@ -260,6 +261,7 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
 -q::
 --quiet::
 	Do not print the names of the generated files to standard output.
+	Progress is not reported to the standard error stream.
 
 --no-binary::
 	Do not output contents of changes in binary files, instead
@@ -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
 	range are always formatted as creation patches, independently
 	of this flag.
 
+--progress::
+	Progress status is reported on the standard error stream
+	by default when it is attached to a terminal, unless -q
+	is specified. This flag forces progress status even if the
+	standard error stream is not directed to a terminal.
+
 CONFIGURATION
 -------------
 You can specify extra mail header lines to be added to each message,
diff --git a/builtin/log.c b/builtin/log.c
index 631fbc984f..02c50431b6 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -26,6 +26,7 @@
 #include "version.h"
 #include "mailmap.h"
 #include "gpg-interface.h"
+#include "progress.h"
 
 /* Set a default date-time format for git log ("log.date" config variable) */
 static const char *default_date_mode = NULL;
@@ -1409,6 +1410,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	char *branch_name = NULL;
 	char *base_commit = NULL;
 	struct base_tree_info bases;
+	int show_progress = 0;
+	struct progress *progress = NULL;
 
 	const struct option builtin_format_patch_options[] = {
 		{ OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
@@ -1480,6 +1483,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		OPT_FILENAME(0, "signature-file", &signature_file,
 				N_("add a signature from a file")),
 		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
+		OPT_BOOL(0, "progress", &show_progress,
+			 N_("show progress")),
 		OPT_END()
 	};
 
@@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		start_number--;
 	}
 	rev.add_signoff = do_signoff;
+
+	if (show_progress && !quiet)
+		progress = start_progress(_("Generating patch"), total);
 	while (0 <= --nr) {
 		int shown;
+		display_progress(progress, total - nr);
 		commit = list[nr];
 		rev.nr = total - nr + (start_number - 1);
 		/* Make the second and subsequent mails replies to the first */
@@ -1805,6 +1814,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (!use_stdout)
 			fclose(rev.diffopt.file);
 	}
+	stop_progress(&progress);
 	free(list);
 	free(branch_name);
 	string_list_clear(&extra_to, 0);
-- 
2.13.0.92.g73a4ce6a77

[PATCH 2/2] rebase: turn on progress option by default for format-patch

[Kevin Willford]

This change passes the progress option of format-patch by
default and passes the -q --quiet option through to the
format-patch call so that it is respected as well.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 git-rebase--am.sh | 5 +++--
 git-rebase.sh     | 2 ++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index 375239341f..ab2be30abf 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -51,8 +51,9 @@ then
 else
 	rm -f "$GIT_DIR/rebased-patches"
 
-	git format-patch -k --stdout --full-index --cherry-pick --right-only \
-		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
+	git format-patch $git_format_patch_opt -k --stdout --full-index \
+		--cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
+		--no-renames --no-cover-letter --progress \
 		"$revisions" ${restrict_revision+^$restrict_revision} \
 		>"$GIT_DIR/rebased-patches"
 	ret=$?
diff --git a/git-rebase.sh b/git-rebase.sh
index db1deed846..b72e319ac9 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -73,6 +73,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
 autostash="$(git config --bool rebase.autostash || echo false)"
 fork_point=auto
 git_am_opt=
+git_format_patch_opt=
 rebase_root=
 force_rebase=
 allow_rerere_autoupdate=
@@ -308,6 +309,7 @@ do
 	--quiet)
 		GIT_QUIET=t
 		git_am_opt="$git_am_opt -q"
+		git_format_patch_opt="$git_format_patch_opt -q"
 		verbose=
 		diffstat=
 		;;
-- 
2.13.0.92.g73a4ce6a77

Re: [PATCH 1/2] format-patch: have progress option while generating patches

[Stefan Beller]

On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com> wrote:
> When generating patches for the rebase command if the user does
> not realize the branch they are rebasing onto is thousands of
> commits different there is no progress indication after initial
> rewinding message.
>
> This patch allows a progress option to be passed to format-patch
> so that the user can be informed the progress of generating the
> patch.  This option will then be used by the rebase command when
> calling format-patch.

After reading the code, I was looking for some explanation
on the underlying assumptions, such as:

  The progress meter as presented in this patch assumes the thousands of
  patches to have a fine granularity as well as assuming to require all the
  same amount of work/time for each, such that a steady progress bar
  is achieved.

  We do not want to estimate the time for each patch based e.g.
  on their size or number of touched files (or parents) as that is too
  expensive for just a progress meter.


>
> Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> ---
>  Documentation/git-format-patch.txt |  8 ++++++++
>  builtin/log.c                      | 10 ++++++++++
>  2 files changed, 18 insertions(+)
>
> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
> index c890328b02..ee5f99f606 100644
> --- a/Documentation/git-format-patch.txt
> +++ b/Documentation/git-format-patch.txt
> @@ -23,6 +23,7 @@ SYNOPSIS
>                    [(--reroll-count|-v) <n>]
>                    [--to=<email>] [--cc=<email>]
>                    [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
> +                  [--progress]
>                    [<common diff options>]
>                    [ <since> | <revision range> ]
>
> @@ -260,6 +261,7 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
>  -q::
>  --quiet::
>         Do not print the names of the generated files to standard output.
> +       Progress is not reported to the standard error stream.
>
>  --no-binary::
>         Do not output contents of changes in binary files, instead
> @@ -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
>         range are always formatted as creation patches, independently
>         of this flag.
>
> +--progress::
> +       Progress status is reported on the standard error stream
> +       by default when it is attached to a terminal, unless -q
> +       is specified. This flag forces progress status even if the
> +       standard error stream is not directed to a terminal.
> +
>  CONFIGURATION
>  -------------
>  You can specify extra mail header lines to be added to each message,
> diff --git a/builtin/log.c b/builtin/log.c
> index 631fbc984f..02c50431b6 100644
> --- a/builtin/log.c
> +++ b/builtin/log.c
> @@ -26,6 +26,7 @@
>  #include "version.h"
>  #include "mailmap.h"
>  #include "gpg-interface.h"
> +#include "progress.h"
>
>  /* Set a default date-time format for git log ("log.date" config variable) */
>  static const char *default_date_mode = NULL;
> @@ -1409,6 +1410,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>         char *branch_name = NULL;
>         char *base_commit = NULL;
>         struct base_tree_info bases;
> +       int show_progress = 0;
> +       struct progress *progress = NULL;
>
>         const struct option builtin_format_patch_options[] = {
>                 { OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
> @@ -1480,6 +1483,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>                 OPT_FILENAME(0, "signature-file", &signature_file,
>                                 N_("add a signature from a file")),
>                 OPT__QUIET(&quiet, N_("don't print the patch filenames")),
> +               OPT_BOOL(0, "progress", &show_progress,
> +                        N_("show progress")),
>                 OPT_END()
>         };
>
> @@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>                 start_number--;
>         }
>         rev.add_signoff = do_signoff;
> +
> +       if (show_progress && !quiet)
> +               progress = start_progress(_("Generating patch"), total);
>         while (0 <= --nr) {
>                 int shown;
> +               display_progress(progress, total - nr);
>                 commit = list[nr];
>                 rev.nr = total - nr + (start_number - 1);
>                 /* Make the second and subsequent mails replies to the first */
> @@ -1805,6 +1814,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>                 if (!use_stdout)
>                         fclose(rev.diffopt.file);
>         }
> +       stop_progress(&progress);
>         free(list);
>         free(branch_name);
>         string_list_clear(&extra_to, 0);
> --
> 2.13.0.92.g73a4ce6a77
>

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

[Stefan Beller]

On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com> wrote:
> This change passes the progress option of format-patch by
> default and passes the -q --quiet option through to the
> format-patch call so that it is respected as well.

This is not conflicting with Johannes rewrite of rebase in C?
(rebase is a huge beast IIUC)

When passing the progress option by default to formatting patches,
maybe we should use start_progress_delay in the previous patch instead
to omit the progress for short lived patch formatting sessions?
(say a delay of one second?)

Thanks,
Stefan

>
> Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> ---
>  git-rebase--am.sh | 5 +++--
>  git-rebase.sh     | 2 ++
>  2 files changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/git-rebase--am.sh b/git-rebase--am.sh
> index 375239341f..ab2be30abf 100644
> --- a/git-rebase--am.sh
> +++ b/git-rebase--am.sh
> @@ -51,8 +51,9 @@ then
>  else
>         rm -f "$GIT_DIR/rebased-patches"
>
> -       git format-patch -k --stdout --full-index --cherry-pick --right-only \
> -               --src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> +       git format-patch $git_format_patch_opt -k --stdout --full-index \
> +               --cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
> +               --no-renames --no-cover-letter --progress \
>                 "$revisions" ${restrict_revision+^$restrict_revision} \
>                 >"$GIT_DIR/rebased-patches"
>         ret=$?
> diff --git a/git-rebase.sh b/git-rebase.sh
> index db1deed846..b72e319ac9 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -73,6 +73,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
>  autostash="$(git config --bool rebase.autostash || echo false)"
>  fork_point=auto
>  git_am_opt=
> +git_format_patch_opt=
>  rebase_root=
>  force_rebase=
>  allow_rerere_autoupdate=
> @@ -308,6 +309,7 @@ do
>         --quiet)
>                 GIT_QUIET=t
>                 git_am_opt="$git_am_opt -q"
> +               git_format_patch_opt="$git_format_patch_opt -q"
>                 verbose=
>                 diffstat=
>                 ;;
> --
> 2.13.0.92.g73a4ce6a77
>

RE: [PATCH 1/2] format-patch: have progress option while generating patches

[Kevin Willford]

> -----Original Message-----
> From: git-owner@vger.kernel.org [mailto:git-owner@vger.kernel.org] On
> Behalf Of Stefan Beller
> Sent: Wednesday, May 31, 2017 12:40 PM
> To: Kevin Willford <kcwillford@gmail.com>
> Cc: git@vger.kernel.org; Junio C Hamano <gitster@pobox.com>; Kevin
> Willford <kewillf@microsoft.com>
> Subject: Re: [PATCH 1/2] format-patch: have progress option while
> generating patches
> 
> On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com>
> wrote:
> > When generating patches for the rebase command if the user does not
> > realize the branch they are rebasing onto is thousands of commits
> > different there is no progress indication after initial rewinding
> > message.
> >
> > This patch allows a progress option to be passed to format-patch so
> > that the user can be informed the progress of generating the patch.
> > This option will then be used by the rebase command when calling
> > format-patch.
> 
> After reading the code, I was looking for some explanation on the underlying
> assumptions, such as:
> 
>   The progress meter as presented in this patch assumes the thousands of
>   patches to have a fine granularity as well as assuming to require all the
>   same amount of work/time for each, such that a steady progress bar
>   is achieved.
> 
>   We do not want to estimate the time for each patch based e.g.
>   on their size or number of touched files (or parents) as that is too
>   expensive for just a progress meter.
> 

Sounds good.  I will add some explanation to the commit message.

> 
> >
> > Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> > ---
> >  Documentation/git-format-patch.txt |  8 ++++++++
> >  builtin/log.c                      | 10 ++++++++++
> >  2 files changed, 18 insertions(+)
> >
> > diff --git a/Documentation/git-format-patch.txt
> > b/Documentation/git-format-patch.txt
> > index c890328b02..ee5f99f606 100644
> > --- a/Documentation/git-format-patch.txt
> > +++ b/Documentation/git-format-patch.txt
> > @@ -23,6 +23,7 @@ SYNOPSIS
> >                    [(--reroll-count|-v) <n>]
> >                    [--to=<email>] [--cc=<email>]
> >                    [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
> > +                  [--progress]
> >                    [<common diff options>]
> >                    [ <since> | <revision range> ]
> >
> > @@ -260,6 +261,7 @@ you can use `--suffix=-patch` to get `0001-
> description-of-my-change-patch`.
> >  -q::
> >  --quiet::
> >         Do not print the names of the generated files to standard output.
> > +       Progress is not reported to the standard error stream.
> >
> >  --no-binary::
> >         Do not output contents of changes in binary files, instead @@
> > -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-
> of-my-change-patch`.
> >         range are always formatted as creation patches, independently
> >         of this flag.
> >
> > +--progress::
> > +       Progress status is reported on the standard error stream
> > +       by default when it is attached to a terminal, unless -q
> > +       is specified. This flag forces progress status even if the
> > +       standard error stream is not directed to a terminal.
> > +
> >  CONFIGURATION
> >  -------------
> >  You can specify extra mail header lines to be added to each message,
> > diff --git a/builtin/log.c b/builtin/log.c index
> > 631fbc984f..02c50431b6 100644
> > --- a/builtin/log.c
> > +++ b/builtin/log.c
> > @@ -26,6 +26,7 @@
> >  #include "version.h"
> >  #include "mailmap.h"
> >  #include "gpg-interface.h"
> > +#include "progress.h"
> >
> >  /* Set a default date-time format for git log ("log.date" config
> > variable) */  static const char *default_date_mode = NULL; @@ -1409,6
> > +1410,8 @@ int cmd_format_patch(int argc, const char **argv, const char
> *prefix)
> >         char *branch_name = NULL;
> >         char *base_commit = NULL;
> >         struct base_tree_info bases;
> > +       int show_progress = 0;
> > +       struct progress *progress = NULL;
> >
> >         const struct option builtin_format_patch_options[] = {
> >                 { OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
> > @@ -1480,6 +1483,8 @@ int cmd_format_patch(int argc, const char **argv,
> const char *prefix)
> >                 OPT_FILENAME(0, "signature-file", &signature_file,
> >                                 N_("add a signature from a file")),
> >                 OPT__QUIET(&quiet, N_("don't print the patch
> > filenames")),
> > +               OPT_BOOL(0, "progress", &show_progress,
> > +                        N_("show progress")),
> >                 OPT_END()
> >         };
> >
> > @@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char
> **argv, const char *prefix)
> >                 start_number--;
> >         }
> >         rev.add_signoff = do_signoff;
> > +
> > +       if (show_progress && !quiet)
> > +               progress = start_progress(_("Generating patch"),
> > + total);
> >         while (0 <= --nr) {
> >                 int shown;
> > +               display_progress(progress, total - nr);
> >                 commit = list[nr];
> >                 rev.nr = total - nr + (start_number - 1);
> >                 /* Make the second and subsequent mails replies to the
> > first */ @@ -1805,6 +1814,7 @@ int cmd_format_patch(int argc, const char
> **argv, const char *prefix)
> >                 if (!use_stdout)
> >                         fclose(rev.diffopt.file);
> >         }
> > +       stop_progress(&progress);
> >         free(list);
> >         free(branch_name);
> >         string_list_clear(&extra_to, 0);
> > --
> > 2.13.0.92.g73a4ce6a77
> >

RE: [PATCH 2/2] rebase: turn on progress option by default for format-patch

[Kevin Willford]

> -----Original Message-----
> From: Stefan Beller [mailto:sbeller@google.com]
> Sent: Wednesday, May 31, 2017 1:09 PM
> To: Kevin Willford <kcwillford@gmail.com>
> Cc: git@vger.kernel.org; Junio C Hamano <gitster@pobox.com>; Kevin
> Willford <kewillf@microsoft.com>
> Subject: Re: [PATCH 2/2] rebase: turn on progress option by default for
> format-patch
> 
> On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com>
> wrote:
> > This change passes the progress option of format-patch by default and
> > passes the -q --quiet option through to the format-patch call so that
> > it is respected as well.
> 
> This is not conflicting with Johannes rewrite of rebase in C?
> (rebase is a huge beast IIUC)

I will check with Johannes and see what possible conflicts there could be.
Since these are flags that get passed to the format-patch code, it shouldn't take much to put it in the C code as well.

> 
> When passing the progress option by default to formatting patches, maybe
> we should use start_progress_delay in the previous patch instead to omit
> the progress for short lived patch formatting sessions?
> (say a delay of one second?)
> 

I thought about that and certainly could do it but I have found it nice to have the number of patches that are generated in the output even for a small number or commits.  For example when I run a `git rebase master` and expect there to be only 2 commits, the message "Generating patch: 100% (2/2), done."  Gives me that good feeling that I did it right and didn't mess something up.  I'm good either way though.

> Thanks,
> Stefan
> 
> >
> > Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> > ---
> >  git-rebase--am.sh | 5 +++--
> >  git-rebase.sh     | 2 ++
> >  2 files changed, 5 insertions(+), 2 deletions(-)
> >
> > diff --git a/git-rebase--am.sh b/git-rebase--am.sh index
> > 375239341f..ab2be30abf 100644
> > --- a/git-rebase--am.sh
> > +++ b/git-rebase--am.sh
> > @@ -51,8 +51,9 @@ then
> >  else
> >         rm -f "$GIT_DIR/rebased-patches"
> >
> > -       git format-patch -k --stdout --full-index --cherry-pick --right-only \
> > -               --src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> > +       git format-patch $git_format_patch_opt -k --stdout --full-index \
> > +               --cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
> > +               --no-renames --no-cover-letter --progress \
> >                 "$revisions" ${restrict_revision+^$restrict_revision} \
> >                 >"$GIT_DIR/rebased-patches"
> >         ret=$?
> > diff --git a/git-rebase.sh b/git-rebase.sh index
> > db1deed846..b72e319ac9 100755
> > --- a/git-rebase.sh
> > +++ b/git-rebase.sh
> > @@ -73,6 +73,7 @@ test "$(git config --bool rebase.stat)" = true &&
> > diffstat=t  autostash="$(git config --bool rebase.autostash || echo false)"
> >  fork_point=auto
> >  git_am_opt=
> > +git_format_patch_opt=
> >  rebase_root=
> >  force_rebase=
> >  allow_rerere_autoupdate=
> > @@ -308,6 +309,7 @@ do
> >         --quiet)
> >                 GIT_QUIET=t
> >                 git_am_opt="$git_am_opt -q"
> > +               git_format_patch_opt="$git_format_patch_opt -q"
> >                 verbose=
> >                 diffstat=
> >                 ;;
> > --
> > 2.13.0.92.g73a4ce6a77
> >

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

[Stefan Beller]

On Wed, May 31, 2017 at 12:46 PM, Kevin Willford <kewillf@microsoft.com> wrote:

>
> I thought about that and certainly could do it but I have found it nice to have the number of patches that are generated in the output even for a small number or commits.  For example when I run a `git rebase master` and expect there to be only 2 commits, the message "Generating patch: 100% (2/2), done."  Gives me that good feeling that I did it right and didn't mess something up.  I'm good either way though.
>

Oh I see, that number matching makes sense.
Though by reading the code, I have the impression
that the final value would not change. So if it would take
a really long time for these 2 patches, you'd still get the
(2/2), to know it was 2 patches indeed.

When it goes quickly, I'd be more concerned about
screen real estate, driving off the recent history from
the screen.

Also we maybe want to have
 "Generating patches: 100% (2/2), done."
plural, as (when using the delay)
it is more likely to kick in for more than just one patch?

Sorry, if this comes across as bike shedding.
It's meant as food for thought. :)

Thanks,
Stefan

Re: [PATCH 1/2] format-patch: have progress option while generating patches

[Jeff King]

On Wed, May 31, 2017 at 08:04:26AM -0700, Kevin Willford wrote:

> When generating patches for the rebase command if the user does
> not realize the branch they are rebasing onto is thousands of
> commits different there is no progress indication after initial
> rewinding message.
> 
> This patch allows a progress option to be passed to format-patch
> so that the user can be informed the progress of generating the
> patch.  This option will then be used by the rebase command when
> calling format-patch.

I'm generally in favor of progress meters, though it does seem a little
funny to me that we'd need one on format-patch. It's basically the same
operation as "git log -p", after all. I guess one difference is that
usually the output of "log" would stream to the pager, so the user would
see immediate output. That's true of format-patch, too, but in the
instance you care about we're probably doing something more like:

  git format-patch "$@" >patches

and the user sees nothing.

That makes me wonder two things:

  1. Should this really be tied to isatty(2), as the documentation
     claims?  It seems like you'd really only want it to kick in for
     certain cases. Arguably whenever stdout is _not_ going to a tty
     you'd want progress, but I think probably the caller should
     probably just decide whether to ask for it or not.

  2. Should this apply to other commands in the log family besides
     format-patch? E.g., should "git log --progress -p >commits" work?

     I added a progress meter to rev-list a while ago (for connectivity
     checks). I don't think we could push this down as far as the
     revision traversal code, because only its callers really understand
     what's the right unit to be counting.

     It may not be worth the trouble, though. Only "format-patch" does a
     pre-pass to find the total number of commits. So "git log
     --progress" would inherently just be counting up, with no clue what
     the final number is.

> @@ -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
>  	range are always formatted as creation patches, independently
>  	of this flag.
>  
> +--progress::
> +	Progress status is reported on the standard error stream
> +	by default when it is attached to a terminal, unless -q
> +	is specified. This flag forces progress status even if the
> +	standard error stream is not directed to a terminal.

Checking whether stderr is a tty would match our usual progress meters.
But I don't actually see any code in the patch implementing this.

As I said above, I think I'd prefer it to require "--progress", as
format-patch is quite often used as plumbing. But we'd need to fix the
documentation here to match.

> @@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  		start_number--;
>  	}
>  	rev.add_signoff = do_signoff;
> +
> +	if (show_progress && !quiet)
> +		progress = start_progress(_("Generating patch"), total);

The meter code here all looks correct, but let me bikeshed for a moment. :)

Should this use start_progress_delay()? In most cases the command will
complete very quickly, and the progress report is just noise. For many
commands (e.g., checkout) we wait 1-2 seconds before bothering to show
progress output.

I would have expected this to say "Generating patches"; most of our
other progress messages are pluralized. You could use Q_() to handle the
mono/plural case, but I think it's fine to just always say "patches"
(that's what other messages do).

One final thought is whether callers would want to customize this
message, since it will often be used as plumbing. E.g., would rebase
want to say something besides "Generating patches". I'm not sure.
Anyway, if you're interested in that direction, there's prior art in
the way rev-list handles "--progress" (and its sole caller,
check_connected()).

-Peff

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

[Jeff King]

On Wed, May 31, 2017 at 08:04:27AM -0700, Kevin Willford wrote:

> This change passes the progress option of format-patch by
> default and passes the -q --quiet option through to the
> format-patch call so that it is respected as well.

That makes sense. Is it a bug that we aren't propagating "-q" already?

I think the answer is "no", because that option only claims to silence
the printing of the filenames that format-patch writes. But since we're
using --stdout, it wouldn't write those anyway.

Come to think of it, I'm not sure that "-q" silencing "--progress" in
your patch 1 actually makes sense. If you do:

  git format-patch -o out/

you don't need progress, because we're already writing the filenames to
stdout. So it's only if you did:

  git format-patch -o out/ -q

that you'd actually want progress. But "-q" would override any
--progress option!  So I actually think we may be better off keeping the
two as distinct features (especially if we follow the "no progress
unless --progress is given" rule I mentioned in the earlier message).

> diff --git a/git-rebase--am.sh b/git-rebase--am.sh
> index 375239341f..ab2be30abf 100644
> --- a/git-rebase--am.sh
> +++ b/git-rebase--am.sh
> @@ -51,8 +51,9 @@ then
>  else
>  	rm -f "$GIT_DIR/rebased-patches"
>  
> -	git format-patch -k --stdout --full-index --cherry-pick --right-only \
> -		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> +	git format-patch $git_format_patch_opt -k --stdout --full-index \
> +		--cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
> +		--no-renames --no-cover-letter --progress \
>  		"$revisions" ${restrict_revision+^$restrict_revision} \
>  		>"$GIT_DIR/rebased-patches"

Here we pass --progress unconditionally, which tells format-patch to
output progress information. Shouldn't we be checking whether stderr is
a tty before making that decision? And that we're not in --quiet mode?

That explains why you want to pass "-q" in the other hunk; to
countermand the explicit --progress here. But if we separate the two as
I mentioned above, you'd want logic more like:

  if test -t 2 && test "$GIT_QUIET" != "t"
	git_format_patch_opt="$git_format_patch_opt --progress"
  fi

-Peff

Re: [PATCH 1/2] format-patch: have progress option while generating patches

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> As I said above, I think I'd prefer it to require "--progress", as
> format-patch is quite often used as plumbing.

Yes, that sounds sensible.

Initially, my reaction was "Why do we even need --progress for
format-patch, when it gives one-line per patch output to show the
progress anyway?", but if that output is redirected to a file, of
course you'd need --progress independently.


> Should this use start_progress_delay()? In most cases the command will
> complete very quickly, and the progress report is just noise. For many
> commands (e.g., checkout) we wait 1-2 seconds before bothering to show
> progress output.

It is better to use the "delay" version for progress meters for
commands that may or may not last very long, and this may be a good
candidate to heed that principle.

The subcommands that use start_progress() tend to be older and more
batch oriented operations, e.g. fsck, pack-objects, etc., that are
expected to last longer.  It may be a good idea to convert them to
the "delay" variant, but obviously that is outside the scope of this
patch.

> I would have expected this to say "Generating patches"; most of our
> other progress messages are pluralized. You could use Q_() to handle the
> mono/plural case, but I think it's fine to just always say "patches"
> (that's what other messages do).
>
> One final thought is whether callers would want to customize this
> message, since it will often be used as plumbing. E.g., would rebase
> want to say something besides "Generating patches". I'm not sure.
> Anyway, if you're interested in that direction, there's prior art in
> the way rev-list handles "--progress" (and its sole caller,
> check_connected()).

These are all good suggestions.

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

[Johannes Schindelin]

Hi Stefan,

On Wed, 31 May 2017, Stefan Beller wrote:

> On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com> wrote:
> > This change passes the progress option of format-patch by
> > default and passes the -q --quiet option through to the
> > format-patch call so that it is respected as well.
> 
> This is not conflicting with Johannes rewrite of rebase in C?
> (rebase is a huge beast IIUC)

No.

Ciao,
Dscho

Re: [PATCH 1/2] format-patch: have progress option while generating patches

[Johannes Schindelin]

Hi Peff,

On Wed, 31 May 2017, Jeff King wrote:

> I'm generally in favor of progress meters, though it does seem a little
> funny to me that we'd need one on format-patch.

When working with huge repositories with a large number of branches, it is
all too easy to pick the wrong branch to rebase to. In that case, it can
take a long time for the `git rebase` call to even generate the mbox.

Kevin's patch is a visual indication when this is happening.

Instead of being puzzled by `git rebase` "hanging", the user will now see
that (.../21957) patch is generated and will have a chance to say "wait a
minute, *that* many patches? I think I meant a different branch".

And since format-patch generates the patches for `git rebase` (actually,
it generates an mbox that then has to be split again, but you get the
gist), it is format-patch that needs to output the progress.

Ciao,
dscho

Re: [PATCH 1/2] format-patch: have progress option while generating patches

[Jeff King]

On Thu, Jun 01, 2017 at 01:15:57PM +0200, Johannes Schindelin wrote:

> On Wed, 31 May 2017, Jeff King wrote:
> 
> > I'm generally in favor of progress meters, though it does seem a little
> > funny to me that we'd need one on format-patch.
> 
> When working with huge repositories with a large number of branches, it is
> all too easy to pick the wrong branch to rebase to. In that case, it can
> take a long time for the `git rebase` call to even generate the mbox.
> 
> Kevin's patch is a visual indication when this is happening.

Right, sorry if it wasn't clear from the rest of my message. My
statement was one of surprise, not objection. I understand his use case.

-Peff

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> ...
> That explains why you want to pass "-q" in the other hunk; to
> countermand the explicit --progress here. But if we separate the two as
> I mentioned above, you'd want logic more like:
>
>   if test -t 2 && test "$GIT_QUIET" != "t"
> 	git_format_patch_opt="$git_format_patch_opt --progress"
>   fi

This looks like a sensible line of thought.  Kevin, do you want to
try this again with the "explicit --progress option" approach?  I
think the gist of the v1 review is that everybody agrees that the
end goal of showing a progress eye-candy from a long-running
"format-patch" phase in a "rebase" is a good idea, but it is dubious
"format-patch" should show it in cases other than when --progress is
explicitly asked for.

Thanks.

[PATCH v2 0/2] Add progress for format-patch and rebase

[Kevin Willford]

Changes since last patch:
1. Use start_progress_delay so progress isn't shown if generating
   the patches is fast enough
2. Updated to have text of "Generating patches"
3. Only show progress when the --progress flag is passed
4. In the rebase script check stderr and the quiet option is not
   set before propagating the progress flag to format-patch

Kevin Willford (2):
  format-patch: have progress option while generating patches
  rebase: turn on progress option by default for format-patch

 Documentation/git-format-patch.txt |  4 ++++
 builtin/log.c                      | 10 ++++++++++
 git-rebase--am.sh                  |  1 +
 git-rebase.sh                      |  6 ++++++
 4 files changed, 21 insertions(+)

-- 
2.14.0.rc0.286.g44127d70e4

[PATCH v2 1/2] format-patch: have progress option while generating patches

[Kevin Willford]

When generating patches for the rebase command if the user does
not realize the branch they are rebasing onto is thousands of
commits different there is no progress indication after initial
rewinding message.

The progress meter as presented in this patch assumes the thousands of
patches to have a fine granularity as well as assuming to require all the
same amount of work/time for each, such that a steady progress bar
is achieved.

We do not want to estimate the time for each patch based e.g.
on their size or number of touched files (or parents) as that is too
expensive for just a progress meter.

This patch allows a progress option to be passed to format-patch
so that the user can be informed the progress of generating the
patch.  This option is then used by the rebase command when
calling format-patch.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 Documentation/git-format-patch.txt |  4 ++++
 builtin/log.c                      | 10 ++++++++++
 2 files changed, 14 insertions(+)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index c890328b02..6cbe462a77 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -23,6 +23,7 @@ SYNOPSIS
 		   [(--reroll-count|-v) <n>]
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
+		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
 
@@ -283,6 +284,9 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
 	range are always formatted as creation patches, independently
 	of this flag.
 
+--progress::
+	Show progress reports on stderr as patches are generated.
+
 CONFIGURATION
 -------------
 You can specify extra mail header lines to be added to each message,
diff --git a/builtin/log.c b/builtin/log.c
index 725c7b8a1a..b07a5529c2 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -27,6 +27,7 @@
 #include "version.h"
 #include "mailmap.h"
 #include "gpg-interface.h"
+#include "progress.h"
 
 /* Set a default date-time format for git log ("log.date" config variable) */
 static const char *default_date_mode = NULL;
@@ -1422,6 +1423,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	char *branch_name = NULL;
 	char *base_commit = NULL;
 	struct base_tree_info bases;
+	int show_progress = 0;
+	struct progress *progress = NULL;
 
 	const struct option builtin_format_patch_options[] = {
 		{ OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
@@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		OPT_FILENAME(0, "signature-file", &signature_file,
 				N_("add a signature from a file")),
 		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
+		OPT_BOOL(0, "progress", &show_progress,
+			 N_("show progress while generating patches")),
 		OPT_END()
 	};
 
@@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		start_number--;
 	}
 	rev.add_signoff = do_signoff;
+
+	if (show_progress)
+		progress = start_progress_delay(_("Generating patches"), total, 0, 1);
 	while (0 <= --nr) {
 		int shown;
+		display_progress(progress, total - nr);
 		commit = list[nr];
 		rev.nr = total - nr + (start_number - 1);
 		/* Make the second and subsequent mails replies to the first */
@@ -1818,6 +1827,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (!use_stdout)
 			fclose(rev.diffopt.file);
 	}
+	stop_progress(&progress);
 	free(list);
 	free(branch_name);
 	string_list_clear(&extra_to, 0);
-- 
2.14.0.rc0.286.g44127d70e4

[PATCH v2 2/2] rebase: turn on progress option by default for format-patch

[Kevin Willford]

This change passes the progress option of format-patch checking
that stderr is attached and rebase is not being run in quiet mode.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 git-rebase--am.sh | 1 +
 git-rebase.sh     | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index 375239341f..ff98fe3a73 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -53,6 +53,7 @@ else
 
 	git format-patch -k --stdout --full-index --cherry-pick --right-only \
 		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
+		$git_format_patch_opt \
 		"$revisions" ${restrict_revision+^$restrict_revision} \
 		>"$GIT_DIR/rebased-patches"
 	ret=$?
diff --git a/git-rebase.sh b/git-rebase.sh
index f8b3d1fd97..ad8415e3cf 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -74,6 +74,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
 autostash="$(git config --bool rebase.autostash || echo false)"
 fork_point=auto
 git_am_opt=
+git_format_patch_opt=
 rebase_root=
 force_rebase=
 allow_rerere_autoupdate=
@@ -445,6 +446,11 @@ else
 	state_dir="$apply_dir"
 fi
 
+if test -t 2 && test -z "$GIT_QUIET"
+then
+	git_format_patch_opt="$git_format_patch_opt --progress"
+fi
+
 if test -z "$rebase_root"
 then
 	case "$#" in
-- 
2.14.0.rc0.286.g44127d70e4

Re: [PATCH v2 0/2] Add progress for format-patch and rebase

[Junio C Hamano]

Kevin Willford <kcwillford@gmail.com> writes:

> Changes since last patch:
> 1. Use start_progress_delay so progress isn't shown if generating
>    the patches is fast enough
> 2. Updated to have text of "Generating patches"
> 3. Only show progress when the --progress flag is passed
> 4. In the rebase script check stderr and the quiet option is not
>    set before propagating the progress flag to format-patch
>
> Kevin Willford (2):
>   format-patch: have progress option while generating patches
>   rebase: turn on progress option by default for format-patch

Do you have a pointer to the previous discussion handy (if you do
not, that is OK---I think I can dig the list archive myself)?

Also, your patch messages appear from you at gmail yet signed off by
you at microsoft.  Please make them consistent (I can fix them up
while queuing _this_ time, but I do not want forever doing that for
future patches from you or other people).  One easy workaround to do
so without convincing your mailer to put your microsoft address on
the sender's From: line in the e-mail is to put an extra

    From: Kevin Without <kewillf@microsoft.com>

line, followed by a blank line, at the very beginning of the body of
the e-mail message.

Thanks.

Re: [PATCH v2 0/2] Add progress for format-patch and rebase

[Jeff King]

On Thu, Aug 10, 2017 at 03:48:31PM -0700, Junio C Hamano wrote:

> Kevin Willford <kcwillford@gmail.com> writes:
> 
> > Changes since last patch:
> > 1. Use start_progress_delay so progress isn't shown if generating
> >    the patches is fast enough
> > 2. Updated to have text of "Generating patches"
> > 3. Only show progress when the --progress flag is passed
> > 4. In the rebase script check stderr and the quiet option is not
> >    set before propagating the progress flag to format-patch
> >
> > Kevin Willford (2):
> >   format-patch: have progress option while generating patches
> >   rebase: turn on progress option by default for format-patch
> 
> Do you have a pointer to the previous discussion handy (if you do
> not, that is OK---I think I can dig the list archive myself)?

https://public-inbox.org/git/20170531150427.7820-1-kewillf@microsoft.com/

is what I turned up.

Overall this version looks good to me, and addresses all of the previous
points. I have two minor points which I'll make inline.

-Peff

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Jeff King]

On Thu, Aug 10, 2017 at 02:32:55PM -0400, Kevin Willford wrote:

> @@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  		OPT_FILENAME(0, "signature-file", &signature_file,
>  				N_("add a signature from a file")),
>  		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
> +		OPT_BOOL(0, "progress", &show_progress,
> +			 N_("show progress while generating patches")),

Earlier I suggested allowing --progress="custom text" since this may be
driven as plumbing for other commands. But I don't think there's any
need to worry about it now. It can be added seamlessly later if we find
such a caller.

> @@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  		start_number--;
>  	}
>  	rev.add_signoff = do_signoff;
> +
> +	if (show_progress)
> +		progress = start_progress_delay(_("Generating patches"), total, 0, 1);

I don't really have an opinion on a 1 second delay versus 2. I thought
we used 2 pretty consistently, though grepping around I do see a couple
of 1's. It probably doesn't matter, but just a curiosity.

-Peff

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Thu, Aug 10, 2017 at 02:32:55PM -0400, Kevin Willford wrote:
>
>> @@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  		OPT_FILENAME(0, "signature-file", &signature_file,
>>  				N_("add a signature from a file")),
>>  		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
>> +		OPT_BOOL(0, "progress", &show_progress,
>> +			 N_("show progress while generating patches")),
>
> Earlier I suggested allowing --progress="custom text" since this may be
> driven as plumbing for other commands. But I don't think there's any
> need to worry about it now. It can be added seamlessly later if we find
> such a caller.
>
>> @@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  		start_number--;
>>  	}
>>  	rev.add_signoff = do_signoff;
>> +
>> +	if (show_progress)
>> +		progress = start_progress_delay(_("Generating patches"), total, 0, 1);
>
> I don't really have an opinion on a 1 second delay versus 2. I thought
> we used 2 pretty consistently, though grepping around I do see a couple
> of 1's. It probably doesn't matter, but just a curiosity.

Yeah, I also thought 2-second was what we used by default.  Perhaps
we would want to bring others in line?

Re: [PATCH v2 2/2] rebase: turn on progress option by default for format-patch

[Junio C Hamano]

Kevin Willford <kcwillford@gmail.com> writes:

> This change passes the progress option of format-patch checking
> that stderr is attached and rebase is not being run in quiet mode.

    Pass the "--progress" option to format-patch when the standard
    error stream goes to the terminal and the command is not run in
    "--quiet" mode.

Thanks for posting a refreshed version.  Both patches make sense to
me.

>
> Signed-off-by: Kevin Willford <kewillf@microsoft.com>

We want to see our authors and signed-off match.  Do you want the
employer address appear in our "shortlog -e" output, or your
personal gmail address?  I'll tenatively "fix" your author address
to match the one at @microsoft.com while queuing.

> ---
>  git-rebase--am.sh | 1 +
>  git-rebase.sh     | 6 ++++++
>  2 files changed, 7 insertions(+)
>
> diff --git a/git-rebase--am.sh b/git-rebase--am.sh
> index 375239341f..ff98fe3a73 100644
> --- a/git-rebase--am.sh
> +++ b/git-rebase--am.sh
> @@ -53,6 +53,7 @@ else
>  
>  	git format-patch -k --stdout --full-index --cherry-pick --right-only \
>  		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> +		$git_format_patch_opt \
>  		"$revisions" ${restrict_revision+^$restrict_revision} \
>  		>"$GIT_DIR/rebased-patches"
>  	ret=$?
> diff --git a/git-rebase.sh b/git-rebase.sh
> index f8b3d1fd97..ad8415e3cf 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -74,6 +74,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
>  autostash="$(git config --bool rebase.autostash || echo false)"
>  fork_point=auto
>  git_am_opt=
> +git_format_patch_opt=
>  rebase_root=
>  force_rebase=
>  allow_rerere_autoupdate=
> @@ -445,6 +446,11 @@ else
>  	state_dir="$apply_dir"
>  fi
>  
> +if test -t 2 && test -z "$GIT_QUIET"
> +then
> +	git_format_patch_opt="$git_format_patch_opt --progress"
> +fi
> +
>  if test -z "$rebase_root"
>  then
>  	case "$#" in

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Philip Oakley]

From: "Junio C Hamano" <gitster@pobox.com>
> Jeff King <peff@peff.net> writes:
>
>> On Thu, Aug 10, 2017 at 02:32:55PM -0400, Kevin Willford wrote:
>>
>>> @@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, 
>>> const char *prefix)
>>>  OPT_FILENAME(0, "signature-file", &signature_file,
>>>  N_("add a signature from a file")),
>>>  OPT__QUIET(&quiet, N_("don't print the patch filenames")),
>>> + OPT_BOOL(0, "progress", &show_progress,
>>> + N_("show progress while generating patches")),
>>
>> Earlier I suggested allowing --progress="custom text" since this may be
>> driven as plumbing for other commands. But I don't think there's any
>> need to worry about it now. It can be added seamlessly later if we find
>> such a caller.
>>
>>> @@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, 
>>> const char *prefix)
>>>  start_number--;
>>>  }
>>>  rev.add_signoff = do_signoff;
>>> +
>>> + if (show_progress)
>>> + progress = start_progress_delay(_("Generating patches"), total, 0, 1);
>>
>> I don't really have an opinion on a 1 second delay versus 2. I thought
>> we used 2 pretty consistently, though grepping around I do see a couple
>> of 1's. It probably doesn't matter, but just a curiosity.
>
> Yeah, I also thought 2-second was what we used by default.  Perhaps
> we would want to bring others in line?

<bikeshed> Surely the choice should depend on the purpose of the delay. IIRC 
the 1 second between patches on the formal 'sent' time was simply to ensure 
the patches had the right sequence. Delays for warnings and progress have 
different purposes, so I think it's more about the purpose than 
standardising the time (along with expectations [least surprise] if other 
messages are displayed).
--
Philip 

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Jeff King]

On Sat, Aug 12, 2017 at 09:06:18AM +0100, Philip Oakley wrote:

> > > > + progress = start_progress_delay(_("Generating patches"), total, 0, 1);
> > > 
> > > I don't really have an opinion on a 1 second delay versus 2. I thought
> > > we used 2 pretty consistently, though grepping around I do see a couple
> > > of 1's. It probably doesn't matter, but just a curiosity.
> > 
> > Yeah, I also thought 2-second was what we used by default.  Perhaps
> > we would want to bring others in line?
> 
> <bikeshed> Surely the choice should depend on the purpose of the delay. IIRC
> the 1 second between patches on the formal 'sent' time was simply to ensure
> the patches had the right sequence. Delays for warnings and progress have
> different purposes, so I think it's more about the purpose than
> standardising the time (along with expectations [least surprise] if other
> messages are displayed).

I think you're confusing two unrelated things. There's a 1-second fake
time-advance on patches, but that's done by send-email, not
format-patch.

Here we're just talking about calls to start_progress_delay(), and how
long it waits before deciding that the operation is slow enough to show
progress. Blame, rename detection, and checkout use 1 second. Prune,
prune-packed, and connectivity checks all use 2 seconds. I doubt it
matters all that much, but presumably the purpose in all is "how long
before a user starts to wonder if things are actually happening", which
is probably command-independent.

-Peff

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Sat, Aug 12, 2017 at 09:06:18AM +0100, Philip Oakley wrote:
>
>> > > > + progress = start_progress_delay(_("Generating patches"), total, 0, 1);
>> > > 
>> > > I don't really have an opinion on a 1 second delay versus 2. I thought
>> > > we used 2 pretty consistently, though grepping around I do see a couple
>> > > of 1's. It probably doesn't matter, but just a curiosity.
>> > 
> ...
> Here we're just talking about calls to start_progress_delay(), and how
> long it waits before deciding that the operation is slow enough to show
> progress. Blame, rename detection, and checkout use 1 second. Prune,
> prune-packed, and connectivity checks all use 2 seconds. I doubt it
> matters all that much, but presumably the purpose in all is "how long
> before a user starts to wonder if things are actually happening", which
> is probably command-independent.

I feel comfortable moving forward, basing our decisions on the
assumption that the "delay before the user wonders is independent of
the command" without anybody actually proving it.

Even though I think that it is natural for people to expect longer
delay from some operation than others (due to some chicken-and-egg
reasons), trying to scientifically measure and to come up with
different delay value that suited for each and every operation is
waste of our brain cycles.

Perhaps we may want to replace the calls to progress_delay() with a
call to a simpler wrapper that does not let the callers give their
own delay threashold to simplify the API.

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Perhaps we may want to replace the calls to progress_delay() with a
> call to a simpler wrapper that does not let the callers give their
> own delay threashold to simplify the API.

... which does not look too bad, but because it makes me wonder if
we even need to make the delay-threshold customizable per callers,
I'll wait further discussions before committing to the approach.

For example, I do not quite understand why 95 is a good value for
prune-packed while 0 is a good value for prune.  

The rename detection in diffcore-rename, delay in blame, and
checkout (via unpack-trees) all of which use 50-percent threshold
with 1 second delay, sort of make sense to me in that if we
completed 50 percent within 1 second, it is likely we will finish
all in 2 seconds (which is the norm for everybody else), perhaps as
a better version of 0-percent 2 seconds rule.

 builtin/blame.c        |  3 +--
 builtin/fsck.c         |  2 +-
 builtin/prune-packed.c |  4 ++--
 builtin/prune.c        |  2 +-
 builtin/rev-list.c     |  2 +-
 diffcore-rename.c      |  4 ++--
 progress.c             | 10 ++++++++--
 progress.h             |  4 ++--
 unpack-trees.c         |  3 +--
 9 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/builtin/blame.c b/builtin/blame.c
index bda1a78726..05115900f3 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -925,8 +925,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
 	sb.found_guilty_entry = &found_guilty_entry;
 	sb.found_guilty_entry_data = &pi;
 	if (show_progress)
-		pi.progress = start_progress_delay(_("Blaming lines"),
-						   sb.num_lines, 50, 1);
+		pi.progress = start_delayed_progress(_("Blaming lines"), sb.num_lines, 50);
 
 	assign_blame(&sb, opt);
 
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 99dea7adf6..6a26079ebc 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -188,7 +188,7 @@ static int traverse_reachable(void)
 	unsigned int nr = 0;
 	int result = 0;
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0, 0);
 	while (pending.nr) {
 		struct object_array_entry *entry;
 		struct object *obj;
diff --git a/builtin/prune-packed.c b/builtin/prune-packed.c
index ac978ad401..4fc6b175de 100644
--- a/builtin/prune-packed.c
+++ b/builtin/prune-packed.c
@@ -37,8 +37,8 @@ static int prune_object(const struct object_id *oid, const char *path,
 void prune_packed_objects(int opts)
 {
 	if (opts & PRUNE_PACKED_VERBOSE)
-		progress = start_progress_delay(_("Removing duplicate objects"),
-			256, 95, 2);
+		progress = start_delayed_progress(_("Removing duplicate objects"),
+						  256, 95);
 
 	for_each_loose_file_in_objdir(get_object_directory(),
 				      prune_object, NULL, prune_subdir, &opts);
diff --git a/builtin/prune.c b/builtin/prune.c
index c378690545..f32adaa0e9 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -138,7 +138,7 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0, 0);
 
 	mark_reachable_objects(&revs, 1, expire, progress);
 	stop_progress(&progress);
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 95d84d5cda..5cfc4b35f4 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -364,7 +364,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 		revs.limited = 1;
 
 	if (show_progress)
-		progress = start_progress_delay(show_progress, 0, 0, 2);
+		progress = start_delayed_progress(show_progress, 0, 0);
 
 	if (use_bitmap_index && !revs.prune) {
 		if (revs.count && !revs.left_right && !revs.cherry_mark) {
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 786f389498..0eafa43618 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -532,9 +532,9 @@ void diffcore_rename(struct diff_options *options)
 	}
 
 	if (options->show_rename_progress) {
-		progress = start_progress_delay(
+		progress = start_delayed_progress(
 				_("Performing inexact rename detection"),
-				rename_dst_nr * rename_src_nr, 50, 1);
+				rename_dst_nr * rename_src_nr, 50);
 	}
 
 	mx = xcalloc(st_mult(NUM_CANDIDATE_PER_DST, num_create), sizeof(*mx));
diff --git a/progress.c b/progress.c
index 73e36d4a42..eeebe15b6f 100644
--- a/progress.c
+++ b/progress.c
@@ -205,8 +205,8 @@ int display_progress(struct progress *progress, unsigned n)
 	return progress ? display(progress, n, NULL) : 0;
 }
 
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay)
+static struct progress *start_progress_delay(const char *title, unsigned total,
+					     unsigned percent_treshold, unsigned delay)
 {
 	struct progress *progress = malloc(sizeof(*progress));
 	if (!progress) {
@@ -227,6 +227,12 @@ struct progress *start_progress_delay(const char *title, unsigned total,
 	return progress;
 }
 
+struct progress *start_delayed_progress(const char *title,
+					unsigned total,	unsigned percent_treshold)
+{
+	return start_progress_delay(title, total, percent_treshold, 2);
+}
+
 struct progress *start_progress(const char *title, unsigned total)
 {
 	return start_progress_delay(title, total, 0, 0);
diff --git a/progress.h b/progress.h
index 611e4c4d42..c7fc431989 100644
--- a/progress.h
+++ b/progress.h
@@ -6,8 +6,8 @@ struct progress;
 void display_throughput(struct progress *progress, off_t total);
 int display_progress(struct progress *progress, unsigned n);
 struct progress *start_progress(const char *title, unsigned total);
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay);
+struct progress *start_delayed_progress(const char *title, unsigned total,
+					unsigned percent_treshold);
 void stop_progress(struct progress **progress);
 void stop_progress_msg(struct progress **progress, const char *msg);
 
diff --git a/unpack-trees.c b/unpack-trees.c
index dd535bc849..03d1b37578 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -343,8 +343,7 @@ static struct progress *get_progress(struct unpack_trees_options *o)
 			total++;
 	}
 
-	return start_progress_delay(_("Checking out files"),
-				    total, 50, 1);
+	return start_delayed_progress(_("Checking out files"), total, 50);
 }
 
 static int check_updates(struct unpack_trees_options *o)

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Jeff King]

On Mon, Aug 14, 2017 at 11:35:33AM -0700, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Perhaps we may want to replace the calls to progress_delay() with a
> > call to a simpler wrapper that does not let the callers give their
> > own delay threashold to simplify the API.
> 
> ... which does not look too bad, but because it makes me wonder if
> we even need to make the delay-threshold customizable per callers,
> I'll wait further discussions before committing to the approach.

For what it's worth, I had a similar "why is this even part of the API"
thought when writing earlier in the thread.

> For example, I do not quite understand why 95 is a good value for
> prune-packed while 0 is a good value for prune.

And I also wondered this same thing. I do not have a good answer.

> The rename detection in diffcore-rename, delay in blame, and
> checkout (via unpack-trees) all of which use 50-percent threshold
> with 1 second delay, sort of make sense to me in that if we
> completed 50 percent within 1 second, it is likely we will finish
> all in 2 seconds (which is the norm for everybody else), perhaps as
> a better version of 0-percent 2 seconds rule.

Just thinking what could go wrong for a moment.

If we reach 51% in one second, would we then never show progress? That
seems like a misfeature when the counter is spiky. E.g., consider
something like object transfer (which doesn't do a delayed progress, but
pretend for a moment as it makes a simple example). Imagine we have 100
objects, 99 of which are 10KB and one of which is 100MB. And that the
big object comes at slot 75.

No matter how the delay works, the counter is going to jump quickly to
75% as we send the small objects, and then appear to stall on the large
one. That's unavoidable without feeding the progress code more data
about the items. But what does the user see?

With (0,2), we start the progress meter at 2 seconds, the user sees it
stall at 75%, and then eventually it unclogs.

With (50,1), we check the percentage after 1 second and see we are
already at 75%. We then disable the meter totally. After 2 seconds, we
get the same stall but the user sees nothing.

So in that case we should always base our decision only on time taken.

And that gives me an answer for your question above: the difference is
whether we expect the counter to move smoothly, or to be spiky.

If it's smooth, the (50,1) case is slightly nicer in that it puts the
progress in front of the user more quickly. I'm not sure if that's
actually worth pushing an additional decision onto the person writing
the calling code, though (especially when we are just now puzzling out
the method for making such a decision from first principles).

So I'd vote to drop that parameter entirely. And if 1 second seems
noticeably snappier, then we should probably just move everything to a 1
second delay (I don't have a strong feeling either way).

-Peff

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> If it's smooth, the (50,1) case is slightly nicer in that it puts the
> progress in front of the user more quickly. I'm not sure if that's
> actually worth pushing an additional decision onto the person writing
> the calling code, though (especially when we are just now puzzling out
> the method for making such a decision from first principles).
>
> So I'd vote to drop that parameter entirely. And if 1 second seems
> noticeably snappier, then we should probably just move everything to a 1
> second delay (I don't have a strong feeling either way).

Sounds like a good idea to me.  

I've already locally tweaked Kevin's patch to use (0,2) instead of
(0,1) without introducing the simpler wrapper.  It should be trivial
to do a wrapper to catch and migrate all the (0,2) users to a
start_delayed_progress() that takes neither percentage or time with
mechanical replacement.

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Jeff King]

On Mon, Aug 14, 2017 at 03:42:14PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > If it's smooth, the (50,1) case is slightly nicer in that it puts the
> > progress in front of the user more quickly. I'm not sure if that's
> > actually worth pushing an additional decision onto the person writing
> > the calling code, though (especially when we are just now puzzling out
> > the method for making such a decision from first principles).
> >
> > So I'd vote to drop that parameter entirely. And if 1 second seems
> > noticeably snappier, then we should probably just move everything to a 1
> > second delay (I don't have a strong feeling either way).
> 
> Sounds like a good idea to me.  
> 
> I've already locally tweaked Kevin's patch to use (0,2) instead of
> (0,1) without introducing the simpler wrapper.  It should be trivial
> to do a wrapper to catch and migrate all the (0,2) users to a
> start_delayed_progress() that takes neither percentage or time with
> mechanical replacement.

I was actually proposing to move (50,1) cases to the simpler wrapper,
too. IOW, drop the delayed_percent_treshold code entirely.

-Peff

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Mon, Aug 14, 2017 at 03:42:14PM -0700, Junio C Hamano wrote:
>
>> Jeff King <peff@peff.net> writes:
>> 
>> > If it's smooth, the (50,1) case is slightly nicer in that it puts the
>> > progress in front of the user more quickly. I'm not sure if that's
>> > actually worth pushing an additional decision onto the person writing
>> > the calling code, though (especially when we are just now puzzling out
>> > the method for making such a decision from first principles).
>> >
>> > So I'd vote to drop that parameter entirely. And if 1 second seems
>> > noticeably snappier, then we should probably just move everything to a 1
>> > second delay (I don't have a strong feeling either way).
>> 
>> Sounds like a good idea to me.  
>> 
>> I've already locally tweaked Kevin's patch to use (0,2) instead of
>> (0,1) without introducing the simpler wrapper.  It should be trivial
>> to do a wrapper to catch and migrate all the (0,2) users to a
>> start_delayed_progress() that takes neither percentage or time with
>> mechanical replacement.
>
> I was actually proposing to move (50,1) cases to the simpler wrapper,
> too. IOW, drop the delayed_percent_treshold code entirely.

I should have mentioned that (50,1) should also be treated just like
(0,2) case--they should mean the same thing.  Other oddness like 95
might merit individual consideration, and I didn't want to add (0,1)
as another oddball to the mix.

[PATCH] progress: simplify "delayed" progress API

[Junio C Hamano]

We used to expose the full power of the delayed progress API to the
callers, so that they can specify, not just the message to show and
expected total amount of work that is used to compute the percentage
of work performed so far, the percent-threshold parameter P and the
delay-seconds parameter N.  The progress meter starts to show at N
seconds into the operation only if the amount of work completed
exceeds P.

Most callers used either (0%, 2s) or (50%, 1s) as (P, N), but there
are oddballs that chose more random-looking values like 95%.

For a smoother workload, (50%, 1s) would allow us to start showing
the progress meter earlier than (0%, 2s), while keeping the chance
of not showing progress meter for long running operation the same as
the latter.  For a task that would take 2s or more to complete, it
is likely that less than half of it would complete within the first
second, if the workload is smooth.  But for a spiky workload whose
earlier part is easier, such a setting is likely to fail to show the
progress meter entirely and (0%, 2s) is more appropriate.

But that is merely a theory.  Realistically, it is of dubious value
to ask each codepath to carefully consider smoothness of their
workload and specify their own setting by passing two extra
parameters.  Let's simplify the API by dropping both parameters and
have everybody use (0%, 2s).

Oh, by the way, the percent-threshold parameter and the structure
member were consistently misspelled, which also is now fixed ;-)

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * So before I forget all about this topic, here is a patch to
   actually do this.

   > So I'd vote to drop that parameter entirely. And if 1 second seems
   > noticeably snappier, then we should probably just move everything to a 1
   > second delay (I don't have a strong feeling either way).

   I was also tempted to do this, but decided to keep it closer to
   the original for majority of callers by leaving the delay at 2s.
   With this patch, such a change as a follow-up is made a lot
   easier (somebody may want to even make a configuration out of it,
   but that would not be me).

 builtin/blame.c        |  3 +--
 builtin/fsck.c         |  2 +-
 builtin/prune-packed.c |  3 +--
 builtin/prune.c        |  2 +-
 builtin/rev-list.c     |  2 +-
 diffcore-rename.c      |  4 ++--
 progress.c             | 15 ++++++++++-----
 progress.h             |  3 +--
 unpack-trees.c         |  3 +--
 9 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/builtin/blame.c b/builtin/blame.c
index bda1a78726..e0daf17548 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -925,8 +925,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
 	sb.found_guilty_entry = &found_guilty_entry;
 	sb.found_guilty_entry_data = &pi;
 	if (show_progress)
-		pi.progress = start_progress_delay(_("Blaming lines"),
-						   sb.num_lines, 50, 1);
+		pi.progress = start_delayed_progress(_("Blaming lines"), sb.num_lines);
 
 	assign_blame(&sb, opt);
 
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 99dea7adf6..0031439fc4 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -188,7 +188,7 @@ static int traverse_reachable(void)
 	unsigned int nr = 0;
 	int result = 0;
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0);
 	while (pending.nr) {
 		struct object_array_entry *entry;
 		struct object *obj;
diff --git a/builtin/prune-packed.c b/builtin/prune-packed.c
index ac978ad401..8f41f7c20e 100644
--- a/builtin/prune-packed.c
+++ b/builtin/prune-packed.c
@@ -37,8 +37,7 @@ static int prune_object(const struct object_id *oid, const char *path,
 void prune_packed_objects(int opts)
 {
 	if (opts & PRUNE_PACKED_VERBOSE)
-		progress = start_progress_delay(_("Removing duplicate objects"),
-			256, 95, 2);
+		progress = start_delayed_progress(_("Removing duplicate objects"), 256);
 
 	for_each_loose_file_in_objdir(get_object_directory(),
 				      prune_object, NULL, prune_subdir, &opts);
diff --git a/builtin/prune.c b/builtin/prune.c
index c378690545..cddabf26a9 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -138,7 +138,7 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0);
 
 	mark_reachable_objects(&revs, 1, expire, progress);
 	stop_progress(&progress);
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 95d84d5cda..dfad8e847a 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -364,7 +364,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 		revs.limited = 1;
 
 	if (show_progress)
-		progress = start_progress_delay(show_progress, 0, 0, 2);
+		progress = start_delayed_progress(show_progress, 0);
 
 	if (use_bitmap_index && !revs.prune) {
 		if (revs.count && !revs.left_right && !revs.cherry_mark) {
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 786f389498..0d8c3d2ee4 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -532,9 +532,9 @@ void diffcore_rename(struct diff_options *options)
 	}
 
 	if (options->show_rename_progress) {
-		progress = start_progress_delay(
+		progress = start_delayed_progress(
 				_("Performing inexact rename detection"),
-				rename_dst_nr * rename_src_nr, 50, 1);
+				rename_dst_nr * rename_src_nr);
 	}
 
 	mx = xcalloc(st_mult(NUM_CANDIDATE_PER_DST, num_create), sizeof(*mx));
diff --git a/progress.c b/progress.c
index 73e36d4a42..289678d43d 100644
--- a/progress.c
+++ b/progress.c
@@ -34,7 +34,7 @@ struct progress {
 	unsigned total;
 	unsigned last_percent;
 	unsigned delay;
-	unsigned delayed_percent_treshold;
+	unsigned delayed_percent_threshold;
 	struct throughput *throughput;
 	uint64_t start_ns;
 };
@@ -88,7 +88,7 @@ static int display(struct progress *progress, unsigned n, const char *done)
 			return 0;
 		if (progress->total) {
 			unsigned percent = n * 100 / progress->total;
-			if (percent > progress->delayed_percent_treshold) {
+			if (percent > progress->delayed_percent_threshold) {
 				/* inhibit this progress report entirely */
 				clear_progress_signal();
 				progress->delay = -1;
@@ -205,8 +205,8 @@ int display_progress(struct progress *progress, unsigned n)
 	return progress ? display(progress, n, NULL) : 0;
 }
 
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay)
+static struct progress *start_progress_delay(const char *title, unsigned total,
+					     unsigned percent_threshold, unsigned delay)
 {
 	struct progress *progress = malloc(sizeof(*progress));
 	if (!progress) {
@@ -219,7 +219,7 @@ struct progress *start_progress_delay(const char *title, unsigned total,
 	progress->total = total;
 	progress->last_value = -1;
 	progress->last_percent = -1;
-	progress->delayed_percent_treshold = percent_treshold;
+	progress->delayed_percent_threshold = percent_threshold;
 	progress->delay = delay;
 	progress->throughput = NULL;
 	progress->start_ns = getnanotime();
@@ -227,6 +227,11 @@ struct progress *start_progress_delay(const char *title, unsigned total,
 	return progress;
 }
 
+struct progress *start_delayed_progress(const char *title, unsigned total)
+{
+	return start_progress_delay(title, total, 0, 2);
+}
+
 struct progress *start_progress(const char *title, unsigned total)
 {
 	return start_progress_delay(title, total, 0, 0);
diff --git a/progress.h b/progress.h
index 611e4c4d42..6392b63371 100644
--- a/progress.h
+++ b/progress.h
@@ -6,8 +6,7 @@ struct progress;
 void display_throughput(struct progress *progress, off_t total);
 int display_progress(struct progress *progress, unsigned n);
 struct progress *start_progress(const char *title, unsigned total);
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay);
+struct progress *start_delayed_progress(const char *title, unsigned total);
 void stop_progress(struct progress **progress);
 void stop_progress_msg(struct progress **progress, const char *msg);
 
diff --git a/unpack-trees.c b/unpack-trees.c
index dd535bc849..e5ae7fe183 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -343,8 +343,7 @@ static struct progress *get_progress(struct unpack_trees_options *o)
 			total++;
 	}
 
-	return start_progress_delay(_("Checking out files"),
-				    total, 50, 1);
+	return start_delayed_progress(_("Checking out files"), total);
 }
 
 static int check_updates(struct unpack_trees_options *o)
-- 
2.14.1-396-gef255185d2

Re: [PATCH] progress: simplify "delayed" progress API

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> We used to expose the full power of the delayed progress API to the
> callers, so that they can specify, not just the message to show and
> expected total amount of work that is used to compute the percentage
> of work performed so far, the percent-threshold parameter P and the
> delay-seconds parameter N.  The progress meter starts to show at N
> seconds into the operation only if the amount of work completed
> exceeds P.

Oops, we inspect the doneness at N seconds and show only if we have
not yet completed P percent of the total work.

Perhaps

	... at N seconds into the operation only if we have not
	completed P per-cent of the total work.

Re: [PATCH] progress: simplify "delayed" progress API

[Jeff King]

On Sat, Aug 19, 2017 at 10:39:41AM -0700, Junio C Hamano wrote:

> We used to expose the full power of the delayed progress API to the
> callers, so that they can specify, not just the message to show and
> expected total amount of work that is used to compute the percentage
> of work performed so far, the percent-threshold parameter P and the
> delay-seconds parameter N.  The progress meter starts to show at N
> seconds into the operation only if the amount of work completed
> exceeds P.
> 
> Most callers used either (0%, 2s) or (50%, 1s) as (P, N), but there
> are oddballs that chose more random-looking values like 95%.
> 
> For a smoother workload, (50%, 1s) would allow us to start showing
> the progress meter earlier than (0%, 2s), while keeping the chance
> of not showing progress meter for long running operation the same as
> the latter.  For a task that would take 2s or more to complete, it
> is likely that less than half of it would complete within the first
> second, if the workload is smooth.  But for a spiky workload whose
> earlier part is easier, such a setting is likely to fail to show the
> progress meter entirely and (0%, 2s) is more appropriate.
> 
> But that is merely a theory.  Realistically, it is of dubious value
> to ask each codepath to carefully consider smoothness of their
> workload and specify their own setting by passing two extra
> parameters.  Let's simplify the API by dropping both parameters and
> have everybody use (0%, 2s).

Nicely explained (modulo the reversal you noticed in the first
paragraph). The patch looks good to me.

> Oh, by the way, the percent-threshold parameter and the structure
> member were consistently misspelled, which also is now fixed ;-)

Heh, that was bugging me, too. :)

>  * So before I forget all about this topic, here is a patch to
>    actually do this.
> 
>    > So I'd vote to drop that parameter entirely. And if 1 second seems
>    > noticeably snappier, then we should probably just move everything to a 1
>    > second delay (I don't have a strong feeling either way).
> 
>    I was also tempted to do this, but decided to keep it closer to
>    the original for majority of callers by leaving the delay at 2s.
>    With this patch, such a change as a follow-up is made a lot
>    easier (somebody may want to even make a configuration out of it,
>    but that would not be me).

Yeah, I agree that it can come later on top (if at all).

Thanks for finishing this off.

-Peff