git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: Martin Langhoff <martin.langhoff@gmail.com>,
	Jonathan Nieder <jrnieder@gmail.com>,
	e@80x24.org, Git Mailing List <git@vger.kernel.org>
Subject: Re: git svn clone/fetch hits issues with gc --auto
Date: Wed, 10 Oct 2018 21:21:48 +0900	[thread overview]
Message-ID: <xmqqzhvmkn4z.fsf@gitster-ct.c.googlers.com> (raw)
In-Reply-To: <878t36f3ed.fsf@evledraar.gmail.com> ("Ævar Arnfjörð Bjarmason"'s message of "Wed, 10 Oct 2018 13:27:06 +0200")

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>  - We use this warning as a proxy for "let's not run for a day",
>    otherwise we'll just grind on gc --auto trying to consolidate
>    possibly many hundreds of K of loose objects only to find none of
>    them can be pruned because the run into the expiry policy. With the
>    warning we retry that once per day, which sucks less.
>
>  - This conflation of the user-visible warning and the policy is an
>    emergent effect of how the different gc pieces interact, which as I
>    note in the linked thread(s) sucks.
>
>    But we can't just yank one piece away (as Jonathan's patch does)
>    without throwing the baby out with the bathwater.
>
>    It will mean that e.g. if you have 10k loose objects in your git.git,
>    and created them just now, that every time you run anything that runs
>    "gc --auto" we'll fork to the background, peg a core at 100% CPU for
>    2-3 minutes or whatever it is, only do get nowhere and do the same
>    thing again in ~3 minutes when you run your next command.

We probably can keep the "let's not run for a day" safety while
pretending that "git gc -auto" succeeded for callers like "git svn"
so that these callers do not hae to do "eval { ... }" to hide our
exit code, no?

I think that is what Jonathan's patch (jn/gc-auto) does.

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Mon, 16 Jul 2018 23:57:40 -0700
Subject: [PATCH] gc: do not return error for prior errors in daemonized mode

diff --git a/builtin/gc.c b/builtin/gc.c
index 95c8afd07b..ce8a663a01 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -438,9 +438,15 @@ static const char *lock_repo_for_gc(int force, pid_t* ret_pid)
 	return NULL;
 }
 
-static void report_last_gc_error(void)
+/*
+ * Returns 0 if there was no previous error and gc can proceed, 1 if
+ * gc should not proceed due to an error in the last run. Prints a
+ * message and returns -1 if an error occured while reading gc.log
+ */
+static int report_last_gc_error(void)
 {
 	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
...
 	if (len < 0)
+		ret = error_errno(_("cannot read '%s'"), gc_log_path);
+	else if (len > 0) {
+		/*
+		 * A previous gc failed.  Report the error, and don't
+		 * bother with an automatic gc run since it is likely
+		 * to fail in the same way.
+		 */
+		warning(_("The last gc run reported the following. "
 			       "Please correct the root cause\n"
 			       "and remove %s.\n"
 			       "Automatic cleanup will not be performed "
 			       "until the file is removed.\n\n"
 			       "%s"),
 			    gc_log_path, sb.buf);
+		ret = 1;
+	}
 	strbuf_release(&sb);
 done:
 	free(gc_log_path);
+	return ret;
 }
 
I.e. report_last_gc_error() returns 1 when finds that the previous
attempt to "gc --auto" failed.  And then

@@ -561,7 +576,13 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 			fprintf(stderr, _("See \"git help gc\" for manual housekeeping.\n"));
 		}
 		if (detach_auto) {
-			report_last_gc_error(); /* dies on error */
+			int ret = report_last_gc_error();
+			if (ret < 0)
+				/* an I/O error occured, already reported */
+				exit(128);
+			if (ret == 1)
+				/* Last gc --auto failed. Skip this one. */
+				return 0;

... it exits with 0 without bothering to rerun "gc".

So it won't get stuck for 3 minutes; the repository after "gc
--auto" punts will stay to be suboptimal for a day, and the user
kill not get an "actionable" error notice (due to this hiding of
previous error), hence cannot make changes that may help like
shortening expiry period, though.


  parent reply	other threads:[~2018-10-10 12:21 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CACPiFCJZ83sqE7Gaj2pa12APkBF5tau-C6t4_GrXBWDwcMnJHg@mail.gmail.com>
2018-10-09 22:51 ` git svn clone/fetch hits issues with gc --auto Martin Langhoff
2018-10-09 23:45   ` Eric Wong
2018-10-10  2:49     ` Junio C Hamano
2018-10-10 11:01       ` Martin Langhoff
2018-10-10 11:27         ` Ævar Arnfjörð Bjarmason
2018-10-10 11:41           ` Martin Langhoff
2018-10-10 11:48             ` Ævar Arnfjörð Bjarmason
2018-10-10 16:51               ` Jonathan Nieder
2018-10-10 17:46                 ` Jeff King
2018-10-10 19:27                   ` [PATCH] gc: introduce an --auto-exit-code option for undoing 3029970275 Ævar Arnfjörð Bjarmason
2018-10-10 20:35                     ` Jeff King
2018-10-10 20:59                       ` Ævar Arnfjörð Bjarmason
2018-10-11  0:38                         ` Jeff King
2018-10-10 20:56                     ` Jonathan Nieder
2018-10-10 21:05                       ` Ævar Arnfjörð Bjarmason
2018-10-10 21:14                         ` Jonathan Nieder
2018-10-10 21:36                           ` Junio C Hamano
2018-10-10 21:51                             ` Jonathan Nieder
2018-10-10 22:16                               ` Ævar Arnfjörð Bjarmason
2018-10-10 22:25                                 ` Jonathan Nieder
2018-10-10 18:38                 ` git svn clone/fetch hits issues with gc --auto Ævar Arnfjörð Bjarmason
2018-10-10 11:43           ` Ævar Arnfjörð Bjarmason
2018-10-10 12:21           ` Junio C Hamano [this message]
2018-10-10 12:37             ` Ævar Arnfjörð Bjarmason
2018-10-10 16:38             ` Martin Langhoff
2018-10-10  8:04   ` Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqzhvmkn4z.fsf@gitster-ct.c.googlers.com \
    --to=gitster@pobox.com \
    --cc=avarab@gmail.com \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=jrnieder@gmail.com \
    --cc=martin.langhoff@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).