git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "SZEDER Gábor" <szeder.dev@gmail.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: "Jeff King" <peff@peff.net>, "Johannes Sixt" <j6t@kdbg.org>,
	"David Turner" <dturner@twosigma.com>,
	git@vger.kernel.org, "SZEDER Gábor" <szeder.dev@gmail.com>
Subject: [PATCHv2.1] t6500: wait for detached auto gc at the end of the test script
Date: Thu, 13 Apr 2017 12:31:38 +0200	[thread overview]
Message-ID: <20170413103138.12717-1-szeder.dev@gmail.com> (raw)
In-Reply-To: <20170412220717.7357-1-szeder.dev@gmail.com>

The last test in 't6500-gc', 'background auto gc does not run if
gc.log is present and recent but does if it is old', added in
a831c06a2 (gc: ignore old gc.log files, 2017-02-10), may sporadically
trigger an error message from the test harness:

  rm: cannot remove 'trash directory.t6500-gc/.git/objects': Directory not empty

The test in question ends with executing an auto gc in the backround,
which occasionally takes so long that it's still running when
'test_done' is about to remove the trash directory.  This 'rm -rf
$trash' in the foreground might race with the detached auto gc to
create and delete files and directories, and gc might (re-)create a
path that 'rm' already visited and removed, triggering the above error
message when 'rm' attempts to remove its parent directory.

Commit bb05510e5 (t5510: run auto-gc in the foreground, 2016-05-01)
fixed the same problem in a different test script by simply
disallowing background gc.  Unfortunately, what worked there is not
applicable here, because the purpose of this test is to check the
behavior of a detached auto gc.

Make sure that the test doesn't continue before the gc is finished in
the background with a clever bit of shell trickery:

  - Open fd 9 in the shell, to be inherited by the background gc
    process, because our daemonize() only closes the standard fds 0,
    1 and 2.
  - Duplicate this fd 9 to stdout.
  - Read 'git gc's stdout, and thus fd 9, through a command
    substitution.  We don't actually care about gc's output, but this
    construct has two useful properties:
  - This read blocks until stdout or fd 9 are open.  While stdout is
    closed after the main gc process creates the background process
    and exits, fd 9 remains open until the backround process exits.
  - The variable assignment from the command substitution gets its
    exit status from the command executed within the command
    substitution, i.e. a failing main gc process will cause the test
    to fail.

Note, that this fd trickery doesn't work on Windows, because due to
MSYS limitations the git process only inherits the standard fds 0, 1
and 2 from the shell.  Luckily, it doesn't matter in this case,
because on Windows daemonize() is basically a noop, thus 'git gc
--auto' always runs in the foreground.

And since we can now continue the test reliably after the detached gc
finished, check that there is only a single packfile left at the end,
i.e. that the detached gc actually did what it was supposed to do.
Also add a comment at the end of the test script to warn developers of
future tests about this issue of long running detached gc processes.

Helped-by: Jeff King <peff@peff.net>
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---

Updated subject line, but otherwise the same as v2.

 t/t6500-gc.sh | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 08de2e8ab..cc7acd101 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -67,6 +67,16 @@ test_expect_success 'auto gc with too many loose objects does not attempt to cre
 	test_line_count = 2 new # There is one new pack and its .idx
 '
 
+run_and_wait_for_auto_gc () {
+	# We read stdout from gc for the side effect of waiting until the
+	# background gc process exits, closing its fd 9.  Furthermore, the
+	# variable assignment from a command substitution preserves the
+	# exit status of the main gc process.
+	# Note: this fd trickery doesn't work on Windows, but there is no
+	# need to, because on Win the auto gc always runs in the foreground.
+	doesnt_matter=$(git gc --auto 9>&1)
+}
+
 test_expect_success 'background auto gc does not run if gc.log is present and recent but does if it is old' '
 	test_commit foo &&
 	test_commit bar &&
@@ -80,7 +90,13 @@ test_expect_success 'background auto gc does not run if gc.log is present and re
 	test-chmtime =-345600 .git/gc.log &&
 	test_must_fail git gc --auto &&
 	test_config gc.logexpiry 2.days &&
-	git gc --auto
+	run_and_wait_for_auto_gc &&
+	ls .git/objects/pack/pack-*.pack >packs &&
+	test_line_count = 1 packs
 '
 
+# DO NOT leave a detached auto gc process running near the end of the
+# test script: it can run long enough in the background to racily
+# interfere with the cleanup in 'test_done'.
+
 test_done
-- 
2.12.2.613.g9c5b79913


  reply	other threads:[~2017-04-13 10:31 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-10 12:59 [PATCH] t6500: don't run detached auto gc at the end of the test script SZEDER Gábor
2017-04-10 13:58 ` Jeff King
2017-04-10 16:31   ` SZEDER Gábor
2017-04-10 16:35     ` Jeff King
2017-04-10 16:56       ` SZEDER Gábor
2017-04-10 17:01         ` Jeff King
2017-04-11 21:32           ` Johannes Sixt
2017-04-12  0:27             ` SZEDER Gábor
2017-04-12  0:50               ` Jeff King
2017-04-12 22:03                 ` SZEDER Gábor
2017-04-12 22:07                   ` [PATCHv2] " SZEDER Gábor
2017-04-13 10:31                     ` SZEDER Gábor [this message]
2017-04-13 16:06                       ` [PATCHv2.1] t6500: wait for " David Turner
2017-04-13 16:44                       ` Jeff King
2017-04-13 18:08                         ` SZEDER Gábor
2017-04-13 18:12                           ` Jeff King
2017-04-13 16:37                   ` [PATCH] t6500: don't run " Jeff King
2017-04-13 17:55                     ` Stefan Beller
2017-04-13 17:57                       ` Jeff King
2017-04-13 19:03                         ` SZEDER Gábor
2017-04-13 19:12                           ` Jeff King
2017-04-13 19:35                             ` SZEDER Gábor
2017-04-14 20:08                               ` Jeff King
2017-04-20 16:42                                 ` SZEDER Gábor
2017-04-20 16:45                                   ` Jeff King
2017-04-20 16:52                                   ` [PATCH] test-lib: abort when can't remove trash directory SZEDER Gábor
2017-04-20 19:06                                     ` Jeff King
2017-04-21  0:48                                     ` Junio C Hamano
2017-04-21 20:06                                       ` SZEDER Gábor
2017-04-21 20:15                                     ` Jeff King
2017-04-24  0:14                                       ` Junio C Hamano
2017-04-24  1:43                                         ` Jeff King
2017-04-24  2:58                                           ` Junio C Hamano
2017-04-24  4:02                                             ` Junio C Hamano
2017-04-24  7:52                                               ` Jeff King
2017-04-24  9:39                                                 ` Torsten Bögershausen
2017-04-24  9:46                                                   ` Jeff King
2017-04-25  2:31                                                   ` Junio C Hamano
2017-04-25  6:05                                                 ` Junio C Hamano
2017-04-25  6:07                                                   ` Jeff King
2017-04-25  6:31                                                     ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170413103138.12717-1-szeder.dev@gmail.com \
    --to=szeder.dev@gmail.com \
    --cc=dturner@twosigma.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=j6t@kdbg.org \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).