git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Eric DeCosta <edecosta@mathworks.com>
Cc: "Đoàn Trần Công Danh" <congdanhqx@gmail.com>,
	"Git ML" <git@vger.kernel.org>,
	"Jeff Hostetler" <jeffhost@microsoft.com>
Subject: Re: fsmonitor: t7527 racy on OSX?
Date: Tue, 22 Nov 2022 23:12:20 +0100	[thread overview]
Message-ID: <221122.86r0xuaawz.gmgdl@evledraar.gmail.com> (raw)
In-Reply-To: <BL0PR05MB55715FF24BD1AD53EE81A5A2D90D9@BL0PR05MB5571.namprd05.prod.outlook.com>


On Tue, Nov 22 2022, Eric DeCosta wrote:

>> -----Original Message-----
>> From: Đoàn Trần Công Danh <congdanhqx@gmail.com>
>> Sent: Monday, November 21, 2022 8:39 AM
>> To: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> Cc: Git ML <git@vger.kernel.org>; Eric DeCosta
>> <edecosta@mathworks.com>; Jeff Hostetler <jeffhost@microsoft.com>
>> Subject: Re: fsmonitor: t7527 racy on OSX?
>> 
>> On 2022-11-21 14:07:13+0100, Ævar Arnfjörð Bjarmason
>> <avarab@gmail.com> wrote:
>> > I have access to a Mac OS X M1 box (gcc104 at [1]) where t7527
>> > reliably fails due to what seems to be a race us doing something, and
>> > assuming that fsmonitor picked up on it.
>> 
>> See also https://lore.kernel.org/git/YvZbGAf+82WtNXcJ@danh.dev/
>> <https://protect-
>> us.mimecast.com/s/580RCpYn6ETDOBoycYVkUq?domain=lore.kernel.org>
>> 
>> I raised 3 months ago and it seems like Jeff Hostetler is too busy.
>> 
>> >
>> > This makes the tests pass:
>> >
>> > diff --git a/t/t7527-builtin-fsmonitor.sh
>> > b/t/t7527-builtin-fsmonitor.sh index 56c0dfffea..ce2555d558 100755
>> > --- a/t/t7527-builtin-fsmonitor.sh
>> > +++ b/t/t7527-builtin-fsmonitor.sh
>> > @@ -428,6 +428,7 @@ test_expect_success 'edit some files' '
>> > start_daemon --tf "$PWD/.git/trace" &&
>> >
>> > edit_files &&
>> > + sleep 1 &&
>> >
>> > test-tool fsmonitor-client query --token 0 &&
>> >
>> > @@ -443,6 +444,7 @@ test_expect_success 'create some files' '
>> > start_daemon --tf "$PWD/.git/trace" &&
>> >
>> > create_files &&
>> > + sleep 1 &&
>> >
>> > test-tool fsmonitor-client query --token 0 &&
>> >
>> > @@ -471,6 +473,7 @@ test_expect_success 'rename some files' '
>> > start_daemon --tf "$PWD/.git/trace" &&
>> >
>> > rename_files &&
>> > + sleep 1 &&
>> >
>> > test-tool fsmonitor-client query --token 0 &&
>> >
>> > @@ -978,6 +981,7 @@ test_expect_success
>> !UNICODE_COMPOSITION_SENSITIVE 'Unicode nfc/nfd' '
>> > mkdir test_unicode/nfd/d_${utf8_nfd} &&
>> >
>> > git -C test_unicode fsmonitor--daemon stop &&
>> > + sleep 1 &&
>> >
>> > if test_have_prereq UNICODE_NFC_PRESERVED then
>> >
>> > The failure is when we grep out the events we expect, which aren't
>> > there, but if you manually inspect them they're there. I.e. they're
>> > just not "in" yet.
>> >
>> > I thought this might be a lack of flushing or syncing in our own trace
>> > code, but adding an fsync() to trace_write() didn't do the trick.
>> >
>> > 1. https://cfarm.tetaneutral.net/news/41#
>> > <https://protect-
>> us.mimecast.com/s/S6YNCqxoXGIWkoNRHEfMzu?domain=cfarm
>> > .tetaneutral.net>
>> 
>> --
>> Danh
>
> Honestly, I'm not surprised. Stopping the daemon and grepping for
> expected results immediately there after is just asking for these
> sorts of races. Sleeping is a bit ugly, but without an explicit means
> of synchronization is probably the best that can be done. I can take a
> look at it some more as I have access to M1 Macs.

I don't see why it would have to do with stopping the daemon. If
anything that should reduce the odds that you're running into a
race. I.e. on OSX in general this will work:

	echo foo >f &&
	grep foo f

Or, the equivalent with an "echo" that's not a shell built-in. I.e. we
had a process start, print to a file, and then we grep data out of it
agin.

The reason I'm saying it should reduce them is if the "echo" were some
long-running daemon process that was still running the "grep" might fail
because the "foo" was still in some buffer and hadn't been written or
fsync'd to disk.

Anyway, all of that seems inapplicable to these failures, as we're not
stopping the daemon yet by the time we run into the synchronization
problem. We just *started* it, then renamed some files, but when we ask
for those events we don't get them back.

Maybe there's some innocuous reason for that, but I have the sinking
feeling that it might be some race between creating the files, the
kernel getting those events, acting on them, but not having sent notice
of those events to the daemon that's listening.

*That* would be much scarier, and would mean that this fsmonitor
implementation would be racy outside of our tests, wouldn't it?

  parent reply	other threads:[~2022-11-22 22:20 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-21 13:07 fsmonitor: t7527 racy on OSX? Ævar Arnfjörð Bjarmason
2022-11-21 13:38 ` Đoàn Trần Công Danh
2022-11-22 17:04   ` Eric DeCosta
2022-11-22 20:50     ` Eric DeCosta
2022-11-22 22:12     ` Ævar Arnfjörð Bjarmason [this message]
2022-11-30 23:18       ` Jeff Hostetler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=221122.86r0xuaawz.gmgdl@evledraar.gmail.com \
    --to=avarab@gmail.com \
    --cc=congdanhqx@gmail.com \
    --cc=edecosta@mathworks.com \
    --cc=git@vger.kernel.org \
    --cc=jeffhost@microsoft.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).