From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 4F1381F953; Wed, 8 Dec 2021 18:22:47 +0000 (UTC) Date: Wed, 8 Dec 2021 18:22:47 +0000 From: Eric Wong To: Dominique Martinet Cc: Julien Moutinho , meta@public-inbox.org Subject: [PATCH] nodatacow: quiet chattr errors [was: Test failures with 1.7.0] Message-ID: <20211208182247.M197857@dcvr> References: <20211208010730.f47xxgzj53nwgvja@sourcephile.fr> <20211208040836.GA27368@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: Dominique Martinet wrote: > Eric Wong wrote on Wed, Dec 08, 2021 at 04:08:36AM +0000: > > > And another reviewer fails 3 times with: > > > > t/extsearch.t (Wstat: 2048 Tests: 145 Failed: 8) > > > > Failed tests: 68-69, 75-76, 86, 98, 102, 139 > > > > Non-zero exit status: 8 > > > > t/imapd.t (Wstat: 256 Tests: 186 Failed: 1) > > > > Failed test: 183 > > > > Non-zero exit status: 1 > > > > t/nntpd.t (Wstat: 256 Tests: 110 Failed: 1) > > > > Failed test: 104 > > > > Non-zero exit status: 1 > > (I'm that one) > > So instead of giving the output of prove -bvw t/nntpd.t (I couldn't > reproduce manually..), I'll go straight to the solution: it looks like > there's a problem with the chattr fallback when btrfs is present on the > machine (to disable cow, lib/PublicInbox/NDC_PP.pm) Hi, thanks for the extra info. Note: You might want to rm -rf ~/.cache/public-inbox/inline-c or try a different HOME directory to get a clean repro (more on this later). I also forget to note, setting TAIL='tail -F' can dump daemon output for tests (works better for GNU tail w/ inotify support). > I had initial messages about chattr not being found; adding chattr to > the build inputs gave another error message that chattr +C doesn't work > on tmpfs. > The problem here seems to be that the fallback here just checks > /proc/self/mounts and calls chattr "just in case", saying it's ok if it > fails, but some tests probably rely on stderr output being empty? Yes on tests requiring stderr to be empty. Below is a patch which should fix it; however it should only be calling chattr on btrfs mounts. So I wonder if our /proc/self/mounts parsing is broken... Can you share your contents of /proc/self/mounts? (maybe spaces/odd chars aren't handled properly...) You can also try: BTRFS_TESTDIR=/path/to/your/btrfs-mount prove -bvw t/nodatacow.t ? > If so it looks more like a test problem than anything else to me, but > perhaps the fallback path could be a bit more prudent in its calling of > chattr (or simply try silencing its output?) Agreed, the patch below silences stderr. > For testing, being a bit forceful and removing that chattr call made all > the tests pass for me. > Now the $100 question is I don't know why the Inline::C version wasn't > used; this one properly calls statfs() and does the ioctl directly only > if required so would have worked. > As far as I can see we're installing Inline::C in the build chroot, I > see it in the PERL5LIB paths; and make/gcc are also available so it > probably should work. Is there a way to check that? For public-inbox-* stuff, Inline::C isn't enabled by default even if installed. Either ~/.cache/public-inbox/inline-c needs to exist or PERL_INLINE_DIRECTORY must be set in env. However, lei will auto-create ~/.cache/public-inbox/inline-c if Socket::MsgHdr is missing (since lei is new w/ no existing users). Thus running lei will enable Inline::C for public-inbox-* commands. Thanks again for all the extra info. Hopefully this below is one step towards making things work more smoothly, but I also suspect /proc/self/mount parsing is off... ------8<----- Subject: [PATCH] nodatacow: quiet chattr errors Maybe this helps when chattr is missing or our /proc/self/mount parsing is broken, somehow. Link: https://public-inbox.org/meta/YbCPWGaJEkV6eWfo@codewreck.org/ --- lib/PublicInbox/NDC_PP.pm | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/PublicInbox/NDC_PP.pm b/lib/PublicInbox/NDC_PP.pm index 57abccbe0625..a84682d371c8 100644 --- a/lib/PublicInbox/NDC_PP.pm +++ b/lib/PublicInbox/NDC_PP.pm @@ -16,11 +16,13 @@ sub nodatacow_dir ($) { # weird chars are escaped as octal $mnt_path =~ s/\\(0[0-9]{2})/chr(oct($1))/egs; $mnt_path .= '/' unless $mnt_path =~ m!/\z!; - if (index($path, $mnt_path) == 0) { - # error goes to stderr, but non-fatal for us - system('chattr', '+C', $path); - last; - } + next if index($path, $mnt_path) < 0; + + open my $olderr, '>&', \*STDERR or die "dup(STDERR): $!"; + open STDERR, '>', '/dev/null' or die "open(/dev/null): $!"; + system(qw(chattr +C), $path); # ignore error + open STDERR, '>&', $olderr or die "dup2(..,STDERR): $!"; + last; } }