Re: Test failures with 1.7.0

[Dominique Martinet <asmadeus@codewreck.org>]

Julien Moutinho wrote on Thu, Dec 09, 2021 at 02:37:43AM +0100:
> I can also reproduce Infinisil's test failure with:
> $ (cd public-inbox-1.7.0; TMPDIR=/var/tmp perl -I$out/lib/perl5/site_perl t/lei_to_mail.t )
> > ok 96 - got Maildir callback
> > Use of uninitialized value in open at t/lei_to_mail.t line 263. 
> > Bail out!  No such file or directory 

I got curious on this one.
strace tells me:
----
2813384 renameat2(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/tmp/badc0ffee", AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/cur/badc0ffee:2,", RENAME_NOREPLACE) = -1 EINVAL (Invalid argument)
2813384 openat(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/new/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
2813384 newfstatat(4, "", {st_mode=S_IFDIR|0755, st_size=2, ...}, AT_EMPTY_PATH) = 0
2813384 brk(0x44f4000)                  = 0x44f4000
2813384 getdents64(4, 0x44b3e40 /* 2 entries */, 131072) = 48
2813384 getdents64(4, 0x44b3e40 /* 0 entries */, 131072) = 0
2813384 close(4)                        = 0
2813384 openat(AT_FDCWD, "/tank/pi-lei_to_mail-2813384-n7sk/maildir/cur/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
2813384 newfstatat(4, "", {st_mode=S_IFDIR|0755, st_size=2, ...}, AT_EMPTY_PATH) = 0
2813384 getdents64(4, 0x44b3e40 /* 2 entries */, 131072) = 48
2813384 getdents64(4, 0x44b3e40 /* 0 entries */, 131072) = 0
2813384 close(4)                        = 0
2813384 write(2, "Use of uninitialized value in op"..., 64) = 64
2813384 openat(AT_FDCWD, "", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
2813384 getpid()                        = 2813384
2813384 getpid()                        = 2813384
2813384 getpid()                        = 2813384
2813384 write(5, "Bail out!  No such file or direc"..., 37) = 37
----

So this one is a real bug, this appears to fix it:
----
From 50a63628d505ca1c8d36f94ab5703f87a2c5e415 Mon Sep 17 00:00:00 2001
From: Dominique Martinet <asmadeus@codewreck.org>
Date: Thu, 9 Dec 2021 11:50:51 +0900
Subject: [PATCH] syscall: fallback to rename on renameat2 EINVAL

ZFS appears to incorrectly return EINVAL on renameat2 when the operation is not
supported:
renameat2(AT_FDCWD, "...", AT_FDCWD, "...", RENAME_NOREPLACE) = -1 EINVAL

Fall back to the racy rename in this case as well:

diff --git a/lib/PublicInbox/Syscall.pm b/lib/PublicInbox/Syscall.pm
index c00385b94db8..78f926ac38f0 100644
--- a/lib/PublicInbox/Syscall.pm
+++ b/lib/PublicInbox/Syscall.pm
@@ -15,7 +15,7 @@ package PublicInbox::Syscall;
 use strict;
 use v5.10.1;
 use parent qw(Exporter);
-use POSIX qw(ENOENT EEXIST ENOSYS O_NONBLOCK);
+use POSIX qw(ENOENT EEXIST ENOSYS EINVAL O_NONBLOCK);
 use Config;
 
 # $VERSION = '0.25'; # Sys::Syscall version
@@ -312,7 +312,7 @@ sub rename_noreplace ($$) {
 		my $ret = syscall($SYS_renameat2, -100, $old, -100, $new, 1);
 		if ($ret == 0) {
 			1; # like rename() perlop
-		} elsif ($! == ENOSYS) {
+		} elsif ($! == ENOSYS || $! == EINVAL) {
 			undef $SYS_renameat2;
 			_rename_noreplace_racy($old, $new);
 		} else {
----

> This test does succeed outside Nix's sandbox:
> $ (cd public-inbox-1.7.0; export PERL_INLINE_DIRECTORY=$PWD/inline-c; rm -rf $PERL_INLINE_DIRECTORY; mkdir $PERL_INLINE_DIRECTORY; prove -bvw t/lei-sigpipe.t )
> > t/lei-sigpipe.t ..               
> > ok 1 - lei import $TMPDIR/lei-daemon/big.eml
> > ok 2 - read one byte             
> > ok 3 - signaled                  
> > ok 4 - got SIGPIPE               
> > ok 5 - quiet after sigpipe 
> > ok 6 - read one byte
> > ok 7 - signaled -f mboxcl2       
> > ok 8 - got SIGPIPE -f mboxcl2    
> > ok 9 - quiet after sigpipe -f mboxcl2
> > ok 10 - read one byte
> > ok 11 - signaled -f text         
> > ok 12 - got SIGPIPE -f text
> > ok 13 - quiet after sigpipe -f text
> > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44)
> > ok 15 - daemon running after t/lei-sigpipe.t:44
> > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44)
> > ok 17 - t/lei-sigpipe.t:44 daemon stopped
> > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty
> > 1..18
> > ok
> > All tests successful.
> > Files=1, Tests=18,  7 wallclock secs ( 0.06 usr  0.06 sys +  3.44 cusr  2.73 csys =  6.29 CPU)
> > Result: PASS
> 
> More surprisingly, it even succeeds when run manually
> inside the hanging Nix sandbox:
> $ sudo nsenter --target 3137110 --all -S 1000 -G 100 $(readlink -e $(which bash))
> $ . /build/env-vars
> $ cd /build
> $ export HOME=$(mktemp -d)
> $ mkdir -p $HOME/.cache/public-inbox/inline-c
> $ LANG=C prove -bvw t/lei-sigpipe.t
> > t/lei-sigpipe.t .. 
> > ok 1 - lei import $TMPDIR/lei-daemon/big.eml
> > ok 2 - read one byte
> > ok 3 - signaled 
> > ok 4 - got SIGPIPE 
> > ok 5 - quiet after sigpipe 
> > ok 6 - read one byte
> > ok 7 - signaled -f mboxcl2
> > ok 8 - got SIGPIPE -f mboxcl2
> > ok 9 - quiet after sigpipe -f mboxcl2
> > ok 10 - read one byte
> > ok 11 - signaled -f text
> > ok 12 - got SIGPIPE -f text
> > ok 13 - quiet after sigpipe -f text
> > ok 14 - lei daemon-pid (daemon-pid after t/lei-sigpipe.t:44)
> > ok 15 - daemon running after t/lei-sigpipe.t:44
> > ok 16 - lei daemon-kill (daemon-kill after t/lei-sigpipe.t:44)
> > ok 17 - t/lei-sigpipe.t:44 daemon stopped
> > ok 18 - t/lei-sigpipe.t:44 daemon XDG_RUNTIME_DIR/lei/errors.log empty
> > 1..18
> > ok
> > All tests successful.
> > Files=1, Tests=18,  4 wallclock secs ( 0.06 usr  0.06 sys +  1.23 cusr  1.48 csys =  2.83 CPU)
> > Result: PASS
> 
> Even more strange, Dominique was able to reproduce
> the hang this morning, but no longer tonight..

Yes, I don't get it, it hanged once but no longer hangs, so as much as
I'd have liked to investigate I'm a bit stuck.

With this, I can confirm running with inline-c also makes the tests that
failed with the btrfs chattr call also pass.
So all that's left is fix the proc mounts parsing there :)

-- 
Dominique