From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS24940 159.69.0.0/16 X-Spam-Status: No, score=-2.4 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS, URIBL_BLACK shortcircuit=no autolearn=no autolearn_force=no version=3.4.2 Received: from todd.t-8ch.de (todd.t-8ch.de [159.69.126.157]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 003451F9F3 for ; Wed, 27 Oct 2021 21:25:01 +0000 (UTC) Date: Wed, 27 Oct 2021 23:24:55 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=t-8ch.de; s=mail; t=1635369897; bh=LfPE0o9YpslsydvDycAjb193LC00KfQ0aJeNxARQbIY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=l4iiQGStdXMwN7Bm2Y7BsTG+krtn2p8PFVoSkulWHm6d6r76l3vFFYhgMtsQJb+zo bz7z9NXcDUDtZlAtH+wqSyAVP30fPuPIYfLXY3s7CMqyZYigZdikBRRD68xdAyucwi 5/gUfhHyI5S4YhGfmAnreYSdt+1zTe2953VAqAac= From: Thomas =?utf-8?Q?Wei=C3=9Fschuh?= To: Eric Wong Cc: meta@public-inbox.org Subject: Re: lei-q-remote-import failures [was: [PATCH] t/index-git-times: support non-master default branch] Message-ID: <52653367-9aac-45a7-9d3d-56dacfe36905@t-8ch.de> References: <20211025222453.252181-1-thomas@t-8ch.de> <20211025225822.GA28875@dcvr> <20211026052815.GA13265@dcvr> <20211027211515.M84746@dcvr> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="PzlkB6R9STvTjunI" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20211027211515.M84746@dcvr> Jabber-ID: thomas@t-8ch.de X-Accept: text/plain, text/html;q=0.2, text/*;q=0.1 X-Accept-Language: en-us, en;q=0.8, de-de;q=0.7, de;q=0.6 List-Id: --PzlkB6R9STvTjunI Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit On 2021-10-27 21:15+0000, Eric Wong wrote: > Thomas Weißschuh wrote: > > On 2021-10-26 05:28+0000, Eric Wong wrote: > > > Thomas Weißschuh wrote: > > > > The only failures I still see are in t/lei-q-remote-import.t which is unrelated > > > > to the git branch and some more that fail only during parallel test execution. > > > > > > > > In t/lei-q-remote-import.t it seems the search results are not memoized to the > > > > local store. (See attachment) > > > > > > Thanks, I'll check out lei-q-remote-import.t separately. > > > That could be related to SMP or slow/fast storage. > > > Any details you can share about CPU core count, speeds > > > or storage speeds, or CPU scheduler, CONFIG_HZ? > > > > CPU: > > > > Model name: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz > > CPU family: 6 > > Model: 140 > > Thread(s) per core: 2 > > Core(s) per socket: 4 > > Socket(s): 1 > > Stepping: 1 > > CPU max MHz: 4700,0000 > > CPU min MHz: 400,0000 > > > > Storage: Samsung EVO 970 (with plenty of memory cache) > > CONFIG_HZ: 300 > > I'm puzzled by this failure; I'm not sure if I've > ever seen it. Does it fail when run standalone? > (prove -bvw t/lei-q-remote-import.t) It also does. The protocol is attached. > Maybe running under TEST_LEI_ERR_LOUD=1 or checking > syslog can reveal something. > > Adding a sleep shouldn't be necessary, but maybe something > else is broken I'm not seeing...: It didn't help, neither did larger sleeps. > diff --git a/t/lei-q-remote-import.t b/t/lei-q-remote-import.t > index 92d8c9b6058c..a2d643cd06c7 100644 > --- a/t/lei-q-remote-import.t > +++ b/t/lei-q-remote-import.t > @@ -35,6 +35,7 @@ test_lei({ tmpdir => $tmpdir }, sub { > lei_ok(@cmd, '-I', $url); > is_deeply($slurp_emls->($o), $exp1, 'got results after remote search'); > unlink $o or BAIL_OUT $!; > + sleep 1; > lei_ok(@cmd); > ok(-f $o && -s _, 'output exists after import but is not empty') or > diag $lei_err; I'll see if I can debug it in the coming days. --PzlkB6R9STvTjunI Content-Type: text/troff; charset=utf-8 Content-Disposition: attachment; filename="output-lei-q-remote-import.t" $ git checkout origin/master Note: switching to 'origin/master'. HEAD is now at ded9dad2 test_common: key test inboxes to init.defaultBranch $ git clean -fdx t/data-gen/ ... $ TEST_LEI_ERR_LOUD=1 prove -bvw t/lei-q-remote-import.t # # /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (0 matches) ok 1 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:qp@example.com ok 2 - output exists but is empty # lei_err=# /usr/bin/curl -Sf -s -d '' http://[::1]:46299/t2/?x=m&q=m%3Aqp%40example.com # # /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # http://[::1]:46299/t2/ 1/1 # # 1 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (1 matches) ok 3 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:qp@example.com -I http://[::1]:46299/t2/ ok 4 - got results after remote search ok 5 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:qp@example.com not ok 6 - output exists after import but is not empty # lei_err=# /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (0 matches) # Failed test 'output exists after import but is not empty' # at t/lei-q-remote-import.t line 39. # # /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (0 matches) not ok 7 - got results w/o remote search # Failed test 'got results w/o remote search' # at t/lei-q-remote-import.t line 41. # Structures begin differing at: # $got->[0] = Does not exist # $expected->[0] = PublicInbox::Eml=HASH(0x565282aeff78) ok 8 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:199707281508.AAA24167@hoyogw.example -I http://[::1]:46299/t2/ --no-import-remote # lei_err=# /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # /usr/bin/curl -Sf -s -d '' http://[::1]:46299/t2/?x=m&q=m%3A199707281508.AAA24167%40hoyogw.example # # http://[::1]:46299/t2/ 1/1 # # 1 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (1 matches) ok 9 - got another after remote search ok 10 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:199707281508.AAA24167@hoyogw.example ok 11 - --no-import-remote did not memoize # lei_err=# /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (0 matches) ok 12 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:qp@example.com --lock=none not ok 13 - --lock=none respected # lei_err=# /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (0 matches) # Failed test '--lock=none respected' # at t/lei-q-remote-import.t line 55. # # /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (0 matches) # lei_err=dotlock timeout /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd.lock ok 14 - dotlock fails ok 15 - timeout noted ok 16 - nothing output on lock failure ok 17 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:qp@example.com --lock=dotlock,timeout=0.000001 (succeeds after lock removal) # lei_err=# /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (0 matches) ok 18 - lei add-external -q /home/t-8ch/Projekte/public-inbox.org/t/data-gen/lei-q-remote-import.local-external-master ok 19 - lei q -q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd --only http://[::1]:46299/t2/ m:testmessage@example.com ok 20 - no warnings or errors ok 21 - got result from remote external ok 22 - got expected result ok 23 - lei q --no-external -o mboxrd:/dev/stdout m:testmessage@example.com ok 24 - message not imported when in local external ok 25 - lei q -o mboxrd:/tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd m:testmessage@example.com # lei_err=# /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/.local/share/lei/store 0/0 # # /home/t-8ch/Projekte/public-inbox.org/t/data-gen/lei-q-remote-import.local-external-master 1/1 # # 1 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/o.mboxrd (1 matches) ok 26 - got expected result after clobber ok 27 - lei q -o mboxrd:/dev/stdout m:never-before-seen@example.com ok 28 - --import-before imported totally unseen message ok 29 - lei q --save z:0.. -o /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/md --only http://[::1]:46299/t2/ # lei_err=# /usr/bin/curl -Sf -s -d '' http://[::1]:46299/t2/?x=m&q=z%3A0.. # # http://[::1]:46299/t2/ 15/15 # # 15 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/md/ (15 matches) ok 30 - lei up /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/md ok 31 - lei up remote dedupe works on maildir # lei_err=# http://[::1]:46299/t2/ limiting to 2021-10-25 23:18 +0200 and newer # # /usr/bin/curl -Sf -s -d '' http://[::1]:46299/t2/?x=m&q=(z%3A0..)+AND+dt%3A20211025211849.. # # http://[::1]:46299/t2/ 11/11 # # 0 written to /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/md/ (11 matches) ok 32 - lei edit-search /tmp/pi-lei-q-remote-import-5668-gmA0/lei-daemon/md ok 33 - lastresult set ok 34 - lei daemon-pid (daemon-pid after t/lei-q-remote-import.t:106) ok 35 - daemon running after t/lei-q-remote-import.t:106 ok 36 - lei daemon-kill (daemon-kill after t/lei-q-remote-import.t:106) ok 37 - t/lei-q-remote-import.t:106 daemon stopped ok 38 - t/lei-q-remote-import.t:106 daemon XDG_RUNTIME_DIR/lei/errors.log empty 1..38 # Looks like you failed 3 tests of 38. Dubious, test returned 3 (wstat 768, 0x300) Failed 3/38 subtests Test Summary Report ------------------- t/lei-q-remote-import.t (Wstat: 768 Tests: 38 Failed: 3) Failed tests: 6-7, 13 Non-zero exit status: 3 Files=1, Tests=38, 1 wallclock secs ( 0.01 usr 0.01 sys + 0.32 cusr 0.08 csys = 0.42 CPU) Result: FAIL --PzlkB6R9STvTjunI--