user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
* 'make test' failures while packaging for openSUSE
@ 2021-11-19 12:42 Jörg Rödel
  2021-11-19 18:54 ` Eric Wong
  0 siblings, 1 reply; 9+ messages in thread
From: Jörg Rödel @ 2021-11-19 12:42 UTC (permalink / raw)
  To: meta

Hi,

first of all thanks a lot to all the developers for writing such an
awesome tool. I am currently setting up lei to watch various Linux
kernel development lists and so far it works like a charm!

To make things more smooth for others I am currently packaging
public-inbox for openSUSE, and I am encountering some failures while
running 'make test'. Here is the error I am seeing in the build log:

	t/lei-lcat.t ................. ok
	
	#   Failed test 'clone + index v1 synced ->created_at'
	#   at t/lei-mirror.t line 175.
	#          got: '1637324685'
	#     expected: undef
	
	#   Failed test 'clone + index v1 synced ->created_at'
	#   at t/lei-mirror.t line 178.
	#          got: '1637324685'
	#     expected: undef
	# Looks like you failed 2 tests of 37.
	t/lei-mirror.t ............... 
	Dubious, test returned 2 (wstat 512, 0x200)
	Failed 2/37 subtests 
		(less 2 skipped subtests: 33 okay)

The full 'make test' excerpt from the build log is also attached. This
happens when building from latest HEAD as well as from tag v1.7.0.

I am not sure why this test fails, maybe it is some problem in one of
the dependencies. Any help here would be great!

Thanks,

	Joerg

[    5s] + make test
[    5s] fatal: not a git repository (or any of the parent directories): .git
[    5s] VERSION=1.7.0 "/usr/bin/perl" -w ./version-gen.perl
[    5s] PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
[    5s] t/address.t .................. ok
[    5s] t/admin.t .................... ok
[    6s] t/altid.t .................... ok
[    6s] t/altid_v2.t ................. ok
[    6s] t/cgi.t ...................... ok
[    6s] t/cmd_ipc.t .................. ok
[    7s] t/config.t ................... ok
[    7s] t/config_limiter.t ........... ok
[    7s] t/content_hash.t ............. ok
[    8s] t/convert-compact.t .......... ok
[    8s] t/dir_idle.t ................. ok
[    8s] t/ds-kqxs.t .................. skipped: DSKQXS is only for *BSD systems
[    8s] t/ds-leak.t .................. ok
[    8s] t/ds-poll.t .................. ok
[   10s] t/edit.t ..................... ok
[   10s] t/emergency.t ................ ok
[   10s] t/eml.t ...................... ok
[   10s] t/eml_content_disposition.t .. ok
[   10s] t/eml_content_type.t ......... ok
[   10s] t/epoll.t .................... ok
[   11s] t/extindex-psgi.t ............ ok
[   15s] t/extsearch.t ................ ok
[   15s] t/fake_inotify.t ............. ok
[   16s] t/feed.t ..................... ok
[   16s] t/filter_base.t .............. ok
[   16s] t/filter_mirror.t ............ ok
[   16s] t/filter_rubylang.t .......... ok
[   16s] t/filter_subjecttag.t ........ ok
[   16s] t/filter_vger.t .............. ok
[   16s] # require PublicInbox::Gcf2: open(/home/abuild/.cache/public-inbox/inline-c/.public-inbox.lock): No such file or directory at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 20.
[   16s] # BEGIN failed--compilation aborted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 78.
[   16s] # Compilation failed in require at (eval 12) line 1.
[   16s] t/gcf2.t ..................... skipped: PublicInbox::Gcf2 missing for t/gcf2.t
[   16s] # require PublicInbox::Gcf2: open(/home/abuild/.cache/public-inbox/inline-c/.public-inbox.lock): No such file or directory at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 20.
[   16s] # BEGIN failed--compilation aborted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 78.
[   16s] # Compilation failed in require at (eval 13) line 1.
[   16s] t/gcf2_client.t .............. skipped: PublicInbox::Gcf2 missing for t/gcf2_client.t
[   17s] t/git.t ...................... ok
[   17s] t/gzip_filter.t .............. ok
[   17s] t/hl_mod.t ................... skipped: highlight missing for t/hl_mod.t
[   19s] t/httpd-corner.t ............. ok
[   19s] t/httpd-https.t .............. ok
[   20s] t/httpd-unix.t ............... ok
[   21s] t/httpd.t .................... ok
[   21s] t/hval.t ..................... ok
[   21s] t/idx_stack.t ................ ok
[   22s] t/imap.t ..................... ok
[   22s] t/imap_searchqp.t ............ ok
[   22s] t/imap_tracker.t ............. ok
[   23s] t/imapd-tls.t ................ ok
[   24s] # waiting for initial fetch...
[   24s] # inbox unlocked on initial fetch, waiting for IDLE
[   25s] # waiting for IMAP IDLE wakeup
[   25s] # inbox unlocked on IDLE wakeup
[   25s] # waiting for -watch reload + initial fetch
[   25s] # waiting for PollInterval wakeup
[   25s] # inbox unlocked (poll)
[   25s] t/imapd.t .................... ok
[   25s] t/import.t ................... ok
[   26s] t/inbox.t .................... ok
[   26s] t/inbox_idle.t ............... ok
[   27s] t/index-git-times.t .......... ok
[   29s] t/indexlevels-mirror-v1.t .... ok
[   32s] t/indexlevels-mirror.t ....... ok
[   34s] # err=no email in From:  or Sender: 
[   34s] # no name in From:  or Sender: 
[   35s] # err=no email in From:  or Sender: 
[   35s] # no name in From:  or Sender: 
[   35s] t/init.t ..................... ok
[   35s] t/ipc.t ...................... ok
[   35s] t/kqnotify.t ................. skipped: KQNotify is only for *BSD systems
[   35s] t/lei-auto-watch.t ........... ok
[   35s] t/lei-convert.t .............. ok
[   35s] t/lei-daemon.t ............... ok
[   36s] t/lei-export-kw.t ............ ok
[   36s] t/lei-externals.t ............ ok
[   36s] t/lei-import-http.t .......... ok
[   36s] t/lei-import-imap.t .......... ok
[   37s] t/lei-import-maildir.t ....... ok
[   37s] t/lei-import-nntp.t .......... ok
[   37s] t/lei-import.t ............... ok
[   37s] t/lei-index.t ................ ok
[   37s] t/lei-inspect.t .............. ok
[   38s] t/lei-lcat.t ................. ok
[   38s] 
[   38s] #   Failed test 'clone + index v1 synced ->created_at'
[   38s] #   at t/lei-mirror.t line 175.
[   38s] #          got: '1637324685'
[   38s] #     expected: undef
[   38s] 
[   38s] #   Failed test 'clone + index v1 synced ->created_at'
[   38s] #   at t/lei-mirror.t line 178.
[   38s] #          got: '1637324685'
[   38s] #     expected: undef
[   38s] # Looks like you failed 2 tests of 37.
[   38s] t/lei-mirror.t ............... 
[   38s] Dubious, test returned 2 (wstat 512, 0x200)
[   38s] Failed 2/37 subtests 
[   38s] 	(less 2 skipped subtests: 33 okay)
[   39s] t/lei-p2q.t .................. ok
[   39s] t/lei-q-kw.t ................. ok
[   39s] t/lei-q-remote-import.t ...... ok
[   39s] t/lei-q-save.t ............... ok
[   39s] t/lei-q-thread.t ............. ok
[   39s] t/lei-refresh-mail-sync.t .... ok
[   40s] t/lei-sigpipe.t .............. ok
[   40s] t/lei-tag.t .................. ok
[   40s] t/lei-up.t ................... ok
[   40s] t/lei-watch.t ................ ok
[   40s] t/lei.t ...................... ok
[   40s] t/lei_dedupe.t ............... ok
[   41s] t/lei_external.t ............. ok
[   41s] t/lei_lcat.t ................. ok
[   41s] t/lei_mail_sync.t ............ ok
[   41s] t/lei_overview.t ............. ok
[   41s] t/lei_saved_search.t ......... ok
[   42s] t/lei_store.t ................ ok
[   43s] t/lei_to_mail.t .............. ok
[   44s] t/lei_xsearch.t .............. ok
[   44s] t/linkify.t .................. ok
[   45s] t/mbox_lock.t ................ ok
[   45s] t/mbox_reader.t .............. ok
[   45s] t/mda.t ...................... ok
[   46s] t/mda_filter_rubylang.t ...... ok
[   46s] t/mdir_reader.t .............. ok
[   46s] t/mid.t ...................... ok
[   46s] t/mime.t ..................... ok
[   47s] t/miscsearch.t ............... ok
[   47s] t/msg_iter.t ................. ok
[   47s] t/msgmap.t ................... ok
[   47s] t/msgtime.t .................. ok
[   48s] t/multi-mid.t ................ ok
[   49s] t/net_reader-imap.t .......... ok
[   49s] t/nntp.t ..................... ok
[   49s] t/nntpd-tls.t ................ ok
[   50s] # waiting for initial fetch...
[   50s] # inbox unlocked on initial fetch
[   51s] t/nntpd-v2.t ................. ok
[   52s] # waiting for initial fetch...
[   52s] # inbox unlocked on initial fetch
[   52s] t/nntpd.t .................... ok
[   52s] t/nodatacow.t ................ ok
[   53s] t/nulsubject.t ............... ok
[   53s] t/on_destroy.t ............... ok
[   53s] t/over.t ..................... ok
[   53s] t/plack.t .................... ok
[   54s] t/precheck.t ................. ok
[   54s] t/psgi_attach.t .............. ok
[   54s] t/psgi_bad_mids.t ............ ok
[   54s] t/psgi_mount.t ............... ok
[   55s] t/psgi_multipart_not.t ....... ok
[   55s] t/psgi_scan_all.t ............ ok
[   55s] t/psgi_search.t .............. ok
[   56s] t/psgi_text.t ................ ok
[   56s] t/psgi_v2.t .................. ok
[   57s] t/purge.t .................... ok
[   59s] t/qspawn.t ................... ok
[   59s] Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master/public-inbox/xapian15/over.sqlite3 at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 194.
[   59s] /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master index failed: Modification of a read-only value attempted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 900, <$r> line 1.
[   59s] 
[   60s] t/reindex-time-range.t ....... ok
[   60s] t/rename_noreplace.t ......... ok
[   60s] # v2 basic replace
[   60s] # v2 basic replace
[   60s] # v2 basic replace
[   61s] # v2 basic replace
[   61s] # v2 medium replace
[   61s] # v2 medium replace
[   61s] # v2 medium replace
[   61s] # v2 medium replace
[   62s] t/replace.t .................. ok
[   62s] t/reply.t .................... ok
[   62s] t/search-thr-index.t ......... ok
[   63s] t/search.t ................... ok
[   64s] t/shared_kv.t ................ ok
[   64s] t/sigfd.t .................... ok
[   64s] t/solver_git.t ............... skipped: t/solver_git.t must be run from a git working tree
[   64s] t/spamcheck_spamc.t .......... ok
[   64s] # waiting for child to reap grandchild...
[   64s] t/spawn.t .................... ok
[   64s] t/thread-cycle.t ............. ok
[   64s] t/thread-index-gap.t ......... ok
[   65s] t/time.t ..................... ok
[   65s] t/uri_imap.t ................. ok
[   65s] t/uri_nntps.t ................ ok
[   65s] t/v1-add-remove-add.t ........ ok
[   66s] t/v1reindex.t ................ ok
[   66s] t/v2-add-remove-add.t ........ ok
[   67s] t/v2dupindex.t ............... ok
[   67s] t/v2index-late-dupe.t ........ ok
[   68s] t/v2mda.t .................... ok
[   71s] t/v2mirror.t ................. ok
[   72s] # messing up all threads with tid=10
[   73s] # messing up all threads with tid=13
[   73s] # messing up all threads with tid=29
[   73s] t/v2reindex.t ................ ok
[   74s] t/v2writable.t ............... ok
[   74s] t/view.t ..................... ok
[   76s] t/watch_filter_rubylang.t .... ok
[   77s] t/watch_imap.t ............... ok
[   77s] # waiting for -watch to import new message
[   77s] # message delivered to `test'
[   77s] t/watch_maildir.t ............ ok
[   78s] t/watch_maildir_v2.t ......... ok
[   79s] t/watch_multiple_headers.t ... ok
[   79s] t/www_altid.t ................ ok
[   79s] t/www_listing.t .............. ok
[   80s] t/www_static.t ............... ok
[   80s] t/xcpdb-reshard.t ............ ok
[   80s] 
[   80s] Test Summary Report
[   80s] -------------------
[   80s] t/lei-mirror.t             (Wstat: 512 Tests: 37 Failed: 2)
[   80s]   Failed tests:  30-31
[   80s]   Non-zero exit status: 2
[   80s] Files=157, Tests=5322, 75 wallclock secs ( 0.66 usr  0.17 sys + 39.13 cusr 18.05 csys = 58.01 CPU)
[   80s] Result: FAIL
[   80s] Failed 1/157 test programs. 2/5322 subtests failed.
[   80s] make: *** [Makefile:1203: test_dynamic] Error 255
[   80s] error: Bad exit status from /var/tmp/rpm-tmp.daUPQe (%check)

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: 'make test' failures while packaging for openSUSE
  2021-11-19 12:42 'make test' failures while packaging for openSUSE Jörg Rödel
@ 2021-11-19 18:54 ` Eric Wong
  2021-11-19 21:46   ` Jörg Rödel
  0 siblings, 1 reply; 9+ messages in thread
From: Eric Wong @ 2021-11-19 18:54 UTC (permalink / raw)
  To: Jörg Rödel; +Cc: meta

Jörg Rödel <joro@8bytes.org> wrote:
> Hi,
> 
> first of all thanks a lot to all the developers for writing such an
> awesome tool. I am currently setting up lei to watch various Linux
> kernel development lists and so far it works like a charm!

Great to know :>  There's still a lot of things to improve,
hopefully next year...

> To make things more smooth for others I am currently packaging
> public-inbox for openSUSE, and I am encountering some failures while
> running 'make test'. Here is the error I am seeing in the build log:
> 
> 	t/lei-lcat.t ................. ok
> 	
> 	#   Failed test 'clone + index v1 synced ->created_at'
> 	#   at t/lei-mirror.t line 175.
> 	#          got: '1637324685'
> 	#     expected: undef
> 	
> 	#   Failed test 'clone + index v1 synced ->created_at'
> 	#   at t/lei-mirror.t line 178.
> 	#          got: '1637324685'
> 	#     expected: undef
> 	# Looks like you failed 2 tests of 37.
> 	t/lei-mirror.t ............... 
> 	Dubious, test returned 2 (wstat 512, 0x200)
> 	Failed 2/37 subtests 
> 		(less 2 skipped subtests: 33 okay)

I don't think I've ever seen this failure before.
It looks like the %created hash didn't get populated by
the first test_lei() block at all.

> The full 'make test' excerpt from the build log is also attached. This
> happens when building from latest HEAD as well as from tag v1.7.0.
> 
> I am not sure why this test fails, maybe it is some problem in one of
> the dependencies. Any help here would be great!

Can you provide a more verbose dump of just t/lei-mirror.t?

   make && prove -bvw t/lei-mirror.t

No rush, though, probably going to be busy the next few days.
This may be related, but lei will fail completely unless
Inline::C or Socket::MsgHdr are available:

> [   16s] # require PublicInbox::Gcf2: open(/home/abuild/.cache/public-inbox/inline-c/.public-inbox.lock): No such file or directory at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 20.
> [   16s] # BEGIN failed--compilation aborted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 78.
> [   16s] # Compilation failed in require at (eval 12) line 1.
> [   16s] t/gcf2.t ..................... skipped: PublicInbox::Gcf2 missing for t/gcf2.t
> [   16s] # require PublicInbox::Gcf2: open(/home/abuild/.cache/public-inbox/inline-c/.public-inbox.lock): No such file or directory at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 20.
> [   16s] # BEGIN failed--compilation aborted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Gcf2.pm line 78.
> [   16s] # Compilation failed in require at (eval 13) line 1.

Also, I should clean that up for testing :x

> [   59s] Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master/public-inbox/xapian15/over.sqlite3 at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 194.
> [   59s] /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master index failed: Modification of a read-only value attempted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 900, <$r> line 1.
> [   59s] 
> [   60s] t/reindex-time-range.t ....... ok

That message is still perplexing to me; and I can't reproduce it
consistently...

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: 'make test' failures while packaging for openSUSE
  2021-11-19 18:54 ` Eric Wong
@ 2021-11-19 21:46   ` Jörg Rödel
  2021-11-22  6:55     ` [PATCH] searchidx: add some extra diagnostics for odd message Eric Wong
  0 siblings, 1 reply; 9+ messages in thread
From: Jörg Rödel @ 2021-11-19 21:46 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

Hi Eric,

thanks a lot for your help!

On Fri, Nov 19, 2021 at 06:54:11PM +0000, Eric Wong wrote:
> Can you provide a more verbose dump of just t/lei-mirror.t?
> 
>    make && prove -bvw t/lei-mirror.t
> 
> No rush, though, probably going to be busy the next few days.
> This may be related, but lei will fail completely unless
> Inline::C or Socket::MsgHdr are available:

Turned out this was the issue. I ran the prove command and found in the
output that either Inline::C or Socket::MsgHdr are missing. Added
Sochet::MsgHdr and now the tests complete without failures.

> > [   59s] Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master/public-inbox/xapian15/over.sqlite3 at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 194.
> > [   59s] /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master index failed: Modification of a read-only value attempted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 900, <$r> line 1.
> > [   59s] 
> > [   60s] t/reindex-time-range.t ....... ok
> 
> That message is still perplexing to me; and I can't reproduce it
> consistently...

I havn't checked whether I see it in all of the 'make test' runs, but at
least it appears pretty frequently here. Let me know if I can help with
tracking that down.

Regards,

	Joerg

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH] searchidx: add some extra diagnostics for odd message
  2021-11-19 21:46   ` Jörg Rödel
@ 2021-11-22  6:55     ` Eric Wong
  2021-11-22  7:42       ` [PATCH] t/lei-mirror: skip lei comparisons if lei missing Eric Wong
  2021-11-22 13:20       ` [PATCH] searchidx: add some extra diagnostics for odd message Jörg Rödel
  0 siblings, 2 replies; 9+ messages in thread
From: Eric Wong @ 2021-11-22  6:55 UTC (permalink / raw)
  To: Jörg Rödel; +Cc: meta

Jörg Rödel <joro@8bytes.org> wrote:
> Hi Eric,
> 
> thanks a lot for your help!

No problem, and sorry for the failing test.

> > On Fri, Nov 19, 2021 at 06:54:11PM +0000, Eric Wong wrote:
> > Can you provide a more verbose dump of just t/lei-mirror.t?
> > 
> >    make && prove -bvw t/lei-mirror.t
> > 
> > No rush, though, probably going to be busy the next few days.
> > This may be related, but lei will fail completely unless
> > Inline::C or Socket::MsgHdr are available:
> 
> Turned out this was the issue. I ran the prove command and found in the
> output that either Inline::C or Socket::MsgHdr are missing. Added
> Sochet::MsgHdr and now the tests complete without failures.

Will work on a patch to ensure lei ran successfully.

> > > [   59s] Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master/public-inbox/xapian15/over.sqlite3 at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 194.
> > > [   59s] /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master index failed: Modification of a read-only value attempted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 900, <$r> line 1.
> > > [   59s] 
> > > [   60s] t/reindex-time-range.t ....... ok
> > 
> > That message is still perplexing to me; and I can't reproduce it
> > consistently...
> 
> I havn't checked whether I see it in all of the 'make test' runs, but at
> least it appears pretty frequently here. Let me know if I can help with
> tracking that down.

Sure thing, can you see if this shows anything useful?
------------8<-----------
Subject: [PATCH] searchidx: add some extra diagnostics for odd message

Perhaps these can help us diagnose occasional
"Modification of a read-only value attempted at ..."
errors in t/reindex-time-range.t (which do not cause test
failures).

Link: https://public-inbox.org/meta/YZgbMSUTSayOijaz@8bytes.org/
---
 lib/PublicInbox/SearchIdx.pm | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
index 6e2e614c..030a5007 100644
--- a/lib/PublicInbox/SearchIdx.pm
+++ b/lib/PublicInbox/SearchIdx.pm
@@ -897,6 +897,7 @@ sub log2stack ($$$) {
 	}
 	my $fh = $git->popen(@cmd, $range);
 	my ($at, $ct, $stk, $cmt);
+eval { # FIXME: diagnosing "Modification of a read-only value attempted .."
 	while (<$fh>) {
 		return if $sync->{quit};
 		if (/\A([0-9]+)-([0-9]+)-($OID)$/o) {
@@ -921,6 +922,8 @@ sub log2stack ($$$) {
 			$stk->push_rec('m', $at, $ct, $oid, $cmt);
 		}
 	}
+}; # /end diagnosing "Modification of a read-only value attempted .."
+Carp::croak($@, Carp::longmess()) if $@;
 	close $fh or die "git log failed: \$?=$?";
 	$stk //= PublicInbox::IdxStack->new;
 	$stk->read_prepare;

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH] t/lei-mirror: skip lei comparisons if lei missing
  2021-11-22  6:55     ` [PATCH] searchidx: add some extra diagnostics for odd message Eric Wong
@ 2021-11-22  7:42       ` Eric Wong
  2021-11-22 13:47         ` Jörg Rödel
  2021-11-22 13:20       ` [PATCH] searchidx: add some extra diagnostics for odd message Jörg Rödel
  1 sibling, 1 reply; 9+ messages in thread
From: Eric Wong @ 2021-11-22  7:42 UTC (permalink / raw)
  To: Jörg Rödel; +Cc: meta

Eric Wong <e@80x24.org> wrote:
> Will work on a patch to ensure lei ran successfully.

---------8<--------
Subject: [PATCH] t/lei-mirror: skip lei comparisons if lei missing

We can't compare created_at times with lei if lei tests are
skipped due to Inline::C or Socket::MsgHdr unavailability.

Reported-by: Jörg Rödel <joro@8bytes.org>
Link: https://public-inbox.org/meta/YZebmAxlFJy4lqAw@8bytes.org/
---
 t/lei-mirror.t | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/t/lei-mirror.t b/t/lei-mirror.t
index 646ff2b19f98..d6fa6db5e3cb 100644
--- a/t/lei-mirror.t
+++ b/t/lei-mirror.t
@@ -170,12 +170,17 @@ SKIP: {
 	local $ENV{HOME} = $tmpdir;
 	ok(run_script([qw(-index -Lbasic), "$d/t1"]), 'index v1');
 	ok(run_script([qw(-index -Lbasic), "$d/t2"]), 'index v2');
-	my $f = "$d/t1/public-inbox/msgmap.sqlite3";
-	my $ca = PublicInbox::Msgmap->new_file($f)->created_at;
-	is($ca, $created{v1}, 'clone + index v1 synced ->created_at');
-	$f = "$d/t2/msgmap.sqlite3";
-	$ca = PublicInbox::Msgmap->new_file($f)->created_at;
-	is($ca, $created{v2}, 'clone + index v1 synced ->created_at');
+
+	SKIP: {
+		skip "lei didn't run", 2 if join(sort keys %created) ne 'v1v2';
+		my $f = "$d/t1/public-inbox/msgmap.sqlite3";
+		my $ca = PublicInbox::Msgmap->new_file($f)->created_at;
+		is($ca, $created{v1}, 'clone + index v1 synced ->created_at');
+
+		$f = "$d/t2/msgmap.sqlite3";
+		$ca = PublicInbox::Msgmap->new_file($f)->created_at;
+		is($ca, $created{v2}, 'clone + index v2 synced ->created_at');
+	}
 	test_lei(sub {
 		lei_ok qw(inspect num:1 --dir), "$d/t1";
 		ok(ref(json_utf8->decode($lei_out)), 'inspect num: on v1');

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] searchidx: add some extra diagnostics for odd message
  2021-11-22  6:55     ` [PATCH] searchidx: add some extra diagnostics for odd message Eric Wong
  2021-11-22  7:42       ` [PATCH] t/lei-mirror: skip lei comparisons if lei missing Eric Wong
@ 2021-11-22 13:20       ` Jörg Rödel
  2021-11-22 18:23         ` [PATCH] searchidx: avoid modification of read-only `$_' Eric Wong
  1 sibling, 1 reply; 9+ messages in thread
From: Jörg Rödel @ 2021-11-22 13:20 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

Hi Eric,

On Mon, Nov 22, 2021 at 06:55:45AM +0000, Eric Wong wrote:
> Sure thing, can you see if this shows anything useful?

This gives me:

[   92s] t/qspawn.t ................... ok
[   92s] Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master/public-inbox/xapian15/over.sqlite3 at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 194.
[   92s] /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master index failed: Modification of a read-only value attempted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 900, <$r> line 1.
[   92s]  at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 942, <$r> line 1.
[   92s] 	PublicInbox::SearchIdx::prepare_stack(HASH(0x564bfd2f7678), "refs/heads/master") called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 1038
[   92s] 	PublicInbox::SearchIdx::_index_sync(PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/InboxWritable.pm line 224
[   92s] 	eval {...} called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/InboxWritable.pm line 224
[   92s] 	PublicInbox::InboxWritable::with_umask(PublicInbox::InboxWritable=HASH(0x564bfcccbb10), CODE(0x564bfd49d0f8), PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 760
[   92s] 	PublicInbox::SearchIdx::with_umask(PublicInbox::SearchIdx=HASH(0x564bfcf5be78), CODE(0x564bfd49d0f8), PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 767
[   92s] 	PublicInbox::SearchIdx::index_sync(PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 194
[   92s] 	eval {...} called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 192
[   92s] 	PublicInbox::Import::_update_git_info(PublicInbox::Import=HASH(0x564bfccbd5f8), 1) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 494
[   92s] 	eval {...} called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 494
[   92s] 	PublicInbox::Import::done(PublicInbox::Import=HASH(0x564bfccbd5f8)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/TestCommon.pm line 707
[   92s] 	PublicInbox::TestCommon::create_inbox("v1", "version", 1, "indexlevel", "basic", "tmpdir", "/tmp/pi-reindex-time-range-22646-lChw/v1") called at t/reindex-time-range.t line 18
[   92s]  at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/InboxWritable.pm line 224.
[   92s] 

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] t/lei-mirror: skip lei comparisons if lei missing
  2021-11-22  7:42       ` [PATCH] t/lei-mirror: skip lei comparisons if lei missing Eric Wong
@ 2021-11-22 13:47         ` Jörg Rödel
  2021-11-22 17:24           ` Eric Wong
  0 siblings, 1 reply; 9+ messages in thread
From: Jörg Rödel @ 2021-11-22 13:47 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

On Mon, Nov 22, 2021 at 07:42:41AM +0000, Eric Wong wrote:
>  t/lei-mirror.t | 17 +++++++++++------
>  1 file changed, 11 insertions(+), 6 deletions(-)

This seems to cause warnings:

[   56s] t/lei-lcat.t ................. ok
[   57s] Useless use of sort in scalar context at t/lei-mirror.t line 175.
[   59s] Use of uninitialized value in join or string at t/lei-mirror.t line 175.
[   59s] t/lei-mirror.t ............... ok

Regards,

	Joerg

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] t/lei-mirror: skip lei comparisons if lei missing
  2021-11-22 13:47         ` Jörg Rödel
@ 2021-11-22 17:24           ` Eric Wong
  0 siblings, 0 replies; 9+ messages in thread
From: Eric Wong @ 2021-11-22 17:24 UTC (permalink / raw)
  To: Jörg Rödel; +Cc: meta

Jörg Rödel <joro@8bytes.org> wrote:
> This seems to cause warnings:
> 
> [   56s] t/lei-lcat.t ................. ok
> [   57s] Useless use of sort in scalar context at t/lei-mirror.t line 175.
> [   59s] Use of uninitialized value in join or string at t/lei-mirror.t line 175.
> [   59s] t/lei-mirror.t ............... ok

Oops :x  I'll squash this in before pushing:

diff --git a/t/lei-mirror.t b/t/lei-mirror.t
index d6fa6db5e3cb..32a5b0390714 100644
--- a/t/lei-mirror.t
+++ b/t/lei-mirror.t
@@ -172,7 +172,8 @@ SKIP: {
 	ok(run_script([qw(-index -Lbasic), "$d/t2"]), 'index v2');
 
 	SKIP: {
-		skip "lei didn't run", 2 if join(sort keys %created) ne 'v1v2';
+		join('', sort(keys %created)) eq 'v1v2' or
+			skip "lei didn't run", 2;
 		my $f = "$d/t1/public-inbox/msgmap.sqlite3";
 		my $ca = PublicInbox::Msgmap->new_file($f)->created_at;
 		is($ca, $created{v1}, 'clone + index v1 synced ->created_at');

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH] searchidx: avoid modification of read-only `$_'
  2021-11-22 13:20       ` [PATCH] searchidx: add some extra diagnostics for odd message Jörg Rödel
@ 2021-11-22 18:23         ` Eric Wong
  0 siblings, 0 replies; 9+ messages in thread
From: Eric Wong @ 2021-11-22 18:23 UTC (permalink / raw)
  To: Jörg Rödel; +Cc: meta

Jörg Rödel <joro@8bytes.org> wrote:
> [   92s] /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/t/data-gen/reindex-time-range.v1-master index failed: Modification of a read-only value attempted at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 900, <$r> line 1.
> [   92s]  at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 942, <$r> line 1.
> [   92s] 	PublicInbox::SearchIdx::prepare_stack(HASH(0x564bfd2f7678), "refs/heads/master") called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 1038
> [   92s] 	PublicInbox::SearchIdx::_index_sync(PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/InboxWritable.pm line 224
> [   92s] 	eval {...} called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/InboxWritable.pm line 224
> [   92s] 	PublicInbox::InboxWritable::with_umask(PublicInbox::InboxWritable=HASH(0x564bfcccbb10), CODE(0x564bfd49d0f8), PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 760
> [   92s] 	PublicInbox::SearchIdx::with_umask(PublicInbox::SearchIdx=HASH(0x564bfcf5be78), CODE(0x564bfd49d0f8), PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/SearchIdx.pm line 767
> [   92s] 	PublicInbox::SearchIdx::index_sync(PublicInbox::SearchIdx=HASH(0x564bfcf5be78), HASH(0x564bfd3f7bc0)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 194
> [   92s] 	eval {...} called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 192
> [   92s] 	PublicInbox::Import::_update_git_info(PublicInbox::Import=HASH(0x564bfccbd5f8), 1) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 494
> [   92s] 	eval {...} called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/Import.pm line 494
> [   92s] 	PublicInbox::Import::done(PublicInbox::Import=HASH(0x564bfccbd5f8)) called at /home/abuild/rpmbuild/BUILD/public-inbox-1.7.0/blib/lib/PublicInbox/TestCommon.pm line 707
> [   92s] 	PublicInbox::TestCommon::create_inbox("v1", "version", 1, "indexlevel", "basic", "tmpdir", "/tmp/pi-reindex-time-range-22646-lChw/v1") called at t/reindex-time-range.t line 18

Aha, so it was the $_ from the map {...} block in t/reindex-time-range.t
which only gets called in the initial run due to caching.  I totally
forgot about that :x  Thanks for that backtrace, this should fix it:

---------8<---------
Subject: [PATCH] searchidx: avoid modification of read-only `$_'

This fixes the "Modification of a read-only value attempted at ..."
error in an initial run of t/reindex-time-range.t.  It was
reproducible by running `rm -rf t/data-gen/reindex-time-range.v*'
before `make && prove -bvw t/reindex-time-range.t'.  Thanks to
Jörg Rödel for providing the backtrace which helped find this.

Debugged-by: Jörg Rödel <joro@8bytes.org>
Link: https://public-inbox.org/meta/YZuZEY+WSnm4wlrS@8bytes.org/
---
 lib/PublicInbox/SearchIdx.pm | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
index 6e2e614c..4e5d7d44 100644
--- a/lib/PublicInbox/SearchIdx.pm
+++ b/lib/PublicInbox/SearchIdx.pm
@@ -896,20 +896,20 @@ sub log2stack ($$$) {
 		push @cmd, "--$k=$v";
 	}
 	my $fh = $git->popen(@cmd, $range);
-	my ($at, $ct, $stk, $cmt);
-	while (<$fh>) {
+	my ($at, $ct, $stk, $cmt, $l);
+	while (defined($l = <$fh>)) {
 		return if $sync->{quit};
-		if (/\A([0-9]+)-([0-9]+)-($OID)$/o) {
+		if ($l =~ /\A([0-9]+)-([0-9]+)-($OID)$/o) {
 			($at, $ct, $cmt) = ($1 + 0, $2 + 0, $3);
 			$stk //= PublicInbox::IdxStack->new($cmt);
-		} elsif (/$del/) {
+		} elsif ($l =~ /$del/) {
 			my $oid = $1;
 			if ($D) { # reindex case
 				$D->{pack('H*', $oid)}++;
 			} else { # non-reindex case:
 				$stk->push_rec('d', $at, $ct, $oid, $cmt);
 			}
-		} elsif (/$add/) {
+		} elsif ($l =~ /$add/) {
 			my $oid = $1;
 			if ($D) {
 				my $oid_bin = pack('H*', $oid);

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2021-11-22 18:23 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-19 12:42 'make test' failures while packaging for openSUSE Jörg Rödel
2021-11-19 18:54 ` Eric Wong
2021-11-19 21:46   ` Jörg Rödel
2021-11-22  6:55     ` [PATCH] searchidx: add some extra diagnostics for odd message Eric Wong
2021-11-22  7:42       ` [PATCH] t/lei-mirror: skip lei comparisons if lei missing Eric Wong
2021-11-22 13:47         ` Jörg Rödel
2021-11-22 17:24           ` Eric Wong
2021-11-22 13:20       ` [PATCH] searchidx: add some extra diagnostics for odd message Jörg Rödel
2021-11-22 18:23         ` [PATCH] searchidx: avoid modification of read-only `$_' Eric Wong

Code repositories for project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).