user/dev discussion of public-inbox itself
 help / color / Atom feed
From: Jonathan Corbet <corbet@lwn.net>
To: meta@public-inbox.org
Cc: Eric Wong <e@80x24.org>
Subject: Race condition in public-inbox-nntpd?
Date: Sat, 13 Oct 2018 12:46:58 -0600
Message-ID: <20181013124658.23b9f9d2@lwn.net> (raw)

So for a little while, I've been seeing occasional NNTP weirdness,
associated with log lines like:

  Oct 13 18:01:06 archive.lwn.net public-inbox-nntpd[12352]: error from:
  XOVER 196731-196731 (BUG: nested long response at /usr/local/share/perl5/PublicInbox/NNTP.pm line 588.

Such complaints tend to be immediately followed by significant
disgruntlement on the client side.

I use gnus to read the various NNTP feeds, and I mark articles (with "u")
when I want to keep them around; I typically have quite a few of them
marked at any given time in a group like linux-kernel.  When I open the
group in gnus, it will do an XOVER on each of those marked articles,
generating dozens of single-article XOVERs in quick succession.  It's
always the single-article ones that fail; the big XOVER for all of the new
stuff at the end works just fine.

Another user has complained that things fail with Thunderbird, again with
the same symptoms on the server side.

I have "fixed" the problem with this little patch:

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 022bb80..017ad88 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -951,6 +951,10 @@ sub event_read {
 		$self->{rbuf} .= $$buf;
 	}
 	my $r = 1;
+	if ($self->{long_res}) {
+		err($self, "long-res on event read");
+		$r = 0;
+	}
 	while ($r > 0 && $self->{rbuf} =~ s/\A\s*([^\r\n]*)\r?\n//) {
 		my $line = $1;
 		return $self->close if $line =~ /[[:cntrl:]]/s;

That makes things work, but it is clearly papering over the real
problem.

I've spent a fair while staring at the code.  As far as I can tell, the
logic there should be sufficient to prevent this from happening; it's not
supposed to be reading while a long response is in the works.  But somehow
it happens.

Does anybody have any thoughts on how this could be coming about?

Thanks,

jon

             reply index

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-13 18:46 Jonathan Corbet [this message]
2018-10-16  6:36 ` Eric Wong
2018-10-16 14:04   ` Jonathan Corbet
2018-12-06  3:59 ` [PATCH] nntp: prevent event_read from firing twice in a row Eric Wong

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181013124658.23b9f9d2@lwn.net \
    --to=corbet@lwn.net \
    --cc=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.org/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git