user/dev discussion of public-inbox itself
 help / color / mirror / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
			download mbox.gz: |
* [PATCH 4/4] Date::Parse is now optional
  2019-11-29 12:25  6% [PATCH 0/4] drop Date::Parse dependency Eric Wong
@ 2019-11-29 12:25  7% ` Eric Wong
  0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-11-29 12:25 UTC (permalink / raw)
  To: meta

-mda should not be dealing with broken Date: headers
nowadays, and deprioritize it in our documentation and
internal checks.
---
 INSTALL                  | 9 +++++----
 Makefile.PL              | 1 -
 TODO                     | 4 ----
 ci/deps.perl             | 2 +-
 lib/PublicInbox/Admin.pm | 2 +-
 lib/PublicInbox/MDA.pm   | 5 ++---
 6 files changed, 9 insertions(+), 14 deletions(-)

diff --git a/INSTALL b/INSTALL
index 4d54e6a0..d41e513d 100644
--- a/INSTALL
+++ b/INSTALL
@@ -32,10 +32,6 @@ To accept incoming mail into a public inbox, you'll likely want:
 
 Beyond that, there is a long list of Perl modules required, starting with:
 
-* Date::Parse                      deb: libtimedate-perl
-                                   pkg: p5-TimeDate
-                                   rpm: perl-TimeDate
-
 * Digest::SHA                      typically installed with Perl
                                    rpm: perl-Digest-SHA
 
@@ -81,6 +77,11 @@ Numerous optional modules are likely to be useful as well:
                                    (speeds up process spawning on Linux,
                                     see public-inbox-daemon(8))
 
+- Date::Parse                      deb: libtimedate-perl
+                                   pkg: p5-TimeDate
+                                   rpm: perl-TimeDate
+                                   (for broken, mostly historical emails)
+
 - Plack::Middleware::ReverseProxy  deb: libplack-middleware-reverseproxy-perl
                                    pkg: p5-Plack-Middleware-ReverseProxy
                                    rpm: perl-Plack-Middleware-ReverseProxy
diff --git a/Makefile.PL b/Makefile.PL
index 4aa0caa7..c211bd5d 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -28,7 +28,6 @@ WriteMakefile(
 		# note: we use spamc(1), NOT the Perl modules
 		# We also depend on git.
 		# Keep this sorted and synced to the INSTALL document
-		'Date::Parse' => 0,
 
 		# libperl$PERL_VERSION,
 		# `perl5' on FreeBSD
diff --git a/TODO b/TODO
index 922163f8..592e306c 100644
--- a/TODO
+++ b/TODO
@@ -104,10 +104,6 @@ all need to be considered for everything we introduce)
 * imperfect scraper importers for obfuscated list archives
   (e.g. obfuscated Mailman stuff, Google Groups, etc...)
 
-* consider using HTTP::Date instead of Date::Parse, since we need the
-  former is capable of parsing RFC822-ish dates, used by Plack, and
-  the latter is missing from OpenBSD and maybe other distros.
-
 * improve performance and avoid head-of-line blocking on slow storage
 
 * share "git cat-file --batch" processes across inboxes to avoid
diff --git a/ci/deps.perl b/ci/deps.perl
index ae6083b9..330ba2f3 100755
--- a/ci/deps.perl
+++ b/ci/deps.perl
@@ -19,7 +19,6 @@ my $profiles = {
 	essential => [ qw(
 		git
 		perl
-		Date::Parse
 		Devel::Peek
 		Digest::SHA
 		Email::Simple
@@ -34,6 +33,7 @@ my $profiles = {
 
 	# everything optional for normal use
 	optional => [ qw(
+		Date::Parse
 		BSD::Resource
 		DBD::SQLite
 		DBI
diff --git a/lib/PublicInbox/Admin.pm b/lib/PublicInbox/Admin.pm
index dddeeae9..3d0d80b9 100644
--- a/lib/PublicInbox/Admin.pm
+++ b/lib/PublicInbox/Admin.pm
@@ -136,7 +136,7 @@ EOF
 }
 
 # TODO: make Devel::Peek optional, only used for daemon
-my @base_mod = qw(Email::MIME Date::Parse Devel::Peek);
+my @base_mod = qw(Email::MIME Devel::Peek);
 my @over_mod = qw(DBD::SQLite DBI);
 my %mod_groups = (
 	-index => [ @base_mod, @over_mod ],
diff --git a/lib/PublicInbox/MDA.pm b/lib/PublicInbox/MDA.pm
index b0dfac45..ef5e7dfa 100644
--- a/lib/PublicInbox/MDA.pm
+++ b/lib/PublicInbox/MDA.pm
@@ -6,7 +6,7 @@ package PublicInbox::MDA;
 use strict;
 use warnings;
 use Email::Simple;
-use Date::Parse qw(strptime);
+use PublicInbox::MsgTime;
 use constant MAX_SIZE => 1024 * 500; # same as spamc default, should be tunable
 use constant MAX_MID_SIZE => 244; # max term size - 1 in Xapian
 
@@ -51,8 +51,7 @@ sub usable_str {
 }
 
 sub usable_date {
-	my @t = eval { strptime(@_) };
-	scalar @t;
+	defined(eval { PublicInbox::MsgTime::str2date_zone($_[0]) });
 }
 
 sub alias_specified {

^ permalink raw reply	[relevance 7%]

* [PATCH 0/4] drop Date::Parse dependency
@ 2019-11-29 12:25  6% Eric Wong
  2019-11-29 12:25  7% ` [PATCH 4/4] Date::Parse is now optional Eric Wong
  0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2019-11-29 12:25 UTC (permalink / raw)
  To: meta

I started working on the async batch interface for git many
months ago, but didn't have a good use case for it.  The
comparison test in msgtime_cmp gives me an excuse to start using
it :)

Spam email I don't care about, but there's some differences for
folks that send valid mails:

The new code gets tripped up by 4 digit dates from non-sensical
times (emails from 1904, really?), so maybe some adjustment
is necessary.

There's also some bogus dates from the year 71685, so that's
a case where falling back on the Received: header is a good
choice.

Maybe some more odd caes when checks are done running...

Eric Wong (4):
  git: async batch interface
  add msgtime_cmp maintainer test
  msgtime: drop Date::Parse for RFC2822
  Date::Parse is now optional

 INSTALL                    |   9 ++-
 MANIFEST                   |   2 +
 Makefile.PL                |   1 -
 TODO                       |   4 -
 ci/deps.perl               |   2 +-
 lib/PublicInbox/Admin.pm   |   2 +-
 lib/PublicInbox/Git.pm     |  94 +++++++++++++++++-----
 lib/PublicInbox/MDA.pm     |   5 +-
 lib/PublicInbox/MsgTime.pm | 118 +++++++++++++++++++++++----
 t/msgtime.t                |   7 ++
 xt/git_async_cmp.t         |  61 ++++++++++++++
 xt/msgtime_cmp.t           | 161 +++++++++++++++++++++++++++++++++++++
 12 files changed, 415 insertions(+), 51 deletions(-)
 create mode 100644 xt/git_async_cmp.t
 create mode 100644 xt/msgtime_cmp.t


^ permalink raw reply	[relevance 6%]

Results 1-2 of 2 | reverse
2019-11-29 12:25  6% [PATCH 0/4] drop Date::Parse dependency Eric Wong
2019-11-29 12:25  7% ` [PATCH 4/4] Date::Parse is now optional Eric Wong

user/dev discussion of public-inbox itself

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 meta meta/ https://public-inbox.org/meta \
		meta@public-inbox.org
	public-inbox-index meta

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git