index: support --max-size / publicinbox.indexMaxSize

In normal mail paths, we can rely on MTAs being configured with
reasonable limits in the -watch and -mda mail injection paths.

However, the MTA is bypassed in a git-only delivery path, a BOFH
could inject a large message and DoS users attempting to mirror
a public-inbox.

This doesn't protect unindexed WWW interfaces from Email::MIME
memory explosions on v1 inboxes.  Probably nobody cares about
unindexed WWW interfaces anymore, especially now that Xapian is
optional for indexing.

1 files changed, 3 insertions, 0 deletions

diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm
index 12cc1f13..01b8bed6 100644
--- a/lib/PublicInbox/V2Writable.pm
+++ b/lib/PublicInbox/V2Writable.pm
@@ -120,6 +120,7 @@ sub new {
                 last_commit => [], # git repo -> commit
         };
         $self->{shards} = count_shards($self) || nproc_shards($creat);
+        $self->{index_max_size} = $v2ibx->{index_max_size};
         bless $self, $class;
 }
 
@@ -867,6 +868,7 @@ sub atfork_child {
 
 sub mark_deleted ($$$$) {
         my ($self, $sync, $git, $oid) = @_;
+        return if PublicInbox::SearchIdx::too_big($self, $git, $oid);
         my $msgref = $git->cat_file($oid);
         my $mime = PublicInbox::MIME->new($$msgref);
         my $mids = mids($mime->header_obj);
@@ -993,6 +995,7 @@ sub multi_mid_q_push ($$$) {
 
 sub reindex_oid ($$$$) {
         my ($self, $sync, $git, $oid) = @_;
+        return if PublicInbox::SearchIdx::too_big($self, $git, $oid);
         my ($num, $mid0, $len);
         my $msgref = $git->cat_file($oid, \$len);
         return if $len == 0; # purged