diff options
| author | Eric Wong (Contractor, The Linux Foundation) <e@80x24.org> | 2018-02-20 21:00:21 +0000 |
|---|---|---|
| committer | Eric Wong (Contractor, The Linux Foundation) <e@80x24.org> | 2018-02-20 21:00:21 +0000 |
| commit | feabfb1809b911fc97538282234c8b1f087ddb6a (patch) | |
| tree | 8fc484f46f1c2c191ccb0a72fcbebfcad38bc115 /lib/PublicInbox/Import.pm | |
| parent | b42bbc915750cf2f0c559514041ba3f5d1a44a12 (diff) | |
| download | public-inbox-feabfb1809b911fc97538282234c8b1f087ddb6a.tar.gz | |
This is too slow, currently. Working with only 2017 LKML
archives:
git-only: ~1 minute
git + SQLite: ~12 minutes
git+Xapian+SQlite: ~45 minutes
So yes, it looks like we'll need to parallelize Xapian indexing,
at least.
Diffstat (limited to 'lib/PublicInbox/Import.pm')
| -rw-r--r-- | lib/PublicInbox/Import.pm | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm index 364ab602..1a2698a7 100644 --- a/lib/PublicInbox/Import.pm +++ b/lib/PublicInbox/Import.pm @@ -285,6 +285,7 @@ sub add { # v2: we need this for Xapian if ($self->{want_object_id}) { chomp($self->{last_object_id} = $self->get_mark(":$blob")); + $self->{last_object_size} = $n; } my $ref = $self->{ref}; |