Re: internal format

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

> v1 or v2?  Some of the reasoning for v2 was here:
>   https://public-inbox.org/meta/20180209205140.GA11047@dcvr/

IIUC, the issues you consider important are:

- Size
- Time to perform "git rev-list --objects --all"
- Flexibility, e.g. to be able to remove messages.

For size your benchmarks seem to indicate that as long as it's kept
inside Git, the choice of format doesn't actually affect it
significantly (and this matches my expectations).
Tho I guess it's probably possible to improve on it with enough efforts
(e.g. storing attachments separately, or splitting large messages into
chunks, e.g. like `bup` does), but I doubt it's worth the effort
(especially if you assume that the mailing-list imposes a limit on
message size).

For timing, I'm curious why you only consider
"git rev-list --objects --all".  Which operation does this corresponds
to in public-inbox and is that really the only one that is
performance-sensitive?

> As for git itself: reliability, ease-of-replication, storage
> efficiency.

Yes, that part I totally understand (same reason I used Git in BuGit
https://gitlab.com/monnier/bugit).  Part of my question was related to
the fact that in BuGit I store the messages in the commit-object rather
than in files (which trivially gives me conflict-free merges as well as
"discussion threads") so I was wondering if it would make sense in the
case of public-inbox to keep the email messages in the commit objects
rather than in files, but since I don't really know which operations are
frequent/important I really have no idea.

One thing that strikes me is that you don't seem to use its
"decentralization": IIUC public-inbox always assumes one of the
repositories is the "master" and others are mirrors (or mirrors of
mirrors), so you get efficient "fast-forward" updates, but you
don't do "merges".

This probably means that keeping the email messages in commit objects
wouldn't bring any benefits.

Also this means that public-inbox could freely rewrite history, for
example (which you'll need to really expunge messages) and just use
"forced updates" in mirrors.

Now I'm left wondering what it would mean for something like
public-inbox to support merging.


        Stefan