Re: GitGitGadget on git/git, was Re: Should we auto-close PRs on git/git?

[Eric Wong <e@80x24.org>]

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi Eric,
> 
> On Tue, 26 Nov 2019, Eric Wong wrote:
> 
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > > On Tue, 26 Nov 2019, Eric Wong wrote:
> > > > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > > > > The biggest obstacle is that at least one of those Pipelines requires
> > > > > access to a clone of public-inbox.org/git, and cloning that is rather
> > > > > expensive. Even a shallow fetch would be super expensive, by virtue of
> > > > > _all_ the mails being blobs reachable from the tip commit's tree.
> > > >
> > > > Fwiw, lore.kernel.org/git/$EPOCH.git ought to be somewhat cheaper,
> > > > but it's a different (more scalable) format which requires SQLite:
> > > >
> > > > 	https://public-inbox.org/public-inbox-v2-format.html
> > >
> > > Is this incremental? GitGitGadget needs this to be incremental ;-)
> >
> > Incremental as far as "git fetch" goes?  Of course :>
> > The "m" file is overwritten with every commit, so the tree size
> > stays at 1 (tree growth was a major scalability problem in v1).
> 
> Let me try again:
> 
> GitGitGadget "reads" the mail via the incremental clone, remembering the
> hash of the latest processed commit. When the Azure Pipeline runs, it
> first fetches, and if the commit is still the same, does nothing but exit
> with success. If the commit is different, it looks at the mails that were
> added, via `git log -p <previous-tip-commit>..<tip-commit>`.
> 
> Is that possible with the v2 format?

Of course, yes.  The Xapian and SQLite indexing also works the
same way "git log prev..tip" and storing the latest commit hash.