Re: Passing revs to git-bundle-create via stdin

[Jeff King <peff@peff.net>]

On Tue, May 23, 2017 at 01:44:55AM +0200, ch wrote:

> I'm using git bundles to create (incremental) backups of my local repositories.
> This works quite well but for certain repositories I'm getting unexpectedly big
> incremental bundles. I did some testing and from what I can tell it seems
> git-bundle-create has issues processing revs passed via stdin. To illustrate
> the problem I have included a small bash script below.
> 
> I'm using Git for Windows 2.13.0.windows.1 (64-bit). Unfortunately I don't have
> access to a non-Windows box to check whether it's a problem specific to the
> Windows port.

Thanks for an easy reproduction recipe. I see the problem on Linux, too.

I think what's happening is that git-bundle actually runs _two_
traversals using the command-line arguments. It kicks off an external
rev-list via compute_and_write_prerequisites(), and then feeds the
arguments again to setup_revisions(). The first one eats all of stdin,
and the second just sees an empty input.

You can see it working if you do:

  $ git bundle create from-terminal.git --all --stdin
  ^feature
  ^master^
  [press ^D, i.e., ctrl-d]
  ^feature
  ^master^
  [press ^D again]

Hitting ^D tells the terminal driver to send an EOF; the first one goes
to the child rev-list, and then we repeat the input to get read by the
second traversal. The result is identical to your command-line-only
output. I have no idea if the ^D thing works at all on Windows, but I
don't mean it even as a workaround. It was just a way of confirming my
guess about the double-read.

The real solutions I can think of are:

  1. Teach git-bundle not to require the double-read. I'm not sure why
     it's written the way it is, but presumably it would be tricky to
     undo it (or we would have just written it the other way in the
     first place!)

  2. Git-bundle could buffer stdin and feed it to the two traversals. I
     think this actually ends up a little tricky, because the second
     traversal is done in-process (so we'd have to actually re-feed the
     buffer to our stdin via a "struct async", which feels pretty
     hacky).

  3. git-bundle could natively support --stdin, reading each line and
     convert it into traversal arguments. This is the quickest way to
     make your example work, but I suspect there will be funky corner
     cases (because we'd have to replicate the same rules that
     revision.c uses to read its input).

None of those are incredibly appealing.

-Peff