Re: Sparse clones (Was: Re: [PATCH 1/2] upload-pack: support subtree packing)

[Elijah Newren <newren@gmail.com>]

On Tue, Jul 27, 2010 at 9:36 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> 2010/7/28 Elijah Newren <newren@gmail.com>:
>> 1) A user controls sparseness by passing rev-list arguments to clone.
<snip>
>> For example,
>>  $ git clone ssh://repo.git dst -- Documentation/
>
> Does pathspec is supported to in addition to prefix?

Basically, whatever git log or git rev-list accepts.  I think I saw
some other discussion about making those adopt some of the
code/capability of git grep, which would automatically benefit sparse
clones.  But until then, no, because I need to be able to take these
arguments and automatically pass them on to log, rev-list, etc.

> So you basically kill off shallow clone too, with "master~6..master".

Yes, that was part of the plan...extend the capabilities of shallow
clones in two ways: allowing the user to specify a cutoff via a
revision identifier as well as a number of commits, and allow people
to clone (and fetch-from/push-to) other "shallow" clones.

> I wonder what happens if user does "git clone ... master~6..master~3"?

Currently, that'd break -- just like it similarly does for fast-export
(see t/t9350-fast-export.sh, 'no exact-ref revisions included').  I
had been thinking of trying to get that fixed for both cases by making
it result in a "master" branch that is "three commits behind" what you
clone/fast-export from.  You'd have to look for and disallow other
special cases like "git fast-export ... master^1 master^2" or "git
clone ... :/searchstring".

I'm not sure how this interacts with Avery's suggestion to just ignore
branch/tag limiting.

> Revision walking is not the only gate to access objects. Others like
> diff machinery needs also be taught about rev-list limits.

Right, good point.  Are there others than the diff machinery (and the
fsck special case) that you know of?

> What information would you send to the server to request new pack in
> sparse clone? Currently we send all commit tips. rev-list has a notion
> to subtract commit trees. I don't know if it can "add" or "subtract"
> tree prefix though.

When "densifying" a sparse clone, I was (initially at least) just
going to treat it like an initial clone and re-download _everything_
(even if sparsifying rather than densifying).  I assumed it'd be rare
to want to do such an operation, but yeah, in the future someone might
want a smarter way to handle it.