Re: Sparse clones (Was: Re: [PATCH 1/2] upload-pack: support subtree packing)

[Elijah Newren <newren@gmail.com>]

Hi,

2010/7/27 Elijah Newren <newren@gmail.com>:
> 2010/7/27 Avery Pennarun <apenwarr@gmail.com>:
>> Note that if you happen to want to implement it in a way that you'll
>> also get all the commit objects from your submodules too (which I
>> highly encourage :)) then downloading the trees is the easiest way.
>> Otherwise you won't know which submodule commits you need.
>
> Makes sense.  Seems like a good reason to include all the trees.

Actually, having thought about it more, I don't see the reason for
getting all the commit objects from submodules (unless those
submodules are at paths specified for download).  If a user has
specified that they just want the Documentation subdirectory, why
would it matter if the submodule under src/widgets was downloaded?
They don't want to do anything with any of its contents, so I don't
see why they'd needs its trees or commits.  Am I missing something?


Also, I'm rethinking the download-all-commits aspect too.  This is
partially due to Nguyễn's stats (and special usecases like
translators), partially because of security issues (it has already
been stated that only including stuff meant to be public is an
important security concern for clone[1], and commit logs for changes
completely outside specified paths might be considered non-public
data[2]), and partially because it reinforces my whole rev-list
limiting args idea (it makes it really clear that 'git log' should
automatically behave like 'git log -- Documentation/' in a sparse
clone of just Documentation/).

[1] e.g. http://article.gmane.org/gmane.comp.version-control.git/115835

[2] This isn't just theoretical either.  I have a couple big important
(to $dayjob and thus me) sparse-clone usecases in this situation and
have for a few years, but gave up on it thinking it wouldn't be
possible with sparse clones.  I instead wrote a fast filtering
mechanism using fast-export/fast-import that creates a new repository
and keeps track of the mapping between sha1sums in unfiltered and
filtered repos, allowing changes to be grafted between the two.  Kind
of a pain, and suboptimal for a few reasons.  It'd be really nice if I
could replace this stuff with sparse clones, but can't do that if
commit logs corresponding to changes completely outside the sparse
paths are included.