[PATCH] doc: explain why file: URLs and bundles don't mix

[PATCH] doc: explain why file: URLs and bundles don't mix

[Alyssa Ross]

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
 Documentation/urls.txt | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index b05da95788..c83d9f859e 100644
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -43,14 +43,24 @@ syntaxes may be used:
 - \file:///path/to/repo.git/
 
 ifndef::git-clone[]
-These two syntaxes are mostly equivalent, except when cloning, when
-the former implies --local option. See linkgit:git-clone[1] for
-details.
+These two syntaxes are mostly equivalent, with some exceptions:
+
+- When cloning, the former implies --local option. See
+  linkgit:git-clone[1] for details.
+
+- The latter is implemented using linkgit:git-upload-pack[1], which
+  expects its repository to be a directory, and therefore does not
+  work for bundles (see linkgit:git-bundle[1]).
 endif::git-clone[]
 
 ifdef::git-clone[]
-These two syntaxes are mostly equivalent, except the former implies
---local option.
+These two syntaxes are mostly equivalent, with some exceptions:
+
+- The former implies --local option.
+
+- The latter is implemented using linkgit:git-upload-pack[1], which
+  expects its repository to be a directory, and therefore does not
+  work for bundles (see linkgit:git-bundle[1]).
 endif::git-clone[]
 
 When Git doesn't know how to handle a certain transport protocol, it
-- 
2.21.0

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

[Junio C Hamano]

Alyssa Ross <hi@alyssa.is> writes:

> Signed-off-by: Alyssa Ross <hi@alyssa.is>
> ---
>  Documentation/urls.txt | 20 +++++++++++++++-----
>  1 file changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/urls.txt b/Documentation/urls.txt
> index b05da95788..c83d9f859e 100644
> --- a/Documentation/urls.txt
> +++ b/Documentation/urls.txt
> @@ -43,14 +43,24 @@ syntaxes may be used:
>  - \file:///path/to/repo.git/
>  
>  ifndef::git-clone[]
> -These two syntaxes are mostly equivalent, except when cloning, when
> -the former implies --local option. See linkgit:git-clone[1] for
> -details.
> +These two syntaxes are mostly equivalent, with some exceptions:
> +
> +- When cloning, the former implies --local option. See
> +  linkgit:git-clone[1] for details.
> +
> +- The latter is implemented using linkgit:git-upload-pack[1], which
> +  expects its repository to be a directory, and therefore does not
> +  work for bundles (see linkgit:git-bundle[1]).

Hmm, I do not think this is quite true.  

If "git clone /path/to/repo.bndl" implied --local, we would end up
trying to hardlink into /path/to/repo.bndl/objects and would fail.

I think what is closer to the reaility is that we check if the
source is a bundle when the local filesystem path is used and try to
clone from the bundle, before using the local filesystem path as a
directory we can "clone --local" from.  On the other hand, when the
<scheme>://<path> syntax is used, we do not even bother seeing if
the named resource is a bundle, or if --local optimization is
possible (because we do not bother seeing if the named resource is a
local filesystem entity, either).

A possibly interesting tangent to think about is what would happen
if we slightly tweak the above design.  What it would require for
the code to take "git clone https://site/repo.bndl", realize that
the named resource is a bundle file, curl/wget it and clone from
that downloaded bundle?  And if it is feasible to implement, would
it even be a good idea to begin with?  I do not have a ready answer
to either of these questions myself.

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 2517 bytes --]

> > +- When cloning, the former implies --local option. See
> > +  linkgit:git-clone[1] for details.
> > +
> > +- The latter is implemented using linkgit:git-upload-pack[1], which
> > +  expects its repository to be a directory, and therefore does not
> > +  work for bundles (see linkgit:git-bundle[1]).
>
> Hmm, I do not think this is quite true.
>
> If "git clone /path/to/repo.bndl" implied --local, we would end up
> trying to hardlink into /path/to/repo.bndl/objects and would fail.
>
> I think what is closer to the reaility is that we check if the
> source is a bundle when the local filesystem path is used and try to
> clone from the bundle, before using the local filesystem path as a
> directory we can "clone --local" from.  On the other hand, when the
> <scheme>://<path> syntax is used, we do not even bother seeing if
> the named resource is a bundle, or if --local optimization is
> possible (because we do not bother seeing if the named resource is a
> local filesystem entity, either).

Would the following work better for you?

When cloning, the former will check to see if the source is a bundle.
If it is, it will clone from the bundle, otherwise it will behave as if
given --local. The latter performs neither of these checks, and
therefore does not support bundles.

> A possibly interesting tangent to think about is what would happen
> if we slightly tweak the above design.  What it would require for
> the code to take "git clone https://site/repo.bndl", realize that
> the named resource is a bundle file, curl/wget it and clone from
> that downloaded bundle?  And if it is feasible to implement, would
> it even be a good idea to begin with?  I do not have a ready answer
> to either of these questions myself.

I was very surprised to find that cloning from a bundle didn't even work
with a file URL, especially since the documentation seemed to imply that
it should. I assumed that file URLs not supporting bundles was an
oversight, and made some attempt to read the code with a view to fixing
it. Once I realised that file URLs went through a completely different
code path to paths, I decided it was better to just document things as
they were.

I think it would be nice if, at least for file URLs, bundles were
supported. I doubt supporting cloning a bundle over a network would
inherently cause problems (although I'm by no means qualified to make
such an assessment) -- if it's not clear whether it's a good idea to do
so, it could always be implemented without being advertised.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

[Jeff King]

On Mon, May 13, 2019 at 05:05:14PM +0900, Junio C Hamano wrote:

> I think what is closer to the reaility is that we check if the
> source is a bundle when the local filesystem path is used and try to
> clone from the bundle, before using the local filesystem path as a
> directory we can "clone --local" from.  On the other hand, when the
> <scheme>://<path> syntax is used, we do not even bother seeing if
> the named resource is a bundle, or if --local optimization is
> possible (because we do not bother seeing if the named resource is a
> local filesystem entity, either).

Yeah. My concern on reading Alyssa's patch is that it goes into too much
detail about "why", when this is basically just "it's a bug that nobody
has yet fixed". I think it makes sense to warn people about the
behavior, but we should probably be vague about the reasons and up-front
that there is really no good reason that it doesn't work (and it might
in the future).

Of course fixing the issue is better still. ;)

> A possibly interesting tangent to think about is what would happen
> if we slightly tweak the above design.  What it would require for
> the code to take "git clone https://site/repo.bndl", realize that
> the named resource is a bundle file, curl/wget it and clone from
> that downloaded bundle?  And if it is feasible to implement, would
> it even be a good idea to begin with?  I do not have a ready answer
> to either of these questions myself.

I've been meaning to come back to this for, oh, going on 8 years now:

  https://public-inbox.org/git/20111110074330.GA27925@sigill.intra.peff.net/

(though it was meant to work with a CDN "try this first then top-off"
strategy; with Jonathan Tan's more flexible CDN work which uses raw
packfiles, I doubt I'd come back to this).

-Peff

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

[Jeff King]

On Mon, May 13, 2019 at 04:54:55PM +0000, Alyssa Ross wrote:

> > I think what is closer to the reaility is that we check if the
> > source is a bundle when the local filesystem path is used and try to
> > clone from the bundle, before using the local filesystem path as a
> > directory we can "clone --local" from.  On the other hand, when the
> > <scheme>://<path> syntax is used, we do not even bother seeing if
> > the named resource is a bundle, or if --local optimization is
> > possible (because we do not bother seeing if the named resource is a
> > local filesystem entity, either).
> 
> Would the following work better for you?
> 
> When cloning, the former will check to see if the source is a bundle.
> If it is, it will clone from the bundle, otherwise it will behave as if
> given --local. The latter performs neither of these checks, and
> therefore does not support bundles.

FWIW, I like that much better than the original proposal. I think it
would also be OK to say something like: ...does not support bundles (but
may in the future).

-Peff