"git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

"git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

[Robert P. J. Day]

  probably doing something idiotic but i'm enumerating variations of
shallow cloning, and tried the following:

$ git clone --shallow-exclude=master https://github.com/django/django.git
Cloning into 'django'...
fatal: the remote end hung up unexpectedly
$

  it is entirely reproducible, and some googling suggests that this
represents an error at the *other* end, which in some weird way does
not support that clone option. that seems strange ... should this
option work? am i using it incorrectly?

  wait, hang on ... i just picked one of django's topic branches at
random, and this did succeed:

$ git clone --shallow-exclude=stable/2.0.x https://github.com/django/django.git
Cloning into 'django'...
remote: Enumerating objects: 33112, done.
remote: Counting objects: 100% (33112/33112), done.
... etc etc ...
$

but all this gave me was the master branch. i clearly don't understand
what this option is supposed to do.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                  http://crashcourse.ca/dokuwiki

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: "git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

[Duy Nguyen]

On Fri, Mar 15, 2019 at 7:17 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:
>
>
>   probably doing something idiotic but i'm enumerating variations of
> shallow cloning, and tried the following:
>
> $ git clone --shallow-exclude=master https://github.com/django/django.git
> Cloning into 'django'...
> fatal: the remote end hung up unexpectedly
> $
>
>   it is entirely reproducible, and some googling suggests that this
> represents an error at the *other* end, which in some weird way does
> not support that clone option. that seems strange ... should this
> option work? am i using it incorrectly?
>
>   wait, hang on ... i just picked one of django's topic branches at
> random, and this did succeed:

Yeah i think when you request shallow clone, by default it only gets
one branch (see --single-branch, often 'master'). So when you specify
--shallow-exclude you essentially say 'give me master branch but
exclude everything from master'.

I should probably make it print a friendlier message than simply
terminateing like that (it's still a guess, I haven't tried it out)

> $ git clone --shallow-exclude=stable/2.0.x https://github.com/django/django.git
> Cloning into 'django'...
> remote: Enumerating objects: 33112, done.
> remote: Counting objects: 100% (33112/33112), done.
> ... etc etc ...
> $
>
> but all this gave me was the master branch. i clearly don't understand
> what this option is supposed to do.

You're saying 'give me master branch, exclude everything that is
reachable from stable/2.0.x'. If there's nothing in common between
master and this branch, you get full 'master' branch. Otherwise you
get a shallow 'master' branch.

This option is meant to give you another way to cut the history.
Instead of saying 'give me X latest commits' with --shallow, you could
say 'give me commits since this cut point' and the cut point is often
a tag, on the same branch. Using an unrelated bracnh or tag could give
some surprising result.
-- 
Duy

Re: "git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

[Robert P. J. Day]

On Fri, 15 Mar 2019, Duy Nguyen wrote:

> On Fri, Mar 15, 2019 at 7:17 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> >
> >
> >   probably doing something idiotic but i'm enumerating variations of
> > shallow cloning, and tried the following:
> >
> > $ git clone --shallow-exclude=master https://github.com/django/django.git
> > Cloning into 'django'...
> > fatal: the remote end hung up unexpectedly
> > $
> >
> >   it is entirely reproducible, and some googling suggests that this
> > represents an error at the *other* end, which in some weird way does
> > not support that clone option. that seems strange ... should this
> > option work? am i using it incorrectly?
> >
> >   wait, hang on ... i just picked one of django's topic branches at
> > random, and this did succeed:
>
> Yeah i think when you request shallow clone, by default it only gets
> one branch (see --single-branch, often 'master'). So when you
> specify --shallow-exclude you essentially say 'give me master branch
> but exclude everything from master'.
>
> I should probably make it print a friendlier message than simply
> terminateing like that (it's still a guess, I haven't tried it out)

  this is the first time i've played with this feature, so i'm still
working my way through the man page, trying to figure out the various
valid combinations for shallow cloning.

  i notice that the SYNOPSIS for "man git-clone" does not contain all
of the supported options (eg., --shallow-exclude is missing, among
others). is that deliberate?

rday
-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                  http://crashcourse.ca/dokuwiki

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: "git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

[Duy Nguyen]

On Fri, Mar 15, 2019 at 8:19 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:
>   this is the first time i've played with this feature, so i'm still
> working my way through the man page, trying to figure out the various
> valid combinations for shallow cloning.
>
>   i notice that the SYNOPSIS for "man git-clone" does not contain all
> of the supported options (eg., --shallow-exclude is missing, among
> others). is that deliberate?

No. It's either laziness or giving up on adding every option in the
SYNOPSIS. Improvements are welcome. I can see now that --single-branch
is mentioned in --depth (the original option to make a shallow clone)
but not on the newer ones. My bad.
-- 
Duy

Re: "git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

[Robert P. J. Day]

On Fri, 15 Mar 2019, Duy Nguyen wrote:

> On Fri, Mar 15, 2019 at 8:19 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> >   this is the first time i've played with this feature, so i'm
> > still working my way through the man page, trying to figure out
> > the various valid combinations for shallow cloning.
> >
> >   i notice that the SYNOPSIS for "man git-clone" does not contain
> > all of the supported options (eg., --shallow-exclude is missing,
> > among others). is that deliberate?
>
> No. It's either laziness or giving up on adding every option in the
> SYNOPSIS. Improvements are welcome. I can see now that
> --single-branch is mentioned in --depth (the original option to make
> a shallow clone) but not on the newer ones. My bad.

  it's fairly obvious that, when you have a git command with a
bazillion options, it's pointless to try to include everything in the
SYNOPSIS part of the man page -- best to just go with something like:

  SYNOPSIS
       git log [<options>] [<revision range>] [[--] <path>...]


what should (IMHO) be avoided are *incomplete* synopses, as those are
exactly the ones that can mislead the reader into thinking that's the
full set of options. "man git-clone" might be at the point of just
reducing the synopsis.

  also, i think "man git-clone" could really use a set of examples for
shallow cloning. i'd offer to write it but i'm still figuring it out.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                  http://crashcourse.ca/dokuwiki

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: "git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

[Duy Nguyen]

On Fri, Mar 15, 2019 at 9:31 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:
>   also, i think "man git-clone" could really use a set of examples for
> shallow cloning. i'd offer to write it but i'm still figuring it out.

Cool. I'll give you a quick introduction to help with those examples :D

Side note: there's also narrow/lazy clone (or something with the
promisor) but I still haven't caught up with that. So none of that
here, but I think we should eventually have some examples/description.

A shallow clone is basically a repository with commit history cut
short. If you have a commit though, you have full content. It's not
cut by paths or anything.

"git clone --depth=X" the the normal way to do this. You get a
repository where all branches and tags have maximum X commits deep.
But since the main purpose of shallow clone is redude download,
--single-branch is made default when --depth is present, so you get
the default branch instead of all the branches (you would need
--no-single-branch for that). You could see these cut points with "git
log --decorate". I think they are marked "crafted" or something.

Once you have made a shallow clone, you could do everything like
usual. Local operations of course can only reach as far as the cut
points. Pull and push to a full clone are possible (it's even possible
to do so to another shallow clone).

When you "git fetch", the cut points remain the same, so you get more
recent commits and the history depth increases. If you do "git fetch
--depth=X" then the cut points are adjusted so that you only have
maximum X commits deep for all select branches, the same way a shallow
clone is made initially. "git fetch --unshallow" can be used to turn a
shallow clone into a complete one. And I'm pretty sure you could turn
a complete one to shallow with "git fetch --depth".

While --depth works fine most of the time, sometimes you want a
different way to define these shallow/cut points. --shallow-since and
--shallow-exclude are added as the result. --shallow-since cut the
history by a specific date (for all select branches).
--shallow-exclude cuts the history by a ref (often a tag). As I
mentioned before, it's mostly used to say "I want a shallow clone from
(but not including) v2.9.0".

And I think that's pretty much it. There's --update-shallow option but
I can't remember when that's needed. Shallow clones may also prevent
you from using some optional features. Pack bitmap is one of those and
I think probably commit graph too. It's not technical limitation, just
lack of time/effort to support them in shallow clones.
-- 
Duy

Re: "git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

[Robert P. J. Day]

On Sat, 16 Mar 2019, Duy Nguyen wrote:

> On Fri, Mar 15, 2019 at 9:31 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> >   also, i think "man git-clone" could really use a set of examples for
> > shallow cloning. i'd offer to write it but i'm still figuring it out.
>
> Cool. I'll give you a quick introduction to help with those examples
> :D

  ... snip ...

  i can see today is going to be busy ... thanks for all that, will
start reading shortly.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                  http://crashcourse.ca/dokuwiki

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================