[BUG] git remote prune removes local tags, depending on fetch config

[BUG] git remote prune removes local tags, depending on fetch config

[Michael Giuffrida]

`git remote prune <name>` should "delete all stale remote-tracking
branches under <name>". I was surprised to discover, after some
troubleshooting, that it also deletes *all* local tags that don't
exist on the remote, if the following refspec is included in the
remote's fetch config:

    +refs/tags/*:refs/tags/*

So, if `remote.origin.fetch` is configured to fetch all tags from the
remote, any tags I create locally will be deleted when running `git
remote prune origin`. This is not intuitive [1], nor is is it
explained in the docs [2]. Is this behavior obvious to someone with a
better understanding of Git internals?

I did find a better way to automatically fetch tags (using tagopt
instead of adding the fetch refspec). However, the refspec doesn't
seem "wrong" in itself; in particular, `git fetch --tags` used to be
considered equivalent to specifying the refspec
"refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec
[3]. So I wouldn't expect it to "break" the behavior of another
command.

[1] https://stackoverflow.com/q/34687657/1327867
[2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
[3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50

Re: [BUG] git remote prune removes local tags, depending on fetch config

[Ævar Arnfjörð Bjarmason]

On Mon, Jan 15 2018, Michael Giuffrida jotted:

> `git remote prune <name>` should "delete all stale remote-tracking
> branches under <name>". I was surprised to discover, after some
> troubleshooting, that it also deletes *all* local tags that don't
> exist on the remote, if the following refspec is included in the
> remote's fetch config:
>
>     +refs/tags/*:refs/tags/*
>
> So, if `remote.origin.fetch` is configured to fetch all tags from the
> remote, any tags I create locally will be deleted when running `git
> remote prune origin`. This is not intuitive [1], nor is is it
> explained in the docs [2]. Is this behavior obvious to someone with a
> better understanding of Git internals?
>
> I did find a better way to automatically fetch tags (using tagopt
> instead of adding the fetch refspec). However, the refspec doesn't
> seem "wrong" in itself; in particular, `git fetch --tags` used to be
> considered equivalent to specifying the refspec
> "refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec
> [3]. So I wouldn't expect it to "break" the behavior of another
> command.
>
> [1] https://stackoverflow.com/q/34687657/1327867
> [2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
> [3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50

These docs are really confusing, but it is working as intended, and
really should be re-documented.

The `git remote prune` subcommand just ends up piggy-backing on
git-fetch, whose behavior is explained here:
https://git-scm.com/docs/git-fetch.html#git-fetch---prune

It's worked this way since at least v1.8.5.6, maybe at some distant
point in the past it only did this for branches when invoked via
git-remote as the documentation says.

RELATED:

I've actually had the reverse problem with this. I want some way to turn
this behavior on without explicitly hacking the refspec, so I can do it
globally in /etc/gitconfig or in ~/.gitconfig without screwing with the
config of each checkout on certain machines.

You can set fetch.prune=true, but that only prunes the branches, you
need to inject remote.origin.fetch into each checkout, unless I've
missed some way of doing this.

I wanted to add fetch.pruneTags that would make it as if you had
refs/tags/*:refs/tags/* in the fetch spec, but I haven't hacked that up
yet, if anyone can see any inherent issue with that plan I'd like to
know about it.

Re: [BUG] git remote prune removes local tags, depending on fetch config

[Michael Giuffrida]

Just to confirm, you're talking about the behavior of removing *all*
tags that aren't found on the remote? (Not just tags that used to be
on some remote, but have since been removed from that remote.) So,
with your proposed workflow, you would never create your own tags
locally, without pushing them to the remote before running `git fetch`
-- otherwise they would simply be deleted.

It doesn't seem like a useful feature -- you wouldn't expect `git
fetch --prune` to remove your local branches that you were developing
on, right?

On Mon, Jan 15, 2018 at 4:38 PM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Mon, Jan 15 2018, Michael Giuffrida jotted:
>
>> `git remote prune <name>` should "delete all stale remote-tracking
>> branches under <name>". I was surprised to discover, after some
>> troubleshooting, that it also deletes *all* local tags that don't
>> exist on the remote, if the following refspec is included in the
>> remote's fetch config:
>>
>>     +refs/tags/*:refs/tags/*
>>
>> So, if `remote.origin.fetch` is configured to fetch all tags from the
>> remote, any tags I create locally will be deleted when running `git
>> remote prune origin`. This is not intuitive [1], nor is is it
>> explained in the docs [2]. Is this behavior obvious to someone with a
>> better understanding of Git internals?
>>
>> I did find a better way to automatically fetch tags (using tagopt
>> instead of adding the fetch refspec). However, the refspec doesn't
>> seem "wrong" in itself; in particular, `git fetch --tags` used to be
>> considered equivalent to specifying the refspec
>> "refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec
>> [3]. So I wouldn't expect it to "break" the behavior of another
>> command.
>>
>> [1] https://stackoverflow.com/q/34687657/1327867
>> [2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
>> [3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50
>
> These docs are really confusing, but it is working as intended, and
> really should be re-documented.
>
> The `git remote prune` subcommand just ends up piggy-backing on
> git-fetch, whose behavior is explained here:
> https://git-scm.com/docs/git-fetch.html#git-fetch---prune
>
> It's worked this way since at least v1.8.5.6, maybe at some distant
> point in the past it only did this for branches when invoked via
> git-remote as the documentation says.
>
> RELATED:
>
> I've actually had the reverse problem with this. I want some way to turn
> this behavior on without explicitly hacking the refspec, so I can do it
> globally in /etc/gitconfig or in ~/.gitconfig without screwing with the
> config of each checkout on certain machines.
>
> You can set fetch.prune=true, but that only prunes the branches, you
> need to inject remote.origin.fetch into each checkout, unless I've
> missed some way of doing this.
>
> I wanted to add fetch.pruneTags that would make it as if you had
> refs/tags/*:refs/tags/* in the fetch spec, but I haven't hacked that up
> yet, if anyone can see any inherent issue with that plan I'd like to
> know about it.

Re: [BUG] git remote prune removes local tags, depending on fetch config

[Ævar Arnfjörð Bjarmason]

On Tue, Jan 16 2018, Michael Giuffrida jotted:

> Just to confirm, you're talking about the behavior of removing *all*
> tags that aren't found on the remote? (Not just tags that used to be
> on some remote, but have since been removed from that remote.) So,
> with your proposed workflow, you would never create your own tags
> locally, without pushing them to the remote before running `git fetch`
> -- otherwise they would simply be deleted.
>
> It doesn't seem like a useful feature -- you wouldn't expect `git
> fetch --prune` to remove your local branches that you were developing
> on, right?

Yes, I realize this isn't everyone's cup of tea, but it's very useful in
our in-house use-case for git.

We have repos that have tags pushed to them on every deployment, thus
the cumulative set of tags is in the hundreds of thousands.

There's a pruning script that prunes both branches and tags from the
server (and archives them to an archive repo that has every ref ever).

Then we need to push these deletions back to client checkouts. For
branches that's easy, just set fetch.prune=true. For tags I need to
chase down checkouts and do:

    git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"

It would be easier to just set fetch.{prune,pruneTags}=true in
/etc/gitconfig, this is on e.g. shared development staging boxes where
you might have a manually cloned checkout in your ~.

Of course it'll wipe away any manually created tag you have, *ideally*
we'd have something like the refs/remotes/ namespace for tags, but that
ship has sailed.

But meanwhile tag pruning is an OK compromise, 99% of our users will
never even think to create a tag (and if they created one manually they
couldn't push it), the advanced users that would know what sort of repo
they're editing and can out-out of the behavior by editing their local
config.

> On Mon, Jan 15, 2018 at 4:38 PM, Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> On Mon, Jan 15 2018, Michael Giuffrida jotted:
>>
>>> `git remote prune <name>` should "delete all stale remote-tracking
>>> branches under <name>". I was surprised to discover, after some
>>> troubleshooting, that it also deletes *all* local tags that don't
>>> exist on the remote, if the following refspec is included in the
>>> remote's fetch config:
>>>
>>>     +refs/tags/*:refs/tags/*
>>>
>>> So, if `remote.origin.fetch` is configured to fetch all tags from the
>>> remote, any tags I create locally will be deleted when running `git
>>> remote prune origin`. This is not intuitive [1], nor is is it
>>> explained in the docs [2]. Is this behavior obvious to someone with a
>>> better understanding of Git internals?
>>>
>>> I did find a better way to automatically fetch tags (using tagopt
>>> instead of adding the fetch refspec). However, the refspec doesn't
>>> seem "wrong" in itself; in particular, `git fetch --tags` used to be
>>> considered equivalent to specifying the refspec
>>> "refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec
>>> [3]. So I wouldn't expect it to "break" the behavior of another
>>> command.
>>>
>>> [1] https://stackoverflow.com/q/34687657/1327867
>>> [2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
>>> [3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50
>>
>> These docs are really confusing, but it is working as intended, and
>> really should be re-documented.
>>
>> The `git remote prune` subcommand just ends up piggy-backing on
>> git-fetch, whose behavior is explained here:
>> https://git-scm.com/docs/git-fetch.html#git-fetch---prune
>>
>> It's worked this way since at least v1.8.5.6, maybe at some distant
>> point in the past it only did this for branches when invoked via
>> git-remote as the documentation says.
>>
>> RELATED:
>>
>> I've actually had the reverse problem with this. I want some way to turn
>> this behavior on without explicitly hacking the refspec, so I can do it
>> globally in /etc/gitconfig or in ~/.gitconfig without screwing with the
>> config of each checkout on certain machines.
>>
>> You can set fetch.prune=true, but that only prunes the branches, you
>> need to inject remote.origin.fetch into each checkout, unless I've
>> missed some way of doing this.
>>
>> I wanted to add fetch.pruneTags that would make it as if you had
>> refs/tags/*:refs/tags/* in the fetch spec, but I haven't hacked that up
>> yet, if anyone can see any inherent issue with that plan I'd like to
>> know about it.

[PATCH 00/11] document & test fetch pruning + WIP fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

Michael Giuffrida noted that the git-remote docs were very confusing,
and upthread I said I wanted this shiny related thing in 11/11.

Along the way I fixed up fetch tests & documentation to hopefully be a
lot less confusing.

I think 1-10/11 of this makes sense for inclusion as-is (pending
review etc.), 11/11 is broken currently, but review / comments on it
welcome, particularly the CLI / config interface / docs etc.

The bug causing it not to work is less of a "I can't figure this out"
and more of a "I won't have time again for hacking in the next couple
of days, and wanted to see what people thought", but if someone wants
to see what I'm screwing up there and do my homework for me that's
also most welcome.

Ævar Arnfjörð Bjarmason (11):
  fetch tests: refactor in preparation for testing tag pruning
  fetch tests: arrange arguments for future readability
  fetch tests: add a tag to be deleted to the pruning tests
  fetch tests: double quote a variable for interpolation
  fetch tests: test --prune and refspec interaction
  git fetch doc: add a new section to explain the ins & outs of pruning
  git remote doc: correct dangerous lies about what prune does
  git-fetch & config doc: link to the new PRUNING section
  fetch: don't redundantly null something calloc() gave us
  fetch tests: add scaffolding for the new fetch.pruneTags
  WIP fetch: add a --fetch-prune option and fetch.pruneTags config

 Documentation/config.txt        |  21 ++++++-
 Documentation/fetch-options.txt |  18 +++++-
 Documentation/git-fetch.txt     |  49 ++++++++++++++++
 Documentation/git-remote.txt    |  14 +++--
 builtin/fetch.c                 |  38 ++++++++++--
 remote.c                        |   2 +
 remote.h                        |   1 +
 t/t5510-fetch.sh                | 125 +++++++++++++++++++++++++++-------------
 8 files changed, 216 insertions(+), 52 deletions(-)

-- 
2.15.1.424.g9478a66081

[PATCH v2 00/12] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

Now v2 and fully non-RFC, changes:

Ævar Arnfjörð Bjarmason (12):
  fetch tests: refactor in preparation for testing tag pruning
  fetch tests: arrange arguments for future readability
  fetch tests: add a tag to be deleted to the pruning tests
  fetch tests: double quote a variable for interpolation
  fetch tests: test --prune and refspec interaction

No changes.

  git fetch doc: add a new section to explain the ins & outs of pruning

Grammar etc. fixes from Eric. Thanks!

  git remote doc: correct dangerous lies about what prune does
  git-fetch & config doc: link to the new PRUNING section

No changes.

  fetch: don't redundantly NULL something calloc() gave us

Minor rephrasing of the commit message.

  fetch: stop accessing "remote" variable indirectly

NEW: Amends some existing confusing code, whose pattern will be used
by 12/12.

  fetch tests: add scaffolding for the new fetch.pruneTags

I screwed up positional arguments in the test description, fixed.

  fetch: add a --fetch-prune option and fetch.pruneTags config

Now actually works, and with a very different implementation which
involves making the previously private add_fetch_refspec() function in
remote.c part of the API.

 Documentation/config.txt        |  20 +++++-
 Documentation/fetch-options.txt |  18 ++++-
 Documentation/git-fetch.txt     |  73 +++++++++++++++++++
 Documentation/git-remote.txt    |  14 ++--
 builtin/fetch.c                 |  37 +++++++++-
 remote.c                        |   5 +-
 remote.h                        |   3 +
 t/t5510-fetch.sh                | 154 +++++++++++++++++++++++++++++-----------
 8 files changed, 272 insertions(+), 52 deletions(-)

-- 
2.15.1.424.g9478a66081

[PATCH v3 00/11] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

Addresses Junio's comments on v2 + more fixes. The 

Ævar Arnfjörð Bjarmason (11):
  fetch: don't redundantly NULL something calloc() gave us
  fetch: stop accessing "remote" variable indirectly

Moved these patches to the beginning. No changes except a note to the
commit message saying how these trivial changes relate (or not) to
subsequent changes.

  fetch tests: refactor in preparation for testing tag pruning

No changes.

  fetch tests: re-arrange arguments for future readability

s/arrange/re-arrange/ in commit message subject.

  fetch tests: add a tag to be deleted to the pruning tests

No changes.

  fetch tests: test --prune and refspec interaction

I'm now just skipping quoting things like +refs/... on the
command-line, which as grepping the rest of the test suite shows is
fine, this eliminated the need for "fetch tests: double quote a
variable for interpolation" so I've ejected it.

  git fetch doc: add a new section to explain the ins & outs of pruning

No changes.

  git remote doc: correct dangerous lies about what prune does

Rewording of doc per Junio's suggestion.


  git-fetch & config doc: link to the new PRUNING section

No changes.

  fetch tests: add scaffolding for the new fetch.pruneTags

No changes except continuing remove refspec quoting changes noted
above & using +refs/tags/... instead of refs/tags/... for consistency
with the last patch...

  fetch: add a --fetch-prune option and fetch.pruneTags config

We now consistently use a + prefixed refspec for tag pruning, even
though it yields the same result. See the discussion in
87tvvdh2vh.fsf@evledraar.gmail.com, I think it's less confusing.

Fix regex in --replace-all command in the commit message.

Rewording to address Junio's comments.

The short help for --prune-tags is now:

    -           N_("prune local tags not found on remote")),
    +           N_("prune local tags no longer on remote and clobber changed tags")),

Add --prune-tags to the bash completion.

 Documentation/config.txt               |  20 ++++-
 Documentation/fetch-options.txt        |  18 +++-
 Documentation/git-fetch.txt            |  76 +++++++++++++++++
 Documentation/git-remote.txt           |  14 +--
 builtin/fetch.c                        |  37 +++++++-
 contrib/completion/git-completion.bash |   2 +-
 remote.c                               |   5 +-
 remote.h                               |   3 +
 t/t5510-fetch.sh                       | 152 ++++++++++++++++++++++++---------
 9 files changed, 275 insertions(+), 52 deletions(-)

-- 
2.15.1.424.g9478a66081

Re: [PATCH v3 00/11] document & test fetch pruning & add fetch.pruneTags

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

>  Documentation/config.txt               |  20 ++++-
>  Documentation/fetch-options.txt        |  18 +++-
>  Documentation/git-fetch.txt            |  76 +++++++++++++++++
>  Documentation/git-remote.txt           |  14 +--
>  builtin/fetch.c                        |  37 +++++++-
>  contrib/completion/git-completion.bash |   2 +-
>  remote.c                               |   5 +-
>  remote.h                               |   3 +
>  t/t5510-fetch.sh                       | 152 ++++++++++++++++++++++++---------
>  9 files changed, 275 insertions(+), 52 deletions(-)

I've queued this round but saw a few minor conflicts with another
topi in flight, so please double check the result when I push the
result out shortly.

Thanks.

Re: [PATCH v3 00/11] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

On Wed, Jan 24 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>>  Documentation/config.txt               |  20 ++++-
>>  Documentation/fetch-options.txt        |  18 +++-
>>  Documentation/git-fetch.txt            |  76 +++++++++++++++++
>>  Documentation/git-remote.txt           |  14 +--
>>  builtin/fetch.c                        |  37 +++++++-
>>  contrib/completion/git-completion.bash |   2 +-
>>  remote.c                               |   5 +-
>>  remote.h                               |   3 +
>>  t/t5510-fetch.sh                       | 152 ++++++++++++++++++++++++---------
>>  9 files changed, 275 insertions(+), 52 deletions(-)
>
> I've queued this round but saw a few minor conflicts with another
> topi in flight, so please double check the result when I push the
> result out shortly.

Yikes. fetch.c is busy these days. Looked it over, looks good to me. In
particular the potential logic error introduced by mis-merging "fetch:
refactor calculation of remote list" has a test which would fail.

Re: [PATCH v3 00/11] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

On Tue, Jan 23 2018, Ævar Arnfjörð Bjarmason jotted:

> I'm now just skipping quoting things like +refs/... on the
> command-line, which as grepping the rest of the test suite shows is
> fine, this eliminated the need for "fetch tests: double quote a
> variable for interpolation" so I've ejected it.

There's a segfault bug in 11/11, which wasn't found because the test
suite doesn't test `git fetch <url>` just `git fetch <named>` and this
is handled differently.

I'll send a fix soon, but don't merge this down from pu for now.

In order to test for that I brought that cmdline quoting patch back, I
can't find a better way to do that, and in addition I have this similar
WIP patch:

    diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
    index 2f5bd966be..8fe4f3c13b 100755
    --- a/t/t5510-fetch.sh
    +++ b/t/t5510-fetch.sh
    @@ -549,13 +549,39 @@ set_config_tristate () {
     }

     test_configured_prune () {
    +	test_configured_prune_guts "$@" "name"
    +	test_configured_prune_guts "$@" "link"
    +}
    +
    +test_configured_prune_guts () {
     	fetch_prune=$1
     	remote_origin_prune=$2
     	expected_branch=$3
     	expected_tag=$4
     	cmdline=$5
    -
    -	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
    +	mode=$6
    +
    +	if ! test -e prune-guts-setup
    +	then
    +		test_expect_success 'prune_guts setup' '
    +			git -C one config remote.origin.url >one.remote-url &&
    +			git -C one config remote.origin.fetch >one.remote-fetch &&
    +			touch prune-guts-setup
    +		'
    +	fi
    +
    +	if test "$mode" = 'link'
    +	then
    +		remote_url="file://$(cat one.remote-url)"
    +		remote_fetch="$(cat one.remote-fetch)"
    +		cmdline_setup="\"$remote_url\" \"$remote_fetch\""
    +		if test "$cmdline" != ""
    +		then
    +			cmdline=$(printf "%s" "$cmdline" | sed -e 's! origin! "'"$remote_url"'"!g')
    +		fi
    +	fi
    +
    +	test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
     		# make sure a newbranch is there in . and also in one
     		git branch -f newbranch &&
     		git tag -f newtag &&
    @@ -563,7 +589,7 @@ test_configured_prune () {
     			cd one &&
     			test_unconfig fetch.prune &&
     			test_unconfig remote.origin.prune &&
    -			git fetch &&
    +			git fetch '"$cmdline_setup"' &&
     			git rev-parse --verify refs/remotes/origin/newbranch &&
     			git rev-parse --verify refs/tags/newtag
     		) &&

It'll be amended a bit more, but the general idea is there, because of
how this whole quoting mess looks like I have to resort to the above
hack outside of the test setup.

[PATCH v2 00/17] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

As noted in my 87h8quytmq.fsf@evledraar.gmail.com there was a bug I
noticed in v3 where it would segfault on some git-fetch invocations,
but there were not tests anywhere that caught that.

So in addition to fixing that issue, this fleshens out the testing
being set up as part of this series so we'll test those sorts of
invocations. It would segfault on some `git fetch <url>`, not `git
fetch <name>`.

Ævar Arnfjörð Bjarmason (17):
  fetch: don't redundantly NULL something calloc() gave us

Rephrased commit message.

  fetch: trivially refactor assignment to ref_nr

New, makes a subsequent change smaller.

  fetch: stop accessing "remote" variable indirectly

Typo fix in commit message noted by Junio.

  remote: add a macro for "refs/tags/*:refs/tags/*"

New, makes a subsequent change smaller.

  fetch tests: refactor in preparation for testing tag pruning
  fetch tests: re-arrange arguments for future readability
  fetch tests: add a tag to be deleted to the pruning tests

No changes.

  fetch tests: test --prune and refspec interaction

Changed +refs/tags/*:refs/tags/ to refs/tags/*:refs/tags/. No
functional difference, since git doesn't care. Just to be consistent
with the macro added earlier & doing the same in commit messages &
tests later in the series.

  fetch tests: double quote a variable for interpolation

Now back from an earlier version, needed for a later change.

  fetch tests: expand case/esac for later change

New, makes the next patch smaller / easier to review.

  fetch tests: fetch <url> <spec> as well as fetch [<remote>]

For all `git fetch <name>` we now run another version of the test
where we test an equivalent `git fetch <url>`. This sort of exhaustive
testing was missing in our whole test suite, and would have caught the
segfault in v3.

  git fetch doc: add a new section to explain the ins & outs of pruning
  git remote doc: correct dangerous lies about what prune does
  git-fetch & config doc: link to the new PRUNING section

No changes except omitting the "+" in front of refs/tags/[...] as
noted above.

  fetch tests: add scaffolding for the new fetch.pruneTags

Ditto "+" change + minor changes carried over from previous patches.

  fetch: add a --fetch-prune option and fetch.pruneTags config

The bug in v3 was that the remote->fetch variable needs to chaned in
lockstep with remote->fetch_refspec, but only the latter was
changed. Codepaths that fetched by URL would under --prune-tags expect
as many items in both, and segfault on the access to remote->fetch.

As explained in the amended commit message the API is not amenable to
ALLOC_GROW, so there's now a add_prune_tags_to_fetch_refspec()
function in remote.c which adds the new element to remote->fetch via
xrealloc() + memcpy().

Careful review of that most welcome.

There's lots more tests that catch the case where it segfaulted.

  fetch: make the --fetch-prune work with <url>

The previous patch was changed to document that this wouldn't work:

    git fetch <url of origin> --prune --prune-tag

This makes it work, at the cost of some complexity in fetch_one(). I
think it makes sense to keep this, I just wanted to split it off from
the previous patch to clearly show the hoops we need to jump through
for that one case.

 Documentation/config.txt               |  20 ++-
 Documentation/fetch-options.txt        |  17 ++-
 Documentation/git-fetch.txt            |  87 ++++++++++++
 Documentation/git-remote.txt           |  14 +-
 builtin/fetch.c                        |  54 ++++++--
 contrib/completion/git-completion.bash |   2 +-
 remote.c                               |  15 ++
 remote.h                               |   5 +
 t/t5510-fetch.sh                       | 242 +++++++++++++++++++++++++++------
 9 files changed, 395 insertions(+), 61 deletions(-)

-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 00/17] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 08 2018, Ævar Arnfjörð Bjarmason jotted:

> As noted in my 87h8quytmq.fsf@evledraar.gmail.com there was a bug I
> noticed in v3 where it would segfault on some git-fetch invocations,
> but there were not tests anywhere that caught that.

...and of course this whole submission this time around should be v4,
not v2, but I didn't notice that I didn't adjust the subject prefix
before sending. Junio: Hopefully you can pick it up anyway without too
much trouble, sorry.

FWIW I've deployed this to production @ work to some tens of k of
machines (low "k" of which have users using git) without any issues.

Re: [PATCH v2 00/17] document & test fetch pruning & add fetch.pruneTags

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Thu, Feb 08 2018, Ævar Arnfjörð Bjarmason jotted:
>
>> As noted in my 87h8quytmq.fsf@evledraar.gmail.com there was a bug I
>> noticed in v3 where it would segfault on some git-fetch invocations,
>> but there were not tests anywhere that caught that.
>
> ...and of course this whole submission this time around should be v4,
> not v2, but I didn't notice that I didn't adjust the subject prefix
> before sending. Junio: Hopefully you can pick it up anyway without too
> much trouble, sorry.
>
> FWIW I've deployed this to production @ work to some tens of k of
> machines (low "k" of which have users using git) without any issues.

Will replace.  I found that the resolution of conflicts necessary
with the jh/partial-clone topic is a bit different from the previous
version due to addition of an extra parameter to fetch_one(), and I
think I didn't botch it, but please double check when I push the
results out in a few hours.

Thanks.

Re: [PATCH v2 00/17] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 08 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Thu, Feb 08 2018, Ævar Arnfjörð Bjarmason jotted:
>>
>>> As noted in my 87h8quytmq.fsf@evledraar.gmail.com there was a bug I
>>> noticed in v3 where it would segfault on some git-fetch invocations,
>>> but there were not tests anywhere that caught that.
>>
>> ...and of course this whole submission this time around should be v4,
>> not v2, but I didn't notice that I didn't adjust the subject prefix
>> before sending. Junio: Hopefully you can pick it up anyway without too
>> much trouble, sorry.
>>
>> FWIW I've deployed this to production @ work to some tens of k of
>> machines (low "k" of which have users using git) without any issues.
>
> Will replace.  I found that the resolution of conflicts necessary
> with the jh/partial-clone topic is a bit different from the previous
> version due to addition of an extra parameter to fetch_one(), and I
> think I didn't botch it, but please double check when I push the
> results out in a few hours.

Thanks. FWIW I've looked over that resolution in pu and it LGTM.

[PATCH v2 01/17] fetch: don't redundantly NULL something calloc() gave us

[Ævar Arnfjörð Bjarmason]

Stop redundantly NULL-ing the last element of the refs structure,
which was retrieved via calloc(), and is thus guaranteed to be
pre-NULL'd.

This code dates back to b888d61c83 ("Make fetch a builtin",
2007-09-10), where wasn't any reason to do this back then either, it's
just boilerplate left over from when git-fetch was initially
introduced.

The motivation for this change was to make a subsequent change which
would also modify the refs variable smaller, since it won't have to
copy this redundant "NULL the last + 1 item" pattern.

We may not end up keeping that change, but as this pattern is still
pointless, so let's fix it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7bbcd26faf..b34665db9e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			} else
 				refs[j++] = argv[i];
 		}
-		refs[j] = NULL;
 		ref_nr = j;
 	}
 
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 01/17] fetch: don't redundantly NULL something calloc() gave us

[Eric Sunshine]

On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Stop redundantly NULL-ing the last element of the refs structure,
> which was retrieved via calloc(), and is thus guaranteed to be
> pre-NULL'd.
> [...]
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> @@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
>                         } else
>                                 refs[j++] = argv[i];
>                 }
> -               refs[j] = NULL;
>                 ref_nr = j;
>         }

This is purely subjective, and I neglected to mention it as early as
v1, but I find that this change hurts readability. Specifically, as
I'm scanning or reading code, explicit termination conditions, like
this NULL assignment, are things I'm expecting to see; they're part of
the idiom of the language. When they're missing, I have to stop, go
back, and study the code more carefully to see if the "missing bit" is
a bug or is intentional. And, it's easy to misread xcalloc() as
xmalloc(), meaning that it's likely that one studying the code would
conclude that the missing NULL assignment is a bug.

If anything, if this patch wants to eliminate redundancy, I'd expect
it to change xcalloc() to xmalloc() and keep the NULL assignment, thus
leaving the idiom intact.

Re: [PATCH v2 01/17] fetch: don't redundantly NULL something calloc() gave us

[Ævar Arnfjörð Bjarmason]

On Fri, Feb 09 2018, Eric Sunshine jotted:

> On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> Stop redundantly NULL-ing the last element of the refs structure,
>> which was retrieved via calloc(), and is thus guaranteed to be
>> pre-NULL'd.
>> [...]
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>> diff --git a/builtin/fetch.c b/builtin/fetch.c
>> @@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
>>                         } else
>>                                 refs[j++] = argv[i];
>>                 }
>> -               refs[j] = NULL;
>>                 ref_nr = j;
>>         }
>
> This is purely subjective, and I neglected to mention it as early as
> v1, but I find that this change hurts readability. Specifically, as
> I'm scanning or reading code, explicit termination conditions, like
> this NULL assignment, are things I'm expecting to see; they're part of
> the idiom of the language. When they're missing, I have to stop, go
> back, and study the code more carefully to see if the "missing bit" is
> a bug or is intentional. And, it's easy to misread xcalloc() as
> xmalloc(), meaning that it's likely that one studying the code would
> conclude that the missing NULL assignment is a bug.
>
> If anything, if this patch wants to eliminate redundancy, I'd expect
> it to change xcalloc() to xmalloc() and keep the NULL assignment, thus
> leaving the idiom intact.

Thanks for all your review, really appreciate it.

I'm going to keep this as-is, reasons:

 * With calloc it's easier to look at the values in a debugger, you get
   NULLs instead of some random garbage for e.g. the ref src/dst. It
   makes it clear it's unset / the tail value.

 * Ditto fewer things to step through / inpect in a debugger. E.g. I
   have a dump of variables before/after in the debugger, with
   assignments like this it's just adding verbosity & something to
   eyeball for something that's never going to change.

 * If there's a bug in the code using calloc is likely to reveal it
   sooner, since you'll be deref-ing NULL instead of some stray
   (possibly still valid) pointer you got from malloc.

 * It looks more in line with established idioms in the codebase. I
   wasn't able to find other cases where we were double-NULL ing
   calloc'd data, but rather stuff like this:

       git grep -W '\bpattern\b' -- '*/ls-remote.c'

[PATCH v2 02/17] fetch: trivially refactor assignment to ref_nr

[Ævar Arnfjörð Bjarmason]

Trivially refactor an assignment to make a subsequent patch
smaller. The "ref_nr" variable is initialized to 0 earlier, just as
"j" is, and "j" is only incremented in that loop, so this change isn't
a logic error.

This change makes a subsequent change which splits the incrementing of
"ref_nr" into two blocks.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index b34665db9e..72085e30b9 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1301,8 +1301,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 						    argv[i], argv[i]);
 			} else
 				refs[j++] = argv[i];
+			ref_nr++;
 		}
-		ref_nr = j;
 	}
 
 	sigchain_push_common(unlock_pack_on_signal);
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 02/17] fetch: trivially refactor assignment to ref_nr

[Eric Sunshine]

On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Trivially refactor an assignment to make a subsequent patch
> smaller. The "ref_nr" variable is initialized to 0 earlier, just as
> "j" is, and "j" is only incremented in that loop, so this change isn't
> a logic error.
>
> This change makes a subsequent change which splits the incrementing of
> "ref_nr" into two blocks.

"This change _simplifies_ a subsequent..."

or

"This change make a subsequent change _easier_..."

> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

[PATCH v2 03/17] fetch: stop accessing "remote" variable indirectly

[Ævar Arnfjörð Bjarmason]

Access the "remote" variable passed to the fetch_one() directly rather
than through the gtransport wrapper struct constructed in this
function for other purposes.

This makes the code more readable, as it's now obvious that the remote
struct doesn't somehow get munged by the prepare_transport() function
above, which takes the "remote" struct as an argument and constructs
the "gtransport" struct, containing among other things the "remote"
struct.

A subsequent change will copy this pattern to access a new
remote->prune_tags field, but without the use of the gtransport
variable. It's useful once that change lands to see that the two
pieces of code behave exactly the same.

This pattern of accessing the container struct was added in
737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this
code was initially introduced.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 72085e30b9..a7705bc150 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 
 	if (prune < 0) {
 		/* no command line request */
-		if (0 <= gtransport->remote->prune)
-			prune = gtransport->remote->prune;
+		if (0 <= remote->prune)
+			prune = remote->prune;
 		else if (0 <= fetch_prune_config)
 			prune = fetch_prune_config;
 		else
-- 
2.15.1.424.g9478a66081

[PATCH v2 04/17] remote: add a macro for "refs/tags/*:refs/tags/*"

[Ævar Arnfjörð Bjarmason]

Add a macro with the refspec string "refs/tags/*:refs/tags/*". There's
been a pre-defined struct version of this since e0aaa29ff3 ("Have a
constant extern refspec for "--tags"", 2008-04-17), but nothing that
could be passed to e.g. add_fetch_refspec().

This will be used in subsequent commits to avoid hardcoding this
string in multiple places.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 remote.c | 1 +
 remote.h | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/remote.c b/remote.c
index 4e93753e19..356c123e3e 100644
--- a/remote.c
+++ b/remote.c
@@ -22,6 +22,7 @@ static struct refspec s_tag_refspec = {
 	"refs/tags/*"
 };
 
+/* See TAG_REFSPEC for the string version */
 const struct refspec *tag_refspec = &s_tag_refspec;
 
 struct counted_string {
diff --git a/remote.h b/remote.h
index 1f6611be21..80fea6dd11 100644
--- a/remote.h
+++ b/remote.h
@@ -297,4 +297,6 @@ extern int parseopt_push_cas_option(const struct option *, const char *arg, int
 extern int is_empty_cas(const struct push_cas_option *);
 void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *);
 
+#define TAG_REFSPEC "refs/tags/*:refs/tags/*"
+
 #endif
-- 
2.15.1.424.g9478a66081

[PATCH v2 05/17] fetch tests: refactor in preparation for testing tag pruning

[Ævar Arnfjörð Bjarmason]

In a subsequent commit this function will learn to test for tag
pruning, prepare for that by making space for more variables, and
making it clear that "expected" here refers to branches.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 668c54be41..11da97f9b7 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -549,9 +549,12 @@ set_config_tristate () {
 }
 
 test_configured_prune () {
-	fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4
+	fetch_prune=$1
+	remote_origin_prune=$2
+	cmdline=$3
+	expected_branch=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -572,7 +575,7 @@ test_configured_prune () {
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
 			git fetch $cmdline &&
-			case "$expected" in
+			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
 				;;
-- 
2.15.1.424.g9478a66081

[PATCH v2 06/17] fetch tests: re-arrange arguments for future readability

[Ævar Arnfjörð Bjarmason]

Re-arrange the arguments to the test_configured_prune() function used
in this test to pass the arguments to --fetch last. A subsequent
change will test for more elaborate fetch arguments, including long
refspecs. It'll be more readable to be able to wrap those on a new
line of their own.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 44 insertions(+), 38 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 11da97f9b7..ab8b25344d 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,10 +551,10 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	cmdline=$3
-	expected_branch=$4
+	expected_branch=$3
+	cmdline=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -587,41 +587,47 @@ test_configured_prune () {
 	'
 }
 
-test_configured_prune unset unset ""		kept
-test_configured_prune unset unset "--no-prune"	kept
-test_configured_prune unset unset "--prune"	pruned
-
-test_configured_prune false unset ""		kept
-test_configured_prune false unset "--no-prune"	kept
-test_configured_prune false unset "--prune"	pruned
-
-test_configured_prune true  unset ""		pruned
-test_configured_prune true  unset "--prune"	pruned
-test_configured_prune true  unset "--no-prune"	kept
-
-test_configured_prune unset false ""		kept
-test_configured_prune unset false "--no-prune"	kept
-test_configured_prune unset false "--prune"	pruned
-
-test_configured_prune false false ""		kept
-test_configured_prune false false "--no-prune"	kept
-test_configured_prune false false "--prune"	pruned
-
-test_configured_prune true  false ""		kept
-test_configured_prune true  false "--prune"	pruned
-test_configured_prune true  false "--no-prune"	kept
-
-test_configured_prune unset true  ""		pruned
-test_configured_prune unset true  "--no-prune"	kept
-test_configured_prune unset true  "--prune"	pruned
-
-test_configured_prune false true  ""		pruned
-test_configured_prune false true  "--no-prune"	kept
-test_configured_prune false true  "--prune"	pruned
-
-test_configured_prune true  true  ""		pruned
-test_configured_prune true  true  "--prune"	pruned
-test_configured_prune true  true  "--no-prune"	kept
+# $1 config: fetch.prune
+# $2 config: remote.<name>.prune
+# $3 expect: branch to be pruned?
+# $4 git-fetch $cmdline:
+#
+#                     $1    $2    $3     $4
+test_configured_prune unset unset kept   ""
+test_configured_prune unset unset kept   "--no-prune"
+test_configured_prune unset unset pruned "--prune"
+
+test_configured_prune false unset kept   ""
+test_configured_prune false unset kept   "--no-prune"
+test_configured_prune false unset pruned "--prune"
+
+test_configured_prune true  unset pruned ""
+test_configured_prune true  unset pruned "--prune"
+test_configured_prune true  unset kept   "--no-prune"
+
+test_configured_prune unset false kept   ""
+test_configured_prune unset false kept   "--no-prune"
+test_configured_prune unset false pruned "--prune"
+
+test_configured_prune false false kept   ""
+test_configured_prune false false kept   "--no-prune"
+test_configured_prune false false pruned "--prune"
+
+test_configured_prune true  false kept   ""
+test_configured_prune true  false pruned "--prune"
+test_configured_prune true  false kept   "--no-prune"
+
+test_configured_prune unset true  pruned ""
+test_configured_prune unset true  kept   "--no-prune"
+test_configured_prune unset true  pruned "--prune"
+
+test_configured_prune false true  pruned ""
+test_configured_prune false true  kept   "--no-prune"
+test_configured_prune false true  pruned "--prune"
+
+test_configured_prune true  true  pruned ""
+test_configured_prune true  true  pruned "--prune"
+test_configured_prune true  true  kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v2 07/17] fetch tests: add a tag to be deleted to the pruning tests

[Ævar Arnfjörð Bjarmason]

Add a tag to be deleted to the fetch --prune tests. The tag is always
kept for now, which is the expected behavior, but now I can add a test
for tag pruning in a later commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 53 insertions(+), 40 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index ab8b25344d..fad65bd885 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -552,21 +552,25 @@ test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
 	expected_branch=$3
-	cmdline=$4
+	expected_tag=$4
+	cmdline=$5
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
+		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
 			test_unconfig remote.origin.prune &&
 			git fetch &&
-			git rev-parse --verify refs/remotes/origin/newbranch
+			git rev-parse --verify refs/remotes/origin/newbranch &&
+			git rev-parse --verify refs/tags/newtag
 		) &&
 
 		# now remove it
 		git branch -d newbranch &&
+		git tag -d newtag &&
 
 		# then test
 		(
@@ -582,6 +586,14 @@ test_configured_prune () {
 			kept)
 				git rev-parse --verify refs/remotes/origin/newbranch
 				;;
+			esac &&
+			case "$expected_tag" in
+			pruned)
+				test_must_fail git rev-parse --verify refs/tags/newtag
+				;;
+			kept)
+				git rev-parse --verify refs/tags/newtag
+				;;
 			esac
 		)
 	'
@@ -590,44 +602,45 @@ test_configured_prune () {
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
 # $3 expect: branch to be pruned?
-# $4 git-fetch $cmdline:
+# $4 expect: tag to be pruned?
+# $5 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4
-test_configured_prune unset unset kept   ""
-test_configured_prune unset unset kept   "--no-prune"
-test_configured_prune unset unset pruned "--prune"
-
-test_configured_prune false unset kept   ""
-test_configured_prune false unset kept   "--no-prune"
-test_configured_prune false unset pruned "--prune"
-
-test_configured_prune true  unset pruned ""
-test_configured_prune true  unset pruned "--prune"
-test_configured_prune true  unset kept   "--no-prune"
-
-test_configured_prune unset false kept   ""
-test_configured_prune unset false kept   "--no-prune"
-test_configured_prune unset false pruned "--prune"
-
-test_configured_prune false false kept   ""
-test_configured_prune false false kept   "--no-prune"
-test_configured_prune false false pruned "--prune"
-
-test_configured_prune true  false kept   ""
-test_configured_prune true  false pruned "--prune"
-test_configured_prune true  false kept   "--no-prune"
-
-test_configured_prune unset true  pruned ""
-test_configured_prune unset true  kept   "--no-prune"
-test_configured_prune unset true  pruned "--prune"
-
-test_configured_prune false true  pruned ""
-test_configured_prune false true  kept   "--no-prune"
-test_configured_prune false true  pruned "--prune"
-
-test_configured_prune true  true  pruned ""
-test_configured_prune true  true  pruned "--prune"
-test_configured_prune true  true  kept   "--no-prune"
+#                     $1    $2    $3     $4     $5
+test_configured_prune unset unset kept   kept   ""
+test_configured_prune unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset pruned kept   "--prune"
+
+test_configured_prune false unset kept   kept   ""
+test_configured_prune false unset kept   kept   "--no-prune"
+test_configured_prune false unset pruned kept   "--prune"
+
+test_configured_prune true  unset pruned kept   ""
+test_configured_prune true  unset pruned kept   "--prune"
+test_configured_prune true  unset kept   kept   "--no-prune"
+
+test_configured_prune unset false kept   kept   ""
+test_configured_prune unset false kept   kept   "--no-prune"
+test_configured_prune unset false pruned kept   "--prune"
+
+test_configured_prune false false kept   kept   ""
+test_configured_prune false false kept   kept   "--no-prune"
+test_configured_prune false false pruned kept   "--prune"
+
+test_configured_prune true  false kept   kept   ""
+test_configured_prune true  false pruned kept   "--prune"
+test_configured_prune true  false kept   kept   "--no-prune"
+
+test_configured_prune unset true  pruned kept   ""
+test_configured_prune unset true  kept   kept   "--no-prune"
+test_configured_prune unset true  pruned kept   "--prune"
+
+test_configured_prune false true  pruned kept   ""
+test_configured_prune false true  kept   kept   "--no-prune"
+test_configured_prune false true  pruned kept   "--prune"
+
+test_configured_prune true  true  pruned kept   ""
+test_configured_prune true  true  pruned kept   "--prune"
+test_configured_prune true  true  kept   kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v2 08/17] fetch tests: test --prune and refspec interaction

[Ævar Arnfjörð Bjarmason]

Add a test for the interaction between explicitly provided refspecs
and fetch.prune.

There's no point in adding this boilerplate to every combination of
unset/false/true, it's instructive and sufficient to show that no
matter if the variable is unset, false or true the refspec on the
command-line overrides any configuration variable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index fad65bd885..dacdb8759c 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -609,6 +609,10 @@ test_configured_prune () {
 test_configured_prune unset unset kept   kept   ""
 test_configured_prune unset unset kept   kept   "--no-prune"
 test_configured_prune unset unset pruned kept   "--prune"
+test_configured_prune unset unset kept   pruned \
+	"--prune origin refs/tags/*:refs/tags/*"
+test_configured_prune unset unset pruned pruned \
+	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_configured_prune false unset kept   kept   ""
 test_configured_prune false unset kept   kept   "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept   "--prune"
 test_configured_prune false false kept   kept   ""
 test_configured_prune false false kept   kept   "--no-prune"
 test_configured_prune false false pruned kept   "--prune"
+test_configured_prune false false kept   pruned \
+	"--prune origin refs/tags/*:refs/tags/*"
+test_configured_prune false false pruned pruned \
+	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_configured_prune true  false kept   kept   ""
 test_configured_prune true  false pruned kept   "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true  pruned kept   "--prune"
 test_configured_prune true  true  pruned kept   ""
 test_configured_prune true  true  pruned kept   "--prune"
 test_configured_prune true  true  kept   kept   "--no-prune"
+test_configured_prune true  true  kept   pruned \
+	"--prune origin refs/tags/*:refs/tags/*"
+test_configured_prune true  true  pruned pruned \
+	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v2 09/17] fetch tests: double quote a variable for interpolation

[Ævar Arnfjörð Bjarmason]

If the $cmdline variable contains arguments with spaces they won't be
interpolated correctly, since the body of the test is single quoted,
and because test-lib.sh does its own eval().

This will be used in a subsequent commit to pass arguments that need
to be quoted to git-fetch, i.e. a file:// path to fetch, which will
have a space in it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dacdb8759c..88d38e0819 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -578,7 +578,7 @@ test_configured_prune () {
 			set_config_tristate fetch.prune $fetch_prune &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
-			git fetch $cmdline &&
+			git fetch '"$cmdline"' &&
 			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
-- 
2.15.1.424.g9478a66081

[PATCH v2 10/17] fetch tests: expand case/esac for later change

[Ævar Arnfjörð Bjarmason]

Expand a compact case/esac statement for a later change that'll add
more logic to the body of the "*" case. This is a whitespace-only
change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 88d38e0819..dfc749f576 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -543,8 +543,12 @@ test_expect_success "should be able to fetch with duplicate refspecs" '
 set_config_tristate () {
 	# var=$1 val=$2
 	case "$2" in
-	unset)  test_unconfig "$1" ;;
-	*)	git config "$1" "$2" ;;
+	unset)
+		test_unconfig "$1"
+		;;
+	*)
+		git config "$1" "$2"
+		;;
 	esac
 }
 
-- 
2.15.1.424.g9478a66081

[PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Ævar Arnfjörð Bjarmason]

When a remote URL is supplied on the command-line the internals of the
fetch are different, in particular the code in get_ref_map(). An
earlier version of the subsequent fetch.pruneTags patch hid a segfault
because the difference wasn't tested for.

Now all the tests are run as both of the variants of:

    git fetch
    git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)

I'm using -c because while the [fetch] config just set by
set_config_tristate will be picked up, the remote.origin.* config
won't override it as intended.

Work around that and turn this into a purely command-line test by
always setting the variables on the command-line, and translate any
setting of remote.origin.X into fetch.X.

The reason for choosing the names "name" and "link" as opposed to
e.g. "named" and "url" is because they're the same length, which makes
the test output easier to read as it will be aligned.

Due to shellscript quoting madness it's not worthwhile to do all of
this within a test_expect_success, but do the parts that can easily be
done there, including the one-time setting of variables that don't
change between runs to be used by subsequent runs in the 'prune_type
setup' test.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 47 +++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 4 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dfc749f576..73ba83454f 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -548,18 +548,52 @@ set_config_tristate () {
 		;;
 	*)
 		git config "$1" "$2"
+		key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
+		git_fetch_c="$git_fetch_c -c $key=$2"
 		;;
 	esac
 }
 
 test_configured_prune () {
+	test_configured_prune_type "$@" "name"
+	test_configured_prune_type "$@" "link"
+}
+
+test_configured_prune_type () {
 	fetch_prune=$1
 	remote_origin_prune=$2
 	expected_branch=$3
 	expected_tag=$4
 	cmdline=$5
-
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+	mode=$6
+
+	if ! test -e prune-type-setup-done
+	then
+		test_expect_success 'prune_type setup' '
+			git -C one config remote.origin.url >one.remote-url &&
+			git -C one config remote.origin.fetch >one.remote-fetch &&
+			remote_url="file://$(cat one.remote-url)" &&
+			remote_fetch="$(cat one.remote-fetch)" &&
+			cmdline_setup="\"$remote_url\" \"$remote_fetch\"" &&
+			touch prune-type-setup-done
+		'
+	fi
+
+	if test "$mode" = 'link'
+	then
+		new_cmdline=""
+
+		if test "$cmdline" = ""
+		then
+			new_cmdline=$cmdline_setup
+		else
+			new_cmdline=$(printf "%s" "$cmdline" | perl -pe 's[origin(?!/)]["'"$remote_url"'"]g')
+		fi
+
+		cmdline="$new_cmdline"
+	fi
+
+	test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		git tag -f newtag &&
@@ -567,7 +601,7 @@ test_configured_prune () {
 			cd one &&
 			test_unconfig fetch.prune &&
 			test_unconfig remote.origin.prune &&
-			git fetch &&
+			git fetch '"$cmdline_setup"' &&
 			git rev-parse --verify refs/remotes/origin/newbranch &&
 			git rev-parse --verify refs/tags/newtag
 		) &&
@@ -579,10 +613,15 @@ test_configured_prune () {
 		# then test
 		(
 			cd one &&
+			git_fetch_c="" &&
 			set_config_tristate fetch.prune $fetch_prune &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
-			git fetch '"$cmdline"' &&
+			if test "$mode" != "link"
+			then
+				git_fetch_c=""
+			fi &&
+			git$git_fetch_c fetch '"$cmdline"' &&
 			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Eric Sunshine]

On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> When a remote URL is supplied on the command-line the internals of the
> fetch are different, in particular the code in get_ref_map(). An
> earlier version of the subsequent fetch.pruneTags patch hid a segfault
> because the difference wasn't tested for.
>
> Now all the tests are run as both of the variants of:
>
>     git fetch
>     git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)
>
> I'm using -c because while the [fetch] config just set by
> set_config_tristate will be picked up, the remote.origin.* config
> won't override it as intended.
>
> Work around that and turn this into a purely command-line test by
> always setting the variables on the command-line, and translate any
> setting of remote.origin.X into fetch.X.
> [...]
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
> @@ -548,18 +548,52 @@ set_config_tristate () {
>         *)
>                 git config "$1" "$2"
> +               key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')

Faster (thus more Windows-friendly) assuming that $1 always starts
with "remote.origin":

    key=fetch${u#remote.origin}

> +               git_fetch_c="$git_fetch_c -c $key=$2"
>                 ;;
>         esac
>  }
>
> +test_configured_prune_type () {
>         fetch_prune=$1
>         remote_origin_prune=$2
>         expected_branch=$3
>         expected_tag=$4
>         cmdline=$5
> -
> -       test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
> +       mode=$6
> +
> +       if ! test -e prune-type-setup-done
> +       then
> +               test_expect_success 'prune_type setup' '
> +                       git -C one config remote.origin.url >one.remote-url &&
> +                       git -C one config remote.origin.fetch >one.remote-fetch &&
> +                       remote_url="file://$(cat one.remote-url)" &&
> +                       remote_fetch="$(cat one.remote-fetch)" &&

Is there a reason that these values need to be captured to files
(which are otherwise not used) before being assigned to variables?
That is, wouldn't this work?

    remote_url="file://$(git -C one config remote.origin.url)" &&
    remote_fetch="$(git -C one config remote.origin.fetch)" &&

> +                       cmdline_setup="\"$remote_url\" \"$remote_fetch\"" &&
> +                       touch prune-type-setup-done

Why does "prune-type-setup-done" need to be a file rather than a
simple shell variable (which is global by default even when assigned
inside test_expect_success)?

Also, since the purpose of this code seems to compute 'cmdline_setup'
just once, can't you do away with 'prune-type-setup-done' altogether
and change:

    if ! test -e prune-type-setup-done

to:

    if test -z "$cmdline_setup"

> +               '
> +       fi

Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Ævar Arnfjörð Bjarmason]

On Fri, Feb 09 2018, Eric Sunshine jotted:

> On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> When a remote URL is supplied on the command-line the internals of the
>> fetch are different, in particular the code in get_ref_map(). An
>> earlier version of the subsequent fetch.pruneTags patch hid a segfault
>> because the difference wasn't tested for.
>>
>> Now all the tests are run as both of the variants of:
>>
>>     git fetch
>>     git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)
>>
>> I'm using -c because while the [fetch] config just set by
>> set_config_tristate will be picked up, the remote.origin.* config
>> won't override it as intended.
>>
>> Work around that and turn this into a purely command-line test by
>> always setting the variables on the command-line, and translate any
>> setting of remote.origin.X into fetch.X.
>> [...]
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
>> @@ -548,18 +548,52 @@ set_config_tristate () {
>>         *)
>>                 git config "$1" "$2"
>> +               key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
>
> Faster (thus more Windows-friendly) assuming that $1 always starts
> with "remote.origin":
>
>     key=fetch${u#remote.origin}

Tests fail with this and I'm not excited to be the first user in git's
test suite to use some novel shell feature, no existing uses of
${u[...].

I also think stuff like this is on the wrong side of cleverness
v.s. benefit. I can't find any reference to this syntax in bash or dash
manpages (forward-search "${u"), but echo | sed is obvious, and it's not
going to make a big difference for Windows.

>> +               git_fetch_c="$git_fetch_c -c $key=$2"
>>                 ;;
>>         esac
>>  }
>>
>> +test_configured_prune_type () {
>>         fetch_prune=$1
>>         remote_origin_prune=$2
>>         expected_branch=$3
>>         expected_tag=$4
>>         cmdline=$5
>> -
>> -       test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
>> +       mode=$6
>> +
>> +       if ! test -e prune-type-setup-done
>> +       then
>> +               test_expect_success 'prune_type setup' '
>> +                       git -C one config remote.origin.url >one.remote-url &&
>> +                       git -C one config remote.origin.fetch >one.remote-fetch &&
>> +                       remote_url="file://$(cat one.remote-url)" &&
>> +                       remote_fetch="$(cat one.remote-fetch)" &&
>
> Is there a reason that these values need to be captured to files
> (which are otherwise not used) before being assigned to variables?
> That is, wouldn't this work?
>
>     remote_url="file://$(git -C one config remote.origin.url)" &&
>     remote_fetch="$(git -C one config remote.origin.fetch)" &&

Nope, I'll just do that. This was cruft left over from an earlier
version which I didn't clean up.

>> +                       cmdline_setup="\"$remote_url\" \"$remote_fetch\"" &&
>> +                       touch prune-type-setup-done
>
> Why does "prune-type-setup-done" need to be a file rather than a
> simple shell variable (which is global by default even when assigned
> inside test_expect_success)?
>
> Also, since the purpose of this code seems to compute 'cmdline_setup'
> just once, can't you do away with 'prune-type-setup-done' altogether
> and change:
>
>     if ! test -e prune-type-setup-done
>
> to:
>
>     if test -z "$cmdline_setup"
>
>> +               '
>> +       fi

Yup, that's much simpler. Thanks.

Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Jeff King]

On Fri, Feb 09, 2018 at 09:05:00PM +0100, Ævar Arnfjörð Bjarmason wrote:

> >> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
> >> @@ -548,18 +548,52 @@ set_config_tristate () {
> >>         *)
> >>                 git config "$1" "$2"
> >> +               key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
> >
> > Faster (thus more Windows-friendly) assuming that $1 always starts
> > with "remote.origin":
> >
> >     key=fetch${u#remote.origin}
> 
> Tests fail with this and I'm not excited to be the first user in git's
> test suite to use some novel shell feature, no existing uses of
> ${u[...].
> 
> I also think stuff like this is on the wrong side of cleverness
> v.s. benefit. I can't find any reference to this syntax in bash or dash
> manpages (forward-search "${u"), but echo | sed is obvious, and it's not
> going to make a big difference for Windows.

The "u" isn't the magic, it's the "#". I.e.:

  key=fetch${1#remote.origin}

and it's used all over the place in our scripts.  I'm not sure why Eric
wrote "u". :)

-Peff

Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Eric Sunshine]

On Fri, Feb 9, 2018 at 3:27 PM, Jeff King <peff@peff.net> wrote:
> On Fri, Feb 09, 2018 at 09:05:00PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> >> +               key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
>> >
>> > Faster (thus more Windows-friendly) assuming that $1 always starts
>> > with "remote.origin":
>> >
>> >     key=fetch${u#remote.origin}
>>
>> Tests fail with this and I'm not excited to be the first user in git's
>> test suite to use some novel shell feature, no existing uses of
>> ${u[...].
>>
>> I also think stuff like this is on the wrong side of cleverness
>> v.s. benefit. I can't find any reference to this syntax in bash or dash
>> manpages (forward-search "${u"), but echo | sed is obvious, and it's not
>> going to make a big difference for Windows.
>
> The "u" isn't the magic, it's the "#". I.e.:
>
>   key=fetch${1#remote.origin}
>
> and it's used all over the place in our scripts.  I'm not sure why Eric
> wrote "u". :)

Because I was testing the expression interactively in the shell and
assigned to a variable arbitrarily named "u". When I copy/pasted the
working expression from the shell session into the email, I stupidly
forgot to change the "u" to "1".

Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>>> +               key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
>>
>> Faster (thus more Windows-friendly) assuming that $1 always starts
>> with "remote.origin":
>>
>>     key=fetch${u#remote.origin}
>
> Tests fail with this and I'm not excited to be the first user in git's
> test suite to use some novel shell feature, no existing uses of
> ${u[...].

s/u/1/, I think.

Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>>>> +               key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
>>>
>>> Faster (thus more Windows-friendly) assuming that $1 always starts
>>> with "remote.origin":
>>>
>>>     key=fetch${u#remote.origin}
>>
>> Tests fail with this and I'm not excited to be the first user in git's
>> test suite to use some novel shell feature, no existing uses of
>> ${u[...].
>
> s/u/1/, I think.

Ah, that's been already pointed out.  Sorry for the noise.

[PATCH v2 12/17] git fetch doc: add a new section to explain the ins & outs of pruning

[Ævar Arnfjörð Bjarmason]

Add a new section to canonically explain how remote reference pruning
works, and how users should be careful about using it in conjunction
with tag refspecs in particular.

A subsequent commit will update the git-remote documentation to refer
to this section, and details the motivation for writing this in the
first place.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index b153aefa68..18fac0da2e 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be
 overridden by giving the `--refmap=<refspec>` parameter(s) on the
 command line.
 
+PRUNING
+-------
+
+Git has a default disposition of keeping data unless it's explicitly
+thrown away; this extends to holding onto local references to branches
+on remotes that have themselves deleted those branches.
+
+If left to accumulate, these stale references might make performance
+worse on big and busy repos that have a lot of branch churn, and
+e.g. make the output of commands like `git branch -a --contains
+<commit>` needlessly verbose, as well as impacting anything else
+that'll work with the complete set of known references.
+
+These remote tracking references can be deleted as a one-off with
+either of:
+
+------------------------------------------------
+# While fetching
+$ git fetch --prune <name>
+
+# Only prune, don't fetch
+$ git remote prune <name>
+------------------------------------------------
+
+To prune references as part of your normal workflow without needing to
+remember to run that set `fetch.prune` globally, or
+`remote.<name>.prune` per-remote in the config. See
+linkgit:git-config[1].
+
+Here's where things get tricky and more specific. The pruning feature
+doesn't actually care about branches, instead it'll prune local <->
+remote references as a function of the refspec of the remote (see
+`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).
+
+Therefore if the refspec for the remote includes
+e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch
+--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote
+tracking branches that are deleted, but any local tag that doesn't
+exist on the remote.
+
+This might not be what you expect, i.e. you want to prune remote
+`<name>`, but also explicitly fetch tags from it, so when you fetch
+from it you delete all your local tags, most of which may not have
+come from the `<name>` remote in the first place.
+
+So be careful when using this with a refspec like
+`refs/tags/*:refs/tags/*`, or any other refspec which might map
+references from multiple remotes to the same local namespace.
+
 OUTPUT
 ------
 
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 12/17] git fetch doc: add a new section to explain the ins & outs of pruning

[Eric Sunshine]

On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Add a new section to canonically explain how remote reference pruning
> works, and how users should be careful about using it in conjunction
> with tag refspecs in particular.
> [...]
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
> @@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be
> +PRUNING
> +-------
> +[...]
> +If left to accumulate, these stale references might make performance
> +worse on big and busy repos that have a lot of branch churn, and
> +e.g. make the output of commands like `git branch -a --contains
> +<commit>` needlessly verbose, as well as impacting anything else
> +that'll work with the complete set of known references.
> +
> +These remote tracking references can be deleted as a one-off with

I think we call these "remote-tracking" (note the hyphen), which are
local but track something remote, rather than "remote tracking" (no
hyphen) which would themselves be remote.

> +either of:
> +
> +------------------------------------------------
> +# While fetching
> +$ git fetch --prune <name>
> +
> +# Only prune, don't fetch
> +$ git remote prune <name>
> +------------------------------------------------
> +
> +To prune references as part of your normal workflow without needing to
> +remember to run that set `fetch.prune` globally, or

s/that/&,/

> +`remote.<name>.prune` per-remote in the config. See
> +linkgit:git-config[1].

[PATCH v2 13/17] git remote doc: correct dangerous lies about what prune does

[Ævar Arnfjörð Bjarmason]

The "git remote prune <name>" command uses the same machinery as "git
fetch <name> --prune", and shares all the same caveats, but its
documentation has suggested that it'll just "delete stale
remote-tracking branches under <name>".

This isn't true, and hasn't been true since at least v1.8.5.6 (the
oldest version I could be bothered to test).

E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of
the remote it'll delete all local tags <name> doesn't know about.

Instead, briefly give the reader just enough of a hint that this
option might constitute a shotgun aimed at their foot, and point them
to the new PRUNING section in the git-fetch documentation which
explains all the nuances of what this facility does.

See "[BUG] git remote prune removes local tags, depending on fetch
config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com)
by Michael Giuffrida for the initial report.

Reported-by: Michael Giuffrida <michaelpg@chromium.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-remote.txt | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-remote.txt b/Documentation/git-remote.txt
index 577b969c1b..04e2410aec 100644
--- a/Documentation/git-remote.txt
+++ b/Documentation/git-remote.txt
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with
 
 'prune'::
 
-Deletes all stale remote-tracking branches under <name>.
-These stale branches have already been removed from the remote repository
-referenced by <name>, but are still locally available in
-"remotes/<name>".
+Deletes stale references associated with <name>. By default stale
+remote-tracking branches under <name>, but depending on global
+configuration and the configuration of the remote we might even prune
+local tags that haven't been pushed there. Equivalent to `git fetch
+--prune <name>`, except that no new references will be fetched.
++
+See the PRUNING section of linkgit:git-fetch[1] for what it'll prune
+depending on various configuration.
 +
 With `--dry-run` option, report what branches will be pruned, but do not
 actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the
 configuration parameter remote.<name>.skipDefaultUpdate set to true will
 be updated.  (See linkgit:git-config[1]).
 +
-With `--prune` option, prune all the remotes that are updated.
+With `--prune` option, run pruning against all the remotes that are updated.
 
 
 DISCUSSION
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 13/17] git remote doc: correct dangerous lies about what prune does

[Eric Sunshine]

On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> The "git remote prune <name>" command uses the same machinery as "git
> fetch <name> --prune", and shares all the same caveats, but its
> documentation has suggested that it'll just "delete stale
> remote-tracking branches under <name>".
>
> This isn't true, and hasn't been true since at least v1.8.5.6 (the
> oldest version I could be bothered to test).
>
> E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of
> the remote it'll delete all local tags <name> doesn't know about.

Not worth a re-roll as this is only the commit message: s/remote/&,/

More below...

> Instead, briefly give the reader just enough of a hint that this
> option might constitute a shotgun aimed at their foot, and point them
> to the new PRUNING section in the git-fetch documentation which
> explains all the nuances of what this facility does.
> [...]
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> diff --git a/Documentation/git-remote.txt b/Documentation/git-remote.txt
> @@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with
> -Deletes all stale remote-tracking branches under <name>.
> -These stale branches have already been removed from the remote repository
> -referenced by <name>, but are still locally available in
> -"remotes/<name>".
> +Deletes stale references associated with <name>. By default stale

Nit: s/default/&,/

> +remote-tracking branches under <name>, but depending on global
> +configuration and the configuration of the remote we might even prune
> +local tags that haven't been pushed there.

This is a bit difficult to digest grammatically due to the phrase
being incomplete. It could say either:

    By default, stale ... under <name> _are deleted_, but
    depending...

or:

    Deletes stale ... <name>; by default, stale...

[PATCH v2 14/17] git-fetch & config doc: link to the new PRUNING section

[Ævar Arnfjörð Bjarmason]

Amend the documentation for fetch.prune, fetch.<name>.prune and
--prune to link to the recently added PRUNING section.

I'd have liked to link directly to it with "<<PRUNING>>" from
fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt
also includes it, but doesn't include that option). However making a
reference across files yields this error:

    [...]/Documentation/git-fetch.xml:226: element xref: validity
    error : IDREF attribute linkend references an unknown ID "PRUNING"

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 6 +++++-
 Documentation/fetch-options.txt | 3 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b..0f27af5760 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1398,7 +1398,8 @@ fetch.unpackLimit::
 
 fetch.prune::
 	If true, fetch will automatically behave as if the `--prune`
-	option was given on the command line.  See also `remote.<name>.prune`.
+	option was given on the command line.  See also `remote.<name>.prune`
+	and the PRUNING section of linkgit:git-fetch[1].
 
 fetch.output::
 	Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
++
+See also `remote.<name>.prune` and the PRUNING section of
+linkgit:git-fetch[1].
 
 remotes.<group>::
 	The list of remotes which are fetched by "git remote update
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index fb6bebbc61..9f5c85ad96 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -74,6 +74,9 @@ ifndef::git-pull[]
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
 	subject to pruning.
++
+See the PRUNING section below for more details.
+
 endif::git-pull[]
 
 ifndef::git-pull[]
-- 
2.15.1.424.g9478a66081

[PATCH v2 15/17] fetch tests: add scaffolding for the new fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

The fetch.pruneTags configuration doesn't exist yet, but will be added
in a subsequent commit. Since testing for it requires adding new
parameters to the test_configured_prune function it's easier to review
this patch first to assert that no functional changes are introduced
yet.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 92 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 50 insertions(+), 42 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 73ba83454f..501d15dd42 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -562,10 +562,12 @@ test_configured_prune () {
 test_configured_prune_type () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	expected_branch=$3
-	expected_tag=$4
-	cmdline=$5
-	mode=$6
+	fetch_prune_tags=$3
+	remote_origin_prune_tags=$4
+	expected_branch=$5
+	expected_tag=$6
+	cmdline=$7
+	mode=$8
 
 	if ! test -e prune-type-setup-done
 	then
@@ -593,14 +595,16 @@ test_configured_prune_type () {
 		cmdline="$new_cmdline"
 	fi
 
-	test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+	test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
+			test_unconfig fetch.pruneTags &&
 			test_unconfig remote.origin.prune &&
+			test_unconfig remote.origin.pruneTags &&
 			git fetch '"$cmdline_setup"' &&
 			git rev-parse --verify refs/remotes/origin/newbranch &&
 			git rev-parse --verify refs/tags/newtag
@@ -615,7 +619,9 @@ test_configured_prune_type () {
 			cd one &&
 			git_fetch_c="" &&
 			set_config_tristate fetch.prune $fetch_prune &&
+			set_config_tristate fetch.pruneTags $fetch_prune_tags &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
+			set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags &&
 
 			if test "$mode" != "link"
 			then
@@ -644,57 +650,59 @@ test_configured_prune_type () {
 
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
-# $3 expect: branch to be pruned?
-# $4 expect: tag to be pruned?
-# $5 git-fetch $cmdline:
+# $3 config: fetch.pruneTags
+# $4 config: remote.<name>.pruneTags
+# $5 expect: branch to be pruned?
+# $6 expect: tag to be pruned?
+# $7 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4     $5
-test_configured_prune unset unset kept   kept   ""
-test_configured_prune unset unset kept   kept   "--no-prune"
-test_configured_prune unset unset pruned kept   "--prune"
-test_configured_prune unset unset kept   pruned \
+#                     $1    $2    $3    $4    $5     $6     $7
+test_configured_prune unset unset unset unset kept   kept   ""
+test_configured_prune unset unset unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset unset unset pruned kept   "--prune"
+test_configured_prune unset unset unset unset kept   pruned \
 	"--prune origin refs/tags/*:refs/tags/*"
-test_configured_prune unset unset pruned pruned \
+test_configured_prune unset unset unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
-test_configured_prune false unset kept   kept   ""
-test_configured_prune false unset kept   kept   "--no-prune"
-test_configured_prune false unset pruned kept   "--prune"
+test_configured_prune false unset unset unset kept   kept   ""
+test_configured_prune false unset unset unset kept   kept   "--no-prune"
+test_configured_prune false unset unset unset pruned kept   "--prune"
 
-test_configured_prune true  unset pruned kept   ""
-test_configured_prune true  unset pruned kept   "--prune"
-test_configured_prune true  unset kept   kept   "--no-prune"
+test_configured_prune true  unset unset unset pruned kept   ""
+test_configured_prune true  unset unset unset pruned kept   "--prune"
+test_configured_prune true  unset unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset false kept   kept   ""
-test_configured_prune unset false kept   kept   "--no-prune"
-test_configured_prune unset false pruned kept   "--prune"
+test_configured_prune unset false unset unset kept   kept   ""
+test_configured_prune unset false unset unset kept   kept   "--no-prune"
+test_configured_prune unset false unset unset pruned kept   "--prune"
 
-test_configured_prune false false kept   kept   ""
-test_configured_prune false false kept   kept   "--no-prune"
-test_configured_prune false false pruned kept   "--prune"
-test_configured_prune false false kept   pruned \
+test_configured_prune false false unset unset kept   kept   ""
+test_configured_prune false false unset unset kept   kept   "--no-prune"
+test_configured_prune false false unset unset pruned kept   "--prune"
+test_configured_prune false false unset unset kept   pruned \
 	"--prune origin refs/tags/*:refs/tags/*"
-test_configured_prune false false pruned pruned \
+test_configured_prune false false unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
-test_configured_prune true  false kept   kept   ""
-test_configured_prune true  false pruned kept   "--prune"
-test_configured_prune true  false kept   kept   "--no-prune"
+test_configured_prune true  false unset unset kept   kept   ""
+test_configured_prune true  false unset unset pruned kept   "--prune"
+test_configured_prune true  false unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset true  pruned kept   ""
-test_configured_prune unset true  kept   kept   "--no-prune"
-test_configured_prune unset true  pruned kept   "--prune"
+test_configured_prune unset true  unset unset pruned kept   ""
+test_configured_prune unset true  unset unset kept   kept   "--no-prune"
+test_configured_prune unset true  unset unset pruned kept   "--prune"
 
-test_configured_prune false true  pruned kept   ""
-test_configured_prune false true  kept   kept   "--no-prune"
-test_configured_prune false true  pruned kept   "--prune"
+test_configured_prune false true  unset unset pruned kept   ""
+test_configured_prune false true  unset unset kept   kept   "--no-prune"
+test_configured_prune false true  unset unset pruned kept   "--prune"
 
-test_configured_prune true  true  pruned kept   ""
-test_configured_prune true  true  pruned kept   "--prune"
-test_configured_prune true  true  kept   kept   "--no-prune"
-test_configured_prune true  true  kept   pruned \
+test_configured_prune true  true  unset unset pruned kept   ""
+test_configured_prune true  true  unset unset pruned kept   "--prune"
+test_configured_prune true  true  unset unset kept   kept   "--no-prune"
+test_configured_prune true  true  unset unset kept   pruned \
 	"--prune origin refs/tags/*:refs/tags/*"
-test_configured_prune true  true  pruned pruned \
+test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_expect_success 'all boundary commits are excluded' '
-- 
2.15.1.424.g9478a66081

[PATCH v2 16/17] fetch: add a --fetch-prune option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
config option. This allows for doing any of:

    git fetch -p -P
    git fetch --prune --prune-tags
    git fetch -p -P origin
    git fetch --prune --prune-tags origin

Or simply:

    git config fetch.prune true &&
    git config fetch.pruneTags true &&
    git fetch

Instead of the much more verbose:

    git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'

Before this feature it was painful to support the use-case of pulling
from a repo which is having both its branches *and* tags deleted
regularly, and have our local references to reflect upstream.

At work we create deployment tags in the repo for each rollout, and
there's *lots* of those, so they're archived within weeks for
performance reasons.

Without this change it's hard to centrally configure such repos in
/etc/gitconfig (on servers that are only used for working with
them). You need to set fetch.prune=true globally, and then for each
repo:

    git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"

Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well,
and users running "git pull" will automatically get the pruning
semantics I want.

Even though "git remote" has corresponding "prune" and "update
--prune" subcommands I'm intentionally not adding a corresponding
prune-tags or "update --prune --prune-tags" mode to that command.

It's advertised (as noted in my recent "git remote doc: correct
dangerous lies about what prune does") as only modifying remote
tracking references, whereas any --prune-tags option is always going
to modify what from the user's perspective is a local copy of the tag,
since there's no such thing as a remote tracking tag.

Ideally add_prune_tags_to_fetch_refspec() would be something that
would use ALLOC_GROW() to grow the 'fetch` member of the 'remote'
struct. Instead I'm realloc-ing remote->fetch and adding the
tag_refspec to the end.

The reason is that parse_{fetch,push}_refspec which allocate the
refspec (ultimately remote->fetch) struct are called many places that
don't have access to a 'remote' struct. It would be hard to change all
their callsites to be amenable to carry around the bookkeeping
variables required for dynamic allocation.

All the other callers of the API first incrementally construct the
string version of the refspec in remote->fetch_refspec via
add_fetch_refspec(), before finally calling parse_fetch_refspec() via
some variation of remote_get().

It's less of a pain to deal with the one special case that needs to
modify already constructed refspecs than to chase down and change all
the other callsites. The API I'm adding is intentionally not
generalized because if we add more of these we'd probably want to
re-visit how this is done.

See my "Re: [BUG] git remote prune removes local tags, depending on
fetch config" (87po6ahx87.fsf@evledraar.gmail.com;
https://public-inbox.org/git/87po6ahx87.fsf@evledraar.gmail.com/) for
more background info.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt               | 14 +++++++
 Documentation/fetch-options.txt        | 14 ++++++-
 Documentation/git-fetch.txt            | 47 +++++++++++++++++++++++
 builtin/fetch.c                        | 32 ++++++++++++++--
 contrib/completion/git-completion.bash |  2 +-
 remote.c                               | 14 +++++++
 remote.h                               |  3 ++
 t/t5510-fetch.sh                       | 69 ++++++++++++++++++++++++++++++++++
 8 files changed, 190 insertions(+), 5 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0f27af5760..e254bfd531 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1401,6 +1401,14 @@ fetch.prune::
 	option was given on the command line.  See also `remote.<name>.prune`
 	and the PRUNING section of linkgit:git-fetch[1].
 
+fetch.pruneTags::
+	If true, fetch will automatically behave as if the
+	`refs/tags/*:refs/tags/*` refspec was provided when pruning,
+	if not set already. This allows for setting both this option
+	and `fetch.prune` to maintain a 1=1 mapping to upstream
+	refs. See also `remote.<name>.pruneTags` and the PRUNING
+	section of linkgit:git-fetch[1].
+
 fetch.output::
 	Control how ref update status is printed. Valid values are
 	`full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
+
+remote.<name>.pruneTags::
+	When set to true, fetching from this remote by default will also
+	remove any local tags that no longer exist on the remote if pruning
+	is activated in general via `remote.<name>.prune`, `fetch.prune` or
+	`--prune`. Overrides `fetch.pruneTags` settings, if any.
 +
 See also `remote.<name>.prune` and the PRUNING section of
 linkgit:git-fetch[1].
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 9f5c85ad96..8631e365f4 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -73,7 +73,19 @@ ifndef::git-pull[]
 	are fetched due to an explicit refspec (either on the command
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
-	subject to pruning.
+	subject to pruning. Supplying `--prune-tags` is a shorthand for
+	providing the tag refspec.
++
+See the PRUNING section below for more details.
+
+-P::
+--prune-tags::
+	Before fetching, remove any local tags that no longer exist on
+	the remote if `--prune` is enabled. This option should be used
+	more carefully, unlike `--prune` it will remove any local
+	references (local tags) that have been created. This option is
+	a shorthand for providing the explicit tag refspec along with
+	`--prune`, see the discussion about that in its documentation.
 +
 See the PRUNING section below for more details.
 
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 18fac0da2e..574206d139 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -148,6 +148,53 @@ So be careful when using this with a refspec like
 `refs/tags/*:refs/tags/*`, or any other refspec which might map
 references from multiple remotes to the same local namespace.
 
+Since keeping up-to-date with both branches and tags on the remote is
+a common use-case the `--prune-tags` option can be supplied along with
+`--prune` to prune local tags that don't exist on the remote, and
+force-update those tags that differ. Tag pruning can also be enabled
+with `fetch.pruneTags` or `remote.<name>.pruneTags` in the config. See
+linkgit:git-config[1].
+
+The `--prune-tags` option is equivalent to having
+`refs/tags/*:refs/tags/*` declared in the refspecs of the remote. This
+can lead to some seemingly strange interactions:
+
+------------------------------------------------
+# These both fetch tags
+$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*'
+$ git fetch --no-tags --prune-tags origin
+------------------------------------------------
+
+The reason it doesn't error out when provided without `--prune` or its
+config versions is for flexibility of the configured versions, and to
+maintain a 1=1 mapping between what the command line flags do, and
+what the configuration versions do.
+
+It's reasonable to e.g. configure `fetch.pruneTags=true` in
+`~/.gitconfig` to have tags pruned whenever `git fetch --prune` is
+run, without making every invocation of `git fetch` without `--prune`
+an error.
+
+Another special case of `--prune-tags` is that
+`refs/tags/*:refs/tags/*` will not be implicitly provided if an URL is
+being fetched. I.e.:
+
+------------------------------------------------
+$ git fetch <url> --prune --prune-tags
+------------------------------------------------
+
+Will prune no tags, as opposed to:
+
+------------------------------------------------
+$ git fetch origin --prune --prune-tags
+------------------------------------------------
+
+To prune tags given a URL supply the refspec explicitly:
+
+------------------------------------------------
+$ git fetch <url> --prune 'refs/tags/*:refs/tags/*'
+------------------------------------------------
+
 OUTPUT
 ------
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a7705bc150..55a0fc37be 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */
 static int prune = -1; /* unspecified */
 #define PRUNE_BY_DEFAULT 0 /* do we prune by default? */
 
+static int fetch_prune_tags_config = -1; /* unspecified */
+static int prune_tags = -1; /* unspecified */
+#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
 static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;
 static int progress = -1;
 static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(k, "fetch.prunetags")) {
+		fetch_prune_tags_config = git_config_bool(k, v);
+		return 0;
+	}
+
 	if (!strcmp(k, "submodule.recurse")) {
 		int r = git_config_bool(k, v) ?
 			RECURSE_SUBMODULES_ON : RECURSE_SUBMODULES_OFF;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = {
 		    N_("number of submodules fetched in parallel")),
 	OPT_BOOL('p', "prune", &prune,
 		 N_("prune remote-tracking branches no longer on remote")),
+	OPT_BOOL('P', "prune-tags", &prune_tags,
+		 N_("prune local tags no longer on remote and clobber changed tags")),
 	{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
 		    N_("control recursive fetching of submodules"),
 		    PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)
 		argv_array_push(argv, "--dry-run");
 	if (prune != -1)
 		argv_array_push(argv, prune ? "--prune" : "--no-prune");
+	if (prune_tags != -1)
+		argv_array_push(argv, prune_tags ? "--prune-tags" : "--no-prune-tags");
 	if (update_head_ok)
 		argv_array_push(argv, "--update-head-ok");
 	if (force)
@@ -1265,7 +1278,7 @@ static int fetch_multiple(struct string_list *list)
 	return result;
 }
 
-static int fetch_one(struct remote *remote, int argc, const char **argv)
+static int fetch_one(struct remote *remote, int argc, const char **argv, int prune_tags_ok)
 {
 	static const char **refs = NULL;
 	struct refspec *refspec;
@@ -1288,6 +1301,19 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			prune = PRUNE_BY_DEFAULT;
 	}
 
+	if (prune_tags < 0) {
+		/* no command line request */
+		if (0 <= remote->prune_tags)
+			prune_tags = remote->prune_tags;
+		else if (0 <= fetch_prune_tags_config)
+			prune_tags = fetch_prune_tags_config;
+		else
+			prune_tags = PRUNE_TAGS_BY_DEFAULT;
+	}
+
+	if (prune_tags_ok && prune_tags && remote_is_configured(remote, 0))
+		add_prune_tags_to_fetch_refspec(remote);
+
 	if (argc > 0) {
 		int j = 0;
 		int i;
@@ -1368,7 +1394,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	} else if (argc == 0) {
 		/* No arguments -- use default remote */
 		remote = remote_get(NULL);
-		result = fetch_one(remote, argc, argv);
+		result = fetch_one(remote, argc, argv, 1);
 	} else if (multiple) {
 		/* All arguments are assumed to be remotes or groups */
 		for (i = 0; i < argc; i++)
@@ -1386,7 +1412,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 		} else {
 			/* Zero or one remotes */
 			remote = remote_get(argv[0]);
-			result = fetch_one(remote, argc-1, argv+1);
+			result = fetch_one(remote, argc-1, argv+1, argc == 1);
 		}
 	}
 
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3683c772c5..4ecd0d4d7a 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1468,7 +1468,7 @@ __git_fetch_recurse_submodules="yes on-demand no"
 __git_fetch_options="
 	--quiet --verbose --append --upload-pack --force --keep --depth=
 	--tags --no-tags --all --prune --dry-run --recurse-submodules=
-	--unshallow --update-shallow
+	--unshallow --update-shallow --prune-tags
 "
 
 _git_fetch ()
diff --git a/remote.c b/remote.c
index 356c123e3e..d670eed4c4 100644
--- a/remote.c
+++ b/remote.c
@@ -104,6 +104,17 @@ static void add_fetch_refspec(struct remote *remote, const char *ref)
 	remote->fetch_refspec[remote->fetch_refspec_nr++] = ref;
 }
 
+void add_prune_tags_to_fetch_refspec(struct remote *remote)
+{
+	int nr = remote->fetch_refspec_nr;
+	int bufsize = nr  + 1;
+	int size = sizeof(struct refspec);
+
+	remote->fetch = xrealloc(remote->fetch, size  * bufsize);
+	memcpy(&remote->fetch[nr], tag_refspec, size);
+	add_fetch_refspec(remote, xstrdup(TAG_REFSPEC));
+}
+
 static void add_url(struct remote *remote, const char *url)
 {
 	ALLOC_GROW(remote->url, remote->url_nr + 1, remote->url_alloc);
@@ -174,6 +185,7 @@ static struct remote *make_remote(const char *name, int len)
 
 	ret = xcalloc(1, sizeof(struct remote));
 	ret->prune = -1;  /* unspecified */
+	ret->prune_tags = -1;  /* unspecified */
 	ALLOC_GROW(remotes, remotes_nr + 1, remotes_alloc);
 	remotes[remotes_nr++] = ret;
 	ret->name = xstrndup(name, len);
@@ -392,6 +404,8 @@ static int handle_config(const char *key, const char *value, void *cb)
 		remote->skip_default_update = git_config_bool(key, value);
 	else if (!strcmp(subkey, "prune"))
 		remote->prune = git_config_bool(key, value);
+	else if (!strcmp(subkey, "prunetags"))
+		remote->prune_tags = git_config_bool(key, value);
 	else if (!strcmp(subkey, "url")) {
 		const char *v;
 		if (git_config_string(&v, key, value))
diff --git a/remote.h b/remote.h
index 80fea6dd11..271afe1bab 100644
--- a/remote.h
+++ b/remote.h
@@ -47,6 +47,7 @@ struct remote {
 	int skip_default_update;
 	int mirror;
 	int prune;
+	int prune_tags;
 
 	const char *receivepack;
 	const char *uploadpack;
@@ -299,4 +300,6 @@ void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *);
 
 #define TAG_REFSPEC "refs/tags/*:refs/tags/*"
 
+void add_prune_tags_to_fetch_refspec(struct remote *remote);
+
 #endif
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 501d15dd42..6f3ab7695a 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -592,6 +592,15 @@ test_configured_prune_type () {
 			new_cmdline=$(printf "%s" "$cmdline" | perl -pe 's[origin(?!/)]["'"$remote_url"'"]g')
 		fi
 
+		if test "$fetch_prune_tags" = 'true' ||
+		   test "$remote_origin_prune_tags" = 'true'
+		then
+			if ! printf '%s' "$cmdline" | grep -q refs/remotes/origin/
+			then
+				new_cmdline="$new_cmdline refs/tags/*:refs/tags/*"
+			fi
+		fi
+
 		cmdline="$new_cmdline"
 	fi
 
@@ -705,6 +714,66 @@ test_configured_prune true  true  unset unset kept   pruned \
 test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
+# --prune-tags on its own does nothing, needs --prune as well, same
+# for for fetch.pruneTags without fetch.prune
+test_configured_prune unset unset unset unset kept kept     "--prune-tags"
+test_configured_prune unset unset true unset  kept kept     ""
+test_configured_prune unset unset unset true  kept kept     ""
+
+# These will prune the tags
+test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags"
+test_configured_prune true  unset true  unset pruned pruned ""
+test_configured_prune unset true  unset true  pruned pruned ""
+
+# remote.<name>.pruneTags overrides fetch.pruneTags, just like
+# remote.<name>.prune overrides fetch.prune if set.
+test_configured_prune true  unset true unset pruned pruned  ""
+test_configured_prune false true  false true  pruned pruned ""
+test_configured_prune true  false true  false kept   kept   ""
+
+# When --prune-tags is supplied it's ignored if an explict refspec is
+# given, same for the configuration options.
+test_configured_prune unset unset unset unset pruned kept \
+	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
+test_configured_prune unset unset true  unset pruned kept \
+	"--prune origin +refs/heads/*:refs/remotes/origin/*"
+test_configured_prune unset unset unset true pruned  kept \
+	"--prune origin +refs/heads/*:refs/remotes/origin/*"
+
+# Pruning that also takes place if s!origin!<file:// url of remote>!,
+# or otherwise uses the file://-specific codepath. However, because
+# there's no implicit +refs/heads/*:refs/remotes/origin/* refspec and
+# supplying it on the command-line negate --prune-tags the branches
+# will not be pruned.
+test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "name"
+test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "link"
+test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
+test_configured_prune_type unset unset unset unset kept   kept   "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name"
+test_configured_prune_type unset unset unset unset kept   kept   "--prune --prune-tags origin" "link"
+test_configured_prune_type unset unset true  unset pruned pruned "--prune origin" "name"
+test_configured_prune_type unset unset true  unset kept   pruned "--prune origin" "link"
+test_configured_prune_type unset unset unset true  pruned pruned "--prune origin" "name"
+test_configured_prune_type unset unset unset true  kept   pruned "--prune origin" "link"
+test_configured_prune_type true  unset true  unset pruned pruned "origin" "name"
+test_configured_prune_type true  unset true  unset kept   pruned "origin" "link"
+test_configured_prune_type unset  true true  unset pruned pruned "origin" "name"
+test_configured_prune_type unset  true true  unset kept   pruned "origin" "link"
+test_configured_prune_type unset  true unset true  pruned pruned "origin" "name"
+test_configured_prune_type unset  true unset true  kept   pruned "origin" "link"
+
+# When all remote.origin.fetch settings are deleted a --prune
+# --prune-tags still implicitly supplies refs/tags/*:refs/tags/* so
+# tags, but not tracking branches, will be deleted.
+test_expect_success 'remove remote.origin.fetch "one"' '
+	(
+		cd one &&
+		git config --unset-all remote.origin.fetch
+	)
+'
+test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name"
+test_configured_prune_type unset unset unset unset kept kept   "origin --prune --prune-tags" "link"
+
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
 	test_commit oneside &&
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 16/17] fetch: add a --fetch-prune option and fetch.pruneTags config

[Eric Sunshine]

On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
> config option. [...]
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
> @@ -592,6 +592,15 @@ test_configured_prune_type () {
> +               if test "$fetch_prune_tags" = 'true' ||
> +                  test "$remote_origin_prune_tags" = 'true'
> +               then
> +                       if ! printf '%s' "$cmdline" | grep -q refs/remotes/origin/

Is $cmdline guaranteed to end with a newline? Historically, not all
'grep's would be able to match the last line if it was not properly
terminated. Perhaps you want '%s\n' instead?

> +                       then
> +                               new_cmdline="$new_cmdline refs/tags/*:refs/tags/*"
> +                       fi
> +               fi
> +
> @@ -705,6 +714,66 @@ test_configured_prune true  true  unset unset kept   pruned \
> +# When --prune-tags is supplied it's ignored if an explict refspec is

s/explict/explicit/

> +# given, same for the configuration options.
> +
> +# Pruning that also takes place if s!origin!<file:// url of remote>!,
> +# or otherwise uses the file://-specific codepath. However, because
> +# there's no implicit +refs/heads/*:refs/remotes/origin/* refspec and
> +# supplying it on the command-line negate --prune-tags the branches

s/negate/&s/
s/--prune-tags/&,/

> +# will not be pruned.

Re: [PATCH v2 16/17] fetch: add a --fetch-prune option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

On Fri, Feb 09 2018, Eric Sunshine jotted:

> On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
>> config option. [...]
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
>> @@ -592,6 +592,15 @@ test_configured_prune_type () {
>> +               if test "$fetch_prune_tags" = 'true' ||
>> +                  test "$remote_origin_prune_tags" = 'true'
>> +               then
>> +                       if ! printf '%s' "$cmdline" | grep -q refs/remotes/origin/
>
> Is $cmdline guaranteed to end with a newline? Historically, not all
> 'grep's would be able to match the last line if it was not properly
> terminated. Perhaps you want '%s\n' instead?

Good point. No it's not, will fix that.

>> +                       then
>> +                               new_cmdline="$new_cmdline refs/tags/*:refs/tags/*"
>> +                       fi
>> +               fi
>> +
>> @@ -705,6 +714,66 @@ test_configured_prune true  true  unset unset kept   pruned \
>> +# When --prune-tags is supplied it's ignored if an explict refspec is
>
> s/explict/explicit/
>
>> +# given, same for the configuration options.
>> +
>> +# Pruning that also takes place if s!origin!<file:// url of remote>!,
>> +# or otherwise uses the file://-specific codepath. However, because
>> +# there's no implicit +refs/heads/*:refs/remotes/origin/* refspec and
>> +# supplying it on the command-line negate --prune-tags the branches
>
> s/negate/&s/
> s/--prune-tags/&,/
>
>> +# will not be pruned.

*nod*

[PATCH v2 17/17] fetch: make the --fetch-prune work with <url>

[Ævar Arnfjörð Bjarmason]

Make the new --prune-tags option work properly when git-fetch is
invoked with a <url> parameter instead of a <remote name>
parameter.

This change is split off from the introduction of --prune-tags due to
the relative complexity of munging the incoming argv, which is easier
to review as a separate change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-fetch.txt | 21 ++++++---------------
 builtin/fetch.c             | 17 ++++++++++++++---
 t/t5510-fetch.sh            |  6 +++---
 3 files changed, 23 insertions(+), 21 deletions(-)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 574206d139..03666f6215 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -175,24 +175,15 @@ It's reasonable to e.g. configure `fetch.pruneTags=true` in
 run, without making every invocation of `git fetch` without `--prune`
 an error.
 
-Another special case of `--prune-tags` is that
-`refs/tags/*:refs/tags/*` will not be implicitly provided if an URL is
-being fetched. I.e.:
-
-------------------------------------------------
-$ git fetch <url> --prune --prune-tags
-------------------------------------------------
-
-Will prune no tags, as opposed to:
+Pruning tags with `--prune-tags` also works when fetching a URL
+instead of a named remote. These will all prune tags not found on
+origin:
 
 ------------------------------------------------
 $ git fetch origin --prune --prune-tags
-------------------------------------------------
-
-To prune tags given a URL supply the refspec explicitly:
-
-------------------------------------------------
-$ git fetch <url> --prune 'refs/tags/*:refs/tags/*'
+$ git fetch origin --prune 'refs/tags/*:refs/tags/*'
+$ git fetch <url of origin> --prune --prune-tags
+$ git fetch <url of origin> --prune 'refs/tags/*:refs/tags/*'
 ------------------------------------------------
 
 OUTPUT
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 55a0fc37be..c96f17a9a3 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1283,7 +1283,10 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru
 	static const char **refs = NULL;
 	struct refspec *refspec;
 	int ref_nr = 0;
+	int j = 0;
 	int exit_code;
+	int maybe_prune_tags;
+	int remote_via_config = remote_is_configured(remote, 0);
 
 	if (!remote)
 		die(_("No remote repository specified.  Please, specify either a URL or a\n"
@@ -1311,13 +1314,21 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru
 			prune_tags = PRUNE_TAGS_BY_DEFAULT;
 	}
 
-	if (prune_tags_ok && prune_tags && remote_is_configured(remote, 0))
+	maybe_prune_tags = prune_tags_ok && prune_tags;
+	if (maybe_prune_tags && remote_via_config)
 		add_prune_tags_to_fetch_refspec(remote);
 
+	if (argc > 0 || (maybe_prune_tags && !remote_via_config)) {
+		size_t nr_alloc = st_add3(argc, maybe_prune_tags, 1);
+		refs = xcalloc(nr_alloc, sizeof(const char *));
+		if (maybe_prune_tags) {
+			refs[j++] = xstrdup("refs/tags/*:refs/tags/*");
+			ref_nr++;
+		}
+	}
+
 	if (argc > 0) {
-		int j = 0;
 		int i;
-		refs = xcalloc(st_add(argc, 1), sizeof(const char *));
 		for (i = 0; i < argc; i++) {
 			if (!strcmp(argv[i], "tag")) {
 				i++;
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 6f3ab7695a..297590814d 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -748,9 +748,9 @@ test_configured_prune unset unset unset true pruned  kept \
 test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "name"
 test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "link"
 test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
-test_configured_prune_type unset unset unset unset kept   kept   "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset kept   pruned "origin --prune --prune-tags" "link"
 test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name"
-test_configured_prune_type unset unset unset unset kept   kept   "--prune --prune-tags origin" "link"
+test_configured_prune_type unset unset unset unset kept   pruned "--prune --prune-tags origin" "link"
 test_configured_prune_type unset unset true  unset pruned pruned "--prune origin" "name"
 test_configured_prune_type unset unset true  unset kept   pruned "--prune origin" "link"
 test_configured_prune_type unset unset unset true  pruned pruned "--prune origin" "name"
@@ -772,7 +772,7 @@ test_expect_success 'remove remote.origin.fetch "one"' '
 	)
 '
 test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name"
-test_configured_prune_type unset unset unset unset kept kept   "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "link"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 17/17] fetch: make the --fetch-prune work with <url>

[Eric Sunshine]

On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> fetch: make the --fetch-prune work with <url>

Do you mean s/--fetch-prune/--prune-tags/ ?

> Make the new --prune-tags option work properly when git-fetch is
> invoked with a <url> parameter instead of a <remote name>
> parameter.
>
> This change is split off from the introduction of --prune-tags due to
> the relative complexity of munging the incoming argv, which is easier
> to review as a separate change.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

Re: [PATCH v2 17/17] fetch: make the --fetch-prune work with <url>

[Ævar Arnfjörð Bjarmason]

On Fri, Feb 09 2018, Eric Sunshine jotted:

> On Thu, Feb 8, 2018 at 11:19 AM, Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> fetch: make the --fetch-prune work with <url>
>
> Do you mean s/--fetch-prune/--prune-tags/ ?

Yes, sorry. Will fix.

>> Make the new --prune-tags option work properly when git-fetch is
>> invoked with a <url> parameter instead of a <remote name>
>> parameter.
>>
>> This change is split off from the introduction of --prune-tags due to
>> the relative complexity of munging the incoming argv, which is easier
>> to review as a separate change.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

[PATCH v3 01/11] fetch: don't redundantly NULL something calloc() gave us

[Ævar Arnfjörð Bjarmason]

Stop redundantly NULL-ing the last element of the refs structure,
which was retrieved via calloc(), and is thus guaranteed to be
pre-NULL'd.

This code dates back to b888d61c83 ("Make fetch a builtin",
2007-09-10), where wasn't any reason to do this back then either, it's
just something left over from when git-fetch was initially introduced.

The initial motivation for this change was to make a subsequent change
which would also modify the refs variable smaller, since it won't have
to copy this redundant "NULL the last + 1 item" pattern.

As it turns out that change was bad, and better done differently, but
as this pattern is still pointless let's fix it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7bbcd26faf..b34665db9e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			} else
 				refs[j++] = argv[i];
 		}
-		refs[j] = NULL;
 		ref_nr = j;
 	}
 
-- 
2.15.1.424.g9478a66081

[PATCH v3 02/11] fetch: stop accessing "remote" variable indirectly

[Ævar Arnfjörð Bjarmason]

Access the "remote" variable passed to the fetch_one() directly rather
than through the gtransport wrapper struct constructed in this
function for other purposes.

This makes the code more readable, as it's now obvious that the remote
struct doesn't somehow get munged by the prepare_transport() function
above, which takes the "remote" struct as an argument and constructs
the "gtransport" struct, containing among other things the "remote"
struct.

A subsequent change will copy this pattern to access a new
remote->prune-tags field, but without hte use of the gtransport
variable. It's useful once that change lands to see that the two
pieces of code behave exactly the same.

This pattern of accessing the container struct was added in
737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this
code was initially introduced.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index b34665db9e..a85c2002a9 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 
 	if (prune < 0) {
 		/* no command line request */
-		if (0 <= gtransport->remote->prune)
-			prune = gtransport->remote->prune;
+		if (0 <= remote->prune)
+			prune = remote->prune;
 		else if (0 <= fetch_prune_config)
 			prune = fetch_prune_config;
 		else
-- 
2.15.1.424.g9478a66081

Re: [PATCH v3 02/11] fetch: stop accessing "remote" variable indirectly

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> remote->prune-tags field, but without hte use of the gtransport

s/-tags/_tags/; s/hte/the/;

No need to resend for fixing the above, as I've done so locally
before queuing.

Thanks.

[PATCH v3 03/11] fetch tests: refactor in preparation for testing tag pruning

[Ævar Arnfjörð Bjarmason]

In a subsequent commit this function will learn to test for tag
pruning, prepare for that by making space for more variables, and
making it clear that "expected" here refers to branches.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 668c54be41..11da97f9b7 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -549,9 +549,12 @@ set_config_tristate () {
 }
 
 test_configured_prune () {
-	fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4
+	fetch_prune=$1
+	remote_origin_prune=$2
+	cmdline=$3
+	expected_branch=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -572,7 +575,7 @@ test_configured_prune () {
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
 			git fetch $cmdline &&
-			case "$expected" in
+			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
 				;;
-- 
2.15.1.424.g9478a66081

[PATCH v3 04/11] fetch tests: re-arrange arguments for future readability

[Ævar Arnfjörð Bjarmason]

Re-arrange the arguments to the test_configured_prune() function used
in this test to pass the arguments to --fetch last. A subsequent
change will test for more elaborate fetch arguments, including long
refspecs. It'll be more readable to be able to wrap those on a new
line of their own.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 44 insertions(+), 38 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 11da97f9b7..ab8b25344d 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,10 +551,10 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	cmdline=$3
-	expected_branch=$4
+	expected_branch=$3
+	cmdline=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -587,41 +587,47 @@ test_configured_prune () {
 	'
 }
 
-test_configured_prune unset unset ""		kept
-test_configured_prune unset unset "--no-prune"	kept
-test_configured_prune unset unset "--prune"	pruned
-
-test_configured_prune false unset ""		kept
-test_configured_prune false unset "--no-prune"	kept
-test_configured_prune false unset "--prune"	pruned
-
-test_configured_prune true  unset ""		pruned
-test_configured_prune true  unset "--prune"	pruned
-test_configured_prune true  unset "--no-prune"	kept
-
-test_configured_prune unset false ""		kept
-test_configured_prune unset false "--no-prune"	kept
-test_configured_prune unset false "--prune"	pruned
-
-test_configured_prune false false ""		kept
-test_configured_prune false false "--no-prune"	kept
-test_configured_prune false false "--prune"	pruned
-
-test_configured_prune true  false ""		kept
-test_configured_prune true  false "--prune"	pruned
-test_configured_prune true  false "--no-prune"	kept
-
-test_configured_prune unset true  ""		pruned
-test_configured_prune unset true  "--no-prune"	kept
-test_configured_prune unset true  "--prune"	pruned
-
-test_configured_prune false true  ""		pruned
-test_configured_prune false true  "--no-prune"	kept
-test_configured_prune false true  "--prune"	pruned
-
-test_configured_prune true  true  ""		pruned
-test_configured_prune true  true  "--prune"	pruned
-test_configured_prune true  true  "--no-prune"	kept
+# $1 config: fetch.prune
+# $2 config: remote.<name>.prune
+# $3 expect: branch to be pruned?
+# $4 git-fetch $cmdline:
+#
+#                     $1    $2    $3     $4
+test_configured_prune unset unset kept   ""
+test_configured_prune unset unset kept   "--no-prune"
+test_configured_prune unset unset pruned "--prune"
+
+test_configured_prune false unset kept   ""
+test_configured_prune false unset kept   "--no-prune"
+test_configured_prune false unset pruned "--prune"
+
+test_configured_prune true  unset pruned ""
+test_configured_prune true  unset pruned "--prune"
+test_configured_prune true  unset kept   "--no-prune"
+
+test_configured_prune unset false kept   ""
+test_configured_prune unset false kept   "--no-prune"
+test_configured_prune unset false pruned "--prune"
+
+test_configured_prune false false kept   ""
+test_configured_prune false false kept   "--no-prune"
+test_configured_prune false false pruned "--prune"
+
+test_configured_prune true  false kept   ""
+test_configured_prune true  false pruned "--prune"
+test_configured_prune true  false kept   "--no-prune"
+
+test_configured_prune unset true  pruned ""
+test_configured_prune unset true  kept   "--no-prune"
+test_configured_prune unset true  pruned "--prune"
+
+test_configured_prune false true  pruned ""
+test_configured_prune false true  kept   "--no-prune"
+test_configured_prune false true  pruned "--prune"
+
+test_configured_prune true  true  pruned ""
+test_configured_prune true  true  pruned "--prune"
+test_configured_prune true  true  kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v3 05/11] fetch tests: add a tag to be deleted to the pruning tests

[Ævar Arnfjörð Bjarmason]

Add a tag to be deleted to the fetch --prune tests. The tag is always
kept for now, which is the expected behavior, but now I can add a test
for tag pruning in a later commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 53 insertions(+), 40 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index ab8b25344d..fad65bd885 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -552,21 +552,25 @@ test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
 	expected_branch=$3
-	cmdline=$4
+	expected_tag=$4
+	cmdline=$5
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
+		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
 			test_unconfig remote.origin.prune &&
 			git fetch &&
-			git rev-parse --verify refs/remotes/origin/newbranch
+			git rev-parse --verify refs/remotes/origin/newbranch &&
+			git rev-parse --verify refs/tags/newtag
 		) &&
 
 		# now remove it
 		git branch -d newbranch &&
+		git tag -d newtag &&
 
 		# then test
 		(
@@ -582,6 +586,14 @@ test_configured_prune () {
 			kept)
 				git rev-parse --verify refs/remotes/origin/newbranch
 				;;
+			esac &&
+			case "$expected_tag" in
+			pruned)
+				test_must_fail git rev-parse --verify refs/tags/newtag
+				;;
+			kept)
+				git rev-parse --verify refs/tags/newtag
+				;;
 			esac
 		)
 	'
@@ -590,44 +602,45 @@ test_configured_prune () {
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
 # $3 expect: branch to be pruned?
-# $4 git-fetch $cmdline:
+# $4 expect: tag to be pruned?
+# $5 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4
-test_configured_prune unset unset kept   ""
-test_configured_prune unset unset kept   "--no-prune"
-test_configured_prune unset unset pruned "--prune"
-
-test_configured_prune false unset kept   ""
-test_configured_prune false unset kept   "--no-prune"
-test_configured_prune false unset pruned "--prune"
-
-test_configured_prune true  unset pruned ""
-test_configured_prune true  unset pruned "--prune"
-test_configured_prune true  unset kept   "--no-prune"
-
-test_configured_prune unset false kept   ""
-test_configured_prune unset false kept   "--no-prune"
-test_configured_prune unset false pruned "--prune"
-
-test_configured_prune false false kept   ""
-test_configured_prune false false kept   "--no-prune"
-test_configured_prune false false pruned "--prune"
-
-test_configured_prune true  false kept   ""
-test_configured_prune true  false pruned "--prune"
-test_configured_prune true  false kept   "--no-prune"
-
-test_configured_prune unset true  pruned ""
-test_configured_prune unset true  kept   "--no-prune"
-test_configured_prune unset true  pruned "--prune"
-
-test_configured_prune false true  pruned ""
-test_configured_prune false true  kept   "--no-prune"
-test_configured_prune false true  pruned "--prune"
-
-test_configured_prune true  true  pruned ""
-test_configured_prune true  true  pruned "--prune"
-test_configured_prune true  true  kept   "--no-prune"
+#                     $1    $2    $3     $4     $5
+test_configured_prune unset unset kept   kept   ""
+test_configured_prune unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset pruned kept   "--prune"
+
+test_configured_prune false unset kept   kept   ""
+test_configured_prune false unset kept   kept   "--no-prune"
+test_configured_prune false unset pruned kept   "--prune"
+
+test_configured_prune true  unset pruned kept   ""
+test_configured_prune true  unset pruned kept   "--prune"
+test_configured_prune true  unset kept   kept   "--no-prune"
+
+test_configured_prune unset false kept   kept   ""
+test_configured_prune unset false kept   kept   "--no-prune"
+test_configured_prune unset false pruned kept   "--prune"
+
+test_configured_prune false false kept   kept   ""
+test_configured_prune false false kept   kept   "--no-prune"
+test_configured_prune false false pruned kept   "--prune"
+
+test_configured_prune true  false kept   kept   ""
+test_configured_prune true  false pruned kept   "--prune"
+test_configured_prune true  false kept   kept   "--no-prune"
+
+test_configured_prune unset true  pruned kept   ""
+test_configured_prune unset true  kept   kept   "--no-prune"
+test_configured_prune unset true  pruned kept   "--prune"
+
+test_configured_prune false true  pruned kept   ""
+test_configured_prune false true  kept   kept   "--no-prune"
+test_configured_prune false true  pruned kept   "--prune"
+
+test_configured_prune true  true  pruned kept   ""
+test_configured_prune true  true  pruned kept   "--prune"
+test_configured_prune true  true  kept   kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v3 06/11] fetch tests: test --prune and refspec interaction

[Ævar Arnfjörð Bjarmason]

Add a test for the interaction between explicitly provided refspecs
and fetch.prune.

There's no point in adding this boilerplate to every combination of
unset/false/true, it's instructive and sufficient to show that no
matter if the variable is unset, false or true the refspec on the
command-line overrides any configuration variable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index fad65bd885..e5e88ee474 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -609,6 +609,10 @@ test_configured_prune () {
 test_configured_prune unset unset kept   kept   ""
 test_configured_prune unset unset kept   kept   "--no-prune"
 test_configured_prune unset unset pruned kept   "--prune"
+test_configured_prune unset unset kept   pruned \
+	"--prune origin +refs/tags/*:refs/tags/*"
+test_configured_prune unset unset pruned pruned \
+	"--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_configured_prune false unset kept   kept   ""
 test_configured_prune false unset kept   kept   "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept   "--prune"
 test_configured_prune false false kept   kept   ""
 test_configured_prune false false kept   kept   "--no-prune"
 test_configured_prune false false pruned kept   "--prune"
+test_configured_prune false false kept   pruned \
+	"--prune origin +refs/tags/*:refs/tags/*"
+test_configured_prune false false pruned pruned \
+	"--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_configured_prune true  false kept   kept   ""
 test_configured_prune true  false pruned kept   "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true  pruned kept   "--prune"
 test_configured_prune true  true  pruned kept   ""
 test_configured_prune true  true  pruned kept   "--prune"
 test_configured_prune true  true  kept   kept   "--no-prune"
+test_configured_prune true  true  kept   pruned \
+	"--prune origin +refs/tags/*:refs/tags/*"
+test_configured_prune true  true  pruned pruned \
+	"--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v3 07/11] git fetch doc: add a new section to explain the ins & outs of pruning

[Ævar Arnfjörð Bjarmason]

Add a new section to canonically explain how remote reference pruning
works, and how users should be careful about using it in conjunction
with tag refspecs in particular.

A subsequent commit will update the git-remote documentation to refer
to this section, and details the motivation for writing this in the
first place.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index b153aefa68..18fac0da2e 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be
 overridden by giving the `--refmap=<refspec>` parameter(s) on the
 command line.
 
+PRUNING
+-------
+
+Git has a default disposition of keeping data unless it's explicitly
+thrown away; this extends to holding onto local references to branches
+on remotes that have themselves deleted those branches.
+
+If left to accumulate, these stale references might make performance
+worse on big and busy repos that have a lot of branch churn, and
+e.g. make the output of commands like `git branch -a --contains
+<commit>` needlessly verbose, as well as impacting anything else
+that'll work with the complete set of known references.
+
+These remote tracking references can be deleted as a one-off with
+either of:
+
+------------------------------------------------
+# While fetching
+$ git fetch --prune <name>
+
+# Only prune, don't fetch
+$ git remote prune <name>
+------------------------------------------------
+
+To prune references as part of your normal workflow without needing to
+remember to run that set `fetch.prune` globally, or
+`remote.<name>.prune` per-remote in the config. See
+linkgit:git-config[1].
+
+Here's where things get tricky and more specific. The pruning feature
+doesn't actually care about branches, instead it'll prune local <->
+remote references as a function of the refspec of the remote (see
+`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).
+
+Therefore if the refspec for the remote includes
+e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch
+--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote
+tracking branches that are deleted, but any local tag that doesn't
+exist on the remote.
+
+This might not be what you expect, i.e. you want to prune remote
+`<name>`, but also explicitly fetch tags from it, so when you fetch
+from it you delete all your local tags, most of which may not have
+come from the `<name>` remote in the first place.
+
+So be careful when using this with a refspec like
+`refs/tags/*:refs/tags/*`, or any other refspec which might map
+references from multiple remotes to the same local namespace.
+
 OUTPUT
 ------
 
-- 
2.15.1.424.g9478a66081

[PATCH v3 08/11] git remote doc: correct dangerous lies about what prune does

[Ævar Arnfjörð Bjarmason]

The "git remote prune <name>" command uses the same machinery as "git
fetch <name> --prune", and shares all the same caveats, but its
documentation has suggested that it'll just "delete stale
remote-tracking branches under <name>".

This isn't true, and hasn't been true since at least v1.8.5.6 (the
oldest version I could be bothered to test).

E.g. if "+refs/tags/*:refs/tags/*" is explicitly set in the refspec of
the remote it'll delete all local tags <name> doesn't know about.

Instead, briefly give the reader just enough of a hint that this
option might constitute a shotgun aimed at their foot, and point them
to the new PRUNING section in the git-fetch documentation which
explains all the nuances of what this facility does.

See "[BUG] git remote prune removes local tags, depending on fetch
config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com)
by Michael Giuffrida for the initial report.

Reported-by: Michael Giuffrida <michaelpg@chromium.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-remote.txt | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-remote.txt b/Documentation/git-remote.txt
index 577b969c1b..04e2410aec 100644
--- a/Documentation/git-remote.txt
+++ b/Documentation/git-remote.txt
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with
 
 'prune'::
 
-Deletes all stale remote-tracking branches under <name>.
-These stale branches have already been removed from the remote repository
-referenced by <name>, but are still locally available in
-"remotes/<name>".
+Deletes stale references associated with <name>. By default stale
+remote-tracking branches under <name>, but depending on global
+configuration and the configuration of the remote we might even prune
+local tags that haven't been pushed there. Equivalent to `git fetch
+--prune <name>`, except that no new references will be fetched.
++
+See the PRUNING section of linkgit:git-fetch[1] for what it'll prune
+depending on various configuration.
 +
 With `--dry-run` option, report what branches will be pruned, but do not
 actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the
 configuration parameter remote.<name>.skipDefaultUpdate set to true will
 be updated.  (See linkgit:git-config[1]).
 +
-With `--prune` option, prune all the remotes that are updated.
+With `--prune` option, run pruning against all the remotes that are updated.
 
 
 DISCUSSION
-- 
2.15.1.424.g9478a66081

[PATCH v3 09/11] git-fetch & config doc: link to the new PRUNING section

[Ævar Arnfjörð Bjarmason]

Amend the documentation for fetch.prune, fetch.<name>.prune and
--prune to link to the recently added PRUNING section.

I'd have liked to link directly to it with "<<PRUNING>>" from
fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt
also includes it, but doesn't include that option). However making a
reference across files yields this error:

    [...]/Documentation/git-fetch.xml:226: element xref: validity
    error : IDREF attribute linkend references an unknown ID "PRUNING"

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 6 +++++-
 Documentation/fetch-options.txt | 3 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b..0f27af5760 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1398,7 +1398,8 @@ fetch.unpackLimit::
 
 fetch.prune::
 	If true, fetch will automatically behave as if the `--prune`
-	option was given on the command line.  See also `remote.<name>.prune`.
+	option was given on the command line.  See also `remote.<name>.prune`
+	and the PRUNING section of linkgit:git-fetch[1].
 
 fetch.output::
 	Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
++
+See also `remote.<name>.prune` and the PRUNING section of
+linkgit:git-fetch[1].
 
 remotes.<group>::
 	The list of remotes which are fetched by "git remote update
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index fb6bebbc61..9f5c85ad96 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -74,6 +74,9 @@ ifndef::git-pull[]
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
 	subject to pruning.
++
+See the PRUNING section below for more details.
+
 endif::git-pull[]
 
 ifndef::git-pull[]
-- 
2.15.1.424.g9478a66081

[PATCH v3 10/11] fetch tests: add scaffolding for the new fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

The fetch.pruneTags configuration doesn't exist yet, but will be added
in a subsequent commit. Since testing for it requires adding new
parameters to the test_configured_prune function it's easier to review
this patch first to assert that no functional changes are introduced
yet.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 90 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 49 insertions(+), 41 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index e5e88ee474..840fd5ef02 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,18 +551,22 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	expected_branch=$3
-	expected_tag=$4
-	cmdline=$5
+	fetch_prune_tags=$3
+	remote_origin_prune_tags=$4
+	expected_branch=$5
+	expected_tag=$6
+	cmdline=$7
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
+			test_unconfig fetch.pruneTags &&
 			test_unconfig remote.origin.prune &&
+			test_unconfig remote.origin.pruneTags &&
 			git fetch &&
 			git rev-parse --verify refs/remotes/origin/newbranch &&
 			git rev-parse --verify refs/tags/newtag
@@ -576,7 +580,9 @@ test_configured_prune () {
 		(
 			cd one &&
 			set_config_tristate fetch.prune $fetch_prune &&
+			set_config_tristate fetch.pruneTags $fetch_prune_tags &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
+			set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags &&
 
 			git fetch $cmdline &&
 			case "$expected_branch" in
@@ -601,57 +607,59 @@ test_configured_prune () {
 
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
-# $3 expect: branch to be pruned?
-# $4 expect: tag to be pruned?
-# $5 git-fetch $cmdline:
+# $3 config: fetch.pruneTags
+# $4 config: remote.<name>.pruneTags
+# $5 expect: branch to be pruned?
+# $6 expect: tag to be pruned?
+# $7 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4     $5
-test_configured_prune unset unset kept   kept   ""
-test_configured_prune unset unset kept   kept   "--no-prune"
-test_configured_prune unset unset pruned kept   "--prune"
-test_configured_prune unset unset kept   pruned \
+#                     $1    $2    $3    $4    $5     $6     $7
+test_configured_prune unset unset unset unset kept   kept   ""
+test_configured_prune unset unset unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset unset unset pruned kept   "--prune"
+test_configured_prune unset unset unset unset kept   pruned \
 	"--prune origin +refs/tags/*:refs/tags/*"
-test_configured_prune unset unset pruned pruned \
+test_configured_prune unset unset unset unset pruned pruned \
 	"--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
-test_configured_prune false unset kept   kept   ""
-test_configured_prune false unset kept   kept   "--no-prune"
-test_configured_prune false unset pruned kept   "--prune"
+test_configured_prune false unset unset unset kept   kept   ""
+test_configured_prune false unset unset unset kept   kept   "--no-prune"
+test_configured_prune false unset unset unset pruned kept   "--prune"
 
-test_configured_prune true  unset pruned kept   ""
-test_configured_prune true  unset pruned kept   "--prune"
-test_configured_prune true  unset kept   kept   "--no-prune"
+test_configured_prune true  unset unset unset pruned kept   ""
+test_configured_prune true  unset unset unset pruned kept   "--prune"
+test_configured_prune true  unset unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset false kept   kept   ""
-test_configured_prune unset false kept   kept   "--no-prune"
-test_configured_prune unset false pruned kept   "--prune"
+test_configured_prune unset false unset unset kept   kept   ""
+test_configured_prune unset false unset unset kept   kept   "--no-prune"
+test_configured_prune unset false unset unset pruned kept   "--prune"
 
-test_configured_prune false false kept   kept   ""
-test_configured_prune false false kept   kept   "--no-prune"
-test_configured_prune false false pruned kept   "--prune"
-test_configured_prune false false kept   pruned \
+test_configured_prune false false unset unset kept   kept   ""
+test_configured_prune false false unset unset kept   kept   "--no-prune"
+test_configured_prune false false unset unset pruned kept   "--prune"
+test_configured_prune false false unset unset kept   pruned \
 	"--prune origin +refs/tags/*:refs/tags/*"
-test_configured_prune false false pruned pruned \
+test_configured_prune false false unset unset pruned pruned \
 	"--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
-test_configured_prune true  false kept   kept   ""
-test_configured_prune true  false pruned kept   "--prune"
-test_configured_prune true  false kept   kept   "--no-prune"
+test_configured_prune true  false unset unset kept   kept   ""
+test_configured_prune true  false unset unset pruned kept   "--prune"
+test_configured_prune true  false unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset true  pruned kept   ""
-test_configured_prune unset true  kept   kept   "--no-prune"
-test_configured_prune unset true  pruned kept   "--prune"
+test_configured_prune unset true  unset unset pruned kept   ""
+test_configured_prune unset true  unset unset kept   kept   "--no-prune"
+test_configured_prune unset true  unset unset pruned kept   "--prune"
 
-test_configured_prune false true  pruned kept   ""
-test_configured_prune false true  kept   kept   "--no-prune"
-test_configured_prune false true  pruned kept   "--prune"
+test_configured_prune false true  unset unset pruned kept   ""
+test_configured_prune false true  unset unset kept   kept   "--no-prune"
+test_configured_prune false true  unset unset pruned kept   "--prune"
 
-test_configured_prune true  true  pruned kept   ""
-test_configured_prune true  true  pruned kept   "--prune"
-test_configured_prune true  true  kept   kept   "--no-prune"
-test_configured_prune true  true  kept   pruned \
+test_configured_prune true  true  unset unset pruned kept   ""
+test_configured_prune true  true  unset unset pruned kept   "--prune"
+test_configured_prune true  true  unset unset kept   kept   "--no-prune"
+test_configured_prune true  true  unset unset kept   pruned \
 	"--prune origin +refs/tags/*:refs/tags/*"
-test_configured_prune true  true  pruned pruned \
+test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_expect_success 'all boundary commits are excluded' '
-- 
2.15.1.424.g9478a66081

[PATCH v3 11/11] fetch: add a --fetch-prune option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
config option. This allows for doing any of:

    git fetch -p -P
    git fetch --prune --prune-tags
    git fetch -p -P origin
    git fetch --prune --prune-tags origin

Or simply:

    git config fetch.prune true &&
    git config fetch.pruneTags true &&
    git fetch

Instead of the much more verbose:

    git fetch --prune origin '+refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'

Before this feature it was painful to support the use-case of pulling
from a repo which is having both its branches *and* tags deleted
regularly, and have our local references to reflect upstream.

At work we create deployment tags in the repo for each rollout, and
there's *lots* of those, so they're archived within weeks for
performance reasons.

Without this change it's hard to centrally configure such repos in
/etc/gitconfig (on servers that are only used for working with
them). You need to set fetch.prune=true globally, and then for each
repo:

    git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"

Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well,
and users running "git pull" will automatically get the pruning
semantics I want.

Even though "git remote" has corresponding "prune" and "update
--prune" subcommands I'm intentionally not adding a corresponding
prune-tags or "update --prune --prune-tags" mode to that command.

It's advertised (as noted in my recent "git remote doc: correct
dangerous lies about what prune does") as only modifying remote
tracking references, whereas any --prune-tags option is always going
to modify what from the user's perspective is a local copy of the tag,
since there's no such thing as a remote tracking tag.

See my "Re: [BUG] git remote prune removes local tags, depending on
fetch config" (87po6ahx87.fsf@evledraar.gmail.com;
https://public-inbox.org/git/87po6ahx87.fsf@evledraar.gmail.com/) for
more background info.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt               | 14 ++++++++++++++
 Documentation/fetch-options.txt        | 15 ++++++++++++++-
 Documentation/git-fetch.txt            | 27 +++++++++++++++++++++++++++
 builtin/fetch.c                        | 32 ++++++++++++++++++++++++++++++++
 contrib/completion/git-completion.bash |  2 +-
 remote.c                               |  5 ++++-
 remote.h                               |  3 +++
 t/t5510-fetch.sh                       | 30 ++++++++++++++++++++++++++++++
 8 files changed, 125 insertions(+), 3 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0f27af5760..ae86455876 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1401,6 +1401,14 @@ fetch.prune::
 	option was given on the command line.  See also `remote.<name>.prune`
 	and the PRUNING section of linkgit:git-fetch[1].
 
+fetch.pruneTags::
+	If true, fetch will automatically behave as if the
+	`refs/tags/*:refs/tags/*` refspec was provided when pruning,
+	if not set already. This allows for setting both this option
+	and `fetch.prune` to maintain a 1=1 mapping to upstrem
+	refs. See also `remote.<name>.pruneTags` and the PRUNING
+	section of linkgit:git-fetch[1].
+
 fetch.output::
 	Control how ref update status is printed. Valid values are
 	`full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
+
+remote.<name>.pruneTags::
+	When set to true, fetching from this remote by default will also
+	remove any local tags that no longer exist on the remote if pruning
+	is activated in general via `remote.<name>.prune`, `fetch.prune` or
+	`--prune`. Overrides `fetch.pruneTags` settings, if any.
 +
 See also `remote.<name>.prune` and the PRUNING section of
 linkgit:git-fetch[1].
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 9f5c85ad96..dc13bed741 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -73,7 +73,20 @@ ifndef::git-pull[]
 	are fetched due to an explicit refspec (either on the command
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
-	subject to pruning.
+	subject to pruning. Supplying `--prune-tags` is a shorthand for
+	providing the tag refspec.
++
+See the PRUNING section below for more details.
+
+-P::
+--prune-tags::
+	Before fetching, remove any local tags that no longer exist on
+	the remote if `--prune` is enabled. This option should be used
+	more carefully, unlike `--prune` it will remove any local
+	references (local tags) that have been created. This option is
+	merely a shorthand for providing the explicit tag refspec
+	along with `--prune`, see the discussion about that in its
+	documentation.
 +
 See the PRUNING section below for more details.
 
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 18fac0da2e..cdde6c50df 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -148,6 +148,33 @@ So be careful when using this with a refspec like
 `refs/tags/*:refs/tags/*`, or any other refspec which might map
 references from multiple remotes to the same local namespace.
 
+Since keeping up-to-date with both branches and tags on the remote is
+a common use-case the `--prune-tags` option can be supplied along with
+`--prune` to prune local tags that don't exist on the remote, and
+force-update those tags that differ. Tag pruning can also be enabled
+with `fetch.pruneTags` or `remote.<name>.pruneTags` in the config. See
+linkgit:git-config[1].
+
+The `--prune-tags` option is equivalent to having
+`+refs/tags/*:refs/tags/*` declared in the refspecs of the
+remote. This can lead to some seemingly strange interactions:
+
+------------------------------------------------
+# These both fetch tags
+$ git fetch --no-tags origin '+refs/tags/*:refs/tags/*'
+$ git fetch --no-tags --prune-tags origin
+------------------------------------------------
+
+The reason it doesn't error out when provided without `--prune` or its
+config versions is for flexibility of the configured versions, and to
+maintain a 1=1 mapping between what the command line flags do, and
+what the configuration versions do.
+
+It's reasonable to e.g. configure `fetch.pruneTags=true` in
+`~/.gitconfig` to have tags pruned whenever `git fetch --prune` is
+run, without making every invocation of `git fetch` without `--prune`
+an error.
+
 OUTPUT
 ------
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a85c2002a9..0c898e64a8 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */
 static int prune = -1; /* unspecified */
 #define PRUNE_BY_DEFAULT 0 /* do we prune by default? */
 
+static int fetch_prune_tags_config = -1; /* unspecified */
+static int prune_tags = -1; /* unspecified */
+#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
 static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;
 static int progress = -1;
 static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(k, "fetch.prunetags")) {
+		fetch_prune_tags_config = git_config_bool(k, v);
+		return 0;
+	}
+
 	if (!strcmp(k, "submodule.recurse")) {
 		int r = git_config_bool(k, v) ?
 			RECURSE_SUBMODULES_ON : RECURSE_SUBMODULES_OFF;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = {
 		    N_("number of submodules fetched in parallel")),
 	OPT_BOOL('p', "prune", &prune,
 		 N_("prune remote-tracking branches no longer on remote")),
+	OPT_BOOL('P', "prune-tags", &prune_tags,
+		 N_("prune local tags no longer on remote and clobber changed tags")),
 	{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
 		    N_("control recursive fetching of submodules"),
 		    PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)
 		argv_array_push(argv, "--dry-run");
 	if (prune != -1)
 		argv_array_push(argv, prune ? "--prune" : "--no-prune");
+	if (prune_tags != -1)
+		argv_array_push(argv, prune_tags ? "--prune-tags" : "--no-prune-tags");
 	if (update_head_ok)
 		argv_array_push(argv, "--update-head-ok");
 	if (force)
@@ -1315,6 +1328,22 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 	return exit_code;
 }
 
+static void maybe_add_prune_tags_refspec(struct remote *remote)
+{
+	if (prune_tags < 0) {
+		/* no command line request */
+		if (0 <= remote->prune_tags)
+			prune_tags = remote->prune_tags;
+		else if (0 <= fetch_prune_tags_config)
+			prune_tags = fetch_prune_tags_config;
+		else
+			prune_tags = PRUNE_TAGS_BY_DEFAULT;
+	}
+
+	if (prune_tags)
+		add_fetch_refspec(remote, xstrdup("+refs/tags/*:refs/tags/*"));
+}
+
 int cmd_fetch(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -1368,6 +1397,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	} else if (argc == 0) {
 		/* No arguments -- use default remote */
 		remote = remote_get(NULL);
+		maybe_add_prune_tags_refspec(remote);
 		result = fetch_one(remote, argc, argv);
 	} else if (multiple) {
 		/* All arguments are assumed to be remotes or groups */
@@ -1386,6 +1416,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 		} else {
 			/* Zero or one remotes */
 			remote = remote_get(argv[0]);
+			if (argc == 1)
+				maybe_add_prune_tags_refspec(remote);
 			result = fetch_one(remote, argc-1, argv+1);
 		}
 	}
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3683c772c5..4ecd0d4d7a 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1468,7 +1468,7 @@ __git_fetch_recurse_submodules="yes on-demand no"
 __git_fetch_options="
 	--quiet --verbose --append --upload-pack --force --keep --depth=
 	--tags --no-tags --all --prune --dry-run --recurse-submodules=
-	--unshallow --update-shallow
+	--unshallow --update-shallow --prune-tags
 "
 
 _git_fetch ()
diff --git a/remote.c b/remote.c
index 4e93753e19..d8716643cb 100644
--- a/remote.c
+++ b/remote.c
@@ -95,7 +95,7 @@ static void add_push_refspec(struct remote *remote, const char *ref)
 	remote->push_refspec[remote->push_refspec_nr++] = ref;
 }
 
-static void add_fetch_refspec(struct remote *remote, const char *ref)
+void add_fetch_refspec(struct remote *remote, const char *ref)
 {
 	ALLOC_GROW(remote->fetch_refspec,
 		   remote->fetch_refspec_nr + 1,
@@ -173,6 +173,7 @@ static struct remote *make_remote(const char *name, int len)
 
 	ret = xcalloc(1, sizeof(struct remote));
 	ret->prune = -1;  /* unspecified */
+	ret->prune_tags = -1;  /* unspecified */
 	ALLOC_GROW(remotes, remotes_nr + 1, remotes_alloc);
 	remotes[remotes_nr++] = ret;
 	ret->name = xstrndup(name, len);
@@ -391,6 +392,8 @@ static int handle_config(const char *key, const char *value, void *cb)
 		remote->skip_default_update = git_config_bool(key, value);
 	else if (!strcmp(subkey, "prune"))
 		remote->prune = git_config_bool(key, value);
+	else if (!strcmp(subkey, "prunetags"))
+		remote->prune_tags = git_config_bool(key, value);
 	else if (!strcmp(subkey, "url")) {
 		const char *v;
 		if (git_config_string(&v, key, value))
diff --git a/remote.h b/remote.h
index 1f6611be21..df6dca24d4 100644
--- a/remote.h
+++ b/remote.h
@@ -47,6 +47,7 @@ struct remote {
 	int skip_default_update;
 	int mirror;
 	int prune;
+	int prune_tags;
 
 	const char *receivepack;
 	const char *uploadpack;
@@ -133,6 +134,8 @@ struct ref {
 #define REF_HEADS	(1u << 1)
 #define REF_TAGS	(1u << 2)
 
+void add_fetch_refspec(struct remote *remote, const char *ref);
+
 extern struct ref *find_ref_by_name(const struct ref *list, const char *name);
 
 struct ref *alloc_ref(const char *name);
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 840fd5ef02..1bca9d0bd2 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -662,6 +662,36 @@ test_configured_prune true  true  unset unset kept   pruned \
 test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
+
+# --prune-tags on its own does nothing, needs --prune as well, same
+# for for fetch.pruneTags without fetch.prune
+test_configured_prune unset unset unset unset kept kept     "--prune-tags"
+test_configured_prune unset unset unset unset kept kept     "origin --prune-tags"
+test_configured_prune unset unset true unset  kept kept     ""
+test_configured_prune unset unset unset true  kept kept     ""
+
+# These will prune the tags
+test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags"
+test_configured_prune unset unset unset unset pruned pruned "origin --prune --prune-tags"
+test_configured_prune true  unset true  unset pruned pruned ""
+test_configured_prune unset true  unset true  pruned pruned ""
+
+# Check that remote.<name>.pruneTags overrides fetch.pruneTags as with
+# remote.<name>.prune and fetch.prune
+test_configured_prune false true  false true  pruned pruned ""
+test_configured_prune true  false true  false kept   kept   ""
+
+# When --prune-tags is supplied it's ignored if an explict refspec is
+# given, same for the configuration options.
+test_configured_prune unset unset unset unset pruned kept \
+	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
+test_configured_prune unset unset true  unset pruned kept \
+	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
+test_configured_prune unset unset unset true pruned  kept \
+	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
+
+test_configured_prune true unset true unset pruned pruned   ""
+
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
 	test_commit oneside &&
-- 
2.15.1.424.g9478a66081

Re: [PATCH v3 11/11] fetch: add a --fetch-prune option and fetch.pruneTags config

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

>  git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"

Shouldn't the last arg be

 '^+\*refs/tags/\*:refs/tags/\*$'

instead?

Re: [PATCH v3 11/11] fetch: add a --fetch-prune option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

On Wed, Jan 24 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>>  git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"
>
> Shouldn't the last arg be
>
>  '^+\*refs/tags/\*:refs/tags/\*$'
>
> instead?

^+\* isn't a valid pattern.

This invocation is to replace both +refs/tags/[...] and refs/tags/[...]
with +refs/tags/[...]. This would be more specific, i.e. ^\+?

     git config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+?refs/tags/\*:refs/tags/\*$"

But I couldn't remeber offhand whether all regcomp() we use supports X?,
whereas they definitely do support X*.

Re: [PATCH v3 11/11] fetch: add a --fetch-prune option and fetch.pruneTags config

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Wed, Jan 24 2018, Junio C. Hamano jotted:
>
>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>
>>>  git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"
>>
>> Shouldn't the last arg be
>>
>>  '^+\*refs/tags/\*:refs/tags/\*$'
>>
>> instead?
>
> ^+\* isn't a valid pattern.

Yikes, sorry for the noise.  You did mean "there might be a plus
sign at the beginning, but there may not be".

[PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

Here's a v5 (correct subject line this time!). Many thanks to Eric for
a thorough review.

I'll spare you the per-patch changelog. These are all minor commit
message / doc / comment wording changes, with the exception of making
a bit of the test code better, and adding a \n for grep
portability. tbdiff at the end.

Ævar Arnfjörð Bjarmason (17):
  fetch: don't redundantly NULL something calloc() gave us
  fetch: trivially refactor assignment to ref_nr
  fetch: stop accessing "remote" variable indirectly
  remote: add a macro for "refs/tags/*:refs/tags/*"
  fetch tests: refactor in preparation for testing tag pruning
  fetch tests: re-arrange arguments for future readability
  fetch tests: add a tag to be deleted to the pruning tests
  fetch tests: test --prune and refspec interaction
  fetch tests: double quote a variable for interpolation
  fetch tests: expand case/esac for later change
  fetch tests: fetch <url> <spec> as well as fetch [<remote>]
  git fetch doc: add a new section to explain the ins & outs of pruning
  git remote doc: correct dangerous lies about what prune does
  git-fetch & config doc: link to the new PRUNING section
  fetch tests: add scaffolding for the new fetch.pruneTags
  fetch: add a --prune-tags option and fetch.pruneTags config
  fetch: make the --prune-tags work with <url>

 Documentation/config.txt               |  20 ++-
 Documentation/fetch-options.txt        |  17 ++-
 Documentation/git-fetch.txt            |  87 ++++++++++++
 Documentation/git-remote.txt           |  14 +-
 builtin/fetch.c                        |  54 ++++++--
 contrib/completion/git-completion.bash |   2 +-
 remote.c                               |  15 +++
 remote.h                               |   5 +
 t/t5510-fetch.sh                       | 238 +++++++++++++++++++++++++++------
 9 files changed, 391 insertions(+), 61 deletions(-)

tbdiff:

 1: da0992d97c =  1: da0992d97c fetch: don't redundantly NULL something calloc() gave us
 2: c781d18a29 !  2: 899430a3b2 fetch: trivially refactor assignment to ref_nr
    @@ -7,8 +7,8 @@
         "j" is, and "j" is only incremented in that loop, so this change isn't
         a logic error.
         
    -    This change makes a subsequent change which splits the incrementing of
    -    "ref_nr" into two blocks.
    +    This change simplifies a subsequent change, which will split the
    +    incrementing of "ref_nr" into two blocks.
         
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
 3: 1203ef6e35 =  3: 98e3a28bdf fetch: stop accessing "remote" variable indirectly
 4: 1d7956c444 =  4: 384c1fc318 remote: add a macro for "refs/tags/*:refs/tags/*"
 5: a0dc3eb024 =  5: ec92777861 fetch tests: refactor in preparation for testing tag pruning
 6: 7ed1561e3d =  6: 1c23526223 fetch tests: re-arrange arguments for future readability
 7: 6dbf0e688d =  7: 59abe07b71 fetch tests: add a tag to be deleted to the pruning tests
 8: 9fc3589793 =  8: af7acef671 fetch tests: test --prune and refspec interaction
 9: a4487f9389 =  9: cb72187362 fetch tests: double quote a variable for interpolation
10: b8c07e2d42 = 10: 300c1c0985 fetch tests: expand case/esac for later change
11: 6341131ee0 ! 11: 550e7df594 fetch tests: fetch <url> <spec> as well as fetch [<remote>]
    @@ -60,15 +60,12 @@
     -	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
     +	mode=$6
     +
    -+	if ! test -e prune-type-setup-done
    ++	if test -z "$cmdline_setup"
     +	then
    -+		test_expect_success 'prune_type setup' '
    -+			git -C one config remote.origin.url >one.remote-url &&
    -+			git -C one config remote.origin.fetch >one.remote-fetch &&
    -+			remote_url="file://$(cat one.remote-url)" &&
    -+			remote_fetch="$(cat one.remote-fetch)" &&
    -+			cmdline_setup="\"$remote_url\" \"$remote_fetch\"" &&
    -+			touch prune-type-setup-done
    ++		test_expect_success 'setup cmdline_setup variable for subsequent test' '
    ++			remote_url="file://$(git -C one config remote.origin.url)" &&
    ++			remote_fetch="$(git -C one config remote.origin.fetch)" &&
    ++			cmdline_setup="\"$remote_url\" \"$remote_fetch\""
     +		'
     +	fi
     +
12: 824c3ed4c1 ! 12: 273f4c603f git fetch doc: add a new section to explain the ins & outs of pruning
    @@ -32,7 +32,7 @@
     +<commit>` needlessly verbose, as well as impacting anything else
     +that'll work with the complete set of known references.
     +
    -+These remote tracking references can be deleted as a one-off with
    ++These remote-tracking references can be deleted as a one-off with
     +either of:
     +
     +------------------------------------------------
    @@ -44,13 +44,13 @@
     +------------------------------------------------
     +
     +To prune references as part of your normal workflow without needing to
    -+remember to run that set `fetch.prune` globally, or
    ++remember to run that, set `fetch.prune` globally, or
     +`remote.<name>.prune` per-remote in the config. See
     +linkgit:git-config[1].
     +
     +Here's where things get tricky and more specific. The pruning feature
     +doesn't actually care about branches, instead it'll prune local <->
    -+remote references as a function of the refspec of the remote (see
    ++remote-references as a function of the refspec of the remote (see
     +`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).
     +
     +Therefore if the refspec for the remote includes
13: 59de997172 ! 13: ada313b523 git remote doc: correct dangerous lies about what prune does
    @@ -11,7 +11,7 @@
         oldest version I could be bothered to test).
         
         E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of
    -    the remote it'll delete all local tags <name> doesn't know about.
    +    the remote, it'll delete all local tags <name> doesn't know about.
         
         Instead, briefly give the reader just enough of a hint that this
         option might constitute a shotgun aimed at their foot, and point them
    @@ -36,11 +36,11 @@
     -These stale branches have already been removed from the remote repository
     -referenced by <name>, but are still locally available in
     -"remotes/<name>".
    -+Deletes stale references associated with <name>. By default stale
    -+remote-tracking branches under <name>, but depending on global
    -+configuration and the configuration of the remote we might even prune
    -+local tags that haven't been pushed there. Equivalent to `git fetch
    -+--prune <name>`, except that no new references will be fetched.
    ++Deletes stale references associated with <name>. By default, stale
    ++remote-tracking branches under <name> are deleted, but depending on
    ++global configuration and the configuration of the remote we might even
    ++prune local tags that haven't been pushed there. Equivalent to `git
    ++fetch --prune <name>`, except that no new references will be fetched.
     ++
     +See the PRUNING section of linkgit:git-fetch[1] for what it'll prune
     +depending on various configuration.
14: 5dfc9177bb = 14: eeb9102d9f git-fetch & config doc: link to the new PRUNING section
15: fc0890eb9c ! 15: 6c1facccda fetch tests: add scaffolding for the new fetch.pruneTags
    @@ -28,7 +28,7 @@
     +	cmdline=$7
     +	mode=$8
      
    - 	if ! test -e prune-type-setup-done
    + 	if test -z "$cmdline_setup"
      	then
     @@
      		cmdline="$new_cmdline"
16: 03b1393ea2 ! 16: 1d8d6e29fc fetch: add a --prune-tags option and fetch.pruneTags config
    @@ -1,9 +1,10 @@
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    -    fetch: add a --fetch-prune option and fetch.pruneTags config
    -    
    -    Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
    -    config option. This allows for doing any of:
    +    fetch: add a --prune-tags option and fetch.pruneTags config
    +    
    +    Add a --prune-tags option to git-fetch, along with fetch.pruneTags
    +    config option and a -P shorthand (-p is --prune). This allows for
    +    doing any of:
         
             git fetch -p -P
             git fetch --prune --prune-tags
    @@ -366,7 +367,7 @@
     +		if test "$fetch_prune_tags" = 'true' ||
     +		   test "$remote_origin_prune_tags" = 'true'
     +		then
    -+			if ! printf '%s' "$cmdline" | grep -q refs/remotes/origin/
    ++			if ! printf '%s' "$cmdline\n" | grep -q refs/remotes/origin/
     +			then
     +				new_cmdline="$new_cmdline refs/tags/*:refs/tags/*"
     +			fi
    @@ -396,7 +397,7 @@
     +test_configured_prune false true  false true  pruned pruned ""
     +test_configured_prune true  false true  false kept   kept   ""
     +
    -+# When --prune-tags is supplied it's ignored if an explict refspec is
    ++# When --prune-tags is supplied it's ignored if an explicit refspec is
     +# given, same for the configuration options.
     +test_configured_prune unset unset unset unset pruned kept \
     +	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
    @@ -405,13 +406,15 @@
     +test_configured_prune unset unset unset true pruned  kept \
     +	"--prune origin +refs/heads/*:refs/remotes/origin/*"
     +
    -+# Pruning that also takes place if s!origin!<file:// url of remote>!,
    -+# or otherwise uses the file://-specific codepath. However, because
    -+# there's no implicit +refs/heads/*:refs/remotes/origin/* refspec and
    -+# supplying it on the command-line negate --prune-tags the branches
    -+# will not be pruned.
    -+test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "name"
    -+test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "link"
    ++# Pruning that also takes place if a file:// url replaces a named
    ++# remote, with the exception of --prune-tags on the command-line
    ++# (arbitrary limitation).
    ++#
    ++# However, because there's no implicit
    ++# +refs/heads/*:refs/remotes/origin/* refspec and supplying it on the
    ++# command-line negates --prune-tags, the branches will not be pruned.
    ++test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "name"
    ++test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "link"
     +test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
     +test_configured_prune_type unset unset unset unset kept   kept   "origin --prune --prune-tags" "link"
     +test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name"
    @@ -427,9 +430,8 @@
     +test_configured_prune_type unset  true unset true  pruned pruned "origin" "name"
     +test_configured_prune_type unset  true unset true  kept   pruned "origin" "link"
     +
    -+# When all remote.origin.fetch settings are deleted a --prune
    -+# --prune-tags still implicitly supplies refs/tags/*:refs/tags/* so
    -+# tags, but not tracking branches, will be deleted.
    ++# Interaction between --prune-tags and no "fetch" config in the remote
    ++# at all.
     +test_expect_success 'remove remote.origin.fetch "one"' '
     +	(
     +		cd one &&
17: f0dca21626 ! 17: ddcd576c5c fetch: make the --prune-tags work with <url>
    @@ -1,6 +1,6 @@
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    -    fetch: make the --fetch-prune work with <url>
    +    fetch: make the --prune-tags work with <url>
         
         Make the new --prune-tags option work properly when git-fetch is
         invoked with a <url> parameter instead of a <remote name>
    @@ -91,8 +91,18 @@
     --- a/t/t5510-fetch.sh
     +++ b/t/t5510-fetch.sh
     @@
    - test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "name"
    - test_configured_prune_type unset unset unset unset kept kept     "origin --prune-tags" "link"
    + 	"--prune origin +refs/heads/*:refs/remotes/origin/*"
    + 
    + # Pruning that also takes place if a file:// url replaces a named
    +-# remote, with the exception of --prune-tags on the command-line
    +-# (arbitrary limitation).
    +-#
    +-# However, because there's no implicit
    ++# remote. However, because there's no implicit
    + # +refs/heads/*:refs/remotes/origin/* refspec and supplying it on the
    + # command-line negates --prune-tags, the branches will not be pruned.
    + test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "name"
    + test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "link"
      test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
     -test_configured_prune_type unset unset unset unset kept   kept   "origin --prune --prune-tags" "link"
     +test_configured_prune_type unset unset unset unset kept   pruned "origin --prune --prune-tags" "link"
    @@ -103,6 +113,18 @@
      test_configured_prune_type unset unset true  unset kept   pruned "--prune origin" "link"
      test_configured_prune_type unset unset unset true  pruned pruned "--prune origin" "name"
     @@
    + test_configured_prune_type unset  true unset true  pruned pruned "origin" "name"
    + test_configured_prune_type unset  true unset true  kept   pruned "origin" "link"
    + 
    +-# Interaction between --prune-tags and no "fetch" config in the remote
    +-# at all.
    ++# When all remote.origin.fetch settings are deleted a --prune
    ++# --prune-tags still implicitly supplies refs/tags/*:refs/tags/* so
    ++# tags, but not tracking branches, will be deleted.
    + test_expect_success 'remove remote.origin.fetch "one"' '
    + 	(
    + 		cd one &&
    +@@
      	)
      '
      test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name"

-- 
2.15.1.424.g9478a66081

Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Here's a v5 (correct subject line this time!). Many thanks to Eric for
> a thorough review.

We haven't seen any comments on this round.  Is everybody happy?

I do not have a strong opinion on the new feature, either for or
against.  I didn't find anything majorly questionable in the
execution, though, so...

Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 22 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> Here's a v5 (correct subject line this time!). Many thanks to Eric for
>> a thorough review.
>
> We haven't seen any comments on this round.  Is everybody happy?
>
> I do not have a strong opinion on the new feature, either for or
> against.  I didn't find anything majorly questionable in the
> execution, though, so...

I've been running that here on thousands of boxes (that are actively
using it) for 2 weeks now without issue. Would be great to have it
merged down & in 2.17.

Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Thu, Feb 22 2018, Junio C. Hamano jotted:
>
>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>
>>> Here's a v5 (correct subject line this time!). Many thanks to Eric for
>>> a thorough review.
>>
>> We haven't seen any comments on this round.  Is everybody happy?
>>
>> I do not have a strong opinion on the new feature, either for or
>> against.  I didn't find anything majorly questionable in the
>> execution, though, so...
>
> I've been running that here on thousands of boxes (that are actively
> using it) for 2 weeks now without issue. Would be great to have it
> merged down & in 2.17.

If those thousands of boxes are all employing one specific workflow
that is helped by these changes, and the workflow is that other
people do not care about (or even worse, actively do not want to let
their junior project members to use without thinking), then a
data-point from the original author does not amount to much ;-)

Let's see how others find it useful and/or if the changed code gets
in the way of others (I am not absolutely sure if the changes are
free of regression to existing users who do not use the new
feature).

Thanks.

Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 22 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Thu, Feb 22 2018, Junio C. Hamano jotted:
>>
>>> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>>>
>>>> Here's a v5 (correct subject line this time!). Many thanks to Eric for
>>>> a thorough review.
>>>
>>> We haven't seen any comments on this round.  Is everybody happy?
>>>
>>> I do not have a strong opinion on the new feature, either for or
>>> against.  I didn't find anything majorly questionable in the
>>> execution, though, so...
>>
>> I've been running that here on thousands of boxes (that are actively
>> using it) for 2 weeks now without issue. Would be great to have it
>> merged down & in 2.17.
>
> If those thousands of boxes are all employing one specific workflow
> that is helped by these changes, and the workflow is that other
> people do not care about (or even worse, actively do not want to let
> their junior project members to use without thinking), then a
> data-point from the original author does not amount to much ;-)

Of course, I should have been clearer. I just meant to chime in with the
datapoint that I'm fairly sure this doesn't have any serious bugs given
the wide internal testing it's gotten.

> Let's see how others find it useful and/or if the changed code gets
> in the way of others (I am not absolutely sure if the changes are
> free of regression to existing users who do not use the new
> feature).

I think if you're on the fence about merging it down (and others don't
chime in saying the want it / like it) it makes sense to merge down
1-14/17 and we could discard 15-17/17 for now for a later re-submission
and discussion once the earlier part of the series lands in master.

The earlier part of the series is just trivial code changes that don't
change any functionality, more documentation for how existing
functionality works, and more thorough testing of existing
functionality.

Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>> Let's see how others find it useful and/or if the changed code gets
>> in the way of others (I am not absolutely sure if the changes are
>> free of regression to existing users who do not use the new
>> feature).
>
> I think if you're on the fence about merging it down (and others don't
> chime in saying the want it / like it) it makes sense to merge down
> 1-14/17 and we could discard 15-17/17 for now for a later re-submission
> and discussion once the earlier part of the series lands in master.

Or we merge everything and let people discover glitches and
complain.  Reverting the last pieces can wait until then ;-).

Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

On Fri, Feb 23 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>>> Let's see how others find it useful and/or if the changed code gets
>>> in the way of others (I am not absolutely sure if the changes are
>>> free of regression to existing users who do not use the new
>>> feature).
>>
>> I think if you're on the fence about merging it down (and others don't
>> chime in saying the want it / like it) it makes sense to merge down
>> 1-14/17 and we could discard 15-17/17 for now for a later re-submission
>> and discussion once the earlier part of the series lands in master.
>
> Or we merge everything and let people discover glitches and
> complain.  Reverting the last pieces can wait until then ;-).

Thanks, that obviously works for me.

[PATCH v5 01/17] fetch: don't redundantly NULL something calloc() gave us

[Ævar Arnfjörð Bjarmason]

Stop redundantly NULL-ing the last element of the refs structure,
which was retrieved via calloc(), and is thus guaranteed to be
pre-NULL'd.

This code dates back to b888d61c83 ("Make fetch a builtin",
2007-09-10), where wasn't any reason to do this back then either, it's
just boilerplate left over from when git-fetch was initially
introduced.

The motivation for this change was to make a subsequent change which
would also modify the refs variable smaller, since it won't have to
copy this redundant "NULL the last + 1 item" pattern.

We may not end up keeping that change, but as this pattern is still
pointless, so let's fix it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7bbcd26faf..b34665db9e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			} else
 				refs[j++] = argv[i];
 		}
-		refs[j] = NULL;
 		ref_nr = j;
 	}
 
-- 
2.15.1.424.g9478a66081

[PATCH v5 02/17] fetch: trivially refactor assignment to ref_nr

[Ævar Arnfjörð Bjarmason]

Trivially refactor an assignment to make a subsequent patch
smaller. The "ref_nr" variable is initialized to 0 earlier, just as
"j" is, and "j" is only incremented in that loop, so this change isn't
a logic error.

This change simplifies a subsequent change, which will split the
incrementing of "ref_nr" into two blocks.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index b34665db9e..72085e30b9 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1301,8 +1301,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 						    argv[i], argv[i]);
 			} else
 				refs[j++] = argv[i];
+			ref_nr++;
 		}
-		ref_nr = j;
 	}
 
 	sigchain_push_common(unlock_pack_on_signal);
-- 
2.15.1.424.g9478a66081

[PATCH v5 03/17] fetch: stop accessing "remote" variable indirectly

[Ævar Arnfjörð Bjarmason]

Access the "remote" variable passed to the fetch_one() directly rather
than through the gtransport wrapper struct constructed in this
function for other purposes.

This makes the code more readable, as it's now obvious that the remote
struct doesn't somehow get munged by the prepare_transport() function
above, which takes the "remote" struct as an argument and constructs
the "gtransport" struct, containing among other things the "remote"
struct.

A subsequent change will copy this pattern to access a new
remote->prune_tags field, but without the use of the gtransport
variable. It's useful once that change lands to see that the two
pieces of code behave exactly the same.

This pattern of accessing the container struct was added in
737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this
code was initially introduced.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 72085e30b9..a7705bc150 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 
 	if (prune < 0) {
 		/* no command line request */
-		if (0 <= gtransport->remote->prune)
-			prune = gtransport->remote->prune;
+		if (0 <= remote->prune)
+			prune = remote->prune;
 		else if (0 <= fetch_prune_config)
 			prune = fetch_prune_config;
 		else
-- 
2.15.1.424.g9478a66081

[PATCH v5 04/17] remote: add a macro for "refs/tags/*:refs/tags/*"

[Ævar Arnfjörð Bjarmason]

Add a macro with the refspec string "refs/tags/*:refs/tags/*". There's
been a pre-defined struct version of this since e0aaa29ff3 ("Have a
constant extern refspec for "--tags"", 2008-04-17), but nothing that
could be passed to e.g. add_fetch_refspec().

This will be used in subsequent commits to avoid hardcoding this
string in multiple places.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 remote.c | 1 +
 remote.h | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/remote.c b/remote.c
index 4e93753e19..356c123e3e 100644
--- a/remote.c
+++ b/remote.c
@@ -22,6 +22,7 @@ static struct refspec s_tag_refspec = {
 	"refs/tags/*"
 };
 
+/* See TAG_REFSPEC for the string version */
 const struct refspec *tag_refspec = &s_tag_refspec;
 
 struct counted_string {
diff --git a/remote.h b/remote.h
index 1f6611be21..80fea6dd11 100644
--- a/remote.h
+++ b/remote.h
@@ -297,4 +297,6 @@ extern int parseopt_push_cas_option(const struct option *, const char *arg, int
 extern int is_empty_cas(const struct push_cas_option *);
 void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *);
 
+#define TAG_REFSPEC "refs/tags/*:refs/tags/*"
+
 #endif
-- 
2.15.1.424.g9478a66081

[PATCH v5 05/17] fetch tests: refactor in preparation for testing tag pruning

[Ævar Arnfjörð Bjarmason]

In a subsequent commit this function will learn to test for tag
pruning, prepare for that by making space for more variables, and
making it clear that "expected" here refers to branches.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 668c54be41..11da97f9b7 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -549,9 +549,12 @@ set_config_tristate () {
 }
 
 test_configured_prune () {
-	fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4
+	fetch_prune=$1
+	remote_origin_prune=$2
+	cmdline=$3
+	expected_branch=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -572,7 +575,7 @@ test_configured_prune () {
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
 			git fetch $cmdline &&
-			case "$expected" in
+			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
 				;;
-- 
2.15.1.424.g9478a66081

[PATCH v5 06/17] fetch tests: re-arrange arguments for future readability

[Ævar Arnfjörð Bjarmason]

Re-arrange the arguments to the test_configured_prune() function used
in this test to pass the arguments to --fetch last. A subsequent
change will test for more elaborate fetch arguments, including long
refspecs. It'll be more readable to be able to wrap those on a new
line of their own.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 44 insertions(+), 38 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 11da97f9b7..ab8b25344d 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,10 +551,10 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	cmdline=$3
-	expected_branch=$4
+	expected_branch=$3
+	cmdline=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -587,41 +587,47 @@ test_configured_prune () {
 	'
 }
 
-test_configured_prune unset unset ""		kept
-test_configured_prune unset unset "--no-prune"	kept
-test_configured_prune unset unset "--prune"	pruned
-
-test_configured_prune false unset ""		kept
-test_configured_prune false unset "--no-prune"	kept
-test_configured_prune false unset "--prune"	pruned
-
-test_configured_prune true  unset ""		pruned
-test_configured_prune true  unset "--prune"	pruned
-test_configured_prune true  unset "--no-prune"	kept
-
-test_configured_prune unset false ""		kept
-test_configured_prune unset false "--no-prune"	kept
-test_configured_prune unset false "--prune"	pruned
-
-test_configured_prune false false ""		kept
-test_configured_prune false false "--no-prune"	kept
-test_configured_prune false false "--prune"	pruned
-
-test_configured_prune true  false ""		kept
-test_configured_prune true  false "--prune"	pruned
-test_configured_prune true  false "--no-prune"	kept
-
-test_configured_prune unset true  ""		pruned
-test_configured_prune unset true  "--no-prune"	kept
-test_configured_prune unset true  "--prune"	pruned
-
-test_configured_prune false true  ""		pruned
-test_configured_prune false true  "--no-prune"	kept
-test_configured_prune false true  "--prune"	pruned
-
-test_configured_prune true  true  ""		pruned
-test_configured_prune true  true  "--prune"	pruned
-test_configured_prune true  true  "--no-prune"	kept
+# $1 config: fetch.prune
+# $2 config: remote.<name>.prune
+# $3 expect: branch to be pruned?
+# $4 git-fetch $cmdline:
+#
+#                     $1    $2    $3     $4
+test_configured_prune unset unset kept   ""
+test_configured_prune unset unset kept   "--no-prune"
+test_configured_prune unset unset pruned "--prune"
+
+test_configured_prune false unset kept   ""
+test_configured_prune false unset kept   "--no-prune"
+test_configured_prune false unset pruned "--prune"
+
+test_configured_prune true  unset pruned ""
+test_configured_prune true  unset pruned "--prune"
+test_configured_prune true  unset kept   "--no-prune"
+
+test_configured_prune unset false kept   ""
+test_configured_prune unset false kept   "--no-prune"
+test_configured_prune unset false pruned "--prune"
+
+test_configured_prune false false kept   ""
+test_configured_prune false false kept   "--no-prune"
+test_configured_prune false false pruned "--prune"
+
+test_configured_prune true  false kept   ""
+test_configured_prune true  false pruned "--prune"
+test_configured_prune true  false kept   "--no-prune"
+
+test_configured_prune unset true  pruned ""
+test_configured_prune unset true  kept   "--no-prune"
+test_configured_prune unset true  pruned "--prune"
+
+test_configured_prune false true  pruned ""
+test_configured_prune false true  kept   "--no-prune"
+test_configured_prune false true  pruned "--prune"
+
+test_configured_prune true  true  pruned ""
+test_configured_prune true  true  pruned "--prune"
+test_configured_prune true  true  kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v5 07/17] fetch tests: add a tag to be deleted to the pruning tests

[Ævar Arnfjörð Bjarmason]

Add a tag to be deleted to the fetch --prune tests. The tag is always
kept for now, which is the expected behavior, but now I can add a test
for tag pruning in a later commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 53 insertions(+), 40 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index ab8b25344d..fad65bd885 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -552,21 +552,25 @@ test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
 	expected_branch=$3
-	cmdline=$4
+	expected_tag=$4
+	cmdline=$5
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
+		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
 			test_unconfig remote.origin.prune &&
 			git fetch &&
-			git rev-parse --verify refs/remotes/origin/newbranch
+			git rev-parse --verify refs/remotes/origin/newbranch &&
+			git rev-parse --verify refs/tags/newtag
 		) &&
 
 		# now remove it
 		git branch -d newbranch &&
+		git tag -d newtag &&
 
 		# then test
 		(
@@ -582,6 +586,14 @@ test_configured_prune () {
 			kept)
 				git rev-parse --verify refs/remotes/origin/newbranch
 				;;
+			esac &&
+			case "$expected_tag" in
+			pruned)
+				test_must_fail git rev-parse --verify refs/tags/newtag
+				;;
+			kept)
+				git rev-parse --verify refs/tags/newtag
+				;;
 			esac
 		)
 	'
@@ -590,44 +602,45 @@ test_configured_prune () {
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
 # $3 expect: branch to be pruned?
-# $4 git-fetch $cmdline:
+# $4 expect: tag to be pruned?
+# $5 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4
-test_configured_prune unset unset kept   ""
-test_configured_prune unset unset kept   "--no-prune"
-test_configured_prune unset unset pruned "--prune"
-
-test_configured_prune false unset kept   ""
-test_configured_prune false unset kept   "--no-prune"
-test_configured_prune false unset pruned "--prune"
-
-test_configured_prune true  unset pruned ""
-test_configured_prune true  unset pruned "--prune"
-test_configured_prune true  unset kept   "--no-prune"
-
-test_configured_prune unset false kept   ""
-test_configured_prune unset false kept   "--no-prune"
-test_configured_prune unset false pruned "--prune"
-
-test_configured_prune false false kept   ""
-test_configured_prune false false kept   "--no-prune"
-test_configured_prune false false pruned "--prune"
-
-test_configured_prune true  false kept   ""
-test_configured_prune true  false pruned "--prune"
-test_configured_prune true  false kept   "--no-prune"
-
-test_configured_prune unset true  pruned ""
-test_configured_prune unset true  kept   "--no-prune"
-test_configured_prune unset true  pruned "--prune"
-
-test_configured_prune false true  pruned ""
-test_configured_prune false true  kept   "--no-prune"
-test_configured_prune false true  pruned "--prune"
-
-test_configured_prune true  true  pruned ""
-test_configured_prune true  true  pruned "--prune"
-test_configured_prune true  true  kept   "--no-prune"
+#                     $1    $2    $3     $4     $5
+test_configured_prune unset unset kept   kept   ""
+test_configured_prune unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset pruned kept   "--prune"
+
+test_configured_prune false unset kept   kept   ""
+test_configured_prune false unset kept   kept   "--no-prune"
+test_configured_prune false unset pruned kept   "--prune"
+
+test_configured_prune true  unset pruned kept   ""
+test_configured_prune true  unset pruned kept   "--prune"
+test_configured_prune true  unset kept   kept   "--no-prune"
+
+test_configured_prune unset false kept   kept   ""
+test_configured_prune unset false kept   kept   "--no-prune"
+test_configured_prune unset false pruned kept   "--prune"
+
+test_configured_prune false false kept   kept   ""
+test_configured_prune false false kept   kept   "--no-prune"
+test_configured_prune false false pruned kept   "--prune"
+
+test_configured_prune true  false kept   kept   ""
+test_configured_prune true  false pruned kept   "--prune"
+test_configured_prune true  false kept   kept   "--no-prune"
+
+test_configured_prune unset true  pruned kept   ""
+test_configured_prune unset true  kept   kept   "--no-prune"
+test_configured_prune unset true  pruned kept   "--prune"
+
+test_configured_prune false true  pruned kept   ""
+test_configured_prune false true  kept   kept   "--no-prune"
+test_configured_prune false true  pruned kept   "--prune"
+
+test_configured_prune true  true  pruned kept   ""
+test_configured_prune true  true  pruned kept   "--prune"
+test_configured_prune true  true  kept   kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v5 08/17] fetch tests: test --prune and refspec interaction

[Ævar Arnfjörð Bjarmason]

Add a test for the interaction between explicitly provided refspecs
and fetch.prune.

There's no point in adding this boilerplate to every combination of
unset/false/true, it's instructive and sufficient to show that no
matter if the variable is unset, false or true the refspec on the
command-line overrides any configuration variable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index fad65bd885..dacdb8759c 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -609,6 +609,10 @@ test_configured_prune () {
 test_configured_prune unset unset kept   kept   ""
 test_configured_prune unset unset kept   kept   "--no-prune"
 test_configured_prune unset unset pruned kept   "--prune"
+test_configured_prune unset unset kept   pruned \
+	"--prune origin refs/tags/*:refs/tags/*"
+test_configured_prune unset unset pruned pruned \
+	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_configured_prune false unset kept   kept   ""
 test_configured_prune false unset kept   kept   "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept   "--prune"
 test_configured_prune false false kept   kept   ""
 test_configured_prune false false kept   kept   "--no-prune"
 test_configured_prune false false pruned kept   "--prune"
+test_configured_prune false false kept   pruned \
+	"--prune origin refs/tags/*:refs/tags/*"
+test_configured_prune false false pruned pruned \
+	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_configured_prune true  false kept   kept   ""
 test_configured_prune true  false pruned kept   "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true  pruned kept   "--prune"
 test_configured_prune true  true  pruned kept   ""
 test_configured_prune true  true  pruned kept   "--prune"
 test_configured_prune true  true  kept   kept   "--no-prune"
+test_configured_prune true  true  kept   pruned \
+	"--prune origin refs/tags/*:refs/tags/*"
+test_configured_prune true  true  pruned pruned \
+	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v5 09/17] fetch tests: double quote a variable for interpolation

[Ævar Arnfjörð Bjarmason]

If the $cmdline variable contains arguments with spaces they won't be
interpolated correctly, since the body of the test is single quoted,
and because test-lib.sh does its own eval().

This will be used in a subsequent commit to pass arguments that need
to be quoted to git-fetch, i.e. a file:// path to fetch, which will
have a space in it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dacdb8759c..88d38e0819 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -578,7 +578,7 @@ test_configured_prune () {
 			set_config_tristate fetch.prune $fetch_prune &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
-			git fetch $cmdline &&
+			git fetch '"$cmdline"' &&
 			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
-- 
2.15.1.424.g9478a66081

[PATCH v5 10/17] fetch tests: expand case/esac for later change

[Ævar Arnfjörð Bjarmason]

Expand a compact case/esac statement for a later change that'll add
more logic to the body of the "*" case. This is a whitespace-only
change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 88d38e0819..dfc749f576 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -543,8 +543,12 @@ test_expect_success "should be able to fetch with duplicate refspecs" '
 set_config_tristate () {
 	# var=$1 val=$2
 	case "$2" in
-	unset)  test_unconfig "$1" ;;
-	*)	git config "$1" "$2" ;;
+	unset)
+		test_unconfig "$1"
+		;;
+	*)
+		git config "$1" "$2"
+		;;
 	esac
 }
 
-- 
2.15.1.424.g9478a66081

[PATCH v5 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]

[Ævar Arnfjörð Bjarmason]

When a remote URL is supplied on the command-line the internals of the
fetch are different, in particular the code in get_ref_map(). An
earlier version of the subsequent fetch.pruneTags patch hid a segfault
because the difference wasn't tested for.

Now all the tests are run as both of the variants of:

    git fetch
    git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)

I'm using -c because while the [fetch] config just set by
set_config_tristate will be picked up, the remote.origin.* config
won't override it as intended.

Work around that and turn this into a purely command-line test by
always setting the variables on the command-line, and translate any
setting of remote.origin.X into fetch.X.

The reason for choosing the names "name" and "link" as opposed to
e.g. "named" and "url" is because they're the same length, which makes
the test output easier to read as it will be aligned.

Due to shellscript quoting madness it's not worthwhile to do all of
this within a test_expect_success, but do the parts that can easily be
done there, including the one-time setting of variables that don't
change between runs to be used by subsequent runs in the 'prune_type
setup' test.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 44 ++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 40 insertions(+), 4 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dfc749f576..9c87fa6106 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -548,18 +548,49 @@ set_config_tristate () {
 		;;
 	*)
 		git config "$1" "$2"
+		key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
+		git_fetch_c="$git_fetch_c -c $key=$2"
 		;;
 	esac
 }
 
 test_configured_prune () {
+	test_configured_prune_type "$@" "name"
+	test_configured_prune_type "$@" "link"
+}
+
+test_configured_prune_type () {
 	fetch_prune=$1
 	remote_origin_prune=$2
 	expected_branch=$3
 	expected_tag=$4
 	cmdline=$5
-
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+	mode=$6
+
+	if test -z "$cmdline_setup"
+	then
+		test_expect_success 'setup cmdline_setup variable for subsequent test' '
+			remote_url="file://$(git -C one config remote.origin.url)" &&
+			remote_fetch="$(git -C one config remote.origin.fetch)" &&
+			cmdline_setup="\"$remote_url\" \"$remote_fetch\""
+		'
+	fi
+
+	if test "$mode" = 'link'
+	then
+		new_cmdline=""
+
+		if test "$cmdline" = ""
+		then
+			new_cmdline=$cmdline_setup
+		else
+			new_cmdline=$(printf "%s" "$cmdline" | perl -pe 's[origin(?!/)]["'"$remote_url"'"]g')
+		fi
+
+		cmdline="$new_cmdline"
+	fi
+
+	test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		git tag -f newtag &&
@@ -567,7 +598,7 @@ test_configured_prune () {
 			cd one &&
 			test_unconfig fetch.prune &&
 			test_unconfig remote.origin.prune &&
-			git fetch &&
+			git fetch '"$cmdline_setup"' &&
 			git rev-parse --verify refs/remotes/origin/newbranch &&
 			git rev-parse --verify refs/tags/newtag
 		) &&
@@ -579,10 +610,15 @@ test_configured_prune () {
 		# then test
 		(
 			cd one &&
+			git_fetch_c="" &&
 			set_config_tristate fetch.prune $fetch_prune &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
-			git fetch '"$cmdline"' &&
+			if test "$mode" != "link"
+			then
+				git_fetch_c=""
+			fi &&
+			git$git_fetch_c fetch '"$cmdline"' &&
 			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
-- 
2.15.1.424.g9478a66081

[PATCH v5 12/17] git fetch doc: add a new section to explain the ins & outs of pruning

[Ævar Arnfjörð Bjarmason]

Add a new section to canonically explain how remote reference pruning
works, and how users should be careful about using it in conjunction
with tag refspecs in particular.

A subsequent commit will update the git-remote documentation to refer
to this section, and details the motivation for writing this in the
first place.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index b153aefa68..e94bcfb8c3 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be
 overridden by giving the `--refmap=<refspec>` parameter(s) on the
 command line.
 
+PRUNING
+-------
+
+Git has a default disposition of keeping data unless it's explicitly
+thrown away; this extends to holding onto local references to branches
+on remotes that have themselves deleted those branches.
+
+If left to accumulate, these stale references might make performance
+worse on big and busy repos that have a lot of branch churn, and
+e.g. make the output of commands like `git branch -a --contains
+<commit>` needlessly verbose, as well as impacting anything else
+that'll work with the complete set of known references.
+
+These remote-tracking references can be deleted as a one-off with
+either of:
+
+------------------------------------------------
+# While fetching
+$ git fetch --prune <name>
+
+# Only prune, don't fetch
+$ git remote prune <name>
+------------------------------------------------
+
+To prune references as part of your normal workflow without needing to
+remember to run that, set `fetch.prune` globally, or
+`remote.<name>.prune` per-remote in the config. See
+linkgit:git-config[1].
+
+Here's where things get tricky and more specific. The pruning feature
+doesn't actually care about branches, instead it'll prune local <->
+remote-references as a function of the refspec of the remote (see
+`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).
+
+Therefore if the refspec for the remote includes
+e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch
+--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote
+tracking branches that are deleted, but any local tag that doesn't
+exist on the remote.
+
+This might not be what you expect, i.e. you want to prune remote
+`<name>`, but also explicitly fetch tags from it, so when you fetch
+from it you delete all your local tags, most of which may not have
+come from the `<name>` remote in the first place.
+
+So be careful when using this with a refspec like
+`refs/tags/*:refs/tags/*`, or any other refspec which might map
+references from multiple remotes to the same local namespace.
+
 OUTPUT
 ------
 
-- 
2.15.1.424.g9478a66081

[PATCH v5 13/17] git remote doc: correct dangerous lies about what prune does

[Ævar Arnfjörð Bjarmason]

The "git remote prune <name>" command uses the same machinery as "git
fetch <name> --prune", and shares all the same caveats, but its
documentation has suggested that it'll just "delete stale
remote-tracking branches under <name>".

This isn't true, and hasn't been true since at least v1.8.5.6 (the
oldest version I could be bothered to test).

E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of
the remote, it'll delete all local tags <name> doesn't know about.

Instead, briefly give the reader just enough of a hint that this
option might constitute a shotgun aimed at their foot, and point them
to the new PRUNING section in the git-fetch documentation which
explains all the nuances of what this facility does.

See "[BUG] git remote prune removes local tags, depending on fetch
config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com)
by Michael Giuffrida for the initial report.

Reported-by: Michael Giuffrida <michaelpg@chromium.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-remote.txt | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-remote.txt b/Documentation/git-remote.txt
index 577b969c1b..4feddc0293 100644
--- a/Documentation/git-remote.txt
+++ b/Documentation/git-remote.txt
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with
 
 'prune'::
 
-Deletes all stale remote-tracking branches under <name>.
-These stale branches have already been removed from the remote repository
-referenced by <name>, but are still locally available in
-"remotes/<name>".
+Deletes stale references associated with <name>. By default, stale
+remote-tracking branches under <name> are deleted, but depending on
+global configuration and the configuration of the remote we might even
+prune local tags that haven't been pushed there. Equivalent to `git
+fetch --prune <name>`, except that no new references will be fetched.
++
+See the PRUNING section of linkgit:git-fetch[1] for what it'll prune
+depending on various configuration.
 +
 With `--dry-run` option, report what branches will be pruned, but do not
 actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the
 configuration parameter remote.<name>.skipDefaultUpdate set to true will
 be updated.  (See linkgit:git-config[1]).
 +
-With `--prune` option, prune all the remotes that are updated.
+With `--prune` option, run pruning against all the remotes that are updated.
 
 
 DISCUSSION
-- 
2.15.1.424.g9478a66081

[PATCH v5 14/17] git-fetch & config doc: link to the new PRUNING section

[Ævar Arnfjörð Bjarmason]

Amend the documentation for fetch.prune, fetch.<name>.prune and
--prune to link to the recently added PRUNING section.

I'd have liked to link directly to it with "<<PRUNING>>" from
fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt
also includes it, but doesn't include that option). However making a
reference across files yields this error:

    [...]/Documentation/git-fetch.xml:226: element xref: validity
    error : IDREF attribute linkend references an unknown ID "PRUNING"

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 6 +++++-
 Documentation/fetch-options.txt | 3 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b..0f27af5760 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1398,7 +1398,8 @@ fetch.unpackLimit::
 
 fetch.prune::
 	If true, fetch will automatically behave as if the `--prune`
-	option was given on the command line.  See also `remote.<name>.prune`.
+	option was given on the command line.  See also `remote.<name>.prune`
+	and the PRUNING section of linkgit:git-fetch[1].
 
 fetch.output::
 	Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
++
+See also `remote.<name>.prune` and the PRUNING section of
+linkgit:git-fetch[1].
 
 remotes.<group>::
 	The list of remotes which are fetched by "git remote update
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index fb6bebbc61..9f5c85ad96 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -74,6 +74,9 @@ ifndef::git-pull[]
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
 	subject to pruning.
++
+See the PRUNING section below for more details.
+
 endif::git-pull[]
 
 ifndef::git-pull[]
-- 
2.15.1.424.g9478a66081

[PATCH v5 15/17] fetch tests: add scaffolding for the new fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

The fetch.pruneTags configuration doesn't exist yet, but will be added
in a subsequent commit. Since testing for it requires adding new
parameters to the test_configured_prune function it's easier to review
this patch first to assert that no functional changes are introduced
yet.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 92 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 50 insertions(+), 42 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 9c87fa6106..1713111006 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -562,10 +562,12 @@ test_configured_prune () {
 test_configured_prune_type () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	expected_branch=$3
-	expected_tag=$4
-	cmdline=$5
-	mode=$6
+	fetch_prune_tags=$3
+	remote_origin_prune_tags=$4
+	expected_branch=$5
+	expected_tag=$6
+	cmdline=$7
+	mode=$8
 
 	if test -z "$cmdline_setup"
 	then
@@ -590,14 +592,16 @@ test_configured_prune_type () {
 		cmdline="$new_cmdline"
 	fi
 
-	test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+	test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
+			test_unconfig fetch.pruneTags &&
 			test_unconfig remote.origin.prune &&
+			test_unconfig remote.origin.pruneTags &&
 			git fetch '"$cmdline_setup"' &&
 			git rev-parse --verify refs/remotes/origin/newbranch &&
 			git rev-parse --verify refs/tags/newtag
@@ -612,7 +616,9 @@ test_configured_prune_type () {
 			cd one &&
 			git_fetch_c="" &&
 			set_config_tristate fetch.prune $fetch_prune &&
+			set_config_tristate fetch.pruneTags $fetch_prune_tags &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
+			set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags &&
 
 			if test "$mode" != "link"
 			then
@@ -641,57 +647,59 @@ test_configured_prune_type () {
 
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
-# $3 expect: branch to be pruned?
-# $4 expect: tag to be pruned?
-# $5 git-fetch $cmdline:
+# $3 config: fetch.pruneTags
+# $4 config: remote.<name>.pruneTags
+# $5 expect: branch to be pruned?
+# $6 expect: tag to be pruned?
+# $7 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4     $5
-test_configured_prune unset unset kept   kept   ""
-test_configured_prune unset unset kept   kept   "--no-prune"
-test_configured_prune unset unset pruned kept   "--prune"
-test_configured_prune unset unset kept   pruned \
+#                     $1    $2    $3    $4    $5     $6     $7
+test_configured_prune unset unset unset unset kept   kept   ""
+test_configured_prune unset unset unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset unset unset pruned kept   "--prune"
+test_configured_prune unset unset unset unset kept   pruned \
 	"--prune origin refs/tags/*:refs/tags/*"
-test_configured_prune unset unset pruned pruned \
+test_configured_prune unset unset unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
-test_configured_prune false unset kept   kept   ""
-test_configured_prune false unset kept   kept   "--no-prune"
-test_configured_prune false unset pruned kept   "--prune"
+test_configured_prune false unset unset unset kept   kept   ""
+test_configured_prune false unset unset unset kept   kept   "--no-prune"
+test_configured_prune false unset unset unset pruned kept   "--prune"
 
-test_configured_prune true  unset pruned kept   ""
-test_configured_prune true  unset pruned kept   "--prune"
-test_configured_prune true  unset kept   kept   "--no-prune"
+test_configured_prune true  unset unset unset pruned kept   ""
+test_configured_prune true  unset unset unset pruned kept   "--prune"
+test_configured_prune true  unset unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset false kept   kept   ""
-test_configured_prune unset false kept   kept   "--no-prune"
-test_configured_prune unset false pruned kept   "--prune"
+test_configured_prune unset false unset unset kept   kept   ""
+test_configured_prune unset false unset unset kept   kept   "--no-prune"
+test_configured_prune unset false unset unset pruned kept   "--prune"
 
-test_configured_prune false false kept   kept   ""
-test_configured_prune false false kept   kept   "--no-prune"
-test_configured_prune false false pruned kept   "--prune"
-test_configured_prune false false kept   pruned \
+test_configured_prune false false unset unset kept   kept   ""
+test_configured_prune false false unset unset kept   kept   "--no-prune"
+test_configured_prune false false unset unset pruned kept   "--prune"
+test_configured_prune false false unset unset kept   pruned \
 	"--prune origin refs/tags/*:refs/tags/*"
-test_configured_prune false false pruned pruned \
+test_configured_prune false false unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
-test_configured_prune true  false kept   kept   ""
-test_configured_prune true  false pruned kept   "--prune"
-test_configured_prune true  false kept   kept   "--no-prune"
+test_configured_prune true  false unset unset kept   kept   ""
+test_configured_prune true  false unset unset pruned kept   "--prune"
+test_configured_prune true  false unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset true  pruned kept   ""
-test_configured_prune unset true  kept   kept   "--no-prune"
-test_configured_prune unset true  pruned kept   "--prune"
+test_configured_prune unset true  unset unset pruned kept   ""
+test_configured_prune unset true  unset unset kept   kept   "--no-prune"
+test_configured_prune unset true  unset unset pruned kept   "--prune"
 
-test_configured_prune false true  pruned kept   ""
-test_configured_prune false true  kept   kept   "--no-prune"
-test_configured_prune false true  pruned kept   "--prune"
+test_configured_prune false true  unset unset pruned kept   ""
+test_configured_prune false true  unset unset kept   kept   "--no-prune"
+test_configured_prune false true  unset unset pruned kept   "--prune"
 
-test_configured_prune true  true  pruned kept   ""
-test_configured_prune true  true  pruned kept   "--prune"
-test_configured_prune true  true  kept   kept   "--no-prune"
-test_configured_prune true  true  kept   pruned \
+test_configured_prune true  true  unset unset pruned kept   ""
+test_configured_prune true  true  unset unset pruned kept   "--prune"
+test_configured_prune true  true  unset unset kept   kept   "--no-prune"
+test_configured_prune true  true  unset unset kept   pruned \
 	"--prune origin refs/tags/*:refs/tags/*"
-test_configured_prune true  true  pruned pruned \
+test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
 test_expect_success 'all boundary commits are excluded' '
-- 
2.15.1.424.g9478a66081

[PATCH v5 16/17] fetch: add a --prune-tags option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

Add a --prune-tags option to git-fetch, along with fetch.pruneTags
config option and a -P shorthand (-p is --prune). This allows for
doing any of:

    git fetch -p -P
    git fetch --prune --prune-tags
    git fetch -p -P origin
    git fetch --prune --prune-tags origin

Or simply:

    git config fetch.prune true &&
    git config fetch.pruneTags true &&
    git fetch

Instead of the much more verbose:

    git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'

Before this feature it was painful to support the use-case of pulling
from a repo which is having both its branches *and* tags deleted
regularly, and have our local references to reflect upstream.

At work we create deployment tags in the repo for each rollout, and
there's *lots* of those, so they're archived within weeks for
performance reasons.

Without this change it's hard to centrally configure such repos in
/etc/gitconfig (on servers that are only used for working with
them). You need to set fetch.prune=true globally, and then for each
repo:

    git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"

Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well,
and users running "git pull" will automatically get the pruning
semantics I want.

Even though "git remote" has corresponding "prune" and "update
--prune" subcommands I'm intentionally not adding a corresponding
prune-tags or "update --prune --prune-tags" mode to that command.

It's advertised (as noted in my recent "git remote doc: correct
dangerous lies about what prune does") as only modifying remote
tracking references, whereas any --prune-tags option is always going
to modify what from the user's perspective is a local copy of the tag,
since there's no such thing as a remote tracking tag.

Ideally add_prune_tags_to_fetch_refspec() would be something that
would use ALLOC_GROW() to grow the 'fetch` member of the 'remote'
struct. Instead I'm realloc-ing remote->fetch and adding the
tag_refspec to the end.

The reason is that parse_{fetch,push}_refspec which allocate the
refspec (ultimately remote->fetch) struct are called many places that
don't have access to a 'remote' struct. It would be hard to change all
their callsites to be amenable to carry around the bookkeeping
variables required for dynamic allocation.

All the other callers of the API first incrementally construct the
string version of the refspec in remote->fetch_refspec via
add_fetch_refspec(), before finally calling parse_fetch_refspec() via
some variation of remote_get().

It's less of a pain to deal with the one special case that needs to
modify already constructed refspecs than to chase down and change all
the other callsites. The API I'm adding is intentionally not
generalized because if we add more of these we'd probably want to
re-visit how this is done.

See my "Re: [BUG] git remote prune removes local tags, depending on
fetch config" (87po6ahx87.fsf@evledraar.gmail.com;
https://public-inbox.org/git/87po6ahx87.fsf@evledraar.gmail.com/) for
more background info.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt               | 14 +++++++
 Documentation/fetch-options.txt        | 14 ++++++-
 Documentation/git-fetch.txt            | 47 +++++++++++++++++++++++
 builtin/fetch.c                        | 32 ++++++++++++++--
 contrib/completion/git-completion.bash |  2 +-
 remote.c                               | 14 +++++++
 remote.h                               |  3 ++
 t/t5510-fetch.sh                       | 70 ++++++++++++++++++++++++++++++++++
 8 files changed, 191 insertions(+), 5 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0f27af5760..e254bfd531 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1401,6 +1401,14 @@ fetch.prune::
 	option was given on the command line.  See also `remote.<name>.prune`
 	and the PRUNING section of linkgit:git-fetch[1].
 
+fetch.pruneTags::
+	If true, fetch will automatically behave as if the
+	`refs/tags/*:refs/tags/*` refspec was provided when pruning,
+	if not set already. This allows for setting both this option
+	and `fetch.prune` to maintain a 1=1 mapping to upstream
+	refs. See also `remote.<name>.pruneTags` and the PRUNING
+	section of linkgit:git-fetch[1].
+
 fetch.output::
 	Control how ref update status is printed. Valid values are
 	`full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
+
+remote.<name>.pruneTags::
+	When set to true, fetching from this remote by default will also
+	remove any local tags that no longer exist on the remote if pruning
+	is activated in general via `remote.<name>.prune`, `fetch.prune` or
+	`--prune`. Overrides `fetch.pruneTags` settings, if any.
 +
 See also `remote.<name>.prune` and the PRUNING section of
 linkgit:git-fetch[1].
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 9f5c85ad96..8631e365f4 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -73,7 +73,19 @@ ifndef::git-pull[]
 	are fetched due to an explicit refspec (either on the command
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
-	subject to pruning.
+	subject to pruning. Supplying `--prune-tags` is a shorthand for
+	providing the tag refspec.
++
+See the PRUNING section below for more details.
+
+-P::
+--prune-tags::
+	Before fetching, remove any local tags that no longer exist on
+	the remote if `--prune` is enabled. This option should be used
+	more carefully, unlike `--prune` it will remove any local
+	references (local tags) that have been created. This option is
+	a shorthand for providing the explicit tag refspec along with
+	`--prune`, see the discussion about that in its documentation.
 +
 See the PRUNING section below for more details.
 
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index e94bcfb8c3..af12310f75 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -148,6 +148,53 @@ So be careful when using this with a refspec like
 `refs/tags/*:refs/tags/*`, or any other refspec which might map
 references from multiple remotes to the same local namespace.
 
+Since keeping up-to-date with both branches and tags on the remote is
+a common use-case the `--prune-tags` option can be supplied along with
+`--prune` to prune local tags that don't exist on the remote, and
+force-update those tags that differ. Tag pruning can also be enabled
+with `fetch.pruneTags` or `remote.<name>.pruneTags` in the config. See
+linkgit:git-config[1].
+
+The `--prune-tags` option is equivalent to having
+`refs/tags/*:refs/tags/*` declared in the refspecs of the remote. This
+can lead to some seemingly strange interactions:
+
+------------------------------------------------
+# These both fetch tags
+$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*'
+$ git fetch --no-tags --prune-tags origin
+------------------------------------------------
+
+The reason it doesn't error out when provided without `--prune` or its
+config versions is for flexibility of the configured versions, and to
+maintain a 1=1 mapping between what the command line flags do, and
+what the configuration versions do.
+
+It's reasonable to e.g. configure `fetch.pruneTags=true` in
+`~/.gitconfig` to have tags pruned whenever `git fetch --prune` is
+run, without making every invocation of `git fetch` without `--prune`
+an error.
+
+Another special case of `--prune-tags` is that
+`refs/tags/*:refs/tags/*` will not be implicitly provided if an URL is
+being fetched. I.e.:
+
+------------------------------------------------
+$ git fetch <url> --prune --prune-tags
+------------------------------------------------
+
+Will prune no tags, as opposed to:
+
+------------------------------------------------
+$ git fetch origin --prune --prune-tags
+------------------------------------------------
+
+To prune tags given a URL supply the refspec explicitly:
+
+------------------------------------------------
+$ git fetch <url> --prune 'refs/tags/*:refs/tags/*'
+------------------------------------------------
+
 OUTPUT
 ------
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a7705bc150..55a0fc37be 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */
 static int prune = -1; /* unspecified */
 #define PRUNE_BY_DEFAULT 0 /* do we prune by default? */
 
+static int fetch_prune_tags_config = -1; /* unspecified */
+static int prune_tags = -1; /* unspecified */
+#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
 static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;
 static int progress = -1;
 static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(k, "fetch.prunetags")) {
+		fetch_prune_tags_config = git_config_bool(k, v);
+		return 0;
+	}
+
 	if (!strcmp(k, "submodule.recurse")) {
 		int r = git_config_bool(k, v) ?
 			RECURSE_SUBMODULES_ON : RECURSE_SUBMODULES_OFF;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = {
 		    N_("number of submodules fetched in parallel")),
 	OPT_BOOL('p', "prune", &prune,
 		 N_("prune remote-tracking branches no longer on remote")),
+	OPT_BOOL('P', "prune-tags", &prune_tags,
+		 N_("prune local tags no longer on remote and clobber changed tags")),
 	{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
 		    N_("control recursive fetching of submodules"),
 		    PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)
 		argv_array_push(argv, "--dry-run");
 	if (prune != -1)
 		argv_array_push(argv, prune ? "--prune" : "--no-prune");
+	if (prune_tags != -1)
+		argv_array_push(argv, prune_tags ? "--prune-tags" : "--no-prune-tags");
 	if (update_head_ok)
 		argv_array_push(argv, "--update-head-ok");
 	if (force)
@@ -1265,7 +1278,7 @@ static int fetch_multiple(struct string_list *list)
 	return result;
 }
 
-static int fetch_one(struct remote *remote, int argc, const char **argv)
+static int fetch_one(struct remote *remote, int argc, const char **argv, int prune_tags_ok)
 {
 	static const char **refs = NULL;
 	struct refspec *refspec;
@@ -1288,6 +1301,19 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			prune = PRUNE_BY_DEFAULT;
 	}
 
+	if (prune_tags < 0) {
+		/* no command line request */
+		if (0 <= remote->prune_tags)
+			prune_tags = remote->prune_tags;
+		else if (0 <= fetch_prune_tags_config)
+			prune_tags = fetch_prune_tags_config;
+		else
+			prune_tags = PRUNE_TAGS_BY_DEFAULT;
+	}
+
+	if (prune_tags_ok && prune_tags && remote_is_configured(remote, 0))
+		add_prune_tags_to_fetch_refspec(remote);
+
 	if (argc > 0) {
 		int j = 0;
 		int i;
@@ -1368,7 +1394,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	} else if (argc == 0) {
 		/* No arguments -- use default remote */
 		remote = remote_get(NULL);
-		result = fetch_one(remote, argc, argv);
+		result = fetch_one(remote, argc, argv, 1);
 	} else if (multiple) {
 		/* All arguments are assumed to be remotes or groups */
 		for (i = 0; i < argc; i++)
@@ -1386,7 +1412,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 		} else {
 			/* Zero or one remotes */
 			remote = remote_get(argv[0]);
-			result = fetch_one(remote, argc-1, argv+1);
+			result = fetch_one(remote, argc-1, argv+1, argc == 1);
 		}
 	}
 
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3683c772c5..4ecd0d4d7a 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1468,7 +1468,7 @@ __git_fetch_recurse_submodules="yes on-demand no"
 __git_fetch_options="
 	--quiet --verbose --append --upload-pack --force --keep --depth=
 	--tags --no-tags --all --prune --dry-run --recurse-submodules=
-	--unshallow --update-shallow
+	--unshallow --update-shallow --prune-tags
 "
 
 _git_fetch ()
diff --git a/remote.c b/remote.c
index 356c123e3e..d670eed4c4 100644
--- a/remote.c
+++ b/remote.c
@@ -104,6 +104,17 @@ static void add_fetch_refspec(struct remote *remote, const char *ref)
 	remote->fetch_refspec[remote->fetch_refspec_nr++] = ref;
 }
 
+void add_prune_tags_to_fetch_refspec(struct remote *remote)
+{
+	int nr = remote->fetch_refspec_nr;
+	int bufsize = nr  + 1;
+	int size = sizeof(struct refspec);
+
+	remote->fetch = xrealloc(remote->fetch, size  * bufsize);
+	memcpy(&remote->fetch[nr], tag_refspec, size);
+	add_fetch_refspec(remote, xstrdup(TAG_REFSPEC));
+}
+
 static void add_url(struct remote *remote, const char *url)
 {
 	ALLOC_GROW(remote->url, remote->url_nr + 1, remote->url_alloc);
@@ -174,6 +185,7 @@ static struct remote *make_remote(const char *name, int len)
 
 	ret = xcalloc(1, sizeof(struct remote));
 	ret->prune = -1;  /* unspecified */
+	ret->prune_tags = -1;  /* unspecified */
 	ALLOC_GROW(remotes, remotes_nr + 1, remotes_alloc);
 	remotes[remotes_nr++] = ret;
 	ret->name = xstrndup(name, len);
@@ -392,6 +404,8 @@ static int handle_config(const char *key, const char *value, void *cb)
 		remote->skip_default_update = git_config_bool(key, value);
 	else if (!strcmp(subkey, "prune"))
 		remote->prune = git_config_bool(key, value);
+	else if (!strcmp(subkey, "prunetags"))
+		remote->prune_tags = git_config_bool(key, value);
 	else if (!strcmp(subkey, "url")) {
 		const char *v;
 		if (git_config_string(&v, key, value))
diff --git a/remote.h b/remote.h
index 80fea6dd11..271afe1bab 100644
--- a/remote.h
+++ b/remote.h
@@ -47,6 +47,7 @@ struct remote {
 	int skip_default_update;
 	int mirror;
 	int prune;
+	int prune_tags;
 
 	const char *receivepack;
 	const char *uploadpack;
@@ -299,4 +300,6 @@ void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *);
 
 #define TAG_REFSPEC "refs/tags/*:refs/tags/*"
 
+void add_prune_tags_to_fetch_refspec(struct remote *remote);
+
 #endif
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 1713111006..227dd70b7b 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -589,6 +589,15 @@ test_configured_prune_type () {
 			new_cmdline=$(printf "%s" "$cmdline" | perl -pe 's[origin(?!/)]["'"$remote_url"'"]g')
 		fi
 
+		if test "$fetch_prune_tags" = 'true' ||
+		   test "$remote_origin_prune_tags" = 'true'
+		then
+			if ! printf '%s' "$cmdline\n" | grep -q refs/remotes/origin/
+			then
+				new_cmdline="$new_cmdline refs/tags/*:refs/tags/*"
+			fi
+		fi
+
 		cmdline="$new_cmdline"
 	fi
 
@@ -702,6 +711,67 @@ test_configured_prune true  true  unset unset kept   pruned \
 test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
 
+# --prune-tags on its own does nothing, needs --prune as well, same
+# for for fetch.pruneTags without fetch.prune
+test_configured_prune unset unset unset unset kept kept     "--prune-tags"
+test_configured_prune unset unset true unset  kept kept     ""
+test_configured_prune unset unset unset true  kept kept     ""
+
+# These will prune the tags
+test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags"
+test_configured_prune true  unset true  unset pruned pruned ""
+test_configured_prune unset true  unset true  pruned pruned ""
+
+# remote.<name>.pruneTags overrides fetch.pruneTags, just like
+# remote.<name>.prune overrides fetch.prune if set.
+test_configured_prune true  unset true unset pruned pruned  ""
+test_configured_prune false true  false true  pruned pruned ""
+test_configured_prune true  false true  false kept   kept   ""
+
+# When --prune-tags is supplied it's ignored if an explicit refspec is
+# given, same for the configuration options.
+test_configured_prune unset unset unset unset pruned kept \
+	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
+test_configured_prune unset unset true  unset pruned kept \
+	"--prune origin +refs/heads/*:refs/remotes/origin/*"
+test_configured_prune unset unset unset true pruned  kept \
+	"--prune origin +refs/heads/*:refs/remotes/origin/*"
+
+# Pruning that also takes place if a file:// url replaces a named
+# remote, with the exception of --prune-tags on the command-line
+# (arbitrary limitation).
+#
+# However, because there's no implicit
+# +refs/heads/*:refs/remotes/origin/* refspec and supplying it on the
+# command-line negates --prune-tags, the branches will not be pruned.
+test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "name"
+test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "link"
+test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
+test_configured_prune_type unset unset unset unset kept   kept   "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name"
+test_configured_prune_type unset unset unset unset kept   kept   "--prune --prune-tags origin" "link"
+test_configured_prune_type unset unset true  unset pruned pruned "--prune origin" "name"
+test_configured_prune_type unset unset true  unset kept   pruned "--prune origin" "link"
+test_configured_prune_type unset unset unset true  pruned pruned "--prune origin" "name"
+test_configured_prune_type unset unset unset true  kept   pruned "--prune origin" "link"
+test_configured_prune_type true  unset true  unset pruned pruned "origin" "name"
+test_configured_prune_type true  unset true  unset kept   pruned "origin" "link"
+test_configured_prune_type unset  true true  unset pruned pruned "origin" "name"
+test_configured_prune_type unset  true true  unset kept   pruned "origin" "link"
+test_configured_prune_type unset  true unset true  pruned pruned "origin" "name"
+test_configured_prune_type unset  true unset true  kept   pruned "origin" "link"
+
+# Interaction between --prune-tags and no "fetch" config in the remote
+# at all.
+test_expect_success 'remove remote.origin.fetch "one"' '
+	(
+		cd one &&
+		git config --unset-all remote.origin.fetch
+	)
+'
+test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name"
+test_configured_prune_type unset unset unset unset kept kept   "origin --prune --prune-tags" "link"
+
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
 	test_commit oneside &&
-- 
2.15.1.424.g9478a66081

[PATCH v5 17/17] fetch: make the --prune-tags work with <url>

[Ævar Arnfjörð Bjarmason]

Make the new --prune-tags option work properly when git-fetch is
invoked with a <url> parameter instead of a <remote name>
parameter.

This change is split off from the introduction of --prune-tags due to
the relative complexity of munging the incoming argv, which is easier
to review as a separate change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-fetch.txt | 21 ++++++---------------
 builtin/fetch.c             | 17 ++++++++++++++---
 t/t5510-fetch.sh            | 16 +++++++---------
 3 files changed, 27 insertions(+), 27 deletions(-)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index af12310f75..e319935597 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -175,24 +175,15 @@ It's reasonable to e.g. configure `fetch.pruneTags=true` in
 run, without making every invocation of `git fetch` without `--prune`
 an error.
 
-Another special case of `--prune-tags` is that
-`refs/tags/*:refs/tags/*` will not be implicitly provided if an URL is
-being fetched. I.e.:
-
-------------------------------------------------
-$ git fetch <url> --prune --prune-tags
-------------------------------------------------
-
-Will prune no tags, as opposed to:
+Pruning tags with `--prune-tags` also works when fetching a URL
+instead of a named remote. These will all prune tags not found on
+origin:
 
 ------------------------------------------------
 $ git fetch origin --prune --prune-tags
-------------------------------------------------
-
-To prune tags given a URL supply the refspec explicitly:
-
-------------------------------------------------
-$ git fetch <url> --prune 'refs/tags/*:refs/tags/*'
+$ git fetch origin --prune 'refs/tags/*:refs/tags/*'
+$ git fetch <url of origin> --prune --prune-tags
+$ git fetch <url of origin> --prune 'refs/tags/*:refs/tags/*'
 ------------------------------------------------
 
 OUTPUT
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 55a0fc37be..c96f17a9a3 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1283,7 +1283,10 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru
 	static const char **refs = NULL;
 	struct refspec *refspec;
 	int ref_nr = 0;
+	int j = 0;
 	int exit_code;
+	int maybe_prune_tags;
+	int remote_via_config = remote_is_configured(remote, 0);
 
 	if (!remote)
 		die(_("No remote repository specified.  Please, specify either a URL or a\n"
@@ -1311,13 +1314,21 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru
 			prune_tags = PRUNE_TAGS_BY_DEFAULT;
 	}
 
-	if (prune_tags_ok && prune_tags && remote_is_configured(remote, 0))
+	maybe_prune_tags = prune_tags_ok && prune_tags;
+	if (maybe_prune_tags && remote_via_config)
 		add_prune_tags_to_fetch_refspec(remote);
 
+	if (argc > 0 || (maybe_prune_tags && !remote_via_config)) {
+		size_t nr_alloc = st_add3(argc, maybe_prune_tags, 1);
+		refs = xcalloc(nr_alloc, sizeof(const char *));
+		if (maybe_prune_tags) {
+			refs[j++] = xstrdup("refs/tags/*:refs/tags/*");
+			ref_nr++;
+		}
+	}
+
 	if (argc > 0) {
-		int j = 0;
 		int i;
-		refs = xcalloc(st_add(argc, 1), sizeof(const char *));
 		for (i = 0; i < argc; i++) {
 			if (!strcmp(argv[i], "tag")) {
 				i++;
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 227dd70b7b..dce2371302 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -738,18 +738,15 @@ test_configured_prune unset unset unset true pruned  kept \
 	"--prune origin +refs/heads/*:refs/remotes/origin/*"
 
 # Pruning that also takes place if a file:// url replaces a named
-# remote, with the exception of --prune-tags on the command-line
-# (arbitrary limitation).
-#
-# However, because there's no implicit
+# remote. However, because there's no implicit
 # +refs/heads/*:refs/remotes/origin/* refspec and supplying it on the
 # command-line negates --prune-tags, the branches will not be pruned.
 test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "name"
 test_configured_prune_type unset unset unset unset kept   kept   "origin --prune-tags" "link"
 test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
-test_configured_prune_type unset unset unset unset kept   kept   "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset kept   pruned "origin --prune --prune-tags" "link"
 test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name"
-test_configured_prune_type unset unset unset unset kept   kept   "--prune --prune-tags origin" "link"
+test_configured_prune_type unset unset unset unset kept   pruned "--prune --prune-tags origin" "link"
 test_configured_prune_type unset unset true  unset pruned pruned "--prune origin" "name"
 test_configured_prune_type unset unset true  unset kept   pruned "--prune origin" "link"
 test_configured_prune_type unset unset unset true  pruned pruned "--prune origin" "name"
@@ -761,8 +758,9 @@ test_configured_prune_type unset  true true  unset kept   pruned "origin" "link"
 test_configured_prune_type unset  true unset true  pruned pruned "origin" "name"
 test_configured_prune_type unset  true unset true  kept   pruned "origin" "link"
 
-# Interaction between --prune-tags and no "fetch" config in the remote
-# at all.
+# When all remote.origin.fetch settings are deleted a --prune
+# --prune-tags still implicitly supplies refs/tags/*:refs/tags/* so
+# tags, but not tracking branches, will be deleted.
 test_expect_success 'remove remote.origin.fetch "one"' '
 	(
 		cd one &&
@@ -770,7 +768,7 @@ test_expect_success 'remove remote.origin.fetch "one"' '
 	)
 '
 test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name"
-test_configured_prune_type unset unset unset unset kept kept   "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "link"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v2 01/12] fetch tests: refactor in preparation for testing tag pruning

[Ævar Arnfjörð Bjarmason]

In a subsequent commit this function will learn to test for tag
pruning, prepare for that by making space for more variables, and
making it clear that "expected" here refers to branches.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 668c54be41..11da97f9b7 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -549,9 +549,12 @@ set_config_tristate () {
 }
 
 test_configured_prune () {
-	fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4
+	fetch_prune=$1
+	remote_origin_prune=$2
+	cmdline=$3
+	expected_branch=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -572,7 +575,7 @@ test_configured_prune () {
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
 			git fetch $cmdline &&
-			case "$expected" in
+			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
 				;;
-- 
2.15.1.424.g9478a66081

[PATCH v2 02/12] fetch tests: arrange arguments for future readability

[Ævar Arnfjörð Bjarmason]

Re-arrange the arguments to the test_configured_prune() function used
in this test to pass the arguments to --fetch last. A subsequent
change will test for more elaborate fetch arguments, including long
refspecs. It'll be more readable to be able to wrap those on a new
line of their own.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 44 insertions(+), 38 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 11da97f9b7..ab8b25344d 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,10 +551,10 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	cmdline=$3
-	expected_branch=$4
+	expected_branch=$3
+	cmdline=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -587,41 +587,47 @@ test_configured_prune () {
 	'
 }
 
-test_configured_prune unset unset ""		kept
-test_configured_prune unset unset "--no-prune"	kept
-test_configured_prune unset unset "--prune"	pruned
-
-test_configured_prune false unset ""		kept
-test_configured_prune false unset "--no-prune"	kept
-test_configured_prune false unset "--prune"	pruned
-
-test_configured_prune true  unset ""		pruned
-test_configured_prune true  unset "--prune"	pruned
-test_configured_prune true  unset "--no-prune"	kept
-
-test_configured_prune unset false ""		kept
-test_configured_prune unset false "--no-prune"	kept
-test_configured_prune unset false "--prune"	pruned
-
-test_configured_prune false false ""		kept
-test_configured_prune false false "--no-prune"	kept
-test_configured_prune false false "--prune"	pruned
-
-test_configured_prune true  false ""		kept
-test_configured_prune true  false "--prune"	pruned
-test_configured_prune true  false "--no-prune"	kept
-
-test_configured_prune unset true  ""		pruned
-test_configured_prune unset true  "--no-prune"	kept
-test_configured_prune unset true  "--prune"	pruned
-
-test_configured_prune false true  ""		pruned
-test_configured_prune false true  "--no-prune"	kept
-test_configured_prune false true  "--prune"	pruned
-
-test_configured_prune true  true  ""		pruned
-test_configured_prune true  true  "--prune"	pruned
-test_configured_prune true  true  "--no-prune"	kept
+# $1 config: fetch.prune
+# $2 config: remote.<name>.prune
+# $3 expect: branch to be pruned?
+# $4 git-fetch $cmdline:
+#
+#                     $1    $2    $3     $4
+test_configured_prune unset unset kept   ""
+test_configured_prune unset unset kept   "--no-prune"
+test_configured_prune unset unset pruned "--prune"
+
+test_configured_prune false unset kept   ""
+test_configured_prune false unset kept   "--no-prune"
+test_configured_prune false unset pruned "--prune"
+
+test_configured_prune true  unset pruned ""
+test_configured_prune true  unset pruned "--prune"
+test_configured_prune true  unset kept   "--no-prune"
+
+test_configured_prune unset false kept   ""
+test_configured_prune unset false kept   "--no-prune"
+test_configured_prune unset false pruned "--prune"
+
+test_configured_prune false false kept   ""
+test_configured_prune false false kept   "--no-prune"
+test_configured_prune false false pruned "--prune"
+
+test_configured_prune true  false kept   ""
+test_configured_prune true  false pruned "--prune"
+test_configured_prune true  false kept   "--no-prune"
+
+test_configured_prune unset true  pruned ""
+test_configured_prune unset true  kept   "--no-prune"
+test_configured_prune unset true  pruned "--prune"
+
+test_configured_prune false true  pruned ""
+test_configured_prune false true  kept   "--no-prune"
+test_configured_prune false true  pruned "--prune"
+
+test_configured_prune true  true  pruned ""
+test_configured_prune true  true  pruned "--prune"
+test_configured_prune true  true  kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v2 03/12] fetch tests: add a tag to be deleted to the pruning tests

[Ævar Arnfjörð Bjarmason]

Add a tag to be deleted to the fetch --prune tests. The tag is always
kept for now, which is the expected behavior, but now I can add a test
for tag pruning in a later commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 53 insertions(+), 40 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index ab8b25344d..fad65bd885 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -552,21 +552,25 @@ test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
 	expected_branch=$3
-	cmdline=$4
+	expected_tag=$4
+	cmdline=$5
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
+		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
 			test_unconfig remote.origin.prune &&
 			git fetch &&
-			git rev-parse --verify refs/remotes/origin/newbranch
+			git rev-parse --verify refs/remotes/origin/newbranch &&
+			git rev-parse --verify refs/tags/newtag
 		) &&
 
 		# now remove it
 		git branch -d newbranch &&
+		git tag -d newtag &&
 
 		# then test
 		(
@@ -582,6 +586,14 @@ test_configured_prune () {
 			kept)
 				git rev-parse --verify refs/remotes/origin/newbranch
 				;;
+			esac &&
+			case "$expected_tag" in
+			pruned)
+				test_must_fail git rev-parse --verify refs/tags/newtag
+				;;
+			kept)
+				git rev-parse --verify refs/tags/newtag
+				;;
 			esac
 		)
 	'
@@ -590,44 +602,45 @@ test_configured_prune () {
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
 # $3 expect: branch to be pruned?
-# $4 git-fetch $cmdline:
+# $4 expect: tag to be pruned?
+# $5 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4
-test_configured_prune unset unset kept   ""
-test_configured_prune unset unset kept   "--no-prune"
-test_configured_prune unset unset pruned "--prune"
-
-test_configured_prune false unset kept   ""
-test_configured_prune false unset kept   "--no-prune"
-test_configured_prune false unset pruned "--prune"
-
-test_configured_prune true  unset pruned ""
-test_configured_prune true  unset pruned "--prune"
-test_configured_prune true  unset kept   "--no-prune"
-
-test_configured_prune unset false kept   ""
-test_configured_prune unset false kept   "--no-prune"
-test_configured_prune unset false pruned "--prune"
-
-test_configured_prune false false kept   ""
-test_configured_prune false false kept   "--no-prune"
-test_configured_prune false false pruned "--prune"
-
-test_configured_prune true  false kept   ""
-test_configured_prune true  false pruned "--prune"
-test_configured_prune true  false kept   "--no-prune"
-
-test_configured_prune unset true  pruned ""
-test_configured_prune unset true  kept   "--no-prune"
-test_configured_prune unset true  pruned "--prune"
-
-test_configured_prune false true  pruned ""
-test_configured_prune false true  kept   "--no-prune"
-test_configured_prune false true  pruned "--prune"
-
-test_configured_prune true  true  pruned ""
-test_configured_prune true  true  pruned "--prune"
-test_configured_prune true  true  kept   "--no-prune"
+#                     $1    $2    $3     $4     $5
+test_configured_prune unset unset kept   kept   ""
+test_configured_prune unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset pruned kept   "--prune"
+
+test_configured_prune false unset kept   kept   ""
+test_configured_prune false unset kept   kept   "--no-prune"
+test_configured_prune false unset pruned kept   "--prune"
+
+test_configured_prune true  unset pruned kept   ""
+test_configured_prune true  unset pruned kept   "--prune"
+test_configured_prune true  unset kept   kept   "--no-prune"
+
+test_configured_prune unset false kept   kept   ""
+test_configured_prune unset false kept   kept   "--no-prune"
+test_configured_prune unset false pruned kept   "--prune"
+
+test_configured_prune false false kept   kept   ""
+test_configured_prune false false kept   kept   "--no-prune"
+test_configured_prune false false pruned kept   "--prune"
+
+test_configured_prune true  false kept   kept   ""
+test_configured_prune true  false pruned kept   "--prune"
+test_configured_prune true  false kept   kept   "--no-prune"
+
+test_configured_prune unset true  pruned kept   ""
+test_configured_prune unset true  kept   kept   "--no-prune"
+test_configured_prune unset true  pruned kept   "--prune"
+
+test_configured_prune false true  pruned kept   ""
+test_configured_prune false true  kept   kept   "--no-prune"
+test_configured_prune false true  pruned kept   "--prune"
+
+test_configured_prune true  true  pruned kept   ""
+test_configured_prune true  true  pruned kept   "--prune"
+test_configured_prune true  true  kept   kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v2 04/12] fetch tests: double quote a variable for interpolation

[Ævar Arnfjörð Bjarmason]

If the $cmdline variable contains multiple arguments they won't be
interpolated correctly since the body of the test is single quoted. I
don't know what part of test-lib.sh is expanding variables within
single-quoted strings, but interpolating this inline is the desired
behavior here.

This will be used in a subsequent commit to pass more than one
variable to git-fetch.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index fad65bd885..542eb53a99 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -578,7 +578,7 @@ test_configured_prune () {
 			set_config_tristate fetch.prune $fetch_prune &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
-			git fetch $cmdline &&
+			git fetch '"$cmdline"' &&
 			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 04/12] fetch tests: double quote a variable for interpolation

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> If the $cmdline variable contains multiple arguments they won't be
> interpolated correctly since the body of the test is single quoted. I
> don't know what part of test-lib.sh is expanding variables within
> single-quoted strings,...

    dothis='echo whatever $IFS separated strings'
    test_expect_success label '
            $dothis
    '

works because test_expect_success ends up beint a glorified 'eval'
and it sees the value of $dothis.  

> but interpolating this inline is the desired
> behavior here.

I am not sure what you meant by this, though.

> -			git fetch $cmdline &&
> +			git fetch '"$cmdline"' &&

Would this work with cmdline that needs to be quoted for the
resulting shell script to be syntactically correct (e.g. cmdline
with a single dq in it)?  By stepping out of sq pair, you are
allowing/asking the shell that forms test_expect_success command
line arguments to interpolate cmdline, instead of asking the shell
that evals test_expect_success with its command line argument
strings.

In other words, I suspect that the caller of test_configured_prune
now must sq_quote the cmdline arguments it passes to this helper,
i.e.

	cmdline="$(git rev-parse --sq-quote arg1 'arg"2' arg3)"
	test_configured_prune ... "$cmdline" ...

for this patch to be correct.

Re: [PATCH v2 04/12] fetch tests: double quote a variable for interpolation

[Ævar Arnfjörð Bjarmason]

On Mon, Jan 22 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> If the $cmdline variable contains multiple arguments they won't be
>> interpolated correctly since the body of the test is single quoted. I
>> don't know what part of test-lib.sh is expanding variables within
>> single-quoted strings,...
>
>     dothis='echo whatever $IFS separated strings'
>     test_expect_success label '
>             $dothis
>     '
>
> works because test_expect_success ends up beint a glorified 'eval'
> and it sees the value of $dothis.
>
>> but interpolating this inline is the desired
>> behavior here.
>
> I am not sure what you meant by this, though.

Looking into this again:

    myvar="a b 'c' \"d\" \"\\\"e\\\"\""
    test_expect_success 'blah' '
    	/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- $myvar &&
    	/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- '"$myvar"' &&
    	echo $myvar &&
    	false
    '

Produces:

    Initialized empty Git repository in /home/avar/g/git/t/trash directory.t5510-fetch/.git/
    expecting success:
    	/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- $myvar &&
    	/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- a b 'c' "d" "\"e\"" &&
    	echo $myvar &&
    	false

    + /usr/bin/perl -MData::Dumper -wE say Dumper \@ARGV -- a b 'c' "d" "\"e\""
    $VAR1 = [
              'a',
              'b',
              '\'c\'',
              '"d"',
              '"\\"e\\""'
            ];

    + /usr/bin/perl -MData::Dumper -wE say Dumper \@ARGV -- a b c d "e"
    $VAR1 = [
              'a',
              'b',
              'c',
              'd',
              '"e"'
            ];

    + echo a b 'c' "d" "\"e\""
    a b 'c' "d" "\"e\""
    + false
    error: last command exited with $?=1
    not ok 1 - blah
    #
    #		/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- $myvar &&
    #		/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- a b 'c' "d" "\"e\"" &&
    #		echo $myvar &&
    #		false
    #

I.e. the desired effect is to get variables like the refspec like/this
not 'like/this'. I could also just apply this on top, which gives the
same end result, but now I wonder if starting some args with
e.g. unescaped + and with : and * in the string is portable:

    diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
    index 48f49b613a..2d311059e9 100755
    --- a/t/t5510-fetch.sh
    +++ b/t/t5510-fetch.sh
    @@ -587 +587 @@ test_configured_prune () {
    -			git fetch '"$cmdline"' &&
    +			git fetch $cmdline &&
    @@ -621 +621 @@ test_configured_prune unset unset unset unset kept   pruned \
    -	"--prune origin 'refs/tags/*:refs/tags/*'"
    +	"--prune origin refs/tags/*:refs/tags/*"
    @@ -623 +623 @@ test_configured_prune unset unset unset unset pruned pruned \
    -	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
    +	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
    @@ -641 +641 @@ test_configured_prune false false unset unset kept   pruned \
    -	"--prune origin 'refs/tags/*:refs/tags/*'"
    +	"--prune origin refs/tags/*:refs/tags/*"
    @@ -643 +643 @@ test_configured_prune false false unset unset pruned pruned \
    -	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
    +	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
    @@ -661 +661 @@ test_configured_prune true  true  unset unset kept   pruned \
    -	"--prune origin 'refs/tags/*:refs/tags/*'"
    +	"--prune origin refs/tags/*:refs/tags/*"
    @@ -663 +663 @@ test_configured_prune true  true  unset unset pruned pruned \
    -	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
    +	"--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
    @@ -684 +684 @@ test_configured_prune true  false true  false kept   kept   ""
    -# When --prune-tags is supplied it's ignored if an explict refspec is
    +# When --prune-tags is supplied its ignored if an explict refspec is
    @@ -687 +687 @@ test_configured_prune unset unset unset unset pruned kept \
    -	"--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
    +	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
    @@ -689 +689 @@ test_configured_prune unset unset true  unset pruned kept \
    -	"--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
    +	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
    @@ -691 +691 @@ test_configured_prune unset unset unset true pruned  kept \
    -	"--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
    +	"--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"


>> -			git fetch $cmdline &&
>> +			git fetch '"$cmdline"' &&
>
> Would this work with cmdline that needs to be quoted for the
> resulting shell script to be syntactically correct (e.g. cmdline
> with a single dq in it)?  By stepping out of sq pair, you are
> allowing/asking the shell that forms test_expect_success command
> line arguments to interpolate cmdline, instead of asking the shell
> that evals test_expect_success with its command line argument
> strings.
>
> In other words, I suspect that the caller of test_configured_prune
> now must sq_quote the cmdline arguments it passes to this helper,
> i.e.
>
> 	cmdline="$(git rev-parse --sq-quote arg1 'arg"2' arg3)"
> 	test_configured_prune ... "$cmdline" ...
>
> for this patch to be correct.

Sorry at this point I'm confused about this whole thing. This doesn't
work and overquotes:

    @@ -559,2 +569,3 @@ test_configured_prune () {

    +	cmdline_q="$(git rev-parse --sq-quote $cmdline)"
     	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" '
    @@ -586,3 +597,4 @@ test_configured_prune () {

    -			git fetch '"$cmdline"' &&
    +			/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- '"$cmdline_q"' &&
    +			git fetch '"$cmdline_q"' &&
     			case "$expected_branch" in

So does just using $cmdline_q without making the calling shell
interpolate it. Is there an idiom I should be following that I'm missing
here?

[PATCH v2 05/12] fetch tests: test --prune and refspec interaction

[Ævar Arnfjörð Bjarmason]

Add a test for the interaction between explicitly provided refspecs
and fetch.prune.

There's no point in adding this boilerplate to every combination of
unset/false/true, it's instructive and sufficient to show that no
matter if the variable is unset, false or true the refspec on the
command-line overrides any configuration variable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 542eb53a99..576c2598c9 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -609,6 +609,10 @@ test_configured_prune () {
 test_configured_prune unset unset kept   kept   ""
 test_configured_prune unset unset kept   kept   "--no-prune"
 test_configured_prune unset unset pruned kept   "--prune"
+test_configured_prune unset unset kept   pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*'"
+test_configured_prune unset unset pruned pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_configured_prune false unset kept   kept   ""
 test_configured_prune false unset kept   kept   "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept   "--prune"
 test_configured_prune false false kept   kept   ""
 test_configured_prune false false kept   kept   "--no-prune"
 test_configured_prune false false pruned kept   "--prune"
+test_configured_prune false false kept   pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*'"
+test_configured_prune false false pruned pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_configured_prune true  false kept   kept   ""
 test_configured_prune true  false pruned kept   "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true  pruned kept   "--prune"
 test_configured_prune true  true  pruned kept   ""
 test_configured_prune true  true  pruned kept   "--prune"
 test_configured_prune true  true  kept   kept   "--no-prune"
+test_configured_prune true  true  kept   pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*'"
+test_configured_prune true  true  pruned pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH v2 06/12] git fetch doc: add a new section to explain the ins & outs of pruning

[Ævar Arnfjörð Bjarmason]

Add a new section to canonically explain how remote reference pruning
works, and how users should be careful about using it in conjunction
with tag refspecs in particular.

A subsequent commit will update the git-remote documentation to refer
to this section, and details the motivation for writing this in the
first place.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index b153aefa68..18fac0da2e 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be
 overridden by giving the `--refmap=<refspec>` parameter(s) on the
 command line.
 
+PRUNING
+-------
+
+Git has a default disposition of keeping data unless it's explicitly
+thrown away; this extends to holding onto local references to branches
+on remotes that have themselves deleted those branches.
+
+If left to accumulate, these stale references might make performance
+worse on big and busy repos that have a lot of branch churn, and
+e.g. make the output of commands like `git branch -a --contains
+<commit>` needlessly verbose, as well as impacting anything else
+that'll work with the complete set of known references.
+
+These remote tracking references can be deleted as a one-off with
+either of:
+
+------------------------------------------------
+# While fetching
+$ git fetch --prune <name>
+
+# Only prune, don't fetch
+$ git remote prune <name>
+------------------------------------------------
+
+To prune references as part of your normal workflow without needing to
+remember to run that set `fetch.prune` globally, or
+`remote.<name>.prune` per-remote in the config. See
+linkgit:git-config[1].
+
+Here's where things get tricky and more specific. The pruning feature
+doesn't actually care about branches, instead it'll prune local <->
+remote references as a function of the refspec of the remote (see
+`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).
+
+Therefore if the refspec for the remote includes
+e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch
+--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote
+tracking branches that are deleted, but any local tag that doesn't
+exist on the remote.
+
+This might not be what you expect, i.e. you want to prune remote
+`<name>`, but also explicitly fetch tags from it, so when you fetch
+from it you delete all your local tags, most of which may not have
+come from the `<name>` remote in the first place.
+
+So be careful when using this with a refspec like
+`refs/tags/*:refs/tags/*`, or any other refspec which might map
+references from multiple remotes to the same local namespace.
+
 OUTPUT
 ------
 
-- 
2.15.1.424.g9478a66081

[PATCH v2 07/12] git remote doc: correct dangerous lies about what prune does

[Ævar Arnfjörð Bjarmason]

The "git remote prune <name>" command uses the same machinery as "git
fetch <name> --prune", and shares all the same caveats, but its
documentation has suggested that it'll just "delete stale
remote-tracking branches under <name>".

This isn't true, and hasn't been true since at least v1.8.5.6 (the
oldest version I could be bothered to test).

E.g. if "+refs/tags/*:refs/tags/*" is explicitly set in the refspec of
the remote it'll delete all local tags <name> doesn't know about.

Instead, briefly give the reader just enough of a hint that this
option might constitute a shotgun aimed at their foot, and point them
to the new PRUNING section in the git-fetch documentation which
explains all the nuances of what this facility does.

See "[BUG] git remote prune removes local tags, depending on fetch
config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com)
by Michael Giuffrida for the initial report.

Reported-by: Michael Giuffrida <michaelpg@chromium.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-remote.txt | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-remote.txt b/Documentation/git-remote.txt
index 577b969c1b..7183a72a23 100644
--- a/Documentation/git-remote.txt
+++ b/Documentation/git-remote.txt
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with
 
 'prune'::
 
-Deletes all stale remote-tracking branches under <name>.
-These stale branches have already been removed from the remote repository
-referenced by <name>, but are still locally available in
-"remotes/<name>".
+Deletes stale references associated with <name>. By default stale
+remote-tracking branches under <name>, but depending on global
+configuration and the configuration of the remote we might even prune
+local tags. Equivalent to `git fetch <name> --prune`, except that no
+new references will be fetched.
++
+See the PRUNING section of linkgit:git-fetch[1] for what it'll prune
+depending on various configuration.
 +
 With `--dry-run` option, report what branches will be pruned, but do not
 actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the
 configuration parameter remote.<name>.skipDefaultUpdate set to true will
 be updated.  (See linkgit:git-config[1]).
 +
-With `--prune` option, prune all the remotes that are updated.
+With `--prune` option, run pruning against all the remotes that are updated.
 
 
 DISCUSSION
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 07/12] git remote doc: correct dangerous lies about what prune does

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> +Deletes stale references associated with <name>. By default stale
> +remote-tracking branches under <name>, but depending on global
> +configuration and the configuration of the remote we might even prune
> +local tags....

An optional clarification

    s/prune local tags/& that haven't been pushed there/

> +... Equivalent to `git fetch <name> --prune`, except that no
> +new references will be fetched.

`git fetch --prune <name>` (dashed options first and then non
options), just like you wrote in the previous step.

Re: [PATCH v2 07/12] git remote doc: correct dangerous lies about what prune does

[Ævar Arnfjörð Bjarmason]

On Mon, Jan 22 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> +Deletes stale references associated with <name>. By default stale
>> +remote-tracking branches under <name>, but depending on global
>> +configuration and the configuration of the remote we might even prune
>> +local tags....
>
> An optional clarification
>
>     s/prune local tags/& that haven't been pushed there/
>
>> +... Equivalent to `git fetch <name> --prune`, except that no
>> +new references will be fetched.
>
> `git fetch --prune <name>` (dashed options first and then non
> options), just like you wrote in the previous step.

Thanks. Will fix both.

[PATCH v2 08/12] git-fetch & config doc: link to the new PRUNING section

[Ævar Arnfjörð Bjarmason]

Amend the documentation for fetch.prune, fetch.<name>.prune and
--prune to link to the recently added PRUNING section.

I'd have liked to link directly to it with "<<PRUNING>>" from
fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt
also includes it, but doesn't include that option). However making a
reference across files yields this error:

    [...]/Documentation/git-fetch.xml:226: element xref: validity
    error : IDREF attribute linkend references an unknown ID "PRUNING"

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 6 +++++-
 Documentation/fetch-options.txt | 3 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b..0f27af5760 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1398,7 +1398,8 @@ fetch.unpackLimit::
 
 fetch.prune::
 	If true, fetch will automatically behave as if the `--prune`
-	option was given on the command line.  See also `remote.<name>.prune`.
+	option was given on the command line.  See also `remote.<name>.prune`
+	and the PRUNING section of linkgit:git-fetch[1].
 
 fetch.output::
 	Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
++
+See also `remote.<name>.prune` and the PRUNING section of
+linkgit:git-fetch[1].
 
 remotes.<group>::
 	The list of remotes which are fetched by "git remote update
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index fb6bebbc61..9f5c85ad96 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -74,6 +74,9 @@ ifndef::git-pull[]
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
 	subject to pruning.
++
+See the PRUNING section below for more details.
+
 endif::git-pull[]
 
 ifndef::git-pull[]
-- 
2.15.1.424.g9478a66081

[PATCH v2 09/12] fetch: don't redundantly NULL something calloc() gave us

[Ævar Arnfjörð Bjarmason]

Stop redundantly NULL-ing the last element of the refs structure,
which was retrieved via calloc(), and is thus guaranteed to be
pre-NULL'd.

This code dates back to b888d61c83 ("Make fetch a builtin",
2007-09-10), where wasn't any reason to do this back then either, it's
just something left over from when git-fetch was initially introduced.

The initial motivation for this change was to make a subsequent change
which'll also modify the refs variable smaller, since it won't have to
copy this redundant "NULL the last + 1 item" pattern.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7bbcd26faf..b34665db9e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			} else
 				refs[j++] = argv[i];
 		}
-		refs[j] = NULL;
 		ref_nr = j;
 	}
 
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 09/12] fetch: don't redundantly NULL something calloc() gave us

[Ævar Arnfjörð Bjarmason]

On Sun, Jan 21 2018, Ævar Arnfjörð Bjarmason jotted:

> 2007-09-10), where wasn't any reason to do this back then either, it's

s/where wasn't/where there wasn't/

> The initial motivation for this change was to make a subsequent change
> which'll also modify the refs variable smaller, since it won't have to
> copy this redundant "NULL the last + 1 item" pattern.

Which, FWIW is not happening later in this series anymore, so this is
just a "cleanup while I'm at it" patch at this point. Will note that in
a v3 commit message pending more comments.

[PATCH v2 10/12] fetch: stop accessing "remote" variable indirectly

[Ævar Arnfjörð Bjarmason]

Access the "remote" variable passed to the fetch_one() directly rather
than through the gtransport wrapper struct constructed in this
function for other purposes.

This makes the code more readable, as it's now obvious that the remote
struct doesn't somehow get munged by the prepare_transport() function
above, which takes the "remote" struct as an argument and constructs
the "gtransport" struct, containing among other things the "remote"
struct.

This pattern of accessing the container struct was added in
737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this
code was initially introduced.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index b34665db9e..a85c2002a9 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 
 	if (prune < 0) {
 		/* no command line request */
-		if (0 <= gtransport->remote->prune)
-			prune = gtransport->remote->prune;
+		if (0 <= remote->prune)
+			prune = remote->prune;
 		else if (0 <= fetch_prune_config)
 			prune = fetch_prune_config;
 		else
-- 
2.15.1.424.g9478a66081

[PATCH v2 11/12] fetch tests: add scaffolding for the new fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

The fetch.pruneTags configuration doesn't exist yet, but will be added
in a subsequent commit. Since testing for it requires adding new
parameters to the test_configured_prune function it's easier to review
this patch first to assert that no functional changes are introduced
yet.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 90 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 49 insertions(+), 41 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 576c2598c9..a1abea7e3f 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,18 +551,22 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	expected_branch=$3
-	expected_tag=$4
-	cmdline=$5
+	fetch_prune_tags=$3
+	remote_origin_prune_tags=$4
+	expected_branch=$5
+	expected_tag=$6
+	cmdline=$7
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
+			test_unconfig fetch.pruneTags &&
 			test_unconfig remote.origin.prune &&
+			test_unconfig remote.origin.pruneTags &&
 			git fetch &&
 			git rev-parse --verify refs/remotes/origin/newbranch &&
 			git rev-parse --verify refs/tags/newtag
@@ -576,7 +580,9 @@ test_configured_prune () {
 		(
 			cd one &&
 			set_config_tristate fetch.prune $fetch_prune &&
+			set_config_tristate fetch.pruneTags $fetch_prune_tags &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
+			set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags &&
 
 			git fetch '"$cmdline"' &&
 			case "$expected_branch" in
@@ -601,57 +607,59 @@ test_configured_prune () {
 
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
-# $3 expect: branch to be pruned?
-# $4 expect: tag to be pruned?
-# $5 git-fetch $cmdline:
+# $3 config: fetch.pruneTags
+# $4 config: remote.<name>.pruneTags
+# $5 expect: branch to be pruned?
+# $6 expect: tag to be pruned?
+# $7 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4     $5
-test_configured_prune unset unset kept   kept   ""
-test_configured_prune unset unset kept   kept   "--no-prune"
-test_configured_prune unset unset pruned kept   "--prune"
-test_configured_prune unset unset kept   pruned \
+#                     $1    $2    $3    $4    $5     $6     $7
+test_configured_prune unset unset unset unset kept   kept   ""
+test_configured_prune unset unset unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset unset unset pruned kept   "--prune"
+test_configured_prune unset unset unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
-test_configured_prune unset unset pruned pruned \
+test_configured_prune unset unset unset unset pruned pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
-test_configured_prune false unset kept   kept   ""
-test_configured_prune false unset kept   kept   "--no-prune"
-test_configured_prune false unset pruned kept   "--prune"
+test_configured_prune false unset unset unset kept   kept   ""
+test_configured_prune false unset unset unset kept   kept   "--no-prune"
+test_configured_prune false unset unset unset pruned kept   "--prune"
 
-test_configured_prune true  unset pruned kept   ""
-test_configured_prune true  unset pruned kept   "--prune"
-test_configured_prune true  unset kept   kept   "--no-prune"
+test_configured_prune true  unset unset unset pruned kept   ""
+test_configured_prune true  unset unset unset pruned kept   "--prune"
+test_configured_prune true  unset unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset false kept   kept   ""
-test_configured_prune unset false kept   kept   "--no-prune"
-test_configured_prune unset false pruned kept   "--prune"
+test_configured_prune unset false unset unset kept   kept   ""
+test_configured_prune unset false unset unset kept   kept   "--no-prune"
+test_configured_prune unset false unset unset pruned kept   "--prune"
 
-test_configured_prune false false kept   kept   ""
-test_configured_prune false false kept   kept   "--no-prune"
-test_configured_prune false false pruned kept   "--prune"
-test_configured_prune false false kept   pruned \
+test_configured_prune false false unset unset kept   kept   ""
+test_configured_prune false false unset unset kept   kept   "--no-prune"
+test_configured_prune false false unset unset pruned kept   "--prune"
+test_configured_prune false false unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
-test_configured_prune false false pruned pruned \
+test_configured_prune false false unset unset pruned pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
-test_configured_prune true  false kept   kept   ""
-test_configured_prune true  false pruned kept   "--prune"
-test_configured_prune true  false kept   kept   "--no-prune"
+test_configured_prune true  false unset unset kept   kept   ""
+test_configured_prune true  false unset unset pruned kept   "--prune"
+test_configured_prune true  false unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset true  pruned kept   ""
-test_configured_prune unset true  kept   kept   "--no-prune"
-test_configured_prune unset true  pruned kept   "--prune"
+test_configured_prune unset true  unset unset pruned kept   ""
+test_configured_prune unset true  unset unset kept   kept   "--no-prune"
+test_configured_prune unset true  unset unset pruned kept   "--prune"
 
-test_configured_prune false true  pruned kept   ""
-test_configured_prune false true  kept   kept   "--no-prune"
-test_configured_prune false true  pruned kept   "--prune"
+test_configured_prune false true  unset unset pruned kept   ""
+test_configured_prune false true  unset unset kept   kept   "--no-prune"
+test_configured_prune false true  unset unset pruned kept   "--prune"
 
-test_configured_prune true  true  pruned kept   ""
-test_configured_prune true  true  pruned kept   "--prune"
-test_configured_prune true  true  kept   kept   "--no-prune"
-test_configured_prune true  true  kept   pruned \
+test_configured_prune true  true  unset unset pruned kept   ""
+test_configured_prune true  true  unset unset pruned kept   "--prune"
+test_configured_prune true  true  unset unset kept   kept   "--no-prune"
+test_configured_prune true  true  unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
-test_configured_prune true  true  pruned pruned \
+test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_expect_success 'all boundary commits are excluded' '
-- 
2.15.1.424.g9478a66081

[PATCH v2 12/12] fetch: add a --fetch-prune option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
config option. This allows for doing any of:

    git fetch -p -P
    git fetch --prune --prune-tags
    git fetch -p -P origin
    git fetch --prune --prune-tags origin

Or simply:

    git config fetch.prune true &&
    git config fetch.pruneTags true &&
    git fetch

Instead of the much more verbose:

    git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'

Before this feature it was painful to support the use-case of pulling
from a repo which is having both its branches *and* tags deleted
regularly, and wanting our local references to reflect upstream.

At work we create deployment tags in the repo for each rollout, and
there's *lots* of those, so they're archived within weeks for
performance reasons.

Without this change it's hard to centrally configure such repos in
/etc/gitconfig (on servers that are only used for working with
them). You need to set fetch.prune=true globally, and then for each
repo:

    git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"

Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well,
and users running "git pull" will automatically get the pruning
semantics we want.

See my "Re: [BUG] git remote prune removes local tags, depending on
fetch config" (87po6ahx87.fsf@evledraar.gmail.com;
https://public-inbox.org/git/87po6ahx87.fsf@evledraar.gmail.com/) for
more background info.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 14 ++++++++++++++
 Documentation/fetch-options.txt | 15 ++++++++++++++-
 Documentation/git-fetch.txt     | 24 ++++++++++++++++++++++++
 builtin/fetch.c                 | 32 ++++++++++++++++++++++++++++++++
 remote.c                        |  5 ++++-
 remote.h                        |  3 +++
 t/t5510-fetch.sh                | 30 ++++++++++++++++++++++++++++++
 7 files changed, 121 insertions(+), 2 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0f27af5760..ae86455876 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1401,6 +1401,14 @@ fetch.prune::
 	option was given on the command line.  See also `remote.<name>.prune`
 	and the PRUNING section of linkgit:git-fetch[1].
 
+fetch.pruneTags::
+	If true, fetch will automatically behave as if the
+	`refs/tags/*:refs/tags/*` refspec was provided when pruning,
+	if not set already. This allows for setting both this option
+	and `fetch.prune` to maintain a 1=1 mapping to upstrem
+	refs. See also `remote.<name>.pruneTags` and the PRUNING
+	section of linkgit:git-fetch[1].
+
 fetch.output::
 	Control how ref update status is printed. Valid values are
 	`full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
+
+remote.<name>.pruneTags::
+	When set to true, fetching from this remote by default will also
+	remove any local tags that no longer exist on the remote if pruning
+	is activated in general via `remote.<name>.prune`, `fetch.prune` or
+	`--prune`. Overrides `fetch.pruneTags` settings, if any.
 +
 See also `remote.<name>.prune` and the PRUNING section of
 linkgit:git-fetch[1].
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 9f5c85ad96..dc13bed741 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -73,7 +73,20 @@ ifndef::git-pull[]
 	are fetched due to an explicit refspec (either on the command
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
-	subject to pruning.
+	subject to pruning. Supplying `--prune-tags` is a shorthand for
+	providing the tag refspec.
++
+See the PRUNING section below for more details.
+
+-P::
+--prune-tags::
+	Before fetching, remove any local tags that no longer exist on
+	the remote if `--prune` is enabled. This option should be used
+	more carefully, unlike `--prune` it will remove any local
+	references (local tags) that have been created. This option is
+	merely a shorthand for providing the explicit tag refspec
+	along with `--prune`, see the discussion about that in its
+	documentation.
 +
 See the PRUNING section below for more details.
 
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index 18fac0da2e..5682ed4ae1 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -148,6 +148,30 @@ So be careful when using this with a refspec like
 `refs/tags/*:refs/tags/*`, or any other refspec which might map
 references from multiple remotes to the same local namespace.
 
+Since keeping up-to-date with both branches and tags on the remote is
+a common use-case the `--prune-tags` option can be supplied along with
+`--prune` to prune local tags that don't exist on the remote. Tag
+pruning can also be enabled with `fetch.pruneTags` or
+`remote.<name>.pruneTags` in the config. See linkgit:git-config[1].
+
+The `--prune-tags` option is equivalent to having
+`refs/tags/*:refs/tags/*` configured in the refspecs for the
+remote. This can lead to some seemingly strange interactions:
+
+------------------------------------------------
+# These both fetch tags
+$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*'
+$ git fetch --no-tags --prune-tags origin
+------------------------------------------------
+
+The reason it doesn't error out when provided without `--prune` or its
+config versions is for flexibility of the configured versions, and to
+maintain a 1=1 mapping between what the command line flags do, and
+what the configuration versions do. It's reasonable to e.g. configure
+`fetch.pruneTags=true` in `~/.gitconfig` to have tags pruned whenever
+`git fetch --prune` is run, without making every invocation of `git
+fetch` without `--prune` an error.
+
 OUTPUT
 ------
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a85c2002a9..046066b6a1 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */
 static int prune = -1; /* unspecified */
 #define PRUNE_BY_DEFAULT 0 /* do we prune by default? */
 
+static int fetch_prune_tags_config = -1; /* unspecified */
+static int prune_tags = -1; /* unspecified */
+#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
 static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;
 static int progress = -1;
 static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(k, "fetch.prunetags")) {
+		fetch_prune_tags_config = git_config_bool(k, v);
+		return 0;
+	}
+
 	if (!strcmp(k, "submodule.recurse")) {
 		int r = git_config_bool(k, v) ?
 			RECURSE_SUBMODULES_ON : RECURSE_SUBMODULES_OFF;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = {
 		    N_("number of submodules fetched in parallel")),
 	OPT_BOOL('p', "prune", &prune,
 		 N_("prune remote-tracking branches no longer on remote")),
+	OPT_BOOL('P', "prune-tags", &prune_tags,
+		 N_("prune local tags not found on remote")),
 	{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
 		    N_("control recursive fetching of submodules"),
 		    PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)
 		argv_array_push(argv, "--dry-run");
 	if (prune != -1)
 		argv_array_push(argv, prune ? "--prune" : "--no-prune");
+	if (prune_tags != -1)
+		argv_array_push(argv, prune_tags ? "--prune-tags" : "--no-prune-tags");
 	if (update_head_ok)
 		argv_array_push(argv, "--update-head-ok");
 	if (force)
@@ -1315,6 +1328,22 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 	return exit_code;
 }
 
+static void maybe_add_prune_tags_refspec(struct remote *remote)
+{
+	if (prune_tags < 0) {
+		/* no command line request */
+		if (0 <= remote->prune_tags)
+			prune_tags = remote->prune_tags;
+		else if (0 <= fetch_prune_tags_config)
+			prune_tags = fetch_prune_tags_config;
+		else
+			prune_tags = PRUNE_TAGS_BY_DEFAULT;
+	}
+
+	if (prune_tags)
+		add_fetch_refspec(remote, xstrdup("refs/tags/*:refs/tags/*"));
+}
+
 int cmd_fetch(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -1368,6 +1397,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	} else if (argc == 0) {
 		/* No arguments -- use default remote */
 		remote = remote_get(NULL);
+		maybe_add_prune_tags_refspec(remote);
 		result = fetch_one(remote, argc, argv);
 	} else if (multiple) {
 		/* All arguments are assumed to be remotes or groups */
@@ -1386,6 +1416,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 		} else {
 			/* Zero or one remotes */
 			remote = remote_get(argv[0]);
+			if (argc == 1)
+				maybe_add_prune_tags_refspec(remote);
 			result = fetch_one(remote, argc-1, argv+1);
 		}
 	}
diff --git a/remote.c b/remote.c
index 4e93753e19..d8716643cb 100644
--- a/remote.c
+++ b/remote.c
@@ -95,7 +95,7 @@ static void add_push_refspec(struct remote *remote, const char *ref)
 	remote->push_refspec[remote->push_refspec_nr++] = ref;
 }
 
-static void add_fetch_refspec(struct remote *remote, const char *ref)
+void add_fetch_refspec(struct remote *remote, const char *ref)
 {
 	ALLOC_GROW(remote->fetch_refspec,
 		   remote->fetch_refspec_nr + 1,
@@ -173,6 +173,7 @@ static struct remote *make_remote(const char *name, int len)
 
 	ret = xcalloc(1, sizeof(struct remote));
 	ret->prune = -1;  /* unspecified */
+	ret->prune_tags = -1;  /* unspecified */
 	ALLOC_GROW(remotes, remotes_nr + 1, remotes_alloc);
 	remotes[remotes_nr++] = ret;
 	ret->name = xstrndup(name, len);
@@ -391,6 +392,8 @@ static int handle_config(const char *key, const char *value, void *cb)
 		remote->skip_default_update = git_config_bool(key, value);
 	else if (!strcmp(subkey, "prune"))
 		remote->prune = git_config_bool(key, value);
+	else if (!strcmp(subkey, "prunetags"))
+		remote->prune_tags = git_config_bool(key, value);
 	else if (!strcmp(subkey, "url")) {
 		const char *v;
 		if (git_config_string(&v, key, value))
diff --git a/remote.h b/remote.h
index 1f6611be21..df6dca24d4 100644
--- a/remote.h
+++ b/remote.h
@@ -47,6 +47,7 @@ struct remote {
 	int skip_default_update;
 	int mirror;
 	int prune;
+	int prune_tags;
 
 	const char *receivepack;
 	const char *uploadpack;
@@ -133,6 +134,8 @@ struct ref {
 #define REF_HEADS	(1u << 1)
 #define REF_TAGS	(1u << 2)
 
+void add_fetch_refspec(struct remote *remote, const char *ref);
+
 extern struct ref *find_ref_by_name(const struct ref *list, const char *name);
 
 struct ref *alloc_ref(const char *name);
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index a1abea7e3f..48f49b613a 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -662,6 +662,36 @@ test_configured_prune true  true  unset unset kept   pruned \
 test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
+
+# --prune-tags on its own does nothing, needs --prune as well, same
+# for for fetch.pruneTags without fetch.prune
+test_configured_prune unset unset unset unset kept kept     "--prune-tags"
+test_configured_prune unset unset unset unset kept kept     "origin --prune-tags"
+test_configured_prune unset unset true unset  kept kept     ""
+test_configured_prune unset unset unset true  kept kept     ""
+
+# These will prune the tags
+test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags"
+test_configured_prune unset unset unset unset pruned pruned "origin --prune --prune-tags"
+test_configured_prune true  unset true  unset pruned pruned ""
+test_configured_prune unset true  unset true  pruned pruned ""
+
+# Check that remote.<name>.pruneTags overrides fetch.pruneTags as with
+# remote.<name>.prune and fetch.prune
+test_configured_prune false true  false true  pruned pruned ""
+test_configured_prune true  false true  false kept   kept   ""
+
+# When --prune-tags is supplied it's ignored if an explict refspec is
+# given, same for the configuration options.
+test_configured_prune unset unset unset unset pruned kept \
+	"--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
+test_configured_prune unset unset true  unset pruned kept \
+	"--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
+test_configured_prune unset unset unset true pruned  kept \
+	"--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
+
+test_configured_prune true unset true unset pruned pruned   ""
+
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
 	test_commit oneside &&
-- 
2.15.1.424.g9478a66081

Re: [PATCH v2 12/12] fetch: add a --fetch-prune option and fetch.pruneTags config

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
> config option. This allows for doing any of:
>
>     git fetch -p -P
>     git fetch --prune --prune-tags
>     git fetch -p -P origin
>     git fetch --prune --prune-tags origin
>
> Or simply:
>
>     git config fetch.prune true &&
>     git config fetch.pruneTags true &&
>     git fetch
>
> Instead of the much more verbose:
>
>     git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'
>
> Before this feature it was painful to support the use-case of pulling
> from a repo which is having both its branches *and* tags deleted
> regularly, and wanting our local references to reflect upstream.
>
> At work we create deployment tags in the repo for each rollout, and
> there's *lots* of those, so they're archived within weeks for
> performance reasons.
>
> Without this change it's hard to centrally configure such repos in
> /etc/gitconfig (on servers that are only used for working with
> them). You need to set fetch.prune=true globally, and then for each
> repo:
>
>     git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"

I think the last one is supposed to be a regular expression on
existing values.  Shouldn't the asterisks be quoted?  

Otherwise, it would appears as if "refs/tags:refs/tags///" are
replaced with "refs/tags/*:refs/tags/*", but it certainly is not
what you are doing.  I also wonder why the existing one does not
expect a leading '+', which I think is what we place by default
when you clone.

> +-P::
> +--prune-tags::
> +	.... This option is
> +	merely a shorthand for providing the explicit tag refspec
> +	along with `--prune`, see the discussion about that in its
> +	documentation.

So would "git fetch -P origin" be like "git fetch --prune --tags
origin"?

>  +
>  See the PRUNING section below for more details.
>  
> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
> index 18fac0da2e..5682ed4ae1 100644
> --- a/Documentation/git-fetch.txt
> +++ b/Documentation/git-fetch.txt
> @@ -148,6 +148,30 @@ So be careful when using this with a refspec like
>  `refs/tags/*:refs/tags/*`, or any other refspec which might map
>  references from multiple remotes to the same local namespace.
>  
> +Since keeping up-to-date with both branches and tags on the remote is
> +a common use-case the `--prune-tags` option can be supplied along with
> +`--prune` to prune local tags that don't exist on the remote. Tag
> +pruning can also be enabled with `fetch.pruneTags` or
> +`remote.<name>.pruneTags` in the config. See linkgit:git-config[1].
> +
> +The `--prune-tags` option is equivalent to having
> +`refs/tags/*:refs/tags/*` configured in the refspecs for the
> +remote. This can lead to some seemingly strange interactions:
> +
> +------------------------------------------------
> +# These both fetch tags
> +$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*'
> +$ git fetch --no-tags --prune-tags origin
> +------------------------------------------------

This description is too confusing.  First you say "having
... configured in the refspecs for the remote", but configured
refspecs for the remote (I presume that you are missing 'fetch' from
that description and are talking about the "remote.$name.fetch"
configuration variable) are overridden when you give explicit
refspecs from the command line, so the above two are not even
equivalent.  The first one gives a refspec explicitly from the
command line, so other configured refspecs like

    [remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/*

should be ignored, while the second one, if --prune-tags tells Git
to behave as if 

    [remote "origin"] fetch = refs/tags/*:refs/tags/*

also exists in the config, would not cause other ones for the same
remote from getting ignored.  So...

Re: [PATCH v2 12/12] fetch: add a --fetch-prune option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

On Mon, Jan 22 2018, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
>> config option. This allows for doing any of:
>>
>>     git fetch -p -P
>>     git fetch --prune --prune-tags
>>     git fetch -p -P origin
>>     git fetch --prune --prune-tags origin
>>
>> Or simply:
>>
>>     git config fetch.prune true &&
>>     git config fetch.pruneTags true &&
>>     git fetch
>>
>> Instead of the much more verbose:
>>
>>     git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'
>>
>> Before this feature it was painful to support the use-case of pulling
>> from a repo which is having both its branches *and* tags deleted
>> regularly, and wanting our local references to reflect upstream.
>>
>> At work we create deployment tags in the repo for each rollout, and
>> there's *lots* of those, so they're archived within weeks for
>> performance reasons.
>>
>> Without this change it's hard to centrally configure such repos in
>> /etc/gitconfig (on servers that are only used for working with
>> them). You need to set fetch.prune=true globally, and then for each
>> repo:
>>
>>     git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"
>
> I think the last one is supposed to be a regular expression on
> existing values.  Shouldn't the asterisks be quoted?
>
> Otherwise, it would appears as if "refs/tags:refs/tags///" are
> replaced with "refs/tags/*:refs/tags/*", but it certainly is not
> what you are doing.

Yes, well spotted. I copied this from an escaped version and forgot to
update that escaping. Will fix.

> I also wonder why the existing one does not expect a leading '+',
> which I think is what we place by default when you clone.

I didn't raise this because I didn't want to get side-tracked with yet
another thing, but it appears to me that a + prefix in tags refspecs
does absolutely nothing. Consider on a fresh git.git clobbering the new
v2.16.1 tag locally:

    $ git tag -a -m"fake" -f v2.16.1 v2.16.0
    Updated tag 'v2.16.1' (was eb5fcb24f)

This should clobber it, and does:

    $ git fetch --prune --dry-run origin '+refs/tags/*:refs/tags/*'
    From github.com:git/git
     t [tag update]          v2.16.1    -> v2.16.1

But so will this without +, which seems like a bug to me:

    $ git fetch --prune --dry-run origin 'refs/tags/*:refs/tags/*'
    From github.com:git/git
     t [tag update]          v2.16.1    -> v2.16.1

But maybe I'm missing something.


>> +-P::
>> +--prune-tags::
>> +	.... This option is
>> +	merely a shorthand for providing the explicit tag refspec
>> +	along with `--prune`, see the discussion about that in its
>> +	documentation.
>
> So would "git fetch -P origin" be like "git fetch --prune --tags
> origin"?

No, as I understand it --tags just has the effect of considering remote
tags that aren't tags of the branches you're fetching, so e.g. on a
fresh git.git:

    $ git tag -a -m"fake" -f v2.16.2 v2.16.0
    $ git fetch --prune --tags --dry-run
    $

I.e. it does nothing. Whereas this will prune the new fake tag:

    $ git fetch --prune origin 'refs/tags/*:refs/tags/*'
    From github.com:git/git
     - [deleted]             (none)     -> v2.16.2

And so does the --prune-tags option:

    $ ~/g/git/git-fetch --prune --prune-tags origin
    From github.com:git/git
     - [deleted]             (none)     -> v2.16.2


>>  +
>>  See the PRUNING section below for more details.
>>
>> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
>> index 18fac0da2e..5682ed4ae1 100644
>> --- a/Documentation/git-fetch.txt
>> +++ b/Documentation/git-fetch.txt
>> @@ -148,6 +148,30 @@ So be careful when using this with a refspec like
>>  `refs/tags/*:refs/tags/*`, or any other refspec which might map
>>  references from multiple remotes to the same local namespace.
>>
>> +Since keeping up-to-date with both branches and tags on the remote is
>> +a common use-case the `--prune-tags` option can be supplied along with
>> +`--prune` to prune local tags that don't exist on the remote. Tag
>> +pruning can also be enabled with `fetch.pruneTags` or
>> +`remote.<name>.pruneTags` in the config. See linkgit:git-config[1].
>> +
>> +The `--prune-tags` option is equivalent to having
>> +`refs/tags/*:refs/tags/*` configured in the refspecs for the
>> +remote. This can lead to some seemingly strange interactions:
>> +
>> +------------------------------------------------
>> +# These both fetch tags
>> +$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*'
>> +$ git fetch --no-tags --prune-tags origin
>> +------------------------------------------------
>
> This description is too confusing.  First you say "having
> ... configured in the refspecs for the remote", but configured
> refspecs for the remote (I presume that you are missing 'fetch' from
> that description and are talking about the "remote.$name.fetch"
> configuration variable) are overridden when you give explicit
> refspecs from the command line, so the above two are not even
> equivalent.  The first one gives a refspec explicitly from the
> command line, so other configured refspecs like
>
>     [remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/*
>
> should be ignored, while the second one, if --prune-tags tells Git
> to behave as if
>
>     [remote "origin"] fetch = refs/tags/*:refs/tags/*
>
> also exists in the config, would not cause other ones for the same
> remote from getting ignored.  So...

Indeed, I'll reword this. FWIW I meant something closer to "declared in
the refspecs of the remote" not "configured ... remote".

[PATCH 01/11] fetch tests: refactor in preparation for testing tag pruning

[Ævar Arnfjörð Bjarmason]

In a subsequent commit this function will learn to test for tag
pruning, prepare for that by making space for more variables, and
making it clear that "expected" here refers to branches.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 668c54be41..11da97f9b7 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -549,9 +549,12 @@ set_config_tristate () {
 }
 
 test_configured_prune () {
-	fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4
+	fetch_prune=$1
+	remote_origin_prune=$2
+	cmdline=$3
+	expected_branch=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -572,7 +575,7 @@ test_configured_prune () {
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
 			git fetch $cmdline &&
-			case "$expected" in
+			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
 				;;
-- 
2.15.1.424.g9478a66081

[PATCH 02/11] fetch tests: arrange arguments for future readability

[Ævar Arnfjörð Bjarmason]

Re-arrange the arguments to the test_configured_prune() function used
in this test to pass the arguments to --fetch last. A subsequent
change will test for more elaborate fetch arguments, including long
refspecs. It'll be more readable to be able to wrap those on a new
line of their own.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 44 insertions(+), 38 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 11da97f9b7..ab8b25344d 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,10 +551,10 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	cmdline=$3
-	expected_branch=$4
+	expected_branch=$3
+	cmdline=$4
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		(
@@ -587,41 +587,47 @@ test_configured_prune () {
 	'
 }
 
-test_configured_prune unset unset ""		kept
-test_configured_prune unset unset "--no-prune"	kept
-test_configured_prune unset unset "--prune"	pruned
-
-test_configured_prune false unset ""		kept
-test_configured_prune false unset "--no-prune"	kept
-test_configured_prune false unset "--prune"	pruned
-
-test_configured_prune true  unset ""		pruned
-test_configured_prune true  unset "--prune"	pruned
-test_configured_prune true  unset "--no-prune"	kept
-
-test_configured_prune unset false ""		kept
-test_configured_prune unset false "--no-prune"	kept
-test_configured_prune unset false "--prune"	pruned
-
-test_configured_prune false false ""		kept
-test_configured_prune false false "--no-prune"	kept
-test_configured_prune false false "--prune"	pruned
-
-test_configured_prune true  false ""		kept
-test_configured_prune true  false "--prune"	pruned
-test_configured_prune true  false "--no-prune"	kept
-
-test_configured_prune unset true  ""		pruned
-test_configured_prune unset true  "--no-prune"	kept
-test_configured_prune unset true  "--prune"	pruned
-
-test_configured_prune false true  ""		pruned
-test_configured_prune false true  "--no-prune"	kept
-test_configured_prune false true  "--prune"	pruned
-
-test_configured_prune true  true  ""		pruned
-test_configured_prune true  true  "--prune"	pruned
-test_configured_prune true  true  "--no-prune"	kept
+# $1 config: fetch.prune
+# $2 config: remote.<name>.prune
+# $3 expect: branch to be pruned?
+# $4 git-fetch $cmdline:
+#
+#                     $1    $2    $3     $4
+test_configured_prune unset unset kept   ""
+test_configured_prune unset unset kept   "--no-prune"
+test_configured_prune unset unset pruned "--prune"
+
+test_configured_prune false unset kept   ""
+test_configured_prune false unset kept   "--no-prune"
+test_configured_prune false unset pruned "--prune"
+
+test_configured_prune true  unset pruned ""
+test_configured_prune true  unset pruned "--prune"
+test_configured_prune true  unset kept   "--no-prune"
+
+test_configured_prune unset false kept   ""
+test_configured_prune unset false kept   "--no-prune"
+test_configured_prune unset false pruned "--prune"
+
+test_configured_prune false false kept   ""
+test_configured_prune false false kept   "--no-prune"
+test_configured_prune false false pruned "--prune"
+
+test_configured_prune true  false kept   ""
+test_configured_prune true  false pruned "--prune"
+test_configured_prune true  false kept   "--no-prune"
+
+test_configured_prune unset true  pruned ""
+test_configured_prune unset true  kept   "--no-prune"
+test_configured_prune unset true  pruned "--prune"
+
+test_configured_prune false true  pruned ""
+test_configured_prune false true  kept   "--no-prune"
+test_configured_prune false true  pruned "--prune"
+
+test_configured_prune true  true  pruned ""
+test_configured_prune true  true  pruned "--prune"
+test_configured_prune true  true  kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH 03/11] fetch tests: add a tag to be deleted to the pruning tests

[Ævar Arnfjörð Bjarmason]

Add a tag to be deleted to the fetch --prune tests. The tag is always
kept for now, which is the expected behavior, but now I can add a test
for tag pruning in a later commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------
 1 file changed, 53 insertions(+), 40 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index ab8b25344d..fad65bd885 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -552,21 +552,25 @@ test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
 	expected_branch=$3
-	cmdline=$4
+	expected_tag=$4
+	cmdline=$5
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" '
+	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
+		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
 			test_unconfig remote.origin.prune &&
 			git fetch &&
-			git rev-parse --verify refs/remotes/origin/newbranch
+			git rev-parse --verify refs/remotes/origin/newbranch &&
+			git rev-parse --verify refs/tags/newtag
 		) &&
 
 		# now remove it
 		git branch -d newbranch &&
+		git tag -d newtag &&
 
 		# then test
 		(
@@ -582,6 +586,14 @@ test_configured_prune () {
 			kept)
 				git rev-parse --verify refs/remotes/origin/newbranch
 				;;
+			esac &&
+			case "$expected_tag" in
+			pruned)
+				test_must_fail git rev-parse --verify refs/tags/newtag
+				;;
+			kept)
+				git rev-parse --verify refs/tags/newtag
+				;;
 			esac
 		)
 	'
@@ -590,44 +602,45 @@ test_configured_prune () {
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
 # $3 expect: branch to be pruned?
-# $4 git-fetch $cmdline:
+# $4 expect: tag to be pruned?
+# $5 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4
-test_configured_prune unset unset kept   ""
-test_configured_prune unset unset kept   "--no-prune"
-test_configured_prune unset unset pruned "--prune"
-
-test_configured_prune false unset kept   ""
-test_configured_prune false unset kept   "--no-prune"
-test_configured_prune false unset pruned "--prune"
-
-test_configured_prune true  unset pruned ""
-test_configured_prune true  unset pruned "--prune"
-test_configured_prune true  unset kept   "--no-prune"
-
-test_configured_prune unset false kept   ""
-test_configured_prune unset false kept   "--no-prune"
-test_configured_prune unset false pruned "--prune"
-
-test_configured_prune false false kept   ""
-test_configured_prune false false kept   "--no-prune"
-test_configured_prune false false pruned "--prune"
-
-test_configured_prune true  false kept   ""
-test_configured_prune true  false pruned "--prune"
-test_configured_prune true  false kept   "--no-prune"
-
-test_configured_prune unset true  pruned ""
-test_configured_prune unset true  kept   "--no-prune"
-test_configured_prune unset true  pruned "--prune"
-
-test_configured_prune false true  pruned ""
-test_configured_prune false true  kept   "--no-prune"
-test_configured_prune false true  pruned "--prune"
-
-test_configured_prune true  true  pruned ""
-test_configured_prune true  true  pruned "--prune"
-test_configured_prune true  true  kept   "--no-prune"
+#                     $1    $2    $3     $4     $5
+test_configured_prune unset unset kept   kept   ""
+test_configured_prune unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset pruned kept   "--prune"
+
+test_configured_prune false unset kept   kept   ""
+test_configured_prune false unset kept   kept   "--no-prune"
+test_configured_prune false unset pruned kept   "--prune"
+
+test_configured_prune true  unset pruned kept   ""
+test_configured_prune true  unset pruned kept   "--prune"
+test_configured_prune true  unset kept   kept   "--no-prune"
+
+test_configured_prune unset false kept   kept   ""
+test_configured_prune unset false kept   kept   "--no-prune"
+test_configured_prune unset false pruned kept   "--prune"
+
+test_configured_prune false false kept   kept   ""
+test_configured_prune false false kept   kept   "--no-prune"
+test_configured_prune false false pruned kept   "--prune"
+
+test_configured_prune true  false kept   kept   ""
+test_configured_prune true  false pruned kept   "--prune"
+test_configured_prune true  false kept   kept   "--no-prune"
+
+test_configured_prune unset true  pruned kept   ""
+test_configured_prune unset true  kept   kept   "--no-prune"
+test_configured_prune unset true  pruned kept   "--prune"
+
+test_configured_prune false true  pruned kept   ""
+test_configured_prune false true  kept   kept   "--no-prune"
+test_configured_prune false true  pruned kept   "--prune"
+
+test_configured_prune true  true  pruned kept   ""
+test_configured_prune true  true  pruned kept   "--prune"
+test_configured_prune true  true  kept   kept   "--no-prune"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH 04/11] fetch tests: double quote a variable for interpolation

[Ævar Arnfjörð Bjarmason]

If the $cmdline variable contains multiple arguments they won't be
interpolated correctly since the body of the test is single quoted. I
don't know what part of test-lib.sh is expanding variables within
single-quoted strings, but interpolating this inline is the desired
behavior here.

This will be used in a subsequent commit to pass more than one
variable to git-fetch.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index fad65bd885..542eb53a99 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -578,7 +578,7 @@ test_configured_prune () {
 			set_config_tristate fetch.prune $fetch_prune &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
 
-			git fetch $cmdline &&
+			git fetch '"$cmdline"' &&
 			case "$expected_branch" in
 			pruned)
 				test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
-- 
2.15.1.424.g9478a66081

[PATCH 05/11] fetch tests: test --prune and refspec interaction

[Ævar Arnfjörð Bjarmason]

Add a test for the interaction between explicitly provided refspecs
and fetch.prune.

There's no point in adding this boilerplate to every combination of
unset/false/true, it's instructive and sufficient to show that no
matter if the variable is unset, false or true the refspec on the
command-line overrides any configuration variable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 542eb53a99..576c2598c9 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -609,6 +609,10 @@ test_configured_prune () {
 test_configured_prune unset unset kept   kept   ""
 test_configured_prune unset unset kept   kept   "--no-prune"
 test_configured_prune unset unset pruned kept   "--prune"
+test_configured_prune unset unset kept   pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*'"
+test_configured_prune unset unset pruned pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_configured_prune false unset kept   kept   ""
 test_configured_prune false unset kept   kept   "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept   "--prune"
 test_configured_prune false false kept   kept   ""
 test_configured_prune false false kept   kept   "--no-prune"
 test_configured_prune false false pruned kept   "--prune"
+test_configured_prune false false kept   pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*'"
+test_configured_prune false false pruned pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_configured_prune true  false kept   kept   ""
 test_configured_prune true  false pruned kept   "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true  pruned kept   "--prune"
 test_configured_prune true  true  pruned kept   ""
 test_configured_prune true  true  pruned kept   "--prune"
 test_configured_prune true  true  kept   kept   "--no-prune"
+test_configured_prune true  true  kept   pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*'"
+test_configured_prune true  true  pruned pruned \
+	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_expect_success 'all boundary commits are excluded' '
 	test_commit base &&
-- 
2.15.1.424.g9478a66081

[PATCH 06/11] git fetch doc: add a new section to explain the ins & outs of pruning

[Ævar Arnfjörð Bjarmason]

Add a new section to canonically explain how remote reference pruning
works, and how users should be careful about using it in conjunction
with tag refspecs in particular.

A subsequent commit will update the git-remote documentation to refer
to this section, and details the motivation for writing this in the
first place.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
index b153aefa68..b07b669a1f 100644
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be
 overridden by giving the `--refmap=<refspec>` parameter(s) on the
 command line.
 
+PRUNING
+-------
+
+Git has a default disposition to keeping data unless it's explicitly
+thrown away, this extends to keeping a hold of local references to
+branches on remotes that have themselves deleted those branches.
+
+If left to accumulate these stale references might make performance
+worse on big and busy repos that have a lot of branch churn, and
+e.g. make the output of commands like `git branch -a --contains
+<commit>` needlessly verbose, as well as impacting anything else
+that'll work with the complete set of known references.
+
+These remote tracking references can be deleted as a one-off with
+either of:
+
+------------------------------------------------
+# While fetching
+$ git fetch --prune <name>
+
+# Only prune, don't fetch
+$ git remote <name>
+------------------------------------------------
+
+To prune references on a remote as part of your normal workflow
+without needing to remember to run that set `fetch.prune` globally, or
+`remote.<name>.prune` per-remote in the config. See
+linkgit:git-config[1].
+
+Here's where things get tricky and more specific. The pruning feature
+doesn't actually care about branches, instead it'll prune local <->
+remote references as a function of the refspec of the remote (see
+`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).
+
+Therefore if the refspec for the remote includes
+e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch
+--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote
+tracking branches that are deleted, but any local tag that doesn't
+exist on the remote.
+
+This might not be what you expect, i.e. you want to prune remote
+`<name>`, but also explicitly fetch tags from it, so when you fetch
+from it you delete all your local tags, most of which may not have
+come from the `<name>` remote in the first place.
+
+So be careful when using this with a refspec like
+`refs/tags/*:refs/tags/*`, or any other refspec which might map
+references from multiple remotes to the same local namespace.
+
 OUTPUT
 ------
 
-- 
2.15.1.424.g9478a66081

Re: [PATCH 06/11] git fetch doc: add a new section to explain the ins & outs of pruning

[Eric Sunshine]

On Thu, Jan 18, 2018 at 7:00 PM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Add a new section to canonically explain how remote reference pruning
> works, and how users should be careful about using it in conjunction
> with tag refspecs in particular.
>
> A subsequent commit will update the git-remote documentation to refer
> to this section, and details the motivation for writing this in the
> first place.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
> @@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be
> +PRUNING
> +-------
> +
> +Git has a default disposition to keeping data unless it's explicitly

s/keeping/keep/ or s/to keeping/of keeping/

> +thrown away, this extends to keeping a hold of local references to

s/away,/away;/
s/to keeping a hold of/holding onto/

> +branches on remotes that have themselves deleted those branches.
> +
> +If left to accumulate these stale references might make performance

s/accumulate/&,/

> +worse on big and busy repos that have a lot of branch churn, and
> +e.g. make the output of commands like `git branch -a --contains
> +<commit>` needlessly verbose, as well as impacting anything else
> +that'll work with the complete set of known references.
> +
> +These remote tracking references can be deleted as a one-off with
> +either of:
> +
> +------------------------------------------------
> +# While fetching
> +$ git fetch --prune <name>
> +
> +# Only prune, don't fetch
> +$ git remote <name>
> +------------------------------------------------

Did you mean "git remote prune <name>"?

> +To prune references on a remote as part of your normal workflow

This reads as if it's possible to prune something on the remote
machine itself. Maybe just say:

    To prune references as part of your normal workflow...

> +without needing to remember to run that set `fetch.prune` globally, or
> +`remote.<name>.prune` per-remote in the config. See
> +linkgit:git-config[1].

[PATCH 07/11] git remote doc: correct dangerous lies about what prune does

[Ævar Arnfjörð Bjarmason]

The "git remote prune <name>" command uses the same machinery as "git
fetch <name> --prune", and shares all the same caveats, but its
documentation has suggested that it'll just "delete stale
remote-tracking branches under <name>".

This isn't true, and hasn't been true since at least v1.8.5.6 (the
oldest version I could be bothered to test).

E.g. if "+refs/tags/*:refs/tags/*" is explicitly set in the refspec of
the remote it'll delete all local tags <name> doesn't know about.

Instead, briefly give the reader just enough of a hint that this
option might constitute a shotgun aimed at their foot, and point them
to the new PRUNING section in the git-fetch documentation which
explains all the nuances of what this facility does.

See "[BUG] git remote prune removes local tags, depending on fetch
config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com)
by Michael Giuffrida for the initial report.

Reported-by: Michael Giuffrida <michaelpg@chromium.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-remote.txt | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-remote.txt b/Documentation/git-remote.txt
index 577b969c1b..7183a72a23 100644
--- a/Documentation/git-remote.txt
+++ b/Documentation/git-remote.txt
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with
 
 'prune'::
 
-Deletes all stale remote-tracking branches under <name>.
-These stale branches have already been removed from the remote repository
-referenced by <name>, but are still locally available in
-"remotes/<name>".
+Deletes stale references associated with <name>. By default stale
+remote-tracking branches under <name>, but depending on global
+configuration and the configuration of the remote we might even prune
+local tags. Equivalent to `git fetch <name> --prune`, except that no
+new references will be fetched.
++
+See the PRUNING section of linkgit:git-fetch[1] for what it'll prune
+depending on various configuration.
 +
 With `--dry-run` option, report what branches will be pruned, but do not
 actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the
 configuration parameter remote.<name>.skipDefaultUpdate set to true will
 be updated.  (See linkgit:git-config[1]).
 +
-With `--prune` option, prune all the remotes that are updated.
+With `--prune` option, run pruning against all the remotes that are updated.
 
 
 DISCUSSION
-- 
2.15.1.424.g9478a66081

[PATCH 08/11] git-fetch & config doc: link to the new PRUNING section

[Ævar Arnfjörð Bjarmason]

Amend the documentation for fetch.prune, fetch.<name>.prune and
--prune to link to the recently added PRUNING section.

I'd have liked to link directly to it with "<<PRUNING>>" from
fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt
also includes it, but doesn't include that option). However making a
reference across files yields this error:

    [...]/Documentation/git-fetch.xml:226: element xref: validity
    error : IDREF attribute linkend references an unknown ID "PRUNING"

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 6 +++++-
 Documentation/fetch-options.txt | 3 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b..0f27af5760 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1398,7 +1398,8 @@ fetch.unpackLimit::
 
 fetch.prune::
 	If true, fetch will automatically behave as if the `--prune`
-	option was given on the command line.  See also `remote.<name>.prune`.
+	option was given on the command line.  See also `remote.<name>.prune`
+	and the PRUNING section of linkgit:git-fetch[1].
 
 fetch.output::
 	Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
++
+See also `remote.<name>.prune` and the PRUNING section of
+linkgit:git-fetch[1].
 
 remotes.<group>::
 	The list of remotes which are fetched by "git remote update
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index fb6bebbc61..9f5c85ad96 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -74,6 +74,9 @@ ifndef::git-pull[]
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
 	subject to pruning.
++
+See the PRUNING section below for more details.
+
 endif::git-pull[]
 
 ifndef::git-pull[]
-- 
2.15.1.424.g9478a66081

[PATCH 09/11] fetch: don't redundantly null something calloc() gave us

[Ævar Arnfjörð Bjarmason]

Stop redundantly NULL-ing the last element of the refs structure,
which was retrieved via calloc() and is thus guaranteed to be
pre-NULL'd.

This code dates back to b888d61c83 ("Make fetch a builtin",
2007-09-10), where wasn't any reason to do this back then either, it's
just something left over from when git-fetch was initially introduced.

The initial motivation for this change was to make a subsequent change
which'll also modify the refs variable smaller, since it won't have to
copy this redundant "NULL the last + 1 item" pattern.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 7bbcd26faf..b34665db9e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			} else
 				refs[j++] = argv[i];
 		}
-		refs[j] = NULL;
 		ref_nr = j;
 	}
 
-- 
2.15.1.424.g9478a66081

[PATCH 10/11] fetch tests: add scaffolding for the new fetch.pruneTags

[Ævar Arnfjörð Bjarmason]

The fetch.pruneTags configuration doesn't exist yet, but will be added
in a subsequent commit. Since testing for it requires adding new
parameters to the test_configured_prune function it's easier to review
this patch first to assert that no functional changes are introduced
yet.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5510-fetch.sh | 90 ++++++++++++++++++++++++++++++--------------------------
 1 file changed, 49 insertions(+), 41 deletions(-)

diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 576c2598c9..18280df4fc 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -551,18 +551,22 @@ set_config_tristate () {
 test_configured_prune () {
 	fetch_prune=$1
 	remote_origin_prune=$2
-	expected_branch=$3
-	expected_tag=$4
-	cmdline=$5
+	fetch_prune_tags=$3
+	remote_origin_prune_tags=$4
+	expected_branch=$5
+	expected_tag=$6
+	cmdline=$7
 
-	test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+	test_expect_success "prune fetch.prune=$1 fetch.pruneTags=$2 remote.origin.prune=$3 remote.origin.prune=$4${7:+ $7}; branch:$5 tag:$6" '
 		# make sure a newbranch is there in . and also in one
 		git branch -f newbranch &&
 		git tag -f newtag &&
 		(
 			cd one &&
 			test_unconfig fetch.prune &&
+			test_unconfig fetch.pruneTags &&
 			test_unconfig remote.origin.prune &&
+			test_unconfig remote.origin.pruneTags &&
 			git fetch &&
 			git rev-parse --verify refs/remotes/origin/newbranch &&
 			git rev-parse --verify refs/tags/newtag
@@ -576,7 +580,9 @@ test_configured_prune () {
 		(
 			cd one &&
 			set_config_tristate fetch.prune $fetch_prune &&
+			set_config_tristate fetch.pruneTags $fetch_prune_tags &&
 			set_config_tristate remote.origin.prune $remote_origin_prune &&
+			set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags &&
 
 			git fetch '"$cmdline"' &&
 			case "$expected_branch" in
@@ -601,57 +607,59 @@ test_configured_prune () {
 
 # $1 config: fetch.prune
 # $2 config: remote.<name>.prune
-# $3 expect: branch to be pruned?
-# $4 expect: tag to be pruned?
-# $5 git-fetch $cmdline:
+# $3 config: fetch.pruneTags
+# $4 config: remote.<name>.pruneTags
+# $5 expect: branch to be pruned?
+# $6 expect: tag to be pruned?
+# $7 git-fetch $cmdline:
 #
-#                     $1    $2    $3     $4     $5
-test_configured_prune unset unset kept   kept   ""
-test_configured_prune unset unset kept   kept   "--no-prune"
-test_configured_prune unset unset pruned kept   "--prune"
-test_configured_prune unset unset kept   pruned \
+#                     $1    $2    $3    $4    $5     $6     $7
+test_configured_prune unset unset unset unset kept   kept   ""
+test_configured_prune unset unset unset unset kept   kept   "--no-prune"
+test_configured_prune unset unset unset unset pruned kept   "--prune"
+test_configured_prune unset unset unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
-test_configured_prune unset unset pruned pruned \
+test_configured_prune unset unset unset unset pruned pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
-test_configured_prune false unset kept   kept   ""
-test_configured_prune false unset kept   kept   "--no-prune"
-test_configured_prune false unset pruned kept   "--prune"
+test_configured_prune false unset unset unset kept   kept   ""
+test_configured_prune false unset unset unset kept   kept   "--no-prune"
+test_configured_prune false unset unset unset pruned kept   "--prune"
 
-test_configured_prune true  unset pruned kept   ""
-test_configured_prune true  unset pruned kept   "--prune"
-test_configured_prune true  unset kept   kept   "--no-prune"
+test_configured_prune true  unset unset unset pruned kept   ""
+test_configured_prune true  unset unset unset pruned kept   "--prune"
+test_configured_prune true  unset unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset false kept   kept   ""
-test_configured_prune unset false kept   kept   "--no-prune"
-test_configured_prune unset false pruned kept   "--prune"
+test_configured_prune unset false unset unset kept   kept   ""
+test_configured_prune unset false unset unset kept   kept   "--no-prune"
+test_configured_prune unset false unset unset pruned kept   "--prune"
 
-test_configured_prune false false kept   kept   ""
-test_configured_prune false false kept   kept   "--no-prune"
-test_configured_prune false false pruned kept   "--prune"
-test_configured_prune false false kept   pruned \
+test_configured_prune false false unset unset kept   kept   ""
+test_configured_prune false false unset unset kept   kept   "--no-prune"
+test_configured_prune false false unset unset pruned kept   "--prune"
+test_configured_prune false false unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
-test_configured_prune false false pruned pruned \
+test_configured_prune false false unset unset pruned pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
-test_configured_prune true  false kept   kept   ""
-test_configured_prune true  false pruned kept   "--prune"
-test_configured_prune true  false kept   kept   "--no-prune"
+test_configured_prune true  false unset unset kept   kept   ""
+test_configured_prune true  false unset unset pruned kept   "--prune"
+test_configured_prune true  false unset unset kept   kept   "--no-prune"
 
-test_configured_prune unset true  pruned kept   ""
-test_configured_prune unset true  kept   kept   "--no-prune"
-test_configured_prune unset true  pruned kept   "--prune"
+test_configured_prune unset true  unset unset pruned kept   ""
+test_configured_prune unset true  unset unset kept   kept   "--no-prune"
+test_configured_prune unset true  unset unset pruned kept   "--prune"
 
-test_configured_prune false true  pruned kept   ""
-test_configured_prune false true  kept   kept   "--no-prune"
-test_configured_prune false true  pruned kept   "--prune"
+test_configured_prune false true  unset unset pruned kept   ""
+test_configured_prune false true  unset unset kept   kept   "--no-prune"
+test_configured_prune false true  unset unset pruned kept   "--prune"
 
-test_configured_prune true  true  pruned kept   ""
-test_configured_prune true  true  pruned kept   "--prune"
-test_configured_prune true  true  kept   kept   "--no-prune"
-test_configured_prune true  true  kept   pruned \
+test_configured_prune true  true  unset unset pruned kept   ""
+test_configured_prune true  true  unset unset pruned kept   "--prune"
+test_configured_prune true  true  unset unset kept   kept   "--no-prune"
+test_configured_prune true  true  unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
-test_configured_prune true  true  pruned pruned \
+test_configured_prune true  true  unset unset pruned pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
 
 test_expect_success 'all boundary commits are excluded' '
-- 
2.15.1.424.g9478a66081

[RFC/PATCH 11/11] WIP fetch: add a --fetch-prune option and fetch.pruneTags config

[Ævar Arnfjörð Bjarmason]

[WIP: This doesn't (yet) work as advertised, see further WIP note
below]

Add a --fetch-prune option to git-fetch along with fetch.pruneTags
config option. This allows for doing:

    git fetch origin -p -P

Or simply:

    git config fetch.prune true &&
    git config fetch.pruneTags true &&
    git fetch origin

Instead of the much more verbose:

    git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'

A use-case of git which is a pain to support now is frequent pulling
from a repo which is having both its branches *and* tags deleted
regularly. At work we create deployment tags in the repo for each
rollout, and there's *lots* of those, so they're archived within weeks
for performance reasons.

Without this change it's hard to centrally configure such repos in
/etc/gitconfig (on servers that are only used for working with
them). You need to set fetch.prune=true globally, and then for each
repo:

    git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"

Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well,
and users running "git pull" will automatically get the pruning
semantics we want.

See my "Re: [BUG] git remote prune removes local tags, depending on
fetch config" (87po6ahx87.fsf@evledraar.gmail.com;
https://public-inbox.org/git/87po6ahx87.fsf@evledraar.gmail.com/) for
more details.

[WIP: This doesn't actually work, since the implementation is to just
pretend that 'refs/tags/*:refs/tags/*' was supplied on the
command-line, we'll consequently stop fetching any branches, since we
don't use the default refspec found in .git/config anymore

I'm not familiar enough with this code and the refspec machinery to
know if this argument injection is crazy, and how to tell it to *also*
do whatever it's doing by default]

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 15 +++++++++++++++
 Documentation/fetch-options.txt | 15 ++++++++++++++-
 builtin/fetch.c                 | 37 ++++++++++++++++++++++++++++++++++---
 remote.c                        |  2 ++
 remote.h                        |  1 +
 t/t5510-fetch.sh                |  1 +
 6 files changed, 67 insertions(+), 4 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0f27af5760..476175ef8d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1401,6 +1401,14 @@ fetch.prune::
 	option was given on the command line.  See also `remote.<name>.prune`
 	and the PRUNING section of linkgit:git-fetch[1].
 
+fetch.pruneTags::
+	If true, fetch will automatically behave as if the
+	`refs/tags/*:refs/tags/*` refspec was provided when pruning,
+	if not set already. This allows for setting both this option
+	and `fetch.prune` to maintain a 1=1 mapping to upstrem
+	refs. See also `remote.<name>.pruneTags` and the PRUNING
+	section of linkgit:git-fetch[1].
+
 fetch.output::
 	Control how ref update status is printed. Valid values are
 	`full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,13 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
+
+remote.<name>.pruneTags::
+	When set to true, fetching from this remote by default will also
+	remove any local tags that no longer exist on the remote (as
+	if the `--prune` option was given on the command line in
+	conjunction with the `refs/tags/*:refs/tags/*` refspec).
+	Overrides `fetch.prune` settings, if any.
 +
 See also `remote.<name>.prune` and the PRUNING section of
 linkgit:git-fetch[1].
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 9f5c85ad96..dc13bed741 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -73,7 +73,20 @@ ifndef::git-pull[]
 	are fetched due to an explicit refspec (either on the command
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
-	subject to pruning.
+	subject to pruning. Supplying `--prune-tags` is a shorthand for
+	providing the tag refspec.
++
+See the PRUNING section below for more details.
+
+-P::
+--prune-tags::
+	Before fetching, remove any local tags that no longer exist on
+	the remote if `--prune` is enabled. This option should be used
+	more carefully, unlike `--prune` it will remove any local
+	references (local tags) that have been created. This option is
+	merely a shorthand for providing the explicit tag refspec
+	along with `--prune`, see the discussion about that in its
+	documentation.
 +
 See the PRUNING section below for more details.
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index b34665db9e..57669a000c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */
 static int prune = -1; /* unspecified */
 #define PRUNE_BY_DEFAULT 0 /* do we prune by default? */
 
+static int fetch_prune_tags_config = -1; /* unspecified */
+static int prune_tags = -1; /* unspecified */
+#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
 static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;
 static int progress = -1;
 static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(k, "fetch.prunetags")) {
+		fetch_prune_tags_config = git_config_bool(k, v);
+		return 0;
+	}
+
 	if (!strcmp(k, "submodule.recurse")) {
 		int r = git_config_bool(k, v) ?
 			RECURSE_SUBMODULES_ON : RECURSE_SUBMODULES_OFF;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = {
 		    N_("number of submodules fetched in parallel")),
 	OPT_BOOL('p', "prune", &prune,
 		 N_("prune remote-tracking branches no longer on remote")),
+	OPT_BOOL('P', "prune-tags", &prune_tags,
+		 N_("prune local tags not found on remote")),
 	{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
 		    N_("control recursive fetching of submodules"),
 		    PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)
 		argv_array_push(argv, "--dry-run");
 	if (prune != -1)
 		argv_array_push(argv, prune ? "--prune" : "--no-prune");
+	if (prune_tags != -1)
+		argv_array_push(argv, prune_tags ? "--prune-tags" : "--no-prune-tags");
 	if (update_head_ok)
 		argv_array_push(argv, "--update-head-ok");
 	if (force)
@@ -1270,6 +1283,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 	static const char **refs = NULL;
 	struct refspec *refspec;
 	int ref_nr = 0;
+	int j = 0;
 	int exit_code;
 
 	if (!remote)
@@ -1288,10 +1302,27 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			prune = PRUNE_BY_DEFAULT;
 	}
 
+	if (prune_tags < 0) {
+		/* no command line request */
+		if (0 <= gtransport->remote->prune_tags)
+			prune_tags = gtransport->remote->prune_tags;
+		else if (0 <= fetch_prune_tags_config)
+			prune_tags = fetch_prune_tags_config;
+		else
+			prune_tags = PRUNE_TAGS_BY_DEFAULT;
+	}
+
+	if (argc > 0 || prune_tags) {
+		size_t nr_alloc = st_add3(argc, prune_tags, 1);
+		refs = xcalloc(nr_alloc, sizeof(const char *));
+		if (prune_tags) {
+			refs[j++] = xstrfmt("refs/tags/*:refs/tags/*");
+			ref_nr++;
+		}
+	}
+
 	if (argc > 0) {
-		int j = 0;
 		int i;
-		refs = xcalloc(st_add(argc, 1), sizeof(const char *));
 		for (i = 0; i < argc; i++) {
 			if (!strcmp(argv[i], "tag")) {
 				i++;
@@ -1301,8 +1332,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 						    argv[i], argv[i]);
 			} else
 				refs[j++] = argv[i];
+			ref_nr++;
 		}
-		ref_nr = j;
 	}
 
 	sigchain_push_common(unlock_pack_on_signal);
diff --git a/remote.c b/remote.c
index 4e93753e19..9eb6d24009 100644
--- a/remote.c
+++ b/remote.c
@@ -391,6 +391,8 @@ static int handle_config(const char *key, const char *value, void *cb)
 		remote->skip_default_update = git_config_bool(key, value);
 	else if (!strcmp(subkey, "prune"))
 		remote->prune = git_config_bool(key, value);
+	else if (!strcmp(subkey, "prunetags"))
+		remote->prune_tags = git_config_bool(key, value);
 	else if (!strcmp(subkey, "url")) {
 		const char *v;
 		if (git_config_string(&v, key, value))
diff --git a/remote.h b/remote.h
index 1f6611be21..04d7994cbf 100644
--- a/remote.h
+++ b/remote.h
@@ -47,6 +47,7 @@ struct remote {
 	int skip_default_update;
 	int mirror;
 	int prune;
+	int prune_tags;
 
 	const char *receivepack;
 	const char *uploadpack;
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 18280df4fc..1a822861e0 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -617,6 +617,7 @@ test_configured_prune () {
 test_configured_prune unset unset unset unset kept   kept   ""
 test_configured_prune unset unset unset unset kept   kept   "--no-prune"
 test_configured_prune unset unset unset unset pruned kept   "--prune"
+test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags" # WIP: broken, see commit msg
 test_configured_prune unset unset unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
 test_configured_prune unset unset unset unset pruned pruned \
-- 
2.15.1.424.g9478a66081

Re: [BUG] git remote prune removes local tags, depending on fetch config

[Andreas Schwab]

On Jan 15 2018, Michael Giuffrida <michaelpg@chromium.org> wrote:

> It doesn't seem like a useful feature -- you wouldn't expect `git
> fetch --prune` to remove your local branches that you were developing
> on, right?

Don't mix local and remote refs.  There is a reason why remote tracking
branches are put in a separate name space.  If you fetch the remote tags
into a separate name space (eg. refs/remote/tags/*:refs/tags/*) then
there is no conflict.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [BUG] git remote prune removes local tags, depending on fetch config

[Kevin Daudt]

On Tue, Jan 16, 2018 at 12:48:32PM +0100, Andreas Schwab wrote:
> On Jan 15 2018, Michael Giuffrida <michaelpg@chromium.org> wrote:
> 
> > It doesn't seem like a useful feature -- you wouldn't expect `git
> > fetch --prune` to remove your local branches that you were developing
> > on, right?
> 
> Don't mix local and remote refs.  There is a reason why remote tracking
> branches are put in a separate name space.  If you fetch the remote tags
> into a separate name space (eg. refs/remote/tags/*:refs/tags/*) then
> there is no conflict.
> 
> Andreas.

But then they are no longer considered tags, but remote tracking
branches. Tools like git tag and git describe won't consider them, and
git branch -r would show them as remote tracking branches.

> 
> -- 
> Andreas Schwab, schwab@linux-m68k.org
> GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
> "And now for something completely different."