GitAttributes feature export-ignore works, but -export-ignore (with dash) should also work

GitAttributes feature export-ignore works, but -export-ignore (with dash) should also work

[M Lewis]

When running 'git archive', the .gitattributes file is used to determine 
paths that should be ignored when building the archive by using 
'export-ignore'.
But it would be useful to include certain sub-paths in the ignored path.

For example, say I have folders like these
     /all_useful                  # this entire sub directory is needed 
in the archive
     /mostly_unuseful             # there are many sub directories in 
here to ignore
/mostly_unuseful/needed_a    # but these specific sub directories are 
desired in the archive
/mostly_unuseful/needed_b

The git-attribute document says a dash '-' prefix should unset the 
attribute. It should work with 'export-ignore'

So if I want for example '/mostly_unuseful/needed_a' in my archive, I 
should be able to specify that
in my .gitattributes file like this:


     /mostly_unuseful                  export-ignore # do not archive
/mostly_unuseful/needed_a -export-ignore # do add to archive
/mostly_unuseful/needed_b    -export-ignore # do add to archive

Thank you

Re: GitAttributes feature export-ignore works, but -export-ignore (with dash) should also work

[Jeff King]

On Wed, Nov 20, 2019 at 08:44:52AM -0800, M Lewis wrote:

> When running 'git archive', the .gitattributes file is used to determine
> paths that should be ignored when building the archive by using
> 'export-ignore'.
> But it would be useful to include certain sub-paths in the ignored path.

Yeah, that seems like a reasonable goal, and I think we can make it
work, but the syntax is slightly different.

> The git-attribute document says a dash '-' prefix should unset the
> attribute. It should work with 'export-ignore'
> 
> So if I want for example '/mostly_unuseful/needed_a' in my archive, I
> should be able to specify that in my .gitattributes file like this:
> 
>     /mostly_unuseful              export-ignore # do not archive
>     /mostly_unuseful/needed_a    -export-ignore # do add to archive
>     /mostly_unuseful/needed_b    -export-ignore # do add to archive

It doesn't work because the attributes are not applied recursively[1];
they are applied to the path you specified. By putting the attribute on
"mostly_unuseful", that doesn't say anything about "needed_a" (from the
perspective of the attributes system). So removing it from that path
likewise does nothing; the attribute is still set on the directory.

Instead, if you write it like this:

  /mostly_unuseful/**        export-ignore
  /mostly_unuseful/needed_a -export-ignore
  /mostly_unuseful/needed_b -export-ignore

Then the attributes are applied directly to the file paths in the first
line, and the subsequent lines counteract them. This does mean that the
subdirectory itself isn't marked as export-ignore, and will always be
included (as an empty directory if need be, though of course in your
example we do actually end up with some files in it).

Now obviously there is recursion happening inside git-archive, as it
walks the tree. And the current behavior is that it sees the ignored
subtree and doesn't walk into it at all. So it _could_ make your
original example work by actually searching within the tree for any
paths marked -export-ignore, and including the directory if and only if
it has unignored entries.

I can't offhand think of anything that would break if we started doing
that, but I haven't thought too hard. And given that there's already a
solution using `**`, it might not be worth it.

-Peff

[1] If you're curious, there's some philosophical discussion about this exact issue in
    https://public-inbox.org/git/7v1uasg8e0.fsf@alter.siamese.dyndns.org/

Re: GitAttributes feature export-ignore works, but -export-ignore (with dash) should also work

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Now obviously there is recursion happening inside git-archive, as it
> walks the tree. And the current behavior is that it sees the ignored
> subtree and doesn't walk into it at all. So it _could_ make your
> original example work by actually searching within the tree for any
> paths marked -export-ignore, and including the directory if and only if
> it has unignored entries.
>
> I can't offhand think of anything that would break if we started doing
> that,...

If there is some effect we want to attach to a directory itself
without affecting the paths inside it, the current and original
design lets you express it naturally.  If we make a pattern that
match a directory to recurse, it is still possible to express it,
but it is a bit cumbersome, e.g.

	/directory	want-to-affect-this
	/directory/**	-want-to-affect-this

I would think.

Git generally does *not* track directories, so in a sense it is
arguable if it even makes sense to think about attaching an
attribute to a directory itself (as opposed to a non-directory
inside the directory) in the first place, though.

Re: GitAttributes feature export-ignore works, but -export-ignore (with dash) should also work

[M Lewis]

Jeff,

Thank you for taking this on. I tried your suggestion copied below. The 
'needed_a' (and _b) directories appear in the archive but unfortunately 
they are empty. We need the contents of the directories in the archive.

On 11/20/2019 11:58 PM, Jeff King wrote:
> Instead, if you write it like this:
>
>    /mostly_unuseful/**        export-ignore
>    /mostly_unuseful/needed_a -export-ignore
>    /mostly_unuseful/needed_b -export-ignore
>
> ... And given that there's already a
> solution using `**`, it might not be worth it.
>

Re: GitAttributes feature export-ignore works, but -export-ignore (with dash) should also work

[Jeff King]

On Thu, Nov 21, 2019 at 07:45:44AM -0800, M Lewis wrote:

> Thank you for taking this on. I tried your suggestion copied below. The
> 'needed_a' (and _b) directories appear in the archive but unfortunately they
> are empty. We need the contents of the directories in the archive.

Ah, I assumed they were files. You should be able to handle that by
setting and clearing the attributes using the "**" syntax. Something
like:

  /subdir/** export-ignore
  /subdir/keep/ -export-ignore
  /subdir/keep/** -export-ignore

would work to re-enable "one" and all of its contents. Or if everything
you want to toggle is at the top-level of the subdirectory, you can use
"*" to avoid assigning export-ignore to all of the recursive paths in
the first place, like:

  /subdir/* export-ignore
  /subdir/keep/ -export-ignore

-Peff

Re: GitAttributes feature export-ignore works, but -export-ignore (with dash) should also work

[Jeff King]

On Thu, Nov 21, 2019 at 10:29:21PM +0900, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Now obviously there is recursion happening inside git-archive, as it
> > walks the tree. And the current behavior is that it sees the ignored
> > subtree and doesn't walk into it at all. So it _could_ make your
> > original example work by actually searching within the tree for any
> > paths marked -export-ignore, and including the directory if and only if
> > it has unignored entries.
> >
> > I can't offhand think of anything that would break if we started doing
> > that,...
> 
> If there is some effect we want to attach to a directory itself
> without affecting the paths inside it, the current and original
> design lets you express it naturally.  If we make a pattern that
> match a directory to recurse, it is still possible to express it,
> but it is a bit cumbersome, e.g.
> 
> 	/directory	want-to-affect-this
> 	/directory/**	-want-to-affect-this
> 
> I would think.

I didn't mean teaching the attribute code anything about recursion. You
convinced me in the thread I linked that the current rules are just fine
for expressing what we want (and especially with "**" now it's easy).
It's git-archive which does the recursion here, so it could have more
convenient semantics for an attribute attached to a directory.

(Though again, I'm happy enough with the solutions I've suggested in
this thread).

> Git generally does *not* track directories, so in a sense it is
> arguable if it even makes sense to think about attaching an
> attribute to a directory itself (as opposed to a non-directory
> inside the directory) in the first place, though.

I mostly agree, though in this case if we are exporting to a tarball,
which _does_ care about directories. So the difference between "don't
export foo/" and "don't export foo/*" can be observed in the output. I
doubt anybody really cares that much in practice, though (i.e., does
anybody really want to be able to output empty directories in their
tarballs?).

-Peff