Feature Request: gitignore recursion

Feature Request: gitignore recursion

[AJ]

I'm hoping to get the following feature implemented into git.

Add the ability to recursively include using:
!/my_dir/**/*

Currently, in order to include a directory with multiple sub-directories
within a excluded directory, you must do the following:
!/my_dir/
!/my_dir/*
!/my_dir/*/*
!/my_dir/*/*/*
!/my_dir/*/*/*/*
!/my_dir/*/*/*/*/*

Here is a use case:
When developing a WordPress site you typically only place the wp-content/themes
directory under version control. Once you start adding custom plugins, README
files, and a task manager like Grunt JS, you then have to starting getting
creative with your .gitignore if your .git directory is in the web root
directory.

Here is an example:
https://gist.github.com/AJ-Acevedo/6859779

Thanks for taking the time to consider this feature
AJ

Re: Feature Request: gitignore recursion

[Duy Nguyen]

On Mon, Oct 7, 2013 at 5:23 AM, AJ <alljeep@gmail.com> wrote:
> I'm hoping to get the following feature implemented into git.
>
> Add the ability to recursively include using:
> !/my_dir/**/*

You can do that since v1.8.2. Actually the pattern should be

!/my_dir/**

Checkout gitignore man page for more information.

> Currently, in order to include a directory with multiple sub-directories
> within a excluded directory, you must do the following:
> !/my_dir/
> !/my_dir/*
> !/my_dir/*/*
> !/my_dir/*/*/*
> !/my_dir/*/*/*/*
> !/my_dir/*/*/*/*/*
>
> Here is a use case:
> When developing a WordPress site you typically only place the wp-content/themes
> directory under version control. Once you start adding custom plugins, README
> files, and a task manager like Grunt JS, you then have to starting getting
> creative with your .gitignore if your .git directory is in the web root
> directory.
>
> Here is an example:
> https://gist.github.com/AJ-Acevedo/6859779
>
> Thanks for taking the time to consider this feature
> AJ
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Duy

Re: Feature Request: gitignore recursion

[Jeff King]

On Mon, Oct 07, 2013 at 05:26:17PM +0700, Nguyen Thai Ngoc Duy wrote:

> On Mon, Oct 7, 2013 at 5:23 AM, AJ <alljeep@gmail.com> wrote:
> > I'm hoping to get the following feature implemented into git.
> >
> > Add the ability to recursively include using:
> > !/my_dir/**/*
> 
> You can do that since v1.8.2. Actually the pattern should be
> 
> !/my_dir/**
> 
> Checkout gitignore man page for more information.

I think that is a reasonable approach to doing what the OP wants.

However, one thing I expected to work but didn't is:

  echo '*' >.gitignore
  echo '!*' >my_dir/.gitignore

That _does_ work for attributes, like:

  echo '* foo=one' >.gitattributes
  echo '* foo=two' >my_dir/.gitattributes

where the more-specific file takes precedence. It works because we keep
an attribute stack, and look from most-specific directory to least.

The directory traversal code, however, tries to avoid entering
directories that are ignored. So if you have told it to ignore "my_dir",
we will not even look at "my_dir/.gitignore".

This is arguably a bug, as the documentation says that more specific
paths should take precedence over toplevel ones. But the optimization of
pruning ignored paths is very important for performance; otherwise, we
would end up enumerating entire ignored subdirectories on the off chance
that they have a negative .gitignore entry buried deep inside them.

So I don't think it is worth fixing, but it might be worth mentioning in
the documentation. Or is it mentioned somewhere that I missed?

-Peff

Re: Feature Request: gitignore recursion

[Duy Nguyen]

On Mon, Oct 07, 2013 at 07:24:11AM -0400, Jeff King wrote:
> However, one thing I expected to work but didn't is:
> 
>   echo '*' >.gitignore
>   echo '!*' >my_dir/.gitignore
> 
> That _does_ work for attributes, like:
> 
>   echo '* foo=one' >.gitattributes
>   echo '* foo=two' >my_dir/.gitattributes
> 
> where the more-specific file takes precedence. It works because we keep
> an attribute stack, and look from most-specific directory to least.
>
> ...

Yeah. I'm still bugged about that because we could at least make your
case work (deep/down/.gitignore will never ever, of course) but I
probably won't do anything about it for the next few months. So how
about this.

The first use of footnote:[] is already in git-notes.txt. The second
instance should not cause any issues with supported asciidoc versions.

-- 8< --
Subject: [PATCH] gitignore.txt: note about a behavior not so obvious much to users


Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/gitignore.txt | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
index 54e334e..cd59f6e 100644
--- a/Documentation/gitignore.txt
+++ b/Documentation/gitignore.txt
@@ -21,7 +21,10 @@ Each line in a `gitignore` file specifies a pattern.
 When deciding whether to ignore a path, Git normally checks
 `gitignore` patterns from multiple sources, with the following
 order of precedence, from highest to lowest (within one level of
-precedence, the last matching pattern decides the outcome):
+precedence, the last matching pattern decides the outcome
+footnote:[Due to optimizations Git will not look further into ignored
+directories. So if you have a rule to ignore directory `foo`,
+`.gitignore` in `foo` and its subdirectories are all ignored]):
 
  * Patterns read from the command line for those commands that support
    them.
-- 
1.8.2.83.gc99314b
-- 8< --

Re: Feature Request: gitignore recursion

[Karsten Blees]

> On Mon, Oct 7, 2013 at 5:23 AM, AJ <allj...@gmail.com> wrote:
>> I'm hoping to get the following feature implemented into git.
>>
>> Add the ability to recursively include using:
>> !/my_dir/**/*
> 
> 
> You can do that since v1.8.2. Actually the pattern should be
> 
> !/my_dir/**
> 

Another solution to include just one specific directory:

  /*
  !/wp-content
  /wp-content/*
  !/wp-content/themes

Note the "/*", instead of "*" which matches everything everywhere.


> Checkout gitignore man page for more information.
> 

The compiled version reads like this:

"A trailing "/" matches everything inside. For example, "abc/" matches all files inside directory "abc"..."


--- 8< ---
Subject: [PATCH] gitignore.txt: fix documentation of "**" patterns

"**" means bold in ASCIIDOC, so we need to escape it.

Signed-off-by: Karsten Blees <blees@dcon.de>
---
 Documentation/gitignore.txt | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
index 54e334e..f971960 100644
--- a/Documentation/gitignore.txt
+++ b/Documentation/gitignore.txt
@@ -113,12 +113,12 @@ full pathname may have special meaning:
 
  - A leading "`**`" followed by a slash means match in all
    directories. For example, "`**/foo`" matches file or directory
-   "`foo`" anywhere, the same as pattern "`foo`". "**/foo/bar"
+   "`foo`" anywhere, the same as pattern "`foo`". "`**/foo/bar`"
    matches file or directory "`bar`" anywhere that is directly
    under directory "`foo`".
 
- - A trailing "/**" matches everything inside. For example,
-   "abc/**" matches all files inside directory "abc", relative
+ - A trailing "`/**`" matches everything inside. For example,
+   "`abc/**`" matches all files inside directory "`abc`", relative
    to the location of the `.gitignore` file, with infinite depth.
 
  - A slash followed by two consecutive asterisks then a slash
-- 
1.8.4.msysgit.1

Re: Feature Request: gitignore recursion

[Karsten Blees]

> On Mon, Oct 07, 2013 at 07:24:11AM -0400, Jeff King wrote:
>> However, one thing I expected to work but didn't is:
>> 
>>   echo '*' >.gitignore
>>   echo '!*' >my_dir/.gitignore
>> 
>> That _does_ work for attributes, like:
>> 
>>   echo '* foo=one' >.gitattributes
>>   echo '* foo=two' >my_dir/.gitattributes
>> 
>> where the more-specific file takes precedence. It works because we keep
>> an attribute stack, and look from most-specific directory to least.

Note that this doesn't work either:

  echo '*' >.gitignore
  echo '!my_dir/*' >>.gitignore

The problem isn't that git doesn't read 'my_dir/.gitignore'. Git simply doesn't recurse into excluded directories, so patterns on excluded content have no effect.

IMO this behavior is reasonable, as there is no way to check whether a negative pattern will match within an excluded untracked directory without scanning the directory.

What about this instead?

--- 8< ---
Subject: [PATCH] gitignore.txt: clarify recursive nature of excluded directories

Additionally, precedence of negated patterns is exactly as outlined in
the description, we don't need to repeat this.

Signed-off-by: Karsten Blees <blees@dcon.de>
---
 Documentation/gitignore.txt | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Documentation/gitignore.txt b/Documentation/gitignore.txt
index f971960..a64a65d 100644
--- a/Documentation/gitignore.txt
+++ b/Documentation/gitignore.txt
@@ -79,8 +79,9 @@ PATTERN FORMAT
 
  - An optional prefix "`!`" which negates the pattern; any
    matching file excluded by a previous pattern will become
-   included again.  If a negated pattern matches, this will
-   override lower precedence patterns sources.
+   included again. It is not possible to re-include a file if a parent
+   directory of that file is excluded (i.e. excluding a directory
+   will recursively and irrevocably exclude the entire content).
    Put a backslash ("`\`") in front of the first "`!`" for patterns
    that begin with a literal "`!`", for example, "`\!important!.txt`".
 
-- 
1.8.4.msysgit.1

Re: Feature Request: gitignore recursion

[Jeff King]

On Thu, Oct 10, 2013 at 09:59:38AM +0200, Karsten Blees wrote:

> > On Mon, Oct 07, 2013 at 07:24:11AM -0400, Jeff King wrote:
> >> However, one thing I expected to work but didn't is:
> >> 
> >>   echo '*' >.gitignore
> >>   echo '!*' >my_dir/.gitignore
> >> 
> >> That _does_ work for attributes, like:
> >> 
> >>   echo '* foo=one' >.gitattributes
> >>   echo '* foo=two' >my_dir/.gitattributes
> >> 
> >> where the more-specific file takes precedence. It works because we keep
> >> an attribute stack, and look from most-specific directory to least.
> 
> Note that this doesn't work either:
> 
>   echo '*' >.gitignore
>   echo '!my_dir/*' >>.gitignore
> 
> The problem isn't that git doesn't read 'my_dir/.gitignore'. Git
> simply doesn't recurse into excluded directories, so patterns on
> excluded content have no effect.

Good point. To make it work you need to un-ignore the subdir, like:

  *
  !my_dir/

at which point "!my_dir/*" will work. But then so will a child
.gitignore in subdir. In fact, I think the only reason that "!my_dir/**"
is needed in the first place is that the "*" is too over-reaching; it
ignores both the top-level directories _and_ all of the individual files
in included directories.

So I think the best solution for the original problem, which is to
ignore everything except for a particular directory, is:

  # ignore everything at the top-level...
  /*
  # ...but specifically include one directory
  !/my_dir/

and then we do not even need to use "**" matching at all (which was
really about overriding the over-reaching "*").

> IMO this behavior is reasonable, as there is no way to check whether a
> negative pattern will match within an excluded untracked directory
> without scanning the directory.

Right. I was focused on not reading the .gitignore, but not descending
into the ignore subdir affects patterns we already know about, too.

> --- 8< ---
> Subject: [PATCH] gitignore.txt: clarify recursive nature of excluded directories
> 
> Additionally, precedence of negated patterns is exactly as outlined in
> the description, we don't need to repeat this.
> 
> Signed-off-by: Karsten Blees <blees@dcon.de>

Yeah, I think your update improves things.

I wonder if it is worth adding the "exclude everything except one
directory" case above to the EXAMPLES section, too.

-Peff