glob vs '*/' vs GLOB_ONLYDIR vs xfs

glob vs '*/' vs GLOB_ONLYDIR vs xfs

[DJ Delorie via Libc-alpha]

in posix/glob.c we have this comment:

  /* POSIX requires all slashes to be matched.  This means that with
     a trailing slash we must match only directories.  */
  if (pattern[0] && pattern[strlen (pattern) - 1] == '/')
    flags |= GLOB_ONLYDIR;

In "man glob" we see:

       GLOB_ONLYDIR
              This is a _hint_ to glob() that the caller is interested
              only in directories that match the pattern.

So, for starters, we're relying on "a hint" to implement a "must".

If you're running glob on an XFS filesystem, readdir() doesn't
reliably fill in d_type, and the "hint" fails:

#include <stdlib.h>
#include <glob.h>

main(int argc, char **argv) {
  glob_t g;
  int i;

  glob (argv[1], 0, NULL, &g);
  for (i=0; i<g.gl_pathc; i++)
    printf("glob[%d] = `%s'\n", i, g.gl_pathv[i]);

  return 0;
}

$ ls -l
total 4
-rw-r--r-- 1 dj games 56 Aug  4 22:45 Makefile
drwxr-xr-x 2 dj games 10 Aug  4 22:44 hellod/
-rw-r--r-- 1 dj games  0 Aug  4 22:44 hellof

$ pwd
/greed/dj/gnu/make-4.3/dj

$ ~/src/globtest 'hello*/'
glob[0] = `hellod/'
glob[1] = `hellof'

$ ~/src/globtest 'hello*'
glob[0] = `hellod'
glob[1] = `hellof'


If we instead run this on a different filesystem type (tmpfs in this
case), it works as I expect:

$ ls -l
total 4
-rw-r--r-- 1 dj games 56 Aug  4 22:45 Makefile
drwxr-xr-x 2 dj games  6 Aug  4 22:44 hellod/
-rw-r--r-- 1 dj games  0 Aug  4 22:44 hellof

$ pwd
/tmp/make-4.3/dj

$ ~/src/globtest 'hello*/'
glob[0] = `hellod/'

$ ~/src/globtest 'hello*'
glob[0] = `hellod'
glob[1] = `hellof'


Is this a bug in glob(), or a misunderstanding of its documentation?

Re: glob vs '*/' vs GLOB_ONLYDIR vs xfs

[Paul Eggert]

On 8/4/21 8:24 PM, DJ Delorie via Libc-alpha wrote:
> If you're running glob on an XFS filesystem, readdir() doesn't
> reliably fill in d_type

Isn't that a bug in XFS? readdir should set d_type to DT_UNKNOWN if it 
doesn't know the type. It shouldn't set d_type to garbage.

Doesn't glob do the right thing if d_type is DT_UNKNOWN?

Re: glob vs '*/' vs GLOB_ONLYDIR vs xfs

[DJ Delorie via Libc-alpha]

Paul Eggert <eggert@cs.ucla.edu> writes:
> Isn't that a bug in XFS? readdir should set d_type to DT_UNKNOWN if it 
> doesn't know the type. It shouldn't set d_type to garbage.

XFS sets d_type to DT_UNKNOWN for most entries, not garbage.

> Doesn't glob do the right thing if d_type is DT_UNKNOWN?

No, if by "right thing" you mean "call lstat and find out the right
type".

Re: glob vs '*/' vs GLOB_ONLYDIR vs xfs

[Paul Eggert]

On 8/5/21 11:10 AM, DJ Delorie wrote:
> Paul Eggert <eggert@cs.ucla.edu> writes:
>> Isn't that a bug in XFS? readdir should set d_type to DT_UNKNOWN if it
>> doesn't know the type. It shouldn't set d_type to garbage.
> 
> XFS sets d_type to DT_UNKNOWN for most entries, not garbage.

Ah, sorry, I misunderstood your previous email.

>> Doesn't glob do the right thing if d_type is DT_UNKNOWN?
> 
> No, if by "right thing" you mean "call lstat and find out the right
> type".

Thanks for reporting the problem. I plan to address this by merging 
recent glibc glob changes into Gnulib (these were for 64-bit time_t), 
fixing the bug in Gnulib, and then we can talk about merging back.

The fix won't be trivial, unfortunately. On the bright side, I think we 
can remove some more stat/lstat calls from glob.

Re: glob vs '*/' vs GLOB_ONLYDIR vs xfs

[DJ Delorie via Libc-alpha]

Paul Eggert <eggert@cs.ucla.edu> writes:
> I plan to address this by merging recent glibc glob changes into
> Gnulib (these were for 64-bit time_t), fixing the bug in Gnulib, and
> then we can talk about merging back.
>
> The fix won't be trivial, unfortunately. On the bright side, I think we 
> can remove some more stat/lstat calls from glob.

Excellent.  Thanks!