[patch v2] glob: resolve DT_UNKNOWN via is_dir

[DJ Delorie <dj@redhat.com>]

[v2: changed malloc failure from ignore to error; added support for
alloca; tested by copying to glibc and testing there]

The DT_* values returned by getdents (readdir) are only hints and
not required.  In fact, some Linux filesystems return DT_UNKNOWN
for most entries, regardless of actual type.  This causes make
to mis-match patterns with a trailing slash (via GLOB_ONLYDIR)
(see make's functions/wildcard test case).  Thus, this patch
detects that case and uses is_dir() to make the type known enough
for proper operation.

Performance in non-DT_UNKNOWN cases is not affected.

The lack of DT_* is a well known issue on older XFS installations
(for example, RHEL 7 and 8, Fedora 28) but can be recreated by
creating an XFS filesystem with flags that mimic older behavior:

$ fallocate -l 10G /xfs.fs
$ mkfs.xfs -n ftype=0 -m crc=0 -f /xfs.fs
$ mkdir /xfs
$ mount -o loop /xfs.fs /xfs

diff --git a/lib/glob.c b/lib/glob.c
index f8d8a306f2..c28c92a42c 100644
--- a/lib/glob.c
+++ b/lib/glob.c
@@ -1381,7 +1381,33 @@ glob_in_dir (const char *pattern, const char *directory, int flags,
               if (flags & GLOB_ONLYDIR)
                 switch (readdir_result_type (d))
                   {
-                  case DT_DIR: case DT_LNK: case DT_UNKNOWN: break;
+                  case DT_DIR: case DT_LNK: break;
+		  case DT_UNKNOWN:
+		    {
+		      /* The filesystem was too lazy to give us a hint,
+			 so we have to do it the hard way.  */
+		      char *fullpath, *p;
+		      bool isdir;
+		      int need = strlen (directory) + strlen (d.name) + 2;
+		      int use_alloca = glob_use_alloca (alloca_used, need);
+		      if (use_alloca)
+			fullpath = alloca_account (need, alloca_used);
+		      else
+			{
+			  fullpath = malloc (need);
+			  if (fullpath == NULL)
+			    goto memory_error;
+			}
+		      p = stpcpy (fullpath, directory);
+		      *p++ = '/';
+		      strcpy (p, d.name);
+		      isdir = is_dir (fullpath, flags, pglob);
+		      if (!use_alloca)
+			free (fullpath);
+		      if (isdir)
+			break;
+		      continue;
+		    }
                   default: continue;
                   }