Bug: Failure if stdin is closed.

Bug: Failure if stdin is closed.

[Dale R. Worley]

(The original problem and the discussion that ensued is on the
git-users mailing list:
https://groups.google.com/forum/#!topic/git-users/lNQ7Cn35EqA)

"git commit" (and probably other operations) fail if standard input
(fd 0) is closed when git starts.  A simple test case follows.  (The
execution is version 1.7.7.6, but the code listed below is from a very
recent commit.)


$ git --version
git version 1.7.7.6
$ git status
# On branch master
nothing to commit (working directory clean)
$ echo This is a test >ffff
$ git status
# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	ffff
nothing added to commit but untracked files present (use "git add" to track)
$ git add ffff
$ # The notation "0<&-" means "close standard input (fd 0) in the process that
$ # executes this command.  It may be specific to the bash shell.
$ git commit -m xxxx 0<&-
error: unable to create temporary sha1 filename : No such file or directory

error: Error building trees
$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#	new file:   ffff
#
$ git commit -m xxxx
[master 54c146c] xxxx
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 ffff
$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
nothing to commit (working directory clean)
$ 


The fundamental error is that git_mkstemps_mode() in wrapper.c assumes
that if open() is successful, its return value must be >0.  That is
not true, because if fd 0 is closed, then open() can successfully
return 0.  I have not tested it, but it looks like the fix is:

 int git_mkstemps_mode(char *pattern, int suffix_len, int mode)
 {
	 static const char letters[] =
		 "abcdefghijklmnopqrstuvwxyz"
		 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
		 "0123456789";
	 static const int num_letters = 62;
	 uint64_t value;
	 struct timeval tv;
	 char *template;
	 size_t len;
	 int fd, count;

	 len = strlen(pattern);

	 if (len < 6 + suffix_len) {
		 errno = EINVAL;
		 return -1;
	 }

	 if (strncmp(&pattern[len - 6 - suffix_len], "XXXXXX", 6)) {
		 errno = EINVAL;
		 return -1;
	 }

	 /*
	  * Replace pattern's XXXXXX characters with randomness.
	  * Try TMP_MAX different filenames.
	  */
	 gettimeofday(&tv, NULL);
	 value = ((size_t)(tv.tv_usec << 16)) ^ tv.tv_sec ^ getpid();
	 template = &pattern[len - 6 - suffix_len];
	 for (count = 0; count < TMP_MAX; ++count) {
		 uint64_t v = value;
		 /* Fill in the random bits. */
		 template[0] = letters[v % num_letters]; v /= num_letters;
		 template[1] = letters[v % num_letters]; v /= num_letters;
		 template[2] = letters[v % num_letters]; v /= num_letters;
		 template[3] = letters[v % num_letters]; v /= num_letters;
		 template[4] = letters[v % num_letters]; v /= num_letters;
		 template[5] = letters[v % num_letters]; v /= num_letters;

		 fd = open(pattern, O_CREAT | O_EXCL | O_RDWR, mode);
-		 if (fd > 0)
+		 if (fd >= 0)
			 return fd;
		 /*
		  * Fatal error (EPERM, ENOSPC etc).
		  * It doesn't make sense to loop.
		  */
		 if (errno != EEXIST)
			 break;
		 /*
		  * This is a random value.  It is only necessary that
		  * the next TMP_MAX values generated by adding 7777 to
		  * VALUE are different with (module 2^32).
		  */
		 value += 7777;
	 }
	 /* We return the null string if we can't find a unique file name.  */
	 pattern[0] = '\0';
	 return -1;
 }


However, when looking at the code, I noticed that few of the functions
have comments describing what they do, and none describe their input
and output values.  In particular, there are no comments specifying
what the error return values are.  This is appalling for a supposedly
professional-quality project!

Dale

Re: Bug: Failure if stdin is closed.

[Thomas Rast]

worley@alum.mit.edu (Dale R. Worley) writes:

> (The original problem and the discussion that ensued is on the
> git-users mailing list:
> https://groups.google.com/forum/#!topic/git-users/lNQ7Cn35EqA)
>
> "git commit" (and probably other operations) fail if standard input
> (fd 0) is closed when git starts.  A simple test case follows.  (The
> execution is version 1.7.7.6, but the code listed below is from a very
> recent commit.)

While you found a real bug in the use of open() below, I'm not convinced
that this scenario is something that should be guarded against.  For
example [1] says

  At program start-up, three streams shall be predefined and need not be
  opened explicitly: standard input (for reading conventional input),
  standard output (for writing conventional output), and standard error
  (for writing diagnostic output).

I haven't found a similar clause relating to the file descriptors 0--2,
but my reading of the above paragraph is that running a program without
an open stdin, stdout and stderr is, in fact, not POSIX.

Closing 2 usually has even funkier consequences if the program proceeds
to open some other file, it ends up as fd 2, and it then dies with an
error.  In that sense it might be saner to simply die whenever open()
gives an FD in the 0..2 range (and we weren't explicitly trying to
reopen one of them).

May I ask why you need this, and to what extent this problem cannot be
solved by instead redirecting from/to /dev/null?

> The fundamental error is that git_mkstemps_mode() in wrapper.c assumes
> that if open() is successful, its return value must be >0.  That is
> not true, because if fd 0 is closed, then open() can successfully
> return 0.  I have not tested it, but it looks like the fix is:
[...]
> 		 template[5] = letters[v % num_letters]; v /= num_letters;
>
> 		 fd = open(pattern, O_CREAT | O_EXCL | O_RDWR, mode);
> -		 if (fd > 0)
> +		 if (fd >= 0)
> 			 return fd;
> 		 /*
> 		  * Fatal error (EPERM, ENOSPC etc).

This is an actual bug, as you say; open() is defined to return -1 on
error, not 0.  You can turn this into a patch, and send it as per
Documentation/SubmittingPatches.

However, does it fully fix the issue you describe?  What if you then run
'git checkout -F -' to read the message from stdin?

> However, when looking at the code, I noticed that few of the functions
> have comments describing what they do, and none describe their input
> and output values.  In particular, there are no comments specifying
> what the error return values are.  This is appalling for a supposedly
> professional-quality project!

You are touching on a sore point of the git code base.  Some
contributors have made a point of adding comments where appropriate, so
we're improving, but round tuits are in short supply as always.  If you
can supply such tuits, they would be appreciated.



[1]  http://pubs.opengroup.org/onlinepubs/009695399/functions/stdin.html

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Bug: Failure if stdin is closed.

[Dale Worley]

> From: Thomas Rast <trast@inf.ethz.ch>

> May I ask why you need this, and to what extent this problem cannot be
> solved by instead redirecting from/to /dev/null?

The situation in which the problem arose is described here:

> worley@alum.mit.edu (Dale R. Worley) writes:
> 
> > (The original problem and the discussion that ensued is on the
> > git-users mailing list:
> > https://groups.google.com/forum/#!topic/git-users/lNQ7Cn35EqA)

It involves invoking git from Apache.  I haven't read up on the
details because I didn't need to in order to debug the problem.

> Closing 2 usually has even funkier consequences if the program proceeds
> to open some other file, it ends up as fd 2, and it then dies with an
> error.  In that sense it might be saner to simply die whenever open()
> gives an FD in the 0..2 range (and we weren't explicitly trying to
> reopen one of them).

True...  But fd 2 may be needed under many unpredictable
circumstances.  In regard to fd 0, we can predict that standard input
(per se) will not be needed, so it's anti-robust to require that it be
open for the code to function at all.

> However, does it fully fix the issue you describe?  What if you then run
> 'git checkout -F -' to read the message from stdin?

Obviously, if the git command explicitly requires use of
standard-input, then standard-input needs to be open.

> > However, when looking at the code, I noticed that few of the functions
> > have comments describing what they do, and none describe their input
> > and output values.  In particular, there are no comments specifying
> > what the error return values are.  This is appalling for a supposedly
> > professional-quality project!
> 
> You are touching on a sore point of the git code base.  Some
> contributors have made a point of adding comments where appropriate, so
> we're improving, but round tuits are in short supply as always.  If you
> can supply such tuits, they would be appreciated.

I will try to put my money where my mouth is.

Dale