Re: [PATCH 1/1] mingw: intercept isatty() to handle /dev/null as Git expects it

[Johannes Sixt <j6t@kdbg.org>]

Am 11.12.2016 um 12:16 schrieb Johannes Schindelin:
> When Git's source code calls isatty(), it really asks whether the
> respective file descriptor is connected to an interactive terminal.
>
> Windows' _isatty() function, however, determines whether the file
> descriptor is associated with a character device. And NUL, Windows'
> equivalent of /dev/null, is a character device.
>
> Which means that for years, Git mistakenly detected an associated
> interactive terminal when being run through the test suite, which
> almost always redirects stdin, stdout and stderr to /dev/null.
>
> This bug only became obvious, and painfully so, when the new
> bisect--helper entered the `pu` branch and made the automatic build & test
> time out because t6030 was waiting for an answer.
>
> For details, see
>
> 	https://msdn.microsoft.com/en-us/library/f4s0ddew.aspx
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  compat/mingw.h   |  3 +++
>  compat/winansi.c | 33 +++++++++++++++++++++++++++++++++
>  2 files changed, 36 insertions(+)
>
> diff --git a/compat/mingw.h b/compat/mingw.h
> index 034fff9479..3350169555 100644
> --- a/compat/mingw.h
> +++ b/compat/mingw.h
> @@ -384,6 +384,9 @@ int mingw_raise(int sig);
>   * ANSI emulation wrappers
>   */
>
> +int winansi_isatty(int fd);
> +#define isatty winansi_isatty
> +
>  void winansi_init(void);
>  HANDLE winansi_get_osfhandle(int fd);
>
> diff --git a/compat/winansi.c b/compat/winansi.c
> index db4a5b0a37..cb725fb02f 100644
> --- a/compat/winansi.c
> +++ b/compat/winansi.c
> @@ -7,6 +7,9 @@
>  #include <wingdi.h>
>  #include <winreg.h>
>
> +/* In this file, we actually want to use Windows' own isatty(). */
> +#undef isatty
> +
>  /*
>   ANSI codes used by git: m, K
>
> @@ -570,6 +573,36 @@ static void detect_msys_tty(int fd)
>
>  #endif
>
> +int winansi_isatty(int fd)
> +{
> +	int res = isatty(fd);
> +
> +	if (res) {
> +		/*
> +		 * Make sure that /dev/null is not fooling Git into believing
> +		 * that we are connected to a terminal, as "_isatty() returns a
> +		 * nonzero value if the descriptor is associated with a
> +		 * character device."; for more information, see
> +		 *
> +		 * https://msdn.microsoft.com/en-us/library/f4s0ddew.aspx
> +		 */
> +		HANDLE handle = (HANDLE)_get_osfhandle(fd);
> +		if (fd == STDIN_FILENO) {
> +			DWORD dummy;
> +
> +			if (!GetConsoleMode(handle, &dummy))
> +				res = 0;
> +		} else if (fd == STDOUT_FILENO || fd == STDERR_FILENO) {
> +			CONSOLE_SCREEN_BUFFER_INFO dummy;
> +
> +			if (!GetConsoleScreenBufferInfo(handle, &dummy))
> +				res = 0;

I am sorry to have to report that this check does not work as expected. 
I am operating Git from CMD, not mintty, BTW.

It fails with GetLastError() == 6 (invalid handle value). The reason for 
this is, I think, that in reality we are not writing to the console 
directly, but to a pipe, which is drained by console_thread(), which 
writes to the console.

I have little clue what this winansi.c file is doing. Do you have any 
pointers where I should start digging?

Wait...

Should we not use winansi_get_osfhandle() instead of _get_osfhandle()?

> +		}
> +	}
> +
> +	return res;
> +}
> +
>  void winansi_init(void)
>  {
>  	int con1, con2;
>