Re: [PATCH 04/15] Set _O_BINARY as default fmode for both MinGW and MSVC

[Alexey Borzenkov <snaury@gmail.com>]

On Wed, Sep 16, 2009 at 8:14 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>> MinGW set the _CRT_fmode to set both the default fmode and
>> _O_BINARY on stdin/stdout/stderr. Rather use the main()
>> define in mingw.h to set this for both MinGW and MSVC.
>>
>> This will ensure that a MinGW and MSVC build will handle
>> input and output identically.
> This one breaks t5302-pack-index.sh (test 15, "[index v1] 2) create a
> stealth corruption in a delta base reference") in my MinGW build. I have
> yet to find out what exactly goes wrong and how it could be fixed.

Looks like calling _setmode in main is too late for something. :-/

I tried defining int _fmode in mingw.c (which sets msvcrt's _fmode
early, without calling _setmode), and it didn't work either (i.e. the
bug was still happening). So the culprit must be in _setmode. I also
tried logging what index_obj_offset and index_obj_nr are showing, and
here are results:

/* with _CRT_fmode = _O_BINARY */
index_obj_offset: 844032
index_obj_nr: 205

/* without _CRT_fmode = _O_BINARY */
index_obj_offset: 844235
index_obj_nr: 298

I then tried to see what git show-index actually returns there, and
here are several top lines of the results:

/* with _CRT_fmode = _O_BINARY */
850186 0106e17481932f5c223fafadc1d26abc6adf40d6
203652 01179d82b18c252824e20e190543e0e84950d820
15576 0246ff570c396c37ff08c0f5f9d88c84fc21a297
851536 035e7e54cee9eb197af435b6b6dcced489c233f6
376062 03a6f56dbbb556425ef8c43c31519b32eb5629a4
39452 0471b3b8ab8da19f3968dc238e9555f4e48ce858
8230 0483297992ed4cd5ab3b5790912b1f03598ea660
72292 0508b858e01f0367552fb14722388ba48dbd36f7

/* without _CRT_fmode = _O_BINARY */
844235 01a8d3f6d063d48baf40ee656804aa58486a34b2
851754 035e7e54cee9eb197af435b6b6dcced489c233f6
851844 04ff7bd225fbb4842697af96c147a4ddecb6a657
848233 088ddcb945af72c47348b65a63992c71b6b9dbe8
849676 088e69e7ab273e35b4f63ecb1b915a3997f13dff
14825 09bd72201f82a0da14a52230e2ccd0e43529e779
9848 0a5916ad398380ba103d98bad6fa4f2b20b74e75
695905 0ae5469e687083bbf954262873ee8c0c7d017e75

As you can see the hashes are completely different! Unfortunately I
also can't pinpoint where this actually happens... :-/

*several minutes later*

Searching which executables set _fmode and which don't I found the
culprit. test-genrandom.c didn't include git-compat-util.h, so mingw.h
was never included. This caused different random data to be generated,
and as it seems more importantly, of different sizes. Can be fixed
with this patch:

diff --git a/test-genrandom.c b/test-genrandom.c
index 8ad276d..b3c28d9 100644
--- a/test-genrandom.c
+++ b/test-genrandom.c
@@ -4,8 +4,7 @@
  * Copyright (C) 2007 by Nicolas Pitre, licensed under the GPL version 2.
  */

-#include <stdio.h>
-#include <stdlib.h>
+#include "git-compat-util.h"

 int main(int argc, char *argv[])
 {

Or maybe there's a hidden bug in git that gets uncovered with slightly
different random data, I don't know. I'm glad I could finally solve
this mindbending issue. :)