fseeko issues in mys2/mingw64

fseeko issues in mys2/mingw64

[John Donoghue]

This is a question/perhaps bug on bigfile support with fseeko in gnulib 
when being used in MSYS2/mingw64.

As a quick overview:

I have a demo program that opens a 5 Gig file add just does a seeko to 
the end of the file and then does a ftellp to get the end position.

I am importing ftello and fseeko and fflush and it replaces the fseeko 
and lseek functions which then in the depths in the gnulib calls _lseek 
rather than _lseeki64.


As an additional note, it doesnt seem to replace the lseek if we arent 
importing fflush

--------------------------------------------------
main.cpp:

#include <config.h>

#include <stdio.h>
#include <stdint.h>

int main()
{
   fprintf(stderr, "test big file\n");

   /* create 5G file if not there yet */
   FILE *f1 = fopen("big.txt", "r");
   if(!f1)
   {
    fprintf(stderr, "creating big file ...");
    fflush(stderr);
    f1 = fopen("big.txt", "w");
    if(f1)
    {
      char buf[1024];

      int64_t sz = 5LL*1024*1024*1024;

      while(sz > 0)
      {
        fwrite(buf, sizeof(buf), 1, f1);
        sz -= sizeof(buf);
      }

    }
    fprintf(stderr, "done\n");
   }
   fclose(f1);

   /* open file and test */
   FILE *f = fopen("big.txt", "r");

   fprintf(stderr, "sizeof(off_t)=%d\n",(int)sizeof(off_t));

   if(f)
   {
     int status = fseeko(f, 0, SEEK_END);
     fprintf(stderr, "seek status=%d\n", status);

     off_t p = ftello(f);
     fprintf(stderr, "tell pos=%lld\n", (long long)p);
     fclose(f);
   }

   return 0;
}

configure.ac:

AC_INIT([bigfile], [1.0.0])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])

AC_CONFIG_HEADERS([config.h])
AC_CONFIG_MACRO_DIRS([m4])

# Checks for programs.
AC_PROG_CXX
AC_PROG_CC
AC_PROG_CC_STDC
gl_EARLY

# For gnulib.
gl_INIT

AC_CONFIG_FILES([lib/Makefile Makefile])
AC_OUTPUT

-------------------------------------------------
Makefile.am:

ACLOCAL_AMFLAGS = -I m4
SUBDIRS = lib

AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib
EXTRA_DIST = m4/gnulib-cache.m4

bin_PROGRAMS = bigtest
bigtest_SOURCES = main.cpp
bigtest_LDADD = lib/libgnu.a

--------------------------------------------------
bootstrap.sh:

if ! [ -d gnulib ]; then \
   git clone git://git.sv.gnu.org/gnulib.git gnulib
fi

./gnulib/gnulib-tool  --import fflush fseeko ftello largefile stdio

touch NEWS README AUTHORS ChangeLog

autoreconf --install

--------------------------------------------------
Running bootsrap shows the actual modules imported as:

Module list with included dependencies (indented):
     absolute-header
     errno
     extensions
     extern-inline
   fflush
     fpurge
     freading
     fseek
   fseeko
     fstat
     ftell
   ftello
     include_next
     intprops
   largefile
     lseek
     msvc-inval
     msvc-nothrow
     pathmax
     snippet/arg-nonnull
     snippet/c++defs
     snippet/unused-parameter
     snippet/warn-on-use
     ssize_t
     stat-time
     stdbool
     stddef
   stdio
     sys_stat
     sys_types
     time
     unistd
     verify

--------------------------------------------------

Running native configure and make in linux builds ok.

The following modules are added to libnu.a:
   freading.o stat-time.o unistd.o fflush.o fpurge.o fseek.o fseeko.o

Running the app:

./bigtest
test big file
sizeof(off_t)=8
seek status=0
tell pos=5368709120

Running in debug, it has replaced fseek with the gnulib fseeko and calls 
lseek at line 117 of fseeko.c

------------------------------------------------
Running in a native configure in msys2/mingw64 and make builds ok;


The following modules are added to libnu.a:
freading.o stat-time.o unistd.o fflush.o fpurge.o fseek.o fseeko.o 
fstat.o lseek.o msvc-inval.o msvc-nothrow.o stat-w32.o

Running the app:

./bigtest.exe
test big file
creating big file ...done
sizeof(off_t)=8
seek status=-1
tell pos=0

Running in debug, and doing a break on _lseek

(gdb) bt
#0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from 
C:\WINDOWS\System32\msvcrt.dll
#1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
     whence=whence@entry=1) at lseek.c:69
#2  0x0000000000401636 in rpl_fseeko (
     fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
     whence=whence@entry=2) at fseeko.c:45
#3  0x000000000040838c in main () at main.cpp:41

And for the lseek call that fails:

(gdb) bt
#0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from 
C:\WINDOWS\System32\msvcrt.dll
#1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
     whence=whence@entry=2) at lseek.c:69
#2  0x0000000000401683 in rpl_fseeko (
     fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
     whence=whence@entry=2) at fseeko.c:117
#3  0x000000000040838c in main () at main.cpp:41

------------------------------------------------

configure output:
.
checking build system type... x86_64-w64-mingw32
checking host system type... x86_64-w64-mingw32
checking for _LARGEFILE_SOURCE value needed for large files... no
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking whether the preprocessor supports include_next... yes
.
checking for 64-bit off_t... yes
checking for 64-bit st_size... no
checking whether fseeko is declared... yes
checking for fseeko... yes
checking whether fflush works on input streams... no
checking whether stat file-mode macros are broken... no
checking for nlink_t... no
checking whether ftello is declared... yes
checking for ftello... yes
checking whether ftello works... yes
.
checking whether fflush works on input streams... (cached) no
checking whether fpurge is declared... no
checking for fseeko... (cached) yes
checking whether fflush works on input streams... (cached) no
checking for _fseeki64... yes
checking whether _fseeki64 is declared... yes
checking for ftello... (cached) yes
checking whether ftello works... (cached) yes
checking whether lseek detects pipes... no
.

------------------------------------------------

Looking at the sources for gnulib lseek.c:

It is entering the lseek call which is really _lseek (parameters that 
are long) rather than 64bit instead of using the _lseeki64 call.

#if _GL_WINDOWS_64_BIT_OFF_T
   return _lseeki64 (fd, offset, whence);
#else
   return lseek (fd, offset, whence);
#endif


Looking for _GL_WINDOWS_64_BIT_OFF_T, it only gets set in 
lib/sys_types.in.h if WINDOWS_64_BIT_OFF_T is set.

Should lseek.c  call _lseeki64 regardless of _GL_WINDOWS_64_BIT_OFF_T 
value if _lseeki64 is available ? Or perhaps if (_FILE_OFFSET_BITS == 64 
&& HAVE_DECL__FSEEKI64 == 1)

Re: fseeko issues in mys2/mingw64

[John Donoghue]

On 1/25/20 11:23 AM, John Donoghue wrote:
> This is a question/perhaps bug on bigfile support with fseeko in 
> gnulib when being used in MSYS2/mingw64.
>
> As a quick overview:
>
> I have a demo program that opens a 5 Gig file add just does a seeko to 
> the end of the file and then does a ftellp to get the end position.
>
> I am importing ftello and fseeko and fflush and it replaces the fseeko 
> and lseek functions which then in the depths in the gnulib calls 
> _lseek rather than _lseeki64.
>
>
> As an additional note, it doesnt seem to replace the lseek if we arent 
> importing fflush
>
> --------------------------------------------------
> main.cpp:
>
> #include <config.h>
>
> #include <stdio.h>
> #include <stdint.h>
>
> int main()
> {
>   fprintf(stderr, "test big file\n");
>
>   /* create 5G file if not there yet */
>   FILE *f1 = fopen("big.txt", "r");
>   if(!f1)
>   {
>    fprintf(stderr, "creating big file ...");
>    fflush(stderr);
>    f1 = fopen("big.txt", "w");
>    if(f1)
>    {
>      char buf[1024];
>
>      int64_t sz = 5LL*1024*1024*1024;
>
>      while(sz > 0)
>      {
>        fwrite(buf, sizeof(buf), 1, f1);
>        sz -= sizeof(buf);
>      }
>
>    }
>    fprintf(stderr, "done\n");
>   }
>   fclose(f1);
>
>   /* open file and test */
>   FILE *f = fopen("big.txt", "r");
>
>   fprintf(stderr, "sizeof(off_t)=%d\n",(int)sizeof(off_t));
>
>   if(f)
>   {
>     int status = fseeko(f, 0, SEEK_END);
>     fprintf(stderr, "seek status=%d\n", status);
>
>     off_t p = ftello(f);
>     fprintf(stderr, "tell pos=%lld\n", (long long)p);
>     fclose(f);
>   }
>
>   return 0;
> }
>
> configure.ac:
>
> AC_INIT([bigfile], [1.0.0])
> AM_INIT_AUTOMAKE([-Wall -Werror foreign])
>
> AC_CONFIG_HEADERS([config.h])
> AC_CONFIG_MACRO_DIRS([m4])
>
> # Checks for programs.
> AC_PROG_CXX
> AC_PROG_CC
> AC_PROG_CC_STDC
> gl_EARLY
>
> # For gnulib.
> gl_INIT
>
> AC_CONFIG_FILES([lib/Makefile Makefile])
> AC_OUTPUT
>
> -------------------------------------------------
> Makefile.am:
>
> ACLOCAL_AMFLAGS = -I m4
> SUBDIRS = lib
>
> AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib
> EXTRA_DIST = m4/gnulib-cache.m4
>
> bin_PROGRAMS = bigtest
> bigtest_SOURCES = main.cpp
> bigtest_LDADD = lib/libgnu.a
>
> --------------------------------------------------
> bootstrap.sh:
>
> if ! [ -d gnulib ]; then \
>   git clone git://git.sv.gnu.org/gnulib.git gnulib
> fi
>
> ./gnulib/gnulib-tool  --import fflush fseeko ftello largefile stdio
>
> touch NEWS README AUTHORS ChangeLog
>
> autoreconf --install
>
> --------------------------------------------------
> Running bootsrap shows the actual modules imported as:
>
> Module list with included dependencies (indented):
>     absolute-header
>     errno
>     extensions
>     extern-inline
>   fflush
>     fpurge
>     freading
>     fseek
>   fseeko
>     fstat
>     ftell
>   ftello
>     include_next
>     intprops
>   largefile
>     lseek
>     msvc-inval
>     msvc-nothrow
>     pathmax
>     snippet/arg-nonnull
>     snippet/c++defs
>     snippet/unused-parameter
>     snippet/warn-on-use
>     ssize_t
>     stat-time
>     stdbool
>     stddef
>   stdio
>     sys_stat
>     sys_types
>     time
>     unistd
>     verify
>
> --------------------------------------------------
>
> Running native configure and make in linux builds ok.
>
> The following modules are added to libnu.a:
>   freading.o stat-time.o unistd.o fflush.o fpurge.o fseek.o fseeko.o
>
> Running the app:
>
> ./bigtest
> test big file
> sizeof(off_t)=8
> seek status=0
> tell pos=5368709120
>
> Running in debug, it has replaced fseek with the gnulib fseeko and 
> calls lseek at line 117 of fseeko.c
>
> ------------------------------------------------
> Running in a native configure in msys2/mingw64 and make builds ok;
>
>
> The following modules are added to libnu.a:
> freading.o stat-time.o unistd.o fflush.o fpurge.o fseek.o fseeko.o 
> fstat.o lseek.o msvc-inval.o msvc-nothrow.o stat-w32.o
>
> Running the app:
>
> ./bigtest.exe
> test big file
> creating big file ...done
> sizeof(off_t)=8
> seek status=-1
> tell pos=0
>
> Running in debug, and doing a break on _lseek
>
> (gdb) bt
> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from 
> C:\WINDOWS\System32\msvcrt.dll
> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>     whence=whence@entry=1) at lseek.c:69
> #2  0x0000000000401636 in rpl_fseeko (
>     fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
>     whence=whence@entry=2) at fseeko.c:45
> #3  0x000000000040838c in main () at main.cpp:41
>
> And for the lseek call that fails:
>
> (gdb) bt
> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from 
> C:\WINDOWS\System32\msvcrt.dll
> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>     whence=whence@entry=2) at lseek.c:69
> #2  0x0000000000401683 in rpl_fseeko (
>     fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
>     whence=whence@entry=2) at fseeko.c:117
> #3  0x000000000040838c in main () at main.cpp:41
>
> ------------------------------------------------
>
> configure output:
> .
> checking build system type... x86_64-w64-mingw32
> checking host system type... x86_64-w64-mingw32
> checking for _LARGEFILE_SOURCE value needed for large files... no
> checking for special C compiler options needed for large files... no
> checking for _FILE_OFFSET_BITS value needed for large files... 64
> checking whether the preprocessor supports include_next... yes
> .
> checking for 64-bit off_t... yes
> checking for 64-bit st_size... no
> checking whether fseeko is declared... yes
> checking for fseeko... yes
> checking whether fflush works on input streams... no
> checking whether stat file-mode macros are broken... no
> checking for nlink_t... no
> checking whether ftello is declared... yes
> checking for ftello... yes
> checking whether ftello works... yes
> .
> checking whether fflush works on input streams... (cached) no
> checking whether fpurge is declared... no
> checking for fseeko... (cached) yes
> checking whether fflush works on input streams... (cached) no
> checking for _fseeki64... yes
> checking whether _fseeki64 is declared... yes
> checking for ftello... (cached) yes
> checking whether ftello works... (cached) yes
> checking whether lseek detects pipes... no
> .
>
> ------------------------------------------------
>
> Looking at the sources for gnulib lseek.c:
>
> It is entering the lseek call which is really _lseek (parameters that 
> are long) rather than 64bit instead of using the _lseeki64 call.
>
> #if _GL_WINDOWS_64_BIT_OFF_T
>   return _lseeki64 (fd, offset, whence);
> #else
>   return lseek (fd, offset, whence);
> #endif
>
>
> Looking for _GL_WINDOWS_64_BIT_OFF_T, it only gets set in 
> lib/sys_types.in.h if WINDOWS_64_BIT_OFF_T is set.
>
> Should lseek.c  call _lseeki64 regardless of _GL_WINDOWS_64_BIT_OFF_T 
> value if _lseeki64 is available ? Or perhaps if (_FILE_OFFSET_BITS == 
> 64 && HAVE_DECL__FSEEKI64 == 1)
>
>
>
>
>
>
Some additional info,


the gnulib fseeko is being used in this case as m4/fseeko.m4 replaces it 
even if it exists, if fflush is one of the checked functions and fflush 
stdin is not yes (in this case configure whether fflush works on input 
streams ... no )

Re: fseeko issues in mys2/mingw64

[Bruno Haible]

Hi John,

> Running in a native configure in msys2/mingw64 and make builds ok;
> 
> 
> The following modules are added to libnu.a:
> freading.o stat-time.o unistd.o fflush.o fpurge.o fseek.o fseeko.o 
> fstat.o lseek.o msvc-inval.o msvc-nothrow.o stat-w32.o
> 
> Running the app:
> 
> ./bigtest.exe
> test big file
> creating big file ...done
> sizeof(off_t)=8
> seek status=-1
> tell pos=0
> 
> Running in debug, and doing a break on _lseek
> 
> (gdb) bt
> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from 
> C:\WINDOWS\System32\msvcrt.dll
> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>      whence=whence@entry=1) at lseek.c:69
> #2  0x0000000000401636 in rpl_fseeko (
>      fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
>      whence=whence@entry=2) at fseeko.c:45
> #3  0x000000000040838c in main () at main.cpp:41
> 
> And for the lseek call that fails:
> 
> (gdb) bt
> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from 
> C:\WINDOWS\System32\msvcrt.dll
> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>      whence=whence@entry=2) at lseek.c:69
> #2  0x0000000000401683 in rpl_fseeko (
>      fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
>      whence=whence@entry=2) at fseeko.c:117
> #3  0x000000000040838c in main () at main.cpp:41

From these backtraces I infer that:
- gnulib's _GL_WINDOWS_64_BIT_OFF_T is not defined, because mingw64 defines
  off_t to a 64-bit type already, due to _FILE_OFFSET_BITS=64, which is
  ensured by AC_SYS_LARGEFILE.
- mingw does a '#define lseek _lseeki64' in this situation (for proper
  large file support).
- The 'undef lseek' in gnulib's unistd.h and lseek.c undoes it.

The patch below should fix it. I've pushed it already, but I would appreciate
your feedback.

> It is entering the lseek call which is really _lseek (parameters that 
> are long) rather than 64bit instead of using the _lseeki64 call.
> 
> #if _GL_WINDOWS_64_BIT_OFF_T
>    return _lseeki64 (fd, offset, whence);
> #else
>    return lseek (fd, offset, whence);
> #endif
> 
> 
> Looking for _GL_WINDOWS_64_BIT_OFF_T, it only gets set in 
> lib/sys_types.in.h if WINDOWS_64_BIT_OFF_T is set.
> 
> Should lseek.c  call _lseeki64 regardless of _GL_WINDOWS_64_BIT_OFF_T 
> value ...

Yup. You hit the nail on the head.


2020-01-25  Bruno Haible  <bruno@clisp.org>

	lseek: Fix the override to not undo the effects of AC_SYS_LARGEFILE.
	Reported by John Donoghue <john.david.donoghue@gmail.com> in
	<https://lists.gnu.org/archive/html/bug-gnulib/2020-01/msg00146.html>.
	* lib/lseek.c (rpl_lseek): When AC_SYS_LARGEFILE has enabled a 64-bit
	off_t on mingw, invoke _lseeki64 instead of lseek.

diff --git a/lib/lseek.c b/lib/lseek.c
index 013f665..b249839 100644
--- a/lib/lseek.c
+++ b/lib/lseek.c
@@ -63,7 +63,7 @@ rpl_lseek (int fd, off_t offset, int whence)
       return -1;
     }
 #endif
-#if _GL_WINDOWS_64_BIT_OFF_T
+#if _GL_WINDOWS_64_BIT_OFF_T || (defined __MINGW32__ && defined _FILE_OFFSET_BITS && (_FILE_OFFSET_BITS == 64))
   return _lseeki64 (fd, offset, whence);
 #else
   return lseek (fd, offset, whence);

Re: fseeko issues in mys2/mingw64

[John Donoghue]

On 1/25/20 3:07 PM, Bruno Haible wrote:
> Hi John,
>
>> Running in a native configure in msys2/mingw64 and make builds ok;
>>
>>
>> The following modules are added to libnu.a:
>> freading.o stat-time.o unistd.o fflush.o fpurge.o fseek.o fseeko.o
>> fstat.o lseek.o msvc-inval.o msvc-nothrow.o stat-w32.o
>>
>> Running the app:
>>
>> ./bigtest.exe
>> test big file
>> creating big file ...done
>> sizeof(off_t)=8
>> seek status=-1
>> tell pos=0
>>
>> Running in debug, and doing a break on _lseek
>>
>> (gdb) bt
>> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from
>> C:\WINDOWS\System32\msvcrt.dll
>> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>>       whence=whence@entry=1) at lseek.c:69
>> #2  0x0000000000401636 in rpl_fseeko (
>>       fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
>>       whence=whence@entry=2) at fseeko.c:45
>> #3  0x000000000040838c in main () at main.cpp:41
>>
>> And for the lseek call that fails:
>>
>> (gdb) bt
>> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from
>> C:\WINDOWS\System32\msvcrt.dll
>> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>>       whence=whence@entry=2) at lseek.c:69
>> #2  0x0000000000401683 in rpl_fseeko (
>>       fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, offset=offset@entry=0,
>>       whence=whence@entry=2) at fseeko.c:117
>> #3  0x000000000040838c in main () at main.cpp:41
>  From these backtraces I infer that:
> - gnulib's _GL_WINDOWS_64_BIT_OFF_T is not defined, because mingw64 defines
>    off_t to a 64-bit type already, due to _FILE_OFFSET_BITS=64, which is
>    ensured by AC_SYS_LARGEFILE.
> - mingw does a '#define lseek _lseeki64' in this situation (for proper
>    large file support).
> - The 'undef lseek' in gnulib's unistd.h and lseek.c undoes it.
>
> The patch below should fix it. I've pushed it already, but I would appreciate
> your feedback.
>
>> It is entering the lseek call which is really _lseek (parameters that
>> are long) rather than 64bit instead of using the _lseeki64 call.
>>
>> #if _GL_WINDOWS_64_BIT_OFF_T
>>     return _lseeki64 (fd, offset, whence);
>> #else
>>     return lseek (fd, offset, whence);
>> #endif
>>
>>
>> Looking for _GL_WINDOWS_64_BIT_OFF_T, it only gets set in
>> lib/sys_types.in.h if WINDOWS_64_BIT_OFF_T is set.
>>
>> Should lseek.c  call _lseeki64 regardless of _GL_WINDOWS_64_BIT_OFF_T
>> value ...
> Yup. You hit the nail on the head.
>
>
> 2020-01-25  Bruno Haible  <bruno@clisp.org>
>
> 	lseek: Fix the override to not undo the effects of AC_SYS_LARGEFILE.
> 	Reported by John Donoghue <john.david.donoghue@gmail.com> in
> 	<https://lists.gnu.org/archive/html/bug-gnulib/2020-01/msg00146.html>.
> 	* lib/lseek.c (rpl_lseek): When AC_SYS_LARGEFILE has enabled a 64-bit
> 	off_t on mingw, invoke _lseeki64 instead of lseek.
>
> diff --git a/lib/lseek.c b/lib/lseek.c
> index 013f665..b249839 100644
> --- a/lib/lseek.c
> +++ b/lib/lseek.c
> @@ -63,7 +63,7 @@ rpl_lseek (int fd, off_t offset, int whence)
>         return -1;
>       }
>   #endif
> -#if _GL_WINDOWS_64_BIT_OFF_T
> +#if _GL_WINDOWS_64_BIT_OFF_T || (defined __MINGW32__ && defined _FILE_OFFSET_BITS && (_FILE_OFFSET_BITS == 64))
>     return _lseeki64 (fd, offset, whence);
>   #else
>     return lseek (fd, offset, whence);
>
The patch is missing the second '_' in __MINGW32__   with that added, it 
works in my example program both in native mingw64 and cross compiled 
mingw64


Thanks!

Re: fseeko issues in mys2/mingw64

[John Donoghue]

On 1/25/20 4:05 PM, John Donoghue wrote:
> On 1/25/20 3:07 PM, Bruno Haible wrote:
>> Hi John,
>>
>>> Running in a native configure in msys2/mingw64 and make builds ok;
>>>
>>>
>>> The following modules are added to libnu.a:
>>> freading.o stat-time.o unistd.o fflush.o fpurge.o fseek.o fseeko.o
>>> fstat.o lseek.o msvc-inval.o msvc-nothrow.o stat-w32.o
>>>
>>> Running the app:
>>>
>>> ./bigtest.exe
>>> test big file
>>> creating big file ...done
>>> sizeof(off_t)=8
>>> seek status=-1
>>> tell pos=0
>>>
>>> Running in debug, and doing a break on _lseek
>>>
>>> (gdb) bt
>>> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from
>>> C:\WINDOWS\System32\msvcrt.dll
>>> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>>>       whence=whence@entry=1) at lseek.c:69
>>> #2  0x0000000000401636 in rpl_fseeko (
>>>       fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, 
>>> offset=offset@entry=0,
>>>       whence=whence@entry=2) at fseeko.c:45
>>> #3  0x000000000040838c in main () at main.cpp:41
>>>
>>> And for the lseek call that fails:
>>>
>>> (gdb) bt
>>> #0  0x00007ffd2f7ad1cb in msvcrt!_lseek () from
>>> C:\WINDOWS\System32\msvcrt.dll
>>> #1  0x00000000004016e7 in rpl_lseek (fd=3, offset=offset@entry=0,
>>>       whence=whence@entry=2) at lseek.c:69
>>> #2  0x0000000000401683 in rpl_fseeko (
>>>       fp=fp@entry=0x7ffd2f81fa90 <msvcrt!_iob+144>, 
>>> offset=offset@entry=0,
>>>       whence=whence@entry=2) at fseeko.c:117
>>> #3  0x000000000040838c in main () at main.cpp:41
>>  From these backtraces I infer that:
>> - gnulib's _GL_WINDOWS_64_BIT_OFF_T is not defined, because mingw64 
>> defines
>>    off_t to a 64-bit type already, due to _FILE_OFFSET_BITS=64, which is
>>    ensured by AC_SYS_LARGEFILE.
>> - mingw does a '#define lseek _lseeki64' in this situation (for proper
>>    large file support).
>> - The 'undef lseek' in gnulib's unistd.h and lseek.c undoes it.
>>
>> The patch below should fix it. I've pushed it already, but I would 
>> appreciate
>> your feedback.
>>
>>> It is entering the lseek call which is really _lseek (parameters that
>>> are long) rather than 64bit instead of using the _lseeki64 call.
>>>
>>> #if _GL_WINDOWS_64_BIT_OFF_T
>>>     return _lseeki64 (fd, offset, whence);
>>> #else
>>>     return lseek (fd, offset, whence);
>>> #endif
>>>
>>>
>>> Looking for _GL_WINDOWS_64_BIT_OFF_T, it only gets set in
>>> lib/sys_types.in.h if WINDOWS_64_BIT_OFF_T is set.
>>>
>>> Should lseek.c  call _lseeki64 regardless of _GL_WINDOWS_64_BIT_OFF_T
>>> value ...
>> Yup. You hit the nail on the head.
>>
>>
>> 2020-01-25  Bruno Haible  <bruno@clisp.org>
>>
>>     lseek: Fix the override to not undo the effects of AC_SYS_LARGEFILE.
>>     Reported by John Donoghue <john.david.donoghue@gmail.com> in
>>     <https://lists.gnu.org/archive/html/bug-gnulib/2020-01/msg00146.html>. 
>>
>>     * lib/lseek.c (rpl_lseek): When AC_SYS_LARGEFILE has enabled a 
>> 64-bit
>>     off_t on mingw, invoke _lseeki64 instead of lseek.
>>
>> diff --git a/lib/lseek.c b/lib/lseek.c
>> index 013f665..b249839 100644
>> --- a/lib/lseek.c
>> +++ b/lib/lseek.c
>> @@ -63,7 +63,7 @@ rpl_lseek (int fd, off_t offset, int whence)
>>         return -1;
>>       }
>>   #endif
>> -#if _GL_WINDOWS_64_BIT_OFF_T
>> +#if _GL_WINDOWS_64_BIT_OFF_T || (defined __MINGW32__ && defined 
>> _FILE_OFFSET_BITS && (_FILE_OFFSET_BITS == 64))
>>     return _lseeki64 (fd, offset, whence);
>>   #else
>>     return lseek (fd, offset, whence);
>>
> The patch is missing the second '_' in __MINGW32__   with that added, 
> it works in my example program both in native mingw64 and cross 
> compiled mingw64
>
>
> Thanks!
>
Ignore my last comment - I guess when I pasted the patch I missed a '_' 
- the patch was fine.


Thanks!

Re: fseeko issues in mys2/mingw64

[Bruno Haible]

> - the patch was fine.

Thanks for the feedback.

Bruno

Re: fseeko issues in mys2/mingw64

[John Donoghue]

On 1/25/20 6:11 PM, Bruno Haible wrote:
>> - the patch was fine.
> Thanks for the feedback.
>
> Bruno
>
Thanks for the quick response times and help in resolving the bug!