mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e

git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed

* mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e
@ 2020-05-31 10:46 Jan Christoph Uhde
  2020-06-01  4:45 ` Jeff King
  0 siblings, 1 reply; 6+ messages in thread
From: Jan Christoph Uhde @ 2020-05-31 10:46 UTC (permalink / raw)
  To: git

[-- Attachment #1: Type: text/plain, Size: 3679 bytes --]

Hi,

when calling `git diff --quiet` on the gcc git repository
my system fails reproducible with a failed mmap. I have
noticed this possible bug with git in debian unstable and
then build git from source. But I am not able to get a core
or backtrace when gdb is attached.

The command I have used to build git is:

make clean; CFLAGS="-O0 -g -fsanitize=address" DEBUG=1 make -j24


Maybe you can give me some advice how to further debug this.

Jan

--------------------------------------------------------------

The output below was created with the system git not
the self-compiled version (that behaves in the same way).

» ulimit -a; git --version; git diff --quiet; git remote -v
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 514868
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 20000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 2000
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
git version 2.27.0.rc2
fatal: mmap failed: Cannot allocate memory
origin  git://gcc.gnu.org/git/gcc.git (fetch)
origin  git://gcc.gnu.org/git/gcc.git (push)

--------------------------------------------------------------

lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
openat(AT_FDCWD, "gcc/testsuite/gdc.test/fail_compilation/b19717a.d", O_RDONLY) = 3
mmap(NULL, 494, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f4a81a58000
close(3)                                = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
openat(AT_FDCWD, "gcc/testsuite/gdc.test/fail_compilation/b3841.d", O_RDONLY) = 3
mmap(NULL, 2643, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "fatal: mmap failed: Cannot alloc"..., 43fatal: mmap failed: Cannot allocate memory
) = 43
exit_group(128)                         = ?
+++ exited with 128 +++

--------------------------------------------------------------

» uname -a; free -m
Linux edward 5.6.0-2-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64 GNU/Linux
               total        used        free      shared  buff/cache   available
Mem:         128842        6279       86976        1084       35586      120439
Swap:        122067           0      122067



[-- Attachment #2: heaptrack.log --]
[-- Type: text/x-log, Size: 40192 bytes --]

reading file "/opt/gcc_repo/heaptrack.git.1558791.gz" - please wait, this might take some time...
Debuggee command was: git diff --quiet
finished reading file, now analyzing data:

MOST CALLS TO ALLOCATION FUNCTIONS
729067 calls to allocation functions with 484.95MB peak consumption from
0x55ed4a44de70
  in /usr/bin/git
66811 calls with 310.23MB peak consumption from:
    0x55ed4a44dcdc
      in /usr/bin/git
    0x55ed4a3b122e
      in /usr/bin/git
    0x55ed4a3acc72
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
66811 calls with 0B peak consumption from:
    0x55ed4a44dcdc
      in /usr/bin/git
    0x55ed4a3aaf4c
      in /usr/bin/git
    0x55ed4a3acc4c
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
66811 calls with 2.54MB peak consumption from:
    0x55ed4a3accc0
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
65357 calls with 1.57MB peak consumption from:
    0x55ed4a351cce
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
65357 calls with 8.61MB peak consumption from:
    0x55ed4a34ffad
      in /usr/bin/git
    0x55ed4a351c50
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 397920 from 210 other places

152095 calls to allocation functions with 7.16KB peak consumption from
inflateInit2_
  in /lib/x86_64-linux-gnu/libz.so.1
66811 calls with 0B peak consumption from:
    0x55ed4a458b9c
      in /usr/bin/git
    0x55ed4a3aaf93
      in /usr/bin/git
    0x55ed4a3acc4c
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
50269 calls with 7.16KB peak consumption from:
    0x55ed4a458b9c
      in /usr/bin/git
    0x55ed4a3aaf93
      in /usr/bin/git
    0x55ed4a3ad084
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
35015 calls with 0B peak consumption from:
    0x55ed4a458b9c
      in /usr/bin/git
    0x55ed4a3abe9d
      in /usr/bin/git
    0x55ed4a3ad369
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502b2
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

22747 calls to allocation functions with 0B peak consumption from
inflate
  in /lib/x86_64-linux-gnu/libz.so.1
22747 calls with 0B peak consumption from:
    0x55ed4a458ebc
      in /usr/bin/git
    0x55ed4a3abeac
      in /usr/bin/git
    0x55ed4a3ad369
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502b2
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

668 calls to allocation functions with 3.13KB peak consumption from
__strdup
  in /lib/x86_64-linux-gnu/libc.so.6
573 calls with 75B peak consumption from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a30ff1c
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
29 calls with 504B peak consumption from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a330abb
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a3330bf
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
29 calls with 2.12KB peak consumption from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a3309a5
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a3330bf
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
8 calls with 109B peak consumption from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a3309a5
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a333219
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
8 calls with 154B peak consumption from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a330abb
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a333219
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 21 from 19 other places

630 calls to allocation functions with 13.37KB peak consumption from
0x7f5839fd679e
  in /lib/x86_64-linux-gnu/libc.so.6
573 calls with 0B peak consumption from:
    0x55ed4a4604fb
      in /usr/bin/git
    0x55ed4a44e53e
      in /usr/bin/git
    0x55ed4a30fa95
      in /usr/bin/git
    0x55ed4a30fca2
      in /usr/bin/git
    0x55ed4a30feda
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
12 calls with 3.73KB peak consumption from:
    0x7f5839fd4bc4
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd3dba
      in /lib/x86_64-linux-gnu/libc.so.6
    setlocale
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a371396
      in /usr/bin/git
    0x55ed4a24afbf
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
12 calls with 3.73KB peak consumption from:
    0x7f5839fd4bc4
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd3dba
      in /lib/x86_64-linux-gnu/libc.so.6
    setlocale
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3713a7
      in /usr/bin/git
    0x55ed4a24afbf
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
2 calls with 0B peak consumption from:
    0x7f5839fd83c2
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd7cdf
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3bd6da
      in /usr/bin/git
    0x55ed4a3b827c
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
2 calls with 2.05KB peak consumption from:
    0x7f5839fda263
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd8497
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd7cdf
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3bd6da
      in /usr/bin/git
    0x55ed4a3b827c
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 29 from 29 other places

20 calls to allocation functions with 1.28KB peak consumption from
0x7f583a275661
  in /lib64/ld-linux-x86-64.so.2
20 calls with 1.28KB peak consumption from:
    _dl_allocate_tls
      in /lib64/ld-linux-x86-64.so.2
    pthread_create@@GLIBC_2.2.5
      in /lib/x86_64-linux-gnu/libpthread.so.0
    0x55ed4a3b8240
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

7 calls to allocation functions with 0B peak consumption from
_IO_file_doallocate
  in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 0B peak consumption from:
    _IO_doallocbuf
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_file_underflow
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_default_uflow
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a32dd71
      in /usr/bin/git
    0x55ed4a32ef94
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a3330bf
      in /usr/bin/git
    0x55ed4a333523
      in /usr/bin/git
    0x55ed4a429fb1
      in /usr/bin/git
    0x55ed4a24afda
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 0B peak consumption from:
    _IO_doallocbuf
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_file_underflow
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_default_uflow
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a32dd71
      in /usr/bin/git
    0x55ed4a32ef94
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a40b9c2
      in /usr/bin/git
    0x55ed4a40bb8c
      in /usr/bin/git
    0x55ed4a40d14c
      in /usr/bin/git
    0x55ed4a277c94
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 0B peak consumption from:
    _IO_doallocbuf
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_file_underflow
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_default_uflow
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a32dd71
      in /usr/bin/git
    0x55ed4a32ef94
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a3330bf
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 0B peak consumption from:
    _IO_doallocbuf
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_file_underflow
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_default_uflow
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a32dd71
      in /usr/bin/git
    0x55ed4a32ef94
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a333219
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 0B peak consumption from:
    _IO_doallocbuf
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_file_underflow
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_default_uflow
      in /lib/x86_64-linux-gnu/libc.so.6
    _IO_getline_info
      in /lib/x86_64-linux-gnu/libc.so.6
    fgets_unlocked
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd9d57
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fda263
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd8497
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd7cdf
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3bd6da
      in /usr/bin/git
    0x55ed4a3b827c
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 2 from 2 other places

3 calls to allocation functions with 72B peak consumption from
tsearch
  in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 24B peak consumption from:
    0x7f5839fdf96d
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a36605a
      in /usr/bin/git
    0x55ed4a24c4e8
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 24B peak consumption from:
    0x7f5839fdf96d
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a366091
      in /usr/bin/git
    0x55ed4a24c4e8
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 calls with 24B peak consumption from:
    0x7f5839fdf96d
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a40d2f2
      in /usr/bin/git
    0x55ed4a277c94
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

1 calls to allocation functions with 24B peak consumption from
pcre2_general_context_create_8
  in /lib/x86_64-linux-gnu/libpcre2-8.so.0
1 calls with 24B peak consumption from:
    0x55ed4a377ba4
      in /usr/bin/git
    0x55ed4a3eefe7
      in /usr/bin/git
    0x55ed4a277d92
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

1 calls to allocation functions with 72.70KB peak consumption from
0x7f5839e53565
  in /lib/x86_64-linux-gnu/libstdc++.so.6
1 calls with 72.70KB peak consumption from:
    0x7f583a273369
      in /lib64/ld-linux-x86-64.so.2
    0x7f583a273468
      in /lib64/ld-linux-x86-64.so.2
    0x7f583a2650c9
      in /lib64/ld-linux-x86-64.so.2


PEAK MEMORY CONSUMERS
484.95MB peak memory consumed over 729067 calls from
0x55ed4a44de70
  in /usr/bin/git
310.23MB consumed over 66811 calls from:
    0x55ed4a44dcdc
      in /usr/bin/git
    0x55ed4a3b122e
      in /usr/bin/git
    0x55ed4a3acc72
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
104.67MB consumed over 50269 calls from:
    0x55ed4a44dcdc
      in /usr/bin/git
    0x55ed4a3aaf4c
      in /usr/bin/git
    0x55ed4a3ad084
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
30.16MB consumed over 4672 calls from:
    0x55ed4a44dcdc
      in /usr/bin/git
    0x55ed4a44ddef
      in /usr/bin/git
    0x55ed4a3ad13c
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
14.58MB consumed over 1 calls from:
    0x55ed4a467ff6
      in /usr/bin/git
    0x55ed4a4680a8
      in /usr/bin/git
    0x55ed4a3c3662
      in /usr/bin/git
    0x55ed4a3c78ac
      in /usr/bin/git
    0x55ed4a3b83e7
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
8.61MB consumed over 65357 calls from:
    0x55ed4a34ffad
      in /usr/bin/git
    0x55ed4a351c50
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 16.68MB from 210 other places

72.70KB peak memory consumed over 1 calls from
0x7f5839e53565
  in /lib/x86_64-linux-gnu/libstdc++.so.6
72.70KB consumed over 1 calls from:
    0x7f583a273369
      in /lib64/ld-linux-x86-64.so.2
    0x7f583a273468
      in /lib64/ld-linux-x86-64.so.2
    0x7f583a2650c9
      in /lib64/ld-linux-x86-64.so.2

13.37KB peak memory consumed over 630 calls from
0x7f5839fd679e
  in /lib/x86_64-linux-gnu/libc.so.6
3.73KB consumed over 12 calls from:
    0x7f5839fd4bc4
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd3dba
      in /lib/x86_64-linux-gnu/libc.so.6
    setlocale
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a371396
      in /usr/bin/git
    0x55ed4a24afbf
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
3.73KB consumed over 12 calls from:
    0x7f5839fd4bc4
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd3dba
      in /lib/x86_64-linux-gnu/libc.so.6
    setlocale
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3713a7
      in /usr/bin/git
    0x55ed4a24afbf
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
2.05KB consumed over 2 calls from:
    0x7f5839fda263
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd8497
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd7cdf
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3bd6da
      in /usr/bin/git
    0x55ed4a3b827c
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1.60KB consumed over 1 calls from:
    0x7f5839fda263
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd8497
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd7cdf
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3bd6da
      in /usr/bin/git
    0x55ed4a3b827c
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1.18KB consumed over 1 calls from:
    0x55ed4a40d2f2
      in /usr/bin/git
    0x55ed4a277c94
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 1.09KB from 29 other places

7.16KB peak memory consumed over 152095 calls from
inflateInit2_
  in /lib/x86_64-linux-gnu/libz.so.1
7.16KB consumed over 50269 calls from:
    0x55ed4a458b9c
      in /usr/bin/git
    0x55ed4a3aaf93
      in /usr/bin/git
    0x55ed4a3ad084
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

3.13KB peak memory consumed over 668 calls from
__strdup
  in /lib/x86_64-linux-gnu/libc.so.6
2.12KB consumed over 29 calls from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a3309a5
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a3330bf
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
504B consumed over 29 calls from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a330abb
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a3330bf
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
154B consumed over 8 calls from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a330abb
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a333219
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
109B consumed over 8 calls from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a3309a5
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a333219
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
75B consumed over 573 calls from:
    0x55ed4a44dd38
      in /usr/bin/git
    0x55ed4a30ff1c
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 168B from 19 other places

1.28KB peak memory consumed over 20 calls from
0x7f583a275661
  in /lib64/ld-linux-x86-64.so.2
1.28KB consumed over 20 calls from:
    _dl_allocate_tls
      in /lib64/ld-linux-x86-64.so.2
    pthread_create@@GLIBC_2.2.5
      in /lib/x86_64-linux-gnu/libpthread.so.0
    0x55ed4a3b8240
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

72B peak memory consumed over 3 calls from
tsearch
  in /lib/x86_64-linux-gnu/libc.so.6
24B consumed over 1 calls from:
    0x7f5839fdf96d
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a36605a
      in /usr/bin/git
    0x55ed4a24c4e8
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
24B consumed over 1 calls from:
    0x7f5839fdf96d
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a366091
      in /usr/bin/git
    0x55ed4a24c4e8
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
24B consumed over 1 calls from:
    0x7f5839fdf96d
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a40d2f2
      in /usr/bin/git
    0x55ed4a277c94
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

24B peak memory consumed over 1 calls from
pcre2_general_context_create_8
  in /lib/x86_64-linux-gnu/libpcre2-8.so.0
24B consumed over 1 calls from:
    0x55ed4a377ba4
      in /usr/bin/git
    0x55ed4a3eefe7
      in /usr/bin/git
    0x55ed4a277d92
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6


MOST TEMPORARY ALLOCATIONS
261461 temporary allocations of 729067 allocations in total (35.86%) from
0x55ed4a44de70
  in /usr/bin/git
65356 temporary allocations of 65356 allocations in total (100.00%) from:
    0x55ed4a41ac66
      in /usr/bin/git
    0x55ed4a41b434
      in /usr/bin/git
    0x55ed4a30fe5a
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
65344 temporary allocations of 65344 allocations in total (100.00%) from:
    0x55ed4a41ac66
      in /usr/bin/git
    0x55ed4a41b434
      in /usr/bin/git
    0x55ed4a30fe5a
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a350217
      in /usr/bin/git
    0x55ed4a350b20
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
65344 temporary allocations of 65344 allocations in total (100.00%) from:
    0x55ed4a41ac66
      in /usr/bin/git
    0x55ed4a41b434
      in /usr/bin/git
    0x55ed4a30fe5a
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35044a
      in /usr/bin/git
    0x55ed4a350acc
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
65342 temporary allocations of 65357 allocations in total (99.98%) from:
    0x55ed4a41ac66
      in /usr/bin/git
    0x55ed4a41b434
      in /usr/bin/git
    0x55ed4a30fe5a
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
29 temporary allocations of 29 allocations in total (100.00%) from:
    0x55ed4a44dcdc
      in /usr/bin/git
    0x55ed4a3306fb
      in /usr/bin/git
    0x55ed4a3308fd
      in /usr/bin/git
    0x55ed4a330988
      in /usr/bin/git
    0x55ed4a3350ee
      in /usr/bin/git
    0x55ed4a32f251
      in /usr/bin/git
    0x55ed4a3327ad
      in /usr/bin/git
    0x55ed4a3330bf
      in /usr/bin/git
    0x55ed4a333354
      in /usr/bin/git
    0x55ed4a333e7b
      in /usr/bin/git
    0x55ed4a277d73
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 46B from 210 other places

129348 temporary allocations of 152095 allocations in total (85.04%) from
inflateInit2_
  in /lib/x86_64-linux-gnu/libz.so.1
66811 temporary allocations of 66811 allocations in total (100.00%) from:
    0x55ed4a458b9c
      in /usr/bin/git
    0x55ed4a3aaf93
      in /usr/bin/git
    0x55ed4a3acc4c
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
50269 temporary allocations of 50269 allocations in total (100.00%) from:
    0x55ed4a458b9c
      in /usr/bin/git
    0x55ed4a3aaf93
      in /usr/bin/git
    0x55ed4a3ad084
      in /usr/bin/git
    0x55ed4a3ad395
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502d0
      in /usr/bin/git
    0x55ed4a350b09
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
12268 temporary allocations of 35015 allocations in total (35.04%) from:
    0x55ed4a458b9c
      in /usr/bin/git
    0x55ed4a3abe9d
      in /usr/bin/git
    0x55ed4a3ad369
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502b2
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

22747 temporary allocations of 22747 allocations in total (100.00%) from
inflate
  in /lib/x86_64-linux-gnu/libz.so.1
22747 temporary allocations of 22747 allocations in total (100.00%) from:
    0x55ed4a458ebc
      in /usr/bin/git
    0x55ed4a3abeac
      in /usr/bin/git
    0x55ed4a3ad369
      in /usr/bin/git
    0x55ed4a412692
      in /usr/bin/git
    0x55ed4a41271a
      in /usr/bin/git
    0x55ed4a3502b2
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6

580 temporary allocations of 630 allocations in total (92.06%) from
0x7f5839fd679e
  in /lib/x86_64-linux-gnu/libc.so.6
572 temporary allocations of 573 allocations in total (99.83%) from:
    0x55ed4a4604fb
      in /usr/bin/git
    0x55ed4a44e53e
      in /usr/bin/git
    0x55ed4a30fa95
      in /usr/bin/git
    0x55ed4a30fca2
      in /usr/bin/git
    0x55ed4a30feda
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
2 temporary allocations of 2 allocations in total (100.00%) from:
    0x7f5839fd83c2
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd7cdf
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3bd6da
      in /usr/bin/git
    0x55ed4a3b827c
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 temporary allocations of 1 allocations in total (100.00%) from:
    0x55ed4a4604fb
      in /usr/bin/git
    0x55ed4a44e53e
      in /usr/bin/git
    0x55ed4a30fa95
      in /usr/bin/git
    0x55ed4a310474
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 temporary allocations of 2 allocations in total (50.00%) from:
    0x7f5839fda263
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd8497
      in /lib/x86_64-linux-gnu/libc.so.6
    0x7f5839fd7cdf
      in /lib/x86_64-linux-gnu/libc.so.6
    0x55ed4a3bd6da
      in /usr/bin/git
    0x55ed4a3b827c
      in /usr/bin/git
    0x55ed4a3b83fc
      in /usr/bin/git
    0x55ed4a278626
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
1 temporary allocations of 1 allocations in total (100.00%) from:
    0x55ed4a4604fb
      in /usr/bin/git
    0x55ed4a44e53e
      in /usr/bin/git
    0x55ed4a30fa95
      in /usr/bin/git
    0x55ed4a3103ec
      in /usr/bin/git
    0x55ed4a310b28
      in /usr/bin/git
    0x55ed4a338cc1
      in /usr/bin/git
    0x55ed4a33ba3d
      in /usr/bin/git
    0x55ed4a35010c
      in /usr/bin/git
    0x55ed4a350ab7
      in /usr/bin/git
    0x55ed4a351ede
      in /usr/bin/git
    0x55ed4a345bae
      in /usr/bin/git
    0x55ed4a27863b
      in /usr/bin/git
    0x55ed4a24b464
      in /usr/bin/git
    0x55ed4a24c523
      in /usr/bin/git
    0x55ed4a24aff9
      in /usr/bin/git
    __libc_start_main
      in /lib/x86_64-linux-gnu/libc.so.6
  and 3B from 29 other places


total runtime: 2.67s.
bytes allocated in total (ignoring deallocations): 3.51GB (1.32GB/s)
calls to allocation functions: 905239 (339294/s)
temporary memory allocations: 414136 (155223/s)
peak heap memory consumption: 485.04MB
peak RSS (including heaptrack overhead): 11.14GB
total memory leaked: 485.04MB

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e
  2020-05-31 10:46 mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e Jan Christoph Uhde
@ 2020-06-01  4:45 ` Jeff King
       [not found]   ` <cfc79aec-ec85-dbec-e37b-6b7035b4c5a4@UhdeJc.com>
  0 siblings, 1 reply; 6+ messages in thread
From: Jeff King @ 2020-06-01  4:45 UTC (permalink / raw)
  To: Jan Christoph Uhde; +Cc: git

On Sun, May 31, 2020 at 12:46:30PM +0200, Jan Christoph Uhde wrote:

> lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
> openat(AT_FDCWD, "gcc/testsuite/gdc.test/fail_compilation/b3841.d", O_RDONLY) = 3
> mmap(NULL, 2643, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)
> openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
> write(2, "fatal: mmap failed: Cannot alloc"..., 43fatal: mmap failed: Cannot allocate memory
> ) = 43

So we open a file in the working tree, and that mmap fails. I guess this
is probably the call to xmmap() in diff_populate_filespec(). That file
isn't particularly large.

Is it possible that your local repository has large number of packs? Git
will leave open maps to each pack's index file, plus some packs
themselves (ones we're accessing, plus we map+close small ones), plus
whatever maps are used by libc to malloc.  The kernel default limit for
the number of maps is 65530. If you have on the order of 30,000 packs
you might run into this limit.

You can check the number of packs with "git count-objects -v", and the
map limit with "sysctl vm.max_map_count".

If that's the problem, the solution is to repack (which should also
generally improve performance). If you have trouble repacking due to the
limits, you can overcome the chicken and egg with:

  sysctl -w vm.max_map_count=131060

-Peff

^ permalink raw reply	[flat|nested] 6+ messages in thread

[parent not found: <cfc79aec-ec85-dbec-e37b-6b7035b4c5a4@UhdeJc.com>]

* Re: mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e
       [not found]   ` <cfc79aec-ec85-dbec-e37b-6b7035b4c5a4@UhdeJc.com>
@ 2020-06-01 16:54     ` Jeff King
  2020-06-01 20:22       ` [PATCH] diff: discard blob data from stat-unmatched pairs Jeff King
  0 siblings, 1 reply; 6+ messages in thread
From: Jeff King @ 2020-06-01 16:54 UTC (permalink / raw)
  To: Jan Christoph Uhde; +Cc: git

[re-adding list to cc; I think there are some interesting bits for
discussion here]

On Mon, Jun 01, 2020 at 08:41:08AM +0200, Jan Christoph Uhde wrote:

> > Is it possible that your local repository has large number of packs? Git
> > will leave open maps to each pack's index file, plus some packs
> > themselves (ones we're accessing, plus we map+close small ones), plus
> > whatever maps are used by libc to malloc.  The kernel default limit for
> > the number of maps is 65530. If you have on the order of 30,000 packs
> > you might run into this limit.
> > 
> > You can check the number of packs with "git count-objects -v", and the
> 
> » git count-objects -v
> count: 0
> size: 0
> in-pack: 2399361
> packs: 1
> size-pack: 919395
> prune-packable: 0
> garbage: 0
> size-garbage: 0

OK, so that's just one pack (with a few million objects, which is
expected).

> > If that's the problem, the solution is to repack (which should also
> > generally improve performance). If you have trouble repacking due to the
> > limits, you can overcome the chicken and egg with:
> > 
> >    sysctl -w vm.max_map_count=131060
> 
> This fixes the problem indeed!

Now that surprises me. However, I can reproduce the issue with a fresh
clone of gcc:

  $ git clone https://github.com/gcc-mirror/gcc
  $ cd gcc
  $ git diff --quiet ;# works!
  $ find . -type f | xargs touch ;# dirty the index entry for each file
  $ git diff --quiet
  fatal: mmap failed: Cannot allocate memory

There are ~100k files in that repo. If we mmap each one as part of
diff_populate_filespec(), we're going to end up with too many maps.

So we can easily reproduce this case without having to download the huge
gcc repo:

  git init repo
  cd repo
  for i in $(seq 70); do
    mkdir $i
    for j in $(seq 1000); do
      echo "foo" >$i/$j
    done
  done
  git add .
  git commit -m 'add a bunch of files'

  git diff --quiet
  git ls-files | xargs touch
  git diff --quiet

> What could be the reason that the problem only occurs with the `--quiet` flag
> that I use in my prompt command, but not when using `git diff` without the flag.

That is curious, and I can reproduce it here.

In the non-quiet case, we queue each filepair that we find to be
stat-dirty. And then we call diffcore_skip_stat_unmatch() from
diffcore_std() on the whole list. We load each file singly, see that
it's not really a change (the index entries are just stat-dirty because
of our touch), and then discard it.

Whereas in the --quiet case, we hit this code in diff_change when
queuing each:

          if (options->flags.quick && options->skip_stat_unmatch &&
              !diff_filespec_check_stat_unmatch(options->repo, p))
                  return;

          options->flags.has_changes = 1;

That code is trying to make sure we accurately set the has_changes flag
in quick/quiet mode (because we can stop diffing as soon as we see a
single change). Since none of these changes is interesting (they're all
just stat-dirty entries), we have to keep going and look at each one.
But this code leaves the populated filespec in the queue. Doing this
makes "diff --quiet" succeed in this case:

diff --git a/diff.c b/diff.c
index d1ad6a3c4a..2535614735 100644
--- a/diff.c
+++ b/diff.c
@@ -6758,8 +6758,15 @@ void diff_change(struct diff_options *options,
 		return;

 	if (options->flags.quick && options->skip_stat_unmatch &&
-	    !diff_filespec_check_stat_unmatch(options->repo, p))
+	    !diff_filespec_check_stat_unmatch(options->repo, p)) {
+		/*
+		 * discard any populated data; this entry is uninteresting;
+		 * we probably ought to avoid queuing the pair at all!
+		 */
+		diff_free_filespec_data(p->one);
+		diff_free_filespec_data(p->two);
 		return;
+	}

 	options->flags.has_changes = 1;
 }

But that's only because these aren't "real" changes. If we swap out our
"touch" for:

  git ls-files | while read fn; do
    echo bar >$fn
  done

then we have real changes. Our --quiet case will exit immediately once
it sees a change, so it's OK. But now the non-quiet one will fail,
because it's going to load each file to confirm that it's an actual
change, but leave the mmap in place for when we do the actual
content-level diff.

So I think in general we do have problems diffing more than 65k working
tree files in a single process because of this limit. It may still be
worth doing something along the lines of the patch above, though,
because it seems more likely for somebody to have 65k stat-dirty files
than 65k actual changes.

-Peff

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH] diff: discard blob data from stat-unmatched pairs
  2020-06-01 16:54     ` Jeff King
@ 2020-06-01 20:22       ` Jeff King
  2020-06-01 20:53         ` Eric Sunshine
  2020-06-02 16:49         ` Junio C Hamano
  0 siblings, 2 replies; 6+ messages in thread
From: Jeff King @ 2020-06-01 20:22 UTC (permalink / raw)
  To: Jan Christoph Uhde; +Cc: git, Junio C Hamano

On Mon, Jun 01, 2020 at 12:54:08PM -0400, Jeff King wrote:

> > What could be the reason that the problem only occurs with the `--quiet` flag
> > that I use in my prompt command, but not when using `git diff` without the flag.
> 
> That is curious, and I can reproduce it here.

This turned out to be quite an interesting bug to look at. The fix is
simple, but I tried to summarize my explorations below.

I'm still curious why all of your files were stat-dirty. :) As a
workaround until you have a version of Git with the fix, you can
manually refresh the index with:

  git update-index --refresh

before running git-diff.

-- >8 --
Subject: [PATCH] diff: discard blob data from stat-unmatched pairs

When performing a tree-level diff against the working tree, we may find
that our index stat information is dirty, so we queue a filepair to be
examined later. If the actual content hasn't changed, we call this a
stat-unmatch; the stat information was out of date, but there's no
actual diff.  Normally diffcore_std() would detect and remove these
identical filepairs via diffcore_skip_stat_unmatch().  However, when
"--quiet" is used, we want to stop the diff as soon as we see any
changes, so we check for stat-unmatches immediately in diff_change().

That check may require us to actually load the file contents into the
pair of diff_filespecs. If we find that the pair isn't a stat-unmatch,
then no big deal; we'd likely load the contents later anyway to generate
a patch, do rename detection, etc, so we want to hold on to it. But if
it a stat-unmatch, then we have no more use for that data; the whole
point is that we're going discard the pair. However, we never free the
allocated diff_filespec data.

In most cases, keeping that data isn't a problem. We don't expect a lot
of stat-unmatch entries, and since we're using --quiet, we'd quit as
soon as we saw such a real change anyway. However, there are extreme
cases where it makes a big difference:

  1. We'd generally mmap() the working tree half of the pair. And since
     the OS may limit the total number of maps, we can run afoul of this
     in large repositories. E.g.:

       $ cd linux
       $ git ls-files | wc -l
       67959
       $ sysctl vm.max_map_count
       vm.max_map_count = 65530
       $ git ls-files | xargs touch ;# everything is stat-dirty!
       $ git diff --quiet
       fatal: mmap failed: Cannot allocate memory

     It should be unusual to have so many files stat-dirty, but it's
     possible if you've just run a script like "sed -i" or similar.

     After this patch, the above correctly exits with code 0.

  2. Even if you don't hit mmap limits, the index half of the pair will
     have been pulled from the object database into heap memory. Again
     in a clone of linux.git, running:

       $ git ls-files | head -n 10000 | xargs touch
       $ git diff --quiet

     peaks at 145MB heap before this patch, and 94MB after.

This patch solves the problem by freeing any diff_filespec data we
picked up during the "--quiet" stat-unmatch check in diff_changes.
Nobody is going to need that data later, so there's no point holding on
to it. There are a few things to note:

  - we could skip queueing the pair entirely, which could in theory save
    a little work. But there's not much to save, as we need a
    diff_filepair to feed to diff_filespec_check_stat_unmatch() anyway.
    And since we cache the result of the stat-unmatch checks, a later
    call to diffcore_skip_stat_unmatch() call will quickly skip over
    them. The diffcore code also counts up the number of stat-unmatched
    pairs as it removes them. It's doubtful any callers would care about
    that in combination with --quiet, but we'd have to reimplement the
    logic here to be on the safe side. So it's not really worth the
    trouble.

  - I didn't write a test, because we always produce the correct output
    unless we run up against system mmap limits, which are both
    unportable and expensive to test against. Measuring peak heap
    would be interesting, but our perf suite isn't yet capable of that.

  - note that diff without "--quiet" does not suffer from the same
    problem. In diffcore_skip_stat_unmatch(), we detect the stat-unmatch
    entries and drop them immediately, so we're not carrying their data
    around.

  - you _can_ still trigger the mmap limit problem if you truly have
    that many files with actual changes. But it's rather unlikely. The
    stat-unmatch check avoids loading the file contents if the sizes
    don't match, so you'd need a pretty trivial change in every single
    file. Likewise, inexact rename detection might load the data for
    many files all at once. But you'd need not just 64k changes, but
    that many deletions and additions. The most likely candidate is
    perhaps break-detection, which would load the data for all pairs and
    keep it around for the content-level diff. But again, you'd need 64k
    actually changed files in the first place.

    So it's still possible to trigger this case, but it seems like "I
    accidentally made all my files stat-dirty" is the most likely case
    in the real world.

Reported-by: Jan Christoph Uhde <Jan@UhdeJc.com>
Signed-off-by: Jeff King <peff@peff.net>
---
 diff.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/diff.c b/diff.c
index d1ad6a3c4a..4d46fab5d3 100644
--- a/diff.c
+++ b/diff.c
@@ -6758,8 +6758,11 @@ void diff_change(struct diff_options *options,
 		return;
 
 	if (options->flags.quick && options->skip_stat_unmatch &&
-	    !diff_filespec_check_stat_unmatch(options->repo, p))
+	    !diff_filespec_check_stat_unmatch(options->repo, p)) {
+		diff_free_filespec_data(p->one);
+		diff_free_filespec_data(p->two);
 		return;
+	}
 
 	options->flags.has_changes = 1;
 }
-- 
2.27.0.rc2.689.g04a94059a7


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] diff: discard blob data from stat-unmatched pairs
  2020-06-01 20:22       ` [PATCH] diff: discard blob data from stat-unmatched pairs Jeff King
@ 2020-06-01 20:53         ` Eric Sunshine
  2020-06-02 16:49         ` Junio C Hamano
  1 sibling, 0 replies; 6+ messages in thread
From: Eric Sunshine @ 2020-06-01 20:53 UTC (permalink / raw)
  To: Jeff King; +Cc: Jan Christoph Uhde, Git List, Junio C Hamano

On Mon, Jun 1, 2020 at 4:22 PM Jeff King <peff@peff.net> wrote:
> Subject: [PATCH] diff: discard blob data from stat-unmatched pairs
>
> When performing a tree-level diff against the working tree, we may find
> that our index stat information is dirty, so we queue a filepair to be
> examined later. If the actual content hasn't changed, we call this a
> stat-unmatch; the stat information was out of date, but there's no
> actual diff.  Normally diffcore_std() would detect and remove these
> identical filepairs via diffcore_skip_stat_unmatch().  However, when
> "--quiet" is used, we want to stop the diff as soon as we see any
> changes, so we check for stat-unmatches immediately in diff_change().
>
> That check may require us to actually load the file contents into the
> pair of diff_filespecs. If we find that the pair isn't a stat-unmatch,
> then no big deal; we'd likely load the contents later anyway to generate
> a patch, do rename detection, etc, so we want to hold on to it. But if
> it a stat-unmatch, then we have no more use for that data; the whole

s/it/& is/

> point is that we're going discard the pair. However, we never free the
> allocated diff_filespec data.
>
> In most cases, keeping that data isn't a problem. We don't expect a lot
> of stat-unmatch entries, and since we're using --quiet, we'd quit as
> soon as we saw such a real change anyway. However, there are extreme
> cases where it makes a big difference:
>
>   1. We'd generally mmap() the working tree half of the pair. And since
>      the OS may limit the total number of maps, we can run afoul of this
>      in large repositories. E.g.:
>
>        $ cd linux
>        $ git ls-files | wc -l
>        67959
>        $ sysctl vm.max_map_count
>        vm.max_map_count = 65530
>        $ git ls-files | xargs touch ;# everything is stat-dirty!
>        $ git diff --quiet
>        fatal: mmap failed: Cannot allocate memory
>
>      It should be unusual to have so many files stat-dirty, but it's
>      possible if you've just run a script like "sed -i" or similar.
>
>      After this patch, the above correctly exits with code 0.
>
>   2. Even if you don't hit mmap limits, the index half of the pair will
>      have been pulled from the object database into heap memory. Again
>      in a clone of linux.git, running:
>
>        $ git ls-files | head -n 10000 | xargs touch
>        $ git diff --quiet
>
>      peaks at 145MB heap before this patch, and 94MB after.
>
> This patch solves the problem by freeing any diff_filespec data we
> picked up during the "--quiet" stat-unmatch check in diff_changes.
> Nobody is going to need that data later, so there's no point holding on
> to it. There are a few things to note:
>
>   - we could skip queueing the pair entirely, which could in theory save
>     a little work. But there's not much to save, as we need a
>     diff_filepair to feed to diff_filespec_check_stat_unmatch() anyway.
>     And since we cache the result of the stat-unmatch checks, a later
>     call to diffcore_skip_stat_unmatch() call will quickly skip over
>     them. The diffcore code also counts up the number of stat-unmatched
>     pairs as it removes them. It's doubtful any callers would care about
>     that in combination with --quiet, but we'd have to reimplement the
>     logic here to be on the safe side. So it's not really worth the
>     trouble.
>
>   - I didn't write a test, because we always produce the correct output
>     unless we run up against system mmap limits, which are both
>     unportable and expensive to test against. Measuring peak heap
>     would be interesting, but our perf suite isn't yet capable of that.
>
>   - note that diff without "--quiet" does not suffer from the same
>     problem. In diffcore_skip_stat_unmatch(), we detect the stat-unmatch
>     entries and drop them immediately, so we're not carrying their data
>     around.
>
>   - you _can_ still trigger the mmap limit problem if you truly have
>     that many files with actual changes. But it's rather unlikely. The
>     stat-unmatch check avoids loading the file contents if the sizes
>     don't match, so you'd need a pretty trivial change in every single
>     file. Likewise, inexact rename detection might load the data for
>     many files all at once. But you'd need not just 64k changes, but
>     that many deletions and additions. The most likely candidate is
>     perhaps break-detection, which would load the data for all pairs and
>     keep it around for the content-level diff. But again, you'd need 64k
>     actually changed files in the first place.
>
>     So it's still possible to trigger this case, but it seems like "I
>     accidentally made all my files stat-dirty" is the most likely case
>     in the real world.
>
> Reported-by: Jan Christoph Uhde <Jan@UhdeJc.com>
> Signed-off-by: Jeff King <peff@peff.net>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] diff: discard blob data from stat-unmatched pairs
  2020-06-01 20:22       ` [PATCH] diff: discard blob data from stat-unmatched pairs Jeff King
  2020-06-01 20:53         ` Eric Sunshine
@ 2020-06-02 16:49         ` Junio C Hamano
  1 sibling, 0 replies; 6+ messages in thread
From: Junio C Hamano @ 2020-06-02 16:49 UTC (permalink / raw)
  To: Jeff King; +Cc: Jan Christoph Uhde, git

Jeff King <peff@peff.net> writes:

> Subject: [PATCH] diff: discard blob data from stat-unmatched pairs
>
> When performing a tree-level diff against the working tree, we may find
> that our index stat information is dirty, so we queue a filepair to be
> examined later. If the actual content hasn't changed, we call this a
> stat-unmatch; the stat information was out of date, but there's no
> actual diff.  Normally diffcore_std() would detect and remove these
> identical filepairs via diffcore_skip_stat_unmatch().  However, when
> "--quiet" is used, we want to stop the diff as soon as we see any
> changes, so we check for stat-unmatches immediately in diff_change().
>
> That check may require us to actually load the file contents into the
> pair of diff_filespecs. If we find that the pair isn't a stat-unmatch,
> then no big deal; we'd likely load the contents later anyway to generate
> a patch, do rename detection, etc, so we want to hold on to it. But if
> it a stat-unmatch, then we have no more use for that data; the whole
> point is that we're going discard the pair. However, we never free the
> allocated diff_filespec data.

Nicely spotted.  So we can discard when quiet is in effect after
this check, which makes sense.

After reading the initial analysis, I wondered if the fix we did in
f34b205f (diff: do not quit early on stat-dirty files, 2014-01-25)
was suboptimal and we should have instead done the "if QUICK, check
if the pair is merely stat-unmatch" in the loop(s) that call
diff_change(), hoping that it may have given us a better control
over the lifetime of the filespecs in each diff_filepair, but I do
not think it would made much difference.

>  	if (options->flags.quick && options->skip_stat_unmatch &&
> -	    !diff_filespec_check_stat_unmatch(options->repo, p))
> +	    !diff_filespec_check_stat_unmatch(options->repo, p)) {
> +		diff_free_filespec_data(p->one);
> +		diff_free_filespec_data(p->two);
>  		return;
> +	}

Thanks.  Will queue (with that s/it/& is/ typofix mentioned
elsewhere).


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-06-02 16:52 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-31 10:46 mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e Jan Christoph Uhde
2020-06-01  4:45 ` Jeff King
     [not found]   ` <cfc79aec-ec85-dbec-e37b-6b7035b4c5a4@UhdeJc.com>
2020-06-01 16:54     ` Jeff King
2020-06-01 20:22       ` [PATCH] diff: discard blob data from stat-unmatched pairs Jeff King
2020-06-01 20:53         ` Eric Sunshine
2020-06-02 16:49         ` Junio C Hamano

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).