From: Lars Schneider <larsxschneider@gmail.com>
To: Martin Ågren <martin.agren@gmail.com>
Cc: Jeff King <peff@peff.net>, Git Users <git@vger.kernel.org>
Subject: Re: [PATCH] pkt-line: re-'static'-ify buffer in packet_write_fmt_1()
Date: Tue, 29 Aug 2017 18:51:52 +0100
Message-ID: <B1E291F2-86FF-4982-A092-92FAED65385C@gmail.com> (raw)
In-Reply-To: <CAN0heSoUqcOqVspZkbPahWQdtVpSdtSZoCFWu0ZQJfN3F0mD2g@mail.gmail.com>
> On 28 Aug 2017, at 06:11, Martin Ågren <martin.agren@gmail.com> wrote:
>
> On 28 August 2017 at 01:23, Jeff King <peff@peff.net> wrote:
>> On Sun, Aug 27, 2017 at 10:04:55PM +0200, Lars Schneider wrote:
>>
>>> I did run all tests under valgrind using "make valgrind" and I found
>>> the following files with potential issues:
>>>
>>> cat valgrind.out | perl -nE 'say /^==.+\((.+)\:.+\)$/' | sort | uniq -c
>>> 7102
>>> 2 clone.c
>>> 33 common-main.c
>>> 6 connect.c
>>> 64 git.c
>>> 4 ls-remote.c
>>> 126 run-command.c
>>> 12 transport.c
>>> 7 worktree.c
>>
>> I'm not sure where valgrind.out comes from. The individual
>> test-results/*.out files may have valgrind output, but I don't think
>> they usually contain leak output.
>>
>> Doing "valgrind ./git-upload-pack . </dev/null >/dev/null" mentions
>> leaked memory but not the locations. Adding --leak-check=full shows that
>> most of it comes from format_packet().
>>
>> And applying Martin's patch drops the "definitely lost" category down to
>> 0 bytes (there's still 550k in "still reachable", but those are in the
>> "exit will free them for us" category).
>>
>>> No mention of "pkt-line.c". Did you run Git with valgrind on one of
>>> your repositories to find it?
>>
>> I'm curious, too. I don't think the valgrind setup in our test suite is
>> great for finding leaks right now.
>
> Sorry for being brief. I've patched t/valgrind/valgrind.sh to say
> "--leak-check=yes". Then I run "./t0000 --valgrind", simply because
> running the complete suite gives more reports than I could possibly
> process.
I set $TOOL_OPTIONS in valgrind.sh: to
'--leak-check=full --errors-for-leak-kinds=definite'
... but I also had to adjust t/test-lib-functions.sh:test_create_repo
as I ran into the error "cannot run git init -- have you built things yet?".
With these changes I was able to see the leak running valgrind with t5110.
---
What if we run a few selected tests with valgrind and count all files that
valgrind mentions (a single leak has multiple file mentions because of
the stack trace and other leak indicators). We record these counts and let
TravisCI scream if one of the numbers increases.
I wonder how stable/fragile such a metric would be as a simple refactoring
could easily change these numbers. Below I ran valgrind on t5510 before and
after Martin's patch. The diff below clearly shows the pkt-line leak.
Would it make sense to pursue something like this in TravisCI to avoid
"pkt-line" kind of leaks in the future?
## Valgrind run with leak
$ ./t5510-fetch.sh --valgrind | perl -nE 'say /^==.+\((.+)\:.+\)$/' | sort | uniq -c
15529
39 abspath.c
30 add.c
34 branch.c
10 bundle.c
268 clone.c
13 commit.c
471 common-main.c
52 config.c
50 connect.c
4 connected.c
1 diff-lib.c
102 dir.c
120 environment.c
4 fetch-pack.c
47 fetch.c
14 files-backend.c
1 git-compat-util.h
871 git.c
96 init-db.c
26 iterator.c
1 list-objects.c
4 log-tree.c
2 object.c
1 pack-objects.c
6 parse-options.c
10 pathspec.c
83 pkt-line.c
6 precompose_utf8.c
2 push.c
67 refs.c
96 remote.c
90 repository.c
4 rev-list.c
10 revision.c
165 run-command.c
42 setup.c
288 strbuf.c
51 strbuf.h
2 tag.c
107 transport.c
10 tree-diff.c
85 upload-pack.c
1 usage.c
455 wrapper.c
## Valgrind run with Matrin's patch to fix the leak
$ ./t5510-fetch.sh --valgrind | perl -nE 'say /^==.+\((.+)\:.+\)$/' | sort | uniq -c
14931
39 abspath.c
30 add.c
34 branch.c
10 bundle.c
268 clone.c
13 commit.c
433 common-main.c
52 config.c
50 connect.c
6 connected.c
1 diff-lib.c
102 dir.c
120 environment.c
4 fetch-pack.c
53 fetch.c
14 files-backend.c
1 git-compat-util.h
879 git.c
96 init-db.c
1 iterator.c
1 list-objects.c
4 log-tree.c
2 object.c
1 pack-objects.c
6 parse-options.c
10 pathspec.c
6 precompose_utf8.c
2 push.c
26 refs.c
96 remote.c
90 repository.c
6 rev-list.c
14 revision.c
171 run-command.c
42 setup.c
246 strbuf.c
50 strbuf.h
2 tag.c
107 transport.c
10 tree-diff.c
2 upload-pack.c
1 usage.c
415 wrapper.c
## Diff
15529 | 14931
39 abspath.c 39 abspath.c
30 add.c 30 add.c
34 branch.c 34 branch.c
10 bundle.c 10 bundle.c
268 clone.c 268 clone.c
13 commit.c 13 commit.c
471 common-main.c | 433 common-main.c
52 config.c 52 config.c
50 connect.c 50 connect.c
4 connected.c | 6 connected.c
1 diff-lib.c 1 diff-lib.c
102 dir.c 102 dir.c
120 environment.c 120 environment.c
4 fetch-pack.c 4 fetch-pack.c
47 fetch.c | 53 fetch.c
14 files-backend.c 14 files-backend.c
1 git-compat-util.h 1 git-compat-util.h
871 git.c | 879 git.c
96 init-db.c 96 init-db.c
26 iterator.c | 1 iterator.c
1 list-objects.c 1 list-objects.c
4 log-tree.c 4 log-tree.c
2 object.c 2 object.c
1 pack-objects.c 1 pack-objects.c
6 parse-options.c 6 parse-options.c
10 pathspec.c 10 pathspec.c
83 pkt-line.c <
6 precompose_utf8.c 6 precompose_utf8.c
2 push.c 2 push.c
67 refs.c | 26 refs.c
96 remote.c 96 remote.c
90 repository.c 90 repository.c
4 rev-list.c | 6 rev-list.c
10 revision.c | 14 revision.c
165 run-command.c | 171 run-command.c
42 setup.c 42 setup.c
288 strbuf.c | 246 strbuf.c
51 strbuf.h | 50 strbuf.h
2 tag.c 2 tag.c
107 transport.c 107 transport.c
10 tree-diff.c 10 tree-diff.c
85 upload-pack.c | 2 upload-pack.c
1 usage.c 1 usage.c
455 wrapper.c | 415 wrapp
Thanks,
Lars
next prev parent reply index
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-08-27 7:37 Martin Ågren
2017-08-27 15:41 ` Jeff King
2017-08-27 18:25 ` Jeff King
2017-08-27 18:21 ` Lars Schneider
2017-08-27 19:09 ` Martin Ågren
2017-08-27 19:15 ` Jeff King
2017-08-27 20:04 ` Lars Schneider
2017-08-27 23:23 ` Jeff King
2017-08-28 4:11 ` Martin Ågren
2017-08-28 17:52 ` Stefan Beller
2017-08-28 17:58 ` Jeff King
2017-09-05 10:03 ` Junio C Hamano
2017-08-29 17:51 ` Lars Schneider [this message]
2017-08-29 18:53 ` Jeff King
2017-08-29 18:58 ` [PATCH] config: use a static lock_file struct Jeff King
2017-08-29 19:09 ` Brandon Williams
2017-08-29 19:10 ` Brandon Williams
2017-08-29 19:12 ` Jeff King
2017-08-30 3:25 ` Michael Haggerty
2017-08-30 4:31 ` Jeff King
2017-08-30 4:55 ` Michael Haggerty
2017-08-30 4:55 ` Jeff King
2017-08-30 5:55 ` Jeff King
2017-08-30 7:07 ` Michael Haggerty
2017-09-02 8:44 ` Jeff King
2017-09-02 13:50 ` Jeff King
2017-08-30 6:55 ` Michael Haggerty
2017-08-30 19:53 ` Jeff King
2017-08-30 19:57 ` Brandon Williams
2017-08-30 20:11 ` Jeff King
2017-08-30 21:06 ` Brandon Williams
2017-08-31 4:09 ` Jeff King
2017-09-06 3:59 ` Junio C Hamano
2017-09-06 12:41 ` Jeff King
2017-08-29 19:22 ` [PATCH] pkt-line: re-'static'-ify buffer in packet_write_fmt_1() Martin Ågren
2017-08-29 21:48 ` Jeff King
2017-08-30 5:31 ` Jeff King
2017-09-05 10:03 ` Junio C Hamano
2017-10-10 4:06 ` [PATCH 0/2] Do not call cmd_*() as a subroutine Junio C Hamano
2017-10-10 4:06 ` [PATCH 1/2] describe: do not use " Junio C Hamano
2017-10-10 4:06 ` [PATCH 2/2] merge-ours: " Junio C Hamano
Reply instructions:
You may reply publically to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=B1E291F2-86FF-4982-A092-92FAED65385C@gmail.com \
--to=larsxschneider@gmail.com \
--cc=git@vger.kernel.org \
--cc=martin.agren@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
git@vger.kernel.org mailing list mirror (one of many)
Archives are clonable:
git clone --mirror https://public-inbox.org/git
git clone --mirror http://ou63pmih66umazou.onion/git
git clone --mirror http://czquwvybam4bgbro.onion/git
git clone --mirror http://hjrcffqmbrq6wope.onion/git
Newsgroups are available over NNTP:
nntp://news.public-inbox.org/inbox.comp.version-control.git
nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
nntp://news.gmane.org/gmane.comp.version-control.git
note: .onion URLs require Tor: https://www.torproject.org/
or Tor2web: https://www.tor2web.org/
AGPL code for this site: git clone https://public-inbox.org/ public-inbox