From: "René Scharfe" <l.s.r@web.de>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: Git List <git@vger.kernel.org>,
Junio C Hamano <gitster@pobox.com>, Taylor Blau <me@ttaylorr.com>,
Christian Couder <chriscool@tuxfamily.org>
Subject: Re: [PATCH 1/3] pack-objects: fix handling of multiple --filter options
Date: Sun, 13 Nov 2022 18:31:12 +0100 [thread overview]
Message-ID: <be477f52-33e0-b2e8-9bcc-9264f1b50460@web.de> (raw)
In-Reply-To: <221112.86bkpcmm6i.gmgdl@evledraar.gmail.com>
Am 12.11.22 um 12:41 schrieb Ævar Arnfjörð Bjarmason:
>
> On Sat, Nov 12 2022, René Scharfe wrote:
>
>> Since 5cb28270a1 (pack-objects: lazily set up "struct rev_info", don't
>> leak, 2022-03-28) --filter options given to git pack-objects overrule
>> earlier ones, letting only the leftmost win and leaking the memory
>> allocated for earlier ones. Fix that by only initializing the rev_info
>> struct once.
>
> If I do e.g. this with SANITIZE=leak:
>
> echo e83c5163316f89bfbde7d9ab23ca2e25604af290 | ./git pack-objects --revs --filter=blob:limit=1001 --filter=object:type=blob --stdout >/dev/null
>
> I see one leak that wasn't there, but two that are gone now. I haven't
> looked into it, but I think the commit message should discuss what leaks
> are fixed, which remain/are new etc.
The leak is insubstantial; I mentioned it just because of the irony. It
is caused by initializing an already initialized struct rev_info without
releasing it in between, as mentioned in the commit message.
.filter_data allocated by filter_combine__init() is not released by
filter_combine__init() in list-objects-filter.c. Plugging that leak
allows your example command to run with SANITIZE=leak. That is a matter
for a separate patch, though.
>
>> @@ -4158,7 +4158,8 @@ static struct list_objects_filter_options *po_filter_revs_init(void *value)
>> {
>> struct po_filter_data *data = value;
>>
>> - repo_init_revisions(the_repository, &data->revs, NULL);
>> + if (!data->have_revs)
>> + repo_init_revisions(the_repository, &data->revs, NULL);
>> data->have_revs = 1;
>>
>> return &data->revs.filter;
>
> FWIW as this goes away in your 2/3 I think just squashing the two with a
> leak fix would be nice, if...
>
>> diff --git a/t/t5317-pack-objects-filter-objects.sh b/t/t5317-pack-objects-filter-objects.sh
>> index bb633c9b09..bd8983bb56 100755
>> --- a/t/t5317-pack-objects-filter-objects.sh
>> +++ b/t/t5317-pack-objects-filter-objects.sh
>> @@ -178,6 +178,25 @@ test_expect_success 'verify blob:limit=1001' '
>> test_cmp expected observed
>> '
>>
>> +test_expect_success 'verify blob:limit=1001+object:type=blob' '
>> + git -C r2 ls-files -s large.1000 |
>
> Aside: Should do "git >tmp && test_parse... <tmp", we lose the exit code
> of "ls-files" here.
OK. Copied that line from the surrounding tests. They used temporary
files before fb2d0db502 (test-lib-functions: add parsing helpers for
ls-files and ls-tree, 2022-04-04).
>
>> + test_parse_ls_files_stage_oids |
>> + sort >expected &&
>> +
>> + git -C r2 pack-objects --revs --stdout --filter=blob:limit=1001 \
>> + --filter=object:type=blob >filter.pack <<-EOF &&
>> + HEAD
>> + EOF
>> + git -C r2 index-pack ../filter.pack &&
>> +
>> + git -C r2 verify-pack -v ../filter.pack >verify_result &&
>> + grep blob verify_result |
>> + parse_verify_pack_blob_oid |
>
> Whereas this one's not a problem, no "git".
>
>> + sort >observed &&
>> +
>> + test_cmp expected observed
>
> Aside: It would be nice if we had a "test_cmp_sort", but some other day...
>
>> +'
>> +
>> test_expect_success 'verify blob:limit=10001' '
>> git -C r2 ls-files -s large.1000 large.10000 |
>> test_parse_ls_files_stage_oids |
>
> ...we can test it, but this test is in a file that's not part of "linux-leaks".
>
> If that one leak I mentioned above can be fixed (or maybe it's not new?)
> this could be tested if we put it in a file of its own with
> TEST_PASSES_SANITIZE_LEAK=true.
Plugging the leak in your example command is not enough to make t5317
leak free.
René
next prev parent reply other threads:[~2022-11-13 17:31 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-12 10:42 [PATCH 0/3] pack-objects: fix and simplify --filter handling René Scharfe
2022-11-12 10:44 ` [PATCH 1/3] pack-objects: fix handling of multiple --filter options René Scharfe
2022-11-12 11:41 ` Ævar Arnfjörð Bjarmason
2022-11-13 17:31 ` René Scharfe [this message]
2022-11-12 16:58 ` Jeff King
2022-11-13 5:01 ` Taylor Blau
2022-11-13 16:44 ` Jeff King
2022-11-13 17:31 ` René Scharfe
2022-11-12 10:44 ` [PATCH 2/3] pack-object: simplify --filter handling René Scharfe
2022-11-12 11:45 ` Ævar Arnfjörð Bjarmason
2022-11-12 17:02 ` Jeff King
2022-11-13 16:49 ` Jeff King
2022-11-13 17:31 ` René Scharfe
2022-11-12 10:46 ` [PATCH 3/3] list-objects-filter: remove OPT_PARSE_LIST_OBJECTS_FILTER_INIT() René Scharfe
2022-11-20 10:03 ` [PATCH v2 0/3] pack-objects: fix and simplify --filter handling René Scharfe
2022-11-20 10:06 ` [PATCH v2 1/3] t5317: stop losing return codes of git ls-files René Scharfe
2022-11-20 10:07 ` [PATCH v2 2/3] t5317: demonstrate failure to handle multiple --filter options René Scharfe
2022-11-20 10:13 ` [PATCH v2 3/3] Revert "pack-objects: lazily set up "struct rev_info", don't leak" René Scharfe
2022-11-28 10:03 ` Junio C Hamano
2022-11-28 11:12 ` Ævar Arnfjörð Bjarmason
2022-11-28 12:00 ` [PATCH] t5314: check exit code of "rev-parse" Ævar Arnfjörð Bjarmason
2022-11-28 13:51 ` René Scharfe
2022-11-28 14:18 ` [PATCH v2] t5314: check exit code of "git" Ævar Arnfjörð Bjarmason
2022-11-28 11:26 ` [PATCH v2 3/3] Revert "pack-objects: lazily set up "struct rev_info", don't leak" René Scharfe
2022-11-28 11:31 ` Ævar Arnfjörð Bjarmason
2022-11-28 12:24 ` Ævar Arnfjörð Bjarmason
2022-11-28 15:16 ` René Scharfe
2022-11-28 15:27 ` Ævar Arnfjörð Bjarmason
2022-11-28 14:29 ` René Scharfe
2022-11-28 14:34 ` Ævar Arnfjörð Bjarmason
2022-11-28 15:56 ` René Scharfe
2022-11-28 17:57 ` René Scharfe
2022-11-28 18:32 ` Ævar Arnfjörð Bjarmason
2022-11-28 21:57 ` René Scharfe
2022-11-29 1:26 ` Jeff King
2022-11-29 1:46 ` Junio C Hamano
2022-11-29 10:25 ` Ævar Arnfjörð Bjarmason
2022-11-29 7:12 ` Ævar Arnfjörð Bjarmason
2022-11-29 19:18 ` René Scharfe
2022-11-28 17:57 ` Ævar Arnfjörð Bjarmason
2022-11-22 19:02 ` [PATCH v2 0/3] pack-objects: fix and simplify --filter handling Jeff King
2022-11-29 12:19 ` [PATCH v3 0/5] " René Scharfe
2022-11-29 12:21 ` [PATCH v3 1/5] t5317: stop losing return codes of git ls-files René Scharfe
2022-11-29 12:22 ` [PATCH v3 2/5] t5317: demonstrate failure to handle multiple --filter options René Scharfe
2022-11-29 12:23 ` [PATCH v3 3/5] pack-objects: fix handling of " René Scharfe
2022-11-30 1:09 ` Junio C Hamano
2022-11-30 7:11 ` René Scharfe
2022-11-29 12:25 ` [PATCH v3 4/5] pack-objects: simplify --filter handling René Scharfe
2022-11-29 13:27 ` Ævar Arnfjörð Bjarmason
2022-11-30 11:23 ` René Scharfe
2022-11-29 12:26 ` [PATCH v3 5/5] list-objects-filter: remove OPT_PARSE_LIST_OBJECTS_FILTER_INIT() René Scharfe
2022-11-30 1:20 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=be477f52-33e0-b2e8-9bcc-9264f1b50460@web.de \
--to=l.s.r@web.de \
--cc=avarab@gmail.com \
--cc=chriscool@tuxfamily.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=me@ttaylorr.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).