From: Kim Gybels <kgybels@infogroep.be>
To: "SZEDER Gábor" <szeder.dev@gmail.com>
Cc: "Junio C Hamano" <gitster@pobox.com>,
"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>,
"Git mailing list" <git@vger.kernel.org>,
kilobyte@angband.pl,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Michael J Gruber" <git@grubix.eu>, "Jeff King" <peff@peff.net>,
tboegi@web.de
Subject: Re: [PATCH] gc --auto: release pack files before auto packing
Date: Sun, 8 Jul 2018 01:16:51 +0200 [thread overview]
Message-ID: <20180707231651.GB6152@infogroep.be> (raw)
In-Reply-To: <CAM0VKj=u0OVad3QDRFOc+NWZ9TfwqAwmZ47s=5e5jGZaPQRH6g@mail.gmail.com>
On (07/07/18 11:40), SZEDER Gábor wrote:
>
> On Fri, Jul 6, 2018 at 6:39 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Duy Nguyen <pclouds@gmail.com> writes:
> >
> > > On Sat, Jun 30, 2018 at 03:38:21PM +0200, Kim Gybels wrote:
> > >> Teach gc --auto to release pack files before auto packing the repository
> > >> to prevent failures when removing them.
> > >>
> > >> Also teach the test 'fetching with auto-gc does not lock up' to complain
> > >> when it is no longer triggering an auto packing of the repository.
> > >>
> > >> Fixes https://github.com/git-for-windows/git/issues/500
> > >>
> > >> Signed-off-by: Kim Gybels <kgybels@infogroep.be>
> > >> ---
> > >>
> > >> Patch based on master, since problem doesn't reproduce on maint,
> > >> however, the improvement to the test might be valuable on maint.
> > >>
> > >> builtin/gc.c | 1 +
> > >> t/t5510-fetch.sh | 2 ++
> > >> 2 files changed, 3 insertions(+)
> > >>
> > >> diff --git a/builtin/gc.c b/builtin/gc.c
> > >> index ccfb1ceaeb..63b62ab57c 100644
> > >> --- a/builtin/gc.c
> > >> +++ b/builtin/gc.c
> > >> @@ -612,6 +612,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
> > >> return -1;
> > >>
> > >> if (!repository_format_precious_objects) {
> > >> + close_all_packs(the_repository->objects);
> > >
> > > We have repo_clear() which does this and potentially closing file
> > > descriptors on other things as well. I suggest we use it, and before
> > > any external command is run. Something like
> > >
> > > diff --git a/builtin/gc.c b/builtin/gc.c
> > > index ccfb1ceaeb..a852c71da6 100644
> > > --- a/builtin/gc.c
> > > +++ b/builtin/gc.c
> > > @@ -473,6 +473,13 @@ static int report_last_gc_error(void)
> > >
> > > static int gc_before_repack(void)
> > > {
> > > + /*
> > > + * Shut down everything, we should have all the info we need
> > > + * at this point. Leaving some file descriptors open may
> > > + * prevent them from being removed on Windows.
> > > + */
> > > + repo_clear(the_repository);
> > > +
> > > if (pack_refs && run_command_v_opt(pack_refs_cmd.argv, RUN_GIT_CMD))
> > > return error(FAILED_RUN, pack_refs_cmd.argv[0]);
> >
> > With
> >
> > https://public-inbox.org/git/20180509170409.13666-1-pclouds@gmail.com/
> >
> > the call to repo_clear() on the_repository itself may be safe [*1*],
> > but if command line parsing code etc. has pointers to in-core object
> > etc. obtained before this function was called, the codeflow after
> > this function returns will be quite disturbed. Has anybody in this
> > review thread audited the codeflow _after_ this function is run to
> > make sure that invalidating in-core repository here does not cause
> > us problems?
>
> It does create us problems, unfortunately.
>
> Among other things, a call to repo_clear(the_repository) does this:
>
> raw_object_store_clear(repo->objects);
> FREE_AND_NULL(repo->objects);
>
> Later on cmd_gc() calls reprepare_packed_git(the_repository), which
> then attempts to:
>
> r->objects->approximate_object_count_valid = 0;
> r->objects->packed_git_initialized = 0;
>
> but with r->objects being NULL at this point that won't work, and in
> turn causes failures in several test scripts.
>
>
> FWIW, the original patch appears to be working fine, at least the test
> suite passes.
Should I post a v3 that goes back to the original fix, but uses
test_i18ngrep instead of grep?
In addition to not breaking any tests, close_all_packs is already used
in a similar way in am and fetch just before running "gc --auto".
-Kim
next prev parent reply other threads:[~2018-07-07 23:16 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-30 13:38 [PATCH] gc --auto: release pack files before auto packing Kim Gybels
2018-06-30 14:58 ` Duy Nguyen
2018-07-06 16:39 ` Junio C Hamano
2018-07-07 9:40 ` SZEDER Gábor
2018-07-07 23:16 ` Kim Gybels [this message]
2018-07-09 14:33 ` Duy Nguyen
2018-07-09 21:10 ` Junio C Hamano
2018-07-11 15:33 ` Duy Nguyen
2018-07-04 20:16 ` [PATCH v2] gc --auto: clear repository " Kim Gybels
2018-07-07 9:57 ` SZEDER Gábor
2018-07-09 20:37 ` [PATCH v3] gc --auto: release pack files " Kim Gybels
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180707231651.GB6152@infogroep.be \
--to=kgybels@infogroep.be \
--cc=Johannes.Schindelin@gmx.de \
--cc=git@grubix.eu \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=kilobyte@angband.pl \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
--cc=szeder.dev@gmail.com \
--cc=tboegi@web.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).