From: Jonathan Nieder <jrnieder@gmail.com>
To: Andreas Krey <a.krey@gmx.de>
Cc: Git Users <git@vger.kernel.org>
Subject: Re: git repack leaks disk space on ENOSPC
Date: Wed, 11 Oct 2017 20:17:03 -0700 [thread overview]
Message-ID: <20171012031702.GB155740@aiede.mtv.corp.google.com> (raw)
In-Reply-To: <20171011150546.GC32090@inner.h.apk.li>
Hi Andreas,
Andreas Krey wrote:
> I observed (again) an annoying behavior of 'git repack':
Do you have context for this 'again'? E.g. was this discussed
previously on-list?
> When the new pack file cannot be fully written because
> the disk gets full beforehand, the tmp_pack file isn't
> deleted, meaning the disk stays full:
>
> $ df -h .; git repack -ad; df -h .; ls -lart .git/objects/pack/tmp*; rm .git/objects/pack/tmp*; df -h .
> Filesystem Size Used Avail Use% Mounted on
> /dev/mapper/vg02-localworkspaces 250G 245G 5.1G 98% /workspaces/calvin
> Counting objects: 4715349, done.
> Delta compression using up to 8 threads.
> Compressing objects: 100% (978051/978051), done.
> fatal: sha1 file '.git/objects/pack/tmp_pack_xB7DMt' write error: No space left on device
> Filesystem Size Used Avail Use% Mounted on
> /dev/mapper/vg02-localworkspaces 250G 250G 20K 100% /workspaces/calvin
> -r--r--r-- 1 andrkrey users 5438435328 Oct 11 17:03 .git/objects/pack/tmp_pack_xB7DMt
> rm: remove write-protected regular file '.git/objects/pack/tmp_pack_xB7DMt'? y
> Filesystem Size Used Avail Use% Mounted on
> /dev/mapper/vg02-localworkspaces 250G 245G 5.1G 98% /workspaces/calvin
>
> git version 2.15.0.rc0
I can imagine this behavior of retaining tmp_pack being useful for
debugging in some circumstances, but I agree with you that it is
certainly not a good default.
Chasing this down, I find:
pack-write.c::create_tmp_packfile chooses the filename
builtin/pack-objects.c::write_pack_file writes to it and the .bitmap, calling
pack-write.c::finish_tmp_packfile to rename it into place
Nothing tries to install an atexit handler to do anything special to it
on exit.
The natural thing, I'd expect, would be for pack-write to use the
tempfile API (see tempfile.h) to create and finish the file. That way,
we'd get such atexit handlers for free. If we want a way to keep temp
files for debugging on abnormal exit, we could set that up separately as
a generic feature of the tempfile API (e.g. an envvar
GIT_KEEP_TEMPFILES_ON_FAILURE), making that an orthogonal topic.
Does using create_tempfile there seem like a good path forward to you?
Would you be interested in working on it (either writing a patch with
such a fix or a test in t/ to make sure it keeps working)?
Thanks,
Jonathan
next prev parent reply other threads:[~2017-10-12 3:17 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-11 15:05 git repack leaks disk space on ENOSPC Andreas Krey
2017-10-12 3:17 ` Jonathan Nieder [this message]
2017-10-12 9:34 ` Andreas Krey
2017-10-12 11:01 ` brian m. carlson
2017-10-12 13:36 ` Jeff King
2017-10-12 13:27 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171012031702.GB155740@aiede.mtv.corp.google.com \
--to=jrnieder@gmail.com \
--cc=a.krey@gmx.de \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).