From: Brandon Casey <bcasey@nvidia.com>
To: <git@vger.kernel.org>
Cc: <gitster@pobox.com>, <peff@peff.net>, <spearce@spearce.org>,
<sunshine@sunshineco.com>, Brandon Casey <drafnel@gmail.com>
Subject: [PATCH 2/2] Don't close pack fd when free'ing pack windows
Date: Wed, 31 Jul 2013 12:51:37 -0700 [thread overview]
Message-ID: <1375300297-6744-2-git-send-email-bcasey@nvidia.com> (raw)
In-Reply-To: <1375300297-6744-1-git-send-email-bcasey@nvidia.com>
From: Brandon Casey <drafnel@gmail.com>
Now that close_one_pack() has been introduced to handle file
descriptor pressure, it is not strictly necessary to close the
pack file descriptor in unuse_one_window() when we're under memory
pressure.
Jeff King provided a justification for leaving the pack file open:
If you close packfile descriptors, you can run into racy situations
where somebody else is repacking and deleting packs, and they go away
while you are trying to access them. If you keep a descriptor open,
you're fine; they last to the end of the process. If you don't, then
they disappear from under you.
For normal object access, this isn't that big a deal; we just rescan
the packs and retry. But if you are packing yourself (e.g., because
you are a pack-objects started by upload-pack for a clone or fetch),
it's much harder to recover (and we print some warnings).
Let's do so (or uh, not do so).
Signed-off-by: Brandon Casey <drafnel@gmail.com>
---
builtin/pack-objects.c | 2 +-
git-compat-util.h | 2 +-
sha1_file.c | 21 +++++++--------------
3 files changed, 9 insertions(+), 16 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f069462..4eb0521 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1809,7 +1809,7 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,
static void try_to_free_from_threads(size_t size)
{
read_lock();
- release_pack_memory(size, -1);
+ release_pack_memory(size);
read_unlock();
}
diff --git a/git-compat-util.h b/git-compat-util.h
index cc4ba4d..29b1ee3 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -517,7 +517,7 @@ int inet_pton(int af, const char *src, void *dst);
const char *inet_ntop(int af, const void *src, char *dst, size_t size);
#endif
-extern void release_pack_memory(size_t, int);
+extern void release_pack_memory(size_t);
typedef void (*try_to_free_t)(size_t);
extern try_to_free_t set_try_to_free_routine(try_to_free_t);
diff --git a/sha1_file.c b/sha1_file.c
index 7731ab1..d26121a 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -614,7 +614,7 @@ static void scan_windows(struct packed_git *p,
}
}
-static int unuse_one_window(struct packed_git *current, int keep_fd)
+static int unuse_one_window(struct packed_git *current)
{
struct packed_git *p, *lru_p = NULL;
struct pack_window *lru_w = NULL, *lru_l = NULL;
@@ -628,15 +628,8 @@ static int unuse_one_window(struct packed_git *current, int keep_fd)
pack_mapped -= lru_w->len;
if (lru_l)
lru_l->next = lru_w->next;
- else {
+ else
lru_p->windows = lru_w->next;
- if (!lru_p->windows && lru_p->pack_fd != -1
- && lru_p->pack_fd != keep_fd) {
- close(lru_p->pack_fd);
- pack_open_fds--;
- lru_p->pack_fd = -1;
- }
- }
free(lru_w);
pack_open_windows--;
return 1;
@@ -644,10 +637,10 @@ static int unuse_one_window(struct packed_git *current, int keep_fd)
return 0;
}
-void release_pack_memory(size_t need, int fd)
+void release_pack_memory(size_t need)
{
size_t cur = pack_mapped;
- while (need >= (cur - pack_mapped) && unuse_one_window(NULL, fd))
+ while (need >= (cur - pack_mapped) && unuse_one_window(NULL))
; /* nothing */
}
@@ -658,7 +651,7 @@ void *xmmap(void *start, size_t length,
if (ret == MAP_FAILED) {
if (!length)
return NULL;
- release_pack_memory(length, fd);
+ release_pack_memory(length);
ret = mmap(start, length, prot, flags, fd, offset);
if (ret == MAP_FAILED)
die_errno("Out of memory? mmap failed");
@@ -954,7 +947,7 @@ unsigned char *use_pack(struct packed_git *p,
win->len = (size_t)len;
pack_mapped += win->len;
while (packed_git_limit < pack_mapped
- && unuse_one_window(p, p->pack_fd))
+ && unuse_one_window(p))
; /* nothing */
win->base = xmmap(NULL, win->len,
PROT_READ, MAP_PRIVATE,
@@ -1000,7 +993,7 @@ static struct packed_git *alloc_packed_git(int extra)
static void try_to_free_pack_memory(size_t size)
{
- release_pack_memory(size, -1);
+ release_pack_memory(size);
}
struct packed_git *add_packed_git(const char *path, int path_len, int local)
--
1.8.4.rc0.2.g6cf5c31
-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
next prev parent reply other threads:[~2013-07-31 19:52 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-07-30 4:05 [PATCH] sha1_file: introduce close_one_pack() to close packs on fd pressure Brandon Casey
2013-07-30 7:51 ` Eric Sunshine
2013-07-30 15:39 ` Junio C Hamano
2013-07-30 19:52 ` Jeff King
2013-07-30 22:59 ` Brandon Casey
2013-07-31 19:51 ` [PATCH v2 1/2] " Brandon Casey
2013-07-31 19:51 ` Brandon Casey [this message]
2013-07-31 21:08 ` [PATCH 2/2] Don't close pack fd when free'ing pack windows Antoine Pelisse
2013-07-31 21:21 ` Fredrik Gustafsson
2013-07-31 21:31 ` Brandon Casey
2013-07-31 21:44 ` Fredrik Gustafsson
2013-07-31 21:23 ` Brandon Casey
2013-07-31 21:28 ` Thomas Rast
2013-08-01 17:12 ` [PATCH v2 1/2] sha1_file: introduce close_one_pack() to close packs on fd pressure Junio C Hamano
2013-08-01 18:01 ` Brandon Casey
2013-08-01 18:39 ` Junio C Hamano
2013-08-01 19:16 ` Brandon Casey
2013-08-01 19:23 ` Brandon Casey
2013-08-01 20:02 ` Junio C Hamano
2013-08-01 20:37 ` Brandon Casey
2013-08-02 5:36 ` [PATCH v3] " Brandon Casey
2013-08-02 16:26 ` Junio C Hamano
2013-08-02 17:12 ` Brandon Casey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1375300297-6744-2-git-send-email-bcasey@nvidia.com \
--to=bcasey@nvidia.com \
--cc=drafnel@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=spearce@spearce.org \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).