From: Junio C Hamano <gitster@pobox.com>
To: Jeff King <peff@peff.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
git@vger.kernel.org, Lars Schneider <larsxschneider@gmail.com>,
Eric Wong <e@80x24.org>,
Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: Re: [PATCH v3 2/3] sha1_file: open window into packfiles with O_CLOEXEC
Date: Thu, 27 Oct 2016 14:49:12 -0700 [thread overview]
Message-ID: <xmqq60od5up3.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <20161027102419.dbzigj7wtr355ofh@sigill.intra.peff.net> (Jeff King's message of "Thu, 27 Oct 2016 06:24:19 -0400")
Jeff King <peff@peff.net> writes:
> +cc Linus as the original author of 144bde78e9 in case there is
> something subtle I'm missing, but this really just seems like it's
> an outdated optimization.
>
> -- >8 --
> Subject: [PATCH] sha1_file: stop opening files with O_NOATIME
>
> When we open object files, we try to do so with O_NOATIME.
> This dates back to 144bde78e9 (Use O_NOATIME when opening
> the sha1 files., 2005-04-23), which is an optimization to
> avoid creating a bunch of dirty inodes when we're accessing
> many objects. But a few things have changed since then:
>
> 1. In June 2005, git learned about packfiles, which means
> we would do a lot fewer atime updates (rather than one
> per object access, we'd generally get one per packfile).
>
> 2. In late 2006, Linux learned about "relatime", which is
> generally the default on modern installs. So
> performance around atimes updates is a non-issue there
> these days.
>
> All the world isn't Linux, but as it turns out, Linux
> is the only platform to implement O_NOATIME in the
> first place.
>
> So it's very unlikely that this code is helping anybody
> these days.
>
> It's not a particularly large amount of code, but the
> fallback-retry creates complexity. E.g., we do a similar
> fallback for CLOEXEC; which one should take precedence, or
> should we try all possible combinations? Dropping O_NOATIME
> makes those questions go away.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
We may want to lose the surrounding for (;;) loop as there is only
one flag to retry without, which was the original code structure
back when 144bde78e9 ("Use O_NOATIME when opening the sha1 files.",
2005-04-23) was written and refactored by 44d1c19ee8 ("Make loose
object file reading more careful", 2008-06-14).
IOW, this on top. The update to ce_compare_data() Lars has in
a0a6cb9662 ("read-cache: make sure file handles are not inherited by
child processes", 2016-10-24) could then made into a call to
git_open().
sha1_file.c | 23 ++++++++---------------
1 file changed, 8 insertions(+), 15 deletions(-)
diff --git a/sha1_file.c b/sha1_file.c
index 6f02a57d8b..e18ea053e6 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1553,24 +1553,17 @@ int check_sha1_signature(const unsigned char *sha1, void *map,
int git_open(const char *name)
{
- static int sha1_file_open_flag = O_CLOEXEC;
-
- for (;;) {
- int fd;
-
- errno = 0;
- fd = open(name, O_RDONLY | sha1_file_open_flag);
- if (fd >= 0)
- return fd;
+ static int cloexec = O_CLOEXEC;
+ int fd;
+ errno = 0;
+ fd = open(name, O_RDONLY | cloexec);
+ if ((cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) {
/* Try again w/o O_CLOEXEC: the kernel might not support it */
- if ((sha1_file_open_flag & O_CLOEXEC) && errno == EINVAL) {
- sha1_file_open_flag &= ~O_CLOEXEC;
- continue;
- }
-
- return -1;
+ cloexec &= ~O_CLOEXEC;
+ fd = open(name, O_RDONLY | cloexec);
}
+ return fd;
}
static int stat_sha1_file(const unsigned char *sha1, struct stat *st)
next prev parent reply other threads:[~2016-10-27 21:49 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-24 18:02 [PATCH v2 0/2] Use CLOEXEC to avoid fd leaks larsxschneider
2016-10-24 18:02 ` [PATCH v2 1/2] sha1_file: open window into packfiles with CLOEXEC larsxschneider
2016-10-25 10:27 ` Johannes Schindelin
2016-10-25 16:58 ` Junio C Hamano
2016-10-24 18:03 ` [PATCH v2 2/2] read-cache: make sure file handles are not inherited by child processes larsxschneider
2016-10-24 18:39 ` Eric Wong
2016-10-24 19:53 ` Junio C Hamano
2016-10-25 10:33 ` Johannes Schindelin
2016-10-25 17:02 ` Junio C Hamano
2016-10-24 19:22 ` Johannes Sixt
2016-10-24 19:53 ` Lars Schneider
2016-10-25 21:39 ` Johannes Sixt
2016-10-24 18:23 ` [PATCH v2 0/2] Use CLOEXEC to avoid fd leaks Junio C Hamano
2016-10-25 11:27 ` Johannes Schindelin
2016-10-25 18:16 ` [PATCH v3 0/3] quick reroll of Lars's git_open() w/ O_CLOEXEC Junio C Hamano
2016-10-25 18:16 ` [PATCH v3 1/3] sha1_file: rename git_open_noatime() to git_open() Junio C Hamano
2016-10-25 18:16 ` [PATCH v3 2/3] sha1_file: open window into packfiles with O_CLOEXEC Junio C Hamano
2016-10-26 4:25 ` Jeff King
2016-10-26 16:23 ` Junio C Hamano
2016-10-26 16:47 ` Jeff King
2016-10-26 17:52 ` Junio C Hamano
2016-10-26 20:17 ` Jeff King
2016-10-26 21:15 ` Junio C Hamano
2016-10-27 10:24 ` Jeff King
2016-10-27 21:49 ` Junio C Hamano [this message]
2016-10-27 22:38 ` Linus Torvalds
2016-10-27 22:56 ` Junio C Hamano
2016-10-27 23:09 ` Linus Torvalds
2016-10-27 23:19 ` Linus Torvalds
2016-10-27 23:36 ` Junio C Hamano
2016-10-27 23:44 ` Linus Torvalds
2016-10-28 1:08 ` Junio C Hamano
2016-10-28 2:37 ` Junio C Hamano
2016-10-28 5:51 ` Eric Wong
2016-10-28 11:11 ` Johannes Schindelin
2016-10-28 16:13 ` Linus Torvalds
2016-10-28 16:48 ` Junio C Hamano
2016-10-28 17:38 ` Linus Torvalds
2016-10-28 17:47 ` Junio C Hamano
2016-10-29 1:26 ` Junio C Hamano
2016-10-29 8:25 ` Johannes Schindelin
2016-10-29 17:06 ` Linus Torvalds
2016-10-31 17:37 ` Junio C Hamano
2016-10-31 13:56 ` Jeff King
2016-10-31 17:55 ` Junio C Hamano
2016-10-31 18:05 ` Jeff King
2016-10-28 13:32 ` Junio C Hamano
2016-10-28 13:33 ` Junio C Hamano
2016-10-28 7:51 ` Jeff King
2016-10-25 18:16 ` [PATCH v3 3/3] read-cache: make sure file handles are not inherited by child processes Junio C Hamano
2016-10-25 21:33 ` Eric Wong
2016-10-25 22:54 ` Junio C Hamano
2016-10-25 21:48 ` [PATCH v3 0/3] quick reroll of Lars's git_open() w/ O_CLOEXEC Lars Schneider
2016-10-25 22:56 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqq60od5up3.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox.com \
--cc=e@80x24.org \
--cc=git@vger.kernel.org \
--cc=johannes.schindelin@gmx.de \
--cc=larsxschneider@gmail.com \
--cc=peff@peff.net \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).