From: Michael Haggerty <mhagger@alum.mit.edu>
To: Junio C Hamano <gitster@pobox.com>
Cc: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>,
"Stefan Beller" <sbeller@google.com>, "Jeff King" <peff@peff.net>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Brandon Williams" <bmwill@google.com>,
git@vger.kernel.org, "Michael Haggerty" <mhagger@alum.mit.edu>
Subject: [PATCH 05/20] read_packed_refs(): use mmap to read the `packed-refs` file
Date: Wed, 13 Sep 2017 19:15:59 +0200 [thread overview]
Message-ID: <67c4bf861beb095a63e8db72f1d37d6bfac39cd0.1505319366.git.mhagger@alum.mit.edu> (raw)
In-Reply-To: <cover.1505319366.git.mhagger@alum.mit.edu>
It's still done in a pretty stupid way, involving more data copying
than necessary. That will improve in future commits.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
refs/packed-backend.c | 42 ++++++++++++++++++++++++++++++++----------
1 file changed, 32 insertions(+), 10 deletions(-)
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 5c50c223ef..154abbd83a 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -215,8 +215,12 @@ static NORETURN void die_invalid_line(const char *path,
*/
static struct packed_ref_cache *read_packed_refs(struct packed_ref_store *refs)
{
- FILE *f;
struct packed_ref_cache *packed_refs = xcalloc(1, sizeof(*packed_refs));
+ int fd;
+ struct stat st;
+ size_t size;
+ char *buf;
+ const char *pos, *eol, *eof;
struct ref_entry *last = NULL;
struct strbuf line = STRBUF_INIT;
enum { PEELED_NONE, PEELED_TAGS, PEELED_FULLY } peeled = PEELED_NONE;
@@ -227,8 +231,8 @@ static struct packed_ref_cache *read_packed_refs(struct packed_ref_store *refs)
packed_refs->cache = create_ref_cache(NULL, NULL);
packed_refs->cache->root->flag &= ~REF_INCOMPLETE;
- f = fopen(refs->path, "r");
- if (!f) {
+ fd = open(refs->path, O_RDONLY);
+ if (fd < 0) {
if (errno == ENOENT) {
/*
* This is OK; it just means that no
@@ -241,16 +245,27 @@ static struct packed_ref_cache *read_packed_refs(struct packed_ref_store *refs)
}
}
- stat_validity_update(&packed_refs->validity, fileno(f));
+ stat_validity_update(&packed_refs->validity, fd);
+
+ if (fstat(fd, &st) < 0)
+ die_errno("couldn't stat %s", refs->path);
+
+ size = xsize_t(st.st_size);
+ buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
+ pos = buf;
+ eof = buf + size;
dir = get_ref_dir(packed_refs->cache->root);
- while (strbuf_getwholeline(&line, f, '\n') != EOF) {
+ while (pos < eof) {
struct object_id oid;
const char *refname;
const char *traits;
- if (!line.len || line.buf[line.len - 1] != '\n')
- die_unterminated_line(refs->path, line.buf, line.len);
+ eol = memchr(pos, '\n', eof - pos);
+ if (!eol)
+ die_unterminated_line(refs->path, pos, eof - pos);
+
+ strbuf_add(&line, pos, eol + 1 - pos);
if (skip_prefix(line.buf, "# pack-refs with:", &traits)) {
if (strstr(traits, " fully-peeled "))
@@ -258,7 +273,7 @@ static struct packed_ref_cache *read_packed_refs(struct packed_ref_store *refs)
else if (strstr(traits, " peeled "))
peeled = PEELED_TAGS;
/* perhaps other traits later as well */
- continue;
+ goto next_line;
}
refname = parse_ref_line(&line, &oid);
@@ -291,11 +306,18 @@ static struct packed_ref_cache *read_packed_refs(struct packed_ref_store *refs)
} else {
die_invalid_line(refs->path, line.buf, line.len);
}
+
+ next_line:
+ /* The "+ 1" is for the LF character. */
+ pos = eol + 1;
+ strbuf_reset(&line);
}
- fclose(f);
- strbuf_release(&line);
+ if (munmap(buf, size))
+ die_errno("error ummapping packed-refs file");
+ close(fd);
+ strbuf_release(&line);
return packed_refs;
}
--
2.14.1
next prev parent reply other threads:[~2017-09-13 17:16 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-13 17:15 [PATCH 00/20] Read `packed-refs` using mmap() Michael Haggerty
2017-09-13 17:15 ` [PATCH 01/20] ref_iterator: keep track of whether the iterator output is ordered Michael Haggerty
2017-09-13 17:15 ` [PATCH 02/20] prefix_ref_iterator: break when we leave the prefix Michael Haggerty
2017-09-13 17:15 ` [PATCH 03/20] packed_ref_cache: add a backlink to the associated `packed_ref_store` Michael Haggerty
2017-09-13 17:15 ` [PATCH 04/20] die_unterminated_line(), die_invalid_line(): new functions Michael Haggerty
2017-09-13 17:15 ` Michael Haggerty [this message]
2017-09-13 17:16 ` [PATCH 06/20] read_packed_refs(): only check for a header at the top of the file Michael Haggerty
2017-09-13 17:16 ` [PATCH 07/20] read_packed_refs(): make parsing of the header line more robust Michael Haggerty
2017-09-13 17:16 ` [PATCH 08/20] read_packed_refs(): read references with minimal copying Michael Haggerty
2017-09-13 17:16 ` [PATCH 09/20] packed_ref_cache: remember the file-wide peeling state Michael Haggerty
2017-09-13 17:16 ` [PATCH 10/20] mmapped_ref_iterator: add iterator over a packed-refs file Michael Haggerty
2017-09-13 17:16 ` [PATCH 11/20] mmapped_ref_iterator_advance(): no peeled value for broken refs Michael Haggerty
2017-09-13 17:16 ` [PATCH 12/20] packed_ref_cache: keep the `packed-refs` file open and mmapped Michael Haggerty
2017-09-13 17:16 ` [PATCH 13/20] read_packed_refs(): ensure that references are ordered when read Michael Haggerty
2017-09-13 17:16 ` [PATCH 14/20] packed_ref_iterator_begin(): iterate using `mmapped_ref_iterator` Michael Haggerty
2017-09-13 17:16 ` [PATCH 15/20] packed_read_raw_ref(): read the reference from the mmapped buffer Michael Haggerty
2017-09-13 17:16 ` [PATCH 16/20] ref_store: implement `refs_peel_ref()` generically Michael Haggerty
2017-09-13 17:16 ` [PATCH 17/20] packed_ref_store: get rid of the `ref_cache` entirely Michael Haggerty
2017-09-13 17:16 ` [PATCH 18/20] ref_cache: remove support for storing peeled values Michael Haggerty
2017-09-13 17:16 ` [PATCH 19/20] mmapped_ref_iterator: inline into `packed_ref_iterator` Michael Haggerty
2017-09-13 17:16 ` [PATCH 20/20] packed-backend.c: rename a bunch of things and update comments Michael Haggerty
2017-09-13 23:00 ` Stefan Beller
2017-09-13 21:35 ` [PATCH 00/20] Read `packed-refs` using mmap() Junio C Hamano
2017-09-14 16:12 ` Michael Haggerty
2017-09-14 22:16 ` Johannes Schindelin
2017-09-14 20:23 ` Johannes Schindelin
2017-09-15 4:21 ` Michael Haggerty
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=67c4bf861beb095a63e8db72f1d37d6bfac39cd0.1505319366.git.mhagger@alum.mit.edu \
--to=mhagger@alum.mit.edu \
--cc=avarab@gmail.com \
--cc=bmwill@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
--cc=sbeller@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).