From: Ben Peart <peartben@gmail.com> To: Jonathan Nieder <jrnieder@gmail.com>, Junio C Hamano <gitster@pobox.com> Cc: git@vger.kernel.org, pclouds@gmail.com, Ben Peart <benpeart@microsoft.com>, jonathantanmy@google.com Subject: Re: [PATCH 1/5] eoie: default to not writing EOIE section Date: Tue, 20 Nov 2018 08:06:16 -0500 Message-ID: <efa1d7fb-1da3-c093-1cb1-873a2e1c445c@gmail.com> (raw) In-Reply-To: <20181120061147.GB144753@google.com> On 11/20/2018 1:11 AM, Jonathan Nieder wrote: > Since 3b1d9e04 (eoie: add End of Index Entry (EOIE) extension, > 2018-10-10) Git defaults to writing the new EOIE section when writing > out an index file. Usually that is a good thing because it improves > threaded performance, but when a Git repository is shared with older > versions of Git, it produces a confusing warning: > > $ git status > ignoring EOIE extension > HEAD detached at 371ed0defa > nothing to commit, working tree clean > > Let's introduce the new index extension more gently. First we'll roll > out the new version of Git that understands it, and then once > sufficiently many users are using such a version, we can flip the > default to writing it by default. > > Introduce a '[index] recordEndOfIndexEntries' configuration variable > to allow interested users to benefit from this index extension early. > > Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> > --- > Rebased. No other change from v1. > > As Jonathan pointed out, it would be nice to have tests here. Ben, > any advice for how I could write some in a followup change? E.g. does > Derrick Stolee's tracing based testing trick apply here? > > Documentation/config/index.txt | 7 +++++++ > read-cache.c | 11 ++++++++++- > t/t1700-split-index.sh | 11 +++++++---- > 3 files changed, 24 insertions(+), 5 deletions(-) > > diff --git a/Documentation/config/index.txt b/Documentation/config/index.txt > index 4b94b6bedc..8e138aba7a 100644 > --- a/Documentation/config/index.txt > +++ b/Documentation/config/index.txt > @@ -1,3 +1,10 @@ > +index.recordEndOfIndexEntries:: > + Specifies whether the index file should include an "End Of Index > + Entry" section. This reduces index load time on multiprocessor > + machines but produces a message "ignoring EOIE extension" when > + reading the index using Git versions before 2.20. Defaults to > + 'false'. > + > index.threads:: > Specifies the number of threads to spawn when loading the index. > This is meant to reduce index load time on multiprocessor machines. > diff --git a/read-cache.c b/read-cache.c > index 4ca81286c0..1e9c772603 100644 > --- a/read-cache.c > +++ b/read-cache.c > @@ -2689,6 +2689,15 @@ void update_index_if_able(struct index_state *istate, struct lock_file *lockfile > rollback_lock_file(lockfile); > } > > +static int record_eoie(void) > +{ > + int val; I believe you are going to want to initialize val to 0 here as it is on the stack so is not guaranteed to be zero. > + > + if (!git_config_get_bool("index.recordendofindexentries", &val)) > + return val; > + return 0; > +} > + > /* > * On success, `tempfile` is closed. If it is the temporary file > * of a `struct lock_file`, we will therefore effectively perform > @@ -2936,7 +2945,7 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile, > * read. Write it out regardless of the strip_extensions parameter as we need it > * when loading the shared index. > */ > - if (offset) { > + if (offset && record_eoie()) { > struct strbuf sb = STRBUF_INIT; > > write_eoie_extension(&sb, &eoie_c, offset); > diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh > index 2ac47aa0e4..0cbac64e28 100755 > --- a/t/t1700-split-index.sh > +++ b/t/t1700-split-index.sh > @@ -25,14 +25,17 @@ test_expect_success 'enable split index' ' > git update-index --split-index && > test-tool dump-split-index .git/index >actual && > indexversion=$(test-tool index-version <.git/index) && > + > + # NEEDSWORK: Stop hard-coding checksums. > if test "$indexversion" = "4" > then > - own=3527df833c6c100d3d1d921a9a782d62a8be4b58 > - base=746f7ab2ed44fb839efdfbffcf399d0b113fb4cb > + own=432ef4b63f32193984f339431fd50ca796493569 > + base=508851a7f0dfa8691e9f69c7f055865389012491 > else > - own=5e9b60117ece18da410ddecc8b8d43766a0e4204 > - base=4370042739b31cd17a5c5cd6043a77c9a00df113 > + own=8299b0bcd1ac364e5f1d7768efb62fa2da79a339 > + base=39d890139ee5356c7ef572216cebcd27aa41f9df > fi && > + > cat >expect <<-EOF && > own $own > base $base >
next prev parent reply other threads:[~2018-11-20 13:06 UTC|newest] Thread overview: 199+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-08-23 15:41 [PATCH v1] read-cache: speed up index load through parallelization Ben Peart 2018-08-23 17:31 ` Stefan Beller 2018-08-23 19:44 ` Ben Peart 2018-08-24 18:40 ` Duy Nguyen 2018-08-28 14:53 ` Ben Peart 2018-08-23 18:06 ` Junio C Hamano 2018-08-23 20:33 ` Ben Peart 2018-08-24 15:37 ` Duy Nguyen 2018-08-24 15:57 ` Duy Nguyen 2018-08-24 17:28 ` Ben Peart 2018-08-25 6:44 ` [PATCH] read-cache.c: optimize reading index format v4 Nguyễn Thái Ngọc Duy 2018-08-27 19:36 ` Junio C Hamano 2018-08-28 19:25 ` Duy Nguyen 2018-08-28 23:54 ` Ben Peart 2018-08-29 17:14 ` Junio C Hamano 2018-09-04 16:08 ` Duy Nguyen 2018-09-02 13:19 ` [PATCH v2 0/1] " Nguyễn Thái Ngọc Duy 2018-09-02 13:19 ` [PATCH v2 1/1] read-cache.c: " Nguyễn Thái Ngọc Duy 2018-09-04 18:58 ` Junio C Hamano 2018-09-04 19:31 ` Junio C Hamano 2018-08-24 18:20 ` [PATCH v1] read-cache: speed up index load through parallelization Duy Nguyen 2018-08-24 18:40 ` Ben Peart 2018-08-24 19:00 ` Duy Nguyen 2018-08-24 19:57 ` Ben Peart 2018-08-29 15:25 ` [PATCH v2 0/3] " Ben Peart 2018-08-29 15:25 ` [PATCH v2 1/3] " Ben Peart 2018-08-29 17:14 ` Junio C Hamano 2018-08-29 21:35 ` Ben Peart 2018-09-03 19:16 ` Duy Nguyen 2018-08-29 15:25 ` [PATCH v2 2/3] read-cache: load cache extensions on worker thread Ben Peart 2018-08-29 17:12 ` Junio C Hamano 2018-08-29 21:42 ` Ben Peart 2018-08-29 22:19 ` Junio C Hamano 2018-09-03 19:21 ` Duy Nguyen 2018-09-03 19:27 ` Duy Nguyen 2018-08-29 15:25 ` [PATCH v2 3/3] read-cache: micro-optimize expand_name_field() to speed up V4 index parsing Ben Peart 2018-09-06 21:03 ` [PATCH v3 0/4] read-cache: speed up index load through parallelization Ben Peart 2018-09-06 21:03 ` [PATCH v3 1/4] read-cache: optimize expand_name_field() to speed up V4 index parsing Ben Peart 2018-09-06 21:03 ` [PATCH v3 2/4] eoie: add End of Index Entry (EOIE) extension Ben Peart 2018-09-07 17:55 ` Junio C Hamano 2018-09-07 20:23 ` Ben Peart 2018-09-08 6:29 ` Martin Ågren 2018-09-08 14:03 ` Ben Peart 2018-09-08 17:08 ` Martin Ågren 2018-09-06 21:03 ` [PATCH v3 3/4] read-cache: load cache extensions on a worker thread Ben Peart 2018-09-07 21:10 ` Junio C Hamano 2018-09-08 14:56 ` Ben Peart 2018-09-06 21:03 ` [PATCH v3 4/4] read-cache: speed up index load through parallelization Ben Peart 2018-09-07 4:16 ` Torsten Bögershausen 2018-09-07 13:43 ` Ben Peart 2018-09-07 17:21 ` [PATCH v3 0/4] " Junio C Hamano 2018-09-07 18:31 ` Ben Peart 2018-09-08 13:18 ` Duy Nguyen 2018-09-11 23:26 ` [PATCH v4 0/5] " Ben Peart 2018-09-11 23:26 ` [PATCH v4 1/5] eoie: add End of Index Entry (EOIE) extension Ben Peart 2018-09-11 23:26 ` [PATCH v4 2/5] read-cache: load cache extensions on a worker thread Ben Peart 2018-09-11 23:26 ` [PATCH v4 3/5] read-cache: speed up index load through parallelization Ben Peart 2018-09-11 23:26 ` [PATCH v4 4/5] read-cache.c: optimize reading index format v4 Ben Peart 2018-09-11 23:26 ` [PATCH v4 5/5] read-cache: clean up casting and byte decoding Ben Peart 2018-09-12 14:34 ` [PATCH v4 0/5] read-cache: speed up index load through parallelization Ben Peart 2018-09-12 16:18 ` [PATCH v5 " Ben Peart 2018-09-12 16:18 ` [PATCH v5 1/5] eoie: add End of Index Entry (EOIE) extension Ben Peart 2018-09-13 22:44 ` Junio C Hamano 2018-09-15 10:02 ` Duy Nguyen 2018-09-17 14:54 ` Ben Peart 2018-09-17 16:05 ` Duy Nguyen 2018-09-17 17:31 ` Junio C Hamano 2018-09-17 17:38 ` Duy Nguyen 2018-09-17 19:08 ` Junio C Hamano 2018-09-12 16:18 ` [PATCH v5 2/5] read-cache: load cache extensions on a worker thread Ben Peart 2018-09-15 10:22 ` Duy Nguyen 2018-09-15 10:24 ` Duy Nguyen 2018-09-17 16:38 ` Ben Peart 2018-09-15 16:23 ` Duy Nguyen 2018-09-17 17:19 ` Junio C Hamano 2018-09-17 16:26 ` Ben Peart 2018-09-17 16:45 ` Duy Nguyen 2018-09-17 21:32 ` Junio C Hamano 2018-09-12 16:18 ` [PATCH v5 3/5] read-cache: load cache entries on worker threads Ben Peart 2018-09-15 10:31 ` Duy Nguyen 2018-09-17 17:25 ` Ben Peart 2018-09-15 11:07 ` Duy Nguyen 2018-09-15 11:09 ` Duy Nguyen 2018-09-17 18:52 ` Ben Peart 2018-09-15 11:29 ` Duy Nguyen 2018-09-12 16:18 ` [PATCH v5 4/5] read-cache.c: optimize reading index format v4 Ben Peart 2018-09-12 16:18 ` [PATCH v5 5/5] read-cache: clean up casting and byte decoding Ben Peart 2018-09-26 19:54 ` [PATCH v6 0/7] speed up index load through parallelization Ben Peart 2018-09-26 19:54 ` [PATCH v6 1/7] read-cache.c: optimize reading index format v4 Ben Peart 2018-09-26 19:54 ` [PATCH v6 2/7] read-cache: clean up casting and byte decoding Ben Peart 2018-09-26 19:54 ` [PATCH v6 3/7] eoie: add End of Index Entry (EOIE) extension Ben Peart 2018-09-28 0:19 ` SZEDER Gábor 2018-09-28 18:38 ` Ben Peart 2018-09-29 0:51 ` SZEDER Gábor 2018-09-29 5:45 ` Duy Nguyen 2018-09-29 18:24 ` Junio C Hamano 2018-09-26 19:54 ` [PATCH v6 4/7] config: add new index.threads config setting Ben Peart 2018-09-28 0:26 ` SZEDER Gábor 2018-09-28 13:39 ` Ben Peart 2018-09-28 17:07 ` Junio C Hamano 2018-09-28 19:41 ` Ben Peart 2018-09-28 20:30 ` Ramsay Jones 2018-09-28 22:15 ` Junio C Hamano 2018-10-01 13:17 ` Ben Peart 2018-10-01 15:06 ` SZEDER Gábor 2018-09-26 19:54 ` [PATCH v6 5/7] read-cache: load cache extensions on a worker thread Ben Peart 2018-09-26 19:54 ` [PATCH v6 6/7] ieot: add Index Entry Offset Table (IEOT) extension Ben Peart 2018-09-26 19:54 ` [PATCH v6 7/7] read-cache: load cache entries on worker threads Ben Peart 2018-09-26 22:06 ` [PATCH v6 0/7] speed up index load through parallelization Junio C Hamano 2018-09-27 17:13 ` Duy Nguyen 2018-10-01 13:45 ` [PATCH v7 " Ben Peart 2018-10-01 13:45 ` [PATCH v7 1/7] read-cache.c: optimize reading index format v4 Ben Peart 2018-10-01 13:45 ` [PATCH v7 2/7] read-cache: clean up casting and byte decoding Ben Peart 2018-10-01 15:10 ` Duy Nguyen 2018-10-01 13:45 ` [PATCH v7 3/7] eoie: add End of Index Entry (EOIE) extension Ben Peart 2018-10-01 15:17 ` SZEDER Gábor 2018-10-02 14:34 ` Ben Peart 2018-10-01 15:30 ` Duy Nguyen 2018-10-02 15:13 ` Ben Peart 2018-10-01 13:45 ` [PATCH v7 4/7] config: add new index.threads config setting Ben Peart 2018-10-01 13:45 ` [PATCH v7 5/7] read-cache: load cache extensions on a worker thread Ben Peart 2018-10-01 15:50 ` Duy Nguyen 2018-10-02 15:00 ` Ben Peart 2018-10-01 13:45 ` [PATCH v7 6/7] ieot: add Index Entry Offset Table (IEOT) extension Ben Peart 2018-10-01 16:27 ` Duy Nguyen 2018-10-02 16:34 ` Ben Peart 2018-10-02 17:02 ` Duy Nguyen 2018-10-01 13:45 ` [PATCH v7 7/7] read-cache: load cache entries on worker threads Ben Peart 2018-10-01 17:09 ` Duy Nguyen 2018-10-02 19:09 ` Ben Peart 2018-10-10 15:59 ` [PATCH v8 0/7] speed up index load through parallelization Ben Peart 2018-10-10 15:59 ` [PATCH v8 1/7] read-cache.c: optimize reading index format v4 Ben Peart 2018-10-10 15:59 ` [PATCH v8 2/7] read-cache: clean up casting and byte decoding Ben Peart 2018-10-10 15:59 ` [PATCH v8 3/7] eoie: add End of Index Entry (EOIE) extension Ben Peart 2018-10-10 15:59 ` [PATCH v8 4/7] config: add new index.threads config setting Ben Peart 2018-10-10 15:59 ` [PATCH v8 5/7] read-cache: load cache extensions on a worker thread Ben Peart 2018-10-10 15:59 ` [PATCH v8 6/7] ieot: add Index Entry Offset Table (IEOT) extension Ben Peart 2018-10-10 15:59 ` [PATCH v8 7/7] read-cache: load cache entries on worker threads Ben Peart 2018-10-19 16:11 ` Jeff King 2018-10-22 2:14 ` Junio C Hamano 2018-10-22 14:40 ` Ben Peart 2018-10-12 3:18 ` [PATCH v8 0/7] speed up index load through parallelization Junio C Hamano 2018-10-14 12:28 ` Duy Nguyen 2018-10-15 17:33 ` Ben Peart 2018-11-13 0:38 ` [PATCH 0/3] Avoid confusing messages from new index extensions (Re: [PATCH v8 0/7] speed up index load through parallelization) Jonathan Nieder 2018-11-13 0:39 ` [PATCH 1/3] eoie: default to not writing EOIE section Jonathan Nieder 2018-11-13 1:05 ` Junio C Hamano 2018-11-13 15:14 ` Ben Peart 2018-11-13 18:25 ` Jonathan Nieder 2018-11-14 1:36 ` Junio C Hamano 2018-11-15 0:19 ` Jonathan Nieder 2018-11-13 0:39 ` [PATCH 2/3] ieot: default to not writing IEOT section Jonathan Nieder 2018-11-13 0:58 ` Jonathan Tan 2018-11-13 1:09 ` Junio C Hamano 2018-11-13 1:12 ` Jonathan Nieder 2018-11-13 15:37 ` Duy Nguyen 2018-11-13 18:09 ` Jonathan Nieder 2018-11-13 15:22 ` Ben Peart 2018-11-13 18:18 ` Jonathan Nieder 2018-11-13 19:15 ` Ben Peart 2018-11-13 21:08 ` Jonathan Nieder 2018-11-14 18:09 ` Ben Peart 2018-11-15 0:05 ` Jonathan Nieder 2018-11-14 3:05 ` Junio C Hamano 2018-11-20 6:09 ` [PATCH v2 0/5] Avoid confusing messages from new index extensions Jonathan Nieder 2018-11-20 6:11 ` [PATCH 1/5] eoie: default to not writing EOIE section Jonathan Nieder 2018-11-20 13:06 ` Ben Peart [this message] 2018-11-20 13:21 ` SZEDER Gábor 2018-11-21 16:46 ` Jeff King 2018-11-22 0:47 ` Junio C Hamano 2018-11-20 15:01 ` Ben Peart 2018-11-20 6:12 ` [PATCH 2/5] ieot: default to not writing IEOT section Jonathan Nieder 2018-11-20 13:07 ` Ben Peart 2018-11-26 19:59 ` Stefan Beller 2018-11-26 21:47 ` Ben Peart 2018-11-26 22:02 ` Stefan Beller 2018-11-27 0:50 ` Junio C Hamano 2018-11-20 6:12 ` [PATCH 3/5] index: do not warn about unrecognized extensions Jonathan Nieder 2018-11-20 6:14 ` [PATCH 4/5] index: make index.threads=true enable ieot and eoie Jonathan Nieder 2018-11-20 13:24 ` Ben Peart 2018-11-20 6:15 ` [PATCH 5/5] index: offer advice for unknown index extensions Jonathan Nieder 2018-11-20 9:26 ` Ævar Arnfjörð Bjarmason 2018-11-20 13:30 ` Ben Peart 2018-11-21 0:22 ` Junio C Hamano 2018-11-21 0:39 ` Jonathan Nieder 2018-11-21 0:44 ` Jonathan Nieder 2018-11-21 5:01 ` Junio C Hamano 2018-11-21 5:04 ` Jonathan Nieder 2018-11-21 5:15 ` Junio C Hamano 2018-11-21 5:31 ` Junio C Hamano 2018-11-21 1:03 ` Jonathan Nieder 2018-11-21 4:23 ` Junio C Hamano 2018-11-21 4:57 ` Jonathan Nieder 2018-11-21 9:30 ` Ævar Arnfjörð Bjarmason 2018-11-13 0:40 ` [PATCH 3/3] index: do not warn about unrecognized extensions Jonathan Nieder 2018-11-13 1:10 ` Junio C Hamano 2018-11-13 15:25 ` Ben Peart 2018-11-14 3:24 ` Junio C Hamano 2018-11-14 18:19 ` Ben Peart
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=efa1d7fb-1da3-c093-1cb1-873a2e1c445c@gmail.com \ --to=peartben@gmail.com \ --cc=benpeart@microsoft.com \ --cc=git@vger.kernel.org \ --cc=gitster@pobox.com \ --cc=jonathantanmy@google.com \ --cc=jrnieder@gmail.com \ --cc=pclouds@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
git@vger.kernel.org list mirror (unofficial, one of many) This inbox may be cloned and mirrored by anyone: git clone --mirror https://public-inbox.org/git git clone --mirror http://ou63pmih66umazou.onion/git git clone --mirror http://czquwvybam4bgbro.onion/git git clone --mirror http://hjrcffqmbrq6wope.onion/git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 git git/ https://public-inbox.org/git \ git@vger.kernel.org public-inbox-index git Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.version-control.git nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git nntp://news.gmane.io/gmane.comp.version-control.git note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/mirrors/git.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git