From: Duy Nguyen <pclouds@gmail.com>
To: "SZEDER Gábor" <szeder.dev@gmail.com>
Cc: "Git Mailing List" <git@vger.kernel.org>,
"Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
"Derrick Stolee" <stolee@gmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>
Subject: Re: [PATCH v2 05/10] split-index.c: dump "link" extension as json
Date: Fri, 5 Jul 2019 06:54:49 +0700 [thread overview]
Message-ID: <CACsJy8CZZAkcuN_hqp6YmMkhKs0ON6b-+Cyo+Q+Jk4zFh0Ve7w@mail.gmail.com> (raw)
In-Reply-To: <20190704200133.GD20404@szeder.dev>
On Fri, Jul 5, 2019 at 3:01 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>
> On Mon, Jun 24, 2019 at 08:02:21PM +0700, Nguyễn Thái Ngọc Duy wrote:
> > diff --git a/t/t3011-ls-files-json.sh b/t/t3011-ls-files-json.sh
> > index 082fe8e966..dbb572ce9d 100755
> > --- a/t/t3011-ls-files-json.sh
> > +++ b/t/t3011-ls-files-json.sh
> > @@ -44,4 +44,18 @@ test_expect_success 'ls-files --json, main entries, UNTR and TREE' '
> > compare_json basic
> > '
> >
> > +test_expect_success 'ls-files --json, split index' '
> > + git init split &&
> > + (
> > + cd split &&
> > + echo one >one &&
> > + git add one &&
> > + git update-index --split-index &&
> > + echo updated >>one &&
> > + test_must_fail git -c splitIndex.maxPercentChange=100 update-index --refresh &&
> > + cp ../filter.sed . &&
> > + compare_json split-index
> > + )
> > +'
> > +
> > test_done
> > diff --git a/t/t3011/split-index b/t/t3011/split-index
> > new file mode 100644
> > index 0000000000..cdcc4ddded
> > --- /dev/null
> > +++ b/t/t3011/split-index
> > @@ -0,0 +1,39 @@
> > +{
> > + "version": 2,
> > + "oid": <string>,
> > + "mtime_sec": <number>,
> > + "mtime_nsec": <number>,
> > + "entries": [
> > + {
> > + "id": 0,
> > + "name": "",
> > + "mode": "100644",
> > + "flags": 0,
> > + "oid": <string>,
> > + "stat": {
> > + "ctime_sec": <number>,
> > + "ctime_nsec": <number>,
> > + "mtime_sec": <number>,
> > + "mtime_nsec": <number>,
> > + "device": <number>,
> > + "inode": <number>,
> > + "uid": <number>,
> > + "gid": <number>,
> > + "size": 4
> > + },
> > + "file_offset": <number>
> > + }
> > + ],
> > + "extensions": {
> > + "link": {
> > + "file_offset": <number>,
> > + "ext_size": <number>,
> > + "oid": <string>,
> > + "delete_bitmap": [
> > + ],
> > + "replace_bitmap": [
> > + 0
> > + ]
> > + }
> > + }
> > +}
>
> This test is flaky, as reported in:
>
> https://public-inbox.org/git/xmqqftno2mku.fsf@gitster-ct.c.googlers.com/
>
> This is because it relies on racy behaviour, namely that the following
> three commands
>
> echo one >one &&
> git add one &&
> git update-index --split-index &&
>
> are executed within the same second, leaving 'one' racily clean. To
> deal with the racily clean file, 5581a019ba (split-index: smudge and
> add racily clean cache entries to split index, 2018-10-11) kicks in,
> and 'one's smudged index entry is stored both in the shared index and
> in the split index. That's why this test expects the offset 0 in the
> "replace_bitmap" array.
>
> However, it's possible that a second boundary is crossed between
> writing to 'one' and splitting the index, and then 'one' is not racily
> clean, and its index entry is only stored in the shared index.
> Consequently, there are no index entries in the split index, so the
> "replace_bitmap" array ends up being empty, ultimately failing the
> test.
Yep. I came up with the same conclusion. But I still have a couple
other things to update before resending.
> A 'test-tool chmtime' invocation or two could make the test
> deterministic (i.e it could make sure that 'one' is either always
> racily clean or it never is, whichever is preferred).
>
> What I still don't understand, however, is that when the test fails
> this way, then the "entries" array ends up being empty as well. It
> looks as if the JSON dump included only index entries that were
> actually stored in '.git/index', but omitted entries that were only
> present in the shared index. I think this is wrong, and it should
> dump the unified view of the split and shared indexes. Or include all
> entries from the shared index as well. Or perhaps I'm completely
> missing something...
The command is to dump .git/index, not the shared one. And since this
is not a split index test, rather a (quite low-level) json dump test,
I did not bother to also dump the shared index, which should look like
a regular one. Producing a unified view in json might not be easy with
the current code because it's tied to file reading code, nearly stream
out json as we read the file, and split-index requires a post
processing step. I could contribute a python script or something to
combine shared/main index together. That way you can still see the
combined one, but we don't have to add/maintain more C code.
--
Duy
next prev parent reply other threads:[~2019-07-04 23:55 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-24 13:02 [PATCH v2 00/10] Add 'ls-files --debug-json' to dump the index in json Nguyễn Thái Ngọc Duy
2019-06-24 13:02 ` [PATCH v2 01/10] ls-files: add --json to dump the index Nguyễn Thái Ngọc Duy
2019-06-24 19:15 ` Jeff Hostetler
2019-06-24 20:04 ` Junio C Hamano
2019-06-25 9:21 ` Johannes Schindelin
2019-06-25 9:52 ` Duy Nguyen
2019-06-25 15:37 ` Jeff Hostetler
2019-06-25 9:05 ` Thomas Gummerer
2019-06-25 9:44 ` Johannes Schindelin
2019-06-25 11:31 ` Johannes Schindelin
2019-06-25 13:57 ` Johannes Schindelin
2019-06-25 22:28 ` Junio C Hamano
2019-06-26 19:51 ` Junio C Hamano
2019-06-24 13:02 ` [PATCH v2 02/10] read-cache.c: dump common extension info in json Nguyễn Thái Ngọc Duy
2019-06-24 13:02 ` [PATCH v2 03/10] cache-tree.c: dump "TREE" extension as json Nguyễn Thái Ngọc Duy
2019-06-24 13:02 ` [PATCH v2 04/10] dir.c: dump "UNTR" " Nguyễn Thái Ngọc Duy
2019-06-24 19:32 ` Jeff Hostetler
2019-06-24 13:02 ` [PATCH v2 05/10] split-index.c: dump "link" " Nguyễn Thái Ngọc Duy
2019-06-24 20:06 ` Jeff Hostetler
2019-06-25 10:29 ` Duy Nguyen
2019-06-25 12:40 ` Derrick Stolee
2019-06-27 10:48 ` Duy Nguyen
2019-06-27 13:24 ` Jeff Hostetler
2019-06-27 13:42 ` Derrick Stolee
2019-06-27 13:47 ` Duy Nguyen
2019-07-03 9:08 ` SZEDER Gábor
2019-07-04 20:01 ` SZEDER Gábor
2019-07-04 23:54 ` Duy Nguyen [this message]
2019-07-08 17:58 ` Junio C Hamano
2019-06-24 13:02 ` [PATCH v2 06/10] fsmonitor.c: dump "FSMN" " Nguyễn Thái Ngọc Duy
2019-06-24 13:02 ` [PATCH v2 07/10] resolve-undo.c: dump "REUC" " Nguyễn Thái Ngọc Duy
2019-06-24 13:02 ` [PATCH v2 08/10] read-cache.c: dump "EOIE" " Nguyễn Thái Ngọc Duy
2019-06-24 13:02 ` [PATCH v2 09/10] read-cache.c: dump "IEOT" " Nguyễn Thái Ngọc Duy
2019-06-24 13:02 ` [PATCH v2 10/10] t3008: use the new SINGLE_CPU prereq Nguyễn Thái Ngọc Duy
2019-06-24 18:00 ` [PATCH v2 00/10] Add 'ls-files --debug-json' to dump the index in json Johannes Schindelin
2019-06-24 18:39 ` Jeff Hostetler
2019-06-25 9:05 ` Duy Nguyen
2019-06-25 9:38 ` Thomas Gummerer
2019-06-25 11:27 ` Johannes Schindelin
2019-06-25 12:06 ` Duy Nguyen
2019-06-25 14:10 ` Johannes Schindelin
2019-06-25 17:08 ` Ramsay Jones
2019-06-26 15:05 ` Johannes Schindelin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACsJy8CZZAkcuN_hqp6YmMkhKs0ON6b-+Cyo+Q+Jk4zFh0Ve7w@mail.gmail.com \
--to=pclouds@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=stolee@gmail.com \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).