From: Johan Herland <johan@herland.net>
To: git@vger.kernel.org
Cc: Johan Herland <johan@herland.net>,
Johannes Schindelin <Johannes.Schindelin@gmx.de>,
"Brian M . Carlson" <sandals@crustytoothpaste.net>
Subject: [PATCH 1/2] t3305: check notes fanout more carefully and robustly
Date: Mon, 3 Feb 2020 22:04:44 +0100 [thread overview]
Message-ID: <20200203210445.2854-1-johan@herland.net> (raw)
In short, before this patch, this test script:
- creates many notes
- verifies that all notes in the notes tree has a fanout of 1
- removes most notes
- verifies that the notes in the notes tree now has a fanout of 0
The fanout verification only happened twice: after creating all the
notes, and after removing most of them.
This patch strengthens the test by checking the fanout after _each_
added/removed note: We assert that the switch from fanout 0 -> 1
happens exactly once while adding notes (and that the switch pervades
the entire notes tree). Likewise, we assert that the switch from
fanout 1 -> 0 happens exactly once while removing notes.
Additionally, we decrease the number of notes left after removal,
from 50 to 15 notes, in order to ensure that fanout 1 -> 0 transition
keeps happening regardless of external factors[1].
[1]: Currently (with the SHA1 hash function and the deterministic
object ids of the test environment) the fanout heuristic in the notes
code happens to switch from 0 -> 1 at 109 notes, and from 1 -> 0 at
59 notes. However, changing the hash function or other external
factors will vary these numbers, and the latter may - in theory - go
as low as 15. For more details, please see the discussion at
https://public-inbox.org/git/20200125230035.136348-4-sandals@crustytoothpaste.net/
Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Cc: Brian M. Carlson <sandals@crustytoothpaste.net>
Signed-off-by: Johan Herland <johan@herland.net>
---
t/t3305-notes-fanout.sh | 101 ++++++++++++++++++++++++++++++----------
1 file changed, 76 insertions(+), 25 deletions(-)
diff --git a/t/t3305-notes-fanout.sh b/t/t3305-notes-fanout.sh
index 831f83d211..402057c83a 100755
--- a/t/t3305-notes-fanout.sh
+++ b/t/t3305-notes-fanout.sh
@@ -4,6 +4,32 @@ test_description='Test that adding/removing many notes triggers automatic fanout
. ./test-lib.sh
+path_has_fanout() {
+ path=$1 &&
+ fanout=$2 &&
+ after_last_slash=$((40 - $fanout * 2)) &&
+ echo $path | grep -q "^\([0-9a-f]\{2\}/\)\{$fanout\}[0-9a-f]\{$after_last_slash\}$"
+}
+
+touched_one_note_with_fanout() {
+ notes_commit=$1 &&
+ modification=$2 && # 'A' for addition, 'D' for deletion
+ fanout=$3 &&
+ diff=$(git diff-tree --no-commit-id --name-status --root -r $notes_commit) &&
+ path=$(echo $diff | sed -e "s/^$modification[\t ]//") &&
+ path_has_fanout "$path" $fanout;
+}
+
+all_notes_have_fanout() {
+ notes_commit=$1 &&
+ fanout=$2 &&
+ git ls-tree -r --name-only $notes_commit 2>/dev/null |
+ while read path
+ do
+ path_has_fanout $path $fanout || return 1
+ done
+}
+
test_expect_success 'creating many notes with git-notes' '
num_notes=300 &&
i=0 &&
@@ -20,7 +46,7 @@ test_expect_success 'creating many notes with git-notes' '
test_expect_success 'many notes created correctly with git-notes' '
git log | grep "^ " > output &&
- i=300 &&
+ i=$num_notes &&
while test $i -gt 0
do
echo " commit #$i" &&
@@ -30,34 +56,46 @@ test_expect_success 'many notes created correctly with git-notes' '
test_cmp expect output
'
-test_expect_success 'many notes created with git-notes triggers fanout' '
- # Expect entire notes tree to have a fanout == 1
- git ls-tree -r --name-only refs/notes/commits |
- while read path
+test_expect_success 'stable fanout 0 is followed by stable fanout 1' '
+ i=$num_notes &&
+ fanout=0 &&
+ while test $i -gt 0
do
- echo $path | grep "^../[0-9a-f]*$" || {
- echo "Invalid path \"$path\"" &&
- return 1;
- }
- done
+ i=$(($i - 1)) &&
+ if touched_one_note_with_fanout refs/notes/commits~$i A $fanout
+ then
+ continue
+ elif test $fanout -eq 0
+ then
+ fanout=1 &&
+ if all_notes_have_fanout refs/notes/commits~$i $fanout
+ then
+ echo "Fanout 0 -> 1 at refs/notes/commits~$i" &&
+ continue
+ fi
+ fi &&
+ echo "Failed fanout=$fanout check at refs/notes/commits~$i" &&
+ git ls-tree -r --name-only refs/notes/commits~$i &&
+ return 1
+ done &&
+ all_notes_have_fanout refs/notes/commits 1
'
test_expect_success 'deleting most notes with git-notes' '
- num_notes=250 &&
+ remove_notes=285 &&
i=0 &&
git rev-list HEAD |
- while test $i -lt $num_notes && read sha1
+ while test $i -lt $remove_notes && read sha1
do
i=$(($i + 1)) &&
test_tick &&
- git notes remove "$sha1" ||
- exit 1
+ git notes remove "$sha1" 2>/dev/null || return 1
done
'
test_expect_success 'most notes deleted correctly with git-notes' '
- git log HEAD~250 | grep "^ " > output &&
- i=50 &&
+ git log HEAD~$remove_notes | grep "^ " > output &&
+ i=$(($num_notes - $remove_notes)) &&
while test $i -gt 0
do
echo " commit #$i" &&
@@ -67,16 +105,29 @@ test_expect_success 'most notes deleted correctly with git-notes' '
test_cmp expect output
'
-test_expect_success 'deleting most notes triggers fanout consolidation' '
- # Expect entire notes tree to have a fanout == 0
- git ls-tree -r --name-only refs/notes/commits |
- while read path
+test_expect_success 'stable fanout 1 is followed by stable fanout 0' '
+ i=$remove_notes &&
+ fanout=1 &&
+ while test $i -gt 0
do
- echo $path | grep -v "^../.*" || {
- echo "Invalid path \"$path\"" &&
- return 1;
- }
- done
+ i=$(($i - 1)) &&
+ if touched_one_note_with_fanout refs/notes/commits~$i D $fanout
+ then
+ continue
+ elif test $fanout -eq 1
+ then
+ fanout=0 &&
+ if all_notes_have_fanout refs/notes/commits~$i $fanout
+ then
+ echo "Fanout 1 -> 0 at refs/notes/commits~$i" &&
+ continue
+ fi
+ fi &&
+ echo "Failed fanout=$fanout check at refs/notes/commits~$i" &&
+ git ls-tree -r --name-only refs/notes/commits~$i &&
+ return 1
+ done &&
+ all_notes_have_fanout refs/notes/commits 0
'
test_done
--
2.23.1
next reply other threads:[~2020-02-03 21:05 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-03 21:04 Johan Herland [this message]
2020-02-03 21:04 ` [PATCH 2/2] notes.c: fix off-by-one error when decreasing notes fanout Johan Herland
2020-02-04 1:05 ` brian m. carlson
2020-02-05 17:16 ` Johan Herland
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200203210445.2854-1-johan@herland.net \
--to=johan@herland.net \
--cc=Johannes.Schindelin@gmx.de \
--cc=git@vger.kernel.org \
--cc=sandals@crustytoothpaste.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).