From: 16657101987@163.com
To: peff@peff.net
Cc: 16657101987@163.com, git@vger.kernel.org, gitster@pobox.com,
mhagger@alum.mit.edu, sunchao9@huawei.com,
worldhello.net@gmail.com
Subject: [PATCH v2 0/1] pack-refs: always refreshing after take the lock file
Date: Thu, 1 Aug 2019 02:35:43 +0800 [thread overview]
Message-ID: <20190731183544.24406-1-16657101987@163.com> (raw)
In-Reply-To: <20190730063634.GA4901@sigill.intra.peff.net>
In-Reply-To: <20190730063634.GA4901@sigill.intra.peff.net>
From: Sun Chao <16657101987@163.com>
On Tue, 30 Jul 2019 02:36:35 -0400, Jeff King wrote:
> You can do this step without the fetch, which makes it hit the race more
> quickly. :) Try this:
>
> # prime it with a single commit
> git commit --allow-empty -m foo
> while true; do
> us=$(git commit-tree -m foo -p HEAD HEAD^{tree}) &&
> git update-ref refs/heads/newbranch $us &&
> git update-ref refs/heads/master $us &&
> git update-ref -d refs/heads/newbranch &&
> them=$(git rev-parse master) &&
> if test "$them" != "$us"; then
> echo >&2 "lost commit: $us"
> exit 1
> fi
> # eye candy
> printf .
> done
Thanks, this could hit the race more quickly and I update it to the
commit log.
> I don't think this is quite the same as racy-git. There we are comparing
> stat entries for a file X to the timestamp of the index (and we are
> concerned they were written in the same second).
>
> But here we have no on-disk stat information to compare to. It's all
> happening in-process. But you're right that it's a racy stat-validity
> problem.
Yes, I agree with you.
> The stat-validity check here is actually more than the timestamp.
> Specifically it's checking the inode and size. But because of the
> specific set of operations you're performing, this ends up correlating
> quite often:
>
> - because our operations involve updating a single ref or
> adding/deleting another ref, we'll oscillate between two sizes
> (either one ref or two)
>
> - likewise if nothing else is happening on the filesystem, pack-refs
> may flip back and forth between two inodes (not the same one,
> because our tempfile-and-rename strategy means we're still using the
> old one while we write the new packed-refs file).
>
> So I actually find this to be a fairly unlikely case in the real world,
> but as your script demonstrates, it's not that hard to trigger it if
> you're trying.
>
> I'm not sure the second one actually fixes things entirely. What if I
> have an older refs/heads/foo, and I do this:
>
> git pack-refs
> git pack-refs --all --prune
>
> We still might hit the race here. The first pack-refs does not pack foo
> (because we didn't say --all), then a simultaneous "update-ref -d" opens
> `packed-refs`, then the second pack-refs packs it all in the same
> second. Now "update-ref -d" uses the old packed-refs file, and we lose
> the ref.
Yes, I agree with you. And in the real word if the git servers has some
3rd-service which update repositories refs or pack-refs frequently may
have this problem, my company's git servers works like this unfortunately.
> So I actually think the best path forward is just always refreshing when
> we take the lock, something like:
>
> Ultimately the best solution there is to move to a better format (like
> the reftables proposal).
I do not know if we could get the new reftables in the next few versions,
So I commit the changes as you suggested, which is also the same as
another way I metioned in `PATCH v1`:
**force `update-ref -d` to update the snapshot before rewrite packed-refs.**
But if the reftables is comeing soon, please just ignore my PATCH :)
**And thank a lot for your reply, it's great to me, because it's my first
PATCh to git myself :)**
Sun Chao (1):
pack-refs: always refreshing after take the lock file
refs/packed-backend.c | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
--
2.22.0.214.g8dca754b1e
next prev parent reply other threads:[~2019-07-31 18:36 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-21 18:17 [PATCH v1 0/1] pack-refs: pack expired loose refs to packed_refs 16657101987
2019-07-21 18:17 ` [PATCH v1 1/1] " 16657101987
2019-07-30 6:36 ` Jeff King
2019-07-31 18:35 ` 16657101987 [this message]
2019-07-31 18:35 ` [PATCH v2 1/1] pack-refs: always refreshing after take the lock file 16657101987
2019-08-16 20:49 ` [PATCH v2 0/1] " Jeff King
2019-08-19 17:36 ` Junio C Hamano
2019-08-20 15:14 ` 16657101987
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190731183544.24406-1-16657101987@163.com \
--to=16657101987@163.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=mhagger@alum.mit.edu \
--cc=peff@peff.net \
--cc=sunchao9@huawei.com \
--cc=worldhello.net@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).