git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: Stefan Beller <sbeller@google.com>
Cc: "Michael Haggerty" <mhagger@alum.mit.edu>,
	"Junio C Hamano" <gitster@pobox.com>,
	"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"David Turner" <novalis@novalis.org>,
	"git@vger.kernel.org" <git@vger.kernel.org>
Subject: Re: [PATCH 17/23] get_packed_ref_cache(): assume "packed-refs" won't change while locked
Date: Wed, 17 May 2017 21:15:18 -0400	[thread overview]
Message-ID: <20170518011517.xklxkbmkkx6cppdf@sigill.intra.peff.net> (raw)
In-Reply-To: <CAGZ79kb65sv8g6XUQMcGTkZ0ubpY2LYpj7g2wv15knXuv7oKhw@mail.gmail.com>

On Wed, May 17, 2017 at 10:57:34AM -0700, Stefan Beller wrote:

> On Wed, May 17, 2017 at 5:05 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> > If we've got the "packed-refs" file locked, then it can't change;
> > there's no need to keep calling `stat_validity_check()` on it.
> 
> This change will work in a world where all Git implementations
> obey a lock. If there is at least one implementation that doesn't
> care about the existence of lock files we may introduce a race
> here.
> 
> I am not sure if it is worth to be extra careful in the common case
> though. But I could imagine some people using a git repo on an
> NFS concurrently with different implementations and one of them
> is an old / careless lock-ignoring implementation.
> 
> My opinion is not strong enough that I'd veto such a patch
> just food for thought.

You're so unbelievably screwed if somebody is not respecting the lock
that I don't think it's worth considering.

This change just drops the stat_validity_check(), so you may fail to
realize that somebody racily (without holding the lock!) changed the
packed refs, and may omit a ref from your traversal if it moved from
loose to packed. That _can_ have lasting corruption effects if your
operation is something like "git prune" that is computing full
reachability.

But even without this change, somebody writing the packed-refs file
without lock is likely to hose over simultaneous writes and lose ref
updates (or even lose refs entirely). So anybody who doesn't respect the
locks is broken, period, and needs to be fixed.

-Peff

  reply	other threads:[~2017-05-18  1:15 UTC|newest]

Thread overview: 73+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-05-17 12:05 [PATCH 00/23] Prepare to separate out a packed_ref_store Michael Haggerty
2017-05-17 12:05 ` [PATCH 01/23] t3600: clean up permissions test properly Michael Haggerty
2017-05-17 12:42   ` Jeff King
2017-05-17 14:01     ` Michael Haggerty
2017-05-18  4:10   ` Junio C Hamano
2017-05-19  3:37     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 02/23] refs.h: clarify docstring for the ref_transaction_update()-related fns Michael Haggerty
2017-05-17 16:46   ` Stefan Beller
2017-05-18  4:13     ` Junio C Hamano
2017-05-17 12:05 ` [PATCH 03/23] ref_iterator_begin_fn(): fix docstring Michael Haggerty
2017-05-17 12:05 ` [PATCH 04/23] prefix_ref_iterator: don't trim too much Michael Haggerty
2017-05-17 12:55   ` Jeff King
2017-05-17 14:11     ` Michael Haggerty
2017-05-17 14:22       ` Jeff King
2017-05-18  4:19   ` Junio C Hamano
2017-05-18  4:50     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 05/23] refs_ref_iterator_begin(): don't check prefixes redundantly Michael Haggerty
2017-05-17 12:59   ` Jeff King
2017-05-17 14:21     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 06/23] refs: use `size_t` indexes when iterating over ref transaction updates Michael Haggerty
2017-05-17 16:59   ` Stefan Beller
2017-05-18  4:55     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 07/23] ref_store: take `logmsg` parameter when deleting references Michael Haggerty
2017-05-17 13:12   ` Jeff King
2017-05-17 15:01     ` Michael Haggerty
2017-05-17 15:03       ` Jeff King
2017-05-17 12:05 ` [PATCH 08/23] lockfile: add a new method, is_lock_file_locked() Michael Haggerty
2017-05-17 13:12   ` Jeff King
2017-05-17 12:05 ` [PATCH 09/23] files-backend: move `lock` member to `files_ref_store` Michael Haggerty
2017-05-17 13:15   ` Jeff King
2017-05-17 15:49     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 10/23] files_ref_store: put the packed files lock directly in this struct Michael Haggerty
2017-05-17 13:17   ` Jeff King
2017-05-17 15:05     ` Michael Haggerty
2017-05-17 17:18     ` Stefan Beller
2017-05-18  0:18       ` Brandon Williams
2017-05-19  4:00       ` Michael Haggerty
2017-05-18  0:17     ` Brandon Williams
2017-05-18  1:11       ` Jeff King
2017-05-18 15:42         ` Brandon Williams
2017-05-17 12:05 ` [PATCH 11/23] files_transaction_cleanup(): new helper function Michael Haggerty
2017-05-17 13:19   ` Jeff King
2017-05-19  4:49     ` Michael Haggerty
2017-05-17 17:26   ` Stefan Beller
2017-05-19  4:42     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 12/23] ref_transaction_commit(): break into multiple functions Michael Haggerty
2017-05-17 17:44   ` Stefan Beller
2017-05-19  7:58     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 13/23] ref_update_reject_duplicates(): expose function to whole refs module Michael Haggerty
2017-05-17 12:05 ` [PATCH 14/23] ref_update_reject_duplicates(): use `size_t` rather than `int` Michael Haggerty
2017-05-17 12:05 ` [PATCH 15/23] ref_update_reject_duplicates(): add a sanity check Michael Haggerty
2017-05-17 12:05 ` [PATCH 16/23] should_pack_ref(): new function, extracted from `files_pack_refs()` Michael Haggerty
2017-05-17 12:05 ` [PATCH 17/23] get_packed_ref_cache(): assume "packed-refs" won't change while locked Michael Haggerty
2017-05-17 17:57   ` Stefan Beller
2017-05-18  1:15     ` Jeff King [this message]
2017-05-18 16:58       ` Stefan Beller
2017-05-17 12:05 ` [PATCH 18/23] read_packed_refs(): do more of the work of reading packed refs Michael Haggerty
2017-05-17 12:05 ` [PATCH 19/23] read_packed_refs(): report unexpected fopen() failures Michael Haggerty
2017-05-17 13:28   ` Jeff King
2017-05-17 15:27     ` Michael Haggerty
2017-05-18  4:57     ` Junio C Hamano
2017-05-18  5:08       ` Jeff King
2017-05-17 12:05 ` [PATCH 20/23] refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA` Michael Haggerty
2017-05-17 13:29   ` Jeff King
2017-05-17 12:05 ` [PATCH 21/23] create_ref_entry(): remove `check_name` option Michael Haggerty
2017-05-17 12:05 ` [PATCH 22/23] ref-filter: limit traversal to prefix Michael Haggerty
2017-05-17 13:38   ` Jeff King
2017-05-19 10:02     ` Michael Haggerty
2017-05-17 12:05 ` [PATCH 23/23] cache_ref_iterator_begin(): avoid priming unneeded directories Michael Haggerty
2017-05-17 13:42 ` [PATCH 00/23] Prepare to separate out a packed_ref_store Jeff King
2017-05-17 18:14   ` Stefan Beller
2017-05-18 17:14 ` Johannes Sixt
2017-05-18 17:22   ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170518011517.xklxkbmkkx6cppdf@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=mhagger@alum.mit.edu \
    --cc=novalis@novalis.org \
    --cc=pclouds@gmail.com \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).