From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: Johannes Sixt <j6t@kdbg.org>,
git@vger.kernel.org, tytso@mit.edu,
Junio C Hamano <gitster@pobox.com>,
Christoph Hellwig <hch@lst.de>,
Linus Torvalds <torvalds@linux-foundation.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [RFC PATCH 1/2] sha1-file: fsync() loose dir entry when core.fsyncObjectFiles
Date: Thu, 19 Nov 2020 12:38:28 +0100 (CET) [thread overview]
Message-ID: <nycvar.QRO.7.76.6.2011191232490.56@tvgsbejvaqbjf.bet> (raw)
In-Reply-To: <87a6xii5gs.fsf@evledraar.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3109 bytes --]
Hi Ævar,
On Tue, 22 Sep 2020, Ævar Arnfjörð Bjarmason wrote:
> On Thu, Sep 17 2020, Johannes Sixt wrote:
>
> > Am 17.09.20 um 13:28 schrieb Ævar Arnfjörð Bjarmason:
> >> Change the behavior of core.fsyncObjectFiles to also sync the
> >> directory entry. I don't have a case where this broke, just going by
> >> paranoia and the fsync(2) manual page's guarantees about its behavior.
> >>
> >> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> >> ---
> >> sha1-file.c | 19 ++++++++++++++-----
> >> 1 file changed, 14 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/sha1-file.c b/sha1-file.c
> >> index dd65bd5c68..d286346921 100644
> >> --- a/sha1-file.c
> >> +++ b/sha1-file.c
> >> @@ -1784,10 +1784,14 @@ int hash_object_file(const struct git_hash_algo *algo, const void *buf,
> >> }
> >>
> >> /* Finalize a file on disk, and close it. */
> >> -static void close_loose_object(int fd)
> >> +static void close_loose_object(int fd, const struct strbuf *dirname)
> >> {
> >> - if (fsync_object_files)
> >> + int dirfd;
> >> + if (fsync_object_files) {
> >> fsync_or_die(fd, "loose object file");
> >> + dirfd = xopen(dirname->buf, O_RDONLY);
> >> + fsync_or_die(dirfd, "loose object directory");
> >
> > Did you have the opportunity to verify that this works on Windows?
> > Opening a directory with open(2), I mean: It's disallowed according to
> > the docs:
> > https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen?view=vs-2019#return-value
>
> I did not, just did a quick hack for an RFC discussion (didn't even
> close() that fd), but if I pursue this I'll do it properly.
>
> Doing some research on it now reveals that we should probably have some
> Windows-specific code here, e.g. browsing GNUlib's source code reveals
> that it uses FlushFileBuffers(), and that code itself is taken from
> sqlite. SQLite also has special-case code for some Unix warts,
> e.g. OSX's and AIX's special fsync behaviors in its src/os_unix.c
If I understand correctly, the idea to `fsync()` directories is to ensure
that metadata updates (such as renames) are flushed, too?
If so (and please note that my understanding of NTFS is not super deep in
this regard), I think that we need not worry on Windows. I have come to
believe that the `rename()` operations are flushed pretty much
immediately, without any `FlushFileBuffers()` (or `_commit()`, as we
actually do in `compat/mingw.h`, to convince yourself see
https://github.com/git/git/blob/v2.29.2/compat/mingw.h#L135-L136).
Directories are not mentioned in `FlushFileBuffers()`'s documentation
(https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-flushfilebuffers)
nor in the documentation of `_commit()`:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/commit?view=msvc-160
Therefore, I believe that there is not even a Win32 equivalent of
`fsync()`ing directories.
Ciao,
Dscho
>
> >> + } if (close(fd) != 0) die_errno(_("error when closing loose object
> >> file")); }
> >
> > -- Hannes
>
>
>
next prev parent reply other threads:[~2020-11-19 11:42 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-17 18:48 [PATCH] enable core.fsyncObjectFiles by default Christoph Hellwig
2018-01-17 19:04 ` Junio C Hamano
2018-01-17 19:35 ` Christoph Hellwig
2018-01-17 20:05 ` Andreas Schwab
2018-01-17 19:37 ` Matthew Wilcox
2018-01-17 19:42 ` Christoph Hellwig
2018-01-17 21:44 ` Ævar Arnfjörð Bjarmason
2018-01-17 22:07 ` Linus Torvalds
2018-01-17 22:25 ` Linus Torvalds
2018-01-17 23:16 ` Ævar Arnfjörð Bjarmason
2018-01-17 23:42 ` Linus Torvalds
2018-01-17 23:52 ` Theodore Ts'o
2018-01-17 23:57 ` Linus Torvalds
2018-01-18 16:27 ` Christoph Hellwig
2018-01-19 19:08 ` Junio C Hamano
2018-01-20 22:14 ` Theodore Ts'o
2018-01-20 22:27 ` Junio C Hamano
2018-01-22 15:09 ` Ævar Arnfjörð Bjarmason
2018-01-22 18:09 ` Theodore Ts'o
2018-01-23 0:47 ` Jeff King
2018-01-23 5:45 ` Theodore Ts'o
2018-01-23 16:17 ` Jeff King
2018-01-23 0:25 ` Jeff King
2018-01-21 21:32 ` Chris Mason
2020-09-17 11:06 ` Ævar Arnfjörð Bjarmason
2020-09-17 11:28 ` [RFC PATCH 0/2] should core.fsyncObjectFiles fsync the dir entry + docs Ævar Arnfjörð Bjarmason
2020-09-17 11:28 ` [RFC PATCH 1/2] sha1-file: fsync() loose dir entry when core.fsyncObjectFiles Ævar Arnfjörð Bjarmason
2020-09-17 13:16 ` Jeff King
2020-09-17 15:09 ` Christoph Hellwig
2020-09-17 14:09 ` Christoph Hellwig
2020-09-17 14:55 ` Jeff King
2020-09-17 14:56 ` Christoph Hellwig
2020-09-17 15:37 ` Junio C Hamano
2020-09-17 17:12 ` Jeff King
2020-09-17 20:37 ` Taylor Blau
2020-09-22 10:42 ` Ævar Arnfjörð Bjarmason
2020-09-17 20:21 ` Johannes Sixt
2020-09-22 8:24 ` Ævar Arnfjörð Bjarmason
2020-11-19 11:38 ` Johannes Schindelin [this message]
2020-09-17 11:28 ` [RFC PATCH 2/2] core.fsyncObjectFiles: make the docs less flippant Ævar Arnfjörð Bjarmason
2020-09-17 14:12 ` Christoph Hellwig
2020-09-17 15:43 ` Junio C Hamano
2020-09-17 20:15 ` Johannes Sixt
2020-10-08 8:13 ` Johannes Schindelin
2020-10-08 15:57 ` Ævar Arnfjörð Bjarmason
2020-10-08 18:53 ` Junio C Hamano
2020-10-09 10:44 ` Johannes Schindelin
2020-09-17 19:21 ` Marc Branchaud
2020-09-17 14:14 ` [PATCH] enable core.fsyncObjectFiles by default Christoph Hellwig
2020-09-17 15:30 ` Junio C Hamano
2018-01-17 20:55 ` Jeff King
2018-01-17 21:10 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=nycvar.QRO.7.76.6.2011191232490.56@tvgsbejvaqbjf.bet \
--to=johannes.schindelin@gmx.de \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=hch@lst.de \
--cc=j6t@kdbg.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).