git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: Ben Wijen <ben@wijen.net>,
	Johannes Schindelin <johannes.schindelin@gmx.de>
Cc: git@vger.kernel.org, Lars Schneider <larsxschneider@gmail.com>
Subject: Re: [PATCH 2/2] mingw: ensure temporary file handles are not inherited by child processes
Date: Wed, 17 Aug 2016 11:22:00 -0700	[thread overview]
Message-ID: <xmqqd1l79rxz.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <77e7c4e4de6c45a1b24bb4d08ca20a1385f43444.1471437637.git.johannes.schindelin@gmx.de> (Johannes Schindelin's message of "Wed, 17 Aug 2016 14:41:19 +0200 (CEST)")

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> From: Ben Wijen <ben@wijen.net>
>
> When the index is locked and child processes inherit the handle to
> said lock and the parent process wants to remove the lock before the
> child process exits, on Windows there is a problem: it won't work
> because files cannot be deleted if a process holds a handle on them.
> The symptom:
>
>     Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
>     Should I try again? (y/n)
>
> Spawning child processes with bInheritHandles==FALSE would not work
> because no file handles would be inherited, not even the hStdXxx
> handles in STARTUPINFO (stdin/stdout/stderr).
>
> Opening every file with O_NOINHERIT does not work, either, as e.g.
> git-upload-pack expects inherited file handles.
>
> This leaves us with the only way out: creating temp files with the
> O_NOINHERIT flag. This flag is Windows-specific, however. For our
> purposes, it is equivalent our purposes) to O_CLOEXEC (which does not
> exist on Windows), so let's just open temporary files with the
> O_CLOEXEC flag and map that flag to O_NOINHERIT on Windows.

The callchain that leads to create_tempfile() eventually goes up to
hold_lock_file_for_update() and its _timeout() variant in the
current codebase.  There is no other create_tempfile() caller.

I think all the callers of these public entry points use them to
open a lockfile for its own use, and never delegate the writing to
it to their children, so this change is safe right now, and I do not
think it is too bad that a new caller that wants to do that have to
explicitly drop O_CLOEXEC (perhaps by dup(2)ing).

But it deserves mention in the lockfile and the tempfile API docs in
<lockfile.h> and <tempfile.h> that the file descriptor returned from
these public entry points does not survive across fork(2).

Something along the line shown by the attached patch, perhaps.

Other than that, this is very nicely analyzed and discusses possible
alternative approaches sufficiently to easily convince readers that
this is the best solution.

Very nicely done.  Thanks.

 lockfile.h | 4 ++++
 tempfile.h | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/lockfile.h b/lockfile.h
index 3d30193..e976c7a 100644
--- a/lockfile.h
+++ b/lockfile.h
@@ -55,6 +55,10 @@
  *   * calling `fdopen_lock_file()` to get a `FILE` pointer for the
  *     open file and writing to the file using stdio.
  *
+ *   Note that the file descriptor returned by hold_lock_file_for_update()
+ *   is marked O_CLOEXEC, so the new contents must be written by
+ *   you, and not by your subprocess.
+ *
  * When finished writing, the caller can:
  *
  * * Close the file descriptor and rename the lockfile to its final
diff --git a/tempfile.h b/tempfile.h
index 4219fe4..d357177 100644
--- a/tempfile.h
+++ b/tempfile.h
@@ -33,6 +33,10 @@
  *   * calling `fdopen_tempfile()` to get a `FILE` pointer for the
  *     open file and writing to the file using stdio.
  *
+ *   Note that the file descriptor returned by create_tempfile()
+ *   is marked O_CLOEXEC, so the new contents must be written by
+ *   you, and not by your subprocess.
+ *
  * When finished writing, the caller can:
  *
  * * Close the file descriptor and remove the temporary file by

  parent reply	other threads:[~2016-08-17 18:22 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-08-17 12:40 [PATCH 0/2] Do not lock temporary files via child processes on Windows Johannes Schindelin
2016-08-17 12:40 ` [PATCH 1/2] t6026-merge-attr: child processes must not inherit index.lock handles Johannes Schindelin
2016-08-17 17:55   ` Junio C Hamano
2016-08-17 12:41 ` [PATCH 2/2] mingw: ensure temporary file handles are not inherited by child processes Johannes Schindelin
2016-08-17 12:48   ` Eric Sunshine
2016-08-18 14:42     ` Johannes Schindelin
2016-08-17 13:14   ` Lars Schneider
2016-08-17 18:22   ` Junio C Hamano [this message]
2016-08-18 14:50     ` Johannes Schindelin
2016-08-18 17:35   ` Eric Wong
2016-08-18 21:53     ` Junio C Hamano
2016-08-18 22:48       ` Eric Wong
2016-08-19 15:57         ` Junio C Hamano
2016-08-22 12:47           ` Johannes Schindelin
2016-08-18 14:51 ` [PATCH v2 0/2] Do not lock temporary files via child processes on Windows Johannes Schindelin
2016-08-18 14:51   ` [PATCH v2 1/2] t6026-merge-attr: child processes must not inherit index.lock handles Johannes Schindelin
2016-08-18 14:51   ` [PATCH v2 2/2] mingw: ensure temporary file handles are not inherited by child processes Johannes Schindelin
2016-08-22 12:47   ` [PATCH v3 0/2] Do not lock temporary files via child processes on Windows Johannes Schindelin
2016-08-22 12:47     ` [PATCH v3 1/2] t6026-merge-attr: child processes must not inherit index.lock handles Johannes Schindelin
2016-08-22 12:47     ` [PATCH v3 2/2] mingw: ensure temporary file handles are not inherited by child processes Johannes Schindelin
2016-08-22 17:58       ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqd1l79rxz.fsf@gitster.mtv.corp.google.com \
    --to=gitster@pobox.com \
    --cc=ben@wijen.net \
    --cc=git@vger.kernel.org \
    --cc=johannes.schindelin@gmx.de \
    --cc=larsxschneider@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).