[PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

[PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

[René Scharfe]

Use the open(2) flag O_EXCL to ensure the file doesn't already exist
instead of (racily) calling stat(2) through file_exists().  While at it
switch to xopen() to reduce code duplication and get more consistent
error messages.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 notes-merge.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/notes-merge.c b/notes-merge.c
index b7814c9..2b29fc4 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -298,12 +298,8 @@ static void write_buf_to_worktree(const unsigned char *obj,
 	char *path = git_pathdup(NOTES_MERGE_WORKTREE "/%s", sha1_to_hex(obj));
 	if (safe_create_leading_directories_const(path))
 		die_errno("unable to create directory for '%s'", path);
-	if (file_exists(path))
-		die("found existing file at '%s'", path);
 
-	fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, 0666);
-	if (fd < 0)
-		die_errno("failed to open '%s'", path);
+	fd = xopen(path, O_WRONLY | O_EXCL | O_CREAT, 0666);
 
 	while (size > 0) {
 		long ret = write_in_full(fd, buf, size);
-- 
2.9.0

Re: [PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

[Jeff King]

On Thu, Jul 07, 2016 at 10:08:30PM +0200, René Scharfe wrote:

> Use the open(2) flag O_EXCL to ensure the file doesn't already exist
> instead of (racily) calling stat(2) through file_exists().  While at it
> switch to xopen() to reduce code duplication and get more consistent
> error messages.

This is definitely an improvement, as it behaves the same except for the
TOCTOU race. But not being very familiar with the notes-merge code, I
have to wonder if this is a system of a larger design issue.

Why do we care that the file exists? Should we instead be using the
lockfile code to get exclusive access to it? That would also switch us
to doing the write-to-tempfile-and-rename dance, but that seems like it
would be a good thing. If we hit a write() error in the code now, we
leave a partially-written file in the notes worktree.

I dunno. From my cursory reading of the code, it seems like we'd never
really expect this file_exists() to trigger in the first place, so
perhaps it's not worth thinking too hard about it.

-Peff

Re: [PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Why do we care that the file exists? Should we instead be using the
> lockfile code to get exclusive access to it? That would also switch us
> to doing the write-to-tempfile-and-rename dance, but that seems like it
> would be a good thing. If we hit a write() error in the code now, we
> leave a partially-written file in the notes worktree.

Yeah, I had the same thought when I saw the change.

> I dunno. From my cursory reading of the code, it seems like we'd never
> really expect this file_exists() to trigger in the first place, so
> perhaps it's not worth thinking too hard about it.

Perhaps.

Re: [PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

[Jeff King]

On Thu, Jul 07, 2016 at 10:08:30PM +0200, René Scharfe wrote:

> diff --git a/notes-merge.c b/notes-merge.c
> index b7814c9..2b29fc4 100644
> --- a/notes-merge.c
> +++ b/notes-merge.c
> @@ -298,12 +298,8 @@ static void write_buf_to_worktree(const unsigned char *obj,
>  	char *path = git_pathdup(NOTES_MERGE_WORKTREE "/%s", sha1_to_hex(obj));
>  	if (safe_create_leading_directories_const(path))
>  		die_errno("unable to create directory for '%s'", path);
> -	if (file_exists(path))
> -		die("found existing file at '%s'", path);
>  
> -	fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, 0666);
> -	if (fd < 0)
> -		die_errno("failed to open '%s'", path);
> +	fd = xopen(path, O_WRONLY | O_EXCL | O_CREAT, 0666);

While working on write_file_buf() elsewhere, I wondered if this was a
good candidate for conversion. But it's not because of the O_EXCL.

I was tempted by something like:

  write_file_buf(path, O_EXCL, buf, len);

but I think that just makes the interface more cluttered for the other
callers. If you are doing something clever with O_EXCL, you probably
should just do it by hand.

However, we can make the loop less painful, as below (on top of your
patch).

-- >8 --
Subject: [PATCH] notes-merge: use write_or_die()

This output loop is overkill. For one thing, we do not need
to loop on write_in_full(); it's entire purpose is to avoid
looping, and it even includes the "disk full?" check itself.

Secondly, the check for EPIPE is pointless. We know this is
an on-disk file that we just opened, so short of somebody
sneaking a FIFO into our notes-merge working tree, it will
not be a pipe. And even if it is, we would generally die of
SIGPIPE before hitting this code. And even if we somehow
ignored SIGPIPE, dying (rather than silently ignoring the
error) seems like the only sensible action anyway.

So what we're left with is calling write_in_full() and dying
if it fails. And that's exactly what write_or_die() does.

Signed-off-by: Jeff King <peff@peff.net>
---
 notes-merge.c | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)

diff --git a/notes-merge.c b/notes-merge.c
index 2b29fc4..a659a62 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -300,21 +300,7 @@ static void write_buf_to_worktree(const unsigned char *obj,
 		die_errno("unable to create directory for '%s'", path);
 
 	fd = xopen(path, O_WRONLY | O_EXCL | O_CREAT, 0666);
-
-	while (size > 0) {
-		long ret = write_in_full(fd, buf, size);
-		if (ret < 0) {
-			/* Ignore epipe */
-			if (errno == EPIPE)
-				break;
-			die_errno("notes-merge");
-		} else if (!ret) {
-			die("notes-merge: disk full?");
-		}
-		size -= ret;
-		buf += ret;
-	}
-
+	write_or_die(fd, buf, size);
 	close(fd);
 	free(path);
 }
-- 
2.9.0.393.g704e522