[PATCH] strbuf: add strbuf_add_real_path()

[PATCH] strbuf: add strbuf_add_real_path()

[René Scharfe]

Add a function for appending the canonized absolute pathname of a given
path to a strbuf.  It keeps the existing contents intact, as expected of
a function of the strbuf_add() family, while avoiding copying the result
if the given strbuf is empty.  It's more consistent with the rest of the
strbuf API than strbuf_realpath(), which it's wrapping.

Also add a semantic patch demonstrating its intended usage and apply it
to the current tree.  Using strbuf_add_real_path() instead of calling
strbuf_addstr() and real_path() avoids an extra copy to a static buffer.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 contrib/coccinelle/strbuf.cocci |  6 ++++++
 setup.c                         |  2 +-
 strbuf.c                        | 11 +++++++++++
 strbuf.h                        | 14 ++++++++++++++
 4 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/contrib/coccinelle/strbuf.cocci b/contrib/coccinelle/strbuf.cocci
index 63995f22ff..1d580e49b0 100644
--- a/contrib/coccinelle/strbuf.cocci
+++ b/contrib/coccinelle/strbuf.cocci
@@ -38,3 +38,9 @@ expression E1, E2, E3;
 @@
 - strbuf_addstr(E1, find_unique_abbrev(E2, E3));
 + strbuf_add_unique_abbrev(E1, E2, E3);
+
+@@
+expression E1, E2;
+@@
+- strbuf_addstr(E1, real_path(E2));
++ strbuf_add_real_path(E1, E2);
diff --git a/setup.c b/setup.c
index 967f289f1e..f14cbcd338 100644
--- a/setup.c
+++ b/setup.c
@@ -254,7 +254,7 @@ int get_common_dir_noenv(struct strbuf *sb, const char *gitdir)
 		if (!is_absolute_path(data.buf))
 			strbuf_addf(&path, "%s/", gitdir);
 		strbuf_addbuf(&path, &data);
-		strbuf_addstr(sb, real_path(path.buf));
+		strbuf_add_real_path(sb, path.buf);
 		ret = 1;
 	} else {
 		strbuf_addstr(sb, gitdir);
diff --git a/strbuf.c b/strbuf.cq
index 8fec6579f7..ace58e7367 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -707,6 +707,17 @@ void strbuf_add_absolute_path(struct strbuf *sb, const char *path)
 	strbuf_addstr(sb, path);
 }
 
+void strbuf_add_real_path(struct strbuf *sb, const char *path)
+{
+	if (sb->len) {
+		struct strbuf resolved = STRBUF_INIT;
+		strbuf_realpath(&resolved, path, 1);
+		strbuf_addbuf(sb, &resolved);
+		strbuf_release(&resolved);
+	} else
+		strbuf_realpath(sb, path, 1);
+}
+
 int printf_ln(const char *fmt, ...)
 {
 	int ret;
diff --git a/strbuf.h b/strbuf.h
index cf1b5409e7..cf8e4bf532 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -441,6 +441,20 @@ extern int strbuf_getcwd(struct strbuf *sb);
  */
 extern void strbuf_add_absolute_path(struct strbuf *sb, const char *path);
 
+/**
+ * Canonize `path` (make it absolute, resolve symlinks, remove extra
+ * slashes) and append it to `sb`.  Die with an informative error
+ * message if there is a problem.
+ *
+ * The directory part of `path` (i.e., everything up to the last
+ * dir_sep) must denote a valid, existing directory, but the last
+ * component need not exist.
+ *
+ * Callers that don't mind links should use the more lightweight
+ * strbuf_add_absolute_path() instead.
+ */
+extern void strbuf_add_real_path(struct strbuf *sb, const char *path);
+
 
 /**
  * Normalize in-place the path contained in the strbuf. See
-- 
2.12.0

Re: [PATCH] strbuf: add strbuf_add_real_path()

[Jeff King]

On Sat, Feb 25, 2017 at 05:00:33PM +0100, René Scharfe wrote:

> Add a function for appending the canonized absolute pathname of a given
> path to a strbuf.  It keeps the existing contents intact, as expected of
> a function of the strbuf_add() family, while avoiding copying the result
> if the given strbuf is empty.  It's more consistent with the rest of the
> strbuf API than strbuf_realpath(), which it's wrapping.
> 
> Also add a semantic patch demonstrating its intended usage and apply it
> to the current tree.  Using strbuf_add_real_path() instead of calling
> strbuf_addstr() and real_path() avoids an extra copy to a static buffer.

It's also re-entrant, which real_path() is not.

> +void strbuf_add_real_path(struct strbuf *sb, const char *path)
> +{
> +	if (sb->len) {
> +		struct strbuf resolved = STRBUF_INIT;
> +		strbuf_realpath(&resolved, path, 1);
> +		strbuf_addbuf(sb, &resolved);
> +		strbuf_release(&resolved);
> +	} else
> +		strbuf_realpath(sb, path, 1);
> +}

The wrapping here seems a little backwards. If strbuf_add_real_path()
were the inner one, then we would not need this extra allocation. I know
that the reasons are historical, but I don't think it would be
impossible to teach the realpath code to do it.

OTOH, it may not be worth the effort. It's not like strbuf_realpath()
doesn't allocate secondary strbufs for its work already, so dropping one
more is probably not that exciting. And certainly think your patch is an
incremental improvement.

Out of curiosity, I took a stab at the patch, which is below. Only
lightly tested by me, and it does make the logic a bit more complicated
to read, as you have to adjust for the original "base" in several
places.

diff --git a/abspath.c b/abspath.c
index 2f0c26e0e..286072f48 100644
--- a/abspath.c
+++ b/abspath.c
@@ -12,9 +12,9 @@ int is_directory(const char *path)
 }
 
 /* removes the last path component from 'path' except if 'path' is root */
-static void strip_last_component(struct strbuf *path)
+static void strip_last_component(struct strbuf *path, size_t base)
 {
-	size_t offset = offset_1st_component(path->buf);
+	size_t offset = base + offset_1st_component(path->buf + base);
 	size_t len = path->len;
 
 	/* Find start of the last component */
@@ -49,14 +49,15 @@ static void get_next_component(struct strbuf *next, struct strbuf *remaining)
 }
 
 /* copies root part from remaining to resolved, canonicalizing it on the way */
-static void get_root_part(struct strbuf *resolved, struct strbuf *remaining)
+static void get_root_part(struct strbuf *resolved, struct strbuf *remaining,
+			  size_t base)
 {
 	int offset = offset_1st_component(remaining->buf);
 
-	strbuf_reset(resolved);
+	strbuf_setlen(resolved, base);
 	strbuf_add(resolved, remaining->buf, offset);
 #ifdef GIT_WINDOWS_NATIVE
-	convert_slashes(resolved->buf);
+	convert_slashes(resolved->buf + base);
 #endif
 	strbuf_remove(remaining, 0, offset);
 }
@@ -78,8 +79,8 @@ static void get_root_part(struct strbuf *resolved, struct strbuf *remaining)
  * informative error message if there is a problem.  Otherwise, return
  * NULL on errors (without generating any output).
  */
-char *strbuf_realpath(struct strbuf *resolved, const char *path,
-		      int die_on_error)
+char *strbuf_add_real_path(struct strbuf *resolved, const char *path,
+			   int die_on_error)
 {
 	struct strbuf remaining = STRBUF_INIT;
 	struct strbuf next = STRBUF_INIT;
@@ -87,6 +88,7 @@ char *strbuf_realpath(struct strbuf *resolved, const char *path,
 	char *retval = NULL;
 	int num_symlinks = 0;
 	struct stat st;
+	size_t base = resolved->len;
 
 	if (!*path) {
 		if (die_on_error)
@@ -96,9 +98,9 @@ char *strbuf_realpath(struct strbuf *resolved, const char *path,
 	}
 
 	strbuf_addstr(&remaining, path);
-	get_root_part(resolved, &remaining);
+	get_root_part(resolved, &remaining, base);
 
-	if (!resolved->len) {
+	if (resolved->len == base) {
 		/* relative path; can use CWD as the initial resolved path */
 		if (strbuf_getcwd(resolved)) {
 			if (die_on_error)
@@ -118,7 +120,7 @@ char *strbuf_realpath(struct strbuf *resolved, const char *path,
 			continue; /* '.' component */
 		} else if (next.len == 2 && !strcmp(next.buf, "..")) {
 			/* '..' component; strip the last path component */
-			strip_last_component(resolved);
+			strip_last_component(resolved, base);
 			continue;
 		}
 
@@ -127,12 +129,12 @@ char *strbuf_realpath(struct strbuf *resolved, const char *path,
 			strbuf_addch(resolved, '/');
 		strbuf_addbuf(resolved, &next);
 
-		if (lstat(resolved->buf, &st)) {
+		if (lstat(resolved->buf + base, &st)) {
 			/* error out unless this was the last component */
 			if (errno != ENOENT || remaining.len) {
 				if (die_on_error)
 					die_errno("Invalid path '%s'",
-						  resolved->buf);
+						  resolved->buf + base);
 				else
 					goto error_out;
 			}
@@ -150,7 +152,7 @@ char *strbuf_realpath(struct strbuf *resolved, const char *path,
 					goto error_out;
 			}
 
-			len = strbuf_readlink(&symlink, resolved->buf,
+			len = strbuf_readlink(&symlink, resolved->buf + base,
 					      st.st_size);
 			if (len < 0) {
 				if (die_on_error)
@@ -162,14 +164,14 @@ char *strbuf_realpath(struct strbuf *resolved, const char *path,
 
 			if (is_absolute_path(symlink.buf)) {
 				/* absolute symlink; set resolved to root */
-				get_root_part(resolved, &symlink);
+				get_root_part(resolved, &symlink, base);
 			} else {
 				/*
 				 * relative symlink
 				 * strip off the last component since it will
 				 * be replaced with the contents of the symlink
 				 */
-				strip_last_component(resolved);
+				strip_last_component(resolved, base);
 			}
 
 			/*
@@ -202,6 +204,12 @@ char *strbuf_realpath(struct strbuf *resolved, const char *path,
 	return retval;
 }
 
+char *strbuf_realpath(struct strbuf *sb, const char *path, int die_on_error)
+{
+	strbuf_reset(sb);
+	return strbuf_add_real_path(sb, path, die_on_error);
+}
+
 const char *real_path(const char *path)
 {
 	static struct strbuf realpath = STRBUF_INIT;

Re: [PATCH] strbuf: add strbuf_add_real_path()

[Brandon Williams]

On 02/25, René Scharfe wrote:
> Add a function for appending the canonized absolute pathname of a given
> path to a strbuf.  It keeps the existing contents intact, as expected of
> a function of the strbuf_add() family, while avoiding copying the result
> if the given strbuf is empty.  It's more consistent with the rest of the
> strbuf API than strbuf_realpath(), which it's wrapping.
> 
> Also add a semantic patch demonstrating its intended usage and apply it
> to the current tree.  Using strbuf_add_real_path() instead of calling
> strbuf_addstr() and real_path() avoids an extra copy to a static buffer.
> 

Seems like a reasonable thing to do.  When I wrote strbuf_realpath() I
think I looked at the strbuf_getcwd() function for what it did (since it
handled paths) and it simply uses the provided buffer disregarding what
is already stored in it.

> Signed-off-by: Rene Scharfe <l.s.r@web.de>
> ---
>  contrib/coccinelle/strbuf.cocci |  6 ++++++
>  setup.c                         |  2 +-
>  strbuf.c                        | 11 +++++++++++
>  strbuf.h                        | 14 ++++++++++++++
>  4 files changed, 32 insertions(+), 1 deletion(-)
> 
> diff --git a/contrib/coccinelle/strbuf.cocci b/contrib/coccinelle/strbuf.cocci
> index 63995f22ff..1d580e49b0 100644
> --- a/contrib/coccinelle/strbuf.cocci
> +++ b/contrib/coccinelle/strbuf.cocci
> @@ -38,3 +38,9 @@ expression E1, E2, E3;
>  @@
>  - strbuf_addstr(E1, find_unique_abbrev(E2, E3));
>  + strbuf_add_unique_abbrev(E1, E2, E3);
> +
> +@@
> +expression E1, E2;
> +@@
> +- strbuf_addstr(E1, real_path(E2));
> ++ strbuf_add_real_path(E1, E2);
> diff --git a/setup.c b/setup.c
> index 967f289f1e..f14cbcd338 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -254,7 +254,7 @@ int get_common_dir_noenv(struct strbuf *sb, const char *gitdir)
>  		if (!is_absolute_path(data.buf))
>  			strbuf_addf(&path, "%s/", gitdir);
>  		strbuf_addbuf(&path, &data);
> -		strbuf_addstr(sb, real_path(path.buf));
> +		strbuf_add_real_path(sb, path.buf);
>  		ret = 1;
>  	} else {
>  		strbuf_addstr(sb, gitdir);
> diff --git a/strbuf.c b/strbuf.cq
> index 8fec6579f7..ace58e7367 100644
> --- a/strbuf.c
> +++ b/strbuf.c
> @@ -707,6 +707,17 @@ void strbuf_add_absolute_path(struct strbuf *sb, const char *path)
>  	strbuf_addstr(sb, path);
>  }
>  
> +void strbuf_add_real_path(struct strbuf *sb, const char *path)
> +{
> +	if (sb->len) {
> +		struct strbuf resolved = STRBUF_INIT;
> +		strbuf_realpath(&resolved, path, 1);
> +		strbuf_addbuf(sb, &resolved);
> +		strbuf_release(&resolved);
> +	} else
> +		strbuf_realpath(sb, path, 1);

I know its not required but I would have braces on the 'else' branch
since they were needed on the 'if' branch.  But that's up to you and
your style :)

> +}
> +
>  int printf_ln(const char *fmt, ...)
>  {
>  	int ret;
> diff --git a/strbuf.h b/strbuf.h
> index cf1b5409e7..cf8e4bf532 100644
> --- a/strbuf.h
> +++ b/strbuf.h
> @@ -441,6 +441,20 @@ extern int strbuf_getcwd(struct strbuf *sb);
>   */
>  extern void strbuf_add_absolute_path(struct strbuf *sb, const char *path);
>  
> +/**
> + * Canonize `path` (make it absolute, resolve symlinks, remove extra
> + * slashes) and append it to `sb`.  Die with an informative error
> + * message if there is a problem.
> + *
> + * The directory part of `path` (i.e., everything up to the last
> + * dir_sep) must denote a valid, existing directory, but the last
> + * component need not exist.
> + *
> + * Callers that don't mind links should use the more lightweight
> + * strbuf_add_absolute_path() instead.
> + */
> +extern void strbuf_add_real_path(struct strbuf *sb, const char *path);
> +
>  
>  /**
>   * Normalize in-place the path contained in the strbuf. See
> -- 
> 2.12.0
> 

-- 
Brandon Williams

Re: [PATCH] strbuf: add strbuf_add_real_path()

[René Scharfe]

Am 27.02.2017 um 19:22 schrieb Brandon Williams:
> On 02/25, René Scharfe wrote:
>> +void strbuf_add_real_path(struct strbuf *sb, const char *path)
>> +{
>> +	if (sb->len) {
>> +		struct strbuf resolved = STRBUF_INIT;
>> +		strbuf_realpath(&resolved, path, 1);
>> +		strbuf_addbuf(sb, &resolved);
>> +		strbuf_release(&resolved);
>> +	} else
>> +		strbuf_realpath(sb, path, 1);
>
> I know its not required but I would have braces on the 'else' branch
> since they were needed on the 'if' branch.  But that's up to you and
> your style :)

Personally I'd actually prefer them as well, but the project's style has 
traditionally been to avoid braces on such trailing single-line branches 
to save lines.  The CodingGuidelines for this topic have been clarified 
recently, though, and seem to require them now.  Interesting.

René

Re: [PATCH] strbuf: add strbuf_add_real_path()

[Brandon Williams]

On 02/27, René Scharfe wrote:
> Am 27.02.2017 um 19:22 schrieb Brandon Williams:
> >On 02/25, René Scharfe wrote:
> >>+void strbuf_add_real_path(struct strbuf *sb, const char *path)
> >>+{
> >>+	if (sb->len) {
> >>+		struct strbuf resolved = STRBUF_INIT;
> >>+		strbuf_realpath(&resolved, path, 1);
> >>+		strbuf_addbuf(sb, &resolved);
> >>+		strbuf_release(&resolved);
> >>+	} else
> >>+		strbuf_realpath(sb, path, 1);
> >
> >I know its not required but I would have braces on the 'else' branch
> >since they were needed on the 'if' branch.  But that's up to you and
> >your style :)
> 
> Personally I'd actually prefer them as well, but the project's style
> has traditionally been to avoid braces on such trailing single-line
> branches to save lines.  The CodingGuidelines for this topic have
> been clarified recently, though, and seem to require them now.
> Interesting.
> 
> René

Having the project's guidelines align with your own preference makes
things a bit easier!

-- 
Brandon Williams