[PATCH] fix pushing to //server/share/dir paths on Windows

[PATCH] fix pushing to //server/share/dir paths on Windows

[Johannes Sixt]

normalize_path_copy() is not prepared to keep the double-slash of a
//server/share/dir kind of path, but treats it like a regular POSIX
style path and transforms it to /server/share/dir.

The bug manifests when 'git push //server/share/dir master' is run,
because tmp_objdir_add_as_alternate() uses the path in normalized
form when it registers the quarantine object database via
link_alt_odb_entries(). Needless to say that the directory cannot be
accessed using the wrongly normalized path.

Fix it by skipping all of the root part, not just a potential drive
prefix. offset_1st_component takes care of this, see the
implementation in compat/mingw.c::mingw_offset_1st_component().

There is a change in behavior: \\server\share is not transformed
into //server/share anymore, but all subsequent directory separators
are rewritten to '/'. This should not make a difference; Windows can
handle the mix. In the context of 'git push' this cannot be verified,
though, as there seems to be an independent bug that transforms the
double '\\' to a single '\' on the way.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 Another long-standing bug uncovered by the quarantine series.

 Dscho, it looks like this could fix the original report at
 https://github.com/git-for-windows/git/issues/979

 This patch should cook well because of the change in behavior.
 I would not be surprised if there is some fall-out.

 The other bug I'm alluding to, I still have to investigate. I do
 not think that it can be counted as fall-out.

 path.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/path.c b/path.c
index 52d889c88e..02dc70fb92 100644
--- a/path.c
+++ b/path.c
@@ -991,7 +991,7 @@ const char *remove_leading_path(const char *in, const char *prefix)
  *
  * Performs the following normalizations on src, storing the result in dst:
  * - Ensures that components are separated by '/' (Windows only)
- * - Squashes sequences of '/'.
+ * - Squashes sequences of '/' except "//server/share" on Windows
  * - Removes "." components.
  * - Removes ".." components, and the components the precede them.
  * Returns failure (non-zero) if a ".." component appears as first path
@@ -1014,17 +1014,23 @@ const char *remove_leading_path(const char *in, const char *prefix)
 int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
 {
 	char *dst0;
-	int i;
-
-	for (i = has_dos_drive_prefix(src); i > 0; i--)
-		*dst++ = *src++;
-	dst0 = dst;
+	int offset;
 
-	if (is_dir_sep(*src)) {
+	/*
+	 * Handle initial part of absolute path: "/", "C:/", "\\server\share/".
+	 */
+	offset = offset_1st_component(src);
+	if (offset) {
+		/* Convert the trailing separator to '/' on Windows. */
+		memcpy(dst, src, offset - 1);
+		dst += offset - 1;
 		*dst++ = '/';
-		while (is_dir_sep(*src))
-			src++;
+		src += offset;
 	}
+	dst0 = dst;
+
+	while (is_dir_sep(*src))
+		src++;
 
 	for (;;) {
 		char c = *src;
-- 
2.11.0.79.g263f27a

Re: [PATCH] fix pushing to //server/share/dir paths on Windows

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> There is a change in behavior: \\server\share is not transformed
> into //server/share anymore, but all subsequent directory separators
> are rewritten to '/'. This should not make a difference; Windows can
> handle the mix.

I saw Dscho had a similar "windows can handle the mix" change in an
earlier development cycle, I think, and this is being consistent.

>  Another long-standing bug uncovered by the quarantine series.
>
>  Dscho, it looks like this could fix the original report at
>  https://github.com/git-for-windows/git/issues/979
>
>  This patch should cook well because of the change in behavior.
>  I would not be surprised if there is some fall-out.
>
>  The other bug I'm alluding to, I still have to investigate. I do
>  not think that it can be counted as fall-out.
>
>  path.c | 24 +++++++++++++++---------
>  1 file changed, 15 insertions(+), 9 deletions(-)

Thanks.

> diff --git a/path.c b/path.c
> index 52d889c88e..02dc70fb92 100644
> --- a/path.c
> +++ b/path.c
> @@ -991,7 +991,7 @@ const char *remove_leading_path(const char *in, const char *prefix)
>   *
>   * Performs the following normalizations on src, storing the result in dst:
>   * - Ensures that components are separated by '/' (Windows only)
> - * - Squashes sequences of '/'.
> + * - Squashes sequences of '/' except "//server/share" on Windows

"on windows" because offset_1st_component() does the magic only
there?  Makes sense.

>   * - Removes "." components.
>   * - Removes ".." components, and the components the precede them.
>   * Returns failure (non-zero) if a ".." component appears as first path
> @@ -1014,17 +1014,23 @@ const char *remove_leading_path(const char *in, const char *prefix)
>  int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
>  {
>  	char *dst0;
> -	int i;
> -
> -	for (i = has_dos_drive_prefix(src); i > 0; i--)
> -		*dst++ = *src++;
> -	dst0 = dst;
> +	int offset;
>  
> -	if (is_dir_sep(*src)) {
> +	/*
> +	 * Handle initial part of absolute path: "/", "C:/", "\\server\share/".
> +	 */
> +	offset = offset_1st_component(src);
> +	if (offset) {
> +		/* Convert the trailing separator to '/' on Windows. */
> +		memcpy(dst, src, offset - 1);
> +		dst += offset - 1;
>  		*dst++ = '/';
> -		while (is_dir_sep(*src))
> -			src++;
> +		src += offset;
>  	}
> +	dst0 = dst;

By resetting dst0 here, we ensure that up_one that is triggered by
seeing "../" will not escape the \\server\share\ part, which makes
sense to me.

> +	while (is_dir_sep(*src))
> +		src++;
>  
>  	for (;;) {
>  		char c = *src;

Re: [PATCH] fix pushing to //server/share/dir paths on Windows

[Jeff King]

On Tue, Dec 13, 2016 at 10:32:01PM +0100, Johannes Sixt wrote:

> normalize_path_copy() is not prepared to keep the double-slash of a
> //server/share/dir kind of path, but treats it like a regular POSIX
> style path and transforms it to /server/share/dir.
> 
> The bug manifests when 'git push //server/share/dir master' is run,
> because tmp_objdir_add_as_alternate() uses the path in normalized
> form when it registers the quarantine object database via
> link_alt_odb_entries(). Needless to say that the directory cannot be
> accessed using the wrongly normalized path.

Thanks for digging this up! I had a feeling that the problem was going
to be in the underlying path code, but I didn't want to just pass the
buck without evidence. :)

> -	if (is_dir_sep(*src)) {
> +	/*
> +	 * Handle initial part of absolute path: "/", "C:/", "\\server\share/".
> +	 */
> +	offset = offset_1st_component(src);
> +	if (offset) {
> +		/* Convert the trailing separator to '/' on Windows. */
> +		memcpy(dst, src, offset - 1);
> +		dst += offset - 1;
>  		*dst++ = '/';

Hmm. So this is the "change-of-behavior" bit. Would it be reasonable to
write:

  /* Copy initial part of absolute path, converting separators on Windows */
  const char *end = src + offset_1st_component(src);
  while (src < end) {
	  char c = *src++;
	  if (c == '\\')
		  c = '/';
	  *dst++ = c;
  }

? I'm not sure if it's wrong to convert backslashes in that first
component or not (but certainly we were before). I don't think we'd need
is_dir_sep() in that "if()", because we can leave slashes as-is. But
maybe it would make the code easier to read.

-Peff

[PATCH v2] fix pushing to //server/share/dir on Windows

[Johannes Sixt]

normalize_path_copy() is not prepared to keep the double-slash of a
//server/share/dir kind of path, but treats it like a regular POSIX
style path and transforms it to /server/share/dir.

The bug manifests when 'git push //server/share/dir master' is run,
because tmp_objdir_add_as_alternate() uses the path in normalized
form when it registers the quarantine object database via
link_alt_odb_entries(). Needless to say that the directory cannot be
accessed using the wrongly normalized path.

Fix it by skipping all of the root part, not just a potential drive
prefix. offset_1st_component takes care of this, see the
implementation in compat/mingw.c::mingw_offset_1st_component().

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
Am 14.12.2016 um 18:30 schrieb Jeff King:
> Would it be reasonable to
> write:
> 
>   /* Copy initial part of absolute path, converting separators on Windows */
>   const char *end = src + offset_1st_component(src);
>   while (src < end) {
> 	  char c = *src++;
> 	  if (c == '\\')
> 		  c = '/';
> 	  *dst++ = c;
>   }

Makes a lot of sense! I haven't had an opportunity, though, to test
on Windows.

> ? I'm not sure if it's wrong to convert backslashes in that first
> component or not (but certainly we were before). I don't think we'd need
> is_dir_sep() in that "if()", because we can leave slashes as-is. But
> maybe it would make the code easier to read.

is_dir_sep() is preferable, IMO.

I also changed the commit message and subject line slightly.

 path.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/path.c b/path.c
index 52d889c88e..efcedafba6 100644
--- a/path.c
+++ b/path.c
@@ -991,7 +991,7 @@ const char *remove_leading_path(const char *in, const char *prefix)
  *
  * Performs the following normalizations on src, storing the result in dst:
  * - Ensures that components are separated by '/' (Windows only)
- * - Squashes sequences of '/'.
+ * - Squashes sequences of '/' except "//server/share" on Windows
  * - Removes "." components.
  * - Removes ".." components, and the components the precede them.
  * Returns failure (non-zero) if a ".." component appears as first path
@@ -1014,17 +1014,22 @@ const char *remove_leading_path(const char *in, const char *prefix)
 int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
 {
 	char *dst0;
-	int i;
+	const char *end;
 
-	for (i = has_dos_drive_prefix(src); i > 0; i--)
-		*dst++ = *src++;
+	/*
+	 * Copy initial part of absolute path: "/", "C:/", "//server/share/".
+	 */
+	end = src + offset_1st_component(src);
+	while (src < end) {
+		char c = *src++;
+		if (is_dir_sep(c))
+			c = '/';
+		*dst++ = c;
+	}
 	dst0 = dst;
 
-	if (is_dir_sep(*src)) {
-		*dst++ = '/';
-		while (is_dir_sep(*src))
-			src++;
-	}
+	while (is_dir_sep(*src))
+		src++;
 
 	for (;;) {
 		char c = *src;
-- 
2.11.0.79.g263f27a

Re: [PATCH v2] fix pushing to //server/share/dir on Windows

[Torsten Bögershausen]

On 14/12/16 20:37, Johannes Sixt wrote:
> normalize_path_copy() is not prepared to keep the double-slash of a
> //server/share/dir kind of path, but treats it like a regular POSIX
> style path and transforms it to /server/share/dir.
>
> The bug manifests when 'git push //server/share/dir master' is run,
> because tmp_objdir_add_as_alternate() uses the path in normalized
> form when it registers the quarantine object database via
> link_alt_odb_entries(). Needless to say that the directory cannot be
> accessed using the wrongly normalized path.
>
> Fix it by skipping all of the root part, not just a potential drive
> prefix. offset_1st_component takes care of this, see the
> implementation in compat/mingw.c::mingw_offset_1st_component().
>
> Signed-off-by: Johannes Sixt <j6t@kdbg.org>
> ---
> Am 14.12.2016 um 18:30 schrieb Jeff King:
>> Would it be reasonable to
>> write:
>>
>>    /* Copy initial part of absolute path, converting separators on Windows */
>>    const char *end = src + offset_1st_component(src);
>>    while (src < end) {
>> 	  char c = *src++;
>> 	  if (c == '\\')
>> 		  c = '/';
>> 	  *dst++ = c;
>>    }
> Makes a lot of sense! I haven't had an opportunity, though, to test
> on Windows.
I'm not sure, if a conversion should be done here, in this part of code.
To my knowledge,

C:\dir1\file
is the same
as
C:/dir1/file
and that is handled by windows.

The \\server\share\dir1\file is native to windows,
and I can't see good reasons to change '\' into '/' somewhere in Git,
when UNC is used.

Cygwin does a translation from
//server/share/dir1/file
into
\\server\share\dir1\file

In other words:
The patch looks good as is, and once I get a Windows machine,
may be able to do some testing and come up with test cases


<https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx>
[]

Re: [PATCH v2] fix pushing to //server/share/dir on Windows

[Jeff King]

On Thu, Dec 15, 2016 at 08:30:52AM +0100, Torsten Bögershausen wrote:

> > > Would it be reasonable to
> > > write:
> > > 
> > >    /* Copy initial part of absolute path, converting separators on Windows */
> > >    const char *end = src + offset_1st_component(src);
> > >    while (src < end) {
> > > 	  char c = *src++;
> > > 	  if (c == '\\')
> > > 		  c = '/';
> > > 	  *dst++ = c;
> > >    }
> > Makes a lot of sense! I haven't had an opportunity, though, to test
> > on Windows.
> I'm not sure, if a conversion should be done here, in this part of code.
> To my knowledge,
> 
> C:\dir1\file
> is the same
> as
> C:/dir1/file
> and that is handled by windows.

I don't have an opinion either way on what Windows would want, but note
that the function already _does_ convert separators to slashes. With
Johannes's original patch, you'd end up with a mix, like:

  \\server\share/dir1/file

So this conversion is really just retaining the original behavior, and
making it consistent throughout the path.

Which isn't to say that the function as it currently exists isn't a
little bit buggy. :)

One of the points of normalizing, though, is that Git can then do
textual comparisons between the output. So I think there's value in
having a canonical internal representation, even if the OS could handle
more exotic ones.

-Peff

Re: [PATCH v2] fix pushing to //server/share/dir on Windows

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I don't have an opinion either way on what Windows would want, but note
> that the function already _does_ convert separators to slashes. With
> Johannes's original patch, you'd end up with a mix, like:
>
>   \\server\share/dir1/file
>
> So this conversion is really just retaining the original behavior, and
> making it consistent throughout the path.

>
> Which isn't to say that the function as it currently exists isn't a
> little bit buggy. :)
>
> One of the points of normalizing, though, is that Git can then do
> textual comparisons between the output. So I think there's value in
> having a canonical internal representation, even if the OS could handle
> more exotic ones.

E.g. the log message of d53c2c6738 ("mingw: fix t9700's assumption
about directory separators", 2016-01-27) says the two kinds of
slashes are equivalent over there, but the patch text ends up doing
exactly that normalization.

5ca6b7bb47 ("config --show-origin: report paths with forward
slashes", 2016-03-23) is an example of us trying to normalize in
order to give consistent output to the users.

Having said all that, I do not have an opinion either way on what
Windows would want, either ;-)

Re: [PATCH] fix pushing to //server/share/dir paths on Windows

[Johannes Schindelin]

Hi Hannes,

On Tue, 13 Dec 2016, Johannes Sixt wrote:

>  Dscho, it looks like this could fix the original report at
>  https://github.com/git-for-windows/git/issues/979

Yes, thank you so much for fixing it!

I think your v2 is safe enough to cherry-pick directly into Git for
Windows' `master`.

Thanks,
Dscho