[PATCH 1/2] submodule: ignore trailing slash on superproject URL

git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed

* [PATCH 1/2] submodule: ignore trailing slash on superproject URL
@ 2016-10-10 17:56 Stefan Beller
  2016-10-10 17:56 ` [PATCH 2/2] submodule: ignore trailing slash in relative url Stefan Beller
                   ` (2 more replies)
  0 siblings, 3 replies; 14+ messages in thread
From: Stefan Beller @ 2016-10-10 17:56 UTC (permalink / raw)
  To: gitster; +Cc: git, venv21, dennis, Stefan Beller

Before 63e95beb0 (2016-04-15, submodule: port resolve_relative_url from
shell to C), it did not matter if the superprojects URL had a trailing
slash or not. It was just chopped off as one of the first steps
(The "remoteurl=${remoteurl%/}" near the beginning of
resolve_relative_url(), which was removed in said commit).

When porting this to the C version, an off-by-one error was introduced
and we did not check the actual last character to be a slash, but the
NULL delimiter.

Reintroduce the behavior from before 63e95beb0, to ignore the trailing
slash.

Reported-by: <venv21@gmail.com>
Helped-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 6 ++++--
 t/t0060-path-utils.sh       | 1 +
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 444ec06..a7841a5 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -95,6 +95,8 @@ static int chop_last_dir(char **remoteurl, int is_relative)
  * NEEDSWORK: This works incorrectly on the domain and protocol part.
  * remote_url      url              outcome          expectation
  * http://a.com/b  ../c             http://a.com/c   as is
+ * http://a.com/b/ ../c             http://a.com/c   same as previous line, but
+ *                                                   ignore trailing slash in url
  * http://a.com/b  ../../c          http://c         error out
  * http://a.com/b  ../../../c       http:/c          error out
  * http://a.com/b  ../../../../c    http:c           error out
@@ -113,8 +115,8 @@ static char *relative_url(const char *remote_url,
 	struct strbuf sb = STRBUF_INIT;
 	size_t len = strlen(remoteurl);
 
-	if (is_dir_sep(remoteurl[len]))
-		remoteurl[len] = '\0';
+	if (is_dir_sep(remoteurl[len-1]))
+		remoteurl[len-1] = '\0';
 
 	if (!url_is_local_not_ssh(remoteurl) || is_absolute_path(remoteurl))
 		is_relative = 0;
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index bf2deee..82b98f8 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -319,6 +319,7 @@ test_submodule_relative_url "../" "foo/bar" "../submodule" "../foo/submodule"
 test_submodule_relative_url "../" "foo" "../submodule" "../submodule"
 
 test_submodule_relative_url "(null)" "../foo/bar" "../sub/a/b/c" "../foo/sub/a/b/c"
+test_submodule_relative_url "(null)" "../foo/bar/" "../sub/a/b/c" "../foo/sub/a/b/c"
 test_submodule_relative_url "(null)" "../foo/bar" "../submodule" "../foo/submodule"
 test_submodule_relative_url "(null)" "../foo/submodule" "../submodule" "../foo/submodule"
 test_submodule_relative_url "(null)" "../foo" "../submodule" "../submodule"
-- 
2.10.1.382.ga23ca1b.dirty


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH 2/2] submodule: ignore trailing slash in relative url
  2016-10-10 17:56 [PATCH 1/2] submodule: ignore trailing slash on superproject URL Stefan Beller
@ 2016-10-10 17:56 ` Stefan Beller
  2016-10-10 19:58 ` [PATCH 1/2] submodule: ignore trailing slash on superproject URL Dennis Kaarsemaker
  2016-10-12 13:30 ` Johannes Schindelin
  2 siblings, 0 replies; 14+ messages in thread
From: Stefan Beller @ 2016-10-10 17:56 UTC (permalink / raw)
  To: gitster; +Cc: git, venv21, dennis, Stefan Beller

This is similar to the previous patch, though no user reported a bug and
I could not find a regressive behavior.

However it is a good thing to be strict on the output and for that we
always omit a trailing slash.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 2 ++
 t/t0060-path-utils.sh       | 1 +
 2 files changed, 3 insertions(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index a7841a5..260f46f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -149,6 +149,8 @@ static char *relative_url(const char *remote_url,
 	}
 	strbuf_reset(&sb);
 	strbuf_addf(&sb, "%s%s%s", remoteurl, colonsep ? ":" : "/", url);
+	if (ends_with(url, "/"))
+		strbuf_setlen(&sb, sb.len - 1);
 	free(remoteurl);
 
 	if (starts_with_dot_slash(sb.buf))
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index 82b98f8..25b48e5 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -319,6 +319,7 @@ test_submodule_relative_url "../" "foo/bar" "../submodule" "../foo/submodule"
 test_submodule_relative_url "../" "foo" "../submodule" "../submodule"
 
 test_submodule_relative_url "(null)" "../foo/bar" "../sub/a/b/c" "../foo/sub/a/b/c"
+test_submodule_relative_url "(null)" "../foo/bar" "../sub/a/b/c/" "../foo/sub/a/b/c"
 test_submodule_relative_url "(null)" "../foo/bar/" "../sub/a/b/c" "../foo/sub/a/b/c"
 test_submodule_relative_url "(null)" "../foo/bar" "../submodule" "../foo/submodule"
 test_submodule_relative_url "(null)" "../foo/submodule" "../submodule" "../foo/submodule"
-- 
2.10.1.382.ga23ca1b.dirty


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-10 17:56 [PATCH 1/2] submodule: ignore trailing slash on superproject URL Stefan Beller
  2016-10-10 17:56 ` [PATCH 2/2] submodule: ignore trailing slash in relative url Stefan Beller
@ 2016-10-10 19:58 ` Dennis Kaarsemaker
  2016-10-12 13:30 ` Johannes Schindelin
  2 siblings, 0 replies; 14+ messages in thread
From: Dennis Kaarsemaker @ 2016-10-10 19:58 UTC (permalink / raw)
  To: Stefan Beller, gitster; +Cc: git, venv21

[And now with CC to the list, sorry Stefan]

On Mon, 2016-10-10 at 10:56 -0700, Stefan Beller wrote:
> Before 63e95beb0 (2016-04-15, submodule: port resolve_relative_url from
> shell to C), it did not matter if the superprojects URL had a trailing
> slash or not. It was just chopped off as one of the first steps
> (The "remoteurl=${remoteurl%/}" near the beginning of
> resolve_relative_url(), which was removed in said commit).
> 
> When porting this to the C version, an off-by-one error was introduced
> and we did not check the actual last character to be a slash, but the
> NULL delimiter.
> 
> Reintroduce the behavior from before 63e95beb0, to ignore the trailing
> slash.

Looks good to me, and fixes my simple testcase and cloning epiphany
with trailing slash. Thanks!

D.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-10 17:56 [PATCH 1/2] submodule: ignore trailing slash on superproject URL Stefan Beller
  2016-10-10 17:56 ` [PATCH 2/2] submodule: ignore trailing slash in relative url Stefan Beller
  2016-10-10 19:58 ` [PATCH 1/2] submodule: ignore trailing slash on superproject URL Dennis Kaarsemaker
@ 2016-10-12 13:30 ` Johannes Schindelin
  2016-10-12 17:06   ` Stefan Beller
  2 siblings, 1 reply; 14+ messages in thread
From: Johannes Schindelin @ 2016-10-12 13:30 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, venv21, dennis

Hi Stefan,

On Mon, 10 Oct 2016, Stefan Beller wrote:

> Before 63e95beb0 (2016-04-15, submodule: port resolve_relative_url from
> shell to C), it did not matter if the superprojects URL had a trailing
> slash or not. It was just chopped off as one of the first steps
> (The "remoteurl=${remoteurl%/}" near the beginning of
> resolve_relative_url(), which was removed in said commit).
> 
> When porting this to the C version, an off-by-one error was introduced
> and we did not check the actual last character to be a slash, but the
> NULL delimiter.

It is a NUL delimiter, not a NULL delimiter.

> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 444ec06..a7841a5 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -95,6 +95,8 @@ static int chop_last_dir(char **remoteurl, int is_relative)
>   * NEEDSWORK: This works incorrectly on the domain and protocol part.
>   * remote_url      url              outcome          expectation
>   * http://a.com/b  ../c             http://a.com/c   as is
> + * http://a.com/b/ ../c             http://a.com/c   same as previous line, but
> + *                                                   ignore trailing slash in url
>   * http://a.com/b  ../../c          http://c         error out
>   * http://a.com/b  ../../../c       http:/c          error out
>   * http://a.com/b  ../../../../c    http:c           error out
> @@ -113,8 +115,8 @@ static char *relative_url(const char *remote_url,
>  	struct strbuf sb = STRBUF_INIT;
>  	size_t len = strlen(remoteurl);
>  
> -	if (is_dir_sep(remoteurl[len]))
> -		remoteurl[len] = '\0';
> +	if (is_dir_sep(remoteurl[len-1]))
> +		remoteurl[len-1] = '\0';
>  
>  	if (!url_is_local_not_ssh(remoteurl) || is_absolute_path(remoteurl))
>  		is_relative = 0;
> diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
> index bf2deee..82b98f8 100755
> --- a/t/t0060-path-utils.sh
> +++ b/t/t0060-path-utils.sh
> @@ -319,6 +319,7 @@ test_submodule_relative_url "../" "foo/bar" "../submodule" "../foo/submodule"
>  test_submodule_relative_url "../" "foo" "../submodule" "../submodule"
>  
>  test_submodule_relative_url "(null)" "../foo/bar" "../sub/a/b/c" "../foo/sub/a/b/c"
> +test_submodule_relative_url "(null)" "../foo/bar/" "../sub/a/b/c" "../foo/sub/a/b/c"
>  test_submodule_relative_url "(null)" "../foo/bar" "../submodule" "../foo/submodule"
>  test_submodule_relative_url "(null)" "../foo/submodule" "../submodule" "../foo/submodule"
>  test_submodule_relative_url "(null)" "../foo" "../submodule" "../submodule"

I see that this already made it to `next`. I saw that because it breaks
the build of Git for Windows (this was not noticed earlier because other
compile failures prevented the tests from running), as now the test cases
173 and 177 of t0060 fail (*not* the newly introduced 163).

Here is the output with -v -x:

-- snip --
[...]
expecting success:
                actual=$(git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' '../.') &&
                test "$actual" = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'

+++ git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' ../.
++ actual=C:/git-sdk-64/usr/src/git/wip/t/.
++ test C:/git-sdk-64/usr/src/git/wip/t/. = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'
error: last command exited with $?=1
not ok 172 - test_submodule_relative_url: (null) /usr/src/git/wip/t/trash directory.t0060-path-utils/. ../. => C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.
#
#                       actual=$(git submodule--helper
#                       resolve-relative-url-test '(null)'
#                       '/usr/src/git/wip/t/trash
#                       directory.t0060-path-utils/.' '../.') &&
#                       test "$actual" =
#                       'C:/git-sdk-64/usr/src/git/wip/t/trash
#                       directory.t0060-path-utils/.'
#
[...]
expecting success:
                actual=$(git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' '../submodule') &&
                test "$actual" = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/submodule'

+++ git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' ../submodule
++ actual=C:/git-sdk-64/usr/src/git/wip/t/submodule
++ test C:/git-sdk-64/usr/src/git/wip/t/submodule = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/submodule'
error: last command exited with $?=1
not ok 176 - test_submodule_relative_url: (null) /usr/src/git/wip/t/trash directory.t0060-path-utils/. ../submodule => C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/submodule
#
#                       actual=$(git submodule--helper
#                       resolve-relative-url-test '(null)'
#                       '/usr/src/git/wip/t/trash
#                       directory.t0060-path-utils/.' '../submodule') &&
#                       test "$actual" =
#                       'C:/git-sdk-64/usr/src/git/wip/t/trash
#                       directory.t0060-path-utils/submodule'
#
[...]
-- snap --

For comparison, this is how it succeeds in an Ubuntu VM:

-- snap --
expecting success:
                actual=$(git submodule--helper resolve-relative-url-test '(null)' '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.' '../.') &&
                test "$actual" = '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.'

+++ git submodule--helper resolve-relative-url-test '(null)' '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.' ../.
++ actual='/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.'
++ test '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.' = '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.'
+ test_eval_ret_=0
+ want_trace
+ test t = t
+ test t = t
+ set +x
ok 173 - test_submodule_relative_url: (null) /home/virtualbox/git/wip/t/trash directory.t0060-path-utils/. ../. => /home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.
-- snap --

The reason that this fails on Windows is that the POSIX->Windows path
mangling of the MSYS2 shell strips the trailing . from "/some/directory/."
when converting it to "C:/git-sdk-64/some/directory", and for a good
reason: most Windows programs do not handle the trailing "." very well.

One very, very ugly workaround for this newly-introduced breakage would be
this:

-- snip --
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index 82b98f8..abd82e9 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -328,11 +328,11 @@ test_submodule_relative_url "(null)" "./foo" "../submodule" "submodule"
 test_submodule_relative_url "(null)" "//somewhere else/repo" "../subrepo" "//somewhere else/subrepo"
 test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
 test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
-test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."
+test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."
 test_submodule_relative_url "(null)" "$PWD" "./." "$(pwd)/."
 test_submodule_relative_url "(null)" "$PWD/addtest" "../repo" "$(pwd)/repo"
 test_submodule_relative_url "(null)" "$PWD" "./ " "$(pwd)/ "
-test_submodule_relative_url "(null)" "$PWD/." "../submodule" "$(pwd)/submodule"
+test_submodule_relative_url "(null)" "$(pwd)/." "../submodule" "$(pwd)/submodule"
 test_submodule_relative_url "(null)" "$PWD/submodule" "../submodule" "$(pwd)/submodule"
 test_submodule_relative_url "(null)" "$PWD/home2/../remote" "../bundle1" "$(pwd)/home2/../bundle1"
 test_submodule_relative_url "(null)" "$PWD/submodule_update_repo" "./." "$(pwd)/submodule_update_repo/."
-- snap --

The reasons this is ugly: we specifically test for *Unixy* paths when we
use $PWD, as opposed to *Windowsy* paths when using $(pwd). We do this to
ensure a certain level of confidence that running things such as

	git clone --recurse-submodules /z/project/.

work. And now that does not work anymore.

So where to go from here?

Ciao,
Dscho

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-12 13:30 ` Johannes Schindelin
@ 2016-10-12 17:06   ` Stefan Beller
  2016-10-13 11:11     ` Johannes Schindelin
  0 siblings, 1 reply; 14+ messages in thread
From: Stefan Beller @ 2016-10-12 17:06 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Junio C Hamano, git@vger.kernel.org, Karl A., Dennis Kaarsemaker

On Wed, Oct 12, 2016 at 6:30 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Stefan,
>
> On Mon, 10 Oct 2016, Stefan Beller wrote:
>
>> Before 63e95beb0 (2016-04-15, submodule: port resolve_relative_url from
>> shell to C), it did not matter if the superprojects URL had a trailing
>> slash or not. It was just chopped off as one of the first steps
>> (The "remoteurl=${remoteurl%/}" near the beginning of
>> resolve_relative_url(), which was removed in said commit).
>>
>> When porting this to the C version, an off-by-one error was introduced
>> and we did not check the actual last character to be a slash, but the
>> NULL delimiter.
>
> It is a NUL delimiter, not a NULL delimiter.

 null character ('\0', called NUL in ASCII)
I see. Thanks for pointing out.

>
>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
>> index 444ec06..a7841a5 100644
>> --- a/builtin/submodule--helper.c
>> +++ b/builtin/submodule--helper.c
>> @@ -95,6 +95,8 @@ static int chop_last_dir(char **remoteurl, int is_relative)
>>   * NEEDSWORK: This works incorrectly on the domain and protocol part.
>>   * remote_url      url              outcome          expectation
>>   * http://a.com/b  ../c             http://a.com/c   as is
>> + * http://a.com/b/ ../c             http://a.com/c   same as previous line, but
>> + *                                                   ignore trailing slash in url
>>   * http://a.com/b  ../../c          http://c         error out
>>   * http://a.com/b  ../../../c       http:/c          error out
>>   * http://a.com/b  ../../../../c    http:c           error out
>> @@ -113,8 +115,8 @@ static char *relative_url(const char *remote_url,
>>       struct strbuf sb = STRBUF_INIT;
>>       size_t len = strlen(remoteurl);
>>
>> -     if (is_dir_sep(remoteurl[len]))
>> -             remoteurl[len] = '\0';
>> +     if (is_dir_sep(remoteurl[len-1]))
>> +             remoteurl[len-1] = '\0';
>>
>>       if (!url_is_local_not_ssh(remoteurl) || is_absolute_path(remoteurl))
>>               is_relative = 0;
>> diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
>> index bf2deee..82b98f8 100755
>> --- a/t/t0060-path-utils.sh
>> +++ b/t/t0060-path-utils.sh
>> @@ -319,6 +319,7 @@ test_submodule_relative_url "../" "foo/bar" "../submodule" "../foo/submodule"
>>  test_submodule_relative_url "../" "foo" "../submodule" "../submodule"
>>
>>  test_submodule_relative_url "(null)" "../foo/bar" "../sub/a/b/c" "../foo/sub/a/b/c"
>> +test_submodule_relative_url "(null)" "../foo/bar/" "../sub/a/b/c" "../foo/sub/a/b/c"
>>  test_submodule_relative_url "(null)" "../foo/bar" "../submodule" "../foo/submodule"
>>  test_submodule_relative_url "(null)" "../foo/submodule" "../submodule" "../foo/submodule"
>>  test_submodule_relative_url "(null)" "../foo" "../submodule" "../submodule"
>
> I see that this already made it to `next`. I saw that because it breaks
> the build of Git for Windows (this was not noticed earlier because other
> compile failures prevented the tests from running), as now the test cases
> 173 and 177 of t0060 fail (*not* the newly introduced 163).
>
> Here is the output with -v -x:
>
> -- snip --
> [...]
> expecting success:
>                 actual=$(git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' '../.') &&
>                 test "$actual" = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'
>
> +++ git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' ../.
> ++ actual=C:/git-sdk-64/usr/src/git/wip/t/.
> ++ test C:/git-sdk-64/usr/src/git/wip/t/. = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'

So this wipes away one dir too much in a test that doesn't end with a
dir separator
(In Windows that is '/' and '\' only, no dots?)

> error: last command exited with $?=1
> not ok 172 - test_submodule_relative_url: (null) /usr/src/git/wip/t/trash directory.t0060-path-utils/. ../. => C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.
> #
> #                       actual=$(git submodule--helper
> #                       resolve-relative-url-test '(null)'
> #                       '/usr/src/git/wip/t/trash
> #                       directory.t0060-path-utils/.' '../.') &&
> #                       test "$actual" =
> #                       'C:/git-sdk-64/usr/src/git/wip/t/trash
> #                       directory.t0060-path-utils/.'
> #
> [...]
> expecting success:
>                 actual=$(git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' '../submodule') &&
>                 test "$actual" = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/submodule'
>
> +++ git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' ../submodule
> ++ actual=C:/git-sdk-64/usr/src/git/wip/t/submodule
> ++ test C:/git-sdk-64/usr/src/git/wip/t/submodule = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/submodule'
> error: last command exited with $?=1
> not ok 176 - test_submodule_relative_url: (null) /usr/src/git/wip/t/trash directory.t0060-path-utils/. ../submodule => C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/submodule
> #
> #                       actual=$(git submodule--helper
> #                       resolve-relative-url-test '(null)'
> #                       '/usr/src/git/wip/t/trash
> #                       directory.t0060-path-utils/.' '../submodule') &&
> #                       test "$actual" =
> #                       'C:/git-sdk-64/usr/src/git/wip/t/trash
> #                       directory.t0060-path-utils/submodule'
> #
> [...]
> -- snap --
>
> For comparison, this is how it succeeds in an Ubuntu VM:
>
> -- snap --
> expecting success:
>                 actual=$(git submodule--helper resolve-relative-url-test '(null)' '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.' '../.') &&
>                 test "$actual" = '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.'
>
> +++ git submodule--helper resolve-relative-url-test '(null)' '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.' ../.
> ++ actual='/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.'
> ++ test '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.' = '/home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.'
> + test_eval_ret_=0
> + want_trace
> + test t = t
> + test t = t
> + set +x
> ok 173 - test_submodule_relative_url: (null) /home/virtualbox/git/wip/t/trash directory.t0060-path-utils/. ../. => /home/virtualbox/git/wip/t/trash directory.t0060-path-utils/.
> -- snap --
>
> The reason that this fails on Windows is that the POSIX->Windows path
> mangling of the MSYS2 shell strips the trailing . from "/some/directory/."
> when converting it to "C:/git-sdk-64/some/directory", and for a good
> reason: most Windows programs do not handle the trailing "." very well.
>
> One very, very ugly workaround for this newly-introduced breakage would be
> this:
>
> -- snip --
> diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
> index 82b98f8..abd82e9 100755
> --- a/t/t0060-path-utils.sh
> +++ b/t/t0060-path-utils.sh
> @@ -328,11 +328,11 @@ test_submodule_relative_url "(null)" "./foo" "../submodule" "submodule"
>  test_submodule_relative_url "(null)" "//somewhere else/repo" "../subrepo" "//somewhere else/subrepo"
>  test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
>  test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
> -test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."
> +test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."
>  test_submodule_relative_url "(null)" "$PWD" "./." "$(pwd)/."
>  test_submodule_relative_url "(null)" "$PWD/addtest" "../repo" "$(pwd)/repo"
>  test_submodule_relative_url "(null)" "$PWD" "./ " "$(pwd)/ "
> -test_submodule_relative_url "(null)" "$PWD/." "../submodule" "$(pwd)/submodule"
> +test_submodule_relative_url "(null)" "$(pwd)/." "../submodule" "$(pwd)/submodule"
>  test_submodule_relative_url "(null)" "$PWD/submodule" "../submodule" "$(pwd)/submodule"
>  test_submodule_relative_url "(null)" "$PWD/home2/../remote" "../bundle1" "$(pwd)/home2/../bundle1"
>  test_submodule_relative_url "(null)" "$PWD/submodule_update_repo" "./." "$(pwd)/submodule_update_repo/."
> -- snap --
>
> The reasons this is ugly: we specifically test for *Unixy* paths when we
> use $PWD, as opposed to *Windowsy* paths when using $(pwd). We do this to
> ensure a certain level of confidence that running things such as
>
>         git clone --recurse-submodules /z/project/.
>
> work. And now that does not work anymore.

After a while of thinking how I could fix it, it occurs to me, I could
claim the removal of the dot as a defect in the Windows path handling. ;)
But that doesn't help users.

Would it be possible to mark the last dir separator special once the
trailing dot is removed? (i.e. put a \ there, and in this patch we
only check for /)
Sounds hacky to me, though.

>
> So where to go from here?

So IIUC this patch fixed a bug in Git and introduced a very similar bug
in Git for Windows?

I have no expertise on how to deal with these path issues, but it sounds like
this dot-stripping is done too early, i.e. you'd want to first let the
Git part handle
the URL concatenation and stuff and only at the end when it comes to using
the path it should get the Windows treatment?

Thanks,
Stefan

>
> Ciao,
> Dscho

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-12 17:06   ` Stefan Beller
@ 2016-10-13 11:11     ` Johannes Schindelin
  2016-10-17  7:10       ` Junio C Hamano
  0 siblings, 1 reply; 14+ messages in thread
From: Johannes Schindelin @ 2016-10-13 11:11 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Junio C Hamano, git@vger.kernel.org, Karl A., Dennis Kaarsemaker

Hi Stefan,

On Wed, 12 Oct 2016, Stefan Beller wrote:

> On Wed, Oct 12, 2016 at 6:30 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Mon, 10 Oct 2016, Stefan Beller wrote:
> >
> >> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> >> index 444ec06..a7841a5 100644
> >> --- a/builtin/submodule--helper.c
> >> +++ b/builtin/submodule--helper.c
> >> @@ -95,6 +95,8 @@ static int chop_last_dir(char **remoteurl, int is_relative)
> >>   * NEEDSWORK: This works incorrectly on the domain and protocol part.
> >>   * remote_url      url              outcome          expectation
> >>   * http://a.com/b  ../c             http://a.com/c   as is
> >> + * http://a.com/b/ ../c             http://a.com/c   same as previous line, but
> >> + *                                                   ignore trailing slash in url
> >>   * http://a.com/b  ../../c          http://c         error out
> >>   * http://a.com/b  ../../../c       http:/c          error out
> >>   * http://a.com/b  ../../../../c    http:c           error out
> >> @@ -113,8 +115,8 @@ static char *relative_url(const char *remote_url,
> >>       struct strbuf sb = STRBUF_INIT;
> >>       size_t len = strlen(remoteurl);
> >>
> >> -     if (is_dir_sep(remoteurl[len]))
> >> -             remoteurl[len] = '\0';
> >> +     if (is_dir_sep(remoteurl[len-1]))
> >> +             remoteurl[len-1] = '\0';
> >>
> >>       if (!url_is_local_not_ssh(remoteurl) || is_absolute_path(remoteurl))
> >>               is_relative = 0;
> >> diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
> >> index bf2deee..82b98f8 100755
> >> --- a/t/t0060-path-utils.sh
> >> +++ b/t/t0060-path-utils.sh
> >> @@ -319,6 +319,7 @@ test_submodule_relative_url "../" "foo/bar" "../submodule" "../foo/submodule"
> >>  test_submodule_relative_url "../" "foo" "../submodule" "../submodule"
> >>
> >>  test_submodule_relative_url "(null)" "../foo/bar" "../sub/a/b/c" "../foo/sub/a/b/c"
> >> +test_submodule_relative_url "(null)" "../foo/bar/" "../sub/a/b/c" "../foo/sub/a/b/c"
> >>  test_submodule_relative_url "(null)" "../foo/bar" "../submodule" "../foo/submodule"
> >>  test_submodule_relative_url "(null)" "../foo/submodule" "../submodule" "../foo/submodule"
> >>  test_submodule_relative_url "(null)" "../foo" "../submodule" "../submodule"
> >
> > I see that this already made it to `next`. I saw that because it breaks
> > the build of Git for Windows (this was not noticed earlier because other
> > compile failures prevented the tests from running), as now the test cases
> > 173 and 177 of t0060 fail (*not* the newly introduced 163).
> >
> > Here is the output with -v -x:
> >
> > -- snip --
> > [...]
> > expecting success:
> >                 actual=$(git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' '../.') &&
> >                 test "$actual" = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'
> >
> > +++ git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' ../.
> > ++ actual=C:/git-sdk-64/usr/src/git/wip/t/.
> > ++ test C:/git-sdk-64/usr/src/git/wip/t/. = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'
> 
> So this wipes away one dir too much in a test that doesn't end with a
> dir separator

The problem is not *that* simple. You see, on Windows, there are no Unixy
paths (I used to say POSIX but that is not correct, if you think of VMS
paths looking quite a bit different from what Git expects). To appease
Git's assumption about the exact form of paths, the Bash (actually, the
POSIX emulation layer called MSYS2) converts paths of the form
/c/Windows/system32/drivers/etc/hosts to
C:/Windows/system32/drivers/etc/hosts.

Please note that paths that are already in the latter form are not
touched.

And note also that URLs (actually, anything matching "^[A-Za-z]+://") are
*also* not converted.

The paths that *are* converted can also be of the form /etc/passwd, in
which case the path is prefixed with the Windows directory in which whose
usr/bin/ subdirectory the MSYS2 runtime lives.

In that latter case, i.e. Unixy paths being converted to Windows ones, the
very special case of a trailing "/." is truncated to "/" (IIRC there are
some Windows programs that do not take well to "." referring to a
directory, but my memory on that is flakey).

> (In Windows that is '/' and '\' only, no dots?)

Most Windows functions handle forward slashes just fine. Certainly all
functions involved in the code path in question.

> > One very, very ugly workaround for this newly-introduced breakage would be
> > this:
> >
> > -- snip --
> > diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
> > index 82b98f8..abd82e9 100755
> > --- a/t/t0060-path-utils.sh
> > +++ b/t/t0060-path-utils.sh
> > @@ -328,11 +328,11 @@ test_submodule_relative_url "(null)" "./foo" "../submodule" "submodule"
> >  test_submodule_relative_url "(null)" "//somewhere else/repo" "../subrepo" "//somewhere else/subrepo"
> >  test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
> >  test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
> > -test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."
> > +test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."
> >  test_submodule_relative_url "(null)" "$PWD" "./." "$(pwd)/."
> >  test_submodule_relative_url "(null)" "$PWD/addtest" "../repo" "$(pwd)/repo"
> >  test_submodule_relative_url "(null)" "$PWD" "./ " "$(pwd)/ "
> > -test_submodule_relative_url "(null)" "$PWD/." "../submodule" "$(pwd)/submodule"
> > +test_submodule_relative_url "(null)" "$(pwd)/." "../submodule" "$(pwd)/submodule"
> >  test_submodule_relative_url "(null)" "$PWD/submodule" "../submodule" "$(pwd)/submodule"
> >  test_submodule_relative_url "(null)" "$PWD/home2/../remote" "../bundle1" "$(pwd)/home2/../bundle1"
> >  test_submodule_relative_url "(null)" "$PWD/submodule_update_repo" "./." "$(pwd)/submodule_update_repo/."
> > -- snap --
> >
> > The reasons this is ugly: we specifically test for *Unixy* paths when we
> > use $PWD, as opposed to *Windowsy* paths when using $(pwd). We do this to
> > ensure a certain level of confidence that running things such as
> >
> >         git clone --recurse-submodules /z/project/.
> >
> > work. And now that does not work anymore.
> 
> After a while of thinking how I could fix it, it occurs to me, I could
> claim the removal of the dot as a defect in the Windows path handling. ;)

Not *quite*. It is not Windows' path handling. It is MSYS2's path
handling, and they must have had good reasons to introduce it. They do not
strip trailing dots just for fun.

> But that doesn't help users.

Exactly.

> Would it be possible to mark the last dir separator special once the
> trailing dot is removed? (i.e. put a \ there, and in this patch we
> only check for /)
> Sounds hacky to me, though.

We could claim that cloning recursively from absolute, Unixy paths is not
supported on Windows.

Given that it still works with relative paths and with absolute Windows
paths and with URLs, I would claim that this is a fair trade-off.

In which case the ugly patch quoted above may be the best way forward.

> > So where to go from here?
> 
> So IIUC this patch fixed a bug in Git and introduced a very similar bug
> in Git for Windows?

Yep. Even if it fixed the very same bug on Windows, too, as the trailing
dot is kept for URLs and absolute Windows paths.

> I have no expertise on how to deal with these path issues, but it sounds
> like this dot-stripping is done too early, i.e. you'd want to first let
> the Git part handle the URL concatenation and stuff and only at the end
> when it comes to using the path it should get the Windows treatment?

Git has no chance to fix this, as the Git executable (thanks to *not*
using the POSIX emulation layer) gets handed a Windows path without the
trailing dot when called from the Bash.

The same, obviously, goes for `git submodule-helper`: it is not using the
POSIX emulation layer, and therefore that layer converts the paths before
executing said subcommand.

And we cannot easily change the behavior of the MSYS2 runtime, as that
would affect too many other users, most likely breaking the use case that
required the stripping of the trailing dot in the first place.

So I fear that we have to live with the fact that the bug you fixed just
hid a bug on Windows, and that we have to either skip the tests or change
them in the way I proposed.

Or we change the tests to work on a URL instead of a Unixy path.

Opinions?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-13 11:11     ` Johannes Schindelin
@ 2016-10-17  7:10       ` Junio C Hamano
  2016-10-17 17:58         ` Stefan Beller
  2016-10-17 19:32         ` Johannes Sixt
  0 siblings, 2 replies; 14+ messages in thread
From: Junio C Hamano @ 2016-10-17  7:10 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Stefan Beller, git@vger.kernel.org, Karl A., Dennis Kaarsemaker

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> > expecting success:
>> >                 actual=$(git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' '../.') &&
>> >                 test "$actual" = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'
>> >
>> > +++ git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' ../.
>> > ++ actual=C:/git-sdk-64/usr/src/git/wip/t/.
>> > ++ test C:/git-sdk-64/usr/src/git/wip/t/. = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'

This may well be total misunderstanding on my side, but is the
expectation of this test even correct?  If it wants to take "../."
relative to "$LEAD/t/trash utils/.", should't it go one level up
with ".." to $LEAD/t and then stay there with ".", expecting
"$LEAD/t" which is what the above is giving us?

IOW, the above makes me wonder why having one of these as the base

	A - path/to/dir
	B - path/to/dir/
	C - path/to/dir/.

to resolve the relative "../." give different results.  Whether bash
on Windows removes the dot at the end of C to turn it into B, as
long as A and B give us the same result we wouldn't be hitting the
problem, no?

>> >  test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
>> >  test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
>> > -test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."
>> > +test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."

>> > The reasons this is ugly: we specifically test for *Unixy* paths when we
>> > use $PWD, as opposed to *Windowsy* paths when using $(pwd).

Just to ensure I am following this correctly, two tests that come
before the one you are touching above have $PWD on the input side
and $(pwd) on the expectation side.  That is what you mean by the
next paragraph, right?  They want to make sure that you honor the
Unixy user input on Windows and still produce Windowsy result, that
is.

>> > We do this to
>> > ensure a certain level of confidence that running things such as
>> >
>> >         git clone --recurse-submodules /z/project/.
>> >
>> > work. And now that does not work anymore.

And I agree from that point of view that having to spell both sides
as $(pwd) would mean you are not testing that "Unixy input to
Windowsy output" expectation, but at the same time, I think you
would want "Windowsy input to Windowsy output" combination also does
produce correct result, which is not tested in the three tests shown
above.  IOW, probably you would want to test both (at least on any
platform where $PWD and $(pwd) textually disagree) for all these
[*1*], and the pair

    "../." taken relative to "$(pwd)/." must be "$(pwd)/."
    "../." taken relative to "$PWD/." must be "$(pwd)/."

test, because of the limitation of your bash, cannot have the latter
half of the pair, so you'd need to comment it out with in-code
explanation, perhaps?  IOW something along the lines of...

 -- >8 -- snip -- >8 --

test_submodule_relative_url "(null)" "$(pwd)/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
test_submodule_relative_url "(null)" "$(pwd)/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."

if test_have_prereq MINGW
then

test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
# This does not work correctly because Win-Bash strips . at the end
# "of $PWD/."
# test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."

fi

 -- >8 -- snip -- >8 --

In any case, I find it more disturbing that we somehow ended up with
a system where these three things are expected to behave differently:

	A - path/to/dir
	B - path/to/dir/
	C - path/to/dir/.

Is that something we can fix?

[Footnote]

*1* It is tempting to update the above test sequence using
    a helper like:

    tsru () {
	test_submodule_relative_url "(null)" "$(pwd)/$1" "$2" "$(pwd)/$3"
	if test_have_prereq MINGW
	then
	    test_submodule_relative_url "(null)" "$PWD/$1" "$2" "$(pwd)/$3"
	fi
    }

    then write the above three tests like so:

	tsru subsuper_update_r ../subsubsuper_update_r subsubsuper_update_r
	tsru super_update_r2 ../subsuper_update_r subsuper_update_r
	tsru . ../. .

    but you would want to disable the MINGW half for only the third
    test, we cannot quite do that.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-17  7:10       ` Junio C Hamano
@ 2016-10-17 17:58         ` Stefan Beller
  2016-10-17 18:28           ` Junio C Hamano
  2016-10-17 19:32         ` Johannes Sixt
  1 sibling, 1 reply; 14+ messages in thread
From: Stefan Beller @ 2016-10-17 17:58 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin, git@vger.kernel.org, Karl A.,
	Dennis Kaarsemaker

On Mon, Oct 17, 2016 at 12:10 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
>>> > expecting success:
>>> >                 actual=$(git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' '../.') &&
>>> >                 test "$actual" = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'
>>> >
>>> > +++ git submodule--helper resolve-relative-url-test '(null)' '/usr/src/git/wip/t/trash directory.t0060-path-utils/.' ../.
>>> > ++ actual=C:/git-sdk-64/usr/src/git/wip/t/.
>>> > ++ test C:/git-sdk-64/usr/src/git/wip/t/. = 'C:/git-sdk-64/usr/src/git/wip/t/trash directory.t0060-path-utils/.'
>
> This may well be total misunderstanding on my side, but is the
> expectation of this test even correct?  If it wants to take "../."
> relative to "$LEAD/t/trash utils/.", should't it go one level up
> with ".." to $LEAD/t and then stay there with ".", expecting
> "$LEAD/t" which is what the above is giving us?
>
> IOW, the above makes me wonder why having one of these as the base
>
>         A - path/to/dir
>         B - path/to/dir/
>         C - path/to/dir/.
>
> to resolve the relative "../." give different results.

Because the shell script originally did "just"

relative="../."
if path/to/dir/ ends with slash, chop it off.
while $relative starts with "../";
do
    chop off starting '../' of relative
    chop of last '/' and following from "path/to/dir/."
done

(Linux:)
As B was made to A first, only C differs as a result, because
you had one more '/' in there.

(Windows:)
However Windows also detects '/.' (C) and makes it an A,
(in C only, because shell code was not treated Windows-sy)
which is where the incompatibility between Windows and other
platforms arises.

So we have a couple of choices (for Git)now:
* go back to using shell only for submodule things as that doesn't
  have the regression and it alos plays nicely with Git for Windows.
* use C for the submodule code in Git and revert the regression fix
  "because consistency across platforms trumps
  consistency over time"
* use C for the submodule code in Git and keep the regression fix
  "because consistency over time in Git proper is more important
  than playing nicely with Git for Windows"
* would it be possible to revert this to shell on Windows only?

> Whether bash
> on Windows removes the dot at the end of C to turn it into B, as
> long as A and B give us the same result we wouldn't be hitting the
> problem, no?

Well in Git proper A,B are the same and C is different.
(B was fixed as a regression)
In Windows C is like B, which was different without the regression
fix, but now it is the same as A, too.

>
>>> >  test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
>>> >  test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
>>> > -test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."
>>> > +test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."
>
>>> > The reasons this is ugly: we specifically test for *Unixy* paths when we
>>> > use $PWD, as opposed to *Windowsy* paths when using $(pwd).
>
> Just to ensure I am following this correctly, two tests that come
> before the one you are touching above have $PWD on the input side
> and $(pwd) on the expectation side.  That is what you mean by the
> next paragraph, right?  They want to make sure that you honor the
> Unixy user input on Windows and still produce Windowsy result, that
> is.
>
>>> > We do this to
>>> > ensure a certain level of confidence that running things such as
>>> >
>>> >         git clone --recurse-submodules /z/project/.
>>> >
>>> > work. And now that does not work anymore.
>
> And I agree from that point of view that having to spell both sides
> as $(pwd) would mean you are not testing that "Unixy input to
> Windowsy output" expectation, but at the same time, I think you
> would want "Windowsy input to Windowsy output" combination also does
> produce correct result, which is not tested in the three tests shown
> above.  IOW, probably you would want to test both (at least on any
> platform where $PWD and $(pwd) textually disagree) for all these
> [*1*], and the pair
>
>     "../." taken relative to "$(pwd)/." must be "$(pwd)/."
>     "../." taken relative to "$PWD/." must be "$(pwd)/."
>
> test, because of the limitation of your bash, cannot have the latter
> half of the pair, so you'd need to comment it out with in-code
> explanation, perhaps?  IOW something along the lines of...
>
>  -- >8 -- snip -- >8 --
>
> test_submodule_relative_url "(null)" "$(pwd)/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
> test_submodule_relative_url "(null)" "$(pwd)/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
> test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."
>
> if test_have_prereq MINGW
> then
>
> test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
> test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
> # This does not work correctly because Win-Bash strips . at the end
> # "of $PWD/."
> # test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."
>
> fi
>
>  -- >8 -- snip -- >8 --
>
> In any case, I find it more disturbing that we somehow ended up with
> a system where these three things are expected to behave differently:
>
>         A - path/to/dir
>         B - path/to/dir/
>         C - path/to/dir/.
>
> Is that something we can fix?

Well A, B are the same.
C is "obviously" different, when it comes to counting slashes for relative
path/url purposes, in the way that there are characters after the last slash
and just by coincidence '.' refers to the directory itself, C behaving like
'path/to/dir/sub' seems right to me.

So how do you imagine this fix going forward?
* Breaking existing users with /. at the end? by treating it the same as A,B
* Do some check based on time/version of Git and cover the old data?
* Forbid /. at the end from now on?

>
>
> [Footnote]
>
> *1* It is tempting to update the above test sequence using
>     a helper like:
>
>     tsru () {
>         test_submodule_relative_url "(null)" "$(pwd)/$1" "$2" "$(pwd)/$3"
>         if test_have_prereq MINGW
>         then
>             test_submodule_relative_url "(null)" "$PWD/$1" "$2" "$(pwd)/$3"
>         fi
>     }
>
>     then write the above three tests like so:
>
>         tsru subsuper_update_r ../subsubsuper_update_r subsubsuper_update_r
>         tsru super_update_r2 ../subsuper_update_r subsuper_update_r
>         tsru . ../. .
>
>     but you would want to disable the MINGW half for only the third
>     test, we cannot quite do that.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-17 17:58         ` Stefan Beller
@ 2016-10-17 18:28           ` Junio C Hamano
  2016-10-17 18:58             ` Stefan Beller
  0 siblings, 1 reply; 14+ messages in thread
From: Junio C Hamano @ 2016-10-17 18:28 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Johannes Schindelin, git@vger.kernel.org, Karl A.,
	Dennis Kaarsemaker

Stefan Beller <sbeller@google.com> writes:

>> In any case, I find it more disturbing that we somehow ended up with
>> a system where these three things are expected to behave differently:
>>
>>         A - path/to/dir
>>         B - path/to/dir/
>>         C - path/to/dir/.
>>
>> Is that something we can fix?
>
> Well A, B are the same.
> C is "obviously" different, when it comes to counting slashes for relative
> path/url purposes, in the way that there are characters after the last slash
> and just by coincidence '.' refers to the directory itself, C behaving like
> 'path/to/dir/sub' seems right to me.

It doesn't look right to me at all.  If you were contrasting

	cd path/to/dir/sub && cd ..
	cd path/to/dir/bus && cd ..

then I would understand, but why should these two

	cd path/to/dir/. && cd ..
	cd path/to/dir/sub && cd ..

behave the same?

> So how do you imagine this fix going forward?
> * Breaking existing users with /. at the end? by treating it the same as A,B
> * Do some check based on time/version of Git and cover the old data?
> * Forbid /. at the end from now on?

Where at the end-user facing level does this trailing "/." surface
and how does the difference appear to them?  I think that is the
crucial question.

Unless there is some convincing argument why "." is not special
(i.e. counter-argument to the above "bus vs sub" and ". vs sub"
example), I would think "existing users with /." does not matter.
If they are "relying" on the behaviour, I would think it is not
because they find that behaviour intuitive, but only because they
learned to live with it.  IOW, treating all of A/B/C the same way
would appear to them a strict bugfix, I would think.

It is totally a different matter if OUR code that consumes the
output from the submodule-helper --resolve-relative" internally is
confused and relies on "../. relative to path/to/dir/. is the same
as ../. relative to path/to/dir/sub" for whatever reason.  Without
fixing that, I would not surprised if fixing things to treat A/B/C
the same way would surface differences in the end-user observable
behaviour in a negative way.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-17 18:28           ` Junio C Hamano
@ 2016-10-17 18:58             ` Stefan Beller
  2016-10-17 19:16               ` Junio C Hamano
  0 siblings, 1 reply; 14+ messages in thread
From: Stefan Beller @ 2016-10-17 18:58 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin, git@vger.kernel.org, Karl A.,
	Dennis Kaarsemaker

On Mon, Oct 17, 2016 at 11:28 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>> In any case, I find it more disturbing that we somehow ended up with
>>> a system where these three things are expected to behave differently:
>>>
>>>         A - path/to/dir
>>>         B - path/to/dir/
>>>         C - path/to/dir/.
>>>
>>> Is that something we can fix?
>>
>> Well A, B are the same.
>> C is "obviously" different, when it comes to counting slashes for relative
>> path/url purposes, in the way that there are characters after the last slash
>> and just by coincidence '.' refers to the directory itself, C behaving like
>> 'path/to/dir/sub' seems right to me.
>
> It doesn't look right to me at all.  If you were contrasting
>
>         cd path/to/dir/sub && cd ..
>         cd path/to/dir/bus && cd ..
>
> then I would understand, but why should these two
>
>         cd path/to/dir/. && cd ..
>         cd path/to/dir/sub && cd ..
>
> behave the same?
>
>> So how do you imagine this fix going forward?
>> * Breaking existing users with /. at the end? by treating it the same as A,B
>> * Do some check based on time/version of Git and cover the old data?
>> * Forbid /. at the end from now on?
>
> Where at the end-user facing level does this trailing "/." surface
> and how does the difference appear to them?  I think that is the
> crucial question.
>
> Unless there is some convincing argument why "." is not special
> (i.e. counter-argument to the above "bus vs sub" and ". vs sub"
> example), I would think "existing users with /." does not matter.
> If they are "relying" on the behaviour, I would think it is not
> because they find that behaviour intuitive, but only because they
> learned to live with it.  IOW, treating all of A/B/C the same way
> would appear to them a strict bugfix, I would think.

I see, so we should adapt the windows style and chop off '/.'
to make A,B,C all the same, because internally we never produced
C AFAICT.

These came in via hand edited .gitmodules files.

>
> It is totally a different matter if OUR code that consumes the
> output from the submodule-helper --resolve-relative" internally is
> confused and relies on "../. relative to path/to/dir/. is the same
> as ../. relative to path/to/dir/sub" for whatever reason.  Without
> fixing that, I would not surprised if fixing things to treat A/B/C
> the same way would surface differences in the end-user observable
> behaviour in a negative way.
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-17 18:58             ` Stefan Beller
@ 2016-10-17 19:16               ` Junio C Hamano
  0 siblings, 0 replies; 14+ messages in thread
From: Junio C Hamano @ 2016-10-17 19:16 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Johannes Schindelin, git@vger.kernel.org, Karl A.,
	Dennis Kaarsemaker

Stefan Beller <sbeller@google.com> writes:

>> Where at the end-user facing level does this trailing "/." surface
>> and how does the difference appear to them?  I think that is the
>> crucial question.
>>
>> Unless there is some convincing argument why "." is not special
>> (i.e. counter-argument to the above "bus vs sub" and ". vs sub"
>> example), I would think "existing users with /." does not matter.
>> If they are "relying" on the behaviour, I would think it is not
>> because they find that behaviour intuitive, but only because they
>> learned to live with it.  IOW, treating all of A/B/C the same way
>> would appear to them a strict bugfix, I would think.
>
> I see, so we should adapt the windows style and chop off '/.'
> to make A,B,C all the same, because internally we never produced
> C AFAICT.
>
> These came in via hand edited .gitmodules files.

Can you elaborate a bit more on this?  

Without seeing "The user added X/. instead of the usual X because
s/he wanted to see Y happen.  If s/he had X there, Z would have
happened instead of Y" and why being able to expect Y to happen is a
good thing (compared to Z), we may fail to notice why the more
"intuitive" (at least to me) "these three must result in the same
outcome: path/to/dir, path/to/dir/, or path/to/dir/." does not serve
a legitimate use case.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-17  7:10       ` Junio C Hamano
  2016-10-17 17:58         ` Stefan Beller
@ 2016-10-17 19:32         ` Johannes Sixt
  2016-10-17 20:07           ` Junio C Hamano
  2016-10-18 20:06           ` Johannes Sixt
  1 sibling, 2 replies; 14+ messages in thread
From: Johannes Sixt @ 2016-10-17 19:32 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin, Stefan Beller, git@vger.kernel.org, Karl A.,
	Dennis Kaarsemaker

Am 17.10.2016 um 09:10 schrieb Junio C Hamano:
> And I agree from that point of view that having to spell both sides
> as $(pwd) would mean you are not testing that "Unixy input to
> Windowsy output" expectation, but at the same time, I think you
> would want "Windowsy input to Windowsy output" combination also does
> produce correct result, which is not tested in the three tests shown
> above.  IOW, probably you would want to test both (at least on any
> platform where $PWD and $(pwd) textually disagree) for all these
> [*1*], and the pair
>
>     "../." taken relative to "$(pwd)/." must be "$(pwd)/."
>     "../." taken relative to "$PWD/." must be "$(pwd)/."
>
> test, because of the limitation of your bash, cannot have the latter
> half of the pair, so you'd need to comment it out with in-code
> explanation, perhaps?

No, we do not have to test both cases. Git, the program, never sees 
Unixy input. It cannot distinguish the two cases. If we did both tests, 
we would just test that *if* the front-end of git is an MSYS program 
(such as bash), *then* that MSYS front-end has converted the Unixy path 
to a Windows-y path in such a way that the end-result is also as 
expected. It's similar to a test that grep produces expected output.

I think that we could reduce the confusion by converting all $PWD to 
$(pwd) in these test cases. I don't remember why I suggested to use $PWD 
for one of the arguments of the test cases (the second must be $(pwd)), 
but the most likely reason is only that we save a process.

-- Hannes

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-17 19:32         ` Johannes Sixt
@ 2016-10-17 20:07           ` Junio C Hamano
  2016-10-18 20:06           ` Johannes Sixt
  1 sibling, 0 replies; 14+ messages in thread
From: Junio C Hamano @ 2016-10-17 20:07 UTC (permalink / raw)
  To: Johannes Sixt
  Cc: Johannes Schindelin, Stefan Beller, git@vger.kernel.org, Karl A.,
	Dennis Kaarsemaker

Johannes Sixt <j6t@kdbg.org> writes:

>>     "../." taken relative to "$(pwd)/." must be "$(pwd)/."
>>     "../." taken relative to "$PWD/." must be "$(pwd)/."
>>
>> test, because of the limitation of your bash, cannot have the latter
>> half of the pair, so you'd need to comment it out with in-code
>> explanation, perhaps?
>
> No, we do not have to test both cases. Git, the program, never sees
> Unixy input. It cannot distinguish the two cases.

Thanks.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 1/2] submodule: ignore trailing slash on superproject URL
  2016-10-17 19:32         ` Johannes Sixt
  2016-10-17 20:07           ` Junio C Hamano
@ 2016-10-18 20:06           ` Johannes Sixt
  1 sibling, 0 replies; 14+ messages in thread
From: Johannes Sixt @ 2016-10-18 20:06 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin, Stefan Beller, git@vger.kernel.org, Karl A.,
	Dennis Kaarsemaker

Am 17.10.2016 um 21:32 schrieb Johannes Sixt:
> I think that we could reduce the confusion by converting all $PWD to
> $(pwd) in these test cases. I don't remember why I suggested to use $PWD
> for one of the arguments of the test cases (the second must be $(pwd)),
> but the most likely reason is only that we save a process.

Something like this works for me on Windows. I can update the patch
after the "normalize funny urls" topic settles.

---- 8< ----
t0060: sidestep surprising path mangling results on Windows

When an MSYS program (such as the bash that drives the test suite)
invokes git on Windows, absolute Unix style paths are transformed into
Windows native absolute paths (drive letter form). However, this
transformation also includes some simplifications that are not just
straight-forward textual substitutions:

- When the path ends in "/.", then the dot is stripped, but not the
  directory separator.

- When the path contains "..", then it is optimized away if possible,
  e.g., "/c/dir/foo/../bar" becomes "c:/dir/bar".

These additional transformations violate the assumptions of some
submodule path tests. We can avoid them when the input is already a
Windows native path, because then MSYS leaves the path unmolested.

Convert the uses of $PWD to $(pwd); the latter returns a native Windows
path.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 t/t0060-path-utils.sh | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index 25b48e5..444b5a4 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -305,8 +305,9 @@ test_git_path GIT_COMMON_DIR=bar config                   bar/config
 test_git_path GIT_COMMON_DIR=bar packed-refs              bar/packed-refs
 test_git_path GIT_COMMON_DIR=bar shallow                  bar/shallow
 
-# In the tests below, the distinction between $PWD and $(pwd) is important:
-# on Windows, $PWD is POSIX style (/c/foo), $(pwd) has drive letter (c:/foo).
+# In the tests below, $(pwd) must be used because it is a native path on
+# Windows and avoids MSYS's path mangling (which simplifies "foo/../bar" and
+# strips the dot from trailing "/.").
 
 test_submodule_relative_url "../" "../foo" "../submodule" "../../submodule"
 test_submodule_relative_url "../" "../foo/bar" "../submodule" "../../foo/submodule"
@@ -314,7 +315,7 @@ test_submodule_relative_url "../" "../foo/submodule" "../submodule" "../../foo/s
 test_submodule_relative_url "../" "./foo" "../submodule" "../submodule"
 test_submodule_relative_url "../" "./foo/bar" "../submodule" "../foo/submodule"
 test_submodule_relative_url "../../../" "../foo/bar" "../sub/a/b/c" "../../../../foo/sub/a/b/c"
-test_submodule_relative_url "../" "$PWD/addtest" "../repo" "$(pwd)/repo"
+test_submodule_relative_url "../" "$(pwd)/addtest" "../repo" "$(pwd)/repo"
 test_submodule_relative_url "../" "foo/bar" "../submodule" "../foo/submodule"
 test_submodule_relative_url "../" "foo" "../submodule" "../submodule"
 
@@ -327,16 +328,16 @@ test_submodule_relative_url "(null)" "../foo" "../submodule" "../submodule"
 test_submodule_relative_url "(null)" "./foo/bar" "../submodule" "foo/submodule"
 test_submodule_relative_url "(null)" "./foo" "../submodule" "submodule"
 test_submodule_relative_url "(null)" "//somewhere else/repo" "../subrepo" "//somewhere else/subrepo"
-test_submodule_relative_url "(null)" "$PWD/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
-test_submodule_relative_url "(null)" "$PWD/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
-test_submodule_relative_url "(null)" "$PWD/." "../." "$(pwd)/."
-test_submodule_relative_url "(null)" "$PWD" "./." "$(pwd)/."
-test_submodule_relative_url "(null)" "$PWD/addtest" "../repo" "$(pwd)/repo"
-test_submodule_relative_url "(null)" "$PWD" "./å äö" "$(pwd)/å äö"
-test_submodule_relative_url "(null)" "$PWD/." "../submodule" "$(pwd)/submodule"
-test_submodule_relative_url "(null)" "$PWD/submodule" "../submodule" "$(pwd)/submodule"
-test_submodule_relative_url "(null)" "$PWD/home2/../remote" "../bundle1" "$(pwd)/home2/../bundle1"
-test_submodule_relative_url "(null)" "$PWD/submodule_update_repo" "./." "$(pwd)/submodule_update_repo/."
+test_submodule_relative_url "(null)" "$(pwd)/subsuper_update_r" "../subsubsuper_update_r" "$(pwd)/subsubsuper_update_r"
+test_submodule_relative_url "(null)" "$(pwd)/super_update_r2" "../subsuper_update_r" "$(pwd)/subsuper_update_r"
+test_submodule_relative_url "(null)" "$(pwd)/." "../." "$(pwd)/."
+test_submodule_relative_url "(null)" "$(pwd)" "./." "$(pwd)/."
+test_submodule_relative_url "(null)" "$(pwd)/addtest" "../repo" "$(pwd)/repo"
+test_submodule_relative_url "(null)" "$(pwd)" "./å äö" "$(pwd)/å äö"
+test_submodule_relative_url "(null)" "$(pwd)/." "../submodule" "$(pwd)/submodule"
+test_submodule_relative_url "(null)" "$(pwd)/submodule" "../submodule" "$(pwd)/submodule"
+test_submodule_relative_url "(null)" "$(pwd)/home2/../remote" "../bundle1" "$(pwd)/home2/../bundle1"
+test_submodule_relative_url "(null)" "$(pwd)/submodule_update_repo" "./." "$(pwd)/submodule_update_repo/."
 test_submodule_relative_url "(null)" "file:///tmp/repo" "../subrepo" "file:///tmp/subrepo"
 test_submodule_relative_url "(null)" "foo/bar" "../submodule" "foo/submodule"
 test_submodule_relative_url "(null)" "foo" "../submodule" "submodule"
-- 
2.10.0.343.g37bc62b


^ permalink raw reply related	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2016-10-18 20:06 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-10 17:56 [PATCH 1/2] submodule: ignore trailing slash on superproject URL Stefan Beller
2016-10-10 17:56 ` [PATCH 2/2] submodule: ignore trailing slash in relative url Stefan Beller
2016-10-10 19:58 ` [PATCH 1/2] submodule: ignore trailing slash on superproject URL Dennis Kaarsemaker
2016-10-12 13:30 ` Johannes Schindelin
2016-10-12 17:06   ` Stefan Beller
2016-10-13 11:11     ` Johannes Schindelin
2016-10-17  7:10       ` Junio C Hamano
2016-10-17 17:58         ` Stefan Beller
2016-10-17 18:28           ` Junio C Hamano
2016-10-17 18:58             ` Stefan Beller
2016-10-17 19:16               ` Junio C Hamano
2016-10-17 19:32         ` Johannes Sixt
2016-10-17 20:07           ` Junio C Hamano
2016-10-18 20:06           ` Johannes Sixt

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).