[PATCH v5 0/2] refs: cleanup and new behavior

[PATCH v5 0/2] refs: cleanup and new behavior

[Jacob Keller]

From: Jacob Keller <jacob.keller@gmail.com>

As per Junio's suggestion, break cleanup/fix and new functionality into
separate patches. Also update description of the new functionality.

The first patch is entirely cleanup of comments with no functionality
change at all (indeed only changes to comment text!). It now correctly
highlights all characters which are disallowed. Discovered by creating a
test function that printed out the whole list.

The second patch introduces the new functionality for "*" and details
how it is now checked per-component. It also explains how the flags are
now passed as a pointer so that we can reject any multi-"*" reference,
by clearing the REFNAME_REFSPEC_PATTERN bit.

Change since v4
- split the cleanup to a separate patch and included all missing values
  from the disposition "4" on the comments.

Jacob Keller (2):
  refs: cleanup comments regarding check_refname_component
  refs: loosen restriction on wildcard "*" refspecs

 Documentation/git-check-ref-format.txt |  4 ++--
 refs.c                                 | 44 +++++++++++++++++++---------------
 refs.h                                 |  4 ++--
 t/t1402-check-ref-format.sh            |  8 ++++---
 4 files changed, 34 insertions(+), 26 deletions(-)

-- 
2.4.3

[PATCH 1/2] refs: cleanup comments regarding check_refname_component

[Jacob Keller]

From: Jacob Keller <jacob.keller@gmail.com>

Correctly specify all characters which are rejected under the '4'
disposition, including '*' even though it does gain special treatment by
callers of check_refname_component.

Cleanup comment style for rejected refs by inserting a ", or" at the end
of each statement.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Cc: Daniel Barkalow <barkalow@iabervon.iabervon.org>
Cc: Junio C Hamano <gitster@pobox.com>
---
 refs.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index ce8cd8d45001..8c08fc0489eb 100644
--- a/refs.c
+++ b/refs.c
@@ -19,7 +19,8 @@ struct ref_lock {
  * 1: End-of-component
  * 2: ., look for a preceding . to reject .. in refs
  * 3: {, look for a preceding @ to reject @{ in refs
- * 4: A bad character: ASCII control characters, "~", "^", ":" or SP
+ * 4: A bad character: ASCII control characters, and
+ *    "*", ":", "?", "[", "\", "^", "~", SP, or TAB
  */
 static unsigned char refname_disposition[256] = {
 	1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
@@ -70,10 +71,11 @@ static unsigned char refname_disposition[256] = {
  *
  * - any path component of it begins with ".", or
  * - it has double dots "..", or
- * - it has ASCII control character, "~", "^", ":" or SP, anywhere, or
- * - it ends with a "/".
- * - it ends with ".lock"
- * - it contains a "\" (backslash)
+ * - it has ASCII control characters, or
+ * - it has "*", ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
+ * - it ends with a "/", or
+ * - it ends with ".lock", or
+ * - it contains a "@{" portion
  */
 static int check_refname_component(const char *refname, int flags)
 {
-- 
2.4.3

[PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Jacob Keller]

From: Jacob Keller <jacob.keller@gmail.com>

Modify logic of check_refname_component and add a new disposition
regarding "*". Allow refspecs that contain a single "*" if
REFNAME_REFSPEC_PATTERN is set. Change the function to pass the flags as
a pointer, and clear REFNAME_REFSPEC_PATTERN after the first "*" so that
only a single "*" is accepted.

This loosens restrictions on refspecs by allowing patterns that have
a "*" within a component instead of only as the whole component. Also
remove the code that hangled refspec patterns in check_refname_format
since it is now handled via the check_refname_component logic.

Now refs such as `for/bar*:foo/bar*` and even `foo/bar*:foo/baz*` will
be accepted. This allows users more control over what is fetched where.
Since users must modify the default by hand to make use of this
functionality it is not considered a large risk. Any refspec which
functioned before shall continue functioning with the new logic.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Cc: Daniel Barkalow <barkalow@iabervon.iabervon.org>
Cc: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-check-ref-format.txt |  4 ++--
 refs.c                                 | 36 +++++++++++++++++++---------------
 refs.h                                 |  4 ++--
 t/t1402-check-ref-format.sh            |  8 +++++---
 4 files changed, 29 insertions(+), 23 deletions(-)

diff --git a/Documentation/git-check-ref-format.txt b/Documentation/git-check-ref-format.txt
index fc02959ba4ab..9044dfaadae1 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -94,8 +94,8 @@ OPTIONS
 	Interpret <refname> as a reference name pattern for a refspec
 	(as used with remote repositories).  If this option is
 	enabled, <refname> is allowed to contain a single `*`
-	in place of a one full pathname component (e.g.,
-	`foo/*/bar` but not `foo/bar*`).
+	in the refspec (e.g., `foo/bar*/baz` or `foo/bar*baz/`
+	but not `foo/bar*/baz*`).
 
 --normalize::
 	Normalize 'refname' by removing any leading slash (`/`)
diff --git a/refs.c b/refs.c
index 8c08fc0489eb..16a1d8305579 100644
--- a/refs.c
+++ b/refs.c
@@ -20,12 +20,13 @@ struct ref_lock {
  * 2: ., look for a preceding . to reject .. in refs
  * 3: {, look for a preceding @ to reject @{ in refs
  * 4: A bad character: ASCII control characters, and
- *    "*", ":", "?", "[", "\", "^", "~", SP, or TAB
+ *    ":", "?", "[", "\", "^", "~", SP, or TAB
+ * 5: *, reject unless REFNAME_REFSPEC_PATTERN is set
  */
 static unsigned char refname_disposition[256] = {
 	1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
 	4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-	4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 2, 1,
+	4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 2, 1,
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 4,
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 4, 0,
@@ -72,12 +73,13 @@ static unsigned char refname_disposition[256] = {
  * - any path component of it begins with ".", or
  * - it has double dots "..", or
  * - it has ASCII control characters, or
- * - it has "*", ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
+ * - it has ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
+ * - it has "*" anywhere unless REFNAME_REFSPEC_PATTERN is set, or
  * - it ends with a "/", or
  * - it ends with ".lock", or
  * - it contains a "@{" portion
  */
-static int check_refname_component(const char *refname, int flags)
+static int check_refname_component(const char *refname, int *flags)
 {
 	const char *cp;
 	char last = '\0';
@@ -98,6 +100,16 @@ static int check_refname_component(const char *refname, int flags)
 			break;
 		case 4:
 			return -1;
+		case 5:
+			if (!(*flags & REFNAME_REFSPEC_PATTERN))
+				return -1; /* refspec can't be a pattern */
+
+			/*
+			 * Unset the pattern flag so that we only accept a single glob for
+			 * the entire refspec.
+			 */
+			*flags &= ~ REFNAME_REFSPEC_PATTERN;
+			break;
 		}
 		last = ch;
 	}
@@ -122,18 +134,10 @@ int check_refname_format(const char *refname, int flags)
 
 	while (1) {
 		/* We are at the start of a path component. */
-		component_len = check_refname_component(refname, flags);
-		if (component_len <= 0) {
-			if ((flags & REFNAME_REFSPEC_PATTERN) &&
-					refname[0] == '*' &&
-					(refname[1] == '\0' || refname[1] == '/')) {
-				/* Accept one wildcard as a full refname component. */
-				flags &= ~REFNAME_REFSPEC_PATTERN;
-				component_len = 1;
-			} else {
-				return -1;
-			}
-		}
+		component_len = check_refname_component(refname, &flags);
+		if (component_len <= 0)
+			return -1;
+
 		component_count++;
 		if (refname[component_len] == '\0')
 			break;
diff --git a/refs.h b/refs.h
index e82fca51f501..1809a1d57577 100644
--- a/refs.h
+++ b/refs.h
@@ -278,8 +278,8 @@ extern int for_each_reflog(each_ref_fn, void *);
  * to the rules described in Documentation/git-check-ref-format.txt.
  * If REFNAME_ALLOW_ONELEVEL is set in flags, then accept one-level
  * reference names.  If REFNAME_REFSPEC_PATTERN is set in flags, then
- * allow a "*" wildcard character in place of one of the name
- * components.  No leading or repeated slashes are accepted.
+ * allow a single "*" wildcard character in the refspec. No leading or
+ * repeated slashes are accepted.
  */
 extern int check_refname_format(const char *refname, int flags);
 
diff --git a/t/t1402-check-ref-format.sh b/t/t1402-check-ref-format.sh
index e5dc62e9efbf..0790edf60de2 100755
--- a/t/t1402-check-ref-format.sh
+++ b/t/t1402-check-ref-format.sh
@@ -62,9 +62,11 @@ invalid_ref 'heads/foo\bar'
 invalid_ref "$(printf 'heads/foo\t')"
 invalid_ref "$(printf 'heads/foo\177')"
 valid_ref "$(printf 'heads/fu\303\237')"
-invalid_ref 'heads/*foo/bar' --refspec-pattern
-invalid_ref 'heads/foo*/bar' --refspec-pattern
-invalid_ref 'heads/f*o/bar' --refspec-pattern
+valid_ref 'heads/*foo/bar' --refspec-pattern
+valid_ref 'heads/foo*/bar' --refspec-pattern
+valid_ref 'heads/f*o/bar' --refspec-pattern
+invalid_ref 'heads/f*o*/bar' --refspec-pattern
+invalid_ref 'heads/foo*/bar*' --refspec-pattern
 
 ref='foo'
 invalid_ref "$ref"
-- 
2.4.3

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Junio C Hamano]

Jacob Keller <jacob.e.keller@intel.com> writes:

> From: Jacob Keller <jacob.keller@gmail.com>
>
> Modify logic of check_refname_component and add a new disposition
> regarding "*". Allow refspecs that contain a single "*" if
> REFNAME_REFSPEC_PATTERN is set. Change the function to pass the flags as
> a pointer, and clear REFNAME_REFSPEC_PATTERN after the first "*" so that
> only a single "*" is accepted.
>
> This loosens restrictions on refspecs by allowing patterns that have
> a "*" within a component instead of only as the whole component. Also
> remove the code that hangled refspec patterns in check_refname_format
> since it is now handled via the check_refname_component logic.
>
> Now refs such as `for/bar*:foo/bar*` and even `foo/bar*:foo/baz*` will
> be accepted. This allows users more control over what is fetched where.
> Since users must modify the default by hand to make use of this
> functionality it is not considered a large risk. Any refspec which
> functioned before shall continue functioning with the new logic.


Thanks.  Now I can read the changes to the code in these two commits
and see that they both make sense ;-)

The above description seem to use "ref" and "refspec" rather
liberally, so I'll rewrite some parts of it to clarify while
queuing.

By the way, have you run test suite before sending this (or any
previous round of this) patch?  This seems to break t5511-refspec.sh
for me.

 

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Jacob Keller]

On Wed, Jul 22, 2015 at 3:04 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jacob Keller <jacob.e.keller@intel.com> writes:
>
>> From: Jacob Keller <jacob.keller@gmail.com>
>>
>> Modify logic of check_refname_component and add a new disposition
>> regarding "*". Allow refspecs that contain a single "*" if
>> REFNAME_REFSPEC_PATTERN is set. Change the function to pass the flags as
>> a pointer, and clear REFNAME_REFSPEC_PATTERN after the first "*" so that
>> only a single "*" is accepted.
>>
>> This loosens restrictions on refspecs by allowing patterns that have
>> a "*" within a component instead of only as the whole component. Also
>> remove the code that hangled refspec patterns in check_refname_format
>> since it is now handled via the check_refname_component logic.
>>
>> Now refs such as `for/bar*:foo/bar*` and even `foo/bar*:foo/baz*` will
>> be accepted. This allows users more control over what is fetched where.
>> Since users must modify the default by hand to make use of this
>> functionality it is not considered a large risk. Any refspec which
>> functioned before shall continue functioning with the new logic.
>
>
> Thanks.  Now I can read the changes to the code in these two commits
> and see that they both make sense ;-)
>
> The above description seem to use "ref" and "refspec" rather
> liberally, so I'll rewrite some parts of it to clarify while
> queuing.
>
> By the way, have you run test suite before sending this (or any
> previous round of this) patch?  This seems to break t5511-refspec.sh
> for me.
>
>
>

Looks like another location I forgot to update. I can send a re-spin
if you need with the following diff. Basically looks like the tests
just didn't get updated to count the new behavior is valid.

diff --git i/t/t5511-refspec.sh w/t/t5511-refspec.sh
index de6db86ccff0..7bfca7962d41 100755
--- i/t/t5511-refspec.sh
+++ w/t/t5511-refspec.sh
@@ -71,11 +71,11 @@ test_refspec fetch ':refs/remotes/frotz/HEAD-to-me'
 test_refspec push ':refs/remotes/frotz/delete me'              invalid
 test_refspec fetch ':refs/remotes/frotz/HEAD to me'            invalid

-test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
-test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
+test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
+test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'

-test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
-test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
+test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*'
+test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*'

 test_refspec fetch 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
 test_refspec push 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid




Regards,
Jake

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Junio C Hamano]

Jacob Keller <jacob.keller@gmail.com> writes:

>> By the way, have you run test suite before sending this (or any
>> previous round of this) patch?  This seems to break t5511-refspec.sh
>> for me.
>
> Looks like another location I forgot to update. I can send a re-spin
> if you need with the following diff. Basically looks like the tests
> just didn't get updated to count the new behavior is valid.

Yeah, basically looks like an untested patch was sent and nobody
noticed during earlier rounds, even the patch was rerolled a few
times, before I finally took it to 'pu' to take a look.  A typical
slow summer moment---people rightfully find it is more important to
have fun themselves than to help polishing others' patches ;-).

Will squash the changes; no need to resend (unless people discover
other issues; let's hope that I wouldn't be the one to do so ;-).

Thanks.

> diff --git i/t/t5511-refspec.sh w/t/t5511-refspec.sh
> index de6db86ccff0..7bfca7962d41 100755
> --- i/t/t5511-refspec.sh
> +++ w/t/t5511-refspec.sh
> @@ -71,11 +71,11 @@ test_refspec fetch ':refs/remotes/frotz/HEAD-to-me'
>  test_refspec push ':refs/remotes/frotz/delete me'              invalid
>  test_refspec fetch ':refs/remotes/frotz/HEAD to me'            invalid
>
> -test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
> -test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
> +test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
> +test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
>
> -test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
> -test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
> +test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*'
> +test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*'
>
>  test_refspec fetch 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
>  test_refspec push 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
>
>
>
>
> Regards,
> Jake

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Jacob Keller]

On Thu, Jul 23, 2015 at 9:44 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jacob Keller <jacob.keller@gmail.com> writes:
>
>>> By the way, have you run test suite before sending this (or any
>>> previous round of this) patch?  This seems to break t5511-refspec.sh
>>> for me.
>>
>> Looks like another location I forgot to update. I can send a re-spin
>> if you need with the following diff. Basically looks like the tests
>> just didn't get updated to count the new behavior is valid.
>
> Yeah, basically looks like an untested patch was sent and nobody
> noticed during earlier rounds, even the patch was rerolled a few
> times, before I finally took it to 'pu' to take a look.  A typical
> slow summer moment---people rightfully find it is more important to
> have fun themselves than to help polishing others' patches ;-).
>

I think what happened, is that I ran some tests when the patch was
"configurable" and I had modified the one set of tests to try with the
option enabled, but it didn't fail in the t5511-refspec.sh since this
series of tests didn't enable the new option. Then, I never re-tested
again (OOPS!) when I removed the optional portion.

> Will squash the changes; no need to resend (unless people discover
> other issues; let's hope that I wouldn't be the one to do so ;-).
>
> Thanks.
>

Thank you! :)

Regards,
Jake

>> diff --git i/t/t5511-refspec.sh w/t/t5511-refspec.sh
>> index de6db86ccff0..7bfca7962d41 100755
>> --- i/t/t5511-refspec.sh
>> +++ w/t/t5511-refspec.sh
>> @@ -71,11 +71,11 @@ test_refspec fetch ':refs/remotes/frotz/HEAD-to-me'
>>  test_refspec push ':refs/remotes/frotz/delete me'              invalid
>>  test_refspec fetch ':refs/remotes/frotz/HEAD to me'            invalid
>>
>> -test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
>> -test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
>> +test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
>> +test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
>>
>> -test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
>> -test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
>> +test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*'
>> +test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*'
>>
>>  test_refspec fetch 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
>>  test_refspec push 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
>>
>>
>>
>>
>> Regards,
>> Jake
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Will squash the changes; no need to resend (unless people discover
> other issues; let's hope that I wouldn't be the one to do so ;-).
>
> Thanks.
>
>> diff --git i/t/t5511-refspec.sh w/t/t5511-refspec.sh
>> index de6db86ccff0..7bfca7962d41 100755
>> --- i/t/t5511-refspec.sh
>> +++ w/t/t5511-refspec.sh
>> @@ -71,11 +71,11 @@ test_refspec fetch ':refs/remotes/frotz/HEAD-to-me'

That was whitespace damaged, so I had to hand-tweak the file in
place.  While at it, I noticed that we do not check a case where
multiple asterisks appear in a single component (which is rejected
for a reason different from having multiple components with an
asterisk in them), which also deserves its own test.

I'll squash in the following instead.

There is a slightly related tangent I noticed while doing so.

I wonder if there is an obvious and unambiguous interpretation of
what this command line wants to do:

    $ git fetch origin refs/heads/*g*/for-linus:refs/remotes/i-*/mine

We _might_ want to allow one (and only one) component with more than
one asterisk on the LHS of a refspec, while requiring only one
asterisk on the RHS to allow "this '*g*' is just like '*' but
excluding things that do not have 'g' in it".

Or it may not be worth the additional complexity.


 t/t5511-refspec.sh | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/t/t5511-refspec.sh b/t/t5511-refspec.sh
index de6db86..f541f30 100755
--- a/t/t5511-refspec.sh
+++ b/t/t5511-refspec.sh
@@ -71,15 +71,18 @@ test_refspec fetch ':refs/remotes/frotz/HEAD-to-me'
 test_refspec push ':refs/remotes/frotz/delete me'		invalid
 test_refspec fetch ':refs/remotes/frotz/HEAD to me'		invalid
 
-test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
-test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
+test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
+test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
 
-test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
-test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
+test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*'
+test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*'
 
 test_refspec fetch 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
 test_refspec push 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
 
+test_refspec fetch 'refs/heads/*g*/for-linus:refs/remotes/mine/*' invalid
+test_refspec push 'refs/heads/*g*/for-linus:refs/remotes/mine/*' invalid
+
 test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*'
 test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*'
 

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Jacob Keller]

On Thu, Jul 23, 2015 at 10:13 AM, Junio C Hamano <gitster@pobox.com> wrote:
> That was whitespace damaged, so I had to hand-tweak the file in
> place.  While at it, I noticed that we do not check a case where
> multiple asterisks appear in a single component (which is rejected
> for a reason different from having multiple components with an
> asterisk in them), which also deserves its own test.
>

Oops. Sorry about that..

> I'll squash in the following instead.
>
> There is a slightly related tangent I noticed while doing so.
>
> I wonder if there is an obvious and unambiguous interpretation of
> what this command line wants to do:
>
>     $ git fetch origin refs/heads/*g*/for-linus:refs/remotes/i-*/mine
>

Personally, I do think this is "obvious" but I don't think that our
parser is at all smart enough to handle it. The  advantage with the
current code, is that the match parser already handles it perfectly.
It was just the rules up front which limited how much we could do. I
don't think the added complexity is that valuable here.. For most
common cases it's just prefixes or suffixes that matter.

> We _might_ want to allow one (and only one) component with more than
> one asterisk on the LHS of a refspec, while requiring only one
> asterisk on the RHS to allow "this '*g*' is just like '*' but
> excluding things that do not have 'g' in it".
>

As above, I think it's obvious the intended meaning, but... the actual
interpretation could be a variety of things. It's not guaranteed to be
interpreted in that way, and while we could document it, I don't think
that it is worth the complexity unless someone actually needs this
behavior.

> Or it may not be worth the additional complexity.
>

Below diff looks fine, thanks!

Regards,
Jake

>
>  t/t5511-refspec.sh | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/t/t5511-refspec.sh b/t/t5511-refspec.sh
> index de6db86..f541f30 100755
> --- a/t/t5511-refspec.sh
> +++ b/t/t5511-refspec.sh
> @@ -71,15 +71,18 @@ test_refspec fetch ':refs/remotes/frotz/HEAD-to-me'
>  test_refspec push ':refs/remotes/frotz/delete me'              invalid
>  test_refspec fetch ':refs/remotes/frotz/HEAD to me'            invalid
>
> -test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
> -test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
> +test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
> +test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
>
> -test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
> -test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
> +test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*'
> +test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*'
>
>  test_refspec fetch 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
>  test_refspec push 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
>
> +test_refspec fetch 'refs/heads/*g*/for-linus:refs/remotes/mine/*' invalid
> +test_refspec push 'refs/heads/*g*/for-linus:refs/remotes/mine/*' invalid
> +
>  test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*'
>  test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*'
>

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Eric Sunshine]

On Wed, Jul 22, 2015 at 5:05 PM, Jacob Keller <jacob.e.keller@intel.com> wrote:
> From: Jacob Keller <jacob.keller@gmail.com>
>
> Modify logic of check_refname_component and add a new disposition
> regarding "*". Allow refspecs that contain a single "*" if
> REFNAME_REFSPEC_PATTERN is set. Change the function to pass the flags as
> a pointer, and clear REFNAME_REFSPEC_PATTERN after the first "*" so that
> only a single "*" is accepted.
>
> This loosens restrictions on refspecs by allowing patterns that have
> a "*" within a component instead of only as the whole component. Also
> remove the code that hangled refspec patterns in check_refname_format

s/hangled/handled/

> since it is now handled via the check_refname_component logic.
>
> Now refs such as `for/bar*:foo/bar*` and even `foo/bar*:foo/baz*` will
> be accepted. This allows users more control over what is fetched where.
> Since users must modify the default by hand to make use of this
> functionality it is not considered a large risk. Any refspec which
> functioned before shall continue functioning with the new logic.
>
> Signed-off-by: Jacob Keller <jacob.keller@gmail.com>

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Wed, Jul 22, 2015 at 5:05 PM, Jacob Keller <jacob.e.keller@intel.com> wrote:
>> From: Jacob Keller <jacob.keller@gmail.com>
>>
>> Modify logic of check_refname_component and add a new disposition
>> regarding "*". Allow refspecs that contain a single "*" if
>> REFNAME_REFSPEC_PATTERN is set. Change the function to pass the flags as
>> a pointer, and clear REFNAME_REFSPEC_PATTERN after the first "*" so that
>> only a single "*" is accepted.
>>
>> This loosens restrictions on refspecs by allowing patterns that have
>> a "*" within a component instead of only as the whole component. Also
>> remove the code that hangled refspec patterns in check_refname_format
>
> s/hangled/handled/
> ...

Thanks; here is what I queued yesterday.

-- >8 --
From: Jacob Keller <jacob.keller@gmail.com>
Date: Wed, 22 Jul 2015 14:05:33 -0700
Subject: [PATCH] refs: loosen restriction on wildcard "*" refspecs

Loosen restrictions on refspecs by allowing patterns that have a "*"
within a component instead of only as the whole component.

Remove the logic to accept a single "*" as a whole component from
check_refname_format(), and implement an extended form of that logic
in check_refname_component().  Pass the pointer to the flags argument
to the latter, as it has to clear REFNAME_REFSPEC_PATTERN bit when
it sees "*".

Teach check_refname_component() function to allow an asterisk "*"
only when REFNAME_REFSPEC_PATTERN is set in the flags, and drop the
bit after seeing a "*", to ensure that one side of a refspec
contains at most one asterisk.

This will allow us to accept refspecs such as `for/bar*:foo/baz*`.
Any refspec which functioned before shall continue functioning with
the new logic.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-check-ref-format.txt |  4 ++--
 refs.c                                 | 36 +++++++++++++++++++---------------
 refs.h                                 |  4 ++--
 t/t1402-check-ref-format.sh            |  8 +++++---
 t/t5511-refspec.sh                     | 11 +++++++----
 5 files changed, 36 insertions(+), 27 deletions(-)

diff --git a/Documentation/git-check-ref-format.txt b/Documentation/git-check-ref-format.txt
index fc02959..9044dfa 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -94,8 +94,8 @@ OPTIONS
 	Interpret <refname> as a reference name pattern for a refspec
 	(as used with remote repositories).  If this option is
 	enabled, <refname> is allowed to contain a single `*`
-	in place of a one full pathname component (e.g.,
-	`foo/*/bar` but not `foo/bar*`).
+	in the refspec (e.g., `foo/bar*/baz` or `foo/bar*baz/`
+	but not `foo/bar*/baz*`).
 
 --normalize::
 	Normalize 'refname' by removing any leading slash (`/`)
diff --git a/refs.c b/refs.c
index 0900f54..3127518 100644
--- a/refs.c
+++ b/refs.c
@@ -21,12 +21,13 @@ struct ref_lock {
  * 2: ., look for a preceding . to reject .. in refs
  * 3: {, look for a preceding @ to reject @{ in refs
  * 4: A bad character: ASCII control characters, and
- *    "*", ":", "?", "[", "\", "^", "~", SP, or TAB
+ *    ":", "?", "[", "\", "^", "~", SP, or TAB
+ * 5: *, reject unless REFNAME_REFSPEC_PATTERN is set
  */
 static unsigned char refname_disposition[256] = {
 	1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
 	4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-	4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 2, 1,
+	4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 2, 1,
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 4,
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 4, 0,
@@ -73,12 +74,13 @@ static unsigned char refname_disposition[256] = {
  * - any path component of it begins with ".", or
  * - it has double dots "..", or
  * - it has ASCII control characters, or
- * - it has "*", ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
+ * - it has ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
+ * - it has "*" anywhere unless REFNAME_REFSPEC_PATTERN is set, or
  * - it ends with a "/", or
  * - it ends with ".lock", or
  * - it contains a "@{" portion
  */
-static int check_refname_component(const char *refname, int flags)
+static int check_refname_component(const char *refname, int *flags)
 {
 	const char *cp;
 	char last = '\0';
@@ -99,6 +101,16 @@ static int check_refname_component(const char *refname, int flags)
 			break;
 		case 4:
 			return -1;
+		case 5:
+			if (!(*flags & REFNAME_REFSPEC_PATTERN))
+				return -1; /* refspec can't be a pattern */
+
+			/*
+			 * Unset the pattern flag so that we only accept
+			 * a single asterisk for one side of refspec.
+			 */
+			*flags &= ~ REFNAME_REFSPEC_PATTERN;
+			break;
 		}
 		last = ch;
 	}
@@ -123,18 +135,10 @@ int check_refname_format(const char *refname, int flags)
 
 	while (1) {
 		/* We are at the start of a path component. */
-		component_len = check_refname_component(refname, flags);
-		if (component_len <= 0) {
-			if ((flags & REFNAME_REFSPEC_PATTERN) &&
-					refname[0] == '*' &&
-					(refname[1] == '\0' || refname[1] == '/')) {
-				/* Accept one wildcard as a full refname component. */
-				flags &= ~REFNAME_REFSPEC_PATTERN;
-				component_len = 1;
-			} else {
-				return -1;
-			}
-		}
+		component_len = check_refname_component(refname, &flags);
+		if (component_len <= 0)
+			return -1;
+
 		component_count++;
 		if (refname[component_len] == '\0')
 			break;
diff --git a/refs.h b/refs.h
index cf642e6..417f2c8 100644
--- a/refs.h
+++ b/refs.h
@@ -224,8 +224,8 @@ extern int for_each_reflog(each_ref_fn, void *);
  * to the rules described in Documentation/git-check-ref-format.txt.
  * If REFNAME_ALLOW_ONELEVEL is set in flags, then accept one-level
  * reference names.  If REFNAME_REFSPEC_PATTERN is set in flags, then
- * allow a "*" wildcard character in place of one of the name
- * components.  No leading or repeated slashes are accepted.
+ * allow a single "*" wildcard character in the refspec. No leading or
+ * repeated slashes are accepted.
  */
 extern int check_refname_format(const char *refname, int flags);
 
diff --git a/t/t1402-check-ref-format.sh b/t/t1402-check-ref-format.sh
index e5dc62e..0790edf 100755
--- a/t/t1402-check-ref-format.sh
+++ b/t/t1402-check-ref-format.sh
@@ -62,9 +62,11 @@ invalid_ref 'heads/foo\bar'
 invalid_ref "$(printf 'heads/foo\t')"
 invalid_ref "$(printf 'heads/foo\177')"
 valid_ref "$(printf 'heads/fu\303\237')"
-invalid_ref 'heads/*foo/bar' --refspec-pattern
-invalid_ref 'heads/foo*/bar' --refspec-pattern
-invalid_ref 'heads/f*o/bar' --refspec-pattern
+valid_ref 'heads/*foo/bar' --refspec-pattern
+valid_ref 'heads/foo*/bar' --refspec-pattern
+valid_ref 'heads/f*o/bar' --refspec-pattern
+invalid_ref 'heads/f*o*/bar' --refspec-pattern
+invalid_ref 'heads/foo*/bar*' --refspec-pattern
 
 ref='foo'
 invalid_ref "$ref"
diff --git a/t/t5511-refspec.sh b/t/t5511-refspec.sh
index de6db86..f541f30 100755
--- a/t/t5511-refspec.sh
+++ b/t/t5511-refspec.sh
@@ -71,15 +71,18 @@ test_refspec fetch ':refs/remotes/frotz/HEAD-to-me'
 test_refspec push ':refs/remotes/frotz/delete me'		invalid
 test_refspec fetch ':refs/remotes/frotz/HEAD to me'		invalid
 
-test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
-test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah' invalid
+test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
+test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*-blah'
 
-test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
-test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*' invalid
+test_refspec fetch 'refs/heads*/for-linus:refs/remotes/mine/*'
+test_refspec push 'refs/heads*/for-linus:refs/remotes/mine/*'
 
 test_refspec fetch 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
 test_refspec push 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid
 
+test_refspec fetch 'refs/heads/*g*/for-linus:refs/remotes/mine/*' invalid
+test_refspec push 'refs/heads/*g*/for-linus:refs/remotes/mine/*' invalid
+
 test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*'
 test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*'
 
-- 
2.5.0-rc3-352-g936684d

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Keller, Jacob E]

On Thu, 2015-07-23 at 15:12 -0700, Junio C Hamano wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> 
> > On Wed, Jul 22, 2015 at 5:05 PM, Jacob Keller <
> > jacob.e.keller@intel.com> wrote:
> > > From: Jacob Keller <jacob.keller@gmail.com>
> > > 
> > > Modify logic of check_refname_component and add a new disposition
> > > regarding "*". Allow refspecs that contain a single "*" if
> > > REFNAME_REFSPEC_PATTERN is set. Change the function to pass the 
> > > flags as
> > > a pointer, and clear REFNAME_REFSPEC_PATTERN after the first "*" 
> > > so that
> > > only a single "*" is accepted.
> > > 
> > > This loosens restrictions on refspecs by allowing patterns that 
> > > have
> > > a "*" within a component instead of only as the whole component. 
> > > Also
> > > remove the code that hangled refspec patterns in 
> > > check_refname_format
> > 
> > s/hangled/handled/
> > ...
> 
> Thanks; here is what I queued yesterday.
> 

Woah. I bow to your imperative commit message abilities. The new commit
message is very nicely worded. Thanks for taking the time to reword my
jumble correctly.

Everything looked great to me.

Regards,
Jake

Re: [PATCH 2/2] refs: loosen restriction on wildcard "*" refspecs

[Junio C Hamano]

"Keller, Jacob E" <jacob.e.keller@intel.com> writes:

>> > s/hangled/handled/
>> > ...
>> 
>> Thanks; here is what I queued yesterday.
>> 
>
> Woah. I bow to your imperative commit message abilities. The new commit
> message is very nicely worded.

Heh, imperative is secondary. I just wanted to straighten out the
use of "refs" and "refspecs" there.

Thanks for a patch that was cleanly done.