[PATCH 0/8] Bundle URIs: Prepatory patches

[PATCH 0/8] Bundle URIs: Prepatory patches

[Derrick Stolee via GitGitGadget]

(This series is based on 2.36.0, but I don't see any conflicts with 'seen'.)

Here is a re-roll of the first eight patches from Ævar's latest RFC. As I
mentioned [1], these patches are useful refactors that help both approaches,
so they can be reviewed while we are still working on combining our
approaches to the bundle URI feature.

[1]
https://lore.kernel.org/git/dd9836e8-bfc9-9a52-199a-3ffce26101f8@github.com/

(My update: I'm still working on a combined version starting from Ævar's
latest changes, but my vacation last week set me back a bit. I'm hoping to
have something in the next two weeks, but my WIP version is based on these
patches.)

Thanks, -Stolee

Here is a range-diff from Ævar's first eight patches. The biggest change is
that his patch 2 was split into my patches 2 and 3. I also renamed
"path_matches_flags()" to "path_starts_with_dotslash_flags()", which makes
the diff there look pretty nasty when it really isn't that bad.

 1:  fcb0b50471 !  1:  bd592ebba4 connect.c: refactor sending of agent & object-format
    @@ Commit message
         object-format. So let's split this into a function.
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
     
      ## connect.c ##
     @@ connect.c: void check_stateless_delimiter(int stateless_rpc,
 -:  ---------- >  2:  15ba1198b4 compat: create is_xplatform_dir_sep()
 2:  60a66c41bd !  3:  22d7ee7d4e dir API: add a generalized path_match_flags() function
    @@ Metadata
      ## Commit message ##
         dir API: add a generalized path_match_flags() function
     
    -    Add a path_match_flags() function and have the two sets of
    -    starts_with_dot_{,dot_}slash() functions added in
    -    63e95beb085 (submodule: port resolve_relative_url from shell to C,
    -    2016-04-15) and a2b26ffb1a8 (fsck: convert gitmodules url to URL
    -    passed to curl, 2020-04-18) be thin wrappers for it.
    +    Add a path_starts_with_dotslash_flags() function and have the two sets
    +    of starts_with_dot_{,dot_}slash() functions added in 63e95beb085
    +    (submodule: port resolve_relative_url from shell to C, 2016-04-15) and
    +    a2b26ffb1a8 (fsck: convert gitmodules url to URL passed to curl,
    +    2020-04-18) be thin wrappers for it.
     
         As the latter of those notes the fsck version was copied from the
         initial builtin/submodule--helper.c version.
     
    -    Since the code added in a2b26ffb1a8 was doing really doing the same as
    -    win32_is_dir_sep() added in 1cadad6f658 (git clone <url>
    -    C:\cygwin\home\USER\repo' is working (again), 2018-12-15) let's move
    -    the latter to git-compat-util.h is a is_xplatform_dir_sep(). We can
    -    then call either it or the platform-specific is_dir_sep() from this
    -    new function.
    -
    -    Let's likewise change code in various other places that was hardcoding
    -    checks for "'/' || '\\'" with the new is_xplatform_dir_sep(). As can
    -    be seen in those callers some of them still concern themselves with
    -    ':' (Mac OS classic?), but let's leave the question of whether that
    -    should be consolidated for some other time.
    -
         As we expect to make wider use of the "native" case in the future,
         define and use two starts_with_dot_{,dot_}slash_native() convenience
         wrappers. This makes the diff in builtin/submodule--helper.c much
         smaller.
     
    -    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
     
      ## builtin/submodule--helper.c ##
     @@ builtin/submodule--helper.c: static char *get_default_remote(void)
    @@ builtin/submodule--helper.c: static int chop_last_dir(char **remoteurl, int is_r
      
     +static int starts_with_dot_slash(const char *const path)
     +{
    -+	return starts_with_dot_slash_native(path);;
    ++	return starts_with_dot_slash_native(path);
     +}
     +
     +static int starts_with_dot_dot_slash(const char *const path)
    @@ builtin/submodule--helper.c: static int chop_last_dir(char **remoteurl, int is_r
       * The `url` argument is the URL that navigates to the submodule origin
       * repo. When relative, this URL is relative to the superproject origin
     
    - ## compat/mingw.c ##
    -@@ compat/mingw.c: int is_valid_win32_path(const char *path, int allow_literal_nul)
    - 			}
    - 
    - 			c = path[i];
    --			if (c && c != '.' && c != ':' && c != '/' && c != '\\')
    -+			if (c && c != '.' && c != ':' && !is_xplatform_dir_sep(c))
    - 				goto not_a_reserved_name;
    - 
    - 			/* contains reserved name */
    -
    - ## compat/win32/path-utils.h ##
    -@@ compat/win32/path-utils.h: int win32_has_dos_drive_prefix(const char *path);
    - 
    - int win32_skip_dos_drive_prefix(char **path);
    - #define skip_dos_drive_prefix win32_skip_dos_drive_prefix
    --static inline int win32_is_dir_sep(int c)
    --{
    --	return c == '/' || c == '\\';
    --}
    --#define is_dir_sep win32_is_dir_sep
    -+#define is_dir_sep is_xplatform_dir_sep
    - static inline char *win32_find_last_dir_sep(const char *path)
    - {
    - 	char *ret = NULL;
    -
      ## dir.c ##
     @@ dir.c: void relocate_gitdir(const char *path, const char *old_git_dir, const char *new_
      
          connect_work_tree_and_git_dir(path, new_git_dir, 0);
      }
     +
    -+int path_match_flags(const char *const str, const enum path_match_flags flags)
    ++int path_starts_with_dotslash_flags(const char *const str,
    ++				    const enum path_match_flags flags)
     +{
     +	const char *p = str;
     +
     +	if (flags & PATH_MATCH_NATIVE &&
     +	    flags & PATH_MATCH_XPLATFORM)
    -+		BUG("path_match_flags() must get one match kind, not multiple!");
    ++		BUG("path_starts_with_dotslash_flags() must get one match kind, not multiple!");
     +	else if (!(flags & PATH_MATCH_KINDS_MASK))
    -+		BUG("path_match_flags() must get at least one match kind!");
    ++		BUG("path_starts_with_dotslash_flags() must get at least one match kind!");
     +
     +	if (flags & PATH_MATCH_STARTS_WITH_DOT_SLASH &&
     +	    flags & PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH)
    -+		BUG("path_match_flags() must get one platform kind, not multiple!");
    ++		BUG("path_starts_with_dotslash_flags() must get one platform kind, not multiple!");
     +	else if (!(flags & PATH_MATCH_PLATFORM_MASK))
    -+		BUG("path_match_flags() must get at least one platform kind!");
    ++		BUG("path_starts_with_dotslash_flags() must get at least one platform kind!");
     +
     +	if (*p++ != '.')
     +		return 0;
    @@ dir.h: void connect_work_tree_and_git_dir(const char *work_tree,
                   const char *new_git_dir);
     +
     +/**
    -+ * The "enum path_matches_kind" determines how path_match_flags() will
    -+ * behave. The flags come in sets, and one (and only one) must be
    ++ * The "enum path_matches_kind" determines how path_starts_with_dotslash_flags()
    ++ * will behave. The flags come in sets, and one (and only one) must be
     + * provided out of each "set":
     + *
     + * PATH_MATCH_NATIVE:
    @@ dir.h: void connect_work_tree_and_git_dir(const char *work_tree,
     +#define PATH_MATCH_PLATFORM_MASK (PATH_MATCH_NATIVE | PATH_MATCH_XPLATFORM)
     +
     +/**
    -+ * path_match_flags() checks if a given "path" matches a given "enum
    -+ * path_match_flags" criteria.
    ++ * path_starts_with_dotslash_flags() checks if a given "path" matches a
    ++ * given "enum path_match_flags" criteria.
     + */
    -+int path_match_flags(const char *const path, const enum path_match_flags f);
    ++int path_starts_with_dotslash_flags(const char *const path,
    ++				    const enum path_match_flags f);
     +
     +/**
     + * starts_with_dot_slash_native(): convenience wrapper for
    -+ * path_match_flags() with PATH_MATCH_STARTS_WITH_DOT_SLASH and
    -+ * PATH_MATCH_NATIVE.
    ++ * path_starts_with_dotslash_flags() with PATH_MATCH_STARTS_WITH_DOT_SLASH
    ++ * and PATH_MATCH_NATIVE.
     + */
     +static inline int starts_with_dot_slash_native(const char *const path)
     +{
     +	const enum path_match_flags what = PATH_MATCH_STARTS_WITH_DOT_SLASH;
     +
    -+	return path_match_flags(path, what | PATH_MATCH_NATIVE);
    ++	return path_starts_with_dotslash_flags(path, what | PATH_MATCH_NATIVE);
     +}
     +
     +/**
     + * starts_with_dot_slash_native(): convenience wrapper for
    -+ * path_match_flags() with PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH and
    -+ * PATH_MATCH_NATIVE.
    ++ * path_starts_with_dotslash_flags() with
    ++ * PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH and PATH_MATCH_NATIVE.
     + */
     +static inline int starts_with_dot_dot_slash_native(const char *const path)
     +{
     +	const enum path_match_flags what = PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH;
     +
    -+	return path_match_flags(path, what | PATH_MATCH_NATIVE);
    ++	return path_starts_with_dotslash_flags(path, what | PATH_MATCH_NATIVE);
     +}
      #endif
     
    @@ fsck.c: int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
     +static int starts_with_dot_slash(const char *const path)
      {
     -	return str[0] == '.' && (str[1] == '/' || str[1] == '\\');
    -+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_SLASH |
    -+				PATH_MATCH_XPLATFORM);
    ++	const int flags = PATH_MATCH_STARTS_WITH_DOT_SLASH |
    ++			  PATH_MATCH_XPLATFORM;
    ++	return path_starts_with_dotslash_flags(path, flags);
      }
      
     -/*
    @@ fsck.c: int fsck_tag_standalone(const struct object_id *oid, const char *buffer,
     +static int starts_with_dot_dot_slash(const char *const path)
      {
     -	return str[0] == '.' && starts_with_dot_slash(str + 1);
    -+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH |
    -+				PATH_MATCH_XPLATFORM);
    ++	const int flags = PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH |
    ++			  PATH_MATCH_XPLATFORM;
    ++	return path_starts_with_dotslash_flags(path, flags);
      }
      
      static int submodule_url_is_relative(const char *url)
    -
    - ## git-compat-util.h ##
    -@@
    - #include <sys/sysctl.h>
    - #endif
    - 
    -+/* Used by compat/win32/path-utils.h, and more */
    -+static inline int is_xplatform_dir_sep(int c)
    -+{
    -+	return c == '/' || c == '\\';
    -+}
    -+
    - #if defined(__CYGWIN__)
    - #include "compat/win32/path-utils.h"
    - #endif
    -@@ git-compat-util.h: static inline int git_skip_dos_drive_prefix(char **path)
    - #define skip_dos_drive_prefix git_skip_dos_drive_prefix
    - #endif
    - 
    --#ifndef is_dir_sep
    - static inline int git_is_dir_sep(int c)
    - {
    - 	return c == '/';
    - }
    -+#ifndef is_dir_sep
    - #define is_dir_sep git_is_dir_sep
    - #endif
    - 
    -
    - ## path.c ##
    -@@ path.c: int is_ntfs_dotgit(const char *name)
    - 
    - 	for (;;) {
    - 		c = *(name++);
    --		if (!c || c == '\\' || c == '/' || c == ':')
    -+		if (!c || is_xplatform_dir_sep(c) || c == ':')
    - 			return 1;
    - 		if (c != '.' && c != ' ')
    - 			return 0;
    -
    - ## submodule-config.c ##
    -@@ submodule-config.c: int check_submodule_name(const char *name)
    - 		return -1;
    - 
    - 	/*
    --	 * Look for '..' as a path component. Check both '/' and '\\' as
    -+	 * Look for '..' as a path component. Check is_xplatform_dir_sep() as
    - 	 * separators rather than is_dir_sep(), because we want the name rules
    - 	 * to be consistent across platforms.
    - 	 */
    - 	goto in_component; /* always start inside component */
    - 	while (*name) {
    - 		char c = *name++;
    --		if (c == '/' || c == '\\') {
    -+		if (is_xplatform_dir_sep(c)) {
    - in_component:
    - 			if (name[0] == '.' && name[1] == '.' &&
    --			    (!name[2] || name[2] == '/' || name[2] == '\\'))
    -+			    (!name[2] || is_xplatform_dir_sep(name[2])))
    - 				return -1;
    - 		}
    - 	}
 -:  ---------- >  4:  01847c9570 amend! compat: create is_xplatform_dir_sep()
 3:  15fea3f4da !  5:  3acb0f9637 fetch-pack: add a deref_without_lazy_fetch_extended()
    @@ Commit message
         subsequent commit.
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
     
      ## fetch-pack.c ##
     @@ fetch-pack.c: static void for_each_cached_alternate(struct fetch_negotiator *negotiator,
    @@ fetch-pack.c: static void for_each_cached_alternate(struct fetch_negotiator *neg
     +	struct object_info info = { .typep = type };
          struct commit *commit;
      
    ++	if (!info.typep)
    ++		BUG("must pass non-NULL type");
    ++
          commit = lookup_commit_in_graph(the_repository, oid);
    -@@ fetch-pack.c: static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
    + 	if (commit)
    + 		return commit;
      
          while (1) {
              if (oid_object_info_extended(the_repository, oid, &info,
    @@ fetch-pack.c: static struct commit *deref_without_lazy_fetch(const struct object
          return NULL;
      }
      
    -+
     +static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
     +					       int mark_tags_complete)
     +{
 4:  009cb42cca !  6:  04508f83c0 fetch-pack: move --keep=* option filling to a function
    @@ Commit message
         function.
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
     
      ## fetch-pack.c ##
     @@ fetch-pack.c: static void parse_gitmodules_oids(int fd, struct oidset *gitmodules_oids)
 5:  55462172ed =  7:  23df88a454 http: make http_get_file() external
 6:  2783486464 !  8:  c410280d2b remote: move relative_url()
    @@ Commit message
            functionally identical "starts_with_dot_{,dot_}slash()" wrappers
            "builtin/submodule--helper.c".
     
    -    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
     
      ## builtin/submodule--helper.c ##
     @@ builtin/submodule--helper.c: static char *get_default_remote(void)
    @@ builtin/submodule--helper.c: static char *get_default_remote(void)
     -
     -static int starts_with_dot_slash(const char *const path)
     -{
    --	return starts_with_dot_slash_native(path);;
    +-	return starts_with_dot_slash_native(path);
     -}
     -
     -static int starts_with_dot_dot_slash(const char *const path)
    @@ builtin/submodule--helper.c: static int module_foreach(int argc, const char **ar
      
     +static int starts_with_dot_slash(const char *const path)
     +{
    -+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_SLASH |
    -+				PATH_MATCH_XPLATFORM);
    ++	const int flags = PATH_MATCH_STARTS_WITH_DOT_SLASH |
    ++			  PATH_MATCH_XPLATFORM;
    ++	return path_starts_with_dotslash_flags(path, flags);
     +}
     +
     +static int starts_with_dot_dot_slash(const char *const path)
     +{
    -+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH |
    -+				PATH_MATCH_XPLATFORM);
    ++	const int flags = PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH |
    ++			  PATH_MATCH_XPLATFORM;
    ++	return path_starts_with_dotslash_flags(path, flags);
     +}
     +
      struct init_cb {
 7:  e0a387c6a0 !  9:  5c112c1994 remote: allow relative_url() to return an absolute url
    @@ Commit message
         The documentation now discusses what happens when supplying two
         absolute URLs.
     
    -    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
     
      ## remote.c ##
     @@ remote.c: char *relative_url(const char *remote_url, const char *url,
 8:  a38b6f388a ! 10:  2afa9c2e73 bundle.h: make "fd" version of read_bundle_header() public
    @@ Commit message
         will be used by code that wants to pass a fd to the bundle API.
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Derrick Stolee <derrickstolee@github.com>
     
      ## bundle.c ##
     @@ bundle.c: static int parse_bundle_signature(struct bundle_header *header, const char *line


Derrick Stolee (3):
  http: make http_get_file() external
  remote: move relative_url()
  remote: allow relative_url() to return an absolute url

Ævar Arnfjörð Bjarmason (5):
  connect.c: refactor sending of agent & object-format
  dir API: add a generalized path_match_flags() function
  fetch-pack: add a deref_without_lazy_fetch_extended()
  fetch-pack: move --keep=* option filling to a function
  bundle.h: make "fd" version of read_bundle_header() public

 builtin/submodule--helper.c | 141 +++---------------------------------
 bundle.c                    |   8 +-
 bundle.h                    |   2 +
 compat/mingw.c              |   2 +-
 compat/win32/path-utils.h   |   6 +-
 connect.c                   |  33 +++++----
 dir.c                       |  29 ++++++++
 dir.h                       |  63 ++++++++++++++++
 fetch-pack.c                |  45 ++++++++----
 fsck.c                      |  23 ++----
 git-compat-util.h           |   8 +-
 http.c                      |   4 +-
 http.h                      |   9 +++
 path.c                      |   2 +-
 remote.c                    |  99 +++++++++++++++++++++++++
 remote.h                    |  32 ++++++++
 submodule-config.c          |   6 +-
 17 files changed, 321 insertions(+), 191 deletions(-)


base-commit: 6cd33dceed60949e2dbc32e3f0f5e67c4c882e1e
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1233%2Fderrickstolee%2Fbundle-redo%2Fprepare-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1233/derrickstolee/bundle-redo/prepare-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1233
-- 
gitgitgadget

[PATCH 1/8] connect.c: refactor sending of agent & object-format

[Ævar Arnfjörð Bjarmason via GitGitGadget]

From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>

Refactor the sending of the "agent" and "object-format" capabilities
into a function.

This was added in its current form in ab67235bc4 (connect: parse v2
refs with correct hash algorithm, 2020-05-25). When we connect to a v2
server we need to know about its object-format, and it needs to know
about ours. Since most things in connect.c and transport.c piggy-back
on the eager getting of remote refs via the handshake() those commands
can make use of the just-sent-over object-format by ls-refs.

But I'm about to add a command that may come after ls-refs, and may
not, but we need the server to know about our user-agent and
object-format. So let's split this into a function.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 connect.c | 33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)

diff --git a/connect.c b/connect.c
index afc79a6236e..e6d0b1d34bd 100644
--- a/connect.c
+++ b/connect.c
@@ -473,6 +473,24 @@ void check_stateless_delimiter(int stateless_rpc,
 		die("%s", error);
 }
 
+static void send_capabilities(int fd_out, struct packet_reader *reader)
+{
+	const char *hash_name;
+
+	if (server_supports_v2("agent", 0))
+		packet_write_fmt(fd_out, "agent=%s", git_user_agent_sanitized());
+
+	if (server_feature_v2("object-format", &hash_name)) {
+		int hash_algo = hash_algo_by_name(hash_name);
+		if (hash_algo == GIT_HASH_UNKNOWN)
+			die(_("unknown object format '%s' specified by server"), hash_name);
+		reader->hash_algo = &hash_algos[hash_algo];
+		packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name);
+	} else {
+		reader->hash_algo = &hash_algos[GIT_HASH_SHA1];
+	}
+}
+
 struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,
 			     struct ref **list, int for_push,
 			     struct transport_ls_refs_options *transport_options,
@@ -480,7 +498,6 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,
 			     int stateless_rpc)
 {
 	int i;
-	const char *hash_name;
 	struct strvec *ref_prefixes = transport_options ?
 		&transport_options->ref_prefixes : NULL;
 	const char **unborn_head_target = transport_options ?
@@ -490,18 +507,8 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,
 	if (server_supports_v2("ls-refs", 1))
 		packet_write_fmt(fd_out, "command=ls-refs\n");
 
-	if (server_supports_v2("agent", 0))
-		packet_write_fmt(fd_out, "agent=%s", git_user_agent_sanitized());
-
-	if (server_feature_v2("object-format", &hash_name)) {
-		int hash_algo = hash_algo_by_name(hash_name);
-		if (hash_algo == GIT_HASH_UNKNOWN)
-			die(_("unknown object format '%s' specified by server"), hash_name);
-		reader->hash_algo = &hash_algos[hash_algo];
-		packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name);
-	} else {
-		reader->hash_algo = &hash_algos[GIT_HASH_SHA1];
-	}
+	/* Send capabilities */
+	send_capabilities(fd_out, reader);
 
 	if (server_options && server_options->nr &&
 	    server_supports_v2("server-option", 1))
-- 
gitgitgadget

[PATCH 2/8] dir API: add a generalized path_match_flags() function

[Ævar Arnfjörð Bjarmason via GitGitGadget]

From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>

Add a path_match_flags() function and have the two sets of
starts_with_dot_{,dot_}slash() functions added in
63e95beb085 (submodule: port resolve_relative_url from shell to C,
2016-04-15) and a2b26ffb1a8 (fsck: convert gitmodules url to URL
passed to curl, 2020-04-18) be thin wrappers for it.

As the latter of those notes the fsck version was copied from the
initial builtin/submodule--helper.c version.

Since the code added in a2b26ffb1a8 was doing really doing the same as
win32_is_dir_sep() added in 1cadad6f658 (git clone <url>
C:\cygwin\home\USER\repo' is working (again), 2018-12-15) let's move
the latter to git-compat-util.h is a is_xplatform_dir_sep(). We can
then call either it or the platform-specific is_dir_sep() from this
new function.

Let's likewise change code in various other places that was hardcoding
checks for "'/' || '\\'" with the new is_xplatform_dir_sep(). As can
be seen in those callers some of them still concern themselves with
':' (Mac OS classic?), but let's leave the question of whether that
should be consolidated for some other time.

As we expect to make wider use of the "native" case in the future,
define and use two starts_with_dot_{,dot_}slash_native() convenience
wrappers. This makes the diff in builtin/submodule--helper.c much
smaller.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 builtin/submodule--helper.c | 20 ++++++------
 compat/mingw.c              |  2 +-
 compat/win32/path-utils.h   |  6 +---
 dir.c                       | 29 +++++++++++++++++
 dir.h                       | 63 +++++++++++++++++++++++++++++++++++++
 fsck.c                      | 23 ++++----------
 git-compat-util.h           |  8 ++++-
 path.c                      |  2 +-
 submodule-config.c          |  6 ++--
 9 files changed, 121 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2c87ef9364f..b68102bb3ed 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -72,16 +72,6 @@ static char *get_default_remote(void)
 	return repo_get_default_remote(the_repository);
 }
 
-static int starts_with_dot_slash(const char *str)
-{
-	return str[0] == '.' && is_dir_sep(str[1]);
-}
-
-static int starts_with_dot_dot_slash(const char *str)
-{
-	return str[0] == '.' && str[1] == '.' && is_dir_sep(str[2]);
-}
-
 /*
  * Returns 1 if it was the last chop before ':'.
  */
@@ -108,6 +98,16 @@ static int chop_last_dir(char **remoteurl, int is_relative)
 	return 0;
 }
 
+static int starts_with_dot_slash(const char *const path)
+{
+	return starts_with_dot_slash_native(path);;
+}
+
+static int starts_with_dot_dot_slash(const char *const path)
+{
+	return starts_with_dot_dot_slash_native(path);
+}
+
 /*
  * The `url` argument is the URL that navigates to the submodule origin
  * repo. When relative, this URL is relative to the superproject origin
diff --git a/compat/mingw.c b/compat/mingw.c
index 6fe80fdf014..b94b473d978 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -2830,7 +2830,7 @@ not_a_reserved_name:
 			}
 
 			c = path[i];
-			if (c && c != '.' && c != ':' && c != '/' && c != '\\')
+			if (c && c != '.' && c != ':' && !is_xplatform_dir_sep(c))
 				goto not_a_reserved_name;
 
 			/* contains reserved name */
diff --git a/compat/win32/path-utils.h b/compat/win32/path-utils.h
index bba2b644080..65fa3b9263a 100644
--- a/compat/win32/path-utils.h
+++ b/compat/win32/path-utils.h
@@ -6,11 +6,7 @@ int win32_has_dos_drive_prefix(const char *path);
 
 int win32_skip_dos_drive_prefix(char **path);
 #define skip_dos_drive_prefix win32_skip_dos_drive_prefix
-static inline int win32_is_dir_sep(int c)
-{
-	return c == '/' || c == '\\';
-}
-#define is_dir_sep win32_is_dir_sep
+#define is_dir_sep is_xplatform_dir_sep
 static inline char *win32_find_last_dir_sep(const char *path)
 {
 	char *ret = NULL;
diff --git a/dir.c b/dir.c
index f2b0f242101..d25aa6ade55 100644
--- a/dir.c
+++ b/dir.c
@@ -3890,3 +3890,32 @@ void relocate_gitdir(const char *path, const char *old_git_dir, const char *new_
 
 	connect_work_tree_and_git_dir(path, new_git_dir, 0);
 }
+
+int path_match_flags(const char *const str, const enum path_match_flags flags)
+{
+	const char *p = str;
+
+	if (flags & PATH_MATCH_NATIVE &&
+	    flags & PATH_MATCH_XPLATFORM)
+		BUG("path_match_flags() must get one match kind, not multiple!");
+	else if (!(flags & PATH_MATCH_KINDS_MASK))
+		BUG("path_match_flags() must get at least one match kind!");
+
+	if (flags & PATH_MATCH_STARTS_WITH_DOT_SLASH &&
+	    flags & PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH)
+		BUG("path_match_flags() must get one platform kind, not multiple!");
+	else if (!(flags & PATH_MATCH_PLATFORM_MASK))
+		BUG("path_match_flags() must get at least one platform kind!");
+
+	if (*p++ != '.')
+		return 0;
+	if (flags & PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH &&
+	    *p++ != '.')
+		return 0;
+
+	if (flags & PATH_MATCH_NATIVE)
+		return is_dir_sep(*p);
+	else if (flags & PATH_MATCH_XPLATFORM)
+		return is_xplatform_dir_sep(*p);
+	BUG("unreachable");
+}
diff --git a/dir.h b/dir.h
index 8e02dfb505d..7bc862030cf 100644
--- a/dir.h
+++ b/dir.h
@@ -578,4 +578,67 @@ void connect_work_tree_and_git_dir(const char *work_tree,
 void relocate_gitdir(const char *path,
 		     const char *old_git_dir,
 		     const char *new_git_dir);
+
+/**
+ * The "enum path_matches_kind" determines how path_match_flags() will
+ * behave. The flags come in sets, and one (and only one) must be
+ * provided out of each "set":
+ *
+ * PATH_MATCH_NATIVE:
+ *	Path separator is is_dir_sep()
+ * PATH_MATCH_XPLATFORM:
+ *	Path separator is is_xplatform_dir_sep()
+ *
+ * Do we use is_dir_sep() to check for a directory separator
+ * (*_NATIVE), or do we always check for '/' or '\' (*_XPLATFORM). The
+ * "*_NATIVE" version on Windows is the same as "*_XPLATFORM",
+ * everywhere else "*_NATIVE" means "only /".
+ *
+ * PATH_MATCH_STARTS_WITH_DOT_SLASH:
+ *	Match a path starting with "./"
+ * PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH:
+ *	Match a path starting with "../"
+ *
+ * The "/" in the above is adjusted based on the "*_NATIVE" and
+ * "*_XPLATFORM" flags.
+ */
+enum path_match_flags {
+	PATH_MATCH_NATIVE = 1 << 0,
+	PATH_MATCH_XPLATFORM = 1 << 1,
+	PATH_MATCH_STARTS_WITH_DOT_SLASH = 1 << 2,
+	PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH = 1 << 3,
+};
+#define PATH_MATCH_KINDS_MASK (PATH_MATCH_STARTS_WITH_DOT_SLASH | \
+	PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH)
+#define PATH_MATCH_PLATFORM_MASK (PATH_MATCH_NATIVE | PATH_MATCH_XPLATFORM)
+
+/**
+ * path_match_flags() checks if a given "path" matches a given "enum
+ * path_match_flags" criteria.
+ */
+int path_match_flags(const char *const path, const enum path_match_flags f);
+
+/**
+ * starts_with_dot_slash_native(): convenience wrapper for
+ * path_match_flags() with PATH_MATCH_STARTS_WITH_DOT_SLASH and
+ * PATH_MATCH_NATIVE.
+ */
+static inline int starts_with_dot_slash_native(const char *const path)
+{
+	const enum path_match_flags what = PATH_MATCH_STARTS_WITH_DOT_SLASH;
+
+	return path_match_flags(path, what | PATH_MATCH_NATIVE);
+}
+
+/**
+ * starts_with_dot_slash_native(): convenience wrapper for
+ * path_match_flags() with PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH and
+ * PATH_MATCH_NATIVE.
+ */
+static inline int starts_with_dot_dot_slash_native(const char *const path)
+{
+	const enum path_match_flags what = PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH;
+
+	return path_match_flags(path, what | PATH_MATCH_NATIVE);
+}
 #endif
diff --git a/fsck.c b/fsck.c
index 3ec500d707a..dd4822ba1be 100644
--- a/fsck.c
+++ b/fsck.c
@@ -975,27 +975,16 @@ done:
 	return ret;
 }
 
-/*
- * Like builtin/submodule--helper.c's starts_with_dot_slash, but without
- * relying on the platform-dependent is_dir_sep helper.
- *
- * This is for use in checking whether a submodule URL is interpreted as
- * relative to the current directory on any platform, since \ is a
- * directory separator on Windows but not on other platforms.
- */
-static int starts_with_dot_slash(const char *str)
+static int starts_with_dot_slash(const char *const path)
 {
-	return str[0] == '.' && (str[1] == '/' || str[1] == '\\');
+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_SLASH |
+				PATH_MATCH_XPLATFORM);
 }
 
-/*
- * Like starts_with_dot_slash, this is a variant of submodule--helper's
- * helper of the same name with the twist that it accepts backslash as a
- * directory separator even on non-Windows platforms.
- */
-static int starts_with_dot_dot_slash(const char *str)
+static int starts_with_dot_dot_slash(const char *const path)
 {
-	return str[0] == '.' && starts_with_dot_slash(str + 1);
+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH |
+				PATH_MATCH_XPLATFORM);
 }
 
 static int submodule_url_is_relative(const char *url)
diff --git a/git-compat-util.h b/git-compat-util.h
index 58fd813bd01..ba3436db9a1 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -236,6 +236,12 @@
 #include <sys/sysctl.h>
 #endif
 
+/* Used by compat/win32/path-utils.h, and more */
+static inline int is_xplatform_dir_sep(int c)
+{
+	return c == '/' || c == '\\';
+}
+
 #if defined(__CYGWIN__)
 #include "compat/win32/path-utils.h"
 #endif
@@ -416,11 +422,11 @@ static inline int git_skip_dos_drive_prefix(char **path)
 #define skip_dos_drive_prefix git_skip_dos_drive_prefix
 #endif
 
-#ifndef is_dir_sep
 static inline int git_is_dir_sep(int c)
 {
 	return c == '/';
 }
+#ifndef is_dir_sep
 #define is_dir_sep git_is_dir_sep
 #endif
 
diff --git a/path.c b/path.c
index d73146b6cd2..2ab78278943 100644
--- a/path.c
+++ b/path.c
@@ -1413,7 +1413,7 @@ int is_ntfs_dotgit(const char *name)
 
 	for (;;) {
 		c = *(name++);
-		if (!c || c == '\\' || c == '/' || c == ':')
+		if (!c || is_xplatform_dir_sep(c) || c == ':')
 			return 1;
 		if (c != '.' && c != ' ')
 			return 0;
diff --git a/submodule-config.c b/submodule-config.c
index 29668b0620d..ce3beaf5d4f 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -204,17 +204,17 @@ int check_submodule_name(const char *name)
 		return -1;
 
 	/*
-	 * Look for '..' as a path component. Check both '/' and '\\' as
+	 * Look for '..' as a path component. Check is_xplatform_dir_sep() as
 	 * separators rather than is_dir_sep(), because we want the name rules
 	 * to be consistent across platforms.
 	 */
 	goto in_component; /* always start inside component */
 	while (*name) {
 		char c = *name++;
-		if (c == '/' || c == '\\') {
+		if (is_xplatform_dir_sep(c)) {
 in_component:
 			if (name[0] == '.' && name[1] == '.' &&
-			    (!name[2] || name[2] == '/' || name[2] == '\\'))
+			    (!name[2] || is_xplatform_dir_sep(name[2])))
 				return -1;
 		}
 	}
-- 
gitgitgadget

[PATCH 3/8] fetch-pack: add a deref_without_lazy_fetch_extended()

[Ævar Arnfjörð Bjarmason via GitGitGadget]

From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>

Add a version of the deref_without_lazy_fetch function which can be
called with custom oi_flags and to grab information about the
"object_type". This will be used for the bundle-uri client in a
subsequent commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 fetch-pack.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/fetch-pack.c b/fetch-pack.c
index 4e1e88eea09..d0aa3a5c229 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -115,11 +115,12 @@ static void for_each_cached_alternate(struct fetch_negotiator *negotiator,
 		cb(negotiator, cache.items[i]);
 }
 
-static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
-					       int mark_tags_complete)
+static struct commit *deref_without_lazy_fetch_extended(const struct object_id *oid,
+							int mark_tags_complete,
+							enum object_type *type,
+							unsigned int oi_flags)
 {
-	enum object_type type;
-	struct object_info info = { .typep = &type };
+	struct object_info info = { .typep = type };
 	struct commit *commit;
 
 	commit = lookup_commit_in_graph(the_repository, oid);
@@ -128,9 +129,9 @@ static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
 
 	while (1) {
 		if (oid_object_info_extended(the_repository, oid, &info,
-					     OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK))
+					     oi_flags))
 			return NULL;
-		if (type == OBJ_TAG) {
+		if (*type == OBJ_TAG) {
 			struct tag *tag = (struct tag *)
 				parse_object(the_repository, oid);
 
@@ -144,7 +145,7 @@ static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
 		}
 	}
 
-	if (type == OBJ_COMMIT) {
+	if (*type == OBJ_COMMIT) {
 		struct commit *commit = lookup_commit(the_repository, oid);
 		if (!commit || repo_parse_commit(the_repository, commit))
 			return NULL;
@@ -154,6 +155,16 @@ static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
 	return NULL;
 }
 
+
+static struct commit *deref_without_lazy_fetch(const struct object_id *oid,
+					       int mark_tags_complete)
+{
+	enum object_type type;
+	unsigned flags = OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK;
+	return deref_without_lazy_fetch_extended(oid, mark_tags_complete,
+						 &type, flags);
+}
+
 static int rev_list_insert_ref(struct fetch_negotiator *negotiator,
 			       const struct object_id *oid)
 {
-- 
gitgitgadget

[PATCH 4/8] fetch-pack: move --keep=* option filling to a function

[Ævar Arnfjörð Bjarmason via GitGitGadget]

From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>

Move the populating of the --keep=* option argument to "index-pack" to
a static function, a subsequent commit will make use of it in another
function.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 fetch-pack.c | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/fetch-pack.c b/fetch-pack.c
index d0aa3a5c229..b1d90d1914f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -847,6 +847,16 @@ static void parse_gitmodules_oids(int fd, struct oidset *gitmodules_oids)
 	} while (1);
 }
 
+static void add_index_pack_keep_option(struct strvec *args)
+{
+	char hostname[HOST_NAME_MAX + 1];
+
+	if (xgethostname(hostname, sizeof(hostname)))
+		xsnprintf(hostname, sizeof(hostname), "localhost");
+	strvec_pushf(args, "--keep=fetch-pack %"PRIuMAX " on %s",
+		     (uintmax_t)getpid(), hostname);
+}
+
 /*
  * If packfile URIs were provided, pass a non-NULL pointer to index_pack_args.
  * The strings to pass as the --index-pack-arg arguments to http-fetch will be
@@ -916,14 +926,8 @@ static int get_pack(struct fetch_pack_args *args,
 			strvec_push(&cmd.args, "-v");
 		if (args->use_thin_pack)
 			strvec_push(&cmd.args, "--fix-thin");
-		if ((do_keep || index_pack_args) && (args->lock_pack || unpack_limit)) {
-			char hostname[HOST_NAME_MAX + 1];
-			if (xgethostname(hostname, sizeof(hostname)))
-				xsnprintf(hostname, sizeof(hostname), "localhost");
-			strvec_pushf(&cmd.args,
-				     "--keep=fetch-pack %"PRIuMAX " on %s",
-				     (uintmax_t)getpid(), hostname);
-		}
+		if ((do_keep || index_pack_args) && (args->lock_pack || unpack_limit))
+			add_index_pack_keep_option(&cmd.args);
 		if (!index_pack_args && args->check_self_contained_and_connected)
 			strvec_push(&cmd.args, "--check-self-contained-and-connected");
 		else
-- 
gitgitgadget

[PATCH 5/8] http: make http_get_file() external

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <derrickstolee@github.com>

This method will be used in an upcoming extension of git-remote-curl to
download a single file over HTTP(S) by request.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 http.c | 4 ++--
 http.h | 9 +++++++++
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/http.c b/http.c
index 229da4d1488..04e73149357 100644
--- a/http.c
+++ b/http.c
@@ -1945,8 +1945,8 @@ int http_get_strbuf(const char *url,
  * If a previous interrupted download is detected (i.e. a previous temporary
  * file is still around) the download is resumed.
  */
-static int http_get_file(const char *url, const char *filename,
-			 struct http_get_options *options)
+int http_get_file(const char *url, const char *filename,
+		  struct http_get_options *options)
 {
 	int ret;
 	struct strbuf tmpfile = STRBUF_INIT;
diff --git a/http.h b/http.h
index df1590e53a4..ba303cfb372 100644
--- a/http.h
+++ b/http.h
@@ -163,6 +163,15 @@ struct http_get_options {
  */
 int http_get_strbuf(const char *url, struct strbuf *result, struct http_get_options *options);
 
+/*
+ * Downloads a URL and stores the result in the given file.
+ *
+ * If a previous interrupted download is detected (i.e. a previous temporary
+ * file is still around) the download is resumed.
+ */
+int http_get_file(const char *url, const char *filename,
+		  struct http_get_options *options);
+
 int http_fetch_ref(const char *base, struct ref *ref);
 
 /* Helpers for fetching packs */
-- 
gitgitgadget

[PATCH 6/8] remote: move relative_url()

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <derrickstolee@github.com>

This method was initially written in 63e95beb0 (submodule: port
resolve_relative_url from shell to C, 2016-05-15). As we will need
similar functionality in the bundle URI feature, extract this to be
available in remote.h.

The code is almost exactly the same, except for the following trivial
differences:

 * Fix whitespace and wrapping issues with the prototype and argument
   lists.

 * Let's call starts_with_dot_{,dot_}slash_native() instead of the
   functionally identical "starts_with_dot_{,dot_}slash()" wrappers
   "builtin/submodule--helper.c".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 builtin/submodule--helper.c | 141 +++---------------------------------
 remote.c                    |  91 +++++++++++++++++++++++
 remote.h                    |  31 ++++++++
 3 files changed, 134 insertions(+), 129 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index b68102bb3ed..86f38e489b0 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -72,135 +72,6 @@ static char *get_default_remote(void)
 	return repo_get_default_remote(the_repository);
 }
 
-/*
- * Returns 1 if it was the last chop before ':'.
- */
-static int chop_last_dir(char **remoteurl, int is_relative)
-{
-	char *rfind = find_last_dir_sep(*remoteurl);
-	if (rfind) {
-		*rfind = '\0';
-		return 0;
-	}
-
-	rfind = strrchr(*remoteurl, ':');
-	if (rfind) {
-		*rfind = '\0';
-		return 1;
-	}
-
-	if (is_relative || !strcmp(".", *remoteurl))
-		die(_("cannot strip one component off url '%s'"),
-			*remoteurl);
-
-	free(*remoteurl);
-	*remoteurl = xstrdup(".");
-	return 0;
-}
-
-static int starts_with_dot_slash(const char *const path)
-{
-	return starts_with_dot_slash_native(path);;
-}
-
-static int starts_with_dot_dot_slash(const char *const path)
-{
-	return starts_with_dot_dot_slash_native(path);
-}
-
-/*
- * The `url` argument is the URL that navigates to the submodule origin
- * repo. When relative, this URL is relative to the superproject origin
- * URL repo. The `up_path` argument, if specified, is the relative
- * path that navigates from the submodule working tree to the superproject
- * working tree. Returns the origin URL of the submodule.
- *
- * Return either an absolute URL or filesystem path (if the superproject
- * origin URL is an absolute URL or filesystem path, respectively) or a
- * relative file system path (if the superproject origin URL is a relative
- * file system path).
- *
- * When the output is a relative file system path, the path is either
- * relative to the submodule working tree, if up_path is specified, or to
- * the superproject working tree otherwise.
- *
- * NEEDSWORK: This works incorrectly on the domain and protocol part.
- * remote_url      url              outcome          expectation
- * http://a.com/b  ../c             http://a.com/c   as is
- * http://a.com/b/ ../c             http://a.com/c   same as previous line, but
- *                                                   ignore trailing slash in url
- * http://a.com/b  ../../c          http://c         error out
- * http://a.com/b  ../../../c       http:/c          error out
- * http://a.com/b  ../../../../c    http:c           error out
- * http://a.com/b  ../../../../../c    .:c           error out
- * NEEDSWORK: Given how chop_last_dir() works, this function is broken
- * when a local part has a colon in its path component, too.
- */
-static char *relative_url(const char *remote_url,
-				const char *url,
-				const char *up_path)
-{
-	int is_relative = 0;
-	int colonsep = 0;
-	char *out;
-	char *remoteurl = xstrdup(remote_url);
-	struct strbuf sb = STRBUF_INIT;
-	size_t len = strlen(remoteurl);
-
-	if (is_dir_sep(remoteurl[len-1]))
-		remoteurl[len-1] = '\0';
-
-	if (!url_is_local_not_ssh(remoteurl) || is_absolute_path(remoteurl))
-		is_relative = 0;
-	else {
-		is_relative = 1;
-		/*
-		 * Prepend a './' to ensure all relative
-		 * remoteurls start with './' or '../'
-		 */
-		if (!starts_with_dot_slash(remoteurl) &&
-		    !starts_with_dot_dot_slash(remoteurl)) {
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "./%s", remoteurl);
-			free(remoteurl);
-			remoteurl = strbuf_detach(&sb, NULL);
-		}
-	}
-	/*
-	 * When the url starts with '../', remove that and the
-	 * last directory in remoteurl.
-	 */
-	while (url) {
-		if (starts_with_dot_dot_slash(url)) {
-			url += 3;
-			colonsep |= chop_last_dir(&remoteurl, is_relative);
-		} else if (starts_with_dot_slash(url))
-			url += 2;
-		else
-			break;
-	}
-	strbuf_reset(&sb);
-	strbuf_addf(&sb, "%s%s%s", remoteurl, colonsep ? ":" : "/", url);
-	if (ends_with(url, "/"))
-		strbuf_setlen(&sb, sb.len - 1);
-	free(remoteurl);
-
-	if (starts_with_dot_slash(sb.buf))
-		out = xstrdup(sb.buf + 2);
-	else
-		out = xstrdup(sb.buf);
-
-	if (!up_path || !is_relative) {
-		strbuf_release(&sb);
-		return out;
-	}
-
-	strbuf_reset(&sb);
-	strbuf_addf(&sb, "%s%s", up_path, out);
-	free(out);
-	return strbuf_detach(&sb, NULL);
-}
-
 static char *resolve_relative_url(const char *rel_url, const char *up_path, int quiet)
 {
 	char *remoteurl, *resolved_url;
@@ -592,6 +463,18 @@ static int module_foreach(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+static int starts_with_dot_slash(const char *const path)
+{
+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_SLASH |
+				PATH_MATCH_XPLATFORM);
+}
+
+static int starts_with_dot_dot_slash(const char *const path)
+{
+	return path_match_flags(path, PATH_MATCH_STARTS_WITH_DOT_DOT_SLASH |
+				PATH_MATCH_XPLATFORM);
+}
+
 struct init_cb {
 	const char *prefix;
 	const char *superprefix;
diff --git a/remote.c b/remote.c
index 42a4e7106e1..87656138645 100644
--- a/remote.c
+++ b/remote.c
@@ -14,6 +14,7 @@
 #include "strvec.h"
 #include "commit-reach.h"
 #include "advice.h"
+#include "connect.h"
 
 enum map_direction { FROM_SRC, FROM_DST };
 
@@ -2727,3 +2728,93 @@ void remote_state_clear(struct remote_state *remote_state)
 	hashmap_clear_and_free(&remote_state->remotes_hash, struct remote, ent);
 	hashmap_clear_and_free(&remote_state->branches_hash, struct remote, ent);
 }
+
+/*
+ * Returns 1 if it was the last chop before ':'.
+ */
+static int chop_last_dir(char **remoteurl, int is_relative)
+{
+	char *rfind = find_last_dir_sep(*remoteurl);
+	if (rfind) {
+		*rfind = '\0';
+		return 0;
+	}
+
+	rfind = strrchr(*remoteurl, ':');
+	if (rfind) {
+		*rfind = '\0';
+		return 1;
+	}
+
+	if (is_relative || !strcmp(".", *remoteurl))
+		die(_("cannot strip one component off url '%s'"),
+			*remoteurl);
+
+	free(*remoteurl);
+	*remoteurl = xstrdup(".");
+	return 0;
+}
+
+char *relative_url(const char *remote_url, const char *url,
+		   const char *up_path)
+{
+	int is_relative = 0;
+	int colonsep = 0;
+	char *out;
+	char *remoteurl = xstrdup(remote_url);
+	struct strbuf sb = STRBUF_INIT;
+	size_t len = strlen(remoteurl);
+
+	if (is_dir_sep(remoteurl[len-1]))
+		remoteurl[len-1] = '\0';
+
+	if (!url_is_local_not_ssh(remoteurl) || is_absolute_path(remoteurl))
+		is_relative = 0;
+	else {
+		is_relative = 1;
+		/*
+		 * Prepend a './' to ensure all relative
+		 * remoteurls start with './' or '../'
+		 */
+		if (!starts_with_dot_slash_native(remoteurl) &&
+		    !starts_with_dot_dot_slash_native(remoteurl)) {
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "./%s", remoteurl);
+			free(remoteurl);
+			remoteurl = strbuf_detach(&sb, NULL);
+		}
+	}
+	/*
+	 * When the url starts with '../', remove that and the
+	 * last directory in remoteurl.
+	 */
+	while (url) {
+		if (starts_with_dot_dot_slash_native(url)) {
+			url += 3;
+			colonsep |= chop_last_dir(&remoteurl, is_relative);
+		} else if (starts_with_dot_slash_native(url))
+			url += 2;
+		else
+			break;
+	}
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "%s%s%s", remoteurl, colonsep ? ":" : "/", url);
+	if (ends_with(url, "/"))
+		strbuf_setlen(&sb, sb.len - 1);
+	free(remoteurl);
+
+	if (starts_with_dot_slash_native(sb.buf))
+		out = xstrdup(sb.buf + 2);
+	else
+		out = xstrdup(sb.buf);
+
+	if (!up_path || !is_relative) {
+		strbuf_release(&sb);
+		return out;
+	}
+
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "%s%s", up_path, out);
+	free(out);
+	return strbuf_detach(&sb, NULL);
+}
diff --git a/remote.h b/remote.h
index 4a1209ae2c8..f18fd27e530 100644
--- a/remote.h
+++ b/remote.h
@@ -409,4 +409,35 @@ int parseopt_push_cas_option(const struct option *, const char *arg, int unset);
 int is_empty_cas(const struct push_cas_option *);
 void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *);
 
+/*
+ * The `url` argument is the URL that navigates to the submodule origin
+ * repo. When relative, this URL is relative to the superproject origin
+ * URL repo. The `up_path` argument, if specified, is the relative
+ * path that navigates from the submodule working tree to the superproject
+ * working tree. Returns the origin URL of the submodule.
+ *
+ * Return either an absolute URL or filesystem path (if the superproject
+ * origin URL is an absolute URL or filesystem path, respectively) or a
+ * relative file system path (if the superproject origin URL is a relative
+ * file system path).
+ *
+ * When the output is a relative file system path, the path is either
+ * relative to the submodule working tree, if up_path is specified, or to
+ * the superproject working tree otherwise.
+ *
+ * NEEDSWORK: This works incorrectly on the domain and protocol part.
+ * remote_url      url              outcome          expectation
+ * http://a.com/b  ../c             http://a.com/c   as is
+ * http://a.com/b/ ../c             http://a.com/c   same as previous line, but
+ *                                                   ignore trailing slash in url
+ * http://a.com/b  ../../c          http://c         error out
+ * http://a.com/b  ../../../c       http:/c          error out
+ * http://a.com/b  ../../../../c    http:c           error out
+ * http://a.com/b  ../../../../../c    .:c           error out
+ * NEEDSWORK: Given how chop_last_dir() works, this function is broken
+ * when a local part has a colon in its path component, too.
+ */
+char *relative_url(const char *remote_url, const char *url,
+		   const char *up_path);
+
 #endif
-- 
gitgitgadget

[PATCH 7/8] remote: allow relative_url() to return an absolute url

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <derrickstolee@github.com>

When the 'url' parameter was absolute, the previous implementation would
concatenate 'remote_url' with 'url'. Instead, we want to return 'url' in
this case.

The documentation now discusses what happens when supplying two
absolute URLs.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 remote.c | 12 ++++++++++--
 remote.h |  1 +
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/remote.c b/remote.c
index 87656138645..7576f673fcd 100644
--- a/remote.c
+++ b/remote.c
@@ -2761,10 +2761,18 @@ char *relative_url(const char *remote_url, const char *url,
 	int is_relative = 0;
 	int colonsep = 0;
 	char *out;
-	char *remoteurl = xstrdup(remote_url);
+	char *remoteurl;
 	struct strbuf sb = STRBUF_INIT;
-	size_t len = strlen(remoteurl);
+	size_t len;
+
+	if (!url_is_local_not_ssh(url) || is_absolute_path(url))
+		return xstrdup(url);
+
+	len = strlen(remote_url);
+	if (!len)
+		BUG("invalid empty remote_url");
 
+	remoteurl = xstrdup(remote_url);
 	if (is_dir_sep(remoteurl[len-1]))
 		remoteurl[len-1] = '\0';
 
diff --git a/remote.h b/remote.h
index f18fd27e530..dd4402436f1 100644
--- a/remote.h
+++ b/remote.h
@@ -434,6 +434,7 @@ void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *);
  * http://a.com/b  ../../../c       http:/c          error out
  * http://a.com/b  ../../../../c    http:c           error out
  * http://a.com/b  ../../../../../c    .:c           error out
+ * http://a.com/b  http://d.org/e   http://d.org/e   as is
  * NEEDSWORK: Given how chop_last_dir() works, this function is broken
  * when a local part has a colon in its path component, too.
  */
-- 
gitgitgadget

[PATCH 8/8] bundle.h: make "fd" version of read_bundle_header() public

[Ævar Arnfjörð Bjarmason via GitGitGadget]

From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>

Change the parse_bundle_header() function to be non-static, and rename
it to parse_bundle_header_fd(). The parse_bundle_header() function is
already public, and it's a thin wrapper around this function. This
will be used by code that wants to pass a fd to the bundle API.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 bundle.c | 8 ++++----
 bundle.h | 2 ++
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/bundle.c b/bundle.c
index d50cfb5aa7e..5fa41a52f11 100644
--- a/bundle.c
+++ b/bundle.c
@@ -66,8 +66,8 @@ static int parse_bundle_signature(struct bundle_header *header, const char *line
 	return -1;
 }
 
-static int parse_bundle_header(int fd, struct bundle_header *header,
-			       const char *report_path)
+int read_bundle_header_fd(int fd, struct bundle_header *header,
+			  const char *report_path)
 {
 	struct strbuf buf = STRBUF_INIT;
 	int status = 0;
@@ -143,7 +143,7 @@ int read_bundle_header(const char *path, struct bundle_header *header)
 
 	if (fd < 0)
 		return error(_("could not open '%s'"), path);
-	return parse_bundle_header(fd, header, path);
+	return read_bundle_header_fd(fd, header, path);
 }
 
 int is_bundle(const char *path, int quiet)
@@ -153,7 +153,7 @@ int is_bundle(const char *path, int quiet)
 
 	if (fd < 0)
 		return 0;
-	fd = parse_bundle_header(fd, &header, quiet ? NULL : path);
+	fd = read_bundle_header_fd(fd, &header, quiet ? NULL : path);
 	if (fd >= 0)
 		close(fd);
 	bundle_header_release(&header);
diff --git a/bundle.h b/bundle.h
index 7fef2108f43..0c052f54964 100644
--- a/bundle.h
+++ b/bundle.h
@@ -24,6 +24,8 @@ void bundle_header_release(struct bundle_header *header);
 
 int is_bundle(const char *path, int quiet);
 int read_bundle_header(const char *path, struct bundle_header *header);
+int read_bundle_header_fd(int fd, struct bundle_header *header,
+			  const char *report_path);
 int create_bundle(struct repository *r, const char *path,
 		  int argc, const char **argv, struct strvec *pack_options,
 		  int version);
-- 
gitgitgadget