[PATCH v3 00/23] backend-struct-db

[PATCH v3 00/23] backend-struct-db

[Ronnie Sahlberg]

List, please review

This series is called backend-struct-db and is also available at
https://github.com/rsahlberg/git/tree/backend-struct-db

This series is built on and follows after the series
ref-transactions-send-pack


This series does not change any logic or behaviour but mainly just shuffles
code around and adds method pointers for the backend functions.

The first patch adds a new public function for checking if a refname is
available or not. This function is needed not because we want to have
different is_refname_available semantics for different backends, we don't,
but because its implementation is quite dependant on the backend type.

15 of the patches, the refs-common.c patches, focuses on moving all backend
agnostic refs functions to a common file. This file will contain all
backend agnostic refs functions.

The last 6 patches adds a backend structure with the methods we need to
describe a pluggable backend. Currently we only have one built in backend,
the current files based backend. These patches do not change any of the
behavior other than that we now call the methods through backend specific
wrapper functions rather than calling them directly.


At this stage we now have a defined set of methods needed for a refs
backend and we can start building and adding new types of ref backends
to git.


Version 3:
- Rework the patches so that we host the common functions in refs.c
  instead of refs-common.c
Version 2:
- Do not use C99 style initializers as suggested by David Turner.
- Make head_ref_namespaced a common function instead of a backend function


Ronnie Sahlberg (23):
  refs.c: create a public function for is_refname_available
  refs-be-files.c: rename refs to refs-be-files
  refs.c: add a new refs.c file to hold all common refs code
  refs.c: move update_ref to refs.c
  refs.c: move delete_ref to the common code
  refs.c: move rename_ref to the common code
  refs.c: move read_ref_at to the common refs file
  refs.c: move the hidden refs functions to the common code
  refs.c: move dwim and friend functions to the common refs code
  refs.c: move warn_if_dangling_symref* to the common code
  refs.c: move read_ref, read_ref_full and ref_exists to the common code
  refs.c: move resolve_refdup to common
  refs.c: move check_refname_component to the common code
  refs.c: move is_branch to the common code
  refs.c: move names_conflict to the common code
  refs.c: move prettify_refname to the common code
  refs.c: move ref iterators to the common code
  refs.c: move head_ref_namespaced to the common code
  refs-be-files.c: add a backend method structure with transaction
    functions
  refs-be-files.c: add reflog backend methods
  refs-be-files.c: add methods for misc ref operations
  refs-be-files.c: add methods for head_ref*
  refs-be-files.c: add methods for the ref iterators

 Makefile        |    1 +
 refs-be-files.c | 3330 +++++++++++++++++++++++++++++++++++++++++
 refs.c          | 4457 +++++++++----------------------------------------------
 refs.h          |  107 ++
 4 files changed, 4110 insertions(+), 3785 deletions(-)
 create mode 100644 refs-be-files.c

-- 
2.0.1.552.g1af257a

[PATCH v3 01/23] refs.c: create a public function for is_refname_available

[Ronnie Sahlberg]

Export a generic is_refname_available() function. We will need this
as a public shared function later when we add additional refs backends
since we want to keep using the same rules for ref naming across
all backends.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs.c | 29 ++++++++++++++++++-----------
 refs.h |  6 ++++++
 2 files changed, 24 insertions(+), 11 deletions(-)

diff --git a/refs.c b/refs.c
index 7e13c0f..4a22513 100644
--- a/refs.c
+++ b/refs.c
@@ -830,9 +830,9 @@ static int name_conflict_fn(struct ref_entry *entry, void *cb_data)
  * operation). skip contains a list of refs we want to skip checking for
  * conflicts with.
  */
-static int is_refname_available(const char *refname,
-				struct ref_dir *dir,
-				const char **skip, int skipnum)
+static int is_refname_available_dir(const char *refname,
+				    struct ref_dir *dir,
+				    const char **skip, int skipnum)
 {
 	struct name_conflict_cb data;
 	data.refname = refname;
@@ -1238,6 +1238,18 @@ static struct ref_dir *get_loose_refs(struct ref_cache *refs)
 	return get_ref_dir(refs->loose);
 }
 
+int is_refname_available(const char *refname, const char **skip, int skipnum)
+{
+	if (!is_refname_available_dir(refname, get_packed_refs(&ref_cache),
+				      skip, skipnum))
+		return 0;
+
+	if (!is_refname_available_dir(refname, get_loose_refs(&ref_cache),
+				      skip, skipnum))
+		return 0;
+	return 1;
+}
+
 /* We allow "recursive" symbolic refs. Only within reason, though */
 #define MAXDEPTH 5
 #define MAXREFLEN (1024)
@@ -2168,8 +2180,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 	 * name is a proper prefix of our refname.
 	 */
 	if (missing &&
-	     !is_refname_available(refname, get_packed_refs(&ref_cache),
-				   skip, skipnum)) {
+	     !is_refname_available_dir(refname, get_packed_refs(&ref_cache),
+				       skip, skipnum)) {
 		last_errno = ENOTDIR;
 		goto error_return;
 	}
@@ -2676,12 +2688,7 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 		return 1;
 	}
 
-	if (!is_refname_available(newrefname, get_packed_refs(&ref_cache),
-				  &oldrefname, 1))
-		return 1;
-
-	if (!is_refname_available(newrefname, get_loose_refs(&ref_cache),
-				  &oldrefname, 1))
+	if (!is_refname_available(newrefname, &oldrefname, 1))
 		return 1;
 
 	log = reflog_exists(oldrefname);
diff --git a/refs.h b/refs.h
index f44b5c8..d526da0 100644
--- a/refs.h
+++ b/refs.h
@@ -131,6 +131,12 @@ extern int ref_exists(const char *);
 extern int is_branch(const char *refname);
 
 /*
+ * Check that a particular refname is available for creation. skip contains
+ * a list of refnames to exclude from the refname collision tests.
+ */
+int is_refname_available(const char *refname, const char **skip, int skipnum);
+
+/*
  * If refname is a non-symbolic reference that refers to a tag object,
  * and the tag can be (recursively) dereferenced to a non-tag object,
  * store the SHA1 of the referred-to object to sha1 and return 0.  If
-- 
2.0.1.552.g1af257a

[PATCH v3 03/23] refs.c: add a new refs.c file to hold all common refs code

[Ronnie Sahlberg]

Create a new erfs.c file that will be used to hold all the refs
code that is backend agnostic and will be shared across all backends.

The reason we renamed everything to refs-be-files.c in the previous patch
and now start moving the common code back to the new refs.c file
instead of the other way around is the etive volumes of code.

With the ref_cache, packed refs and loose ref handling that are all
part of the files based implementation the backend specific part
of the old refs.c file is several times larger than the backend agnostic
part. Therefore it makes more sense to first rename everything to be
part of the files based backend and then move the parts that can be used
as common code back to refs.c.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 Makefile | 1 +
 refs.c   | 3 +++
 2 files changed, 4 insertions(+)
 create mode 100644 refs.c

diff --git a/Makefile b/Makefile
index e010ad1..937d22a 100644
--- a/Makefile
+++ b/Makefile
@@ -857,6 +857,7 @@ LIB_OBJS += quote.o
 LIB_OBJS += reachable.o
 LIB_OBJS += read-cache.o
 LIB_OBJS += reflog-walk.o
+LIB_OBJS += refs.o
 LIB_OBJS += refs-be-files.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace_object.o
diff --git a/refs.c b/refs.c
new file mode 100644
index 0000000..77492ff
--- /dev/null
+++ b/refs.c
@@ -0,0 +1,3 @@
+/*
+ * Common refs code for all backends.
+ */
-- 
2.0.1.552.g1af257a

[PATCH v3 04/23] refs.c: move update_ref to refs.c

[Ronnie Sahlberg]

This change moves update_ref() to the refs.c file since this function
does not contain any backend specific code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 23 -----------------------
 refs.c          | 25 +++++++++++++++++++++++++
 2 files changed, 25 insertions(+), 23 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 4a22513..eb66cf7 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3576,29 +3576,6 @@ int transaction_delete_sha1(struct ref_transaction *transaction,
 				      old_sha1, flags, have_old, msg, err);
 }
 
-int update_ref(const char *action, const char *refname,
-	       const unsigned char *sha1, const unsigned char *oldval,
-	       int flags, struct strbuf *e)
-{
-	struct ref_transaction *t;
-	struct strbuf err = STRBUF_INIT;
-
-	t = transaction_begin(&err);
-	if (!t ||
-	    transaction_update_sha1(t, refname, sha1, oldval, flags,
-				    !!oldval, action, &err) ||
-	    transaction_commit(t, &err)) {
-		const char *str = "update_ref failed for ref '%s': %s";
-
-		transaction_free(t);
-		if (e)
-			strbuf_addf(e, str, refname, err.buf);
-		strbuf_release(&err);
-		return 1;
-	}
-	return 0;
-}
-
 static int ref_update_compare(const void *r1, const void *r2)
 {
 	const struct ref_update * const *u1 = r1;
diff --git a/refs.c b/refs.c
index 77492ff..5f6a1e7 100644
--- a/refs.c
+++ b/refs.c
@@ -1,3 +1,28 @@
 /*
  * Common refs code for all backends.
  */
+#include "cache.h"
+#include "refs.h"
+
+int update_ref(const char *action, const char *refname,
+	       const unsigned char *sha1, const unsigned char *oldval,
+	       int flags, struct strbuf *e)
+{
+	struct ref_transaction *t;
+	struct strbuf err = STRBUF_INIT;
+
+	t = transaction_begin(&err);
+	if (!t ||
+	    transaction_update_sha1(t, refname, sha1, oldval, flags,
+				    !!oldval, action, &err) ||
+	    transaction_commit(t, &err)) {
+		const char *str = "update_ref failed for ref '%s': %s";
+
+		transaction_free(t);
+		if (e)
+			strbuf_addf(e, str, refname, err.buf);
+		strbuf_release(&err);
+		return 1;
+	}
+	return 0;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 05/23] refs.c: move delete_ref to the common code

[Ronnie Sahlberg]

This change moves delete_ref() to the refs.c file since this function
does not contain any backend specific code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 19 -------------------
 refs.c          | 19 +++++++++++++++++++
 2 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index eb66cf7..faf794c 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2622,25 +2622,6 @@ static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
 	return 0;
 }
 
-int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
-{
-	struct ref_transaction *transaction;
-	struct strbuf err = STRBUF_INIT;
-
-	transaction = transaction_begin(&err);
-	if (!transaction ||
-	    transaction_delete_sha1(transaction, refname, sha1, delopt,
-				    sha1 && !is_null_sha1(sha1), NULL, &err) ||
-	    transaction_commit(transaction, &err)) {
-		error("%s", err.buf);
-		transaction_free(transaction);
-		strbuf_release(&err);
-		return 1;
-	}
-	transaction_free(transaction);
-	return 0;
-}
-
 struct rename_reflog_cb {
 	struct ref_transaction *transaction;
 	const char *refname;
diff --git a/refs.c b/refs.c
index 5f6a1e7..70b12eb 100644
--- a/refs.c
+++ b/refs.c
@@ -26,3 +26,22 @@ int update_ref(const char *action, const char *refname,
 	}
 	return 0;
 }
+
+int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
+{
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+
+	transaction = transaction_begin(&err);
+	if (!transaction ||
+	    transaction_delete_sha1(transaction, refname, sha1, delopt,
+				    sha1 && !is_null_sha1(sha1), NULL, &err) ||
+	    transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		transaction_free(transaction);
+		strbuf_release(&err);
+		return 1;
+	}
+	transaction_free(transaction);
+	return 0;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 06/23] refs.c: move rename_ref to the common code

[Ronnie Sahlberg]

This change moves rename_ref() to the refs.c file since this function
does not contain any backend specific code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 92 ---------------------------------------------------------
 refs.c          | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 92 insertions(+), 92 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index faf794c..7d579be 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2622,98 +2622,6 @@ static int delete_ref_loose(struct ref_lock *lock, int flag, struct strbuf *err)
 	return 0;
 }
 
-struct rename_reflog_cb {
-	struct ref_transaction *transaction;
-	const char *refname;
-	struct strbuf *err;
-};
-
-static int rename_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
-			     const char *id, unsigned long timestamp, int tz,
-			     const char *message, void *cb_data)
-{
-	struct rename_reflog_cb *cb = cb_data;
-	struct reflog_committer_info ci;
-
-	memset(&ci, 0, sizeof(ci));
-	ci.id = id;
-	ci.timestamp = timestamp;
-	ci.tz = tz;
-	return transaction_update_reflog(cb->transaction, cb->refname,
-					 nsha1, osha1, &ci, message, 0,
-					 cb->err);
-}
-
-int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)
-{
-	unsigned char sha1[20];
-	int flag = 0, log;
-	struct ref_transaction *transaction = NULL;
-	struct strbuf err = STRBUF_INIT;
-	const char *symref = NULL;
-	struct rename_reflog_cb cb;
-	struct reflog_committer_info ci;
-
-	memset(&ci, 0, sizeof(ci));
-	ci.committer_info = git_committer_info(0);
-
-	symref = resolve_ref_unsafe(oldrefname, sha1,
-				    RESOLVE_REF_READING, &flag);
-	if (flag & REF_ISSYMREF) {
-		error("refname %s is a symbolic ref, renaming it is not supported",
-			oldrefname);
-		return 1;
-	}
-	if (!symref) {
-		error("refname %s not found", oldrefname);
-		return 1;
-	}
-
-	if (!is_refname_available(newrefname, &oldrefname, 1))
-		return 1;
-
-	log = reflog_exists(oldrefname);
-	transaction = transaction_begin(&err);
-	if (!transaction)
-		goto fail;
-
-	if (strcmp(oldrefname, newrefname)) {
-		if (log && transaction_update_reflog(transaction, newrefname,
-						     sha1, sha1, &ci, NULL,
-						     REFLOG_TRUNCATE, &err))
-			goto fail;
-		cb.transaction = transaction;
-		cb.refname = newrefname;
-		cb.err = &err;
-		if (log && for_each_reflog_ent(oldrefname, rename_reflog_ent,
-					       &cb))
-			goto fail;
-
-		if (transaction_delete_sha1(transaction, oldrefname, sha1,
-					    REF_NODEREF,
-					    1, NULL, &err))
-			goto fail;
-	}
-	if (transaction_update_sha1(transaction, newrefname, sha1,
-				    NULL, 0, 0, NULL, &err))
-		goto fail;
-	if (log && transaction_update_reflog(transaction, newrefname, sha1,
-					     sha1, &ci, logmsg,
-					     REFLOG_COMMITTER_INFO_IS_VALID,
-					     &err))
-		goto fail;
-	if (transaction_commit(transaction, &err))
-		goto fail;
-	transaction_free(transaction);
-	return 0;
-
- fail:
-	error("rename_ref failed: %s", err.buf);
-	strbuf_release(&err);
-	transaction_free(transaction);
-	return 1;
-}
-
 static int close_ref(struct ref_lock *lock)
 {
 	if (close_lock_file(lock->lk))
diff --git a/refs.c b/refs.c
index 70b12eb..319eafa 100644
--- a/refs.c
+++ b/refs.c
@@ -45,3 +45,95 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 	transaction_free(transaction);
 	return 0;
 }
+
+struct rename_reflog_cb {
+	struct ref_transaction *transaction;
+	const char *refname;
+	struct strbuf *err;
+};
+
+static int rename_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
+			     const char *id, unsigned long timestamp, int tz,
+			     const char *message, void *cb_data)
+{
+	struct rename_reflog_cb *cb = cb_data;
+	struct reflog_committer_info ci;
+
+	memset(&ci, 0, sizeof(ci));
+	ci.id = id;
+	ci.timestamp = timestamp;
+	ci.tz = tz;
+	return transaction_update_reflog(cb->transaction, cb->refname,
+					 nsha1, osha1, &ci, message, 0,
+					 cb->err);
+}
+
+int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsg)
+{
+	unsigned char sha1[20];
+	int flag = 0, log;
+	struct ref_transaction *transaction = NULL;
+	struct strbuf err = STRBUF_INIT;
+	const char *symref = NULL;
+	struct rename_reflog_cb cb;
+	struct reflog_committer_info ci;
+
+	memset(&ci, 0, sizeof(ci));
+	ci.committer_info = git_committer_info(0);
+
+	symref = resolve_ref_unsafe(oldrefname, sha1,
+				    RESOLVE_REF_READING, &flag);
+	if (flag & REF_ISSYMREF) {
+		error("refname %s is a symbolic ref, renaming it is not supported",
+			oldrefname);
+		return 1;
+	}
+	if (!symref) {
+		error("refname %s not found", oldrefname);
+		return 1;
+	}
+
+	if (!is_refname_available(newrefname, &oldrefname, 1))
+		return 1;
+
+	log = reflog_exists(oldrefname);
+	transaction = transaction_begin(&err);
+	if (!transaction)
+		goto fail;
+
+	if (strcmp(oldrefname, newrefname)) {
+		if (log && transaction_update_reflog(transaction, newrefname,
+						     sha1, sha1, &ci, NULL,
+						     REFLOG_TRUNCATE, &err))
+			goto fail;
+		cb.transaction = transaction;
+		cb.refname = newrefname;
+		cb.err = &err;
+		if (log && for_each_reflog_ent(oldrefname, rename_reflog_ent,
+					       &cb))
+			goto fail;
+
+		if (transaction_delete_sha1(transaction, oldrefname, sha1,
+					    REF_NODEREF,
+					    1, NULL, &err))
+			goto fail;
+	}
+	if (transaction_update_sha1(transaction, newrefname, sha1,
+				    NULL, 0, 0, NULL, &err))
+		goto fail;
+	if (log && transaction_update_reflog(transaction, newrefname, sha1,
+					     sha1, &ci, logmsg,
+					     REFLOG_COMMITTER_INFO_IS_VALID,
+					     &err))
+		goto fail;
+	if (transaction_commit(transaction, &err))
+		goto fail;
+	transaction_free(transaction);
+	return 0;
+
+ fail:
+	error("rename_ref failed: %s", err.buf);
+	strbuf_release(&err);
+	transaction_free(transaction);
+	return 1;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 07/23] refs.c: move read_ref_at to the common refs file

[Ronnie Sahlberg]

This change moves read_ref_at() to the refs.c file since this function
does not contain any backend specific code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 114 --------------------------------------------------------
 refs.c          | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 114 insertions(+), 114 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 7d579be..52ca0bb 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2935,120 +2935,6 @@ int create_symref(const char *ref_target, const char *refs_heads_master,
 	return 0;
 }
 
-struct read_ref_at_cb {
-	const char *refname;
-	unsigned long at_time;
-	int cnt;
-	int reccnt;
-	unsigned char *sha1;
-	int found_it;
-
-	unsigned char osha1[20];
-	unsigned char nsha1[20];
-	int tz;
-	unsigned long date;
-	char **msg;
-	unsigned long *cutoff_time;
-	int *cutoff_tz;
-	int *cutoff_cnt;
-};
-
-static int read_ref_at_ent(unsigned char *osha1, unsigned char *nsha1,
-		const char *id, unsigned long timestamp, int tz,
-		const char *message, void *cb_data)
-{
-	struct read_ref_at_cb *cb = cb_data;
-
-	cb->reccnt++;
-	cb->tz = tz;
-	cb->date = timestamp;
-
-	if (timestamp <= cb->at_time || cb->cnt == 0) {
-		if (cb->msg)
-			*cb->msg = xstrdup(message);
-		if (cb->cutoff_time)
-			*cb->cutoff_time = timestamp;
-		if (cb->cutoff_tz)
-			*cb->cutoff_tz = tz;
-		if (cb->cutoff_cnt)
-			*cb->cutoff_cnt = cb->reccnt - 1;
-		/*
-		 * we have not yet updated cb->[n|o]sha1 so they still
-		 * hold the values for the previous record.
-		 */
-		if (!is_null_sha1(cb->osha1)) {
-			hashcpy(cb->sha1, nsha1);
-			if (hashcmp(cb->osha1, nsha1))
-				warning("Log for ref %s has gap after %s.",
-					cb->refname, show_date(cb->date, cb->tz, DATE_RFC2822));
-		}
-		else if (cb->date == cb->at_time)
-			hashcpy(cb->sha1, nsha1);
-		else if (hashcmp(nsha1, cb->sha1))
-			warning("Log for ref %s unexpectedly ended on %s.",
-				cb->refname, show_date(cb->date, cb->tz,
-						   DATE_RFC2822));
-		hashcpy(cb->osha1, osha1);
-		hashcpy(cb->nsha1, nsha1);
-		cb->found_it = 1;
-		return 1;
-	}
-	hashcpy(cb->osha1, osha1);
-	hashcpy(cb->nsha1, nsha1);
-	if (cb->cnt > 0)
-		cb->cnt--;
-	return 0;
-}
-
-static int read_ref_at_ent_oldest(unsigned char *osha1, unsigned char *nsha1,
-				  const char *id, unsigned long timestamp,
-				  int tz, const char *message, void *cb_data)
-{
-	struct read_ref_at_cb *cb = cb_data;
-
-	if (cb->msg)
-		*cb->msg = xstrdup(message);
-	if (cb->cutoff_time)
-		*cb->cutoff_time = timestamp;
-	if (cb->cutoff_tz)
-		*cb->cutoff_tz = tz;
-	if (cb->cutoff_cnt)
-		*cb->cutoff_cnt = cb->reccnt;
-	hashcpy(cb->sha1, osha1);
-	if (is_null_sha1(cb->sha1))
-		hashcpy(cb->sha1, nsha1);
-	/* We just want the first entry */
-	return 1;
-}
-
-int read_ref_at(const char *refname, unsigned long at_time, int cnt,
-		unsigned char *sha1, char **msg,
-		unsigned long *cutoff_time, int *cutoff_tz, int *cutoff_cnt)
-{
-	struct read_ref_at_cb cb;
-
-	memset(&cb, 0, sizeof(cb));
-	cb.refname = refname;
-	cb.at_time = at_time;
-	cb.cnt = cnt;
-	cb.msg = msg;
-	cb.cutoff_time = cutoff_time;
-	cb.cutoff_tz = cutoff_tz;
-	cb.cutoff_cnt = cutoff_cnt;
-	cb.sha1 = sha1;
-
-	for_each_reflog_ent_reverse(refname, read_ref_at_ent, &cb);
-
-	if (!cb.reccnt)
-		die("Log for %s is empty.", refname);
-	if (cb.found_it)
-		return 0;
-
-	for_each_reflog_ent(refname, read_ref_at_ent_oldest, &cb);
-
-	return 1;
-}
-
 int reflog_exists(const char *refname)
 {
 	struct stat st;
diff --git a/refs.c b/refs.c
index 319eafa..072cd39 100644
--- a/refs.c
+++ b/refs.c
@@ -137,3 +137,117 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logms
 	transaction_free(transaction);
 	return 1;
 }
+
+struct read_ref_at_cb {
+	const char *refname;
+	unsigned long at_time;
+	int cnt;
+	int reccnt;
+	unsigned char *sha1;
+	int found_it;
+
+	unsigned char osha1[20];
+	unsigned char nsha1[20];
+	int tz;
+	unsigned long date;
+	char **msg;
+	unsigned long *cutoff_time;
+	int *cutoff_tz;
+	int *cutoff_cnt;
+};
+
+static int read_ref_at_ent(unsigned char *osha1, unsigned char *nsha1,
+		const char *id, unsigned long timestamp, int tz,
+		const char *message, void *cb_data)
+{
+	struct read_ref_at_cb *cb = cb_data;
+
+	cb->reccnt++;
+	cb->tz = tz;
+	cb->date = timestamp;
+
+	if (timestamp <= cb->at_time || cb->cnt == 0) {
+		if (cb->msg)
+			*cb->msg = xstrdup(message);
+		if (cb->cutoff_time)
+			*cb->cutoff_time = timestamp;
+		if (cb->cutoff_tz)
+			*cb->cutoff_tz = tz;
+		if (cb->cutoff_cnt)
+			*cb->cutoff_cnt = cb->reccnt - 1;
+		/*
+		 * we have not yet updated cb->[n|o]sha1 so they still
+		 * hold the values for the previous record.
+		 */
+		if (!is_null_sha1(cb->osha1)) {
+			hashcpy(cb->sha1, nsha1);
+			if (hashcmp(cb->osha1, nsha1))
+				warning("Log for ref %s has gap after %s.",
+					cb->refname, show_date(cb->date, cb->tz, DATE_RFC2822));
+		}
+		else if (cb->date == cb->at_time)
+			hashcpy(cb->sha1, nsha1);
+		else if (hashcmp(nsha1, cb->sha1))
+			warning("Log for ref %s unexpectedly ended on %s.",
+				cb->refname, show_date(cb->date, cb->tz,
+						   DATE_RFC2822));
+		hashcpy(cb->osha1, osha1);
+		hashcpy(cb->nsha1, nsha1);
+		cb->found_it = 1;
+		return 1;
+	}
+	hashcpy(cb->osha1, osha1);
+	hashcpy(cb->nsha1, nsha1);
+	if (cb->cnt > 0)
+		cb->cnt--;
+	return 0;
+}
+
+static int read_ref_at_ent_oldest(unsigned char *osha1, unsigned char *nsha1,
+				  const char *id, unsigned long timestamp,
+				  int tz, const char *message, void *cb_data)
+{
+	struct read_ref_at_cb *cb = cb_data;
+
+	if (cb->msg)
+		*cb->msg = xstrdup(message);
+	if (cb->cutoff_time)
+		*cb->cutoff_time = timestamp;
+	if (cb->cutoff_tz)
+		*cb->cutoff_tz = tz;
+	if (cb->cutoff_cnt)
+		*cb->cutoff_cnt = cb->reccnt;
+	hashcpy(cb->sha1, osha1);
+	if (is_null_sha1(cb->sha1))
+		hashcpy(cb->sha1, nsha1);
+	/* We just want the first entry */
+	return 1;
+}
+
+int read_ref_at(const char *refname, unsigned long at_time, int cnt,
+		unsigned char *sha1, char **msg,
+		unsigned long *cutoff_time, int *cutoff_tz, int *cutoff_cnt)
+{
+	struct read_ref_at_cb cb;
+
+	memset(&cb, 0, sizeof(cb));
+	cb.refname = refname;
+	cb.at_time = at_time;
+	cb.cnt = cnt;
+	cb.msg = msg;
+	cb.cutoff_time = cutoff_time;
+	cb.cutoff_tz = cutoff_tz;
+	cb.cutoff_cnt = cutoff_cnt;
+	cb.sha1 = sha1;
+
+	for_each_reflog_ent_reverse(refname, read_ref_at_ent, &cb);
+
+	if (!cb.reccnt)
+		die("Log for %s is empty.", refname);
+	if (cb.found_it)
+		return 0;
+
+	for_each_reflog_ent(refname, read_ref_at_ent_oldest, &cb);
+
+	return 1;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 08/23] refs.c: move the hidden refs functions to the common code

[Ronnie Sahlberg]

This change moves the hidden refs functions to the refs.c file since
these functions do not contain any backend specific code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 43 -------------------------------------------
 refs.c          | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 43 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 52ca0bb..6181edf 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -3796,46 +3796,3 @@ char *shorten_unambiguous_ref(const char *refname, int strict)
 	free(short_name);
 	return xstrdup(refname);
 }
-
-static struct string_list *hide_refs;
-
-int parse_hide_refs_config(const char *var, const char *value, const char *section)
-{
-	if (!strcmp("transfer.hiderefs", var) ||
-	    /* NEEDSWORK: use parse_config_key() once both are merged */
-	    (starts_with(var, section) && var[strlen(section)] == '.' &&
-	     !strcmp(var + strlen(section), ".hiderefs"))) {
-		char *ref;
-		int len;
-
-		if (!value)
-			return config_error_nonbool(var);
-		ref = xstrdup(value);
-		len = strlen(ref);
-		while (len && ref[len - 1] == '/')
-			ref[--len] = '\0';
-		if (!hide_refs) {
-			hide_refs = xcalloc(1, sizeof(*hide_refs));
-			hide_refs->strdup_strings = 1;
-		}
-		string_list_append(hide_refs, ref);
-	}
-	return 0;
-}
-
-int ref_is_hidden(const char *refname)
-{
-	struct string_list_item *item;
-
-	if (!hide_refs)
-		return 0;
-	for_each_string_list_item(item, hide_refs) {
-		int len;
-		if (!starts_with(refname, item->string))
-			continue;
-		len = strlen(item->string);
-		if (!refname[len] || refname[len] == '/')
-			return 1;
-	}
-	return 0;
-}
diff --git a/refs.c b/refs.c
index 072cd39..9e2059b 100644
--- a/refs.c
+++ b/refs.c
@@ -3,6 +3,7 @@
  */
 #include "cache.h"
 #include "refs.h"
+#include "string-list.h"
 
 int update_ref(const char *action, const char *refname,
 	       const unsigned char *sha1, const unsigned char *oldval,
@@ -251,3 +252,46 @@ int read_ref_at(const char *refname, unsigned long at_time, int cnt,
 
 	return 1;
 }
+
+static struct string_list *hide_refs;
+
+int parse_hide_refs_config(const char *var, const char *value, const char *section)
+{
+	if (!strcmp("transfer.hiderefs", var) ||
+	    /* NEEDSWORK: use parse_config_key() once both are merged */
+	    (starts_with(var, section) && var[strlen(section)] == '.' &&
+	     !strcmp(var + strlen(section), ".hiderefs"))) {
+		char *ref;
+		int len;
+
+		if (!value)
+			return config_error_nonbool(var);
+		ref = xstrdup(value);
+		len = strlen(ref);
+		while (len && ref[len - 1] == '/')
+			ref[--len] = '\0';
+		if (!hide_refs) {
+			hide_refs = xcalloc(1, sizeof(*hide_refs));
+			hide_refs->strdup_strings = 1;
+		}
+		string_list_append(hide_refs, ref);
+	}
+	return 0;
+}
+
+int ref_is_hidden(const char *refname)
+{
+	struct string_list_item *item;
+
+	if (!hide_refs)
+		return 0;
+	for_each_string_list_item(item, hide_refs) {
+		int len;
+		if (!starts_with(refname, item->string))
+			continue;
+		len = strlen(item->string);
+		if (!refname[len] || refname[len] == '/')
+			return 1;
+	}
+	return 0;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 09/23] refs.c: move dwim and friend functions to the common refs code

[Ronnie Sahlberg]

These functions do not contain any backend specific code so we can move
them to the common code and share across all backends.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 202 --------------------------------------------------------
 refs.c          | 202 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 202 insertions(+), 202 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 6181edf..56e146f 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1956,30 +1956,6 @@ const char *prettify_refname(const char *name)
 		0);
 }
 
-static const char *ref_rev_parse_rules[] = {
-	"%.*s",
-	"refs/%.*s",
-	"refs/tags/%.*s",
-	"refs/heads/%.*s",
-	"refs/remotes/%.*s",
-	"refs/remotes/%.*s/HEAD",
-	NULL
-};
-
-int refname_match(const char *abbrev_name, const char *full_name)
-{
-	const char **p;
-	const int abbrev_name_len = strlen(abbrev_name);
-
-	for (p = ref_rev_parse_rules; *p; p++) {
-		if (!strcmp(full_name, mkpath(*p, abbrev_name_len, abbrev_name))) {
-			return 1;
-		}
-	}
-
-	return 0;
-}
-
 static void unlock_ref(struct ref_lock *lock)
 {
 	/* Do not free lock->lk -- atexit() still looks at them */
@@ -2033,91 +2009,6 @@ static int remove_empty_directories(const char *file)
 	return result;
 }
 
-/*
- * *string and *len will only be substituted, and *string returned (for
- * later free()ing) if the string passed in is a magic short-hand form
- * to name a branch.
- */
-static char *substitute_branch_name(const char **string, int *len)
-{
-	struct strbuf buf = STRBUF_INIT;
-	int ret = interpret_branch_name(*string, *len, &buf);
-
-	if (ret == *len) {
-		size_t size;
-		*string = strbuf_detach(&buf, &size);
-		*len = size;
-		return (char *)*string;
-	}
-
-	return NULL;
-}
-
-int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
-{
-	char *last_branch = substitute_branch_name(&str, &len);
-	const char **p, *r;
-	int refs_found = 0;
-
-	*ref = NULL;
-	for (p = ref_rev_parse_rules; *p; p++) {
-		char fullref[PATH_MAX];
-		unsigned char sha1_from_ref[20];
-		unsigned char *this_result;
-		int flag;
-
-		this_result = refs_found ? sha1_from_ref : sha1;
-		mksnpath(fullref, sizeof(fullref), *p, len, str);
-		r = resolve_ref_unsafe(fullref, this_result,
-				       RESOLVE_REF_READING, &flag);
-		if (r) {
-			if (!refs_found++)
-				*ref = xstrdup(r);
-			if (!warn_ambiguous_refs)
-				break;
-		} else if ((flag & REF_ISSYMREF) && strcmp(fullref, "HEAD")) {
-			warning("ignoring dangling symref %s.", fullref);
-		} else if ((flag & REF_ISBROKEN) && strchr(fullref, '/')) {
-			warning("ignoring broken ref %s.", fullref);
-		}
-	}
-	free(last_branch);
-	return refs_found;
-}
-
-int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
-{
-	char *last_branch = substitute_branch_name(&str, &len);
-	const char **p;
-	int logs_found = 0;
-
-	*log = NULL;
-	for (p = ref_rev_parse_rules; *p; p++) {
-		unsigned char hash[20];
-		char path[PATH_MAX];
-		const char *ref, *it;
-
-		mksnpath(path, sizeof(path), *p, len, str);
-		ref = resolve_ref_unsafe(path, hash, RESOLVE_REF_READING, NULL);
-		if (!ref)
-			continue;
-		if (reflog_exists(path))
-			it = path;
-		else if (strcmp(ref, path) && reflog_exists(ref))
-			it = ref;
-		else
-			continue;
-		if (!logs_found++) {
-			*log = xstrdup(it);
-			hashcpy(sha1, hash);
-		}
-		if (!warn_ambiguous_refs)
-			break;
-	}
-	free(last_branch);
-	return logs_found;
-}
-
 /* This function should make sure errno is meaningful on error */
 static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 					    const unsigned char *old_sha1,
@@ -3703,96 +3594,3 @@ cleanup:
 		ret = -2;
 	return ret;
 }
-
-char *shorten_unambiguous_ref(const char *refname, int strict)
-{
-	int i;
-	static char **scanf_fmts;
-	static int nr_rules;
-	char *short_name;
-
-	if (!nr_rules) {
-		/*
-		 * Pre-generate scanf formats from ref_rev_parse_rules[].
-		 * Generate a format suitable for scanf from a
-		 * ref_rev_parse_rules rule by interpolating "%s" at the
-		 * location of the "%.*s".
-		 */
-		size_t total_len = 0;
-		size_t offset = 0;
-
-		/* the rule list is NULL terminated, count them first */
-		for (nr_rules = 0; ref_rev_parse_rules[nr_rules]; nr_rules++)
-			/* -2 for strlen("%.*s") - strlen("%s"); +1 for NUL */
-			total_len += strlen(ref_rev_parse_rules[nr_rules]) - 2 + 1;
-
-		scanf_fmts = xmalloc(nr_rules * sizeof(char *) + total_len);
-
-		offset = 0;
-		for (i = 0; i < nr_rules; i++) {
-			assert(offset < total_len);
-			scanf_fmts[i] = (char *)&scanf_fmts[nr_rules] + offset;
-			offset += snprintf(scanf_fmts[i], total_len - offset,
-					   ref_rev_parse_rules[i], 2, "%s") + 1;
-		}
-	}
-
-	/* bail out if there are no rules */
-	if (!nr_rules)
-		return xstrdup(refname);
-
-	/* buffer for scanf result, at most refname must fit */
-	short_name = xstrdup(refname);
-
-	/* skip first rule, it will always match */
-	for (i = nr_rules - 1; i > 0 ; --i) {
-		int j;
-		int rules_to_fail = i;
-		int short_name_len;
-
-		if (1 != sscanf(refname, scanf_fmts[i], short_name))
-			continue;
-
-		short_name_len = strlen(short_name);
-
-		/*
-		 * in strict mode, all (except the matched one) rules
-		 * must fail to resolve to a valid non-ambiguous ref
-		 */
-		if (strict)
-			rules_to_fail = nr_rules;
-
-		/*
-		 * check if the short name resolves to a valid ref,
-		 * but use only rules prior to the matched one
-		 */
-		for (j = 0; j < rules_to_fail; j++) {
-			const char *rule = ref_rev_parse_rules[j];
-			char refname[PATH_MAX];
-
-			/* skip matched rule */
-			if (i == j)
-				continue;
-
-			/*
-			 * the short name is ambiguous, if it resolves
-			 * (with this previous rule) to a valid ref
-			 * read_ref() returns 0 on success
-			 */
-			mksnpath(refname, sizeof(refname),
-				 rule, short_name_len, short_name);
-			if (ref_exists(refname))
-				break;
-		}
-
-		/*
-		 * short name is non-ambiguous if all previous rules
-		 * haven't resolved to a valid ref
-		 */
-		if (j == rules_to_fail)
-			return short_name;
-	}
-
-	free(short_name);
-	return xstrdup(refname);
-}
diff --git a/refs.c b/refs.c
index 9e2059b..adf0c29 100644
--- a/refs.c
+++ b/refs.c
@@ -295,3 +295,205 @@ int ref_is_hidden(const char *refname)
 	}
 	return 0;
 }
+
+static const char *ref_rev_parse_rules[] = {
+	"%.*s",
+	"refs/%.*s",
+	"refs/tags/%.*s",
+	"refs/heads/%.*s",
+	"refs/remotes/%.*s",
+	"refs/remotes/%.*s/HEAD",
+	NULL
+};
+
+int refname_match(const char *abbrev_name, const char *full_name)
+{
+	const char **p;
+	const int abbrev_name_len = strlen(abbrev_name);
+
+	for (p = ref_rev_parse_rules; *p; p++) {
+		if (!strcmp(full_name, mkpath(*p, abbrev_name_len, abbrev_name))) {
+			return 1;
+		}
+	}
+
+	return 0;
+}
+
+/*
+ * *string and *len will only be substituted, and *string returned (for
+ * later free()ing) if the string passed in is a magic short-hand form
+ * to name a branch.
+ */
+static char *substitute_branch_name(const char **string, int *len)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int ret = interpret_branch_name(*string, *len, &buf);
+
+	if (ret == *len) {
+		size_t size;
+		*string = strbuf_detach(&buf, &size);
+		*len = size;
+		return (char *)*string;
+	}
+
+	return NULL;
+}
+
+int dwim_ref(const char *str, int len, unsigned char *sha1, char **ref)
+{
+	char *last_branch = substitute_branch_name(&str, &len);
+	const char **p, *r;
+	int refs_found = 0;
+
+	*ref = NULL;
+	for (p = ref_rev_parse_rules; *p; p++) {
+		char fullref[PATH_MAX];
+		unsigned char sha1_from_ref[20];
+		unsigned char *this_result;
+		int flag;
+
+		this_result = refs_found ? sha1_from_ref : sha1;
+		mksnpath(fullref, sizeof(fullref), *p, len, str);
+		r = resolve_ref_unsafe(fullref, this_result,
+				       RESOLVE_REF_READING, &flag);
+		if (r) {
+			if (!refs_found++)
+				*ref = xstrdup(r);
+			if (!warn_ambiguous_refs)
+				break;
+		} else if ((flag & REF_ISSYMREF) && strcmp(fullref, "HEAD")) {
+			warning("ignoring dangling symref %s.", fullref);
+		} else if ((flag & REF_ISBROKEN) && strchr(fullref, '/')) {
+			warning("ignoring broken ref %s.", fullref);
+		}
+	}
+	free(last_branch);
+	return refs_found;
+}
+
+int dwim_log(const char *str, int len, unsigned char *sha1, char **log)
+{
+	char *last_branch = substitute_branch_name(&str, &len);
+	const char **p;
+	int logs_found = 0;
+
+	*log = NULL;
+	for (p = ref_rev_parse_rules; *p; p++) {
+		unsigned char hash[20];
+		char path[PATH_MAX];
+		const char *ref, *it;
+
+		mksnpath(path, sizeof(path), *p, len, str);
+		ref = resolve_ref_unsafe(path, hash, RESOLVE_REF_READING, NULL);
+		if (!ref)
+			continue;
+		if (reflog_exists(path))
+			it = path;
+		else if (strcmp(ref, path) && reflog_exists(ref))
+			it = ref;
+		else
+			continue;
+		if (!logs_found++) {
+			*log = xstrdup(it);
+			hashcpy(sha1, hash);
+		}
+		if (!warn_ambiguous_refs)
+			break;
+	}
+	free(last_branch);
+	return logs_found;
+}
+
+char *shorten_unambiguous_ref(const char *refname, int strict)
+{
+	int i;
+	static char **scanf_fmts;
+	static int nr_rules;
+	char *short_name;
+
+	if (!nr_rules) {
+		/*
+		 * Pre-generate scanf formats from ref_rev_parse_rules[].
+		 * Generate a format suitable for scanf from a
+		 * ref_rev_parse_rules rule by interpolating "%s" at the
+		 * location of the "%.*s".
+		 */
+		size_t total_len = 0;
+		size_t offset = 0;
+
+		/* the rule list is NULL terminated, count them first */
+		for (nr_rules = 0; ref_rev_parse_rules[nr_rules]; nr_rules++)
+			/* -2 for strlen("%.*s") - strlen("%s"); +1 for NUL */
+			total_len += strlen(ref_rev_parse_rules[nr_rules]) - 2 + 1;
+
+		scanf_fmts = xmalloc(nr_rules * sizeof(char *) + total_len);
+
+		offset = 0;
+		for (i = 0; i < nr_rules; i++) {
+			assert(offset < total_len);
+			scanf_fmts[i] = (char *)&scanf_fmts[nr_rules] + offset;
+			offset += snprintf(scanf_fmts[i], total_len - offset,
+					   ref_rev_parse_rules[i], 2, "%s") + 1;
+		}
+	}
+
+	/* bail out if there are no rules */
+	if (!nr_rules)
+		return xstrdup(refname);
+
+	/* buffer for scanf result, at most refname must fit */
+	short_name = xstrdup(refname);
+
+	/* skip first rule, it will always match */
+	for (i = nr_rules - 1; i > 0 ; --i) {
+		int j;
+		int rules_to_fail = i;
+		int short_name_len;
+
+		if (1 != sscanf(refname, scanf_fmts[i], short_name))
+			continue;
+
+		short_name_len = strlen(short_name);
+
+		/*
+		 * in strict mode, all (except the matched one) rules
+		 * must fail to resolve to a valid non-ambiguous ref
+		 */
+		if (strict)
+			rules_to_fail = nr_rules;
+
+		/*
+		 * check if the short name resolves to a valid ref,
+		 * but use only rules prior to the matched one
+		 */
+		for (j = 0; j < rules_to_fail; j++) {
+			const char *rule = ref_rev_parse_rules[j];
+			char refname[PATH_MAX];
+
+			/* skip matched rule */
+			if (i == j)
+				continue;
+
+			/*
+			 * the short name is ambiguous, if it resolves
+			 * (with this previous rule) to a valid ref
+			 * read_ref() returns 0 on success
+			 */
+			mksnpath(refname, sizeof(refname),
+				 rule, short_name_len, short_name);
+			if (ref_exists(refname))
+				break;
+		}
+
+		/*
+		 * short name is non-ambiguous if all previous rules
+		 * haven't resolved to a valid ref
+		 */
+		if (j == rules_to_fail)
+			return short_name;
+	}
+
+	free(short_name);
+	return xstrdup(refname);
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 10/23] refs.c: move warn_if_dangling_symref* to the common code

[Ronnie Sahlberg]

These functions do not use any backend specific code so we can move
them to the common code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 52 ----------------------------------------------------
 refs.c          | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 52 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 56e146f..40c329b 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1667,58 +1667,6 @@ int peel_ref(const char *refname, unsigned char *sha1)
 	return peel_object(base, sha1);
 }
 
-struct warn_if_dangling_data {
-	FILE *fp;
-	const char *refname;
-	const struct string_list *refnames;
-	const char *msg_fmt;
-};
-
-static int warn_if_dangling_symref(const char *refname, const unsigned char *sha1,
-				   int flags, void *cb_data)
-{
-	struct warn_if_dangling_data *d = cb_data;
-	const char *resolves_to;
-	unsigned char junk[20];
-
-	if (!(flags & REF_ISSYMREF))
-		return 0;
-
-	resolves_to = resolve_ref_unsafe(refname, junk, 0, NULL);
-	if (!resolves_to
-	    || (d->refname
-		? strcmp(resolves_to, d->refname)
-		: !string_list_has_string(d->refnames, resolves_to))) {
-		return 0;
-	}
-
-	fprintf(d->fp, d->msg_fmt, refname);
-	fputc('\n', d->fp);
-	return 0;
-}
-
-void warn_dangling_symref(FILE *fp, const char *msg_fmt, const char *refname)
-{
-	struct warn_if_dangling_data data;
-
-	data.fp = fp;
-	data.refname = refname;
-	data.refnames = NULL;
-	data.msg_fmt = msg_fmt;
-	for_each_rawref(warn_if_dangling_symref, &data);
-}
-
-void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct string_list *refnames)
-{
-	struct warn_if_dangling_data data;
-
-	data.fp = fp;
-	data.refname = NULL;
-	data.refnames = refnames;
-	data.msg_fmt = msg_fmt;
-	for_each_rawref(warn_if_dangling_symref, &data);
-}
-
 /*
  * Call fn for each reference in the specified ref_cache, omitting
  * references not in the containing_dir of base.  fn is called for all
diff --git a/refs.c b/refs.c
index adf0c29..3011452 100644
--- a/refs.c
+++ b/refs.c
@@ -497,3 +497,55 @@ char *shorten_unambiguous_ref(const char *refname, int strict)
 	free(short_name);
 	return xstrdup(refname);
 }
+
+struct warn_if_dangling_data {
+	FILE *fp;
+	const char *refname;
+	const struct string_list *refnames;
+	const char *msg_fmt;
+};
+
+static int warn_if_dangling_symref(const char *refname, const unsigned char *sha1,
+				   int flags, void *cb_data)
+{
+	struct warn_if_dangling_data *d = cb_data;
+	const char *resolves_to;
+	unsigned char junk[20];
+
+	if (!(flags & REF_ISSYMREF))
+		return 0;
+
+	resolves_to = resolve_ref_unsafe(refname, junk, 0, NULL);
+	if (!resolves_to
+	    || (d->refname
+		? strcmp(resolves_to, d->refname)
+		: !string_list_has_string(d->refnames, resolves_to))) {
+		return 0;
+	}
+
+	fprintf(d->fp, d->msg_fmt, refname);
+	fputc('\n', d->fp);
+	return 0;
+}
+
+void warn_dangling_symref(FILE *fp, const char *msg_fmt, const char *refname)
+{
+	struct warn_if_dangling_data data;
+
+	data.fp = fp;
+	data.refname = refname;
+	data.refnames = NULL;
+	data.msg_fmt = msg_fmt;
+	for_each_rawref(warn_if_dangling_symref, &data);
+}
+
+void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct string_list *refnames)
+{
+	struct warn_if_dangling_data data;
+
+	data.fp = fp;
+	data.refname = NULL;
+	data.refnames = refnames;
+	data.msg_fmt = msg_fmt;
+	for_each_rawref(warn_if_dangling_symref, &data);
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 11/23] refs.c: move read_ref, read_ref_full and ref_exists to the common code

[Ronnie Sahlberg]

These functions do not depend on the backend implementation so we
can move them to the common code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 18 ------------------
 refs.c          | 18 ++++++++++++++++++
 2 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 40c329b..a94378e 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1514,24 +1514,6 @@ struct ref_filter {
 	void *cb_data;
 };
 
-int read_ref_full(const char *refname, unsigned char *sha1, int flags, int *ref_flag)
-{
-	if (resolve_ref_unsafe(refname, sha1, flags, ref_flag))
-		return 0;
-	return -1;
-}
-
-int read_ref(const char *refname, unsigned char *sha1)
-{
-	return read_ref_full(refname, sha1, RESOLVE_REF_READING, NULL);
-}
-
-int ref_exists(const char *refname)
-{
-	unsigned char sha1[20];
-	return !!resolve_ref_unsafe(refname, sha1, RESOLVE_REF_READING, NULL);
-}
-
 static int filter_refs(const char *refname, const unsigned char *sha1, int flags,
 		       void *data)
 {
diff --git a/refs.c b/refs.c
index 3011452..a0e6d81 100644
--- a/refs.c
+++ b/refs.c
@@ -549,3 +549,21 @@ void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct string_li
 	data.msg_fmt = msg_fmt;
 	for_each_rawref(warn_if_dangling_symref, &data);
 }
+
+int read_ref_full(const char *refname, unsigned char *sha1, int flags, int *ref_flag)
+{
+	if (resolve_ref_unsafe(refname, sha1, flags, ref_flag))
+		return 0;
+	return -1;
+}
+
+int read_ref(const char *refname, unsigned char *sha1)
+{
+	return read_ref_full(refname, sha1, RESOLVE_REF_READING, NULL);
+}
+
+int ref_exists(const char *refname)
+{
+	unsigned char sha1[20];
+	return !!resolve_ref_unsafe(refname, sha1, RESOLVE_REF_READING, NULL);
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 12/23] refs.c: move resolve_refdup to common

[Ronnie Sahlberg]

This function can be shared across all refs backends so move it
to the common code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 6 ------
 refs.c          | 6 ++++++
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index a94378e..ed7bc61 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1501,12 +1501,6 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int fla
 	}
 }
 
-char *resolve_refdup(const char *ref, unsigned char *sha1, int flags, int *ref_flag)
-{
-	const char *ret = resolve_ref_unsafe(ref, sha1, flags, ref_flag);
-	return ret ? xstrdup(ret) : NULL;
-}
-
 /* The argument to filter_refs */
 struct ref_filter {
 	const char *pattern;
diff --git a/refs.c b/refs.c
index a0e6d81..b8582f8 100644
--- a/refs.c
+++ b/refs.c
@@ -567,3 +567,9 @@ int ref_exists(const char *refname)
 	unsigned char sha1[20];
 	return !!resolve_ref_unsafe(refname, sha1, RESOLVE_REF_READING, NULL);
 }
+
+char *resolve_refdup(const char *ref, unsigned char *sha1, int flags, int *ref_flag)
+{
+	const char *ret = resolve_ref_unsafe(ref, sha1, flags, ref_flag);
+	return ret ? xstrdup(ret) : NULL;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 13/23] refs.c: move check_refname_component to the common code

[Ronnie Sahlberg]

This function does not contain any backend specific code so we
can move it to the common code.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 110 --------------------------------------------------------
 refs.c          | 110 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 110 insertions(+), 110 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index ed7bc61..55bced9 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -6,25 +6,6 @@
 #include "string-list.h"
 
 /*
- * How to handle various characters in refnames:
- * 0: An acceptable character for refs
- * 1: End-of-component
- * 2: ., look for a preceding . to reject .. in refs
- * 3: {, look for a preceding @ to reject @{ in refs
- * 4: A bad character: ASCII control characters, "~", "^", ":" or SP
- */
-static unsigned char refname_disposition[256] = {
-	1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-	4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-	4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 2, 1,
-	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 4,
-	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 4, 0,
-	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4
-};
-
-/*
  * Used as a flag to transaction_delete_sha1 when a loose ref is being
  * pruned.
  */
@@ -35,97 +16,6 @@ static unsigned char refname_disposition[256] = {
  */
 #define UPDATE_REFLOG_NOLOCK 0x0200
 
-/*
- * Try to read one refname component from the front of refname.
- * Return the length of the component found, or -1 if the component is
- * not legal.  It is legal if it is something reasonable to have under
- * ".git/refs/"; We do not like it if:
- *
- * - any path component of it begins with ".", or
- * - it has double dots "..", or
- * - it has ASCII control character, "~", "^", ":" or SP, anywhere, or
- * - it ends with a "/".
- * - it ends with ".lock"
- * - it contains a "\" (backslash)
- */
-static int check_refname_component(const char *refname, int flags)
-{
-	const char *cp;
-	char last = '\0';
-
-	for (cp = refname; ; cp++) {
-		int ch = *cp & 255;
-		unsigned char disp = refname_disposition[ch];
-		switch (disp) {
-		case 1:
-			goto out;
-		case 2:
-			if (last == '.')
-				return -1; /* Refname contains "..". */
-			break;
-		case 3:
-			if (last == '@')
-				return -1; /* Refname contains "@{". */
-			break;
-		case 4:
-			return -1;
-		}
-		last = ch;
-	}
-out:
-	if (cp == refname)
-		return 0; /* Component has zero length. */
-	if (refname[0] == '.') {
-		if (!(flags & REFNAME_DOT_COMPONENT))
-			return -1; /* Component starts with '.'. */
-		/*
-		 * Even if leading dots are allowed, don't allow "."
-		 * as a component (".." is prevented by a rule above).
-		 */
-		if (refname[1] == '\0')
-			return -1; /* Component equals ".". */
-	}
-	if (cp - refname >= 5 && !memcmp(cp - 5, ".lock", 5))
-		return -1; /* Refname ends with ".lock". */
-	return cp - refname;
-}
-
-int check_refname_format(const char *refname, int flags)
-{
-	int component_len, component_count = 0;
-
-	if (!strcmp(refname, "@"))
-		/* Refname is a single character '@'. */
-		return -1;
-
-	while (1) {
-		/* We are at the start of a path component. */
-		component_len = check_refname_component(refname, flags);
-		if (component_len <= 0) {
-			if ((flags & REFNAME_REFSPEC_PATTERN) &&
-					refname[0] == '*' &&
-					(refname[1] == '\0' || refname[1] == '/')) {
-				/* Accept one wildcard as a full refname component. */
-				flags &= ~REFNAME_REFSPEC_PATTERN;
-				component_len = 1;
-			} else {
-				return -1;
-			}
-		}
-		component_count++;
-		if (refname[component_len] == '\0')
-			break;
-		/* Skip to next component. */
-		refname += component_len + 1;
-	}
-
-	if (refname[component_len - 1] == '.')
-		return -1; /* Refname ends with '.'. */
-	if (!(flags & REFNAME_ALLOW_ONELEVEL) && component_count < 2)
-		return -1; /* Refname has only one component. */
-	return 0;
-}
-
 struct ref_entry;
 
 /*
diff --git a/refs.c b/refs.c
index b8582f8..ea5f276 100644
--- a/refs.c
+++ b/refs.c
@@ -573,3 +573,113 @@ char *resolve_refdup(const char *ref, unsigned char *sha1, int flags, int *ref_f
 	const char *ret = resolve_ref_unsafe(ref, sha1, flags, ref_flag);
 	return ret ? xstrdup(ret) : NULL;
 }
+
+/*
+ * How to handle various characters in refnames:
+ * 0: An acceptable character for refs
+ * 1: End-of-component
+ * 2: ., look for a preceding . to reject .. in refs
+ * 3: {, look for a preceding @ to reject @{ in refs
+ * 4: A bad character: ASCII control characters, "~", "^", ":" or SP
+ */
+static unsigned char refname_disposition[256] = {
+	1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
+	4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
+	4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 2, 1,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 4,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 4, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4
+};
+
+/*
+ * Try to read one refname component from the front of refname.
+ * Return the length of the component found, or -1 if the component is
+ * not legal.  It is legal if it is something reasonable to have under
+ * ".git/refs/"; We do not like it if:
+ *
+ * - any path component of it begins with ".", or
+ * - it has double dots "..", or
+ * - it has ASCII control character, "~", "^", ":" or SP, anywhere, or
+ * - it ends with a "/".
+ * - it ends with ".lock"
+ * - it contains a "\" (backslash)
+ */
+static int check_refname_component(const char *refname, int flags)
+{
+	const char *cp;
+	char last = '\0';
+
+	for (cp = refname; ; cp++) {
+		int ch = *cp & 255;
+		unsigned char disp = refname_disposition[ch];
+		switch (disp) {
+		case 1:
+			goto out;
+		case 2:
+			if (last == '.')
+				return -1; /* Refname contains "..". */
+			break;
+		case 3:
+			if (last == '@')
+				return -1; /* Refname contains "@{". */
+			break;
+		case 4:
+			return -1;
+		}
+		last = ch;
+	}
+out:
+	if (cp == refname)
+		return 0; /* Component has zero length. */
+	if (refname[0] == '.') {
+		if (!(flags & REFNAME_DOT_COMPONENT))
+			return -1; /* Component starts with '.'. */
+		/*
+		 * Even if leading dots are allowed, don't allow "."
+		 * as a component (".." is prevented by a rule above).
+		 */
+		if (refname[1] == '\0')
+			return -1; /* Component equals ".". */
+	}
+	if (cp - refname >= 5 && !memcmp(cp - 5, ".lock", 5))
+		return -1; /* Refname ends with ".lock". */
+	return cp - refname;
+}
+
+int check_refname_format(const char *refname, int flags)
+{
+	int component_len, component_count = 0;
+
+	if (!strcmp(refname, "@"))
+		/* Refname is a single character '@'. */
+		return -1;
+
+	while (1) {
+		/* We are at the start of a path component. */
+		component_len = check_refname_component(refname, flags);
+		if (component_len <= 0) {
+			if ((flags & REFNAME_REFSPEC_PATTERN) &&
+					refname[0] == '*' &&
+					(refname[1] == '\0' || refname[1] == '/')) {
+				/* Accept one wildcard as a full refname component. */
+				flags &= ~REFNAME_REFSPEC_PATTERN;
+				component_len = 1;
+			} else {
+				return -1;
+			}
+		}
+		component_count++;
+		if (refname[component_len] == '\0')
+			break;
+		/* Skip to next component. */
+		refname += component_len + 1;
+	}
+
+	if (refname[component_len - 1] == '.')
+		return -1; /* Refname ends with '.'. */
+	if (!(flags & REFNAME_ALLOW_ONELEVEL) && component_count < 2)
+		return -1; /* Refname has only one component. */
+	return 0;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 14/23] refs.c: move is_branch to the common code

[Ronnie Sahlberg]

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 5 -----
 refs.c          | 5 +++++
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 55bced9..70c034c 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2483,11 +2483,6 @@ static int log_ref_write(const char *refname, const unsigned char *old_sha1,
 	return 0;
 }
 
-int is_branch(const char *refname)
-{
-	return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
-}
-
 static int write_sha1_update_reflog(struct ref_lock *lock,
 	const unsigned char *sha1, const char *logmsg)
 {
diff --git a/refs.c b/refs.c
index ea5f276..9bc0a31 100644
--- a/refs.c
+++ b/refs.c
@@ -683,3 +683,8 @@ int check_refname_format(const char *refname, int flags)
 		return -1; /* Refname has only one component. */
 	return 0;
 }
+
+int is_branch(const char *refname)
+{
+	return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 15/23] refs.c: move names_conflict to the common code

[Ronnie Sahlberg]

Move names_conflict to the common code and make it public.
We want to use the same name conflict checks across all backends
so we guarantee compatibility across backends.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 14 --------------
 refs.c          |  8 ++++++++
 refs.h          |  9 +++++++++
 3 files changed, 17 insertions(+), 14 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 70c034c..6542969 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -676,20 +676,6 @@ static void prime_ref_dir(struct ref_dir *dir)
 			prime_ref_dir(get_ref_dir(entry));
 	}
 }
-/*
- * Return true iff refname1 and refname2 conflict with each other.
- * Two reference names conflict if one of them exactly matches the
- * leading components of the other; e.g., "foo/bar" conflicts with
- * both "foo" and with "foo/bar/baz" but not with "foo/bar" or
- * "foo/barbados".
- */
-static int names_conflict(const char *refname1, const char *refname2)
-{
-	for (; *refname1 && *refname1 == *refname2; refname1++, refname2++)
-		;
-	return (*refname1 == '\0' && *refname2 == '/')
-		|| (*refname1 == '/' && *refname2 == '\0');
-}
 
 struct name_conflict_cb {
 	const char *refname;
diff --git a/refs.c b/refs.c
index 9bc0a31..177bed6 100644
--- a/refs.c
+++ b/refs.c
@@ -688,3 +688,11 @@ int is_branch(const char *refname)
 {
 	return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
 }
+
+int names_conflict(const char *refname1, const char *refname2)
+{
+	for (; *refname1 && *refname1 == *refname2; refname1++, refname2++)
+		;
+	return (*refname1 == '\0' && *refname2 == '/')
+		|| (*refname1 == '/' && *refname2 == '\0');
+}
diff --git a/refs.h b/refs.h
index d526da0..a14fc5d 100644
--- a/refs.h
+++ b/refs.h
@@ -128,6 +128,15 @@ int pack_refs(unsigned int flags, struct strbuf *err);
 
 extern int ref_exists(const char *);
 
+/*
+ * Return true iff refname1 and refname2 conflict with each other.
+ * Two reference names conflict if one of them exactly matches the
+ * leading components of the other; e.g., "foo/bar" conflicts with
+ * both "foo" and with "foo/bar/baz" but not with "foo/bar" or
+ * "foo/barbados".
+ */
+int names_conflict(const char *refname1, const char *refname2);
+
 extern int is_branch(const char *refname);
 
 /*
-- 
2.0.1.552.g1af257a

[PATCH v3 16/23] refs.c: move prettify_refname to the common code

[Ronnie Sahlberg]

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 9 ---------
 refs.c          | 9 +++++++++
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 6542969..fb9c614 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1747,15 +1747,6 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)
 			       DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
 }
 
-const char *prettify_refname(const char *name)
-{
-	return name + (
-		starts_with(name, "refs/heads/") ? 11 :
-		starts_with(name, "refs/tags/") ? 10 :
-		starts_with(name, "refs/remotes/") ? 13 :
-		0);
-}
-
 static void unlock_ref(struct ref_lock *lock)
 {
 	/* Do not free lock->lk -- atexit() still looks at them */
diff --git a/refs.c b/refs.c
index 177bed6..d163e61 100644
--- a/refs.c
+++ b/refs.c
@@ -696,3 +696,12 @@ int names_conflict(const char *refname1, const char *refname2)
 	return (*refname1 == '\0' && *refname2 == '/')
 		|| (*refname1 == '/' && *refname2 == '\0');
 }
+
+const char *prettify_refname(const char *name)
+{
+	return name + (
+		starts_with(name, "refs/heads/") ? 11 :
+		starts_with(name, "refs/tags/") ? 10 :
+		starts_with(name, "refs/remotes/") ? 13 :
+		0);
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 17/23] refs.c: move ref iterators to the common code

[Ronnie Sahlberg]

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 81 ---------------------------------------------------------
 refs.c          | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 81 insertions(+), 81 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index fb9c614..9aa88ef 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1377,22 +1377,6 @@ const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int fla
 	}
 }
 
-/* The argument to filter_refs */
-struct ref_filter {
-	const char *pattern;
-	each_ref_fn *fn;
-	void *cb_data;
-};
-
-static int filter_refs(const char *refname, const unsigned char *sha1, int flags,
-		       void *data)
-{
-	struct ref_filter *filter = (struct ref_filter *)data;
-	if (wildmatch(filter->pattern, refname, 0, NULL))
-		return 0;
-	return filter->fn(refname, sha1, flags, filter->cb_data);
-}
-
 enum peel_status {
 	/* object was peeled successfully: */
 	PEEL_PEELED = 0,
@@ -1646,36 +1630,6 @@ int for_each_ref_in_submodule(const char *submodule, const char *prefix,
 	return do_for_each_ref(get_ref_cache(submodule), prefix, fn, strlen(prefix), 0, cb_data);
 }
 
-int for_each_tag_ref(each_ref_fn fn, void *cb_data)
-{
-	return for_each_ref_in("refs/tags/", fn, cb_data);
-}
-
-int for_each_tag_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
-{
-	return for_each_ref_in_submodule(submodule, "refs/tags/", fn, cb_data);
-}
-
-int for_each_branch_ref(each_ref_fn fn, void *cb_data)
-{
-	return for_each_ref_in("refs/heads/", fn, cb_data);
-}
-
-int for_each_branch_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
-{
-	return for_each_ref_in_submodule(submodule, "refs/heads/", fn, cb_data);
-}
-
-int for_each_remote_ref(each_ref_fn fn, void *cb_data)
-{
-	return for_each_ref_in("refs/remotes/", fn, cb_data);
-}
-
-int for_each_remote_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
-{
-	return for_each_ref_in_submodule(submodule, "refs/remotes/", fn, cb_data);
-}
-
 int for_each_replace_ref(each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(&ref_cache, "refs/replace/", fn, 13, 0, cb_data);
@@ -1706,41 +1660,6 @@ int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
 	return ret;
 }
 
-int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
-	const char *prefix, void *cb_data)
-{
-	struct strbuf real_pattern = STRBUF_INIT;
-	struct ref_filter filter;
-	int ret;
-
-	if (!prefix && !starts_with(pattern, "refs/"))
-		strbuf_addstr(&real_pattern, "refs/");
-	else if (prefix)
-		strbuf_addstr(&real_pattern, prefix);
-	strbuf_addstr(&real_pattern, pattern);
-
-	if (!has_glob_specials(pattern)) {
-		/* Append implied '/' '*' if not present. */
-		if (real_pattern.buf[real_pattern.len - 1] != '/')
-			strbuf_addch(&real_pattern, '/');
-		/* No need to check for '*', there is none. */
-		strbuf_addch(&real_pattern, '*');
-	}
-
-	filter.pattern = real_pattern.buf;
-	filter.fn = fn;
-	filter.cb_data = cb_data;
-	ret = for_each_ref(filter_refs, &filter);
-
-	strbuf_release(&real_pattern);
-	return ret;
-}
-
-int for_each_glob_ref(each_ref_fn fn, const char *pattern, void *cb_data)
-{
-	return for_each_glob_ref_in(fn, pattern, NULL, cb_data);
-}
-
 int for_each_rawref(each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(&ref_cache, "", fn, 0,
diff --git a/refs.c b/refs.c
index d163e61..964a513 100644
--- a/refs.c
+++ b/refs.c
@@ -705,3 +705,84 @@ const char *prettify_refname(const char *name)
 		starts_with(name, "refs/remotes/") ? 13 :
 		0);
 }
+
+/* The argument to filter_refs */
+struct ref_filter {
+	const char *pattern;
+	each_ref_fn *fn;
+	void *cb_data;
+};
+
+static int filter_refs(const char *refname, const unsigned char *sha1, int flags,
+		       void *data)
+{
+	struct ref_filter *filter = (struct ref_filter *)data;
+	if (wildmatch(filter->pattern, refname, 0, NULL))
+		return 0;
+	return filter->fn(refname, sha1, flags, filter->cb_data);
+}
+
+int for_each_tag_ref(each_ref_fn fn, void *cb_data)
+{
+	return for_each_ref_in("refs/tags/", fn, cb_data);
+}
+
+int for_each_tag_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+	return for_each_ref_in_submodule(submodule, "refs/tags/", fn, cb_data);
+}
+
+int for_each_branch_ref(each_ref_fn fn, void *cb_data)
+{
+	return for_each_ref_in("refs/heads/", fn, cb_data);
+}
+
+int for_each_branch_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+	return for_each_ref_in_submodule(submodule, "refs/heads/", fn, cb_data);
+}
+
+int for_each_remote_ref(each_ref_fn fn, void *cb_data)
+{
+	return for_each_ref_in("refs/remotes/", fn, cb_data);
+}
+
+int for_each_remote_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+	return for_each_ref_in_submodule(submodule, "refs/remotes/", fn, cb_data);
+}
+
+int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
+	const char *prefix, void *cb_data)
+{
+	struct strbuf real_pattern = STRBUF_INIT;
+	struct ref_filter filter;
+	int ret;
+
+	if (!prefix && !starts_with(pattern, "refs/"))
+		strbuf_addstr(&real_pattern, "refs/");
+	else if (prefix)
+		strbuf_addstr(&real_pattern, prefix);
+	strbuf_addstr(&real_pattern, pattern);
+
+	if (!has_glob_specials(pattern)) {
+		/* Append implied '/' '*' if not present. */
+		if (real_pattern.buf[real_pattern.len - 1] != '/')
+			strbuf_addch(&real_pattern, '/');
+		/* No need to check for '*', there is none. */
+		strbuf_addch(&real_pattern, '*');
+	}
+
+	filter.pattern = real_pattern.buf;
+	filter.fn = fn;
+	filter.cb_data = cb_data;
+	ret = for_each_ref(filter_refs, &filter);
+
+	strbuf_release(&real_pattern);
+	return ret;
+}
+
+int for_each_glob_ref(each_ref_fn fn, const char *pattern, void *cb_data)
+{
+	return for_each_glob_ref_in(fn, pattern, NULL, cb_data);
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 18/23] refs.c: move head_ref_namespaced to the common code

[Ronnie Sahlberg]

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 15 ---------------
 refs.c          | 15 +++++++++++++++
 2 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 9aa88ef..e58a7e1 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1635,21 +1635,6 @@ int for_each_replace_ref(each_ref_fn fn, void *cb_data)
 	return do_for_each_ref(&ref_cache, "refs/replace/", fn, 13, 0, cb_data);
 }
 
-int head_ref_namespaced(each_ref_fn fn, void *cb_data)
-{
-	struct strbuf buf = STRBUF_INIT;
-	int ret = 0;
-	unsigned char sha1[20];
-	int flag;
-
-	strbuf_addf(&buf, "%sHEAD", get_git_namespace());
-	if (!read_ref_full(buf.buf, sha1, RESOLVE_REF_READING, &flag))
-		ret = fn(buf.buf, sha1, flag, cb_data);
-	strbuf_release(&buf);
-
-	return ret;
-}
-
 int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
 {
 	struct strbuf buf = STRBUF_INIT;
diff --git a/refs.c b/refs.c
index 964a513..6b434ad 100644
--- a/refs.c
+++ b/refs.c
@@ -786,3 +786,18 @@ int for_each_glob_ref(each_ref_fn fn, const char *pattern, void *cb_data)
 {
 	return for_each_glob_ref_in(fn, pattern, NULL, cb_data);
 }
+
+int head_ref_namespaced(each_ref_fn fn, void *cb_data)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int ret = 0;
+	unsigned char sha1[20];
+	int flag;
+
+	strbuf_addf(&buf, "%sHEAD", get_git_namespace());
+	if (!read_ref_full(buf.buf, sha1, RESOLVE_REF_READING, &flag))
+		ret = fn(buf.buf, sha1, flag, cb_data);
+	strbuf_release(&buf);
+
+	return ret;
+}
-- 
2.0.1.552.g1af257a

[PATCH v3 19/23] refs-be-files.c: add a backend method structure with transaction functions

[Ronnie Sahlberg]

Add a ref structure for backend methods. Start by adding method pointers
for the transaction functions.

Rename the existing transaction functions to files_* and make them static.
Add new transaction functions that just pass through to the appropriate
methods for the backend.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 68 ++++++++++++++++++++++++++++++++++-----------------------
 refs.c          | 55 ++++++++++++++++++++++++++++++++++++++++++++++
 refs.h          | 35 +++++++++++++++++++++++++++++
 3 files changed, 131 insertions(+), 27 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index e58a7e1..27eafd0 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2777,12 +2777,12 @@ struct ref_transaction {
 	enum ref_transaction_state state;
 };
 
-struct ref_transaction *transaction_begin(struct strbuf *err)
+static struct ref_transaction *files_transaction_begin(struct strbuf *err)
 {
 	return xcalloc(1, sizeof(struct ref_transaction));
 }
 
-void transaction_free(struct ref_transaction *transaction)
+static void files_transaction_free(struct ref_transaction *transaction)
 {
 	int i;
 
@@ -2812,13 +2812,13 @@ static struct ref_update *add_update(struct ref_transaction *transaction,
 	return update;
 }
 
-int transaction_update_reflog(struct ref_transaction *transaction,
-			      const char *refname,
-			      const unsigned char *new_sha1,
-			      const unsigned char *old_sha1,
-			      struct reflog_committer_info *ci,
-			      const char *msg, int flags,
-			      struct strbuf *err)
+static int files_transaction_update_reflog(struct ref_transaction *transaction,
+					   const char *refname,
+					   const unsigned char *new_sha1,
+					   const unsigned char *old_sha1,
+					   struct reflog_committer_info *ci,
+					   const char *msg, int flags,
+					   struct strbuf *err)
 {
 	struct ref_update *update;
 	int i;
@@ -2865,12 +2865,13 @@ int transaction_update_reflog(struct ref_transaction *transaction,
 	return 0;
 }
 
-int transaction_update_sha1(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *new_sha1,
-			    const unsigned char *old_sha1,
-			    int flags, int have_old, const char *msg,
-			    struct strbuf *err)
+static int files_transaction_update_sha1(struct ref_transaction *transaction,
+					 const char *refname,
+					 const unsigned char *new_sha1,
+					 const unsigned char *old_sha1,
+					 int flags, int have_old,
+					 const char *msg,
+					 struct strbuf *err)
 {
 	struct ref_update *update;
 
@@ -2897,11 +2898,11 @@ int transaction_update_sha1(struct ref_transaction *transaction,
 	return 0;
 }
 
-int transaction_create_sha1(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *new_sha1,
-			    int flags, const char *msg,
-			    struct strbuf *err)
+static int files_transaction_create_sha1(struct ref_transaction *transaction,
+					 const char *refname,
+					 const unsigned char *new_sha1,
+					 int flags, const char *msg,
+					 struct strbuf *err)
 {
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: create called for transaction that is not open");
@@ -2913,11 +2914,12 @@ int transaction_create_sha1(struct ref_transaction *transaction,
 				       null_sha1, flags, 1, msg, err);
 }
 
-int transaction_delete_sha1(struct ref_transaction *transaction,
-			    const char *refname,
-			    const unsigned char *old_sha1,
-			    int flags, int have_old, const char *msg,
-			    struct strbuf *err)
+static int files_transaction_delete_sha1(struct ref_transaction *transaction,
+					 const char *refname,
+					 const unsigned char *old_sha1,
+					 int flags, int have_old,
+					 const char *msg,
+					 struct strbuf *err)
 {
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		die("BUG: delete called for transaction that is not open");
@@ -2959,8 +2961,8 @@ static int ref_update_reject_duplicates(struct ref_update **updates, int n,
 	return 0;
 }
 
-int transaction_commit(struct ref_transaction *transaction,
-		       struct strbuf *err)
+static int files_transaction_commit(struct ref_transaction *transaction,
+				    struct strbuf *err)
 {
 	int ret = 0, delnum = 0, i, df_conflict = 0, need_repack = 0;
 	int num_updates = 0;
@@ -3284,3 +3286,15 @@ cleanup:
 		ret = -2;
 	return ret;
 }
+
+struct ref_be refs_files = {
+	files_transaction_begin,
+	files_transaction_update_sha1,
+	files_transaction_create_sha1,
+	files_transaction_delete_sha1,
+	files_transaction_update_reflog,
+	files_transaction_commit,
+	files_transaction_free,
+};
+
+struct ref_be *refs = &refs_files;
diff --git a/refs.c b/refs.c
index 6b434ad..b8c942f 100644
--- a/refs.c
+++ b/refs.c
@@ -801,3 +801,58 @@ int head_ref_namespaced(each_ref_fn fn, void *cb_data)
 
 	return ret;
 }
+
+
+/* backend functions */
+struct ref_transaction *transaction_begin(struct strbuf *err)
+{
+	return refs->transaction_begin(err);
+}
+
+int transaction_update_sha1(struct ref_transaction *transaction,
+			    const char *refname, const unsigned char *new_sha1,
+			    const unsigned char *old_sha1, int flags,
+			    int have_old, const char *msg, struct strbuf *err)
+{
+	return refs->transaction_update_sha1(transaction, refname, new_sha1,
+					     old_sha1, flags, have_old, msg,
+					     err);
+}
+
+int transaction_create_sha1(struct ref_transaction *transaction,
+			    const char *refname, const unsigned char *new_sha1,
+			    int flags, const char *msg, struct strbuf *err)
+{
+	return refs->transaction_create_sha1(transaction, refname, new_sha1,
+					     flags, msg, err);
+}
+int transaction_delete_sha1(struct ref_transaction *transaction,
+			    const char *refname, const unsigned char *old_sha1,
+			    int flags, int have_old, const char *msg,
+			    struct strbuf *err)
+{
+	return refs->transaction_delete_sha1(transaction, refname, old_sha1,
+					     flags, have_old, msg, err);
+}
+
+int transaction_update_reflog(struct ref_transaction *transaction,
+			      const char *refname,
+			      const unsigned char *new_sha1,
+			      const unsigned char *old_sha1,
+			      struct reflog_committer_info *ci,
+			      const char *msg, int flags,
+			      struct strbuf *err)
+{
+	return refs->transaction_update_reflog(transaction, refname, new_sha1,
+					       old_sha1, ci, msg, flags, err);
+}
+
+int transaction_commit(struct ref_transaction *transaction, struct strbuf *err)
+{
+	return refs->transaction_commit(transaction, err);
+}
+
+void transaction_free(struct ref_transaction *transaction)
+{
+	return refs->transaction_free(transaction);
+}
diff --git a/refs.h b/refs.h
index a14fc5d..4b669f5 100644
--- a/refs.h
+++ b/refs.h
@@ -350,4 +350,39 @@ int update_ref(const char *action, const char *refname,
 extern int parse_hide_refs_config(const char *var, const char *value, const char *);
 extern int ref_is_hidden(const char *);
 
+
+/* refs backends */
+typedef struct ref_transaction *(*transaction_begin_fn)(struct strbuf *err);
+typedef int (*transaction_update_sha1_fn)(struct ref_transaction *transaction,
+		const char *refname, const unsigned char *new_sha1,
+		const unsigned char *old_sha1, int flags, int have_old,
+		const char *msg, struct strbuf *err);
+typedef int (*transaction_create_sha1_fn)(struct ref_transaction *transaction,
+		const char *refname, const unsigned char *new_sha1,
+		int flags, const char *msg, struct strbuf *err);
+typedef int (*transaction_delete_sha1_fn)(struct ref_transaction *transaction,
+		const char *refname, const unsigned char *old_sha1,
+		int flags, int have_old, const char *msg, struct strbuf *err);
+typedef int (*transaction_update_reflog_fn)(
+		struct ref_transaction *transaction,
+		const char *refname, const unsigned char *new_sha1,
+		const unsigned char *old_sha1,
+		struct reflog_committer_info *ci,
+		const char *msg, int flags, struct strbuf *err);
+typedef int (*transaction_commit_fn)(struct ref_transaction *transaction,
+				       struct strbuf *err);
+typedef void (*transaction_free_fn)(struct ref_transaction *transaction);
+
+struct ref_be {
+	transaction_begin_fn transaction_begin;
+	transaction_update_sha1_fn transaction_update_sha1;
+	transaction_create_sha1_fn transaction_create_sha1;
+	transaction_delete_sha1_fn transaction_delete_sha1;
+	transaction_update_reflog_fn transaction_update_reflog;
+	transaction_commit_fn transaction_commit;
+	transaction_free_fn transaction_free;
+};
+
+extern struct ref_be *refs;
+
 #endif /* REFS_H */
-- 
2.0.1.552.g1af257a

[PATCH v3 20/23] refs-be-files.c: add reflog backend methods

[Ronnie Sahlberg]

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 21 +++++++++++++++------
 refs.c          | 32 ++++++++++++++++++++++++++++++++
 refs.h          | 16 ++++++++++++++++
 3 files changed, 63 insertions(+), 6 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 27eafd0..464d488 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -2251,7 +2251,7 @@ static int copy_msg(char *buf, const char *msg)
 }
 
 /* This function must set a meaningful errno on failure */
-int create_reflog(const char *refname)
+static int files_create_reflog(const char *refname)
 {
 	int logfd, oflags = O_APPEND | O_WRONLY;
 	char logfile[PATH_MAX];
@@ -2516,7 +2516,7 @@ int create_symref(const char *ref_target, const char *refs_heads_master,
 	return 0;
 }
 
-int reflog_exists(const char *refname)
+static int files_reflog_exists(const char *refname)
 {
 	struct stat st;
 
@@ -2524,7 +2524,7 @@ int reflog_exists(const char *refname)
 		S_ISREG(st.st_mode);
 }
 
-int delete_reflog(const char *refname)
+static int files_delete_reflog(const char *refname)
 {
 	return remove_path(git_path("logs/%s", refname));
 }
@@ -2568,7 +2568,9 @@ static char *find_beginning_of_line(char *bob, char *scan)
 	return scan;
 }
 
-int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn, void *cb_data)
+static int files_for_each_reflog_ent_reverse(const char *refname,
+					     each_reflog_ent_fn fn,
+					     void *cb_data)
 {
 	struct strbuf sb = STRBUF_INIT;
 	FILE *logfp;
@@ -2645,7 +2647,8 @@ int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn, void
 	return ret;
 }
 
-int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn, void *cb_data)
+static int files_for_each_reflog_ent(const char *refname,
+				     each_reflog_ent_fn fn, void *cb_data)
 {
 	FILE *logfp;
 	struct strbuf sb = STRBUF_INIT;
@@ -2706,7 +2709,7 @@ static int do_for_each_reflog(struct strbuf *name, each_ref_fn fn, void *cb_data
 	return retval;
 }
 
-int for_each_reflog(each_ref_fn fn, void *cb_data)
+static int files_for_each_reflog(each_ref_fn fn, void *cb_data)
 {
 	int retval;
 	struct strbuf name;
@@ -3295,6 +3298,12 @@ struct ref_be refs_files = {
 	files_transaction_update_reflog,
 	files_transaction_commit,
 	files_transaction_free,
+	files_for_each_reflog_ent,
+	files_for_each_reflog_ent_reverse,
+	files_for_each_reflog,
+	files_reflog_exists,
+	files_create_reflog,
+	files_delete_reflog,
 };
 
 struct ref_be *refs = &refs_files;
diff --git a/refs.c b/refs.c
index b8c942f..2db1a74 100644
--- a/refs.c
+++ b/refs.c
@@ -856,3 +856,35 @@ void transaction_free(struct ref_transaction *transaction)
 {
 	return refs->transaction_free(transaction);
 }
+
+int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn,
+				void *cb_data)
+{
+	return refs->for_each_reflog_ent_reverse(refname, fn, cb_data);
+}
+
+int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn,
+			void *cb_data)
+{
+	return refs->for_each_reflog_ent(refname, fn, cb_data);
+}
+
+int for_each_reflog(each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_reflog(fn, cb_data);
+}
+
+int reflog_exists(const char *refname)
+{
+	return refs->reflog_exists(refname);
+}
+
+int create_reflog(const char *refname)
+{
+	return refs->create_reflog(refname);
+}
+
+int delete_reflog(const char *refname)
+{
+	return refs->delete_reflog(refname);
+}
diff --git a/refs.h b/refs.h
index 4b669f5..0a68986 100644
--- a/refs.h
+++ b/refs.h
@@ -372,6 +372,16 @@ typedef int (*transaction_update_reflog_fn)(
 typedef int (*transaction_commit_fn)(struct ref_transaction *transaction,
 				       struct strbuf *err);
 typedef void (*transaction_free_fn)(struct ref_transaction *transaction);
+typedef int (*for_each_reflog_ent_fn)(const char *refname,
+				      each_reflog_ent_fn fn,
+				      void *cb_data);
+typedef int (*for_each_reflog_ent_reverse_fn)(const char *refname,
+					      each_reflog_ent_fn fn,
+					      void *cb_data);
+typedef int (*for_each_reflog_fn)(each_ref_fn fn, void *cb_data);
+typedef int (*reflog_exists_fn)(const char *refname);
+typedef int (*create_reflog_fn)(const char *refname);
+typedef int (*delete_reflog_fn)(const char *refname);
 
 struct ref_be {
 	transaction_begin_fn transaction_begin;
@@ -381,6 +391,12 @@ struct ref_be {
 	transaction_update_reflog_fn transaction_update_reflog;
 	transaction_commit_fn transaction_commit;
 	transaction_free_fn transaction_free;
+	for_each_reflog_ent_fn for_each_reflog_ent;
+	for_each_reflog_ent_reverse_fn for_each_reflog_ent_reverse;
+	for_each_reflog_fn for_each_reflog;
+	reflog_exists_fn reflog_exists;
+	create_reflog_fn create_reflog;
+	delete_reflog_fn delete_reflog;
 };
 
 extern struct ref_be *refs;
-- 
2.0.1.552.g1af257a

[PATCH v3 21/23] refs-be-files.c: add methods for misc ref operations

[Ronnie Sahlberg]

Add ref backend methods for:
resolve_ref_unsafe, is_refname_available, pack_refs, peel_ref,
create_symref, resolve_gitlink_ref.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 25 ++++++++++++++++++-------
 refs.c          | 33 +++++++++++++++++++++++++++++++++
 refs.h          | 18 ++++++++++++++++++
 3 files changed, 69 insertions(+), 7 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 464d488..b09f0fc 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1114,7 +1114,8 @@ static struct ref_dir *get_loose_refs(struct ref_cache *refs)
 	return get_ref_dir(refs->loose);
 }
 
-int is_refname_available(const char *refname, const char **skip, int skipnum)
+static int files_is_refname_available(const char *refname, const char **skip,
+				      int skipnum)
 {
 	if (!is_refname_available_dir(refname, get_packed_refs(&ref_cache),
 				      skip, skipnum))
@@ -1188,7 +1189,8 @@ static int resolve_gitlink_ref_recursive(struct ref_cache *refs,
 	return resolve_gitlink_ref_recursive(refs, p, sha1, recursion+1);
 }
 
-int resolve_gitlink_ref(const char *path, const char *refname, unsigned char *sha1)
+static int files_resolve_gitlink_ref(const char *path, const char *refname,
+				     unsigned char *sha1)
 {
 	int len = strlen(path), retval;
 	char *submodule;
@@ -1247,7 +1249,9 @@ static const char *handle_missing_loose_ref(const char *refname,
 }
 
 /* This function needs to return a meaningful errno on failure */
-const char *resolve_ref_unsafe(const char *refname, unsigned char *sha1, int flags, int *ref_flag)
+static const char *files_resolve_ref_unsafe(const char *refname,
+					    unsigned char *sha1, int flags,
+					    int *ref_flag)
 {
 	int depth = MAXDEPTH;
 	ssize_t len;
@@ -1466,7 +1470,7 @@ static enum peel_status peel_entry(struct ref_entry *entry, int repeel)
 	return status;
 }
 
-int peel_ref(const char *refname, unsigned char *sha1)
+static int files_peel_ref(const char *refname, unsigned char *sha1)
 {
 	int flag;
 	unsigned char base[20];
@@ -2080,7 +2084,7 @@ static void prune_refs(struct ref_to_prune *r)
 	}
 }
 
-int pack_refs(unsigned int flags, struct strbuf *err)
+static int files_pack_refs(unsigned int flags, struct strbuf *err)
 {
 	struct pack_refs_cb_data cbdata;
 
@@ -2453,8 +2457,9 @@ static int write_ref_sha1(struct ref_lock *lock,
 	return 0;
 }
 
-int create_symref(const char *ref_target, const char *refs_heads_master,
-		  const char *logmsg)
+static int files_create_symref(const char *ref_target,
+			       const char *refs_heads_master,
+			       const char *logmsg)
 {
 	const char *lockpath;
 	char ref[1000];
@@ -3304,6 +3309,12 @@ struct ref_be refs_files = {
 	files_reflog_exists,
 	files_create_reflog,
 	files_delete_reflog,
+	files_resolve_ref_unsafe,
+	files_is_refname_available,
+	files_pack_refs,
+	files_peel_ref,
+	files_create_symref,
+	files_resolve_gitlink_ref,
 };
 
 struct ref_be *refs = &refs_files;
diff --git a/refs.c b/refs.c
index 2db1a74..60b6241 100644
--- a/refs.c
+++ b/refs.c
@@ -888,3 +888,36 @@ int delete_reflog(const char *refname)
 {
 	return refs->delete_reflog(refname);
 }
+
+const char *resolve_ref_unsafe(const char *ref, unsigned char *sha1,
+			       int reading, int *flag)
+{
+	return refs->resolve_ref_unsafe(ref, sha1, reading, flag);
+}
+
+int is_refname_available(const char *refname, const char **skip, int skipnum)
+{
+	return refs->is_refname_available(refname, skip, skipnum);
+}
+
+int pack_refs(unsigned int flags, struct strbuf *err)
+{
+	return refs->pack_refs(flags, err);
+}
+
+int peel_ref(const char *refname, unsigned char *sha1)
+{
+	return refs->peel_ref(refname, sha1);
+}
+
+int create_symref(const char *ref_target, const char *refs_heads_master,
+		  const char *logmsg)
+{
+	return refs->create_symref(ref_target, refs_heads_master, logmsg);
+}
+
+int resolve_gitlink_ref(const char *path, const char *refname,
+			unsigned char *sha1)
+{
+	return refs->resolve_gitlink_ref(path, refname, sha1);
+}
diff --git a/refs.h b/refs.h
index 0a68986..5257437 100644
--- a/refs.h
+++ b/refs.h
@@ -382,6 +382,18 @@ typedef int (*for_each_reflog_fn)(each_ref_fn fn, void *cb_data);
 typedef int (*reflog_exists_fn)(const char *refname);
 typedef int (*create_reflog_fn)(const char *refname);
 typedef int (*delete_reflog_fn)(const char *refname);
+typedef const char *(*resolve_ref_unsafe_fn)(const char *ref,
+		unsigned char *sha1, int reading, int *flag);
+
+typedef int (*is_refname_available_fn)(const char *refname, const char **skip,
+				       int skipnum);
+typedef int (*pack_refs_fn)(unsigned int flags, struct strbuf *err);
+typedef int (*peel_ref_fn)(const char *refname, unsigned char *sha1);
+typedef int (*create_symref_fn)(const char *ref_target,
+				const char *refs_heads_master,
+				const char *logmsg);
+typedef int (*resolve_gitlink_ref_fn)(const char *path, const char *refname,
+				      unsigned char *sha1);
 
 struct ref_be {
 	transaction_begin_fn transaction_begin;
@@ -397,6 +409,12 @@ struct ref_be {
 	reflog_exists_fn reflog_exists;
 	create_reflog_fn create_reflog;
 	delete_reflog_fn delete_reflog;
+	resolve_ref_unsafe_fn resolve_ref_unsafe;
+	is_refname_available_fn is_refname_available;
+	pack_refs_fn pack_refs;
+	peel_ref_fn peel_ref;
+	create_symref_fn create_symref;
+	resolve_gitlink_ref_fn resolve_gitlink_ref;
 };
 
 extern struct ref_be *refs;
-- 
2.0.1.552.g1af257a

[PATCH v3 22/23] refs-be-files.c: add methods for head_ref*

[Ronnie Sahlberg]

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c |  7 +++++--
 refs.c          | 10 ++++++++++
 refs.h          |  5 +++++
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index b09f0fc..910663b 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1603,12 +1603,13 @@ static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
 	return 0;
 }
 
-int head_ref(each_ref_fn fn, void *cb_data)
+static int files_head_ref(each_ref_fn fn, void *cb_data)
 {
 	return do_head_ref(NULL, fn, cb_data);
 }
 
-int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+static int files_head_ref_submodule(const char *submodule, each_ref_fn fn,
+				    void *cb_data)
 {
 	return do_head_ref(submodule, fn, cb_data);
 }
@@ -3315,6 +3316,8 @@ struct ref_be refs_files = {
 	files_peel_ref,
 	files_create_symref,
 	files_resolve_gitlink_ref,
+	files_head_ref,
+	files_head_ref_submodule,
 };
 
 struct ref_be *refs = &refs_files;
diff --git a/refs.c b/refs.c
index 60b6241..841d905 100644
--- a/refs.c
+++ b/refs.c
@@ -921,3 +921,13 @@ int resolve_gitlink_ref(const char *path, const char *refname,
 {
 	return refs->resolve_gitlink_ref(path, refname, sha1);
 }
+
+int head_ref(each_ref_fn fn, void *cb_data)
+{
+	return refs->head_ref(fn, cb_data);
+}
+
+int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+	return refs->head_ref_submodule(submodule, fn, cb_data);
+}
diff --git a/refs.h b/refs.h
index 5257437..92f8f44 100644
--- a/refs.h
+++ b/refs.h
@@ -394,6 +394,9 @@ typedef int (*create_symref_fn)(const char *ref_target,
 				const char *logmsg);
 typedef int (*resolve_gitlink_ref_fn)(const char *path, const char *refname,
 				      unsigned char *sha1);
+typedef int (*head_ref_fn)(each_ref_fn fn, void *cb_data);
+typedef int (*head_ref_submodule_fn)(const char *submodule, each_ref_fn fn,
+				     void *cb_data);
 
 struct ref_be {
 	transaction_begin_fn transaction_begin;
@@ -415,6 +418,8 @@ struct ref_be {
 	peel_ref_fn peel_ref;
 	create_symref_fn create_symref;
 	resolve_gitlink_ref_fn resolve_gitlink_ref;
+	head_ref_fn head_ref;
+	head_ref_submodule_fn head_ref_submodule;
 };
 
 extern struct ref_be *refs;
-- 
2.0.1.552.g1af257a

[PATCH v3 23/23] refs-be-files.c: add methods for the ref iterators

[Ronnie Sahlberg]

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 refs-be-files.c | 21 ++++++++++++++-------
 refs.c          | 36 ++++++++++++++++++++++++++++++++++++
 refs.h          | 18 ++++++++++++++++++
 3 files changed, 68 insertions(+), 7 deletions(-)

diff --git a/refs-be-files.c b/refs-be-files.c
index 910663b..7c0ab25 100644
--- a/refs-be-files.c
+++ b/refs-be-files.c
@@ -1614,33 +1614,33 @@ static int files_head_ref_submodule(const char *submodule, each_ref_fn fn,
 	return do_head_ref(submodule, fn, cb_data);
 }
 
-int for_each_ref(each_ref_fn fn, void *cb_data)
+static int files_for_each_ref(each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(&ref_cache, "", fn, 0, 0, cb_data);
 }
 
-int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+static int files_for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(get_ref_cache(submodule), "", fn, 0, 0, cb_data);
 }
 
-int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)
+static int files_for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(&ref_cache, prefix, fn, strlen(prefix), 0, cb_data);
 }
 
-int for_each_ref_in_submodule(const char *submodule, const char *prefix,
+static int files_for_each_ref_in_submodule(const char *submodule, const char *prefix,
 		each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(get_ref_cache(submodule), prefix, fn, strlen(prefix), 0, cb_data);
 }
 
-int for_each_replace_ref(each_ref_fn fn, void *cb_data)
+static int files_for_each_replace_ref(each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(&ref_cache, "refs/replace/", fn, 13, 0, cb_data);
 }
 
-int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
+static int files_for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
 {
 	struct strbuf buf = STRBUF_INIT;
 	int ret;
@@ -1650,7 +1650,7 @@ int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
 	return ret;
 }
 
-int for_each_rawref(each_ref_fn fn, void *cb_data)
+static int files_for_each_rawref(each_ref_fn fn, void *cb_data)
 {
 	return do_for_each_ref(&ref_cache, "", fn, 0,
 			       DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
@@ -3318,6 +3318,13 @@ struct ref_be refs_files = {
 	files_resolve_gitlink_ref,
 	files_head_ref,
 	files_head_ref_submodule,
+	files_for_each_ref,
+	files_for_each_ref_submodule,
+	files_for_each_ref_in,
+	files_for_each_ref_in_submodule,
+	files_for_each_rawref,
+	files_for_each_namespaced_ref,
+	files_for_each_replace_ref,
 };
 
 struct ref_be *refs = &refs_files;
diff --git a/refs.c b/refs.c
index 841d905..ceee979 100644
--- a/refs.c
+++ b/refs.c
@@ -931,3 +931,39 @@ int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
 {
 	return refs->head_ref_submodule(submodule, fn, cb_data);
 }
+
+int for_each_ref(each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_ref(fn, cb_data);
+}
+
+int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_ref_submodule(submodule, fn, cb_data);
+}
+
+int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_ref_in(prefix, fn, cb_data);
+}
+
+int for_each_ref_in_submodule(const char *submodule, const char *prefix,
+			      each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_ref_in_submodule(submodule, prefix, fn, cb_data);
+}
+
+int for_each_rawref(each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_rawref(fn, cb_data);
+}
+
+int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_namespaced_ref(fn, cb_data);
+}
+
+int for_each_replace_ref(each_ref_fn fn, void *cb_data)
+{
+	return refs->for_each_replace_ref(fn, cb_data);
+}
diff --git a/refs.h b/refs.h
index 92f8f44..bd3a0d4 100644
--- a/refs.h
+++ b/refs.h
@@ -397,6 +397,17 @@ typedef int (*resolve_gitlink_ref_fn)(const char *path, const char *refname,
 typedef int (*head_ref_fn)(each_ref_fn fn, void *cb_data);
 typedef int (*head_ref_submodule_fn)(const char *submodule, each_ref_fn fn,
 				     void *cb_data);
+typedef int (*for_each_ref_fn)(each_ref_fn fn, void *cb_data);
+typedef int (*for_each_ref_submodule_fn)(const char *submodule, each_ref_fn fn,
+					 void *cb_data);
+typedef int (*for_each_ref_in_fn)(const char *prefix, each_ref_fn fn,
+				  void *cb_data);
+typedef int (*for_each_ref_in_submodule_fn)(const char *submodule,
+					    const char *prefix,
+					    each_ref_fn fn, void *cb_data);
+typedef int (*for_each_rawref_fn)(each_ref_fn fn, void *cb_data);
+typedef int (*for_each_namespaced_ref_fn)(each_ref_fn fn, void *cb_data);
+typedef int (*for_each_replace_ref_fn)(each_ref_fn fn, void *cb_data);
 
 struct ref_be {
 	transaction_begin_fn transaction_begin;
@@ -420,6 +431,13 @@ struct ref_be {
 	resolve_gitlink_ref_fn resolve_gitlink_ref;
 	head_ref_fn head_ref;
 	head_ref_submodule_fn head_ref_submodule;
+	for_each_ref_fn for_each_ref;
+	for_each_ref_submodule_fn for_each_ref_submodule;
+	for_each_ref_in_fn for_each_ref_in;
+	for_each_ref_in_submodule_fn for_each_ref_in_submodule;
+	for_each_rawref_fn for_each_rawref;
+	for_each_namespaced_ref_fn for_each_namespaced_ref;
+	for_each_replace_ref_fn for_each_replace_ref;
 };
 
 extern struct ref_be *refs;
-- 
2.0.1.552.g1af257a

Re: [PATCH v3 03/23] refs.c: add a new refs.c file to hold all common refs code

[Junio C Hamano]

Ronnie Sahlberg <sahlberg@google.com> writes:

> Create a new erfs.c file that will be used to hold all the refs
> code that is backend agnostic and will be shared across all backends.
>
> The reason we renamed everything to refs-be-files.c in the previous patch
> and now start moving the common code back to the new refs.c file
> instead of the other way around is the etive volumes of code.

Huh?  Why not create refs-be-files.c and move whatever need to be
there over there, instead of rename the file and move things that
shouldn't have been moved back like this?

Puzzled.

I do not see 02/23 here, but I am assuming that is is just

    git mv refs.c refs-be-files.c

which may have been a seven-line patch with "format-patch -M" ;-)

>
> With the ref_cache, packed refs and loose ref handling that are all
> part of the files based implementation the backend specific part
> of the old refs.c file is several times larger than the backend agnostic
> part. Therefore it makes more sense to first rename everything to be
> part of the files based backend and then move the parts that can be used
> as common code back to refs.c.
>
> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
> ---
>  Makefile | 1 +
>  refs.c   | 3 +++
>  2 files changed, 4 insertions(+)
>  create mode 100644 refs.c
>
> diff --git a/Makefile b/Makefile
> index e010ad1..937d22a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -857,6 +857,7 @@ LIB_OBJS += quote.o
>  LIB_OBJS += reachable.o
>  LIB_OBJS += read-cache.o
>  LIB_OBJS += reflog-walk.o
> +LIB_OBJS += refs.o
>  LIB_OBJS += refs-be-files.o
>  LIB_OBJS += remote.o
>  LIB_OBJS += replace_object.o
> diff --git a/refs.c b/refs.c
> new file mode 100644
> index 0000000..77492ff
> --- /dev/null
> +++ b/refs.c
> @@ -0,0 +1,3 @@
> +/*
> + * Common refs code for all backends.
> + */

Re: [PATCH v3 19/23] refs-be-files.c: add a backend method structure with transaction functions

[Junio C Hamano]

Ronnie Sahlberg <sahlberg@google.com> writes:

> diff --git a/refs-be-files.c b/refs-be-files.c
> index e58a7e1..27eafd0 100644
> --- a/refs-be-files.c
> +++ b/refs-be-files.c
> ...
> +struct ref_be refs_files = {
> +	files_transaction_begin,
> +	files_transaction_update_sha1,
> +	files_transaction_create_sha1,
> +	files_transaction_delete_sha1,
> +	files_transaction_update_reflog,
> +	files_transaction_commit,
> +	files_transaction_free,
> +};
> +
> +struct ref_be *refs = &refs_files;

> diff --git a/refs.c b/refs.c
> index 6b434ad..b8c942f 100644
> --- a/refs.c
> +++ b/refs.c
> ...
> +void transaction_free(struct ref_transaction *transaction)
> +{
> +	return refs->transaction_free(transaction);
> +}
> diff --git a/refs.h b/refs.h
> index a14fc5d..4b669f5 100644
> --- a/refs.h
> +++ b/refs.h
> ...
> +struct ref_be {
> +	transaction_begin_fn transaction_begin;
> +	transaction_update_sha1_fn transaction_update_sha1;
> +	transaction_create_sha1_fn transaction_create_sha1;
> +	transaction_delete_sha1_fn transaction_delete_sha1;
> +	transaction_update_reflog_fn transaction_update_reflog;
> +	transaction_commit_fn transaction_commit;
> +	transaction_free_fn transaction_free;
> +};
> +
> +extern struct ref_be *refs;
> +
>  #endif /* REFS_H */

The overall structure is certainly nice, but this means you only can
LINK with one backend.  Is that what we really want?

I would have expected something like this:

  * In refs.c, there is a "static struct ref_be *the_refs_backend"
    that points at the chosen singleton backend;

  * Upon start-up, set_refs_backend() function that is exported from
    refs.c can be used to set the_refs_backend;

  * Each refs-be-frotz.c will export "struct ref_be refs_frotz" (or
    perhaps "struct refs_be refs_be_frotz") to the outside world, so
    that the start-up code can call set_refs_backend() with it.

  * It is probably sensible to keep the_refs_backend default to
    &refs_be_files.

Re: [PATCH v3 03/23] refs.c: add a new refs.c file to hold all common refs code

[Ronnie Sahlberg]

On Tue, Aug 26, 2014 at 2:31 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Ronnie Sahlberg <sahlberg@google.com> writes:
>
>> Create a new erfs.c file that will be used to hold all the refs
>> code that is backend agnostic and will be shared across all backends.
>>
>> The reason we renamed everything to refs-be-files.c in the previous patch
>> and now start moving the common code back to the new refs.c file
>> instead of the other way around is the etive volumes of code.
>
> Huh?  Why not create refs-be-files.c and move whatever need to be
> there over there, instead of rename the file and move things that
> shouldn't have been moved back like this?
>

The reason is the relative size of the code. I could do it the other
way but then
the changes that is moving the code would be much bigger.

Moving it like this, by first renaming it to refs-be-files.c and then
moving the backend agnostic parts back
is that the backend agnostic parts are mostly helper functions that
are independent of eachother.
This makes it possible to move just a few functions at a time making
the individual changes smaller and easier to manage when there are
merge conflicts.

A lot of the code that implements the actual files implementation for
refs storage implements the ref-cache/ref-dir/packed-refs/loose refs
etc.
This is all code that is intertwined and is difficult to split up.
Thus almost forcing me to move the whole 3000 lines of implementation
in one single monolithic patch.

I think "first rename, then move the agnostic parts a small section at
a time" was the least bad solution.




> Puzzled.
>
> I do not see 02/23 here, but I am assuming that is is just
>
>     git mv refs.c refs-be-files.c
>
> which may have been a seven-line patch with "format-patch -M" ;-)
>
>>
>> With the ref_cache, packed refs and loose ref handling that are all
>> part of the files based implementation the backend specific part
>> of the old refs.c file is several times larger than the backend agnostic
>> part. Therefore it makes more sense to first rename everything to be
>> part of the files based backend and then move the parts that can be used
>> as common code back to refs.c.
>>
>> Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
>> ---
>>  Makefile | 1 +
>>  refs.c   | 3 +++
>>  2 files changed, 4 insertions(+)
>>  create mode 100644 refs.c
>>
>> diff --git a/Makefile b/Makefile
>> index e010ad1..937d22a 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -857,6 +857,7 @@ LIB_OBJS += quote.o
>>  LIB_OBJS += reachable.o
>>  LIB_OBJS += read-cache.o
>>  LIB_OBJS += reflog-walk.o
>> +LIB_OBJS += refs.o
>>  LIB_OBJS += refs-be-files.o
>>  LIB_OBJS += remote.o
>>  LIB_OBJS += replace_object.o
>> diff --git a/refs.c b/refs.c
>> new file mode 100644
>> index 0000000..77492ff
>> --- /dev/null
>> +++ b/refs.c
>> @@ -0,0 +1,3 @@
>> +/*
>> + * Common refs code for all backends.
>> + */

Re: [PATCH v3 19/23] refs-be-files.c: add a backend method structure with transaction functions

[Ronnie Sahlberg]

On Tue, Aug 26, 2014 at 2:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Ronnie Sahlberg <sahlberg@google.com> writes:
>
>> diff --git a/refs-be-files.c b/refs-be-files.c
>> index e58a7e1..27eafd0 100644
>> --- a/refs-be-files.c
>> +++ b/refs-be-files.c
>> ...
>> +struct ref_be refs_files = {
>> +     files_transaction_begin,
>> +     files_transaction_update_sha1,
>> +     files_transaction_create_sha1,
>> +     files_transaction_delete_sha1,
>> +     files_transaction_update_reflog,
>> +     files_transaction_commit,
>> +     files_transaction_free,
>> +};
>> +
>> +struct ref_be *refs = &refs_files;
>
>> diff --git a/refs.c b/refs.c
>> index 6b434ad..b8c942f 100644
>> --- a/refs.c
>> +++ b/refs.c
>> ...
>> +void transaction_free(struct ref_transaction *transaction)
>> +{
>> +     return refs->transaction_free(transaction);
>> +}
>> diff --git a/refs.h b/refs.h
>> index a14fc5d..4b669f5 100644
>> --- a/refs.h
>> +++ b/refs.h
>> ...
>> +struct ref_be {
>> +     transaction_begin_fn transaction_begin;
>> +     transaction_update_sha1_fn transaction_update_sha1;
>> +     transaction_create_sha1_fn transaction_create_sha1;
>> +     transaction_delete_sha1_fn transaction_delete_sha1;
>> +     transaction_update_reflog_fn transaction_update_reflog;
>> +     transaction_commit_fn transaction_commit;
>> +     transaction_free_fn transaction_free;
>> +};
>> +
>> +extern struct ref_be *refs;
>> +
>>  #endif /* REFS_H */
>
> The overall structure is certainly nice, but this means you only can
> LINK with one backend.  Is that what we really want?
>
> I would have expected something like this:
>
>   * In refs.c, there is a "static struct ref_be *the_refs_backend"
>     that points at the chosen singleton backend;

Done.
It is also initialized to default to the files backend :
refs.c:

  /* We always have a files backend and it is the default */
  struct ref_be *the_refs_backend = &refs_be_files;


This does make "refs_be_files" and later "refs_be_db" public symbols instead of
the singleton. But we probably want/need these structures to be public anyway
if we at some stage want to be able to switch between backends at runtime.
refs.h:

   extern struct ref_be refs_be_files;
   void set_refs_backend(struct ref_be *ref_be);

Thus allowing us to
   set_refs_backend(&refs_be_files) to switch back to the files backend.




>
>   * Upon start-up, set_refs_backend() function that is exported from
>     refs.c can be used to set the_refs_backend;
>
>   * Each refs-be-frotz.c will export "struct ref_be refs_frotz" (or
>     perhaps "struct refs_be refs_be_frotz") to the outside world, so
>     that the start-up code can call set_refs_backend() with it.

Yepp.
refs-be-db.c: does this.

>
>   * It is probably sensible to keep the_refs_backend default to
>     &refs_be_files.
>

Yepp.

https://github.com/rsahlberg/git/tree/backend-struct-db
https://github.com/rsahlberg/git/tree/backend-struct-db-2 (adds a db
backend and daemon)


Thanks. Good suggestions.