[PATCH] decorate: clean up and document API

[PATCH] decorate: clean up and document API

[Jonathan Tan]

Improve the names of the identifiers in decorate.h, document them, and
add an example of how to use these functions.

The example is compiled and run as part of the test suite.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
This patch contains some example code in a test helper.

There is a discussion at [1] about where example code for hashmap should
go. For something relatively simple, like decorate, perhaps example code
isn't necessary; but if we include example code anyway, I think that we
might as well make it compiled and tested, so this patch contains that
example code in a test helper.

[1] https://public-inbox.org/git/466dd5331907754ca5c25cc83173ca895220ca81.1511999045.git.johannes.schindelin@gmx.de/
---
 Documentation/technical/api-decorate.txt |  6 ---
 Makefile                                 |  1 +
 builtin/fast-export.c                    |  2 +-
 decorate.c                               | 28 ++++++------
 decorate.h                               | 49 +++++++++++++++++++--
 t/helper/.gitignore                      |  1 +
 t/helper/test-example-decorate.c         | 74 ++++++++++++++++++++++++++++++++
 t/t9004-example.sh                       | 10 +++++
 8 files changed, 146 insertions(+), 25 deletions(-)
 delete mode 100644 Documentation/technical/api-decorate.txt
 create mode 100644 t/helper/test-example-decorate.c
 create mode 100755 t/t9004-example.sh

diff --git a/Documentation/technical/api-decorate.txt b/Documentation/technical/api-decorate.txt
deleted file mode 100644
index 1d52a6ce1..000000000
--- a/Documentation/technical/api-decorate.txt
+++ /dev/null
@@ -1,6 +0,0 @@
-decorate API
-============
-
-Talk about <decorate.h>
-
-(Linus)
diff --git a/Makefile b/Makefile
index fef9c8d27..df52cc1c3 100644
--- a/Makefile
+++ b/Makefile
@@ -651,6 +651,7 @@ TEST_PROGRAMS_NEED_X += test-dump-cache-tree
 TEST_PROGRAMS_NEED_X += test-dump-fsmonitor
 TEST_PROGRAMS_NEED_X += test-dump-split-index
 TEST_PROGRAMS_NEED_X += test-dump-untracked-cache
+TEST_PROGRAMS_NEED_X += test-example-decorate
 TEST_PROGRAMS_NEED_X += test-fake-ssh
 TEST_PROGRAMS_NEED_X += test-genrandom
 TEST_PROGRAMS_NEED_X += test-hashmap
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index f8fe04ca5..796d0cd66 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -895,7 +895,7 @@ static void export_marks(char *file)
 {
 	unsigned int i;
 	uint32_t mark;
-	struct object_decoration *deco = idnums.hash;
+	struct decoration_entry *deco = idnums.entries;
 	FILE *f;
 	int e = 0;
 
diff --git a/decorate.c b/decorate.c
index 270eb2519..de31331fa 100644
--- a/decorate.c
+++ b/decorate.c
@@ -14,20 +14,20 @@ static unsigned int hash_obj(const struct object *obj, unsigned int n)
 static void *insert_decoration(struct decoration *n, const struct object *base, void *decoration)
 {
 	int size = n->size;
-	struct object_decoration *hash = n->hash;
+	struct decoration_entry *entries = n->entries;
 	unsigned int j = hash_obj(base, size);
 
-	while (hash[j].base) {
-		if (hash[j].base == base) {
-			void *old = hash[j].decoration;
-			hash[j].decoration = decoration;
+	while (entries[j].base) {
+		if (entries[j].base == base) {
+			void *old = entries[j].decoration;
+			entries[j].decoration = decoration;
 			return old;
 		}
 		if (++j >= size)
 			j = 0;
 	}
-	hash[j].base = base;
-	hash[j].decoration = decoration;
+	entries[j].base = base;
+	entries[j].decoration = decoration;
 	n->nr++;
 	return NULL;
 }
@@ -36,24 +36,23 @@ static void grow_decoration(struct decoration *n)
 {
 	int i;
 	int old_size = n->size;
-	struct object_decoration *old_hash = n->hash;
+	struct decoration_entry *old_entries = n->entries;
 
 	n->size = (old_size + 1000) * 3 / 2;
-	n->hash = xcalloc(n->size, sizeof(struct object_decoration));
+	n->entries = xcalloc(n->size, sizeof(struct decoration_entry));
 	n->nr = 0;
 
 	for (i = 0; i < old_size; i++) {
-		const struct object *base = old_hash[i].base;
-		void *decoration = old_hash[i].decoration;
+		const struct object *base = old_entries[i].base;
+		void *decoration = old_entries[i].decoration;
 
 		if (!decoration)
 			continue;
 		insert_decoration(n, base, decoration);
 	}
-	free(old_hash);
+	free(old_entries);
 }
 
-/* Add a decoration pointer, return any old one */
 void *add_decoration(struct decoration *n, const struct object *obj,
 		void *decoration)
 {
@@ -64,7 +63,6 @@ void *add_decoration(struct decoration *n, const struct object *obj,
 	return insert_decoration(n, obj, decoration);
 }
 
-/* Lookup a decoration pointer */
 void *lookup_decoration(struct decoration *n, const struct object *obj)
 {
 	unsigned int j;
@@ -74,7 +72,7 @@ void *lookup_decoration(struct decoration *n, const struct object *obj)
 		return NULL;
 	j = hash_obj(obj, n->size);
 	for (;;) {
-		struct object_decoration *ref = n->hash + j;
+		struct decoration_entry *ref = n->entries + j;
 		if (ref->base == obj)
 			return ref->decoration;
 		if (!ref->base)
diff --git a/decorate.h b/decorate.h
index e7328044f..9014c1e99 100644
--- a/decorate.h
+++ b/decorate.h
@@ -1,18 +1,61 @@
 #ifndef DECORATE_H
 #define DECORATE_H
 
-struct object_decoration {
+/*
+ * A data structure that associates Git objects to void pointers. See
+ * t/helper/test-example-decorate.c for a demonstration of how to use these
+ * functions.
+ */
+
+/*
+ * An entry in the data structure.
+ */
+struct decoration_entry {
 	const struct object *base;
 	void *decoration;
 };
 
+/*
+ * The data structure.
+ *
+ * This data structure must be zero-initialized.
+ */
 struct decoration {
+	/*
+	 * Not used by the decoration mechanism. Clients may use this for
+	 * whatever they want.
+	 */
 	const char *name;
-	unsigned int size, nr;
-	struct object_decoration *hash;
+
+	/*
+	 * The capacity of "entries".
+	 */
+	unsigned int size;
+
+	/*
+	 * The number of real Git objects (that is, entries with non-NULL
+	 * "base").
+	 */
+	unsigned int nr;
+
+	/*
+	 * The entries. This is an array of size "size", containing nr entries
+	 * with non-NULL "base" and (size - nr) entries with NULL "base".
+	 */
+	struct decoration_entry *entries;
 };
 
+/*
+ * Add an association from the given object to the given pointer (which may be
+ * NULL), returning the previously associated pointer. If there is no previous
+ * association, this function returns NULL.
+ */
 extern void *add_decoration(struct decoration *n, const struct object *obj, void *decoration);
+
+/*
+ * Return the pointer associated to the given object. If there is no
+ * association, this function returns NULL.
+ */
 extern void *lookup_decoration(struct decoration *n, const struct object *obj);
 
 #endif
diff --git a/t/helper/.gitignore b/t/helper/.gitignore
index d02f9b39a..fff6aef22 100644
--- a/t/helper/.gitignore
+++ b/t/helper/.gitignore
@@ -8,6 +8,7 @@
 /test-dump-fsmonitor
 /test-dump-split-index
 /test-dump-untracked-cache
+/test-example-decorate
 /test-fake-ssh
 /test-scrap-cache-tree
 /test-genrandom
diff --git a/t/helper/test-example-decorate.c b/t/helper/test-example-decorate.c
new file mode 100644
index 000000000..90dc97a9d
--- /dev/null
+++ b/t/helper/test-example-decorate.c
@@ -0,0 +1,74 @@
+#include "cache.h"
+#include "object.h"
+#include "decorate.h"
+
+int cmd_main(int argc, const char **argv)
+{
+	struct decoration n;
+	struct object_id one_oid = { {1} };
+	struct object_id two_oid = { {2} };
+	struct object_id three_oid = { {3} };
+	struct object *one, *two, *three;
+
+	int decoration_a, decoration_b;
+
+	void *ret;
+
+	int i, objects_noticed = 0;
+
+	/*
+	 * The struct must be zero-initialized.
+	 */
+	memset(&n, 0, sizeof(n));
+
+	/*
+	 * Add 2 objects, one with a non-NULL decoration and one with a NULL
+	 * decoration.
+	 */
+	one = lookup_unknown_object(one_oid.hash);
+	two = lookup_unknown_object(two_oid.hash);
+	ret = add_decoration(&n, one, &decoration_a);
+	if (ret)
+		die("BUG: when adding a brand-new object, NULL should be returned");
+	ret = add_decoration(&n, two, NULL);
+	if (ret)
+		die("BUG: when adding a brand-new object, NULL should be returned");
+
+	/*
+	 * When re-adding an already existing object, the old decoration is
+	 * returned.
+	 */
+	ret = add_decoration(&n, one, NULL);
+	if (ret != &decoration_a)
+		die("BUG: when readding an already existing object, existing decoration should be returned");
+	ret = add_decoration(&n, two, &decoration_b);
+	if (ret)
+		die("BUG: when readding an already existing object, existing decoration should be returned");
+
+	/*
+	 * Lookup returns the added declarations, or NULL if the object was
+	 * never added.
+	 */
+	ret = lookup_decoration(&n, one);
+	if (ret)
+		die("BUG: lookup should return added declaration");
+	ret = lookup_decoration(&n, two);
+	if (ret != &decoration_b)
+		die("BUG: lookup should return added declaration");
+	three = lookup_unknown_object(three_oid.hash);
+	ret = lookup_decoration(&n, three);
+	if (ret)
+		die("BUG: lookup for unknown object should return NULL");
+
+	/*
+	 * The user can also loop through all entries.
+	 */
+	for (i = 0; i < n.size; i++) {
+		if (n.entries[i].base)
+			objects_noticed++;
+	}
+	if (objects_noticed != 2)
+		die("BUG: should have 2 objects");
+
+	return 0;
+}
diff --git a/t/t9004-example.sh b/t/t9004-example.sh
new file mode 100755
index 000000000..b28a028f5
--- /dev/null
+++ b/t/t9004-example.sh
@@ -0,0 +1,10 @@
+#!/bin/sh
+
+test_description='check that example code compiles and runs'
+. ./test-lib.sh
+
+test_expect_success 'decorate' '
+	test-example-decorate
+'
+
+test_done
-- 
2.15.1.424.g9478a66081-goog

Re: [PATCH] decorate: clean up and document API

[Jeff King]

On Thu, Dec 07, 2017 at 04:14:24PM -0800, Jonathan Tan wrote:

> Improve the names of the identifiers in decorate.h, document them, and
> add an example of how to use these functions.
> 
> The example is compiled and run as part of the test suite.
> 
> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
> ---
> This patch contains some example code in a test helper.
> 
> There is a discussion at [1] about where example code for hashmap should
> go. For something relatively simple, like decorate, perhaps example code
> isn't necessary; but if we include example code anyway, I think that we
> might as well make it compiled and tested, so this patch contains that
> example code in a test helper.

I have mixed feelings. On the one hand, compiling and running the code
ensures that those things actually work. On the other hand, I expect you
can make a much clearer example if instead of having running code, you
show snippets of almost-code.

E.g.:

  struct decoration d = { NULL };

  add_decoration(&d, obj, "foo");
  ...
  str = lookup_decoration(obj);

pretty much gives the relevant overview, with very little boilerplate.
Yes, it omits things like the return value of add_decoration(), but
those sorts of details are probably better left to the function
docstrings.

Other than that philosophical point, the documentation you added looks
pretty good to me. Two possible improvements to the API we could do on
top:

  1. Should there be a DECORATION_INIT macro (possibly taking the "name"
     as an argument)? (Actually, the whole name thing seems like a
     confusing and bad API design in the first place).

  2. This is really just an oidmap to a void pointer. I wonder if we
     ought to be wrapping that code (I think we still want some
     interface so that the caller doesn't have to declare their own
     structs).

-Peff

Re: [PATCH] decorate: clean up and document API

[Jonathan Tan]

On Fri, 8 Dec 2017 04:55:11 -0500
Jeff King <peff@peff.net> wrote:

> I have mixed feelings. On the one hand, compiling and running the code
> ensures that those things actually work. On the other hand, I expect you
> can make a much clearer example if instead of having running code, you
> show snippets of almost-code.
> 
> E.g.:
> 
>   struct decoration d = { NULL };
> 
>   add_decoration(&d, obj, "foo");
>   ...
>   str = lookup_decoration(obj);
> 
> pretty much gives the relevant overview, with very little boilerplate.
> Yes, it omits things like the return value of add_decoration(), but
> those sorts of details are probably better left to the function
> docstrings.

The part about iterating over all entries should probably also be shown
too. Besides that, I'm OK with having a simplified example in
documentation too, but I'll wait and see if others have any opinions
before making any changes.

> Other than that philosophical point, the documentation you added looks
> pretty good to me. Two possible improvements to the API we could do on
> top:
> 
>   1. Should there be a DECORATION_INIT macro (possibly taking the "name"
>      as an argument)? (Actually, the whole name thing seems like a
>      confusing and bad API design in the first place).

Agreed about the "name" thing. I'll add a DECORATION_INIT when I make
the next reroll, but I think that having it with no argument is best
(and instantiating "name" with NULL).

>   2. This is really just an oidmap to a void pointer. I wonder if we
>      ought to be wrapping that code (I think we still want some
>      interface so that the caller doesn't have to declare their own
>      structs).

It is slightly different from oidmap in that this uses "struct object *"
as a key whereas oidmap uses "struct object_id", meaning that a user of
"decorate" must already have objects allocated or be willing to allocate
them, whereas a user of "oidmap" doesn't.

Having said that, it is true that perhaps we have too many data
structures doing similar things.

Re: [PATCH] decorate: clean up and document API

[Jeff King]

On Mon, Dec 11, 2017 at 10:32:49AM -0800, Jonathan Tan wrote:

> > Other than that philosophical point, the documentation you added looks
> > pretty good to me. Two possible improvements to the API we could do on
> > top:
> > 
> >   1. Should there be a DECORATION_INIT macro (possibly taking the "name"
> >      as an argument)? (Actually, the whole name thing seems like a
> >      confusing and bad API design in the first place).
> 
> Agreed about the "name" thing. I'll add a DECORATION_INIT when I make
> the next reroll, but I think that having it with no argument is best
> (and instantiating "name" with NULL).

That will leave callers like the one in log-tree unable to use the
macro, since it uses a static initializer. I didn't dig, though. That
may be the only one. Most of the rest seem to just get explicitly
zero-initialized (some via xcalloc of a larger struct, so maybe we
should just promise that zero-initialization is always OK).

> >   2. This is really just an oidmap to a void pointer. I wonder if we
> >      ought to be wrapping that code (I think we still want some
> >      interface so that the caller doesn't have to declare their own
> >      structs).
> 
> It is slightly different from oidmap in that this uses "struct object *"
> as a key whereas oidmap uses "struct object_id", meaning that a user of
> "decorate" must already have objects allocated or be willing to allocate
> them, whereas a user of "oidmap" doesn't.

Ah, right. I was thinking the difference was only on the "value" half
being a pointer versus a struct.

It's nice in the current code that decorations do not incur the extra
cost of storing the oid twice (once in the "struct object", and then
again in the map key).  OTOH, that might well be premature optimization.

-Peff