Re: [PATCH] fast-import: replace custom hash with hashmap.c

[Jeff King <peff@peff.net>]

On Mon, Apr 06, 2020 at 02:35:42PM -0400, Jeff King wrote:

> Pointing to uninitialized bits is fine according to the standard (and I
> think that's what you're asking about for chapter and verse). But we
> really do lie to container_of(). See remote.c, for example. In
> make_remote(), we call hashmap_get() with a pointer to lookup_entry,
> which is a bare "struct hashmap_entry". That should end up in
> remotes_hash_cmp(), which unconditionally computes a pointer to a
> "struct remote".
> 
> Now the hashmap_entry there is at the beginning of the struct, so the
> offset is 0 and the two pointers are the same. So while the pointer's
> type is incorrect, we didn't compute an invalid pointer value. And
> traditionally the hashmap code required that it be at the front of the
> containing struct, but that was loosened recently (and container_of
> introduced) in 5efabc7ed9 (Merge branch 'ew/hashmap', 2019-10-15).
> 
> And grepping around, I don't see any cases where it's _not_ at the
> beginning. So perhaps this is a problem waiting to bite us.

I was curious wither ASan/UBSan might complain if we did something like
this:

diff --git a/oidmap.h b/oidmap.h
index c66a83ab1d..1abfe966f2 100644
--- a/oidmap.h
+++ b/oidmap.h
@@ -12,10 +12,10 @@
  * internal_entry field.
  */
 struct oidmap_entry {
+	struct object_id oid;
+
 	/* For internal use only */
 	struct hashmap_entry internal_entry;
-
-	struct object_id oid;
 };
 

It does, but not because of the subtle UB issue. It's just that the rest
of oidmap still assumes the hashmap-is-first logic, and we need:

diff --git a/oidmap.c b/oidmap.c
index 423aa014a3..4065ea4b79 100644
--- a/oidmap.c
+++ b/oidmap.c
@@ -35,7 +35,10 @@ void *oidmap_get(const struct oidmap *map, const struct object_id *key)
 	if (!map->map.cmpfn)
 		return NULL;
 
-	return hashmap_get_from_hash(&map->map, oidhash(key), key);
+	return container_of_or_null(
+		hashmap_get_from_hash(&map->map, oidhash(key), key),
+		struct oidmap_entry,
+		internal_entry);
 }
 
 void *oidmap_remove(struct oidmap *map, const struct object_id *key)
 struct oidmap {
@@ -79,7 +79,10 @@ static inline void oidmap_iter_init(struct oidmap *map, struct oidmap_iter *iter
 static inline void *oidmap_iter_next(struct oidmap_iter *iter)
 {
 	/* TODO: this API could be reworked to do compile-time type checks */
-	return (void *)hashmap_iter_next(&iter->h_iter);
+	return container_of_or_null(
+		hashmap_iter_next(&iter->h_iter),
+		struct oidmap_entry,
+		internal_entry);
 }
 
 static inline void *oidmap_iter_first(struct oidmap *map,

-Peff