[PATCH 1/2] object.c: free replace map in raw_object_store_clear

[PATCH 1/2] object.c: free replace map in raw_object_store_clear

[Stefan Beller]

The replace map for objects was missed to free in the object store in
the conversion of 174774cd519 (Merge branch 'sb/object-store-replace',
2018-05-08)

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/object.c b/object.c
index 9d5b10d5a20..ff28f90c5ef 100644
--- a/object.c
+++ b/object.c
@@ -497,6 +497,7 @@ void raw_object_store_clear(struct raw_object_store *o)
 {
 	FREE_AND_NULL(o->objectdir);
 	FREE_AND_NULL(o->alternate_db);
+	FREE_AND_NULL(o->replace_map);
 
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
-- 
2.17.0.255.g8bfb7c0704

[PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

[Stefan Beller]

This was missed in 5982da9d2ce (replace-object: allow
prepare_replace_object to handle arbitrary repositories, 2018-04-11)

Technically the code works correctly as the replace_map is the same
size in different repositories, however it is hard to read. So convert
the code to the familiar pattern of dereferencing the pointer that we
assign in the sizeof itself.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 replace_object.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/replace_object.c b/replace_object.c
index 246b98cd4f1..801b5c16789 100644
--- a/replace_object.c
+++ b/replace_object.c
@@ -37,7 +37,7 @@ static void prepare_replace_object(struct repository *r)
 		return;
 
 	r->objects->replace_map =
-		xmalloc(sizeof(*the_repository->objects->replace_map));
+		xmalloc(sizeof(*r->objects->replace_map));
 	oidmap_init(r->objects->replace_map, 0);
 
 	for_each_replace_ref(r, register_replace_ref, NULL);
-- 
2.17.0.255.g8bfb7c0704

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> This was missed in 5982da9d2ce (replace-object: allow
> prepare_replace_object to handle arbitrary repositories, 2018-04-11)
>
> Technically the code works correctly as the replace_map is the same
> size in different repositories, however it is hard to read. So convert
> the code to the familiar pattern of dereferencing the pointer that we
> assign in the sizeof itself.

;-)

We say

	ptr = xmalloc(sizeof(*ptr))

is better because 

	ptr = xmalloc(sizeof(typeof(*ptr)))

is easy to go stale unless we actually use typeof and instead say a
concrete type like "struct oidmap".

This one was doing

	ptr = xmalloc(sizeof(*another_ptr))

and it was OK because ptr and another_ptr happened to be of the same
type.  I wonder if we are making it safer, or making it more obscure
to seasoned C programmers, if we introduced a pair of helper macros,
perhaps like these:

	#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
	#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))

The change looks obviously good.  Will queue.

Thanks.

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

[Jeff King]

On Thu, May 10, 2018 at 07:23:13PM +0900, Junio C Hamano wrote:

> This one was doing
> 
> 	ptr = xmalloc(sizeof(*another_ptr))
> 
> and it was OK because ptr and another_ptr happened to be of the same
> type.  I wonder if we are making it safer, or making it more obscure
> to seasoned C programmers, if we introduced a pair of helper macros,
> perhaps like these:
> 
> 	#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
> 	#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))

I've often wondered that, too. It's the natural endgame of the
ALLOC_ARRAY() road we've been going down.

-Peff

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

[Derrick Stolee]

On 5/10/2018 7:56 AM, Jeff King wrote:
> On Thu, May 10, 2018 at 07:23:13PM +0900, Junio C Hamano wrote:
>
>> This one was doing
>>
>> 	ptr = xmalloc(sizeof(*another_ptr))
>>
>> and it was OK because ptr and another_ptr happened to be of the same
>> type.  I wonder if we are making it safer, or making it more obscure
>> to seasoned C programmers, if we introduced a pair of helper macros,
>> perhaps like these:
>>
>> 	#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
>> 	#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))
> I've often wondered that, too. It's the natural endgame of the
> ALLOC_ARRAY() road we've been going down.

I'll chime in that I like this idea.

Because I'm trying to learn more about Coccinelle, I added the diff 
below and ran 'make coccicheck'. It resulted in a 1000-line patch on 
'next'. I'll refrain from sending that patch series, but it's nice to 
know a "simple" transformation is possible.

Using `git grep` I see 230 instances of 'xmalloc' and 261 instances of 
'xcalloc'. After the Coccinelle transformation, these are down to 194 
and 190, respectively, because the rest allocate in the same line as the 
definition. It's worth thinking about the macro pattern for those cases.

diff --git a/contrib/coccinelle/alloc.cocci b/contrib/coccinelle/alloc.cocci
new file mode 100644
index 0000000000..95f426c4dc
--- /dev/null
+++ b/contrib/coccinelle/alloc.cocci
@@ -0,0 +1,13 @@
+@@
+expression p;
+@@
+- p = xmalloc(sizeof(*p));
++ ALLOCATE(p);
+
+@@
+expression c;
+expression p;
+@@
+- p = xcalloc(c, sizeof(*p));
++ CALLOCATE(p,c);
+
diff --git a/git-compat-util.h b/git-compat-util.h
index f9e4c5f9bc..5398f217d7 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -843,6 +843,9 @@ extern FILE *fopen_or_warn(const char *path, const 
char *mode);
   */
  #define FREE_AND_NULL(p) do { free(p); (p) = NULL; } while (0)

+#define ALLOCATE(ptr) (ptr) = xmalloc(sizeof(*(ptr)))
+#define CALLOCATE(ptr,cnt) (ptr) = xcalloc((cnt), sizeof(*(ptr)))
+
  #define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), 
(alloc)))
  #define REALLOC_ARRAY(x, alloc) (x) = xrealloc((x), 
st_mult(sizeof(*(x)), (alloc)))

Re: [PATCH 2/2] replace-object.c: remove the_repository from prepare_replace_object

[Stefan Beller]

> Using `git grep` I see 230 instances of 'xmalloc' and 261 instances of
> 'xcalloc'. After the Coccinelle transformation, these are down to 194 and
> 190, respectively, because the rest allocate in the same line as the
> definition. It's worth thinking about the macro pattern for those cases.

Thanks for reporting the coccinelle experiment!

As we follow a strict declare before code, and we do not know if further
declarations make use of this already, e.g. given

    struct foo *f = xmalloc(sizeof(*f));
    struct bar b = &f->baz;

we cannot split up the line declaring and assigning f, but the macro
has to recreate the assignment upon declaration, for that we'd
need to have something like

    ALLOCATE_TYPE(type, name);

which over complicates things IMHO.

Maybe it is worth identifying the pattern where 'f' is not used in further
declarations, such that we can make patches as

-    struct foo *f = xmalloc(sizeof(*f));
+   struct foo *f;
    struct baz b = &unrelated;
+
+ ALLOCATE(f);
+

Thanks,
Stefan

Re: [PATCH 1/2] object.c: free replace map in raw_object_store_clear

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> The replace map for objects was missed to free in the object store in
> the conversion of 174774cd519 (Merge branch 'sb/object-store-replace',
> 2018-05-08)

I need a bit of clarification wrt the above.  The above makes it
sound like the merge needed a semantic conflict resolution (e.g. one
side turned replace_map into a pointer while the other side added a
place where the containing structure is freed and now the merge
result needs to free the pointer in the new place that frees the
containing structure, but the merge forgot to do so).  Is that what
is going on?

Or is this just a simple "the topic that ends at 174774cd519^2 had
this leak that needs to be fixed by this patch; instead of rerolling
this is an incremental, because the topic has already been merged to
'master' and it is too late now"?

Looking at this patch in the context of the side branch (instead of
in the merged result) already makes sense to me, so I am guessing it
is the latter (i.e. not a botched merge that missed semantic
conflicts), in which case the proposed log message is a bit too
alarming and points readers in a wrong direction.  Shouldn't it
point at, say, c1274495 ("replace-object: eliminate replace objects
prepared flag", 2018-04-11) that turned the oidmap instance into a
pointer in raw_object_store?

Thanks.

>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  object.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/object.c b/object.c
> index 9d5b10d5a20..ff28f90c5ef 100644
> --- a/object.c
> +++ b/object.c
> @@ -497,6 +497,7 @@ void raw_object_store_clear(struct raw_object_store *o)
>  {
>  	FREE_AND_NULL(o->objectdir);
>  	FREE_AND_NULL(o->alternate_db);
> +	FREE_AND_NULL(o->replace_map);
>  
>  	free_alt_odbs(o);
>  	o->alt_odb_tail = NULL;

Re: [PATCH 1/2] object.c: free replace map in raw_object_store_clear

[Stefan Beller]

Hi Junio,

On Thu, May 10, 2018 at 3:16 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> The replace map for objects was missed to free in the object store in
>> the conversion of 174774cd519 (Merge branch 'sb/object-store-replace',
>> 2018-05-08)
>
>
> Or is this just a simple "the topic that ends at 174774cd519^2 had
> this leak that needs to be fixed by this patch; instead of rerolling
> this is an incremental, because the topic has already been merged to
> 'master' and it is too late now"?

This is the case. I wondered if I want to point to the exact commit
(which I have trouble pointing to as the whole topic has no memory
leak fixes, Closest would be d88f9fdf8b2 (replace-object: move
replace_map to object store, 2018-04-11))
or rather just point at the series. I did not think of the confusion
that might arise there.

> Looking at this patch in the context of the side branch (instead of
> in the merged result) already makes sense to me, so I am guessing it
> is the latter (i.e. not a botched merge that missed semantic
> conflicts), in which case the proposed log message is a bit too
> alarming and points readers in a wrong direction.  Shouldn't it
> point at, say, c1274495 ("replace-object: eliminate replace objects
> prepared flag", 2018-04-11) that turned the oidmap instance into a
> pointer in raw_object_store?

Ah, that is the best place to point at. Makes sense.
I'll reroll this,

Thanks,
Stefan

Re: What's cooking in git.git (May 2018, #02; Thu, 17)

[Stefan Beller]

> * sb/object-store-replace (2018-05-10) 2 commits
>   (merged to 'next' on 2018-05-16 at 41bbedcc81)
>  + replace-object.c: remove the_repository from prepare_replace_object
>  + object.c: free replace map in raw_object_store_clear
>
>  Hotfix.
>
>  Will merge to 'master'.

Please do not.
(Or do, but then be prepared for another hotfix.)

The commit sb/object-store-replace^ needs more free'ing and shall be
replaced with
https://public-inbox.org/git/20180510195849.28023-4-sbeller@google.com/
I'll resend shortly.

>
> * sb/submodule-merge-in-merge-recursive (2018-05-16) 3 commits
>  - merge-recursive: give notice when submodule commit gets fast-forwarded
>  - merge-recursive: i18n submodule merge output and respect verbosity
>  - submodule.c: move submodule merging to merge-recursive.c
>
>  By code restructuring of submodule merge in merge-recursive,
>  informational messages from the codepath are now given using the
>  same mechanism as other output, and honor the merge.verbosity
>  configuration.  The code also learned to give a few new messages
>  when a submodule three-way merge resolves cleanly when one side
>  records a descendant of the commit chosen by the other side.
>
>  Will merge to 'next'.

Merging would be ok, but I would rather not.
A resend will be only for cosmetic effect, as I messed up the last commit

So, please hold in pu.

> * sb/diff-color-move-more (2018-04-25) 7 commits
>  - diff.c: add --color-moved-ignore-space-delta option
>  - diff.c: decouple white space treatment from move detection algorithm
>  - diff.c: add a blocks mode for moved code detection
>  - diff.c: adjust hash function signature to match hashmap expectation
>  - diff.c: do not pass diff options as keydata to hashmap
>  - xdiff/xdiffi.c: remove unneeded function declarations
>  - xdiff/xdiff.h: remove unused flags
>
>  "git diff --color-moved" feature has further been tweaked.
>
>  Will merge to 'next'.

I did not get around to fix it up, there are still review
comments outstanding. (The test is broken in the last commit.)

Please hold in pu;

Thanks,
Stefan

[PATCH 0/2] Reroll 2 last commits of sb/object-store-replace

[Stefan Beller]

>> * sb/object-store-replace (2018-05-10) 2 commits
>>   (merged to 'next' on 2018-05-16 at 41bbedcc81)
>>  + replace-object.c: remove the_repository from prepare_replace_object
>>  + object.c: free replace map in raw_object_store_clear
>>
>>  Hotfix.
>>
>>  Will merge to 'master'.
>
>Please do not.

Here comes a reroll with another fix, diff below, as the branch-diff did
not give a good output[1].

Thanks,
Stefan 

$ git diff origin/sb/object-store-replace
diff --git c/object.c w/object.c
index cdf520084df..97245fdea25 100644
--- c/object.c
+++ w/object.c
@@ -480,6 +480,8 @@ void raw_object_store_clear(struct raw_object_store *o)
 {
        FREE_AND_NULL(o->objectdir);
        FREE_AND_NULL(o->alternate_db);
+
+       oidmap_free(o->replace_map, 1);
        FREE_AND_NULL(o->replace_map);
 
        free_alt_odbs(o);

[1]
$ ./git-branch-diff origin/sb/object-store-replace^^..origin/sb/object-store-replace sb/object-store-replace^^..sb/object-store-replace
1:  d5873072054 < -:  ----------- object.c: free replace map in raw_object_store_clear
-:  ----------- > 1:  88d456c5d24 object.c: free replace map in raw_object_store_clear
2:  74fd0705bb3 = 2:  6da07fe2f9c replace-object.c: remove the_repository from prepare_replace_object

Stefan Beller (2):
  object.c: free replace map in raw_object_store_clear
  replace-object.c: remove the_repository from prepare_replace_object

 object.c         | 3 +++
 replace_object.c | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

-- 
2.17.0.582.gccdcbd54c44.dirty

[PATCH 1/2] object.c: free replace map in raw_object_store_clear

[Stefan Beller]

The replace map for objects was missed to free in the object store in
the conversion of c1274495 ("replace-object: eliminate replace objects
prepared flag", 2018-04-11). We also missed to free the replaced objects
that are put into the replace map in that whole series.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/object.c b/object.c
index 66cffaf6e51..97245fdea25 100644
--- a/object.c
+++ b/object.c
@@ -481,6 +481,9 @@ void raw_object_store_clear(struct raw_object_store *o)
 	FREE_AND_NULL(o->objectdir);
 	FREE_AND_NULL(o->alternate_db);
 
+	oidmap_free(o->replace_map, 1);
+	FREE_AND_NULL(o->replace_map);
+
 	free_alt_odbs(o);
 	o->alt_odb_tail = NULL;
 
-- 
2.17.0.582.gccdcbd54c44.dirty