git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Stefan Beller <sbeller@google.com>
To: Duy Nguyen <pclouds@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>,
	Jonathan Tan <jonathantanmy@google.com>,
	Junio C Hamano <gitster@pobox.com>,
	Jameson Miller <jamill@microsoft.com>
Subject: Re: [PATCH v2 13/13] alloc: allow arbitrary repositories for alloc functions
Date: Tue, 8 May 2018 11:38:01 -0700	[thread overview]
Message-ID: <CAGZ79kZhAjArSksrWMzJc49BKu1HF-R6kBHEzkus-in4fbTZvQ@mail.gmail.com> (raw)
In-Reply-To: <CACsJy8D-e-bff3S+LQAMfwB-w8OpkjrfFrV9O5S3ku+M5aAjQA@mail.gmail.com>

On Tue, May 8, 2018 at 8:00 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Tue, May 8, 2018 at 12:59 AM, Stefan Beller <sbeller@google.com> wrote:
>> @@ -501,9 +509,31 @@ void raw_object_store_clear(struct raw_object_store *o)
>>  void parsed_object_pool_clear(struct parsed_object_pool *o)
>>  {
>>         /*
>> -        * TOOD free objects in o->obj_hash.
>> -        *
>>          * As objects are allocated in slabs (see alloc.c), we do
>>          * not need to free each object, but each slab instead.
>> +        *
>> +        * Before doing so, we need to free any additional memory
>> +        * the objects may hold.
>>          */
>> +       unsigned i;
>> +
>> +       for (i = 0; i < o->obj_hash_size; i++) {
>> +               struct object *obj = o->obj_hash[i];
>> +
>> +               if (!obj)
>> +                       continue;
>> +
>> +               if (obj->type == OBJ_TREE) {
>> +                       free(((struct tree*)obj)->buffer);
>
> It would be nicer to keep this in separate functions, e.g.
> release_tree_node() and release_commit_node() to go with
> alloc_xxx_node().

ok, I can introduce that, although it seems unnecessary complicated
for now.

On top of this series I started an experiment (which rewrites alloc
and object.c a whole lot more; for performance reasons), which gets
rid of the multiple alloc_states. There will be only one allocation for
one repository, it can allocate across multiple types without alignment
overhead. It will reduce memory footprint of obj_hash by half, via
storing indexes instead of pointers in there.
That said, the experiment shall not influence the
direction of this series. Will fix.

>> +               } else if (obj->type == OBJ_COMMIT) {
>> +                       free_commit_list(((struct commit*)obj)->parents);
>> +                       free(&((struct commit*)obj)->util);
>> +               }
>> +       }
>
> I still don't see who frees obj_hash[] (or at least clears it if not
> freed). If I'm going to use this to free memory in pack-objects then
> I'd really prefer obj_hash[] freed because it's a big _big_ array.

gah!

> Just to be clear, what I mean is
>
> FREE_AND_NULL(o->obj_hash);
> o->obj_hash_size = 0;

ok, I just put it here, just before the calls
to clear_alloc_state()s.

  reply	other threads:[~2018-05-08 18:38 UTC|newest]

Thread overview: 95+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-01 21:33 [PATCH 00/13] object store: alloc Stefan Beller
2018-05-01 21:33 ` [PATCH 01/13] repository: introduce object parser field Stefan Beller
2018-05-02 17:17   ` Duy Nguyen
2018-05-02 17:26     ` Stefan Beller
2018-05-02 17:58       ` Duy Nguyen
2018-05-02 20:30   ` Jonathan Tan
2018-05-01 21:33 ` [PATCH 02/13] object: add repository argument to create_object Stefan Beller
2018-05-01 21:43   ` Eric Sunshine
2018-05-01 21:33 ` [PATCH 03/13] object: add repository argument to grow_object_hash Stefan Beller
2018-05-01 21:33 ` [PATCH 04/13] alloc: add repository argument to alloc_blob_node Stefan Beller
2018-05-02 20:34   ` Jonathan Tan
2018-05-01 21:33 ` [PATCH 05/13] alloc: add repository argument to alloc_tree_node Stefan Beller
2018-05-01 21:33 ` [PATCH 06/13] alloc: add repository argument to alloc_commit_node Stefan Beller
2018-05-01 21:33 ` [PATCH 07/13] alloc: add repository argument to alloc_tag_node Stefan Beller
2018-05-01 21:33 ` [PATCH 08/13] alloc: add repository argument to alloc_object_node Stefan Beller
2018-05-01 21:33 ` [PATCH 09/13] alloc: add repository argument to alloc_report Stefan Beller
2018-05-01 21:34 ` [PATCH 10/13] alloc: add repository argument to alloc_commit_index Stefan Beller
2018-05-01 21:34 ` [PATCH 11/13] object: allow grow_object_hash to handle arbitrary repositories Stefan Beller
2018-05-01 21:34 ` [PATCH 12/13] object: allow create_object " Stefan Beller
2018-05-02 20:36   ` Jonathan Tan
2018-05-01 21:34 ` [PATCH 13/13] alloc: allow arbitrary repositories for alloc functions Stefan Beller
2018-05-02 17:44   ` Duy Nguyen
2018-05-03 17:24     ` Stefan Beller
2018-05-03 17:35       ` Duy Nguyen
2018-05-02 20:50   ` Jonathan Tan
2018-05-03 17:25     ` Stefan Beller
2018-05-03 14:58   ` Duy Nguyen
2018-05-02 17:01 ` [PATCH 00/13] object store: alloc Duy Nguyen
2018-05-02 18:07   ` Jameson Miller
2018-05-02 18:22     ` Duy Nguyen
2018-05-02 18:44       ` Jameson Miller
2018-05-03 22:45       ` Stefan Beller
2018-05-07 14:05 ` Junio C Hamano
2018-05-07 20:53   ` Stefan Beller
2018-05-07 22:59 ` [PATCH v2 " Stefan Beller
2018-05-07 22:59   ` [PATCH v2 01/13] repository: introduce parsed objects field Stefan Beller
2018-05-08 17:23     ` Jonathan Tan
2018-05-07 22:59   ` [PATCH v2 02/13] object: add repository argument to create_object Stefan Beller
2018-05-07 22:59   ` [PATCH v2 03/13] object: add repository argument to grow_object_hash Stefan Beller
2018-05-07 22:59   ` [PATCH v2 04/13] alloc: add repository argument to alloc_blob_node Stefan Beller
2018-05-07 22:59   ` [PATCH v2 05/13] alloc: add repository argument to alloc_tree_node Stefan Beller
2018-05-07 22:59   ` [PATCH v2 06/13] alloc: add repository argument to alloc_commit_node Stefan Beller
2018-05-07 22:59   ` [PATCH v2 07/13] alloc: add repository argument to alloc_tag_node Stefan Beller
2018-05-07 22:59   ` [PATCH v2 08/13] alloc: add repository argument to alloc_object_node Stefan Beller
2018-05-07 22:59   ` [PATCH v2 09/13] alloc: add repository argument to alloc_report Stefan Beller
2018-05-07 22:59   ` [PATCH v2 10/13] alloc: add repository argument to alloc_commit_index Stefan Beller
2018-05-07 22:59   ` [PATCH v2 11/13] object: allow grow_object_hash to handle arbitrary repositories Stefan Beller
2018-05-07 22:59   ` [PATCH v2 12/13] object: allow create_object " Stefan Beller
2018-05-07 22:59   ` [PATCH v2 13/13] alloc: allow arbitrary repositories for alloc functions Stefan Beller
2018-05-08 10:10     ` Jeff King
2018-05-08 15:00     ` Duy Nguyen
2018-05-08 18:38       ` Stefan Beller [this message]
2018-05-08 17:45     ` Jonathan Tan
2018-05-08 19:37   ` [PATCH v3 00/13] object store: alloc Stefan Beller
2018-05-08 19:37     ` [PATCH v3 01/13] repository: introduce parsed objects field Stefan Beller
2018-05-08 19:37     ` [PATCH v3 02/13] object: add repository argument to create_object Stefan Beller
2018-05-08 19:37     ` [PATCH v3 03/13] object: add repository argument to grow_object_hash Stefan Beller
2018-05-08 19:37     ` [PATCH v3 04/13] alloc: add repository argument to alloc_blob_node Stefan Beller
2018-05-08 19:37     ` [PATCH v3 05/13] alloc: add repository argument to alloc_tree_node Stefan Beller
2018-05-08 19:37     ` [PATCH v3 06/13] alloc: add repository argument to alloc_commit_node Stefan Beller
2018-05-08 19:37     ` [PATCH v3 07/13] alloc: add repository argument to alloc_tag_node Stefan Beller
2018-05-08 19:37     ` [PATCH v3 08/13] alloc: add repository argument to alloc_object_node Stefan Beller
2018-05-08 19:37     ` [PATCH v3 09/13] alloc: add repository argument to alloc_report Stefan Beller
2018-05-08 19:37     ` [PATCH v3 10/13] alloc: add repository argument to alloc_commit_index Stefan Beller
2018-05-08 19:37     ` [PATCH v3 11/13] object: allow grow_object_hash to handle arbitrary repositories Stefan Beller
2018-05-08 19:37     ` [PATCH v3 12/13] object: allow create_object " Stefan Beller
2018-05-08 19:37     ` [PATCH v3 13/13] alloc: allow arbitrary repositories for alloc functions Stefan Beller
2018-05-08 20:04       ` Jonathan Tan
2018-05-08 20:37         ` Stefan Beller
2018-05-09 15:54           ` Duy Nguyen
2018-05-09 17:18         ` Duy Nguyen
2018-05-09 19:20           ` Stefan Beller
2018-05-10 15:43             ` Duy Nguyen
2018-05-10  0:40     ` [PATCH v4 00/13] object store: alloc Stefan Beller
2018-05-10  0:40       ` [PATCH v4 01/13] repository: introduce parsed objects field Stefan Beller
2018-05-10  0:40       ` [PATCH v4 02/13] object: add repository argument to create_object Stefan Beller
2018-05-10  0:40       ` [PATCH v4 03/13] object: add repository argument to grow_object_hash Stefan Beller
2018-05-10  0:40       ` [PATCH v4 04/13] alloc: add repository argument to alloc_blob_node Stefan Beller
2018-05-10  0:40       ` [PATCH v4 05/13] alloc: add repository argument to alloc_tree_node Stefan Beller
2018-05-10  0:40       ` [PATCH v4 06/13] alloc: add repository argument to alloc_commit_node Stefan Beller
2018-05-10  0:40       ` [PATCH v4 07/13] alloc: add repository argument to alloc_tag_node Stefan Beller
2018-05-10  0:40       ` [PATCH v4 08/13] alloc: add repository argument to alloc_object_node Stefan Beller
2018-05-10  0:40       ` [PATCH v4 09/13] alloc: add repository argument to alloc_report Stefan Beller
2018-05-10  0:40       ` [PATCH v4 10/13] alloc: add repository argument to alloc_commit_index Stefan Beller
2018-05-10  0:40       ` [PATCH v4 11/13] object: allow grow_object_hash to handle arbitrary repositories Stefan Beller
2018-05-10  0:40       ` [PATCH v4 12/13] object: allow create_object " Stefan Beller
2018-05-10  0:40       ` [PATCH v4 13/13] alloc: allow arbitrary repositories for alloc functions Stefan Beller
2018-05-10 17:16       ` [PATCH v4 00/13] object store: alloc Jonathan Tan
2018-05-10 17:32         ` Stefan Beller
2018-05-10 20:56           ` Jonathan Tan
2018-05-10 22:36             ` Stefan Beller
2018-05-11 19:17         ` [PATCH] alloc: allow arbitrary repositories for alloc functions Stefan Beller
2018-05-11 19:38           ` Eric Sunshine
2018-05-15 21:48             ` Stefan Beller
2018-05-16  2:27               ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAGZ79kZhAjArSksrWMzJc49BKu1HF-R6kBHEzkus-in4fbTZvQ@mail.gmail.com \
    --to=sbeller@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jamill@microsoft.com \
    --cc=jonathantanmy@google.com \
    --cc=pclouds@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).