From: Junio C Hamano <gitster@pobox.com>
To: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
Cc: git@vger.kernel.org, sandals@crustytoothpaste.net,
avarab@gmail.com, peff@peff.net,
Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH v4 11/11] commit-graph: extract write_commit_graph_file()
Date: Mon, 13 May 2019 14:09:44 +0900 [thread overview]
Message-ID: <xmqqpnonc547.fsf@gitster-ct.c.googlers.com> (raw)
In-Reply-To: <fc81c8946dd53d926b03ac94e1d9e11b4f9b8fc3.1557411749.git.gitgitgadget@gmail.com> (Derrick Stolee via GitGitGadget's message of "Thu, 09 May 2019 07:22:40 -0700 (PDT)")
"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> From: Derrick Stolee <dstolee@microsoft.com>
>
> The write_commit_graph() method is too complex, so we are
> extracting methods one by one.
>
> Extract write_commit_graph_file() that takes all of the information
> in the context struct and writes the data to a commit-graph file.
The later parts of splitting pieces out of write_commit_graph() into
separate helper functions look all sensible. One big benefit of
doing this, even if each of these helper functions have a single
caller, is that each of these individual steps now has a descriptive
name.
Module a few nits (and possibly s/method/helper function/g), the
series look good to me.
Thanks.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> commit-graph.c | 155 +++++++++++++++++++++++++------------------------
> 1 file changed, 80 insertions(+), 75 deletions(-)
>
> diff --git a/commit-graph.c b/commit-graph.c
> index 16cdd7afb2..7723156964 100644
> --- a/commit-graph.c
> +++ b/commit-graph.c
> @@ -1015,21 +1015,91 @@ static void copy_oids_to_commits(struct write_commit_graph_context *ctx)
> stop_progress(&ctx->progress);
> }
>
> -int write_commit_graph(const char *obj_dir,
> - struct string_list *pack_indexes,
> - struct string_list *commit_hex,
> - unsigned int flags)
> +static int write_commit_graph_file(struct write_commit_graph_context *ctx)
> {
> - struct write_commit_graph_context *ctx;
> + uint32_t i;
> struct hashfile *f;
> - uint32_t i, count_distinct = 0;
> - char *graph_name = NULL;
> struct lock_file lk = LOCK_INIT;
> uint32_t chunk_ids[5];
> uint64_t chunk_offsets[5];
> - int num_chunks;
> const unsigned hashsz = the_hash_algo->rawsz;
> struct strbuf progress_title = STRBUF_INIT;
> + int num_chunks = ctx->num_extra_edges ? 4 : 3;
> +
> + ctx->graph_name = get_commit_graph_filename(ctx->obj_dir);
> + if (safe_create_leading_directories(ctx->graph_name)) {
> + UNLEAK(ctx->graph_name);
> + error(_("unable to create leading directories of %s"),
> + ctx->graph_name);
> + return errno;
> + }
> +
> + hold_lock_file_for_update(&lk, ctx->graph_name, LOCK_DIE_ON_ERROR);
> + f = hashfd(lk.tempfile->fd, lk.tempfile->filename.buf);
> +
> + hashwrite_be32(f, GRAPH_SIGNATURE);
> +
> + hashwrite_u8(f, GRAPH_VERSION);
> + hashwrite_u8(f, oid_version());
> + hashwrite_u8(f, num_chunks);
> + hashwrite_u8(f, 0); /* unused padding byte */
> +
> + chunk_ids[0] = GRAPH_CHUNKID_OIDFANOUT;
> + chunk_ids[1] = GRAPH_CHUNKID_OIDLOOKUP;
> + chunk_ids[2] = GRAPH_CHUNKID_DATA;
> + if (ctx->num_extra_edges)
> + chunk_ids[3] = GRAPH_CHUNKID_EXTRAEDGES;
> + else
> + chunk_ids[3] = 0;
> + chunk_ids[4] = 0;
> +
> + chunk_offsets[0] = 8 + (num_chunks + 1) * GRAPH_CHUNKLOOKUP_WIDTH;
> + chunk_offsets[1] = chunk_offsets[0] + GRAPH_FANOUT_SIZE;
> + chunk_offsets[2] = chunk_offsets[1] + hashsz * ctx->commits.nr;
> + chunk_offsets[3] = chunk_offsets[2] + (hashsz + 16) * ctx->commits.nr;
> + chunk_offsets[4] = chunk_offsets[3] + 4 * ctx->num_extra_edges;
> +
> + for (i = 0; i <= num_chunks; i++) {
> + uint32_t chunk_write[3];
> +
> + chunk_write[0] = htonl(chunk_ids[i]);
> + chunk_write[1] = htonl(chunk_offsets[i] >> 32);
> + chunk_write[2] = htonl(chunk_offsets[i] & 0xffffffff);
> + hashwrite(f, chunk_write, 12);
> + }
> +
> + if (ctx->report_progress) {
> + strbuf_addf(&progress_title,
> + Q_("Writing out commit graph in %d pass",
> + "Writing out commit graph in %d passes",
> + num_chunks),
> + num_chunks);
> + ctx->progress = start_delayed_progress(
> + progress_title.buf,
> + num_chunks * ctx->commits.nr);
> + }
> + write_graph_chunk_fanout(f, ctx);
> + write_graph_chunk_oids(f, hashsz, ctx);
> + write_graph_chunk_data(f, hashsz, ctx);
> + if (ctx->num_extra_edges)
> + write_graph_chunk_extra_edges(f, ctx);
> + stop_progress(&ctx->progress);
> + strbuf_release(&progress_title);
> +
> + close_commit_graph(ctx->r);
> + finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC);
> + commit_lock_file(&lk);
> +
> + return 0;
> +}
> +
> +int write_commit_graph(const char *obj_dir,
> + struct string_list *pack_indexes,
> + struct string_list *commit_hex,
> + unsigned int flags)
> +{
> + struct write_commit_graph_context *ctx;
> + uint32_t i, count_distinct = 0;
> int res = 0;
>
> if (!commit_graph_compatible(the_repository))
> @@ -1096,75 +1166,10 @@ int write_commit_graph(const char *obj_dir,
>
> compute_generation_numbers(ctx);
>
> - num_chunks = ctx->num_extra_edges ? 4 : 3;
> -
> - ctx->graph_name = get_commit_graph_filename(ctx->obj_dir);
> - if (safe_create_leading_directories(ctx->graph_name)) {
> - UNLEAK(ctx->graph_name);
> - error(_("unable to create leading directories of %s"),
> - ctx->graph_name);
> - res = errno;
> - goto cleanup;
> - }
> -
> - hold_lock_file_for_update(&lk, ctx->graph_name, LOCK_DIE_ON_ERROR);
> - f = hashfd(lk.tempfile->fd, lk.tempfile->filename.buf);
> -
> - hashwrite_be32(f, GRAPH_SIGNATURE);
> -
> - hashwrite_u8(f, GRAPH_VERSION);
> - hashwrite_u8(f, oid_version());
> - hashwrite_u8(f, num_chunks);
> - hashwrite_u8(f, 0); /* unused padding byte */
> -
> - chunk_ids[0] = GRAPH_CHUNKID_OIDFANOUT;
> - chunk_ids[1] = GRAPH_CHUNKID_OIDLOOKUP;
> - chunk_ids[2] = GRAPH_CHUNKID_DATA;
> - if (ctx->num_extra_edges)
> - chunk_ids[3] = GRAPH_CHUNKID_EXTRAEDGES;
> - else
> - chunk_ids[3] = 0;
> - chunk_ids[4] = 0;
> -
> - chunk_offsets[0] = 8 + (num_chunks + 1) * GRAPH_CHUNKLOOKUP_WIDTH;
> - chunk_offsets[1] = chunk_offsets[0] + GRAPH_FANOUT_SIZE;
> - chunk_offsets[2] = chunk_offsets[1] + hashsz * ctx->commits.nr;
> - chunk_offsets[3] = chunk_offsets[2] + (hashsz + 16) * ctx->commits.nr;
> - chunk_offsets[4] = chunk_offsets[3] + 4 * ctx->num_extra_edges;
> -
> - for (i = 0; i <= num_chunks; i++) {
> - uint32_t chunk_write[3];
> -
> - chunk_write[0] = htonl(chunk_ids[i]);
> - chunk_write[1] = htonl(chunk_offsets[i] >> 32);
> - chunk_write[2] = htonl(chunk_offsets[i] & 0xffffffff);
> - hashwrite(f, chunk_write, 12);
> - }
> -
> - if (ctx->report_progress) {
> - strbuf_addf(&progress_title,
> - Q_("Writing out commit graph in %d pass",
> - "Writing out commit graph in %d passes",
> - num_chunks),
> - num_chunks);
> - ctx->progress = start_delayed_progress(
> - progress_title.buf,
> - num_chunks * ctx->commits.nr);
> - }
> - write_graph_chunk_fanout(f, ctx);
> - write_graph_chunk_oids(f, hashsz, ctx);
> - write_graph_chunk_data(f, hashsz, ctx);
> - if (ctx->num_extra_edges)
> - write_graph_chunk_extra_edges(f, ctx);
> - stop_progress(&ctx->progress);
> - strbuf_release(&progress_title);
> -
> - close_commit_graph(ctx->r);
> - finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC);
> - commit_lock_file(&lk);
> + res = write_commit_graph_file(ctx);
>
> cleanup:
> - free(graph_name);
> + free(ctx->graph_name);
> free(ctx->commits.list);
> free(ctx->oids.list);
> free(ctx);
next prev parent reply other threads:[~2019-05-13 5:10 UTC|newest]
Thread overview: 89+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-23 21:59 [PATCH 0/6] Create commit-graph file format v2 Derrick Stolee via GitGitGadget
2019-01-23 21:59 ` [PATCH 1/6] commit-graph: return with errors during write Derrick Stolee via GitGitGadget
2019-01-23 21:59 ` [PATCH 2/6] commit-graph: collapse parameters into flags Derrick Stolee via GitGitGadget
2019-01-23 21:59 ` [PATCH 3/6] commit-graph: create new version flags Derrick Stolee via GitGitGadget
2019-01-23 21:59 ` [PATCH 4/6] commit-graph: add --version=<n> option Derrick Stolee via GitGitGadget
2019-01-24 9:31 ` Ævar Arnfjörð Bjarmason
2019-01-23 21:59 ` [PATCH 5/6] commit-graph: implement file format version 2 Derrick Stolee via GitGitGadget
2019-01-23 23:56 ` Jonathan Tan
2019-01-24 9:40 ` Ævar Arnfjörð Bjarmason
2019-01-24 14:34 ` Derrick Stolee
2019-03-21 9:21 ` Ævar Arnfjörð Bjarmason
2019-01-23 21:59 ` [PATCH 6/6] commit-graph: test verifying a corrupt v2 header Derrick Stolee via GitGitGadget
2019-01-23 23:59 ` Jonathan Tan
2019-01-24 23:05 ` [PATCH 0/6] Create commit-graph file format v2 Junio C Hamano
2019-01-24 23:39 ` Junio C Hamano
2019-01-25 13:54 ` Derrick Stolee
2019-04-24 19:58 ` [PATCH v2 0/5] " Derrick Stolee via GitGitGadget
2019-04-24 19:58 ` [PATCH v2 1/5] commit-graph: return with errors during write Derrick Stolee via GitGitGadget
2019-04-24 19:58 ` [PATCH v2 2/5] commit-graph: collapse parameters into flags Derrick Stolee via GitGitGadget
2019-04-25 5:21 ` Junio C Hamano
2019-04-24 19:58 ` [PATCH v2 3/5] commit-graph: create new version flags Derrick Stolee via GitGitGadget
2019-04-25 5:29 ` Junio C Hamano
2019-04-25 11:09 ` Derrick Stolee
2019-04-25 21:31 ` Ævar Arnfjörð Bjarmason
2019-04-26 2:20 ` Junio C Hamano
2019-04-24 19:58 ` [PATCH v2 4/5] commit-graph: add --version=<n> option Derrick Stolee via GitGitGadget
2019-04-24 19:58 ` [PATCH v2 5/5] commit-graph: implement file format version 2 Derrick Stolee via GitGitGadget
2019-04-25 22:09 ` [PATCH v2 0/5] Create commit-graph file format v2 Ævar Arnfjörð Bjarmason
2019-04-26 2:28 ` Junio C Hamano
2019-04-26 8:33 ` Ævar Arnfjörð Bjarmason
2019-04-26 12:06 ` Derrick Stolee
2019-04-26 13:55 ` Ævar Arnfjörð Bjarmason
2019-04-27 12:57 ` Ævar Arnfjörð Bjarmason
2019-05-01 13:11 ` [PATCH v3 0/6] " Derrick Stolee via GitGitGadget
2019-05-01 13:11 ` [PATCH v3 1/6] commit-graph: return with errors during write Derrick Stolee via GitGitGadget
2019-05-01 14:46 ` Ævar Arnfjörð Bjarmason
2019-05-01 13:11 ` [PATCH v3 2/6] commit-graph: collapse parameters into flags Derrick Stolee via GitGitGadget
2019-05-01 13:11 ` [PATCH v3 3/6] commit-graph: create new version parameter Derrick Stolee via GitGitGadget
2019-05-01 13:11 ` [PATCH v3 4/6] commit-graph: add --version=<n> option Derrick Stolee via GitGitGadget
2019-05-01 13:11 ` [PATCH v3 5/6] commit-graph: implement file format version 2 Derrick Stolee via GitGitGadget
2019-05-01 19:12 ` Ævar Arnfjörð Bjarmason
2019-05-01 19:56 ` Derrick Stolee
2019-05-01 13:11 ` [PATCH v3 6/6] commit-graph: remove Future Work section Derrick Stolee via GitGitGadget
2019-05-01 14:58 ` Ævar Arnfjörð Bjarmason
2019-05-01 19:59 ` Derrick Stolee
2019-05-01 20:25 ` [PATCH v3 0/6] Create commit-graph file format v2 Ævar Arnfjörð Bjarmason
2019-05-02 13:26 ` Derrick Stolee
2019-05-02 18:02 ` Ævar Arnfjörð Bjarmason
2019-05-03 12:47 ` Derrick Stolee
2019-05-03 13:41 ` Ævar Arnfjörð Bjarmason
2019-05-06 8:27 ` Christian Couder
2019-05-06 13:47 ` Derrick Stolee
2019-05-03 14:16 ` SZEDER Gábor
2019-05-03 15:11 ` Derrick Stolee
2019-05-09 14:22 ` [PATCH v4 00/11] Commit-graph write refactor (was: Create commit-graph file format v2) Derrick Stolee via GitGitGadget
2019-05-09 14:22 ` [PATCH v4 01/11] commit-graph: fix the_repository reference Derrick Stolee via GitGitGadget
2019-05-13 2:56 ` Junio C Hamano
2019-05-09 14:22 ` [PATCH v4 02/11] commit-graph: return with errors during write Derrick Stolee via GitGitGadget
2019-05-13 3:13 ` Junio C Hamano
2019-05-13 11:04 ` Derrick Stolee
2019-05-13 11:22 ` Derrick Stolee
2019-05-09 14:22 ` [PATCH v4 03/11] commit-graph: collapse parameters into flags Derrick Stolee via GitGitGadget
2019-05-13 3:44 ` Junio C Hamano
2019-05-13 11:07 ` Derrick Stolee
2019-05-09 14:22 ` [PATCH v4 04/11] commit-graph: remove Future Work section Derrick Stolee via GitGitGadget
2019-05-09 14:22 ` [PATCH v4 05/11] commit-graph: create write_commit_graph_context Derrick Stolee via GitGitGadget
2019-05-09 14:22 ` [PATCH v4 07/11] commit-graph: extract fill_oids_from_commit_hex() Derrick Stolee via GitGitGadget
2019-05-09 14:22 ` [PATCH v4 06/11] commit-graph: extract fill_oids_from_packs() Derrick Stolee via GitGitGadget
2019-05-13 5:05 ` Junio C Hamano
2019-05-09 14:22 ` [PATCH v4 08/11] commit-graph: extract fill_oids_from_all_packs() Derrick Stolee via GitGitGadget
2019-05-09 14:22 ` [PATCH v4 09/11] commit-graph: extract count_distinct_commits() Derrick Stolee via GitGitGadget
2019-05-09 14:22 ` [PATCH v4 10/11] commit-graph: extract copy_oids_to_commits() Derrick Stolee via GitGitGadget
2019-05-09 14:22 ` [PATCH v4 11/11] commit-graph: extract write_commit_graph_file() Derrick Stolee via GitGitGadget
2019-05-13 5:09 ` Junio C Hamano [this message]
2019-05-09 17:58 ` [PATCH v4 00/11] Commit-graph write refactor (was: Create commit-graph file format v2) Josh Steadmon
2019-06-12 13:29 ` [PATCH v5 " Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 01/11] commit-graph: fix the_repository reference Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 02/11] commit-graph: return with errors during write Derrick Stolee via GitGitGadget
2019-06-29 17:23 ` SZEDER Gábor
2019-07-01 12:19 ` Derrick Stolee
2019-06-12 13:29 ` [PATCH v5 03/11] commit-graph: collapse parameters into flags Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 04/11] commit-graph: remove Future Work section Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 05/11] commit-graph: create write_commit_graph_context Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 06/11] commit-graph: extract fill_oids_from_packs() Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 08/11] commit-graph: extract fill_oids_from_all_packs() Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 07/11] commit-graph: extract fill_oids_from_commit_hex() Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 09/11] commit-graph: extract count_distinct_commits() Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 10/11] commit-graph: extract copy_oids_to_commits() Derrick Stolee via GitGitGadget
2019-06-12 13:29 ` [PATCH v5 11/11] commit-graph: extract write_commit_graph_file() Derrick Stolee via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqpnonc547.fsf@gitster-ct.c.googlers.com \
--to=gitster@pobox.com \
--cc=avarab@gmail.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitgitgadget@gmail.com \
--cc=peff@peff.net \
--cc=sandals@crustytoothpaste.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).