From: Ramsay Jones <ramsay@ramsayjones.plus.com>
To: Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>,
git@vger.kernel.org
Cc: peff@peff.net, avarab@gmail.com, git@jeffhostetler.com,
jrnieder@google.com, steadmon@google.com,
johannes.schindelin@gmx.de, Junio C Hamano <gitster@pobox.com>,
Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH v4 12/14] commit-graph: create options for split files
Date: Thu, 6 Jun 2019 19:41:46 +0100 [thread overview]
Message-ID: <02c9a0e2-4505-14a4-fa68-958a7f0c7935@ramsayjones.plus.com> (raw)
In-Reply-To: <ac5586a20f7ed600f8912b1aed07dea5098ea046.1559830527.git.gitgitgadget@gmail.com>
On 06/06/2019 15:15, Derrick Stolee via GitGitGadget wrote:
> From: Derrick Stolee <dstolee@microsoft.com>
>
> The split commit-graph feature is now fully implemented, but needs
> some more run-time configurability. Allow direct callers to 'git
> commit-graph write --split' to specify the values used in the
> merge strategy and the expire time.
>
> Update the documentation to specify these values.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> Documentation/git-commit-graph.txt | 21 +++++++++++++-
> Documentation/technical/commit-graph.txt | 7 +++--
> builtin/commit-graph.c | 20 +++++++++++---
> builtin/commit.c | 2 +-
> builtin/gc.c | 3 +-
> commit-graph.c | 35 ++++++++++++++++--------
> commit-graph.h | 12 ++++++--
> t/t5323-split-commit-graph.sh | 35 ++++++++++++++++++++++++
> 8 files changed, 112 insertions(+), 23 deletions(-)
>
> diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt
> index 624470e198..365e145e82 100644
> --- a/Documentation/git-commit-graph.txt
> +++ b/Documentation/git-commit-graph.txt
> @@ -26,7 +26,7 @@ OPTIONS
> Use given directory for the location of packfiles and commit-graph
> file. This parameter exists to specify the location of an alternate
> that only has the objects directory, not a full `.git` directory. The
> - commit-graph file is expected to be at `<dir>/info/commit-graph` and
> + commit-graph file is expected to be in the `<dir>/info` directory and
> the packfiles are expected to be in `<dir>/pack`.
>
>
> @@ -51,6 +51,25 @@ or `--stdin-packs`.)
> +
> With the `--append` option, include all commits that are present in the
> existing commit-graph file.
> ++
> +With the `--split` option, write the commit-graph as a chain of multiple
> +commit-graph files stored in `<dir>/info/commit-graphs`. The new commits
> +not already in the commit-graph are added in a new "tip" file. This file
> +is merged with the existing file if the following merge conditions are
> +met:
> ++
> +* If `--size-multiple=<X>` is not specified, let `X` equal 2. If the new
> +tip file would have `N` commits and the previous tip has `M` commits and
> +`X` times `N` is greater than `M`, instead merge the two files into a
> +single file.
> ++
> +* If `--max-commits=<M>` is specified with `M` a positive integer, and the
> +new tip file would have more than `M` commits, then instead merge the new
> +tip with the previous tip.
> ++
> +Finally, if `--expire-time=<datetime>` is not specified, let `datetime`
> +be the current time. After writing the split commit-graph, delete all
> +unused commit-graph whose modified times are older than `datetime`.
>
> 'read'::
>
> diff --git a/Documentation/technical/commit-graph.txt b/Documentation/technical/commit-graph.txt
> index aed4350a59..729fbcb32f 100644
> --- a/Documentation/technical/commit-graph.txt
> +++ b/Documentation/technical/commit-graph.txt
> @@ -248,10 +248,11 @@ When writing a set of commits that do not exist in the commit-graph stack of
> height N, we default to creating a new file at level N + 1. We then decide to
> merge with the Nth level if one of two conditions hold:
>
> - 1. The expected file size for level N + 1 is at least half the file size for
> - level N.
> + 1. `--size-multiple=<X>` is specified or X = 2, and the number of commits in
> + level N is less than X times the number of commits in level N + 1.
>
> - 2. Level N + 1 contains more than 64,0000 commits.
> + 2. `--max-commits=<C>` is specified with non-zero C and the number of commits
> + in level N + 1 is more than C commits.
>
> This decision cascades down the levels: when we merge a level we create a new
> set of commits that then compares to the next level.
> diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
> index c2c07d3917..18e3b61fb6 100644
> --- a/builtin/commit-graph.c
> +++ b/builtin/commit-graph.c
> @@ -10,7 +10,7 @@ static char const * const builtin_commit_graph_usage[] = {
> N_("git commit-graph [--object-dir <objdir>]"),
> N_("git commit-graph read [--object-dir <objdir>]"),
> N_("git commit-graph verify [--object-dir <objdir>]"),
> - N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits]"),
> + N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits] <split options>"),
> NULL
> };
>
> @@ -25,7 +25,7 @@ static const char * const builtin_commit_graph_read_usage[] = {
> };
>
> static const char * const builtin_commit_graph_write_usage[] = {
> - N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits]"),
> + N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits] <split options>"),
> NULL
> };
>
> @@ -135,6 +135,7 @@ static int graph_read(int argc, const char **argv)
> }
>
> extern int read_replace_refs;
> +struct split_commit_graph_opts split_opts;
This 'split_opts' variable needs to be marked 'static'.
Thanks.
ATB,
Ramsay Jones
next prev parent reply other threads:[~2019-06-06 18:42 UTC|newest]
Thread overview: 136+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-08 15:53 [PATCH 00/17] [RFC] Commit-graph: Write incremental files Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 01/17] commit-graph: fix the_repository reference Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 02/17] commit-graph: return with errors during write Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 03/17] commit-graph: collapse parameters into flags Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 04/17] commit-graph: remove Future Work section Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 05/17] commit-graph: create write_commit_graph_context Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 06/17] commit-graph: extract fill_oids_from_packs() Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 07/17] commit-graph: extract fill_oids_from_commit_hex() Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 08/17] commit-graph: extract fill_oids_from_all_packs() Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 10/17] commit-graph: extract copy_oids_to_commits() Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 09/17] commit-graph: extract count_distinct_commits() Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 11/17] commit-graph: extract write_commit_graph_file() Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 12/17] Documentation: describe split commit-graphs Derrick Stolee via GitGitGadget
2019-05-08 17:20 ` SZEDER Gábor
2019-05-08 19:00 ` Derrick Stolee
2019-05-08 20:11 ` Ævar Arnfjörð Bjarmason
2019-05-09 4:49 ` Junio C Hamano
2019-05-09 12:25 ` Derrick Stolee
2019-05-09 13:45 ` Derrick Stolee
2019-05-09 15:48 ` Ævar Arnfjörð Bjarmason
2019-05-09 17:08 ` Derrick Stolee
2019-05-09 21:45 ` Ævar Arnfjörð Bjarmason
2019-05-10 12:44 ` Derrick Stolee
2019-05-08 15:53 ` [PATCH 13/17] commit-graph: lay groundwork for incremental files Derrick Stolee via GitGitGadget
2019-05-08 15:53 ` [PATCH 14/17] commit-graph: load split commit-graph files Derrick Stolee via GitGitGadget
2019-05-08 15:54 ` [PATCH 15/17] commit-graph: write " Derrick Stolee via GitGitGadget
2019-05-08 15:54 ` [PATCH 16/17] commit-graph: add --split option Derrick Stolee via GitGitGadget
2019-05-08 15:54 ` [PATCH 17/17] fetch: add fetch.writeCommitGraph config setting Derrick Stolee via GitGitGadget
2019-05-09 8:07 ` Ævar Arnfjörð Bjarmason
2019-05-09 14:21 ` Derrick Stolee
2019-05-08 19:27 ` [PATCH 00/17] [RFC] Commit-graph: Write incremental files Ævar Arnfjörð Bjarmason
2019-05-22 19:53 ` [PATCH v2 00/11] " Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 01/11] commit-graph: document commit-graph chains Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 02/11] commit-graph: prepare for " Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 03/11] commit-graph: rename commit_compare to oid_compare Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 05/11] commit-graph: add base graphs chunk Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 04/11] commit-graph: load commit-graph chains Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 06/11] commit-graph: rearrange chunk count logic Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 07/11] commit-graph: write commit-graph chains Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 08/11] commit-graph: add --split option to builtin Derrick Stolee via GitGitGadget
2019-05-27 11:28 ` SZEDER Gábor
2019-05-22 19:53 ` [PATCH v2 09/11] commit-graph: merge commit-graph chains Derrick Stolee via GitGitGadget
2019-05-23 0:43 ` Ævar Arnfjörð Bjarmason
2019-05-23 13:00 ` Derrick Stolee
2019-05-22 19:53 ` [PATCH v2 10/11] commit-graph: allow cross-alternate chains Derrick Stolee via GitGitGadget
2019-05-22 19:53 ` [PATCH v2 11/11] commit-graph: expire commit-graph files Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 00/14] Commit-graph: Write incremental files Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 01/14] commit-graph: document commit-graph chains Derrick Stolee via GitGitGadget
2019-06-05 17:22 ` Junio C Hamano
2019-06-05 18:09 ` Derrick Stolee
2019-06-06 12:10 ` Philip Oakley
2019-06-06 17:09 ` Derrick Stolee
2019-06-06 21:59 ` Philip Oakley
2019-06-03 16:03 ` [PATCH v3 02/14] commit-graph: prepare for " Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 03/14] commit-graph: rename commit_compare to oid_compare Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 04/14] commit-graph: load commit-graph chains Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 06/14] commit-graph: rearrange chunk count logic Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 05/14] commit-graph: add base graphs chunk Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 07/14] commit-graph: write commit-graph chains Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 08/14] commit-graph: add --split option to builtin Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 09/14] commit-graph: merge commit-graph chains Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 10/14] commit-graph: allow cross-alternate chains Derrick Stolee via GitGitGadget
2019-06-03 16:03 ` [PATCH v3 11/14] commit-graph: expire commit-graph files Derrick Stolee via GitGitGadget
2019-06-03 16:04 ` [PATCH v3 12/14] commit-graph: create options for split files Derrick Stolee via GitGitGadget
2019-06-03 16:04 ` [PATCH v3 13/14] commit-graph: verify chains with --shallow mode Derrick Stolee via GitGitGadget
2019-06-03 16:04 ` [PATCH v3 14/14] commit-graph: clean up chains after flattened write Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 00/14] Commit-graph: Write incremental files Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 01/14] commit-graph: document commit-graph chains Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 02/14] commit-graph: prepare for " Derrick Stolee via GitGitGadget
2019-06-06 15:19 ` Philip Oakley
2019-06-06 21:28 ` Junio C Hamano
2019-06-07 12:44 ` Derrick Stolee
2019-06-06 14:15 ` [PATCH v4 03/14] commit-graph: rename commit_compare to oid_compare Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 04/14] commit-graph: load commit-graph chains Derrick Stolee via GitGitGadget
2019-06-06 22:20 ` Junio C Hamano
2019-06-07 12:53 ` Derrick Stolee
2019-06-06 14:15 ` [PATCH v4 05/14] commit-graph: add base graphs chunk Derrick Stolee via GitGitGadget
2019-06-07 18:15 ` Junio C Hamano
2019-06-06 14:15 ` [PATCH v4 06/14] commit-graph: rearrange chunk count logic Derrick Stolee via GitGitGadget
2019-06-07 18:23 ` Junio C Hamano
2019-06-06 14:15 ` [PATCH v4 08/14] commit-graph: add --split option to builtin Derrick Stolee via GitGitGadget
2019-06-07 21:57 ` Junio C Hamano
2019-06-11 12:51 ` Derrick Stolee
2019-06-11 19:45 ` Junio C Hamano
2019-06-06 14:15 ` [PATCH v4 07/14] commit-graph: write commit-graph chains Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 09/14] commit-graph: merge " Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 10/14] commit-graph: allow cross-alternate chains Derrick Stolee via GitGitGadget
2019-06-06 17:00 ` Philip Oakley
2019-06-06 14:15 ` [PATCH v4 11/14] commit-graph: expire commit-graph files Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 12/14] commit-graph: create options for split files Derrick Stolee via GitGitGadget
2019-06-06 18:41 ` Ramsay Jones [this message]
2019-06-06 14:15 ` [PATCH v4 13/14] commit-graph: verify chains with --shallow mode Derrick Stolee via GitGitGadget
2019-06-06 14:15 ` [PATCH v4 14/14] commit-graph: clean up chains after flattened write Derrick Stolee via GitGitGadget
2019-06-06 16:57 ` [PATCH v4 00/14] Commit-graph: Write incremental files Junio C Hamano
2019-06-07 12:37 ` Derrick Stolee
2019-06-07 18:38 ` [PATCH v5 00/16] " Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 01/16] commit-graph: document commit-graph chains Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 02/16] commit-graph: prepare for " Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 03/16] commit-graph: rename commit_compare to oid_compare Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 04/16] commit-graph: load commit-graph chains Derrick Stolee via GitGitGadget
2019-06-10 21:47 ` Junio C Hamano
2019-06-10 23:41 ` Derrick Stolee
2019-06-07 18:38 ` [PATCH v5 05/16] commit-graph: add base graphs chunk Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 07/16] commit-graph: write commit-graph chains Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 06/16] commit-graph: rearrange chunk count logic Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 08/16] commit-graph: add --split option to builtin Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 09/16] commit-graph: merge commit-graph chains Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 10/16] commit-graph: allow cross-alternate chains Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 11/16] commit-graph: expire commit-graph files Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 13/16] commit-graph: verify chains with --shallow mode Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 12/16] commit-graph: create options for split files Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 14/16] commit-graph: clean up chains after flattened write Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 15/16] commit-graph: test octopus merges with --split Derrick Stolee via GitGitGadget
2019-06-07 18:38 ` [PATCH v5 16/16] commit-graph: test --split across alternate without --split Derrick Stolee via GitGitGadget
2019-06-17 15:02 ` [PATCH] commit-graph: normalize commit-graph filenames Derrick Stolee
2019-06-17 15:07 ` Derrick Stolee
2019-06-17 18:07 ` [PATCH v2] " Derrick Stolee
2019-06-18 18:14 ` [PATCH v6 00/18] Commit-graph: Write incremental files Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 01/18] commit-graph: document commit-graph chains Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 03/18] commit-graph: rename commit_compare to oid_compare Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 02/18] commit-graph: prepare for commit-graph chains Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 04/18] commit-graph: load " Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 05/18] commit-graph: add base graphs chunk Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 06/18] commit-graph: rearrange chunk count logic Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 07/18] commit-graph: write commit-graph chains Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 08/18] commit-graph: add --split option to builtin Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 09/18] commit-graph: merge commit-graph chains Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 10/18] commit-graph: allow cross-alternate chains Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 11/18] commit-graph: expire commit-graph files Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 13/18] commit-graph: verify chains with --shallow mode Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 12/18] commit-graph: create options for split files Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 14/18] commit-graph: clean up chains after flattened write Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 15/18] commit-graph: test octopus merges with --split Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 16/18] commit-graph: test --split across alternate without --split Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 18/18] commit-graph: test verify across alternates Derrick Stolee via GitGitGadget
2019-06-18 18:14 ` [PATCH v6 17/18] commit-graph: normalize commit-graph filenames Derrick Stolee via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=02c9a0e2-4505-14a4-fa68-958a7f0c7935@ramsayjones.plus.com \
--to=ramsay@ramsayjones.plus.com \
--cc=avarab@gmail.com \
--cc=dstolee@microsoft.com \
--cc=git@jeffhostetler.com \
--cc=git@vger.kernel.org \
--cc=gitgitgadget@gmail.com \
--cc=gitster@pobox.com \
--cc=johannes.schindelin@gmx.de \
--cc=jrnieder@google.com \
--cc=peff@peff.net \
--cc=steadmon@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).