From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_PASS, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by dcvr.yhbt.net (Postfix) with ESMTP id BB8D81F5AE for ; Thu, 23 Jul 2020 17:57:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730270AbgGWR5C (ORCPT ); Thu, 23 Jul 2020 13:57:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730255AbgGWR45 (ORCPT ); Thu, 23 Jul 2020 13:56:57 -0400 Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96EC8C0619DC for ; Thu, 23 Jul 2020 10:56:56 -0700 (PDT) Received: by mail-wm1-x342.google.com with SMTP id x5so5190870wmi.2 for ; Thu, 23 Jul 2020 10:56:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=tIb63pX0YeW2srRqGdL3BdyiWo8uWN8WA7i9PyUQcko=; b=ixe/m4dBeL+P7I1ufPyItsAIKzqaBl+/7knRwzxAANpYmNbEGuu05YIZ8FqAbUuS1V ig0B3riPIL3xSKpw4Fpjkfcbx0WgtjRwsPsXQ0l2t0B0ZZLpH5ht0mZvH7YyVDM66zbC Hvwpw3m3kP2ehVWi9cbQbojmhNaLz4/xec1i5/IkQsYduZ6H1jysZkBvrIcg0SnmQa6K /nwMLqAaVbx1JdDy+sk8ibMOhriDE2L67twvY6dOmurcVY1NNpdm9/bFOgAeEHu4+fTR p8vmXHOohDmYzRRWenPFDgsb9Sd6/q6bb3q/YfTYrRU96nx0f4PNrDsA8RJqjYow9ZY6 byjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=tIb63pX0YeW2srRqGdL3BdyiWo8uWN8WA7i9PyUQcko=; b=Prx+OddV6Mh2V98vIc2JE2QAbD0wg0cU1lu5kCan44A6x/PnchiAxWZnOyP8VczxrE 7+7oDFmswexCPvxm50F3qbu8kQQPhRx6kh1aTpznm4pOxzvWqTnoet9cozrn6CJg2QIc 10JeSS9+6Ua25CcqP/RRqNP5lMycSCpEUs7ubQi+9mMP9bV0eZX725ZFBrlLWcToPE27 +THDt/Wy4qPkb67V7foZo015TaDrWdUMW53we1FOq5IXk4jdShEq52lEjELGLn8h5Y3b vSGrqw7bLj68to3yPsK2Tg7P0h6Y6HDU5hTE6viF3rrFNcIbESssp+b0Kla8XW9EnjRK VcIQ== X-Gm-Message-State: AOAM533tiZbSmaepo+M48CTrMkqrDTMAEm46XuIDE/WXWtP3srAMr7C6 G94mfwsJgTgvjhGs+94Q3VNktNBF X-Google-Smtp-Source: ABdhPJyX3mAAY6Jy2ddnmXVeoxUOdQ/WYGc9qwGTQgxPHxQSEeQeYxSPNKC9A2tyRVPmKvIb1prGVA== X-Received: by 2002:a1c:9650:: with SMTP id y77mr5032849wmd.101.1595527015179; Thu, 23 Jul 2020 10:56:55 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g18sm4868495wru.27.2020.07.23.10.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2020 10:56:54 -0700 (PDT) Message-Id: <9af2309f0804cc3a2b26c3cc5c5526d51e86aac2.1595527000.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Date: Thu, 23 Jul 2020 17:56:36 +0000 Subject: [PATCH v2 14/18] maintenance: add auto condition for commit-graph task Fcc: Sent Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes.Schindelin@gmx.de, sandals@crustytoothpaste.net, steadmon@google.com, jrnieder@gmail.com, peff@peff.net, congdanhqx@gmail.com, phillip.wood123@gmail.com, emilyshaffer@google.com, sluongng@gmail.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee Instead of writing a new commit-graph in every 'git maintenance run --auto' process (when maintenance.commit-graph.enalbed is configured to be true), only write when there are "enough" commits not in a commit-graph file. This count is controlled by the maintenance.commit-graph.auto config option. To compute the count, use a depth-first search starting at each ref, and leaving markers using the PARENT1 flag. If this count reaches the limit, then terminate early and start the task. Otherwise, this operation will peel every ref and parse the commit it points to. If these are all in the commit-graph, then this is typically a very fast operation. Users with many refs might feel a slow-down, and hence could consider updating their limit to be very small. A negative value will force the step to run every time. Signed-off-by: Derrick Stolee --- Documentation/config/maintenance.txt | 10 ++++ builtin/gc.c | 76 ++++++++++++++++++++++++++++ object.h | 1 + 3 files changed, 87 insertions(+) diff --git a/Documentation/config/maintenance.txt b/Documentation/config/maintenance.txt index 370cbfb42f..9bd69b9df3 100644 --- a/Documentation/config/maintenance.txt +++ b/Documentation/config/maintenance.txt @@ -2,3 +2,13 @@ maintenance..enabled:: This boolean config option controls whether the maintenance task with name `` is run when no `--task` option is specified. By default, only `maintenance.gc.enabled` is true. + +maintenance.commit-graph.auto:: + This integer config option controls how often the `commit-graph` task + should be run as part of `git maintenance run --auto`. If zero, then + the `commit-graph` task will not run with the `--auto` option. A + negative value will force the task to run every time. Otherwise, a + positive value implies the command should run when the number of + reachable commits that are not in the commit-graph file is at least + the value of `maintenance.commit-graph.auto`. The default value is + 100. diff --git a/builtin/gc.c b/builtin/gc.c index 31696a2595..84ad360d17 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -30,6 +30,7 @@ #include "promisor-remote.h" #include "remote.h" #include "midx.h" +#include "refs.h" #define FAILED_RUN "failed to run %s" @@ -715,6 +716,80 @@ static struct maintenance_opts { int tasks_selected; } opts; +/* Remember to update object flag allocation in object.h */ +#define PARENT1 (1u<<16) + +static int num_commits_not_in_graph = 0; +static int limit_commits_not_in_graph = 100; + +static int dfs_on_ref(const char *refname, + const struct object_id *oid, int flags, + void *cb_data) +{ + int result = 0; + struct object_id peeled; + struct commit_list *stack = NULL; + struct commit *commit; + + if (!peel_ref(refname, &peeled)) + oid = &peeled; + if (oid_object_info(the_repository, oid, NULL) != OBJ_COMMIT) + return 0; + + commit = lookup_commit(the_repository, oid); + if (!commit) + return 0; + if (parse_commit(commit)) + return 0; + + commit_list_append(commit, &stack); + + while (!result && stack) { + struct commit_list *parent; + + commit = pop_commit(&stack); + + for (parent = commit->parents; parent; parent = parent->next) { + if (parse_commit(parent->item) || + commit_graph_position(parent->item) != COMMIT_NOT_FROM_GRAPH || + parent->item->object.flags & PARENT1) + continue; + + parent->item->object.flags |= PARENT1; + num_commits_not_in_graph++; + + if (num_commits_not_in_graph >= limit_commits_not_in_graph) { + result = 1; + break; + } + + commit_list_append(parent->item, &stack); + } + } + + free_commit_list(stack); + return result; +} + +static int should_write_commit_graph(void) +{ + int result; + + git_config_get_int("maintenance.commit-graph.auto", + &limit_commits_not_in_graph); + + if (!limit_commits_not_in_graph) + return 0; + if (limit_commits_not_in_graph < 0) + return 1; + + result = for_each_ref(dfs_on_ref, NULL); + + clear_commit_marks_all(PARENT1); + + return result; +} + static int run_write_commit_graph(void) { int result; @@ -1224,6 +1299,7 @@ static void initialize_tasks(void) tasks[num_tasks]->name = "commit-graph"; tasks[num_tasks]->fn = maintenance_task_commit_graph; + tasks[num_tasks]->auto_condition = should_write_commit_graph; num_tasks++; for (i = 0; i < num_tasks; i++) { diff --git a/object.h b/object.h index 38dc2d5a6c..4f886495d7 100644 --- a/object.h +++ b/object.h @@ -73,6 +73,7 @@ struct object_array { * list-objects-filter.c: 21 * builtin/fsck.c: 0--3 * builtin/index-pack.c: 2021 + * builtin/maintenance.c: 16 * builtin/pack-objects.c: 20 * builtin/reflog.c: 10--12 * builtin/show-branch.c: 0-------------------------------------------26 -- gitgitgadget