From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_PASS, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by dcvr.yhbt.net (Postfix) with ESMTP id 61C411F66E for ; Tue, 25 Aug 2020 18:34:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726706AbgHYSeY (ORCPT ); Tue, 25 Aug 2020 14:34:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726466AbgHYSeA (ORCPT ); Tue, 25 Aug 2020 14:34:00 -0400 Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 562FDC061757 for ; Tue, 25 Aug 2020 11:33:59 -0700 (PDT) Received: by mail-wm1-x342.google.com with SMTP id 83so3568044wme.4 for ; Tue, 25 Aug 2020 11:33:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=2uAnxf2B5tiekxp+tVpWDPB8CQkwx46JIztlNDXM5IM=; b=c2BmGxBrYPp4lYJz1u8zrtxuq21ZRB6CWcEM0h7K5zEyqQ3GIv7uY5eGUdaecn7lqo JdOFxC1e2f4PyZ9TnBPS4L5LEMmzYFiiX8ezgefPNIW9GV+k/u+aMKNXrLLB8G+ai2+4 sZG2O7/QjlWewj44ycM2g6uExGyrbH+kKVUy29inc/JYJ6XF3uKeC9TLcNNIcV2PCngK M/f8P7zxYria8ApHhQ342YZng63WjD4kVS6J7/+/lNtAXco1u8vb7EUpCXTeA6kvkSqp XFDjouDhnESjB9omtIn9+K1BxQMj6wSDfg5FT4eYLUnByqaM4Pm2ANAtLVh6kV1HVeu4 WQJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=2uAnxf2B5tiekxp+tVpWDPB8CQkwx46JIztlNDXM5IM=; b=L6RnkRg+9zba50cjwearnfnW7cR53LyY8Gcc+BIdlcrKeujDfhErlPL9LMqowRBRMc 7dGvd/JR3kNk6NmFkSwIwjTMSEbevzrQ2LK00cPmBJa8tlRYcei6g+bQGPEGG8DBzBZl xIRWfhxZ1XHcQNZYqP5K4SHy96Z04r6q6y+tAt/SXKjpi/urCk6FKI7MK3TCneoTk/kc ttoYInuPJwUiGpKjeNhcaChCx7RkLqPAEURne/KEmtmkS5ktkAyUtVRW4YxlXLlhHH08 0JHM1dxwreG+6dLPJXQTNztaWLImwB9kluh2jXY4DF3BQVG2LWIXkk6jz2DyE9R/q4tf g2JA== X-Gm-Message-State: AOAM533EoFC7dgiAyu2LEddmT8bUgwCDS0cFuPXCKSryS4J8LHkT/Zfa LQKrDtSncbftAaEe4L0Lv0APFpDWHm8= X-Google-Smtp-Source: ABdhPJy/fYfMpploBYXiyo7c0HIr5sTNuOHiFF53d9oArFrnxbqd33i5VlhcXSEM7+60EBIuLLN6KQ== X-Received: by 2002:a1c:a585:: with SMTP id o127mr3399226wme.102.1598380437826; Tue, 25 Aug 2020 11:33:57 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f6sm10785358wme.32.2020.08.25.11.33.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 11:33:57 -0700 (PDT) Message-Id: <4c3115fe3522bee47ba1f8f5e847e99ad7e56d40.1598380427.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Date: Tue, 25 Aug 2020 18:33:45 +0000 Subject: [PATCH v3 10/11] maintenance: add auto condition for commit-graph task Fcc: Sent Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 To: git@vger.kernel.org Cc: sandals@crustytoothpaste.net, steadmon@google.com, jrnieder@gmail.com, peff@peff.net, congdanhqx@gmail.com, phillip.wood123@gmail.com, emilyshaffer@google.com, sluongng@gmail.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee Instead of writing a new commit-graph in every 'git maintenance run --auto' process (when maintenance.commit-graph.enalbed is configured to be true), only write when there are "enough" commits not in a commit-graph file. This count is controlled by the maintenance.commit-graph.auto config option. To compute the count, use a depth-first search starting at each ref, and leaving markers using the PARENT1 flag. If this count reaches the limit, then terminate early and start the task. Otherwise, this operation will peel every ref and parse the commit it points to. If these are all in the commit-graph, then this is typically a very fast operation. Users with many refs might feel a slow-down, and hence could consider updating their limit to be very small. A negative value will force the step to run every time. Signed-off-by: Derrick Stolee --- Documentation/config/maintenance.txt | 10 ++++ builtin/gc.c | 82 ++++++++++++++++++++++++++++ object.h | 1 + 3 files changed, 93 insertions(+) diff --git a/Documentation/config/maintenance.txt b/Documentation/config/maintenance.txt index 4402b8b49f..7cc6700d57 100644 --- a/Documentation/config/maintenance.txt +++ b/Documentation/config/maintenance.txt @@ -4,3 +4,13 @@ maintenance..enabled:: `git maintenance run`. These config values are ignored if a `--task` option exists. By default, only `maintenance.gc.enabled` is true. + +maintenance.commit-graph.auto:: + This integer config option controls how often the `commit-graph` task + should be run as part of `git maintenance run --auto`. If zero, then + the `commit-graph` task will not run with the `--auto` option. A + negative value will force the task to run every time. Otherwise, a + positive value implies the command should run when the number of + reachable commits that are not in the commit-graph file is at least + the value of `maintenance.commit-graph.auto`. The default value is + 100. diff --git a/builtin/gc.c b/builtin/gc.c index 709d13553b..8c4edf19ba 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -28,6 +28,7 @@ #include "blob.h" #include "tree.h" #include "promisor-remote.h" +#include "refs.h" #define FAILED_RUN "failed to run %s" @@ -710,6 +711,86 @@ struct maintenance_run_opts { int quiet; }; +/* Remember to update object flag allocation in object.h */ +#define SEEN (1u<<0) + +struct cg_auto_data { + int num_not_in_graph; + int limit; +}; + +static int dfs_on_ref(const char *refname, + const struct object_id *oid, int flags, + void *cb_data) +{ + struct cg_auto_data *data = (struct cg_auto_data *)cb_data; + int result = 0; + struct object_id peeled; + struct commit_list *stack = NULL; + struct commit *commit; + + if (!peel_ref(refname, &peeled)) + oid = &peeled; + if (oid_object_info(the_repository, oid, NULL) != OBJ_COMMIT) + return 0; + + commit = lookup_commit(the_repository, oid); + if (!commit) + return 0; + if (parse_commit(commit)) + return 0; + + commit_list_append(commit, &stack); + + while (!result && stack) { + struct commit_list *parent; + + commit = pop_commit(&stack); + + for (parent = commit->parents; parent; parent = parent->next) { + if (parse_commit(parent->item) || + commit_graph_position(parent->item) != COMMIT_NOT_FROM_GRAPH || + parent->item->object.flags & SEEN) + continue; + + parent->item->object.flags |= SEEN; + data->num_not_in_graph++; + + if (data->num_not_in_graph >= data->limit) { + result = 1; + break; + } + + commit_list_append(parent->item, &stack); + } + } + + free_commit_list(stack); + return result; +} + +static int should_write_commit_graph(void) +{ + int result; + struct cg_auto_data data; + + data.num_not_in_graph = 0; + data.limit = 100; + git_config_get_int("maintenance.commit-graph.auto", + &data.limit); + + if (!data.limit) + return 0; + if (data.limit < 0) + return 1; + + result = for_each_ref(dfs_on_ref, &data); + + clear_commit_marks_all(SEEN); + + return result; +} + static int run_write_commit_graph(struct maintenance_run_opts *opts) { struct child_process child = CHILD_PROCESS_INIT; @@ -790,6 +871,7 @@ static struct maintenance_task tasks[] = { [TASK_COMMIT_GRAPH] = { "commit-graph", maintenance_task_commit_graph, + should_write_commit_graph, }, }; diff --git a/object.h b/object.h index 96a2105859..20b18805f0 100644 --- a/object.h +++ b/object.h @@ -73,6 +73,7 @@ struct object_array { * sha1-name.c: 20 * list-objects-filter.c: 21 * builtin/fsck.c: 0--3 + * builtin/gc.c: 0 * builtin/index-pack.c: 2021 * builtin/pack-objects.c: 20 * builtin/reflog.c: 10--12 -- gitgitgadget