From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-2.8 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,T_RP_MATCHES_RCVD shortcircuit=no autolearn=no autolearn_force=no version=3.4.0 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by dcvr.yhbt.net (Postfix) with ESMTP id C62571F406 for ; Wed, 3 Jan 2018 16:36:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751262AbeACQem (ORCPT ); Wed, 3 Jan 2018 11:34:42 -0500 Received: from mail-wr0-f194.google.com ([209.85.128.194]:41566 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750831AbeACQei (ORCPT ); Wed, 3 Jan 2018 11:34:38 -0500 Received: by mail-wr0-f194.google.com with SMTP id p69so2141381wrb.8 for ; Wed, 03 Jan 2018 08:34:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wP3U1xM93sY/U+aBkYgb2muCBEKDN5f7mcuvwEvYA50=; b=Af1YJMXfvchi13kWUyV/pvz0stsVJHvapipHAjCfBO2y8hdB5p8P2Z6XZep/GSIWVT G3v2YIQriJHLQ/AmvMC2TW9NlBV16bAQSD+N6cn2l5w+m1c70Ssoogcq4AYPVQzp+61D KUDaZoSGTq/qHzMLf1ArjMb4J2x90XsQIFWkwa03/TJUHHHJs3nZGdlKeoFaNiPZ0nC0 tQxgD+XxRGVpKhaVbK4nhCoz7Ci/hguJkphwO0rqEIUruQr2QpGQI4EFLJfPTZF31Gt3 NeQM0BiWAgc+ViFkGS5YX8m8xCVbuZIGBuoyO/gqTzWPpK4ibf+bY8nk0DDXvnHzlQib FIKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wP3U1xM93sY/U+aBkYgb2muCBEKDN5f7mcuvwEvYA50=; b=cRCrvFoLN+42zzecu3oCTyY1xI06YSdyTixnESzkCSAkaZCNIJZbGHga4ba4i2vyde 9rRm5T4znN1ytPAnt38Q+5O3oSnROPADVsHZfoBhovO8guBP3wsgrS72L0Pq9WdwsY9X DCGT0k7d190idTEVY6MsY/cFqmzaLpCfXckg13yoYaDH6oiiZh8FgyedzLWedayn7VOC /8bI/BVmnleUCLBMOvGTqGADTisavll3F74anFxwJvS1cYCg3XLf6It1noXrD6k2u6XW oEWoFNwfx/0l+DJQdGSqgWTWh95ubTDEBEl55vRjVk3HA+tTejRmnSYdG21d+DrkDwIO IhnA== X-Gm-Message-State: AKGB3mLM6XCgaIMMwFQOj9Y5KXKabRQUHgJwUIM7yZj8bxEwW/9vPtkS e1di+WirmGwzZA2Ikgbs7W26jxNF X-Google-Smtp-Source: ACJfBotI4uYSxRMe2jF+FEaL/DUrTiVWtyuW2cbjoPjbRkviw+TQiZB8iHhu83MyV5YpKIkc9+tUYA== X-Received: by 10.223.152.129 with SMTP id w1mr2232108wrb.20.1514997276353; Wed, 03 Jan 2018 08:34:36 -0800 (PST) Received: from localhost.localdomain (sud35-h04-89-95-107-230.dsl.sta.abo.bbox.fr. [89.95.107.230]) by smtp.gmail.com with ESMTPSA id s11sm1356384wrb.30.2018.01.03.08.34.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Jan 2018 08:34:35 -0800 (PST) From: Christian Couder X-Google-Original-From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Ben Peart , Jonathan Tan , Nguyen Thai Ngoc Duy , Mike Hommey , Lars Schneider , Eric Wong , Christian Couder , Jeff Hostetler Subject: [PATCH 13/40] gc: do not repack promisor packfiles Date: Wed, 3 Jan 2018 17:33:36 +0100 Message-Id: <20180103163403.11303-14-chriscool@tuxfamily.org> X-Mailer: git-send-email 2.16.0.rc0.16.g82191dbc6c.dirty In-Reply-To: <20180103163403.11303-1-chriscool@tuxfamily.org> References: <20180103163403.11303-1-chriscool@tuxfamily.org> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jonathan Tan Teach gc to stop traversal at promisor objects, and to leave promisor packfiles alone. This has the effect of only repacking non-promisor packfiles, and preserves the distinction between promisor packfiles and non-promisor packfiles. Signed-off-by: Jonathan Tan Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano --- Documentation/git-pack-objects.txt | 11 ++++++++ builtin/gc.c | 4 +++ builtin/pack-objects.c | 37 ++++++++++++++++++++++++-- builtin/prune.c | 7 +++++ builtin/repack.c | 9 +++++-- t/t0410-partial-clone.sh | 54 ++++++++++++++++++++++++++++++++++++-- 6 files changed, 116 insertions(+), 6 deletions(-) diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt index aa403d02f3..81bc490ac5 100644 --- a/Documentation/git-pack-objects.txt +++ b/Documentation/git-pack-objects.txt @@ -255,6 +255,17 @@ a missing object is encountered. This is the default action. The form '--missing=allow-any' will allow object traversal to continue if a missing object is encountered. Missing objects will silently be omitted from the results. ++ +The form '--missing=allow-promisor' is like 'allow-any', but will only +allow object traversal to continue for EXPECTED promisor missing objects. +Unexpected missing object will raise an error. + +--exclude-promisor-objects:: + Omit objects that are known to be in the promisor remote. (This + option has the purpose of operating only on locally created objects, + so that when we repack, we still maintain a distinction between + locally created objects [without .promisor] and objects from the + promisor remote [with .promisor].) This is used with partial clone. SEE ALSO -------- diff --git a/builtin/gc.c b/builtin/gc.c index 3c5eae0edf..cef1461d1a 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -20,6 +20,7 @@ #include "argv-array.h" #include "commit.h" #include "packfile.h" +#include "external-odb.h" #define FAILED_RUN "failed to run %s" @@ -458,6 +459,9 @@ int cmd_gc(int argc, const char **argv, const char *prefix) argv_array_push(&prune, prune_expire); if (quiet) argv_array_push(&prune, "--no-progress"); + if (has_external_odb()) + argv_array_push(&prune, + "--exclude-promisor-objects"); if (run_command_v_opt(prune.argv, RUN_GIT_CMD)) return error(FAILED_RUN, prune.argv[0]); } diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 6b9cfc289d..6c71552cdf 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -75,6 +75,8 @@ static int use_bitmap_index = -1; static int write_bitmap_index; static uint16_t write_bitmap_options; +static int exclude_promisor_objects; + static unsigned long delta_cache_size = 0; static unsigned long max_delta_cache_size = 256 * 1024 * 1024; static unsigned long cache_max_small_delta_size = 1000; @@ -84,8 +86,9 @@ static unsigned long window_memory_limit = 0; static struct list_objects_filter_options filter_options; enum missing_action { - MA_ERROR = 0, /* fail if any missing objects are encountered */ - MA_ALLOW_ANY, /* silently allow ALL missing objects */ + MA_ERROR = 0, /* fail if any missing objects are encountered */ + MA_ALLOW_ANY, /* silently allow ALL missing objects */ + MA_ALLOW_PROMISOR, /* silently allow all missing PROMISOR objects */ }; static enum missing_action arg_missing_action; static show_object_fn fn_show_object; @@ -2578,6 +2581,20 @@ static void show_object__ma_allow_any(struct object *obj, const char *name, void show_object(obj, name, data); } +static void show_object__ma_allow_promisor(struct object *obj, const char *name, void *data) +{ + assert(arg_missing_action == MA_ALLOW_PROMISOR); + + /* + * Quietly ignore EXPECTED missing objects. This avoids problems with + * staging them now and getting an odd error later. + */ + if (!has_object_file(&obj->oid) && is_promisor_object(&obj->oid)) + return; + + show_object(obj, name, data); +} + static int option_parse_missing_action(const struct option *opt, const char *arg, int unset) { @@ -2592,10 +2609,18 @@ static int option_parse_missing_action(const struct option *opt, if (!strcmp(arg, "allow-any")) { arg_missing_action = MA_ALLOW_ANY; + fetch_if_missing = 0; fn_show_object = show_object__ma_allow_any; return 0; } + if (!strcmp(arg, "allow-promisor")) { + arg_missing_action = MA_ALLOW_PROMISOR; + fetch_if_missing = 0; + fn_show_object = show_object__ma_allow_promisor; + return 0; + } + die(_("invalid value for --missing")); return 0; } @@ -3009,6 +3034,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) { OPTION_CALLBACK, 0, "missing", NULL, N_("action"), N_("handling for missing objects"), PARSE_OPT_NONEG, option_parse_missing_action }, + OPT_BOOL(0, "exclude-promisor-objects", &exclude_promisor_objects, + N_("do not pack objects in promisor packfiles")), OPT_END(), }; @@ -3054,6 +3081,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) argv_array_push(&rp, "--unpacked"); } + if (exclude_promisor_objects) { + use_internal_rev_list = 1; + fetch_if_missing = 0; + argv_array_push(&rp, "--exclude-promisor-objects"); + } + if (!reuse_object) reuse_delta = 0; if (pack_compression_level == -1) diff --git a/builtin/prune.c b/builtin/prune.c index d2fdae680a..4cfec82f40 100644 --- a/builtin/prune.c +++ b/builtin/prune.c @@ -101,12 +101,15 @@ int cmd_prune(int argc, const char **argv, const char *prefix) { struct rev_info revs; struct progress *progress = NULL; + int exclude_promisor_objects = 0; const struct option options[] = { OPT__DRY_RUN(&show_only, N_("do not remove, show only")), OPT__VERBOSE(&verbose, N_("report pruned objects")), OPT_BOOL(0, "progress", &show_progress, N_("show progress")), OPT_EXPIRY_DATE(0, "expire", &expire, N_("expire objects older than