From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-2.0 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by dcvr.yhbt.net (Postfix) with ESMTP id A8E811F452 for ; Wed, 12 Apr 2023 10:23:04 +0000 (UTC) Authentication-Results: dcvr.yhbt.net; dkim=pass (2048-bit key; unprotected) header.d=pks.im header.i=@pks.im header.a=rsa-sha256 header.s=fm3 header.b=YDUWb7Gh; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm3 header.b=SWEq7ww1; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231480AbjDLKW7 (ORCPT ); Wed, 12 Apr 2023 06:22:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230099AbjDLKWx (ORCPT ); Wed, 12 Apr 2023 06:22:53 -0400 Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30EE87A9B for ; Wed, 12 Apr 2023 03:22:48 -0700 (PDT) Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 40C2D3200979; Wed, 12 Apr 2023 06:22:48 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 12 Apr 2023 06:22:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1681294967; x=1681381367; bh=WW xzsL+aogjS+iPsHi7EjPHY6gJwOSprOj5oHJjX0tA=; b=YDUWb7GhNTdlUxldhJ XR6m9jKtDYZhTvOd/rP8yOb8+Wm1UWfcmTPmAyJtrRznD67hACtVlOT8v+ZxI7Ue sy6vOgtmTkQNUi1VtBI9inJeWFGwIFK0FEiRG3hGvs19+khRNlVKBV5iTEMYEUF+ B0h0l8pDI1Oreun4tp9yoIIcvUX3fKwZaOklOW8LnemdP0bp4WRxRM+iYzsVKHD7 GCxiDUdErgU3oruJmueMD7l1yQFkvz6036zmMeZJ1rJKieFDbejZJqGip7gO3GYs pEZFbCWbhoOJCKl8Jy/1RLL45pF8Y320ZkT9tcdYKQM6SYmZnwlVRwSqf2MgtY2t BbHw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1681294967; x=1681381367; bh=WWxzsL+aogjS+ iPsHi7EjPHY6gJwOSprOj5oHJjX0tA=; b=SWEq7ww1ud4GSItsWebbM+Sirg4ND jezfrea+/zegSgNmi1HKvDDM4xCgeeIdxcQlDKUdVcuQcwVcdK2FAPBFqy5dRTL0 F8kVp7VbVh1bT2MbwmK6op8iRm3V9EDtGXeg49ANoccs6DdaxGKWg/0WPEKRFpde KY9ajTZoZBBr6x0soV4ea2lE/D2Sa3MSG9Hurb9miMKe45TtGSFGt+1nLPE87z6Q ZeLKd5fwdehBQxlm0V2/QTrsmmjtv/PgskUisGZYWy3+M2blaZ0aLiJZMhaFFNzl WBx8fAFgApy46wC/SIbgo8vQTkDHRbmgmGJPzK1PbFWB/dMpblhCOSmFQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdekiedgvdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvvefukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeeukedtvedtffevleejtefgheehieegke eluddvfeefgeehgfeltddtheejleffteenucevlhhushhtvghrufhiiigvpedvnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 12 Apr 2023 06:22:46 -0400 (EDT) Received: by pks.im (OpenSMTPD) with ESMTPSA id b78cfa58 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 12 Apr 2023 10:22:25 +0000 (UTC) Date: Wed, 12 Apr 2023 12:22:44 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Taylor Blau , peff@peff.net, dstolee@microsoft.com Subject: [PATCH v2 4/8] pack-objects: fix error when packing same pack twice Message-ID: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Y0ZVHOLJiAVeFlZq" Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org --Y0ZVHOLJiAVeFlZq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable When passed the same packfile twice via `--stdin-packs` we return an error that the packfile supposedly was not found. This is because when reading packs into the list of included or excluded packfiles, we will happily re-add packfiles even if they are part of the lists already. And while the list can now contain duplicates, we will only set the `util` pointer of the first list entry to the `packed_git` structure. We notice that at a later point when checking that all list entries have their `util` pointer set and die with an error. While this is kind of a nonsensical request, this scenario can be hit when doing geometric repacks. When a repository is connected to an alternate object directory and both have the exact same packfile then both would get added to the geometric sequence. And when we then decide to perform the repack, we will invoke git-pack-objects(1) with the same packfile twice. Fix this bug by removing any duplicates from both the included and excluded packs. Signed-off-by: Patrick Steinhardt --- builtin/pack-objects.c | 2 ++ t/t5331-pack-objects-stdin.sh | 29 +++++++++++++++++++++++++++++ t/t7703-repack-geometric.sh | 25 +++++++++++++++++++++++++ 3 files changed, 56 insertions(+) create mode 100755 t/t5331-pack-objects-stdin.sh diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 77d88f85b0..fdf3f440be 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3359,7 +3359,9 @@ static void read_packs_list_from_stdin(void) } =20 string_list_sort(&include_packs); + string_list_remove_duplicates(&include_packs, 0); string_list_sort(&exclude_packs); + string_list_remove_duplicates(&exclude_packs, 0); =20 for (p =3D get_all_packs(the_repository); p; p =3D p->next) { const char *pack_name =3D pack_basename(p); diff --git a/t/t5331-pack-objects-stdin.sh b/t/t5331-pack-objects-stdin.sh new file mode 100755 index 0000000000..ab34cfc729 --- /dev/null +++ b/t/t5331-pack-objects-stdin.sh @@ -0,0 +1,29 @@ +#!/bin/sh + +test_description=3D'pack-objects --stdin' +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=3Dmain +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +TEST_PASSES_SANITIZE_LEAK=3Dtrue +. ./test-lib.sh + +test_expect_success 'pack-objects --stdin with duplicate packfile' ' + test_when_finished "rm -fr repo" && + + git init repo && + ( + cd repo && + test_commit "commit" && + git repack -ad && + + ( + basename .git/objects/pack/pack-*.pack && + basename .git/objects/pack/pack-*.pack + ) >packfiles && + + git pack-objects --stdin-packs generated-pack