From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-4.4 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by dcvr.yhbt.net (Postfix) with ESMTP id 5098C2018E for ; Mon, 8 Aug 2016 16:28:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752360AbcHHQ2e (ORCPT ); Mon, 8 Aug 2016 12:28:34 -0400 Received: from pb-smtp1.pobox.com ([64.147.108.70]:53613 "EHLO sasl.smtp.pobox.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752312AbcHHQ2d (ORCPT ); Mon, 8 Aug 2016 12:28:33 -0400 Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-smtp1.pobox.com (Postfix) with ESMTP id CDBC734D03; Mon, 8 Aug 2016 12:28:31 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=bZr7M/2TFU63NqBFCF8hrvlu7QA=; b=VUdtnk f8f4TEYCXD2wgf7Ghg/dk1DxdaPutp+2DGT2kacf9EY0hrrwmvO7zNZVzIKhwE2T VW+Unu1PVn2nz1xl6bEpr9OS0wVxSM1iXsMdu6f2BW7bgfPIvwrg4jCvSs/1PjiV nCc8S7hw2XJwyBpr4pOEmWLG3+cy5AsmWUL3w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=X8eYQbCTOdXM/iuynoe25n1E/+2mE5A1 POLq6ltG0oVSQUjZZFYQk67lceHhBCbMmbxUmCsa9En0hOgcCh3IWMVUjYAS75E3 NaVASy4aoqArIKL3PyR49/rhk2gW3qH+uUawJRU9zavo+8wX1RzYgTyGZROlBYoj Y3eMurLPSvc= Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1]) by pb-smtp1.pobox.com (Postfix) with ESMTP id B3DB134D02; Mon, 8 Aug 2016 12:28:31 -0400 (EDT) Received: from pobox.com (unknown [104.132.0.95]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 28A1134D01; Mon, 8 Aug 2016 12:28:31 -0400 (EDT) From: Junio C Hamano To: Jeff King Cc: git@vger.kernel.org, Michael Haggerty Subject: Re: [PATCH v2 7/7] pack-objects: use mru list when iterating over packs References: <20160729040422.GA19678@sigill.intra.peff.net> <20160729041524.GG22408@sigill.intra.peff.net> <20160729054536.GA27343@sigill.intra.peff.net> <20160808145042.uwrk2m6jq3m4li37@sigill.intra.peff.net> Date: Mon, 08 Aug 2016 09:28:29 -0700 In-Reply-To: <20160808145042.uwrk2m6jq3m4li37@sigill.intra.peff.net> (Jeff King's message of "Mon, 8 Aug 2016 10:50:43 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Pobox-Relay-ID: 24DB40D6-5D85-11E6-9A48-89D312518317-77302942!pb-smtp1.pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Jeff King writes: > Here's a list of approaches I think we can use to fix this: > > 1. squelch the warning and ignore it. The downside here, besides not > warning the user about true in-pack cycles, is that we have no > opportunity to actually find a new delta (because we realize the > problem only after the delta-compression phase). > > My test repository is a bad packing of all of the forks of > torvalds/linux, with 3600 packs. I'm happy to share it if anybody > wants to see it, but note that it is 11GB. > > The space overhead of the resulting pack in this case is ~3.2% > (versus a pack generated by the original code, using the static > pack order). Which is really not that bad, but I'm sure there are > more pathological cases (i.e., there were on the order of hundreds > or maybe thousands of cycles that needed broken, out of about 15 > million total objects; but one could imagine the worst case as > nr_of_objects/2). > ... > > So I dunno. I really like the MRU approach if we can salvage it. I think I share the same feeling. As long as the chance is small enough that the pack reordering creates a new cycle, the resulting pack would not become too bloated by the last-ditch cycle breaking code and finding a replacement delta instead of inflating it may not be worth the trouble. It worries me a lot to lose the warning unconditionally, though. That's the (only) coal-mine canary that lets us notice a problem when we actually start hitting that last-ditch cycle breaking too often.