git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Cc: git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>
Subject: Re: [PATCH 0/7] nd/repack-keep-pack update
Date: Sat, 14 Apr 2018 21:47:12 +0200	[thread overview]
Message-ID: <878t9pfu4f.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <20180414152642.4666-1-pclouds@gmail.com>


On Sat, Apr 14 2018, Nguyễn Thái Ngọc Duy wrote:

> This is basically a resend from the last round but rebased on
> 'master'. The old base results in some conflicts with packfile and oid
> conversion series. This one should reduce merge conflicts on 'pu' to
> trivial ones.

Thanks. I've been running this at work and as noted in
https://public-inbox.org/git/87vadpxv27.fsf@evledraar.gmail.com/ it's
had big performance impact to the better, users even started noticing it
(they'd previously get noticeable slowdowns while doing other task on
GC).

I also tried to see just how much worse this was making performance, my
hunch was that the difference should be trivial but noticeable since
we'll produce a less efficient pack.

What I found was the opposite, under real-world conditions it seems to
be making things 1-2% better on common git operations, which I suspect
is because once we've done a few pulls and coalesced those into their
own pack(s) there's more cache locality for the data we're actually
looking at.

I.e. once you've got a repo has a big pack you're not touching, and a
few weeks of updates from upstream that you've coalesced into another
pack there's a higher density of stuff you care about near HEAD per FS
page in the recent smaller pack, which if you're pressed for memory and
parts of your pack are getting paged out of the FS cache is a win. I
haven't confirmed that, it's just a hypothesis.

The only (trivial) issue I found in the patches themselves was that
between 4/5 and 5/7 you're adding an empty line to config.txt in 4/7
just to erase it in 5/7, better not to add it to begin with, but
hopefully Junio can fix that up (if he cares).

  parent reply	other threads:[~2018-04-14 19:47 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-14 15:26 [PATCH 0/7] nd/repack-keep-pack update Nguyễn Thái Ngọc Duy
2018-04-14 15:26 ` [PATCH 1/7] t7700: have closing quote of a test at the beginning of line Nguyễn Thái Ngọc Duy
2018-04-14 15:26 ` [PATCH 2/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-04-14 15:26 ` [PATCH 3/7] gc: add --keep-largest-pack option Nguyễn Thái Ngọc Duy
2018-04-14 15:26 ` [PATCH 4/7] gc: add gc.bigPackThreshold config Nguyễn Thái Ngọc Duy
2018-04-14 15:26 ` [PATCH 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy
2018-04-14 15:26 ` [PATCH 6/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-04-14 15:26 ` [PATCH 7/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-04-14 19:47 ` Ævar Arnfjörð Bjarmason [this message]
2018-04-14 22:38   ` [PATCH 0/7] nd/repack-keep-pack update Junio C Hamano
2018-04-15 15:36 ` [PATCH v2 " Nguyễn Thái Ngọc Duy
2018-04-15 15:36   ` [PATCH v2 1/7] t7700: have closing quote of a test at the beginning of line Nguyễn Thái Ngọc Duy
2018-04-15 15:36   ` [PATCH v2 2/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-04-15 15:36   ` [PATCH v2 3/7] gc: add --keep-largest-pack option Nguyễn Thái Ngọc Duy
2018-04-15 15:36   ` [PATCH v2 4/7] gc: add gc.bigPackThreshold config Nguyễn Thái Ngọc Duy
2018-04-15 15:36   ` [PATCH v2 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy
2018-04-15 15:36   ` [PATCH v2 6/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-04-15 15:36   ` [PATCH v2 7/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-04-16  4:51   ` [PATCH v2 0/7] nd/repack-keep-pack update Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=878t9pfu4f.fsf@evledraar.gmail.com \
    --to=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=pclouds@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).