git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Theodore Tso <tytso@mit.edu>
To: Nicolas Pitre <nico@cam.org>
Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>,
	Junio C Hamano <junkio@cox.net>,
	Git Mailing List <git@vger.kernel.org>
Subject: Re: [PATCH] Add pack.depth option to git-pack-objects and change default depth to 50
Date: Wed, 9 May 2007 09:49:58 -0400	[thread overview]
Message-ID: <20070509134958.GA21489@thunk.org> (raw)
In-Reply-To: <alpine.LFD.0.99.0705081439140.24220@xanadu.home>

On Tue, May 08, 2007 at 02:46:35PM -0400, Nicolas Pitre wrote:
> The window size has absolutely no effect on the runtime pack access, 
> except maybe for the increased number of deltas.  It is really a pack 
> time cost.  The delta depth is the opposite: it has no effect on the 
> packing time, but it has the potential to slow down runtime access.  But 
> the delta base cache is apparently working really well to mitigate that 
> cost, as long as it is big enough of course.

Exactly, and the numbers bear out with the theory.  Junio's already
applied the change to up the default to 50, but for the record, here
are the results using a kernel git repository, with times for doing a
git-gc (with the --no-reuse-delta, although I like the suggestion to
change it to just be "--hard", or maybe "--agressive"), and with a git
pickaxe and a git-log with pathname restrictions.  It essentially
confirms that the delta base cache is doing the job just fine, up to
even a depth of 100.  However, there isn't much difference in pack
size between a depth of 50 and 100.

Increasing the window size from 10 to 30 increases the pack run time
by roughly 40%, and saves an extra 5% or so on the pack size.
Increasing the window beyond 30 has ever-smaller diminishing returns,
while the time to repack gets bigger and bigger.

The timing information is real/user/system/minor pagefaults, and as
before, the results are done using a hot cache.  Apologies in advance
for the long lines in the benchmark results.  These results were
generated using a relatively recent, post-2.6.21 kernel git tree on a
Thinkpad T60p laptop with a 2GHz T2500 Intel Core Duo processor.

Regards,

             git-gc                      pack size   git-log -S'object'    git-log include/scsi drivers/scsi
w=10,d=10   5:08.70/293.20/5.56/1027802  162316k  5:31.12/322.20/4.14/153298  0:01.80/1.72/0.03/19077
w=10,d=30   4:14.79/245.46/2.57/398754   149636k  5:41.56/331.21/4.86/517220  0:01.83/1.74/0.05/17880
w=10,d=50   4:31.89/257.16/3.50/576538   146608k  5:51.58/336.63/5.07/259643  0:01.88/1.80/0.04/17658
w=10,d=100  4:35.93/262.77/4.08/715195   144152k  5:58.23/341.26/6.22/624510  0:01.89/1.80/0.04/17571

w=30,d=10   7:23.06/424.08/6.35/1222640  153752k  5:32.85/323.96/2.13/213150  0:01.73/1.64/0.06/18429
w=30,d=30   6:15.76/364.27/3.53/407546   141200k  5:42.29/333.53/2.10/338301  0:01.81/1.71/0.04/17237
w=30,d=50   6:26.85/372.24/5.38/578343   139408k  5:42.21/336.65/1.30/260234  0:01.77/1.70/0.05/17108
w=30,d=100  6:34.31/381.72/5.40/744886   138040k  5:59.03/342.65/4.90/607681  0:01.93/1.79/0.06/17043

w=50,d=10   8:51.08/508.80/5.75/1050358  152168k  5:35.43/327.48/3.70/209655  0:01.70/1.64/0.04/18264
w=50,d=30   8:04.53/471.65/5.58/423755   139000k  5:42.06/335.67/1.55/335333  0:01.78/1.72/0.04/17046
w=50,d=50   8:14.87/479.93/7.09/617310   137244k  5:47.22/339.26/2.73/462412  0:01.79/1.72/0.04/16871
w=50,d=100  8:23.13/490.09/4.10/751742   136152k  5:51.10/343.89/2.29/503573  0:01.83/1.73/0.06/16920

w=100,d=10  12:00.34/702.07/6.27/1167403 150736k  5:34.42/328.88/1.09/207522  0:01.72/1.63/0.07/18024
w=100,d=30  11:56.34/702.36/2.93/436950  137240k  5:42.64/335.82/2.07/422743  0:01.77/1.67/0.05/16852
w=100,d=50  12:20.39/722.99/3.94/671655  135488k  5:47.20/339.38/2.44/468974  0:01.78/1.68/0.03/16621
w=100,d=100 12:37.69/740.63/4.12/733593  134492k  5:52.26/344.16/3.00/636120  0:01.85/1.80/0.03/16681

							- Ted

  reply	other threads:[~2007-05-09 13:50 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-05-08  2:54 [PATCH] Add --no-reuse-delta, --window, and --depth options to git-gc Theodore Ts'o
2007-05-08  3:13 ` Nicolas Pitre
2007-05-08  3:21   ` Theodore Tso
2007-05-08  3:38     ` Dana How
2007-05-08  4:43     ` Junio C Hamano
2007-05-08 13:46       ` Nicolas Pitre
2007-05-08 13:28 ` [PATCH] Add --no-reuse-delta, --window, and --depth options to Theodore Ts'o
2007-05-08 13:28   ` [PATCH] Add pack.depth option to git-pack-objects and change default depth to 50 Theodore Ts'o
2007-05-08 13:28     ` [PATCH] Add --no-reuse-delta option to git-gc Theodore Ts'o
2007-05-08 15:35       ` Nicolas Pitre
2007-05-09  5:05       ` Daniel Barkalow
2007-05-09  8:15         ` Junio C Hamano
2007-05-09  9:02           ` Steven Grimm
2007-05-09 11:35             ` Other compression?, was " Johannes Schindelin
2007-05-09 15:15             ` Junio C Hamano
2007-05-09 19:10             ` Shawn O. Pearce
2007-06-10  7:40               ` Sam Vilain
2007-06-11  1:51                 ` Nicolas Pitre
2007-06-11  6:20                   ` Steven Grimm
2007-06-11  6:31                     ` Shawn O. Pearce
2007-06-11 10:20                   ` Johannes Schindelin
2007-06-11 14:01                     ` Nicolas Pitre
2007-06-11 21:40                       ` Johannes Schindelin
2007-05-09 19:48           ` [PATCH] Add --aggressive option to 'git gc' Theodore Tso
2007-05-09 20:19             ` Junio C Hamano
2007-05-09 22:22               ` Theodore Tso
2007-05-10  7:38             ` Junio C Hamano
2007-05-08 15:38     ` [PATCH] Add pack.depth option to git-pack-objects and change default depth to 50 Nicolas Pitre
2007-05-08 16:30       ` Theodore Tso
2007-05-08 16:49         ` Johannes Schindelin
2007-05-08 18:09           ` Theodore Tso
2007-05-08 18:46             ` Nicolas Pitre
2007-05-09 13:49               ` Theodore Tso [this message]
2007-05-09 14:17                 ` Johannes Schindelin
2007-05-08 17:07         ` Dana How
2007-05-08 17:35         ` Nicolas Pitre
2007-05-09  5:03           ` Junio C Hamano
2007-05-08 15:30   ` [PATCH] Add --no-reuse-delta, --window, and --depth options to Nicolas Pitre
2007-05-08 21:12     ` Junio C Hamano
2007-05-08 23:59       ` Nicolas Pitre

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070509134958.GA21489@thunk.org \
    --to=tytso@mit.edu \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=junkio@cox.net \
    --cc=nico@cam.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).