git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: Doug Kelly <dougk.ff7@gmail.com>
Cc: Git List <git@vger.kernel.org>
Subject: Re: Question: .idx without .pack causes performance issues?
Date: Mon, 03 Aug 2015 18:27:27 -0700	[thread overview]
Message-ID: <xmqqk2tb6dxc.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <CAEtYS8SNksc0m5rn_tRk8bGLBeq_8QcBLHyHo=cOfZ+aE6n0gA@mail.gmail.com> (Doug Kelly's message of "Mon, 3 Aug 2015 17:17:28 -0500")

Doug Kelly <dougk.ff7@gmail.com> writes:

> Here's a change to prune.c that at least addresses the issue by removing
> .idx files without an associated pack, but it's by no means pretty.  If anyone
> has any feedback before I turn this into a formal patch, it's more than welcome!

I'd hesitate to see removal of a file (for that matter, a creation
too) inside a "while (de = readdir)" loop.  As the original function
is about temporary files, and the new thing is not about temporary
files at all, I'd further prefer that we do not do it in the same
loop.

I am wondering if we can add a new mode to report_pack_garbage() in
sha1_file.c to allow it to remove stale and lone ".idx".  Most of
the time we are accessing packs read-only, and I do not want the
function to unconditionally remove lone ".idx", but perhaps we
can teach "prune" to set a custom report_garbage() routine and
react to a call to its custom report_garbage()?

Perhaps that custom report_garbage() can make a list of ".idx"
files, iterate over it to pick the lone one without ".pack" and
remove them.  Or the custom report_garbage() can make a list of lone
".idx" files, if you tweak the interface to report_garbage() to
contain th seen_bits value, avoiding the need to check the existence
of ".pack" for the second time.

  reply	other threads:[~2015-08-04  1:27 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-07-21 18:41 Question: .idx without .pack causes performance issues? Doug Kelly
2015-07-21 18:57 ` Junio C Hamano
2015-07-21 19:15   ` Junio C Hamano
2015-07-21 20:48     ` Junio C Hamano
2015-07-21 21:37       ` Doug Kelly
2015-08-03 22:17         ` Doug Kelly
2015-08-04  1:27           ` Junio C Hamano [this message]
2015-08-07 21:36             ` Doug Kelly
2015-08-07 22:27               ` Junio C Hamano
2015-08-13 18:02                 ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Doug Kelly
2015-08-13 18:02                   ` [PATCH 2/2] gc: Remove garbage .idx files from pack dir Doug Kelly
2015-08-17 16:35                     ` Junio C Hamano
2015-08-17 20:30                     ` Junio C Hamano
2015-08-13 18:46                   ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Eric Sunshine
2015-08-17 16:53                     ` Junio C Hamano
2015-10-28 17:48                       ` Junio C Hamano
2015-10-28 22:43                         ` Doug Kelly
2015-11-04  3:05                           ` [PATCH 1/3] " Doug Kelly
2015-11-04  3:05                             ` [PATCH 2/3] t5304: Add test for cleaning pack garbage Doug Kelly
2015-11-04  3:05                             ` [PATCH 3/3] gc: Remove garbage .idx files from pack dir Doug Kelly
2015-11-04  3:12                           ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Doug Kelly
2015-11-04 19:35                             ` Junio C Hamano
2015-11-04 19:56                               ` Doug Kelly
2015-11-04 20:02                                 ` Jeff King
2015-11-04 20:08                                   ` Doug Kelly
2015-11-04 20:15                                     ` Jeff King
2015-12-30  7:37                                     ` Jeff King
2016-01-13 17:14                                       ` Doug Kelly
2016-01-13 20:08                                         ` Junio C Hamano
2016-01-13 20:19                                           ` Doug Kelly
2016-01-13 20:23                                             ` Jeff King
2015-11-04 19:56                               ` Jeff King
     [not found]     ` <CABYiQpn7r2Vcf=S5RaWHBN85eBYGPV_e02+BY=4L98qfUzDT1Q@mail.gmail.com>
2015-11-11 14:58       ` Fwd: Question: .idx without .pack causes performance issues? Thomas Berg
2015-07-21 19:49   ` Doug Kelly

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqk2tb6dxc.fsf@gitster.dls.corp.google.com \
    --to=gitster@pobox.com \
    --cc=dougk.ff7@gmail.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).