git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Subject: [PATCH 64/68] fsck: drop inode-sorting code
Date: Thu, 24 Sep 2015 17:08:28 -0400	[thread overview]
Message-ID: <20150924210828.GI30946@sigill.intra.peff.net> (raw)
In-Reply-To: <20150924210225.GA23624@sigill.intra.peff.net>

Fsck tries to access loose objects in order of inode number,
with the hope that this would make cold cache access faster
on a spinning disk. This dates back to 7e8c174 (fsck-cache:
sort entries by inode number, 2005-05-02), which predates
the invention of packfiles.

These days, there's not much point in trying to optimize
cold cache for a large number of loose objects. You are much
better off to simply pack the objects, which will reduce the
disk footprint _and_ provide better locality of data access.

So while you can certainly construct pathological cases
where this code might help, it is not worth the trouble
anymore.

Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/fsck.c | 70 ++--------------------------------------------------------
 1 file changed, 2 insertions(+), 68 deletions(-)

diff --git a/builtin/fsck.c b/builtin/fsck.c
index a019f4a..73c3596 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -39,14 +39,6 @@ static int show_dangling = 1;
 #define ERROR_REACHABLE 02
 #define ERROR_PACK 04
 
-#ifdef NO_D_INO_IN_DIRENT
-#define SORT_DIRENT 0
-#define DIRENT_SORT_HINT(de) 0
-#else
-#define SORT_DIRENT 1
-#define DIRENT_SORT_HINT(de) ((de)->d_ino)
-#endif
-
 static int fsck_config(const char *var, const char *value, void *cb)
 {
 	if (strcmp(var, "fsck.skiplist") == 0) {
@@ -373,64 +365,6 @@ static int fsck_obj_buffer(const unsigned char *sha1, enum object_type type,
 	return fsck_obj(obj);
 }
 
-/*
- * This is the sorting chunk size: make it reasonably
- * big so that we can sort well..
- */
-#define MAX_SHA1_ENTRIES (1024)
-
-struct sha1_entry {
-	unsigned long ino;
-	unsigned char sha1[20];
-};
-
-static struct {
-	unsigned long nr;
-	struct sha1_entry *entry[MAX_SHA1_ENTRIES];
-} sha1_list;
-
-static int ino_compare(const void *_a, const void *_b)
-{
-	const struct sha1_entry *a = _a, *b = _b;
-	unsigned long ino1 = a->ino, ino2 = b->ino;
-	return ino1 < ino2 ? -1 : ino1 > ino2 ? 1 : 0;
-}
-
-static void fsck_sha1_list(void)
-{
-	int i, nr = sha1_list.nr;
-
-	if (SORT_DIRENT)
-		qsort(sha1_list.entry, nr,
-		      sizeof(struct sha1_entry *), ino_compare);
-	for (i = 0; i < nr; i++) {
-		struct sha1_entry *entry = sha1_list.entry[i];
-		unsigned char *sha1 = entry->sha1;
-
-		sha1_list.entry[i] = NULL;
-		if (fsck_sha1(sha1))
-			errors_found |= ERROR_OBJECT;
-		free(entry);
-	}
-	sha1_list.nr = 0;
-}
-
-static void add_sha1_list(unsigned char *sha1, unsigned long ino)
-{
-	struct sha1_entry *entry = xmalloc(sizeof(*entry));
-	int nr;
-
-	entry->ino = ino;
-	hashcpy(entry->sha1, sha1);
-	nr = sha1_list.nr;
-	if (nr == MAX_SHA1_ENTRIES) {
-		fsck_sha1_list();
-		nr = 0;
-	}
-	sha1_list.entry[nr] = entry;
-	sha1_list.nr = ++nr;
-}
-
 static inline int is_loose_object_file(struct dirent *de,
 				       char *name, unsigned char *sha1)
 {
@@ -459,7 +393,8 @@ static void fsck_dir(int i, char *path)
 		if (is_dot_or_dotdot(de->d_name))
 			continue;
 		if (is_loose_object_file(de, name, sha1)) {
-			add_sha1_list(sha1, DIRENT_SORT_HINT(de));
+			if (fsck_sha1(sha1))
+				errors_found |= ERROR_OBJECT;
 			continue;
 		}
 		if (starts_with(de->d_name, "tmp_obj_"))
@@ -573,7 +508,6 @@ static void fsck_object_dir(const char *path)
 		display_progress(progress, i+1);
 	}
 	stop_progress(&progress);
-	fsck_sha1_list();
 }
 
 static int fsck_head_link(void)
-- 
2.6.0.rc3.454.g204ad51

  parent reply	other threads:[~2015-09-24 21:09 UTC|newest]

Thread overview: 93+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-09-24 21:02 [PATCH v2 0/68] war on sprintf Jeff King
2015-09-24 21:02 ` [PATCH 01/68] show-branch: avoid segfault with --reflog of unborn branch Jeff King
2015-09-24 21:03 ` [PATCH 02/68] mailsplit: fix FILE* leak in split_maildir Jeff King
2015-09-24 21:03 ` [PATCH 03/68] archive-tar: fix minor indentation violation Jeff King
2015-09-24 21:05 ` [PATCH 04/68] fsck: don't fsck alternates for connectivity-only check Jeff King
2015-09-24 21:05 ` [PATCH 05/68] add xsnprintf helper function Jeff King
2015-09-24 21:05 ` [PATCH 06/68] add git_path_buf " Jeff King
2015-09-24 21:05 ` [PATCH 07/68] strbuf: make strbuf_complete_line more generic Jeff King
2015-09-24 21:05 ` [PATCH 08/68] add reentrant variants of sha1_to_hex and find_unique_abbrev Jeff King
2015-09-24 21:05 ` [PATCH 09/68] fsck: use strbuf to generate alternate directories Jeff King
2015-09-24 21:05 ` [PATCH 10/68] mailsplit: make PATH_MAX buffers dynamic Jeff King
2015-09-24 21:05 ` [PATCH 11/68] trace: use strbuf for quote_crnl output Jeff King
2015-09-24 21:05 ` [PATCH 12/68] progress: store throughput display in a strbuf Jeff King
2015-09-24 21:06 ` [PATCH 13/68] test-dump-cache-tree: avoid overflow of cache-tree name Jeff King
2015-09-24 21:06 ` [PATCH 14/68] compat/inet_ntop: fix off-by-one in inet_ntop4 Jeff King
2015-09-24 21:06 ` [PATCH 15/68] convert trivial sprintf / strcpy calls to xsnprintf Jeff King
2015-09-24 21:06 ` [PATCH 16/68] archive-tar: use xsnprintf for trivial formatting Jeff King
2015-09-24 21:06 ` [PATCH 17/68] use xsnprintf for generating git object headers Jeff King
2015-09-24 21:06 ` [PATCH 18/68] find_short_object_filename: convert sprintf to xsnprintf Jeff King
2015-09-24 21:06 ` [PATCH 19/68] stop_progress_msg: " Jeff King
2015-09-24 21:06 ` [PATCH 20/68] compat/hstrerror: convert sprintf to snprintf Jeff King
2015-09-24 21:06 ` [PATCH 21/68] grep: use xsnprintf to format failure message Jeff King
2015-09-24 21:06 ` [PATCH 22/68] entry.c: convert strcpy to xsnprintf Jeff King
2015-09-24 21:06 ` [PATCH 23/68] add_packed_git: convert strcpy into xsnprintf Jeff King
2015-09-24 21:06 ` [PATCH 24/68] http-push: replace strcat with xsnprintf Jeff King
2015-09-24 21:07 ` [PATCH 25/68] receive-pack: convert strncpy to xsnprintf Jeff King
2015-09-24 21:07 ` [PATCH 26/68] replace trivial malloc + sprintf / strcpy calls with xstrfmt Jeff King
2015-09-24 21:07 ` [PATCH 27/68] config: use xstrfmt in normalize_value Jeff King
2015-09-24 21:07 ` [PATCH 28/68] fetch: replace static buffer with xstrfmt Jeff King
2015-09-24 21:07 ` [PATCH 29/68] use strip_suffix and xstrfmt to replace suffix Jeff King
2015-09-24 21:07 ` [PATCH 30/68] ref-filter: drop sprintf and strcpy calls Jeff King
2015-09-24 21:07 ` [PATCH 31/68] help: drop prepend function in favor of xstrfmt Jeff King
2015-09-24 21:07 ` [PATCH 32/68] mailmap: replace strcpy with xstrdup Jeff King
2015-09-24 21:07 ` [PATCH 33/68] read_branches_file: simplify string handling Jeff King
2015-09-24 21:07 ` [PATCH 34/68] read_remotes_file: " Jeff King
2015-09-24 21:07 ` [PATCH 35/68] resolve_ref: use strbufs for internal buffers Jeff King
2015-09-24 21:07 ` [PATCH 36/68] upload-archive: convert sprintf to strbuf Jeff King
2015-09-24 21:07 ` [PATCH 37/68] remote-ext: simplify git pkt-line generation Jeff King
2015-09-24 21:07 ` [PATCH 38/68] http-push: use strbuf instead of fwrite_buffer Jeff King
2015-09-24 21:07 ` [PATCH 39/68] http-walker: store url in a strbuf Jeff King
2015-09-24 21:07 ` [PATCH 40/68] sha1_get_pack_name: use " Jeff King
2015-09-24 21:07 ` [PATCH 41/68] init: use strbufs to store paths Jeff King
2015-09-29 23:50   ` Michael Blume
2015-09-30  0:23     ` Jeff King
2015-09-30 20:00       ` Junio C Hamano
2015-10-01  2:51         ` Jeff King
2015-10-02  6:00           ` Torsten Bögershausen
2015-10-02 15:33             ` Jeff King
2015-10-03  5:58       ` Torsten Bögershausen
2015-10-03 16:54         ` Junio C Hamano
2015-10-03 21:12           ` Torsten Bögershausen
2015-10-04  3:37             ` Jeff King
2015-10-04  6:31               ` Torsten Bögershausen
2015-10-05  3:41                 ` Jeff King
2015-10-05  3:43                   ` [PATCH 1/3] precompose_utf8: drop unused variable Jeff King
2015-10-06  3:24                     ` Torsten Bögershausen
2015-10-05  3:45                   ` [PATCH 2/3] probe_utf8_pathname_composition: use internal strbuf Jeff King
2015-10-05  3:46                   ` [PATCH 3/3] init: use strbufs to store paths Jeff King
2015-09-24 21:07 ` [PATCH 42/68] apply: convert root string to strbuf Jeff King
2015-09-24 21:07 ` [PATCH 43/68] transport: use strbufs for status table "quickref" strings Jeff King
2015-09-24 21:07 ` [PATCH 44/68] merge-recursive: convert malloc / strcpy to strbuf Jeff King
2015-09-24 21:07 ` [PATCH 45/68] enter_repo: convert fixed-size buffers to strbufs Jeff King
2015-09-24 21:07 ` [PATCH 46/68] remove_leading_path: use a strbuf for internal storage Jeff King
2015-09-24 21:07 ` [PATCH 47/68] write_loose_object: convert to strbuf Jeff King
2015-09-24 21:07 ` [PATCH 48/68] diagnose_invalid_index_path: use strbuf to avoid strcpy/strcat Jeff King
2015-09-24 21:07 ` [PATCH 49/68] fetch-pack: use argv_array for index-pack / unpack-objects Jeff King
2015-09-24 21:07 ` [PATCH 50/68] http-push: use an argv_array for setup_revisions Jeff King
2015-09-24 21:07 ` [PATCH 51/68] stat_tracking_info: convert to argv_array Jeff King
2015-09-24 21:08 ` [PATCH 52/68] daemon: use cld->env_array when re-spawning Jeff King
2015-09-24 21:08 ` [PATCH 53/68] use sha1_to_hex_r() instead of strcpy Jeff King
2015-09-24 21:08 ` [PATCH 54/68] drop strcpy in favor of raw sha1_to_hex Jeff King
2015-09-24 23:42   ` Eric Sunshine
2015-09-25  1:36     ` Jeff King
2015-09-24 21:08 ` [PATCH 55/68] color: add overflow checks for parsing colors Jeff King
2015-09-24 21:08 ` [PATCH 56/68] use alloc_ref rather than hand-allocating "struct ref" Jeff King
2015-09-24 21:08 ` [PATCH 57/68] avoid sprintf and strcpy with flex arrays Jeff King
2015-09-24 21:08 ` [PATCH 58/68] receive-pack: simplify keep_arg computation Jeff King
2015-09-24 21:08 ` [PATCH 59/68] help: clean up kfmclient munging Jeff King
2015-09-24 21:08 ` [PATCH 60/68] prefer memcpy to strcpy Jeff King
2015-09-27 11:19   ` René Scharfe
2015-09-27 13:06     ` Torsten Bögershausen
2015-09-27 13:13       ` René Scharfe
2015-09-27 13:24         ` René Scharfe
2015-09-28  7:09   ` Rasmus Villemoes
2015-09-24 21:08 ` [PATCH 61/68] color: add color_set helper for copying raw colors Jeff King
2015-09-24 21:08 ` [PATCH 62/68] notes: document length of fanout path with a constant Jeff King
2015-09-24 21:08 ` [PATCH 63/68] convert strncpy to memcpy Jeff King
2015-09-24 21:08 ` Jeff King [this message]
2015-09-24 21:08 ` [PATCH 65/68] Makefile: drop D_INO_IN_DIRENT build knob Jeff King
2015-09-24 21:08 ` [PATCH 66/68] fsck: use for_each_loose_file_in_objdir Jeff King
2015-09-26  3:36   ` Jeff King
2015-09-24 21:08 ` [PATCH 67/68] use strbuf_complete to conditionally append slash Jeff King
2015-09-24 21:08 ` [PATCH 68/68] name-rev: use strip_suffix to avoid magic numbers Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150924210828.GI30946@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).