From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS6315 166.70.0.0/16 X-Spam-Status: No, score=-3.7 required=3.0 tests=AWL,BAYES_00, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from out02.mta.xmission.com (out02.mta.xmission.com [166.70.13.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id E9EBC1F85E; Thu, 12 Jul 2018 22:19:21 +0000 (UTC) Received: from in01.mta.xmission.com ([166.70.13.51]) by out02.mta.xmission.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1fdjvj-0006Fj-Vl; Thu, 12 Jul 2018 16:19:20 -0600 Received: from [97.119.167.31] (helo=x220.xmission.com) by in01.mta.xmission.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1fdjvi-0001cI-PI; Thu, 12 Jul 2018 16:19:19 -0600 From: ebiederm@xmission.com (Eric W. Biederman) To: Konstantin Ryabitsev Cc: Eric Wong , meta@public-inbox.org References: <87a7r6z1cy.fsf@xmission.com> <20180705231346.GA6524@dcvr> <20180712183151.GA9085@chatter> Date: Thu, 12 Jul 2018 17:19:09 -0500 In-Reply-To: <20180712183151.GA9085@chatter> (Konstantin Ryabitsev's message of "Thu, 12 Jul 2018 14:31:51 -0400") Message-ID: <87va9k3xnm.fsf@xmission.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-SPF: eid=1fdjvi-0001cI-PI;;;mid=<87va9k3xnm.fsf@xmission.com>;;;hst=in01.mta.xmission.com;;;ip=97.119.167.31;;;frm=ebiederm@xmission.com;;;spf=neutral X-XM-AID: U2FsdGVkX190MfiIjNQAPkRocfsktZ+Al3jp0mxK49E= X-SA-Exim-Connect-IP: 97.119.167.31 X-SA-Exim-Mail-From: ebiederm@xmission.com Subject: Re: Warnings from git fsck after lkml import X-SA-Exim-Version: 4.2.1 (built Thu, 05 May 2016 13:38:54 -0600) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) List-Id: Konstantin Ryabitsev writes: > On Thu, Jul 05, 2018 at 11:13:46PM +0000, Eric Wong wrote: >>"Eric W. Biederman" wrote: >>> It looks like public-inbox has some challenges when importing some >>> questionable emails. The import of lkml has resulted in several commits >>> with bad dates that git fsck complains about. I have previously >>> reported this to Konstantin Ryabitsev who maintains kernel.org but since >>> I have not seen any discussion of this I thought I should report it >>> directly here as well. >> >>Thanks for bringing this up publically. >> >>Yes, I early during v2 development I noticed old mails had some >>-1400 timezone values (but the furthest is -1200). I opted to >>attempt to preserve the wonky timezones since fast-import >>happily accepts -1400 and I didn't anticipate problems... > > So, I can fix those in the archives, but this obviously requires > rebasing the whole repo, and I'm not sure what kind of impact that > would have. I'm assuming it's not sufficient to just fix the git repo, > as all commit IDs after the modified commit are going to be different > -- so additional changes to sqlite and xapian dbs would be required? Unless I am mistaken the cheap/clever version is to - Rebuild the 3 .git trees. - Notice that the object id's (aka sha1 hashes) of the emails remains the same. - Use sqlite3 to update the meta table of msgmap.sqlite3 My currently msgmap.sqlite3 contains: CREATE TABLE meta (key VARCHAR(32) PRIMARY KEY, val VARCHAR(255) NOT NULL); /* No STAT tables available */ sqlite> select * from meta; created_at|1530525399 last_xap15-6|c8f95c6728579303c200adbfb5469215da7e7836 last_xap15-5|31ed379430c456f90bdd172b223020c0e6d7cb8d last_xap15-4|88294f6d487193f5984791ee81213a25130d0559 last_xap15-3|93d9eace2721494d8457c7f5f6de803c0d648172 last_xap15-2|d48078ceeec1f51313253a56ed3ba0eae7fde909 last_xap15-1|6b67b9f5e0cd82d3c734e6cdc44c1f722ab6fb6a last_xap15-0|b67bf7f62c8125d67461cc6e7d1736ddc8844a18 Which matches the HEAD commits the lkml git trees. eric@x220:~/public-inbox/vger.kernel.org/linux-kernel-good$ git --git-dir=git/0.git show --pretty=oneline | head -1 b67bf7f62c8125d67461cc6e7d1736ddc8844a18 [-mm patch] drivers/firewire/: cleanups eric@x220:~/public-inbox/vger.kernel.org/linux-kernel-good$ git --git-dir=git/1.git show --pretty=oneline | head -1 6b67b9f5e0cd82d3c734e6cdc44c1f722ab6fb6a Re: [git patches] libata updates for 2.6.34 eric@x220:~/public-inbox/vger.kernel.org/linux-kernel-good$ git --git-dir=git/2.git show --pretty=oneline | head -1 d48078ceeec1f51313253a56ed3ba0eae7fde909 Re: linux-next: Tree for Jan 10 (staging/sb105x) eric@x220:~/public-inbox/vger.kernel.org/linux-kernel-good$ git --git-dir=git/3.git show --pretty=oneline | head -1 93d9eace2721494d8457c7f5f6de803c0d648172 Re: randconfig bug: ARM/KVM link error in hyp_idmap section eric@x220:~/public-inbox/vger.kernel.org/linux-kernel-good$ git --git-dir=git/4.git show --pretty=oneline | head -1 88294f6d487193f5984791ee81213a25130d0559 Re: [PATCH 2/2] sdhci-of-arasan: Set controller to test mode when fails-without-test-cd is present eric@x220:~/public-inbox/vger.kernel.org/linux-kernel-good$ git --git-dir=git/5.git show --pretty=oneline | head -1 31ed379430c456f90bdd172b223020c0e6d7cb8d Re: [PATCH 0/2] of: change overlay apply input data from EDT to FDT eric@x220:~/public-inbox/vger.kernel.org/linux-kernel-good$ git --git-dir=git/6.git show --pretty=oneline | head -1 c8f95c6728579303c200adbfb5469215da7e7836 [PATCH] slimbus: stream: Fix htmldocs warnings However all you have to do is ensure you preserve msgmap.sqlite3 and public-inbox-index is capable of rebuilding everything else. Eric