From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
"Johannes Schindelin" <johannes.schindelin@gmx.de>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH 4/4] fetch: implement fetch.fsck.*
Date: Thu, 24 May 2018 19:35:16 +0000 [thread overview]
Message-ID: <20180524193516.28713-5-avarab@gmail.com> (raw)
In-Reply-To: <20180524193516.28713-1-avarab@gmail.com>
In-Reply-To: <20180524190214.GA21354@sigill.intra.peff.net>
Implement support for fetch.fsck.* corresponding with the existing
receive.fsck.*. This allows for pedantically cloning repositories with
specific issues without turning off fetch.fsckObjects.
One such repository is https://github.com/robbyrussell/oh-my-zsh.git
which before this change will emit this error when cloned with
fetch.fsckObjects:
error: object 2b7227859263b6aabcc28355b0b994995b7148b6: zeroPaddedFilemode: contains zero-padded file modes
fatal: Error in object
fatal: index-pack failed
Now with fetch.fsck.zeroPaddedFilemode=warn we'll warn about that
issue, but the clone will succeed:
warning: object 2b7227859263b6aabcc28355b0b994995b7148b6: zeroPaddedFilemode: contains zero-padded file modes
warning: object a18c4d13c2a5fa2d4ecd5346c50e119b999b807d: zeroPaddedFilemode: contains zero-padded file modes
warning: object 84df066176c8da3fd59b13731a86d90f4f1e5c9d: zeroPaddedFilemode: contains zero-padded file modes
The motivation for this is to be able to turn on fetch.fsckObjects
globally across a fleet of computers but still be able to manually
clone various legacy repositories by either white-listing specific
issues, or better yet whitelist specific objects.
The use of --git-dir=* instead of -C in the tests could be considered
somewhat archaic, but the tests I'm adding here are duplicating the
corresponding receive.* tests with as few changes as possible.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
Documentation/config.txt | 21 +++++++++++----
fetch-pack.c | 32 +++++++++++++++++++++--
t/t5504-fetch-receive-strict.sh | 46 +++++++++++++++++++++++++++++++++
3 files changed, 92 insertions(+), 7 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 124f7a187c..af6187f6b5 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1426,6 +1426,16 @@ fetch.fsckObjects::
checked. Defaults to false. If not set, the value of
`transfer.fsckObjects` is used instead.
+fetch.fsck.<msg-id>::
+ Acts like `fsck.<msg-id>`, but is used by
+ linkgit:git-fetch-pack[1] instead of linkgit:git-fsck[1]. See
+ the `fsck.<msg-id>` documentation for details.
+
+fetch.fsck.skipList::
+ Acts like `fsck.skipList`, but is used by
+ linkgit:git-fetch-pack[1] instead of linkgit:git-fsck[1]. See
+ the `fsck.skipList` documentation for details.
+
fetch.unpackLimit::
If the number of objects fetched over the Git native
transfer is below this
@@ -1561,9 +1571,10 @@ fsck.<msg-id>::
repositories containing such data.
+
Setting `fsck.<msg-id>` will be picked up by linkgit:git-fsck[1], but
-to accept pushes of such data set `receive.fsck.<msg-id>` instead. The
-rest of the documentation discusses `fsck.*` for brevity, but the same
-applies for the corresponding `receive.fsck.*` variables.
+to accept pushes of such data set `receive.fsck.<msg-id>` instead, or
+to clone or fetch it set `fetch.fsck.<msg-id>`. The rest of the
+documentation discusses `fsck.*` for brevity, but the same applies for
+the corresponding `receive.fsck.*` and `fetch.<msg-id>.*`. variables.
+
When `fsck.<msg-id>` is set, errors can be switched to warnings and
vice versa by configuring the `fsck.<msg-id>` setting where the
@@ -1580,8 +1591,8 @@ hidden going forward, although that's unlikely to happen in practice
unless someone is being deliberately malicious.
fsck.skipList::
- Like `fsck.<msg-id>` this variable has a corresponding
- `receive.fsck.skipList` variant.
+ Like `fsck.<msg-id>` this variable has corresponding
+ `receive.fsck.skipList` and `fetch.fsck.skipList` variants.
+
The path to a sorted list of object names (i.e. one SHA-1 per line)
that are known to be broken in a non-fatal way and should be
diff --git a/fetch-pack.c b/fetch-pack.c
index 490c38f833..9e4282788e 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -19,6 +19,7 @@
#include "sha1-array.h"
#include "oidset.h"
#include "packfile.h"
+#include "fsck.h"
static int transfer_unpack_limit = -1;
static int fetch_unpack_limit = -1;
@@ -33,6 +34,7 @@ static int agent_supported;
static int server_supports_filtering;
static struct lock_file shallow_lock;
static const char *alternate_shallow_file;
+static struct strbuf fsck_msg_types = STRBUF_INIT;
/* Remember to update object flag allocation in object.h */
#define COMPLETE (1U << 0)
@@ -935,7 +937,8 @@ static int get_pack(struct fetch_pack_args *args,
*/
argv_array_push(&cmd.args, "--fsck-objects");
else
- argv_array_push(&cmd.args, "--strict");
+ argv_array_pushf(&cmd.args, "--strict%s",
+ fsck_msg_types.buf);
}
cmd.in = demux.out;
@@ -1409,6 +1412,31 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
return ref;
}
+static int fetch_pack_config_cb(const char *var, const char *value, void *cb)
+{
+ if (strcmp(var, "fetch.fsck.skiplist") == 0) {
+ const char *path;
+
+ if (git_config_pathname(&path, var, value))
+ return 1;
+ strbuf_addf(&fsck_msg_types, "%cskiplist=%s",
+ fsck_msg_types.len ? ',' : '=', path);
+ free((char *)path);
+ return 0;
+ }
+
+ if (skip_prefix(var, "fetch.fsck.", &var)) {
+ if (is_valid_msg_type(var, value))
+ strbuf_addf(&fsck_msg_types, "%c%s=%s",
+ fsck_msg_types.len ? ',' : '=', var, value);
+ else
+ warning("Skipping unknown msg id '%s'", var);
+ return 0;
+ }
+
+ return git_default_config(var, value, cb);
+}
+
static void fetch_pack_config(void)
{
git_config_get_int("fetch.unpacklimit", &fetch_unpack_limit);
@@ -1417,7 +1445,7 @@ static void fetch_pack_config(void)
git_config_get_bool("fetch.fsckobjects", &fetch_fsck_objects);
git_config_get_bool("transfer.fsckobjects", &transfer_fsck_objects);
- git_config(git_default_config, NULL);
+ git_config(fetch_pack_config_cb, NULL);
}
static void fetch_pack_setup(void)
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index 49d3621a92..b7f5222c2b 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -135,6 +135,20 @@ test_expect_success 'push with receive.fsck.skipList' '
git push --porcelain dst bogus
'
+test_expect_success 'fetch with fetch.fsck.skipList' '
+ commit="$(git hash-object -t commit -w --stdin <bogus-commit)" &&
+ refspec=refs/heads/bogus:refs/heads/bogus &&
+ git push . $commit:refs/heads/bogus &&
+ rm -rf dst &&
+ git init dst &&
+ git --git-dir=dst/.git config fetch.fsckObjects true &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+ git --git-dir=dst/.git config fetch.fsck.skipList dst/.git/SKIP &&
+ echo $commit >dst/.git/SKIP &&
+ git --git-dir=dst/.git fetch "file://$(pwd)" $refspec
+'
+
+
test_expect_success 'push with receive.fsck.missingEmail=warn' '
commit="$(git hash-object -t commit -w --stdin <bogus-commit)" &&
git push . $commit:refs/heads/bogus &&
@@ -155,6 +169,27 @@ test_expect_success 'push with receive.fsck.missingEmail=warn' '
! grep "missingEmail" act
'
+test_expect_success 'fetch with fetch.fsck.missingEmail=warn' '
+ commit="$(git hash-object -t commit -w --stdin <bogus-commit)" &&
+ refspec=refs/heads/bogus:refs/heads/bogus &&
+ git push . $commit:refs/heads/bogus &&
+ rm -rf dst &&
+ git init dst &&
+ git --git-dir=dst/.git config fetch.fsckobjects true &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+ git --git-dir=dst/.git config \
+ fetch.fsck.missingEmail warn &&
+ git --git-dir=dst/.git fetch "file://$(pwd)" $refspec >act 2>&1 &&
+ grep "missingEmail" act &&
+ git --git-dir=dst/.git branch -D bogus &&
+ git --git-dir=dst/.git config --add \
+ receive.fsck.missingEmail ignore &&
+ git --git-dir=dst/.git config --add \
+ receive.fsck.badDate warn &&
+ git --git-dir=dst/.git fetch "file://$(pwd)" $refspec >act 2>&1 &&
+ ! grep "missingEmail" act
+'
+
test_expect_success \
'receive.fsck.unterminatedHeader=warn triggers error' '
rm -rf dst &&
@@ -166,4 +201,15 @@ test_expect_success \
grep "Cannot demote unterminatedheader" act
'
+test_expect_success \
+ 'fetch.fsck.unterminatedHeader=warn triggers error' '
+ rm -rf dst &&
+ git init dst &&
+ git --git-dir=dst/.git config fetch.fsckobjects true &&
+ git --git-dir=dst/.git config \
+ fetch.fsck.unterminatedheader warn &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" HEAD &&
+ grep "Cannot demote unterminatedheader" act
+'
+
test_done
--
2.17.0.290.gded63e768a
next prev parent reply other threads:[~2018-05-24 19:35 UTC|newest]
Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-24 15:25 BUG: No way to set fsck.<msg-id> when cloning Ævar Arnfjörð Bjarmason
2018-05-24 15:58 ` Kevin Daudt
2018-05-24 17:04 ` Ævar Arnfjörð Bjarmason
2018-05-24 19:02 ` Jeff King
2018-05-24 19:35 ` [PATCH 0/4] fsck: doc fixes & fetch.fsck.* implementation Ævar Arnfjörð Bjarmason
2018-05-25 19:28 ` [PATCH v2 0/5] " Ævar Arnfjörð Bjarmason
2018-07-27 14:37 ` [PATCH v3 00/10] " Ævar Arnfjörð Bjarmason
2018-07-30 22:13 ` SZEDER Gábor
2018-07-27 14:37 ` [PATCH v3 01/10] receive.fsck.<msg-id> tests: remove dead code Ævar Arnfjörð Bjarmason
2018-07-27 19:11 ` Junio C Hamano
2018-07-27 19:45 ` Ævar Arnfjörð Bjarmason
2018-07-27 22:19 ` Junio C Hamano
2018-07-27 14:37 ` [PATCH v3 02/10] config doc: don't describe *.fetchObjects twice Ævar Arnfjörð Bjarmason
2018-07-27 19:19 ` Junio C Hamano
2018-07-27 14:37 ` [PATCH v3 03/10] config doc: unify the description of fsck.* and receive.fsck.* Ævar Arnfjörð Bjarmason
2018-07-27 19:29 ` Junio C Hamano
2018-07-27 14:37 ` [PATCH v3 04/10] config doc: elaborate on what transfer.fsckObjects does Ævar Arnfjörð Bjarmason
2018-07-27 19:41 ` Junio C Hamano
2018-07-27 14:37 ` [PATCH v3 05/10] config doc: elaborate on fetch.fsckObjects security Ævar Arnfjörð Bjarmason
2018-07-27 19:45 ` Junio C Hamano
2018-07-28 14:09 ` Ævar Arnfjörð Bjarmason
2018-07-27 14:37 ` [PATCH v3 06/10] transfer.fsckObjects tests: untangle confusing setup Ævar Arnfjörð Bjarmason
2018-07-27 14:37 ` [PATCH v3 07/10] fetch: implement fetch.fsck.* Ævar Arnfjörð Bjarmason
2018-07-27 20:18 ` Junio C Hamano
2018-07-27 21:08 ` Junio C Hamano
2018-07-30 14:58 ` Duy Nguyen
2018-07-30 15:06 ` Ævar Arnfjörð Bjarmason
2018-07-27 14:37 ` [PATCH v3 08/10] fsck: test & document {fetch,receive}.fsck.* config fallback Ævar Arnfjörð Bjarmason
2018-07-27 21:28 ` Junio C Hamano
2018-07-27 14:37 ` [PATCH v3 09/10] fsck: add stress tests for fsck.skipList Ævar Arnfjörð Bjarmason
2018-07-27 14:37 ` [PATCH v3 10/10] fsck: test and document unknown fsck.<msg-id> values Ævar Arnfjörð Bjarmason
2018-07-27 19:50 ` Ævar Arnfjörð Bjarmason
2018-07-27 21:43 ` Junio C Hamano
2018-07-28 13:55 ` Ævar Arnfjörð Bjarmason
2018-07-30 14:47 ` Junio C Hamano
2018-05-25 19:28 ` [PATCH v2 1/5] config doc: don't describe *.fetchObjects twice Ævar Arnfjörð Bjarmason
2018-05-25 21:07 ` Eric Sunshine
2018-05-25 19:28 ` [PATCH v2 2/5] config doc: unify the description of fsck.* and receive.fsck.* Ævar Arnfjörð Bjarmason
2018-05-25 21:16 ` Eric Sunshine
2018-05-28 9:45 ` Junio C Hamano
2018-05-28 16:44 ` Ævar Arnfjörð Bjarmason
2018-05-30 3:05 ` Junio C Hamano
2018-05-30 3:39 ` Junio C Hamano
2018-05-31 7:20 ` Ævar Arnfjörð Bjarmason
2018-06-01 0:11 ` Junio C Hamano
2018-05-25 19:28 ` [PATCH v2 3/5] config doc: elaborate on what transfer.fsckObjects does Ævar Arnfjörð Bjarmason
2018-05-25 21:19 ` Eric Sunshine
2018-05-25 19:28 ` [PATCH v2 4/5] config doc: mention future aspirations for transfer.fsckObjects Ævar Arnfjörð Bjarmason
2018-05-25 20:33 ` Christian Couder
2018-05-25 19:28 ` [PATCH v2 5/5] fetch: implement fetch.fsck.* Ævar Arnfjörð Bjarmason
2018-05-30 3:47 ` Junio C Hamano
2018-05-31 7:23 ` Ævar Arnfjörð Bjarmason
2018-05-28 9:48 ` [PATCH 0/4] fsck: doc fixes & fetch.fsck.* implementation Junio C Hamano
2018-05-24 19:35 ` [PATCH 1/4] config doc: don't describe *.fetchObjects twice Ævar Arnfjörð Bjarmason
2018-05-25 3:18 ` Junio C Hamano
2018-05-24 19:35 ` [PATCH 2/4] config doc: unify the description of fsck.* and receive.fsck.* Ævar Arnfjörð Bjarmason
2018-05-24 19:53 ` Eric Sunshine
2018-05-24 20:12 ` Ævar Arnfjörð Bjarmason
2018-05-24 22:49 ` Eric Sunshine
2018-05-25 2:07 ` Junio C Hamano
2018-05-24 19:35 ` [PATCH 3/4] config doc: elaborate on what transfer.fsckObjects does Ævar Arnfjörð Bjarmason
2018-05-24 20:15 ` Eric Sunshine
2018-05-25 3:22 ` Junio C Hamano
2018-05-31 7:32 ` Ævar Arnfjörð Bjarmason
2018-05-24 19:35 ` Ævar Arnfjörð Bjarmason [this message]
2018-05-25 4:09 ` [PATCH 4/4] fetch: implement fetch.fsck.* Junio C Hamano
2018-05-24 17:04 ` BUG: No way to set fsck.<msg-id> when cloning Jeff King
2018-05-24 20:48 ` Thomas Braun
2018-05-25 7:36 ` Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180524193516.28713-5-avarab@gmail.com \
--to=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=johannes.schindelin@gmx.de \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).