From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by dcvr.yhbt.net (Postfix) with ESMTP id 71AB01F85D for ; Wed, 11 Jul 2018 12:51:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732925AbeGKMzX (ORCPT ); Wed, 11 Jul 2018 08:55:23 -0400 Received: from cloud.peff.net ([104.130.231.41]:55090 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1726457AbeGKMzX (ORCPT ); Wed, 11 Jul 2018 08:55:23 -0400 Received: (qmail 6599 invoked by uid 109); 11 Jul 2018 12:51:12 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Wed, 11 Jul 2018 12:51:12 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 6760 invoked by uid 111); 11 Jul 2018 12:51:13 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Wed, 11 Jul 2018 08:51:13 -0400 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Wed, 11 Jul 2018 08:51:10 -0400 Date: Wed, 11 Jul 2018 08:51:10 -0400 From: Jeff King To: Henning Schild Cc: git@vger.kernel.org, Eric Sunshine , Junio C Hamano , Martin =?utf-8?B?w4VncmVu?= , Ben Toews , Taylor Blau , "brian m . carlson" Subject: Re: [PATCH v2 9/9] gpg-interface t: extend the existing GPG tests with GPGSM Message-ID: <20180711125109.GC23835@sigill.intra.peff.net> References: <20180710170901.GH23624@sigill.intra.peff.net> <20180711123824.7e0be91a@md1pvb1c.ad001.siemens.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20180711123824.7e0be91a@md1pvb1c.ad001.siemens.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org On Wed, Jul 11, 2018 at 12:38:24PM +0200, Henning Schild wrote: > > Can we save a dummy generated key and just import it? That's what we > > do for the regular gpg case. > > I will look into storing a binary and leaving notes how it was > generated, just like regular gpg does. The reason i did not do that in > the first place is that x509 certs have a validity and we introduce > time into the picture. But i will see if i can generate epoch->infinity > to get the host clock or just the future out of the picture. That would be preferable. But even if we can just get something like a 10-year expiration, that may be enough. Somebody dealing with failing tests and regenerating keys in ten years is probably not the end of the world. It could hurt people with drastically incorrect system clocks, but I suspect there are other tests with similar problems (especially if your clock is in the past). > > We're going to have a lot of duplicated tests here. That's a > > maintenance burden when one of them needs fixes later. And when new > > tests are added, we won't automatically get them tested under each > > format. > > > > Can we move the battery of tests into a function that takes a few > > parameters (prereq name, branch to look at, etc) and then call it for > > both the gpg/gpgsm cases? > > I guess this is part of the earlier "allow GPGSM without GPG" and i > can ignore it if we agree that this is not needed? I think it's orthogonal. Even if GPGSM requires GPG, you'd still want to make sure that whatever exercise we give to the GPG code is also exercised using GPGSM. I will note, though, that _some_ tests are not really exercising gpg-specific bits, but more how we react to it (e.g., how we format %G placeholders). And it's probably OK for those to just be run once. So in an ideal world, the test script would probably look something like: # this function holds tests which exercise the interactions # with the gpg binary itself type_specific_tests() { prereq=$1 branch=$2 test_expect_success $prereq "test whatever ($prereq)" ' some test using $branch here ' } test_expect_success 'setup' ' set up both openpgp and x509 branches here ' type_specific_tests GPG openpgp type_specific_tests GPGSM x509 # and now tests that generically care about getting _some_ signature # result (e.g., the way we format signature info) # and then probably a few tests specific to how the config is handled, # like your new gpg.format coverage But in practice, the type-specific bits are often muddled together with the type-independent ones (e.g., we happen to test the parsing of a failed signature from gpg by checking how %G? is formatted or similar). So it may be simplest to just run most of the tests twice, once with gpg and once with gpgsm. I kind of wonder if all of t7510 could just be bumped into a function. Or even into a sourced file and run from two different scripts. See the way that t8001 and t8002 use annotate-tests.sh for an example. -Peff