git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
* Stable GnuPG interface, git should use GPGME
@ 2017-03-10 10:00 Bernhard E. Reiter
  2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason
                   ` (2 more replies)
  0 siblings, 3 replies; 19+ messages in thread
From: Bernhard E. Reiter @ 2017-03-10 10:00 UTC (permalink / raw)
  To: git; +Cc: gnupg-devel

[-- Attachment #1: Type: text/plain, Size: 2459 bytes --]

Dear Git-Devs,

git uses an pipe-and-exec approach to running a GnuPG binary 
as writen in the documentation [1]:

    gpg.program
           Use this custom program instead of "gpg" found on $PATH when making
           or verifying a PGP signature. The program must support the same
           command-line interface as GPG

please consider using libgpgme interfacing to GnuPG, because the gpg 
command-line interface is not considered an official API to GnuPG by the 
GnuPG-devs and thus potentially unstable. 

== Details

I'm involved in GnuPG development. For most applications using libgpgme is the 
way what GnuPG-devs would recommend, also see 

  https://wiki.gnupg.org/APIs .

GnuPG devs are making a good effort of trying to keep the command-line 
interface stable, though it is not for sure. Git is only using a small part 
of the interface, so the risk when keeping the current way is small. 
Still I believe git's stability and usability would profit when moving to 
libgpgme, especially with the coming move to GnuPG 2.2, better diagnosing 
messages and for cross-plattform usage.

== Usability problem with `gpg2` vs `gpg`

My use case today was signing and git by default found the `gpg` binary by 
default and the command failed. The reason is that I have `gpg2` installed 
and most applications use it right away. So git failed signing because 
the .gnupg configuration of the user was not ready for the old `gpg` which is 
still installed on Debian GNU/Linux for purposes of the operating system. If 
git would have used libgpgme, gpgme would have choosen the most uptodate 
version of `gpg` available (or configured) without me intervening via 
gpg.program. Now because of this problem you could adding a check for `gpg2` 
and fallback to `gpg`, but even better would be to move to libgpgme. >:)

Best Regards and thanks for maintaining Git as Free Software,
Bernhard

== how to respond

ps: Please copy me on replies as I am not on git@vger.kernel.org. 
pps: I've copied gnupg-devel@ so they can see I've send this report, you don't 
have to.


[1] 
https://github.com/git/git/blob/3bc53220cb2dcf709f7a027a3f526befd021d858/Documentation/config.txt
search for 'gpg.program'.

-- 
www.intevation.de/~bernhard (CEO)     +49 541 33 508 3-3
Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998
Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter
@ 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason
  2017-03-13 10:14   ` Michael J Gruber
  2017-03-13 10:30   ` Bernhard Reiter
  2017-03-10 18:54 ` Linus Torvalds
  2017-03-11  0:10 ` brian m. carlson
  2 siblings, 2 replies; 19+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-03-10 14:23 UTC (permalink / raw)
  To: Bernhard E. Reiter
  Cc: Git Mailing List, gnupg-devel, Lukas Puehringer, Michael J Gruber,
	Linus Torvalds

On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter
<bernhard.reiter@intevation.de> wrote:
> Dear Git-Devs,

I haven't contributed to Git's GPG code, but I'm taking the liberty of
CC-ing some people who have.

> git uses an pipe-and-exec approach to running a GnuPG binary
> as writen in the documentation [1]:
>
>     gpg.program
>            Use this custom program instead of "gpg" found on $PATH when making
>            or verifying a PGP signature. The program must support the same
>            command-line interface as GPG
>
> please consider using libgpgme interfacing to GnuPG, because the gpg
> command-line interface is not considered an official API to GnuPG by the
> GnuPG-devs and thus potentially unstable.
>
> == Details
>
> I'm involved in GnuPG development. For most applications using libgpgme is the
> way what GnuPG-devs would recommend, also see
>
>   https://wiki.gnupg.org/APIs .
>
> GnuPG devs are making a good effort of trying to keep the command-line
> interface stable, though it is not for sure. Git is only using a small part
> of the interface, so the risk when keeping the current way is small.
> Still I believe git's stability and usability would profit when moving to
> libgpgme, especially with the coming move to GnuPG 2.2, better diagnosing
> messages and for cross-plattform usage.
>
> == Usability problem with `gpg2` vs `gpg`
>
> My use case today was signing and git by default found the `gpg` binary by
> default and the command failed. The reason is that I have `gpg2` installed
> and most applications use it right away. So git failed signing because
> the .gnupg configuration of the user was not ready for the old `gpg` which is
> still installed on Debian GNU/Linux for purposes of the operating system. If
> git would have used libgpgme, gpgme would have choosen the most uptodate
> version of `gpg` available (or configured) without me intervening via
> gpg.program. Now because of this problem you could adding a check for `gpg2`
> and fallback to `gpg`, but even better would be to move to libgpgme. >:)

I'm on Debian but haven't had these issues. What's your gpg & gpg2
--version & Debian release? And what in particular failed?

And what git version was this? I see we've had a couple of workarounds
for gpg2, in particular Linus's v2.8.4-1-gb624a3e67f, but if you have
v2.10.0 or later that won't fix whatever issue you had.

Using the library sounds good, but a shorter-term immediate fix would
be to figure out what bug you encountered in our use of the
command-line version, and see if we've fixed that already or not.
Regardless of what we do with a gpg library in the future some distros
might want to backport such a small patch if we can come up with it.

> Best Regards and thanks for maintaining Git as Free Software,
> Bernhard
>
> == how to respond
>
> ps: Please copy me on replies as I am not on git@vger.kernel.org.
> pps: I've copied gnupg-devel@ so they can see I've send this report, you don't
> have to.
>
>
> [1]
> https://github.com/git/git/blob/3bc53220cb2dcf709f7a027a3f526befd021d858/Documentation/config.txt
> search for 'gpg.program'.
>
> --
> www.intevation.de/~bernhard (CEO)     +49 541 33 508 3-3
> Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998
> Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter
  2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason
@ 2017-03-10 18:54 ` Linus Torvalds
  2017-03-10 20:26   ` Theodore Ts'o
  2017-03-13 11:14   ` Bernhard E. Reiter
  2017-03-11  0:10 ` brian m. carlson
  2 siblings, 2 replies; 19+ messages in thread
From: Linus Torvalds @ 2017-03-10 18:54 UTC (permalink / raw)
  To: Bernhard E. Reiter; +Cc: Git Mailing List, gnupg-devel

On Fri, Mar 10, 2017 at 2:00 AM, Bernhard E. Reiter
<bernhard.reiter@intevation.de> wrote:
>
> git uses an pipe-and-exec approach to running a GnuPG binary
> as writen in the documentation [1]:
>
>     gpg.program
>            Use this custom program instead of "gpg" found on $PATH when making
>            or verifying a PGP signature. The program must support the same
>            command-line interface as GPG
>
> please consider using libgpgme interfacing to GnuPG, because the gpg
> command-line interface is not considered an official API to GnuPG by the
> GnuPG-devs and thus potentially unstable.

Quite frankly, I will NAK this just based on previous bad experiences
with using "helpful" libraries.

Maybe you can lay my worries to rest, but the problems with libraries
in this context tend to be

 - hard to personalize.

   At least right now, we just allow people to set their gpg binary.
I'm betting that the library would pick whatever random preferred
version, and in the process possibly screwed us.

   Example: what if somebody is actually using another pgp
implementation entirely for some reason, and is just scripting around
it?

   Or maybe he's using the regular gnupg, but using different keys for
different projects (using "--homedir"). That's trivial with the
current model. How trivial is that with a library?

 - existing configuration

   This is the main problem I've seen in the past. Using the "ssh"
_program_ is easy. You add your keys, your config files, whatever, and
it "just works" (or rather, you fight it once and it definitely
doesn't "just" work, but then you copy your .ssh directory around for
the rest of your and forget how it ever worked, but it does).

   Using "libssh2" is an exercise in futility, and you have to do a
crazy amount of stupid "look up keys" and simple configuration in your
.ssh/config (like per-host keys, hostname swizzling etc) just don't
pick up the configurations you already did for the program.

 - UI

   For things like gpg, the UI is traditionally horrible. But there
tends to be various things like password agents that help with caching
passwords and/or add a graphical UI to get the password when
necessary.

 - library versioning.

   I don't know why, but I've never *ever* met a library developer who
realized that libraries were all about stable API's, and the library
users don't want to fight different versions.

   And to make matters worse, the different versions (particularly if
you end up having to use a development version due to bugs or required
features etc) are always made horribly bad to even detect at
built-time automatically with simple #ifdef etc, so now you have to do
autoconf crap etc.

Now, it may be that the pgpme library "just works" across
architectures and handles all of the above situations as gracefully as
the external program does. In that case - but _ONLY_ in that case -
would a switch-over to the library possibly be a good thing.

I'd be pleasantly surprised. But I *would* be surprised, because every
time I've seen that "library vs program" model, I've seen the above
issues.

In fact, we have those exact issues very much in git itself too. Yes,
I've used libgit2 (for subsurface). It's a pain in the arse to do
*exactly* the above kinds of things, and the thing is, that isn't
git-specific.

So I'm very down on using external libraries unless they are stable
and have no need for configuration etc. Things like zlib is fine -
there just isn't much to configure outside of the "just how hard do
you want me to try to compress". Nobody has a ".zlib/config" file that
you need to worry about accessing etc.

Of course, maybe pgpme is a world first, and actually does read your
.gnupg/config file trivially, and has all the gpg agent integration
that it picks up automatically, and allows various per-user
configurations, and all my worries are bogus.

But that would literally be the first time I've ever seen that.

                   Linus

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-10 18:54 ` Linus Torvalds
@ 2017-03-10 20:26   ` Theodore Ts'o
  2017-03-13 11:14   ` Bernhard E. Reiter
  1 sibling, 0 replies; 19+ messages in thread
From: Theodore Ts'o @ 2017-03-10 20:26 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Bernhard E. Reiter, Git Mailing List, gnupg-devel

On Fri, Mar 10, 2017 at 10:54:19AM -0800, Linus Torvalds wrote:
>  - library versioning.
> 
>    I don't know why, but I've never *ever* met a library developer who
> realized that libraries were all about stable API's, and the library
> users don't want to fight different versions.

Actually, you have.  (Raises hand :-)

libext2fs has a stable API *and* ABI.  We add new functions instead of
changing function parameters (so ext2fs_block_iterate2() is
implemented in terms of ext2fs_block_iterate3(), and so on).  And
structures have magic numbers that have served as versioning signal.
This is actually not rocket science.  If you've met anyone who's
programmed for Multics, they did something similar.  And of course,
that's why we have the wait3(2) and wait(4) system calls.

I do have to agree with your general point, that most developers tend
to be *incredibly* sloppy with their interfaces.  That being said, not
all library developers are as bad as GNOME.  :-)

    	    	       	      	     - Ted

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter
  2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason
  2017-03-10 18:54 ` Linus Torvalds
@ 2017-03-11  0:10 ` brian m. carlson
  2017-03-13 12:29   ` Bernhard E. Reiter
  2017-03-13 19:48   ` Christian Neukirchen
  2 siblings, 2 replies; 19+ messages in thread
From: brian m. carlson @ 2017-03-11  0:10 UTC (permalink / raw)
  To: Bernhard E. Reiter; +Cc: git, gnupg-devel

[-- Attachment #1: Type: text/plain, Size: 3122 bytes --]

On Fri, Mar 10, 2017 at 11:00:07AM +0100, Bernhard E. Reiter wrote:
> My use case today was signing and git by default found the `gpg` binary by
> default and the command failed. The reason is that I have `gpg2` installed
> and most applications use it right away. So git failed signing because
> the .gnupg configuration of the user was not ready for the old `gpg` which is
> still installed on Debian GNU/Linux for purposes of the operating system. If
> git would have used libgpgme, gpgme would have choosen the most uptodate
> version of `gpg` available (or configured) without me intervening via
> gpg.program. Now because of this problem you could adding a check for `gpg2`
> and fallback to `gpg`, but even better would be to move to libgpgme. >:)

There are a couple potential problems I see with this approach.  First,
I'd want to know whether gpgme supports gpgsm, which I know some people
use to sign commits and tags.

Another issue is what happens to the git verify-* --raw output.  Some
people want the ability to script signature verification.  This can be
really important when you have automated systems verifying tags and
commits.

For example, running the following commands, we can determine that Junio
signs his tags with SHA-1 (algorithm 2), while I sign my commits with
SHA-512 (algorithm 10).

genre ok % git verify-tag --raw v2.12.0 2>&1 | grep VALIDSIG
[GNUPG:] VALIDSIG E1F036B1FEE7221FC778ECEFB0B5E88696AFE6CB 2017-02-24 1487962205 0 4 0 1 2 00 96E07AF25771955980DAD10020D04E5A713660A7
genre ok % git verify-commit --raw object-id-part10 2>&1 | grep VALIDSIG
[GNUPG:] VALIDSIG 5FC3A781776B26DF87F70C37BF535D811F52F68B 2017-03-06 1488760639 0 4 0 1 10 00 88ACE9B29196305BA9947552F1BA225C0223B187

There's literally no other way to get this information at the moment
(which is why I added the --raw option).  A gpgme implementation would
need to expose this same information, at which point, we might as well
have used gpg directly.

This is not an idle consideration; we have automated systems at work
that update software automatically and submit it for human review,
including verifying signatures and hashes.  This saves hundreds of hours
of staff time and results in better security.

Because the amount of the gpg API we actually use is very small, a user
who wants to use a custom signature program (say, OpenBSD's signify),
can actually write a simple wrapper that mimics it and use that instead.

Finally, I work on a development system where work is done both as an
unprivileged user and as root.  Because I use the same socket for both,
GnuPG screams bloody murder that the permissions are wrong.  I know this
is secure in my scenario, but without a custom wrapper, I have to deal
with GnuPG polluting my terminal every time I sign a commit or a tag.  A
gpgme implementation would need to honor the same wrapper script or
otherwise not scream to the terminal.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 868 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason
@ 2017-03-13 10:14   ` Michael J Gruber
  2017-03-13 12:49     ` Bernhard E. Reiter
  2017-03-13 10:30   ` Bernhard Reiter
  1 sibling, 1 reply; 19+ messages in thread
From: Michael J Gruber @ 2017-03-13 10:14 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Bernhard E. Reiter
  Cc: Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds

Ævar Arnfjörð Bjarmason venit, vidit, dixit 10.03.2017 15:23:
> On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter
> <bernhard.reiter@intevation.de> wrote:
>> Dear Git-Devs,
> 
> I haven't contributed to Git's GPG code, but I'm taking the liberty of
> CC-ing some people who have.
> 
>> git uses an pipe-and-exec approach to running a GnuPG binary
>> as writen in the documentation [1]:
>>
>>     gpg.program
>>            Use this custom program instead of "gpg" found on $PATH when making
>>            or verifying a PGP signature. The program must support the same
>>            command-line interface as GPG
>>
>> please consider using libgpgme interfacing to GnuPG, because the gpg
>> command-line interface is not considered an official API to GnuPG by the
>> GnuPG-devs and thus potentially unstable.
>>
>> == Details
>>
>> I'm involved in GnuPG development. For most applications using libgpgme is the
>> way what GnuPG-devs would recommend, also see
>>
>>   https://wiki.gnupg.org/APIs .
>>
>> GnuPG devs are making a good effort of trying to keep the command-line
>> interface stable, though it is not for sure. Git is only using a small part
>> of the interface, so the risk when keeping the current way is small.
>> Still I believe git's stability and usability would profit when moving to
>> libgpgme, especially with the coming move to GnuPG 2.2, better diagnosing
>> messages and for cross-plattform usage.
>>
>> == Usability problem with `gpg2` vs `gpg`
>>
>> My use case today was signing and git by default found the `gpg` binary by
>> default and the command failed. The reason is that I have `gpg2` installed
>> and most applications use it right away. So git failed signing because
>> the .gnupg configuration of the user was not ready for the old `gpg` which is
>> still installed on Debian GNU/Linux for purposes of the operating system. If
>> git would have used libgpgme, gpgme would have choosen the most uptodate
>> version of `gpg` available (or configured) without me intervening via
>> gpg.program. Now because of this problem you could adding a check for `gpg2`
>> and fallback to `gpg`, but even better would be to move to libgpgme. >:)
> 
> I'm on Debian but haven't had these issues. What's your gpg & gpg2
> --version & Debian release? And what in particular failed?
> 
> And what git version was this? I see we've had a couple of workarounds
> for gpg2, in particular Linus's v2.8.4-1-gb624a3e67f, but if you have
> v2.10.0 or later that won't fix whatever issue you had.
> 
> Using the library sounds good, but a shorter-term immediate fix would
> be to figure out what bug you encountered in our use of the
> command-line version, and see if we've fixed that already or not.
> Regardless of what we do with a gpg library in the future some distros
> might want to backport such a small patch if we can come up with it.

As far as I know, Git handles different GPG versions just fine.

The problem is the "difficult" upgrade path and mixed installations with
gpg and gpg2.1+ that some distributions force upon you:

As soon as you start gpg2.1, your (secret) key store is migrated to a
new format without technically invalidating it. Similarly, users may
enter gpg2.1+-only comand in the config that is actually shared with
gpg, throwing off any use of gpg - not just by git, but also by anything
that your distro requires gpg for (such as packaging tools and the like).

In short: Users will run into problems anyway; git provides the quick
way out (git config gpg.program gpg2), users won't be as lucky with
other things that require gpg.

As for the library: While - technically speaking - the command line is
not a stable API for gpg, it does work across versions of gpg, and gpg
2.2 will be the first real stable branch that uses the new key store
layout. So I'd rather wait for that to stabilize before going away from
what turned out to be most stable so far.

Note that we (git) refrain from parsing ordinary output/return codes of
gpg and use status-fd as we should (and as documented).

Michael

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason
  2017-03-13 10:14   ` Michael J Gruber
@ 2017-03-13 10:30   ` Bernhard Reiter
  1 sibling, 0 replies; 19+ messages in thread
From: Bernhard Reiter @ 2017-03-13 10:30 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Git Mailing List, gnupg-devel, Lukas Puehringer, Michael J Gruber,
	Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 3870 bytes --]

Am Freitag 10 März 2017 15:23:27 schrieb Ævar Arnfjörð Bjarmason:
> On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter wrote
> > please consider using libgpgme interfacing to GnuPG, because the gpg
> > command-line interface is not considered an official API to GnuPG by the
> > GnuPG-devs and thus potentially unstable.

> > == Usability problem with `gpg2` vs `gpg`
> >
> > My use case today was signing and git by default found the `gpg` binary
> > by default and the command failed.

I've mentioned this as one example for a possible advantage using libgpgme
when interfacing with GnuPG.

> > The reason is that I have `gpg2` 
> > installed and most applications use it right away. So git failed signing
> > because the .gnupg configuration of the user was not ready for the old
> > `gpg` which is still installed on Debian GNU/Linux for purposes of the
> > operating system. If git would have used libgpgme, gpgme would have
> > choosen the most uptodate version of `gpg` available (or configured)
> > without me intervening via gpg.program. Now because of this problem you
> > could adding a check for `gpg2` and fallback to `gpg`, but even better
> > would be to move to libgpgme. >:)
>
> I'm on Debian but haven't had these issues. What's your gpg & gpg2
> --version & Debian release? And what in particular failed?

If you use options in your configuration that only gpg2 understands, gpg(1)
will barf. For example the following lines in ~/.gnupg/gpg.conf

  debug-level basic
  log-file socket:///home/bern/.gnupg/log-socket

will lead to
LANG=C gpg -K
  gpg: /powerhome/bern/.gnupg/gpg.conf:102: argument not expected
  gpg: /powerhome/bern/.gnupg/gpg.conf:103: invalid option
where gpg2 works as expected.

As a number of application already uses gpg2 (via libgpgme or not), this may 
go unnoticed for a while. So when I've started to sign with git on this 
machine I ran into the problem (current Jessie default versions):

  dpkg -s gnupg | grep ^Version
  #Version: 1.4.18-7+deb8u3
  dpkg -s gnupg2 | grep ^Version
  #Version: 2.0.26-6+deb8u1

Workarounds are:
 * Use a different config for gpg2 and gpg, e.g. ~/.gnupg/gpg.conf-2
   (https://www.gnupg.org/documentation/manuals/gnupg-2.0/Invoking-GPG.html )
 * or set gpg.program for git to gpg2.

> And what git version was this? I see we've had a couple of workarounds
> for gpg2, in particular Linus's v2.8.4-1-gb624a3e67f, but if you have
> v2.10.0 or later that won't fix whatever issue you had.

dpkg -s git | grep ^Version
Version: 1:2.1.4-2.1+deb8u2
(I've checked the most current master source to see that git still calls 
gpg.program, otherwise followed advise on https://git-scm.com/community
to send reports and questions to the list.)

> Using the library sounds good, but a shorter-term immediate fix would
> be to figure out what bug you encountered in our use of the
> command-line version, and see if we've fixed that already or not.
> Regardless of what we do with a gpg library in the future some distros
> might want to backport such a small patch if we can come up with it.

I guess a good simple approach would be to try "gpg2" first and then fall back 
to "gpg" or "gpgv" in case only these version are available.
(Here is a report that puts forward using gpgv in some situations 
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852684 )

As there are other subtle potential issues with directly calling a gpg binary,
using libgpgme by default probably has other advantages as well. And if there 
are important functions missing the GnuPG-devs would like to hear about them.

Regards,
Bernhard

-- 
www.intevation.de/~bernhard   +49 541 33 508 3-3
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-10 18:54 ` Linus Torvalds
  2017-03-10 20:26   ` Theodore Ts'o
@ 2017-03-13 11:14   ` Bernhard E. Reiter
  2017-03-13 12:53     ` Jeff King
  1 sibling, 1 reply; 19+ messages in thread
From: Bernhard E. Reiter @ 2017-03-13 11:14 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Git Mailing List, gnupg-devel

[-- Attachment #1: Type: text/plain, Size: 6185 bytes --]

Am Freitag 10 März 2017 19:54:19 schrieb Linus Torvalds:
> On Fri, Mar 10, 2017 at 2:00 AM, Bernhard E. Reiter

> > please consider using libgpgme interfacing to GnuPG, because the gpg
> > command-line interface is not considered an official API to GnuPG by the
> > GnuPG-devs and thus potentially unstable.
>
> Quite frankly, I will NAK this just based on previous bad experiences
> with using "helpful" libraries.
>
> Maybe you can lay my worries to rest, but the problems with libraries
> in this context tend to be

As gpgme is not just a helpful library, but the official API to GnuPG, it is 
well supported by the GnuPG-Initiative itself and stable. Still there could 
be problems and of course in some situations the disadvantages outweigh the 
advantages. On the other hand we have seen a number of systematic problems 
with "just calling gpg" that libgpgme tries to provide a solution to.

So it is too early to say that libgpgme would be right choice for git to me, 
but it should be seriously considered. Grateful that you have written down 
some of your concern, let me try give you some pointers.

>  - hard to personalize.
>
>    At least right now, we just allow people to set their gpg binary.
> I'm betting that the library would pick whatever random preferred
> version, and in the process possibly screwed us.
>
>    Example: what if somebody is actually using another pgp
> implementation entirely for some reason, and is just scripting around
> it?

>    Or maybe he's using the regular gnupg, but using different keys for
> different projects (using "--homedir"). That's trivial with the
> current model. How trivial is that with a library?

https://www.gnupg.org/documentation/manuals/gpgme/Engine-Configuration.html
"
You can change the configuration of a backend engine, and thus change the 
executable program and configuration directory to be used. You can make these 
changes the default or set them for some contexts individually. 
"

Using a completely different OpenPGP implementation maybe a potential use case 
for keeping a configuration option around. I did not deeply examine what git 
really needs. Usually a different implementation will have quite a different 
command line interface, so it may require substaintial work to come up with a 
wrapper about that other OpenPGP implementation to provide the same command 
line interface as GnuPG.

>  - existing configuration
>
>    This is the main problem I've seen in the past. Using the "ssh"
> _program_ is easy. You add your keys, your config files, whatever, and
> it "just works" (or rather, you fight it once and it definitely
> doesn't "just" work, but then you copy your .ssh directory around for
> the rest of your and forget how it ever worked, but it does).

gpgme via gpg uses the existing configuration files (which you can also read 
and modify with gpgconf for implementiong GUIs).

>  - UI
>
>    For things like gpg, the UI is traditionally horrible. But there
> tends to be various things like password agents that help with caching
> passwords and/or add a graphical UI to get the password when
> necessary.

As the gpg binary itself speaks to gpg-agent, this is fully integrated when 
used via gpgme. Our GPA and Kleopatra GUIs work fine with gpgme and others as 
well, because they call come together in the deeper engine functions of 
GnuPG.

>  - library versioning.
>
>    I don't know why, but I've never *ever* met a library developer who
> realized that libraries were all about stable API's, and the library
> users don't want to fight different versions.

In my experience Werner (the lead GnuPG developers) is quite reasonable about 
keeping APIs stable (he often goes out of his way to keep even the command 
line version stable, maybe he shouldn't do that to the command line options 
so you are more motivated to go to this official API gpgme. >:) )

>    And to make matters worse, the different versions (particularly if
> you end up having to use a development version due to bugs or required
> features etc) are always made horribly bad to even detect at
> built-time automatically with simple #ifdef etc, so now you have to do
> autoconf crap etc.

https://www.gnupg.org/documentation/manuals/gpgme/Library-Version-Check.html
"
The function gpgme_check_version has four purposes. It can be used to retrieve 
the version number of the library. In addition it can verify that the version 
number is higher than a certain required version number. In either case, the 
function initializes some sub-systems, and for this reason alone it must be 
invoked early in your program, before you make use of the other functions in 
GPGME. The last purpose is to run selftests.

As a side effect for W32 based systems, the socket layer will get initialized. 
"

> Now, it may be that the pgpme library "just works" across
> architectures and handles all of the above situations as gracefully as
> the external program does. In that case - but _ONLY_ in that case -
> would a switch-over to the library possibly be a good thing.

At least gpgme aims to fulfill these goals (and is used on many 
architectures).

> I'd be pleasantly surprised. But I *would* be surprised, because every
> time I've seen that "library vs program" model, I've seen the above
> issues.

Your concerns are understandable, I've seen similiar problems with "library vs 
program" and the unix tools box approach gives a number of lessons on how to 
losely couple components. Thanks again for taking the time and writing them 
down. I've given you some pointers why gpgme indeed could be different and 
may be an improvement for git (or other applications). I guess one of the 
next steps would be for someone to look for specific points or try gpgme for 
git purposes. Me and gnupg-devel@ are happy to take your questions or get 
feedback.

Regards,
Bernhard


-- 
www.intevation.de/~bernhard (CEO)     +49 541 33 508 3-3
Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998
Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-11  0:10 ` brian m. carlson
@ 2017-03-13 12:29   ` Bernhard E. Reiter
  2017-03-13 19:48   ` Christian Neukirchen
  1 sibling, 0 replies; 19+ messages in thread
From: Bernhard E. Reiter @ 2017-03-13 12:29 UTC (permalink / raw)
  To: brian m. carlson; +Cc: git, gnupg-devel

[-- Attachment #1: Type: text/plain, Size: 3510 bytes --]

Am Samstag 11 März 2017 01:10:31 schrieb brian m. carlson:
> On Fri, Mar 10, 2017 at 11:00:07AM +0100, Bernhard E. Reiter wrote:
> >  but even better would be to move to libgpgme. >:)
>
> There are a couple potential problems I see with this approach.  First,
> I'd want to know whether gpgme supports gpgsm, which I know some people
> use to sign commits and tags.

Yes, gpgme supports gpgsm.
https://www.gnupg.org/documentation/manuals/gpgme/Cryptographic-Message-Syntax.html
"CMS is implemented by GpgSM, the S/MIME implementation for GnuPG."

> Another issue is what happens to the git verify-* --raw output.  Some
> people want the ability to script signature verification.  This can be
> really important when you have automated systems verifying tags and
> commits.
>
> For example, running the following commands, we can determine that Junio
> signs his tags with SHA-1 (algorithm 2), while I sign my commits with
> SHA-512 (algorithm 10).
>
> genre ok % git verify-tag --raw v2.12.0 2>&1 | grep VALIDSIG
> [GNUPG:] VALIDSIG E1F036B1FEE7221FC778ECEFB0B5E88696AFE6CB 2017-02-24
> 1487962205 0 4 0 1 2 00 96E07AF25771955980DAD10020D04E5A713660A7 genre ok %
> git verify-commit --raw object-id-part10 2>&1 | grep VALIDSIG [GNUPG:]
> VALIDSIG 5FC3A781776B26DF87F70C37BF535D811F52F68B 2017-03-06 1488760639 0 4
> 0 1 10 00 88ACE9B29196305BA9947552F1BA225C0223B187
>
> There's literally no other way to get this information at the moment
> (which is why I added the --raw option).  A gpgme implementation would
> need to expose this same information, at which point, we might as well
> have used gpg directly.

I'm quite optimistic that the use case can be covered implementing the 
git --raw option using gpgme, but I don't know the best way right away.
(I also wonder what would happen if someone manages to put in two or more 
signatures in the object you are verifying.)

> Because the amount of the gpg API we actually use is very small, a user
> who wants to use a custom signature program (say, OpenBSD's signify),
> can actually write a simple wrapper that mimics it and use that instead.

I agree that this can be a use case against using libgpgme.

> Finally, I work on a development system where work is done both as an
> unprivileged user and as root.  Because I use the same socket for both,
> GnuPG screams bloody murder that the permissions are wrong.  I know this
> is secure in my scenario, but without a custom wrapper, I have to deal
> with GnuPG polluting my terminal every time I sign a commit or a tag.  A
> gpgme implementation would need to honor the same wrapper script or
> otherwise not scream to the terminal.

I don't understand  the scenario well enought to advise. Using the same socket 
sounds strange at the onset. In general libgpgme allows someone to use other 
binaries (see my answer to Linus) and offers quite a number of possibilities. 
However there may be special cases that are not covered as good as using 
everything raw and manually. Libgpgme has to be this way and offering some 
standard default otherwise it would use some of its merits. On the other hand
the code doing special things or calling gpg directly can have defects as well
which is a significant drawback.

Regards,
Bernhard
-- 
www.intevation.de/~bernhard (CEO)     +49 541 33 508 3-3
Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998
Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-13 10:14   ` Michael J Gruber
@ 2017-03-13 12:49     ` Bernhard E. Reiter
  2017-03-14 10:39       ` Michael J Gruber
  0 siblings, 1 reply; 19+ messages in thread
From: Bernhard E. Reiter @ 2017-03-13 12:49 UTC (permalink / raw)
  To: Michael J Gruber
  Cc: Ævar Arnfjörð Bjarmason, Git Mailing List,
	gnupg-devel, Lukas Puehringer, Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 3047 bytes --]

Am Montag 13 März 2017 11:14:57 schrieb Michael J Gruber:
> Ævar Arnfjörð Bjarmason venit, vidit, dixit 10.03.2017 15:23:
> > On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter

> >> please consider using libgpgme interfacing to GnuPG, because the gpg
> >> command-line interface is not considered an official API to GnuPG by the
> >> GnuPG-devs and thus potentially unstable.

[example of gpg2 vs gpg option incompatibility cut]

> > Using the library sounds good, but a shorter-term immediate fix would
> > be to figure out what bug you encountered in our use of the
> > command-line version, and see if we've fixed that already or not.

> As far as I know, Git handles different GPG versions just fine.

As mentioned before: explicitely setting gpg.program to gpg2 helps if gpg
chokes on the new config. Trying the `gpg2` binary first can be a simple fix. 
Using libgpgme potentially solves this and other compatility options.

> The problem is the "difficult" upgrade path and mixed installations with
> gpg and gpg2.1+ that some distributions force upon you:
>
> As soon as you start gpg2.1, your (secret) key store is migrated to a
> new format without technically invalidating it. Similarly, users may
> enter gpg2.1+-only comand in the config that is actually shared with
> gpg, throwing off any use of gpg - not just by git, but also by anything
> that your distro requires gpg for (such as packaging tools and the like).

Yes, this is another example why trying `gpg2? first by default or using 
libgpgme keeps trouble away from users.

> In short: Users will run into problems anyway; git provides the quick
> way out (git config gpg.program gpg2), users won't be as lucky with
> other things that require gpg.

Application using libgpgme will behave fine and many user facing components 
use it already. 

> As for the library: While - technically speaking - the command line is
> not a stable API for gpg, it does work across versions of gpg, and gpg

... to some extend.

> 2.2 will be the first real stable branch that uses the new key store
> layout. So I'd rather wait for that to stabilize before going away from
> what turned out to be most stable so far.

It is not just about the key-store change as mentioned before. However
I agree that a potential switch should be done with a current version of gpgme 
that already has support for GnuPG 2.1/2, e.g. gpgme v1.8.0.

> Note that we (git) refrain from parsing ordinary output/return codes of
> gpg and use status-fd as we should (and as documented).

It is good to use --status-fd and --with-colons when calling gpg, you still 
have to parse the results of status-fd as described in doc/DETAILs. 
https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=doc/DETAILS;hb=HEAD

Regards,
Bernhard

-- 
www.intevation.de/~bernhard (CEO)     +49 541 33 508 3-3
Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998
Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-13 11:14   ` Bernhard E. Reiter
@ 2017-03-13 12:53     ` Jeff King
  0 siblings, 0 replies; 19+ messages in thread
From: Jeff King @ 2017-03-13 12:53 UTC (permalink / raw)
  To: Bernhard E. Reiter; +Cc: Linus Torvalds, Git Mailing List, gnupg-devel

On Mon, Mar 13, 2017 at 12:14:17PM +0100, Bernhard E. Reiter wrote:

> Using a completely different OpenPGP implementation maybe a potential use case 
> for keeping a configuration option around. I did not deeply examine what git 
> really needs. Usually a different implementation will have quite a different 
> command line interface, so it may require substaintial work to come up with a 
> wrapper about that other OpenPGP implementation to provide the same command 
> line interface as GnuPG.

The existing config option will have to stay, period. If we take it
away, then we'd be breaking existing setups that do anything more exotic
than point it at a particular gpg binary.

So I think the possible patches in this area are:

  - adding gpgme support (probably as an optional build-time dependency)
    along with a config option to use it instead of gpg.program (or
    possibly just "gpg.program = gpgme" to trigger it).

  - defaulting to gpgme (when it's compiled in) instead of gpg.program

The first one by itself shouldn't hurt anybody, aside from carrying some
extra code around.

We'd have to see how the first one goes before considering the second.
If we hit a point where people have problems and the advice is "just set
gpg.program to gpgme", then that's a good sign that the default should
be flipped.

-Peff

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-11  0:10 ` brian m. carlson
  2017-03-13 12:29   ` Bernhard E. Reiter
@ 2017-03-13 19:48   ` Christian Neukirchen
  1 sibling, 0 replies; 19+ messages in thread
From: Christian Neukirchen @ 2017-03-13 19:48 UTC (permalink / raw)
  To: git

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> Because the amount of the gpg API we actually use is very small, a user
> who wants to use a custom signature program (say, OpenBSD's signify),
> can actually write a simple wrapper that mimics it and use that instead.

FWIW, I did this, and it was quite some effort to figure out the
actual API that is needed:

http://chneukirchen.org/dotfiles/bin/git-signify

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-13 12:49     ` Bernhard E. Reiter
@ 2017-03-14 10:39       ` Michael J Gruber
  2017-03-17  9:56         ` Bernhard E. Reiter
  0 siblings, 1 reply; 19+ messages in thread
From: Michael J Gruber @ 2017-03-14 10:39 UTC (permalink / raw)
  To: Bernhard E. Reiter
  Cc: Ævar Arnfjörð Bjarmason, Git Mailing List,
	gnupg-devel, Lukas Puehringer, Linus Torvalds

Bernhard E. Reiter venit, vidit, dixit 13.03.2017 13:49:
> Am Montag 13 März 2017 11:14:57 schrieb Michael J Gruber:
>> Ævar Arnfjörð Bjarmason venit, vidit, dixit 10.03.2017 15:23:
>>> On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter
> 
>>>> please consider using libgpgme interfacing to GnuPG, because the gpg
>>>> command-line interface is not considered an official API to GnuPG by the
>>>> GnuPG-devs and thus potentially unstable.
> 
> [example of gpg2 vs gpg option incompatibility cut]
> 
>>> Using the library sounds good, but a shorter-term immediate fix would
>>> be to figure out what bug you encountered in our use of the
>>> command-line version, and see if we've fixed that already or not.
> 
>> As far as I know, Git handles different GPG versions just fine.
> 
> As mentioned before: explicitely setting gpg.program to gpg2 helps if gpg
> chokes on the new config. Trying the `gpg2` binary first can be a simple fix. 
> Using libgpgme potentially solves this and other compatility options.
> 
>> The problem is the "difficult" upgrade path and mixed installations with
>> gpg and gpg2.1+ that some distributions force upon you:
>>
>> As soon as you start gpg2.1, your (secret) key store is migrated to a
>> new format without technically invalidating it. Similarly, users may
>> enter gpg2.1+-only comand in the config that is actually shared with
>> gpg, throwing off any use of gpg - not just by git, but also by anything
>> that your distro requires gpg for (such as packaging tools and the like).
> 
> Yes, this is another example why trying `gpg2? first by default or using 
> libgpgme keeps trouble away from users.

No, not at all. On the contrary: Using gpg2.1 without being asked to by
the user will migrate his key store! This can lead to tremendous
problems when a user manages his secret key store using gpg and git uses
the other secret key store (via gpg2.1)!

>> In short: Users will run into problems anyway; git provides the quick
>> way out (git config gpg.program gpg2), users won't be as lucky with
>> other things that require gpg.
> 
> Application using libgpgme will behave fine and many user facing components 
> use it already. 
> 
>> As for the library: While - technically speaking - the command line is
>> not a stable API for gpg, it does work across versions of gpg, and gpg
> 
> ... to some extend.

I have no idea what you're implying here - but I have a pretty good idea
of what works in current git and what not, including gpg usage (which is
the qualifier that you cut out).

>> 2.2 will be the first real stable branch that uses the new key store
>> layout. So I'd rather wait for that to stabilize before going away from
>> what turned out to be most stable so far.
> 
> It is not just about the key-store change as mentioned before. However
> I agree that a potential switch should be done with a current version of gpgme 
> that already has support for GnuPG 2.1/2, e.g. gpgme v1.8.0.

Unfortunately, gpgme does not solve the interoperability problems
between gpg (1, classic, stable maint mode) or gpg2.0 (stable) and
gpg2.1 (modern) key stores, and gpg2.2 (modern, stable) is not released yet.

>> Note that we (git) refrain from parsing ordinary output/return codes of
>> gpg and use status-fd as we should (and as documented).
> 
> It is good to use --status-fd and --with-colons when calling gpg, you still 
> have to parse the results of status-fd as described in doc/DETAILs. 
> https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=doc/DETAILS;hb=HEAD

Again, have you checked what we are doing in git land?

Your agenda is pretty clear. It's also pretty clear from the above that
gpgme is not the solution to the problem which is introduced by the
migration path to versions of gpg which are not declared stable by the
gpg project, away from a gpg version which is not obsoleted by the gpg
project (2.0, maybe 1).

And, really, key store migration was the only "major" thing we had to
think about to cope with gpg 2.1+ - it affected the test suite setup.

So, once 2.2 is out and stable and mainstream and we don't risk
migrating users' secret key store sneakily I'm all for defaulting to
gpg2, and then is a good time for us to look into gpgme.

Michael

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-14 10:39       ` Michael J Gruber
@ 2017-03-17  9:56         ` Bernhard E. Reiter
  2017-03-22 17:15           ` Werner Koch
  0 siblings, 1 reply; 19+ messages in thread
From: Bernhard E. Reiter @ 2017-03-17  9:56 UTC (permalink / raw)
  To: Michael J Gruber, Werner Koch
  Cc: Ævar Arnfjörð Bjarmason, Git Mailing List,
	gnupg-devel, Lukas Puehringer, Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 6648 bytes --]

Am Dienstag 14 März 2017 11:39:12 schrieb Michael J Gruber:
> >> The problem is the "difficult" upgrade path and mixed installations with
> >> gpg and gpg2.1+ that some distributions force upon you:
> >>
> >> As soon as you start gpg2.1, your (secret) key store is migrated to a
> >> new format without technically invalidating it. Similarly, users may
> >> enter gpg2.1+-only comand in the config that is actually shared with
> >> gpg, throwing off any use of gpg - not just by git, but also by anything
> >> that your distro requires gpg for (such as packaging tools and the
> >> like).
> >
> > Yes, this is another example why trying `gpg2? first by default or using
> > libgpgme keeps trouble away from users.
>
> No, not at all. On the contrary: Using gpg2.1 without being asked to by
> the user will migrate his key store! This can lead to tremendous
> problems when a user manages his secret key store using gpg and git uses
> the other secret key store (via gpg2.1)!

In the following scenario I agree with you:
* The user only uses gpg (1) and only applications that use gpg (1)
   and does not have gpg2 installed.
* Now a GnuPG package for 2.1/2.2 is installed and only the "gpg2"
  binary now points to it.
* If then git will just switches using `gpg2` which is v2.1 because it 
  came available, it will force the migration to the new store.

My statement was assuming a scenario where
* the user has gpg v1 and gpg2 v2.0 installed and many (or all) applications 
  use gpg2 via gpgme or by other means. I consider this typical for desktops
  or application servers using for desktop or development work.
* Now git also uses gpg2 (via gpgme or by other means, a possible improvement 
   we are just talking about)
* If GnuPG 2.1/2.2 becomes available and of the already used application will
  trigger the keystore migration. git users will have a smooth experience
  because they were using gpg2 all along and then to use the new keystore.

(It is still possible for some to configure libgpgme to use `gpg` v1 and all 
applications using gpgme will use it and you do not have to configure each 
one separately. Because you should either use gpg v2.1 or gpg v1 
exclusively, it makes sense to configure it close to gpg in its official api 
and not within each application. Again you can deviate from this.)

> >> As for the library: While - technically speaking - the command line is
> >> not a stable API for gpg, it does work across versions of gpg, and gpg
> >
> > ... to some extend.
>
> I have no idea what you're implying here - but I have a pretty good idea
> of what works in current git and what not, including gpg usage (which is
> the qualifier that you cut out).

As the command line is not a stable API to GnuPG, there were changes and there 
will be changes to the command line over several versions. It maybe in the 
past that git's usage of the command line was stable and works well, because 
of the subset of functionality of GnuPG that git is using and good work done 
by you and other developers of the git GnuPG support. It may or may not be a 
good choice for the future. As I've written before I'm not familiar enough 
with git to make a final call here, but there are some signs and examples 
where using gpgme may be an improvements in the areas:

* better configuration which gpg and which options to use for all applications
* consistent behaviour from git with other applications
* more robust programming interface over several gpg versions
* (better performance when reusing the same gpgme "context" for many 
consecutive crypto operations)

> Unfortunately, gpgme does not solve the interoperability problems
> between gpg (1, classic, stable maint mode) or gpg2.0 (stable) and
> gpg2.1 (modern) key stores, and gpg2.2 (modern, stable) is not released
> yet.

That is right, I also believe gpgme does not solve all interoperability 
problems. I guess it solves some, but I would do more research to come up 
with some specific examples.

> >> Note that we (git) refrain from parsing ordinary output/return codes of
> >> gpg and use status-fd as we should (and as documented).
> >
> > It is good to use --status-fd and --with-colons when calling gpg, you
> > still have to parse the results of status-fd as described in doc/DETAILs.
> > https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=doc/DETAILS
> >;hb=HEAD
>
> Again, have you checked what we are doing in git land?

As mentioned before: Only on the surface. I'm not deeply familiar with all git 
use case of GnuPG. 

> Your agenda is pretty clear.

Yes, I'd like to improve the user experience of git users regarding crypto
operations done with GnuPG. There are some signs that indicate that using 
gpgme may potentially improve this user experience (as outlined).

> It's also pretty clear from the above that 
> gpgme is not the solution to the problem which is introduced by the
> migration path to versions of gpg which are not declared stable by the
> gpg project, away from a gpg version which is not obsoleted by the gpg
> project (2.0, maybe 1).

While I agree that gpgme does not solve all problems with this migration,
it solves some problems with it (see above and my other email) 
and offers other advantages as the official API of GnuPG. Still it is the call 
of people more familiar with git usage and use cases to see if they have a 
good reason to deviate from the common case and use a non-official way of 
automating crypto operations with GnuPG.

> And, really, key store migration was the only "major" thing we had to
> think about to cope with gpg 2.1+ - it affected the test suite setup.
>
> So, once 2.2 is out and stable and mainstream and we don't risk
> migrating users' secret key store sneakily I'm all for defaulting to
> gpg2, and then is a good time for us to look into gpgme.

The key store migration is mainly an orthogonal issue and the problems will 
happen with or without using gpgme. As GnuPG 2.2 is under way, it makes sense 
to look into an improved support now, to be ready if GnuPG 2.2 becomes more 
widely used. It probably will get a lot more usage this year because of the 
ECC support (and some other reasons).

Thanks all for your thoughts, using GnuPG, considering gpgme 
and in general for maintaining git as Free Software! :)

Best Regards,
Bernhard

-- 
www.intevation.de/~bernhard (CEO)     +49 541 33 508 3-3
Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998
Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-17  9:56         ` Bernhard E. Reiter
@ 2017-03-22 17:15           ` Werner Koch
  2017-03-22 18:46             ` Peter Lebbing
  2017-03-23  7:29             ` Bernhard E. Reiter
  0 siblings, 2 replies; 19+ messages in thread
From: Werner Koch @ 2017-03-22 17:15 UTC (permalink / raw)
  To: Bernhard E. Reiter
  Cc: Michael J Gruber, Ævar Arnfjörð Bjarmason,
	Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 2636 bytes --]

On Fri, 17 Mar 2017 10:56, bernhard.reiter@intevation.de said:

> As the command line is not a stable API to GnuPG, there were changes and there 
> will be changes to the command line over several versions. It maybe in the 

That is not true.  The command line interface has been stable for the
last 19 years.  We only removed a left over PGP-2 backward compatibility
in 2.1 (-kvv).  I doubt that this has even been noticed.

>> Unfortunately, gpgme does not solve the interoperability problems
>> between gpg (1, classic, stable maint mode) or gpg2.0 (stable) and
>> gpg2.1 (modern) key stores, and gpg2.2 (modern, stable) is not released
>> yet.

It actually does.  For the tasks git uses gpg you should not notice a
difference in gpgme between any of these versions.  BTW, 2.1 was
realeased more than 2 years ago and 2.0 will reach EOL in 9 months.

> That is right, I also believe gpgme does not solve all interoperability 
> problems. I guess it solves some, but I would do more research to come up 

The main arguments pro GPGME are

 - There is generic stable API and ABI which did not changes since 0.4.1
   (2003-06-06) when we decided to redesign the API.

 - Iff verification of signatures needs a speedup we can do this inside
   GPGME and GnuPG without the GPGME user noticing that.  Technically we
   will then run gpg as co-process controlled by GPGME; for gpgsm
   (S/MIME) this is already done but there has not yet been a need to do
   this also for gpg.  Git could be the trigger to implement that.

 - The GPGME API would allow to provide a rich and stable output with
   the pretty print format.  AFAICS the current %G* format characters
   are a bit limited and require that scripts need to look at the key
   for further information.

> The key store migration is mainly an orthogonal issue and the problems will 
> happen with or without using gpgme. As GnuPG 2.2 is under way, it makes sense 

Please note that 2.2 is more of a marketing term than a change.  2.1 is
already in use and will be the standard gpg version in several distros,
including Debian.

Interoperability with 1.4 is a bit cumbersome if you often add new keys.
However, "gpg --export | gpg1 --import" is not too complicated. 

Be aware that ECC keys will be used more and more and they are not
supported by gpg1.  Further we are currently defining a v5 key format to
be prepared for weaknesses in the SHA-1 based fingerprint.  It is very
unlikely that a v6 key format will be added to gpg1.


Salam-Shalom,

   Werner

-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-22 17:15           ` Werner Koch
@ 2017-03-22 18:46             ` Peter Lebbing
  2017-03-23  6:52               ` Werner Koch
  2017-03-23  7:29             ` Bernhard E. Reiter
  1 sibling, 1 reply; 19+ messages in thread
From: Peter Lebbing @ 2017-03-22 18:46 UTC (permalink / raw)
  To: Bernhard E. Reiter, Michael J Gruber,
	Ævar Arnfjörð Bjarmason, Git Mailing List,
	gnupg-devel, Lukas Puehringer, Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 1662 bytes --]

On 22/03/17 18:15, Werner Koch wrote:
> It actually does.  For the tasks git uses gpg you should not notice a
> difference in gpgme between any of these versions.

Bernhard wrote "interoperability problems between [...] key stores". I'm
under the impression you are actually answering the question "can GPGME
be used in the same way regardless of the GnuPG version" instead?

> Interoperability with 1.4 is a bit cumbersome if you often add new keys.
> However, "gpg --export | gpg1 --import" is not too complicated. 

This presumes that

1) Keys are only updated on the 2.1 side
2) Keys are not deleted
3) Secret keys are never changed

right?

1) is trivially solvable. 2) is trickier, but can be done.

3) is because GnuPG 1.4 cannot update a secret key at all. Adding a new
subkey fails with:

gpg: key DCDFDFA4: already in secret keyring
gpg: Total number processed: 1
gpg:       secret keys read: 1
gpg:  secret keys unchanged: 1

You could delete before re-importing, but what if the key on the 1.4
side is actually the newer one? You'd lose all changes. Worst case:
updates of a single key on both sides. But that is perhaps pushing the
limit of sane use. Perhaps not, perhaps the user likes to use two
different frontends which use different GnuPG versions as their backend.

Luckily, expiration extensions are picked up by just transferring the
public key part of a secret key, so that does work.

Peter.

-- 
I use the GNU Privacy Guard (GnuPG) in combination with Enigmail.
You can send me encrypted mail if you want some privacy.
My key is available at <http://digitalbrains.com/2012/openpgp-key-peter>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-22 18:46             ` Peter Lebbing
@ 2017-03-23  6:52               ` Werner Koch
  0 siblings, 0 replies; 19+ messages in thread
From: Werner Koch @ 2017-03-23  6:52 UTC (permalink / raw)
  To: Peter Lebbing
  Cc: Bernhard E. Reiter, Michael J Gruber,
	Ævar Arnfjörð Bjarmason, Git Mailing List,
	gnupg-devel, Lukas Puehringer, Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

On Wed, 22 Mar 2017 19:46, peter@digitalbrains.com said:
> under the impression you are actually answering the question "can GPGME
> be used in the same way regardless of the GnuPG version" instead?

Right.

> 3) is because GnuPG 1.4 cannot update a secret key at all. Adding a new
> subkey fails with:

Indeed, I was not anymore aware of this limitation in versions < 2.1.
There are even more conflicts when PGP-2 keys (to me the only valid
reason to use gpg1 along with gpg2) or ECC keys need to be considered.


Shalom-Salam,

   Werner

-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-22 17:15           ` Werner Koch
  2017-03-22 18:46             ` Peter Lebbing
@ 2017-03-23  7:29             ` Bernhard E. Reiter
  2017-03-23 10:56               ` Werner Koch
  1 sibling, 1 reply; 19+ messages in thread
From: Bernhard E. Reiter @ 2017-03-23  7:29 UTC (permalink / raw)
  To: Werner Koch
  Cc: Michael J Gruber, Ævar Arnfjörð Bjarmason,
	Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 1349 bytes --]

Am Mittwoch 22 März 2017 18:15:36 schrieb Werner Koch:
> On Fri, 17 Mar 2017 10:56, bernhard.reiter@intevation.de said:
> > As the command line is not a stable API to GnuPG, there were changes and
> > there will be changes to the command line over several versions. It maybe
> > in the
>
> That is not true.  The command line interface has been stable for the
> last 19 years.  We only removed a left over PGP-2 backward compatibility
> in 2.1 (-kvv).  I doubt that this has even been noticed.

I was under the impression (and I do remember you telling me a few times)
that the output of the command line interaction did change a lot over times
and using applications had issues. I've experienced a few of those over the 
last 17 years. :) Still I would have to dig up the examples.
You seem to refer more to the existence and principal meaning of command line 
options, which is very stable because of your efforts, which I've mentioned 
before. Contents of the resulting input and output strings has changed though 
and gpgme does this parsing for a using application in a more stable way.

Best Regards,
Bernhard

-- 
www.intevation.de/~bernhard (CEO)     +49 541 33 508 3-3
Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998
Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Stable GnuPG interface, git should use GPGME
  2017-03-23  7:29             ` Bernhard E. Reiter
@ 2017-03-23 10:56               ` Werner Koch
  0 siblings, 0 replies; 19+ messages in thread
From: Werner Koch @ 2017-03-23 10:56 UTC (permalink / raw)
  To: Bernhard E. Reiter
  Cc: Michael J Gruber, Ævar Arnfjörð Bjarmason,
	Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds

[-- Attachment #1: Type: text/plain, Size: 539 bytes --]

On Thu, 23 Mar 2017 08:29, bernhard.reiter@intevation.de said:

> I was under the impression (and I do remember you telling me a few times)
> that the output of the command line interaction did change a lot over times
> and using applications had issues. I've experienced a few of those over the 

Sure, but that is only for human consumption and not for scripts.

gpg introduced its --status-fd machine interface 19 years ago.


Shalom-Salam,

   Werner


-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2017-03-23 11:03 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter
2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason
2017-03-13 10:14   ` Michael J Gruber
2017-03-13 12:49     ` Bernhard E. Reiter
2017-03-14 10:39       ` Michael J Gruber
2017-03-17  9:56         ` Bernhard E. Reiter
2017-03-22 17:15           ` Werner Koch
2017-03-22 18:46             ` Peter Lebbing
2017-03-23  6:52               ` Werner Koch
2017-03-23  7:29             ` Bernhard E. Reiter
2017-03-23 10:56               ` Werner Koch
2017-03-13 10:30   ` Bernhard Reiter
2017-03-10 18:54 ` Linus Torvalds
2017-03-10 20:26   ` Theodore Ts'o
2017-03-13 11:14   ` Bernhard E. Reiter
2017-03-13 12:53     ` Jeff King
2017-03-11  0:10 ` brian m. carlson
2017-03-13 12:29   ` Bernhard E. Reiter
2017-03-13 19:48   ` Christian Neukirchen

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).