* Stable GnuPG interface, git should use GPGME @ 2017-03-10 10:00 Bernhard E. Reiter 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason ` (2 more replies) 0 siblings, 3 replies; 19+ messages in thread From: Bernhard E. Reiter @ 2017-03-10 10:00 UTC (permalink / raw) To: git; +Cc: gnupg-devel [-- Attachment #1: Type: text/plain, Size: 2459 bytes --] Dear Git-Devs, git uses an pipe-and-exec approach to running a GnuPG binary as writen in the documentation [1]: gpg.program Use this custom program instead of "gpg" found on $PATH when making or verifying a PGP signature. The program must support the same command-line interface as GPG please consider using libgpgme interfacing to GnuPG, because the gpg command-line interface is not considered an official API to GnuPG by the GnuPG-devs and thus potentially unstable. == Details I'm involved in GnuPG development. For most applications using libgpgme is the way what GnuPG-devs would recommend, also see https://wiki.gnupg.org/APIs . GnuPG devs are making a good effort of trying to keep the command-line interface stable, though it is not for sure. Git is only using a small part of the interface, so the risk when keeping the current way is small. Still I believe git's stability and usability would profit when moving to libgpgme, especially with the coming move to GnuPG 2.2, better diagnosing messages and for cross-plattform usage. == Usability problem with `gpg2` vs `gpg` My use case today was signing and git by default found the `gpg` binary by default and the command failed. The reason is that I have `gpg2` installed and most applications use it right away. So git failed signing because the .gnupg configuration of the user was not ready for the old `gpg` which is still installed on Debian GNU/Linux for purposes of the operating system. If git would have used libgpgme, gpgme would have choosen the most uptodate version of `gpg` available (or configured) without me intervening via gpg.program. Now because of this problem you could adding a check for `gpg2` and fallback to `gpg`, but even better would be to move to libgpgme. >:) Best Regards and thanks for maintaining Git as Free Software, Bernhard == how to respond ps: Please copy me on replies as I am not on git@vger.kernel.org. pps: I've copied gnupg-devel@ so they can see I've send this report, you don't have to. [1] https://github.com/git/git/blob/3bc53220cb2dcf709f7a027a3f526befd021d858/Documentation/config.txt search for 'gpg.program'. -- www.intevation.de/~bernhard (CEO) +49 541 33 508 3-3 Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998 Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter @ 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason 2017-03-13 10:14 ` Michael J Gruber 2017-03-13 10:30 ` Bernhard Reiter 2017-03-10 18:54 ` Linus Torvalds 2017-03-11 0:10 ` brian m. carlson 2 siblings, 2 replies; 19+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2017-03-10 14:23 UTC (permalink / raw) To: Bernhard E. Reiter Cc: Git Mailing List, gnupg-devel, Lukas Puehringer, Michael J Gruber, Linus Torvalds On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter <bernhard.reiter@intevation.de> wrote: > Dear Git-Devs, I haven't contributed to Git's GPG code, but I'm taking the liberty of CC-ing some people who have. > git uses an pipe-and-exec approach to running a GnuPG binary > as writen in the documentation [1]: > > gpg.program > Use this custom program instead of "gpg" found on $PATH when making > or verifying a PGP signature. The program must support the same > command-line interface as GPG > > please consider using libgpgme interfacing to GnuPG, because the gpg > command-line interface is not considered an official API to GnuPG by the > GnuPG-devs and thus potentially unstable. > > == Details > > I'm involved in GnuPG development. For most applications using libgpgme is the > way what GnuPG-devs would recommend, also see > > https://wiki.gnupg.org/APIs . > > GnuPG devs are making a good effort of trying to keep the command-line > interface stable, though it is not for sure. Git is only using a small part > of the interface, so the risk when keeping the current way is small. > Still I believe git's stability and usability would profit when moving to > libgpgme, especially with the coming move to GnuPG 2.2, better diagnosing > messages and for cross-plattform usage. > > == Usability problem with `gpg2` vs `gpg` > > My use case today was signing and git by default found the `gpg` binary by > default and the command failed. The reason is that I have `gpg2` installed > and most applications use it right away. So git failed signing because > the .gnupg configuration of the user was not ready for the old `gpg` which is > still installed on Debian GNU/Linux for purposes of the operating system. If > git would have used libgpgme, gpgme would have choosen the most uptodate > version of `gpg` available (or configured) without me intervening via > gpg.program. Now because of this problem you could adding a check for `gpg2` > and fallback to `gpg`, but even better would be to move to libgpgme. >:) I'm on Debian but haven't had these issues. What's your gpg & gpg2 --version & Debian release? And what in particular failed? And what git version was this? I see we've had a couple of workarounds for gpg2, in particular Linus's v2.8.4-1-gb624a3e67f, but if you have v2.10.0 or later that won't fix whatever issue you had. Using the library sounds good, but a shorter-term immediate fix would be to figure out what bug you encountered in our use of the command-line version, and see if we've fixed that already or not. Regardless of what we do with a gpg library in the future some distros might want to backport such a small patch if we can come up with it. > Best Regards and thanks for maintaining Git as Free Software, > Bernhard > > == how to respond > > ps: Please copy me on replies as I am not on git@vger.kernel.org. > pps: I've copied gnupg-devel@ so they can see I've send this report, you don't > have to. > > > [1] > https://github.com/git/git/blob/3bc53220cb2dcf709f7a027a3f526befd021d858/Documentation/config.txt > search for 'gpg.program'. > > -- > www.intevation.de/~bernhard (CEO) +49 541 33 508 3-3 > Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998 > Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason @ 2017-03-13 10:14 ` Michael J Gruber 2017-03-13 12:49 ` Bernhard E. Reiter 2017-03-13 10:30 ` Bernhard Reiter 1 sibling, 1 reply; 19+ messages in thread From: Michael J Gruber @ 2017-03-13 10:14 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason, Bernhard E. Reiter Cc: Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds Ævar Arnfjörð Bjarmason venit, vidit, dixit 10.03.2017 15:23: > On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter > <bernhard.reiter@intevation.de> wrote: >> Dear Git-Devs, > > I haven't contributed to Git's GPG code, but I'm taking the liberty of > CC-ing some people who have. > >> git uses an pipe-and-exec approach to running a GnuPG binary >> as writen in the documentation [1]: >> >> gpg.program >> Use this custom program instead of "gpg" found on $PATH when making >> or verifying a PGP signature. The program must support the same >> command-line interface as GPG >> >> please consider using libgpgme interfacing to GnuPG, because the gpg >> command-line interface is not considered an official API to GnuPG by the >> GnuPG-devs and thus potentially unstable. >> >> == Details >> >> I'm involved in GnuPG development. For most applications using libgpgme is the >> way what GnuPG-devs would recommend, also see >> >> https://wiki.gnupg.org/APIs . >> >> GnuPG devs are making a good effort of trying to keep the command-line >> interface stable, though it is not for sure. Git is only using a small part >> of the interface, so the risk when keeping the current way is small. >> Still I believe git's stability and usability would profit when moving to >> libgpgme, especially with the coming move to GnuPG 2.2, better diagnosing >> messages and for cross-plattform usage. >> >> == Usability problem with `gpg2` vs `gpg` >> >> My use case today was signing and git by default found the `gpg` binary by >> default and the command failed. The reason is that I have `gpg2` installed >> and most applications use it right away. So git failed signing because >> the .gnupg configuration of the user was not ready for the old `gpg` which is >> still installed on Debian GNU/Linux for purposes of the operating system. If >> git would have used libgpgme, gpgme would have choosen the most uptodate >> version of `gpg` available (or configured) without me intervening via >> gpg.program. Now because of this problem you could adding a check for `gpg2` >> and fallback to `gpg`, but even better would be to move to libgpgme. >:) > > I'm on Debian but haven't had these issues. What's your gpg & gpg2 > --version & Debian release? And what in particular failed? > > And what git version was this? I see we've had a couple of workarounds > for gpg2, in particular Linus's v2.8.4-1-gb624a3e67f, but if you have > v2.10.0 or later that won't fix whatever issue you had. > > Using the library sounds good, but a shorter-term immediate fix would > be to figure out what bug you encountered in our use of the > command-line version, and see if we've fixed that already or not. > Regardless of what we do with a gpg library in the future some distros > might want to backport such a small patch if we can come up with it. As far as I know, Git handles different GPG versions just fine. The problem is the "difficult" upgrade path and mixed installations with gpg and gpg2.1+ that some distributions force upon you: As soon as you start gpg2.1, your (secret) key store is migrated to a new format without technically invalidating it. Similarly, users may enter gpg2.1+-only comand in the config that is actually shared with gpg, throwing off any use of gpg - not just by git, but also by anything that your distro requires gpg for (such as packaging tools and the like). In short: Users will run into problems anyway; git provides the quick way out (git config gpg.program gpg2), users won't be as lucky with other things that require gpg. As for the library: While - technically speaking - the command line is not a stable API for gpg, it does work across versions of gpg, and gpg 2.2 will be the first real stable branch that uses the new key store layout. So I'd rather wait for that to stabilize before going away from what turned out to be most stable so far. Note that we (git) refrain from parsing ordinary output/return codes of gpg and use status-fd as we should (and as documented). Michael ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-13 10:14 ` Michael J Gruber @ 2017-03-13 12:49 ` Bernhard E. Reiter 2017-03-14 10:39 ` Michael J Gruber 0 siblings, 1 reply; 19+ messages in thread From: Bernhard E. Reiter @ 2017-03-13 12:49 UTC (permalink / raw) To: Michael J Gruber Cc: Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 3047 bytes --] Am Montag 13 März 2017 11:14:57 schrieb Michael J Gruber: > Ævar Arnfjörð Bjarmason venit, vidit, dixit 10.03.2017 15:23: > > On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter > >> please consider using libgpgme interfacing to GnuPG, because the gpg > >> command-line interface is not considered an official API to GnuPG by the > >> GnuPG-devs and thus potentially unstable. [example of gpg2 vs gpg option incompatibility cut] > > Using the library sounds good, but a shorter-term immediate fix would > > be to figure out what bug you encountered in our use of the > > command-line version, and see if we've fixed that already or not. > As far as I know, Git handles different GPG versions just fine. As mentioned before: explicitely setting gpg.program to gpg2 helps if gpg chokes on the new config. Trying the `gpg2` binary first can be a simple fix. Using libgpgme potentially solves this and other compatility options. > The problem is the "difficult" upgrade path and mixed installations with > gpg and gpg2.1+ that some distributions force upon you: > > As soon as you start gpg2.1, your (secret) key store is migrated to a > new format without technically invalidating it. Similarly, users may > enter gpg2.1+-only comand in the config that is actually shared with > gpg, throwing off any use of gpg - not just by git, but also by anything > that your distro requires gpg for (such as packaging tools and the like). Yes, this is another example why trying `gpg2? first by default or using libgpgme keeps trouble away from users. > In short: Users will run into problems anyway; git provides the quick > way out (git config gpg.program gpg2), users won't be as lucky with > other things that require gpg. Application using libgpgme will behave fine and many user facing components use it already. > As for the library: While - technically speaking - the command line is > not a stable API for gpg, it does work across versions of gpg, and gpg ... to some extend. > 2.2 will be the first real stable branch that uses the new key store > layout. So I'd rather wait for that to stabilize before going away from > what turned out to be most stable so far. It is not just about the key-store change as mentioned before. However I agree that a potential switch should be done with a current version of gpgme that already has support for GnuPG 2.1/2, e.g. gpgme v1.8.0. > Note that we (git) refrain from parsing ordinary output/return codes of > gpg and use status-fd as we should (and as documented). It is good to use --status-fd and --with-colons when calling gpg, you still have to parse the results of status-fd as described in doc/DETAILs. https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=doc/DETAILS;hb=HEAD Regards, Bernhard -- www.intevation.de/~bernhard (CEO) +49 541 33 508 3-3 Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998 Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-13 12:49 ` Bernhard E. Reiter @ 2017-03-14 10:39 ` Michael J Gruber 2017-03-17 9:56 ` Bernhard E. Reiter 0 siblings, 1 reply; 19+ messages in thread From: Michael J Gruber @ 2017-03-14 10:39 UTC (permalink / raw) To: Bernhard E. Reiter Cc: Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds Bernhard E. Reiter venit, vidit, dixit 13.03.2017 13:49: > Am Montag 13 März 2017 11:14:57 schrieb Michael J Gruber: >> Ævar Arnfjörð Bjarmason venit, vidit, dixit 10.03.2017 15:23: >>> On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter > >>>> please consider using libgpgme interfacing to GnuPG, because the gpg >>>> command-line interface is not considered an official API to GnuPG by the >>>> GnuPG-devs and thus potentially unstable. > > [example of gpg2 vs gpg option incompatibility cut] > >>> Using the library sounds good, but a shorter-term immediate fix would >>> be to figure out what bug you encountered in our use of the >>> command-line version, and see if we've fixed that already or not. > >> As far as I know, Git handles different GPG versions just fine. > > As mentioned before: explicitely setting gpg.program to gpg2 helps if gpg > chokes on the new config. Trying the `gpg2` binary first can be a simple fix. > Using libgpgme potentially solves this and other compatility options. > >> The problem is the "difficult" upgrade path and mixed installations with >> gpg and gpg2.1+ that some distributions force upon you: >> >> As soon as you start gpg2.1, your (secret) key store is migrated to a >> new format without technically invalidating it. Similarly, users may >> enter gpg2.1+-only comand in the config that is actually shared with >> gpg, throwing off any use of gpg - not just by git, but also by anything >> that your distro requires gpg for (such as packaging tools and the like). > > Yes, this is another example why trying `gpg2? first by default or using > libgpgme keeps trouble away from users. No, not at all. On the contrary: Using gpg2.1 without being asked to by the user will migrate his key store! This can lead to tremendous problems when a user manages his secret key store using gpg and git uses the other secret key store (via gpg2.1)! >> In short: Users will run into problems anyway; git provides the quick >> way out (git config gpg.program gpg2), users won't be as lucky with >> other things that require gpg. > > Application using libgpgme will behave fine and many user facing components > use it already. > >> As for the library: While - technically speaking - the command line is >> not a stable API for gpg, it does work across versions of gpg, and gpg > > ... to some extend. I have no idea what you're implying here - but I have a pretty good idea of what works in current git and what not, including gpg usage (which is the qualifier that you cut out). >> 2.2 will be the first real stable branch that uses the new key store >> layout. So I'd rather wait for that to stabilize before going away from >> what turned out to be most stable so far. > > It is not just about the key-store change as mentioned before. However > I agree that a potential switch should be done with a current version of gpgme > that already has support for GnuPG 2.1/2, e.g. gpgme v1.8.0. Unfortunately, gpgme does not solve the interoperability problems between gpg (1, classic, stable maint mode) or gpg2.0 (stable) and gpg2.1 (modern) key stores, and gpg2.2 (modern, stable) is not released yet. >> Note that we (git) refrain from parsing ordinary output/return codes of >> gpg and use status-fd as we should (and as documented). > > It is good to use --status-fd and --with-colons when calling gpg, you still > have to parse the results of status-fd as described in doc/DETAILs. > https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=doc/DETAILS;hb=HEAD Again, have you checked what we are doing in git land? Your agenda is pretty clear. It's also pretty clear from the above that gpgme is not the solution to the problem which is introduced by the migration path to versions of gpg which are not declared stable by the gpg project, away from a gpg version which is not obsoleted by the gpg project (2.0, maybe 1). And, really, key store migration was the only "major" thing we had to think about to cope with gpg 2.1+ - it affected the test suite setup. So, once 2.2 is out and stable and mainstream and we don't risk migrating users' secret key store sneakily I'm all for defaulting to gpg2, and then is a good time for us to look into gpgme. Michael ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-14 10:39 ` Michael J Gruber @ 2017-03-17 9:56 ` Bernhard E. Reiter 2017-03-22 17:15 ` Werner Koch 0 siblings, 1 reply; 19+ messages in thread From: Bernhard E. Reiter @ 2017-03-17 9:56 UTC (permalink / raw) To: Michael J Gruber, Werner Koch Cc: Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 6648 bytes --] Am Dienstag 14 März 2017 11:39:12 schrieb Michael J Gruber: > >> The problem is the "difficult" upgrade path and mixed installations with > >> gpg and gpg2.1+ that some distributions force upon you: > >> > >> As soon as you start gpg2.1, your (secret) key store is migrated to a > >> new format without technically invalidating it. Similarly, users may > >> enter gpg2.1+-only comand in the config that is actually shared with > >> gpg, throwing off any use of gpg - not just by git, but also by anything > >> that your distro requires gpg for (such as packaging tools and the > >> like). > > > > Yes, this is another example why trying `gpg2? first by default or using > > libgpgme keeps trouble away from users. > > No, not at all. On the contrary: Using gpg2.1 without being asked to by > the user will migrate his key store! This can lead to tremendous > problems when a user manages his secret key store using gpg and git uses > the other secret key store (via gpg2.1)! In the following scenario I agree with you: * The user only uses gpg (1) and only applications that use gpg (1) and does not have gpg2 installed. * Now a GnuPG package for 2.1/2.2 is installed and only the "gpg2" binary now points to it. * If then git will just switches using `gpg2` which is v2.1 because it came available, it will force the migration to the new store. My statement was assuming a scenario where * the user has gpg v1 and gpg2 v2.0 installed and many (or all) applications use gpg2 via gpgme or by other means. I consider this typical for desktops or application servers using for desktop or development work. * Now git also uses gpg2 (via gpgme or by other means, a possible improvement we are just talking about) * If GnuPG 2.1/2.2 becomes available and of the already used application will trigger the keystore migration. git users will have a smooth experience because they were using gpg2 all along and then to use the new keystore. (It is still possible for some to configure libgpgme to use `gpg` v1 and all applications using gpgme will use it and you do not have to configure each one separately. Because you should either use gpg v2.1 or gpg v1 exclusively, it makes sense to configure it close to gpg in its official api and not within each application. Again you can deviate from this.) > >> As for the library: While - technically speaking - the command line is > >> not a stable API for gpg, it does work across versions of gpg, and gpg > > > > ... to some extend. > > I have no idea what you're implying here - but I have a pretty good idea > of what works in current git and what not, including gpg usage (which is > the qualifier that you cut out). As the command line is not a stable API to GnuPG, there were changes and there will be changes to the command line over several versions. It maybe in the past that git's usage of the command line was stable and works well, because of the subset of functionality of GnuPG that git is using and good work done by you and other developers of the git GnuPG support. It may or may not be a good choice for the future. As I've written before I'm not familiar enough with git to make a final call here, but there are some signs and examples where using gpgme may be an improvements in the areas: * better configuration which gpg and which options to use for all applications * consistent behaviour from git with other applications * more robust programming interface over several gpg versions * (better performance when reusing the same gpgme "context" for many consecutive crypto operations) > Unfortunately, gpgme does not solve the interoperability problems > between gpg (1, classic, stable maint mode) or gpg2.0 (stable) and > gpg2.1 (modern) key stores, and gpg2.2 (modern, stable) is not released > yet. That is right, I also believe gpgme does not solve all interoperability problems. I guess it solves some, but I would do more research to come up with some specific examples. > >> Note that we (git) refrain from parsing ordinary output/return codes of > >> gpg and use status-fd as we should (and as documented). > > > > It is good to use --status-fd and --with-colons when calling gpg, you > > still have to parse the results of status-fd as described in doc/DETAILs. > > https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=doc/DETAILS > >;hb=HEAD > > Again, have you checked what we are doing in git land? As mentioned before: Only on the surface. I'm not deeply familiar with all git use case of GnuPG. > Your agenda is pretty clear. Yes, I'd like to improve the user experience of git users regarding crypto operations done with GnuPG. There are some signs that indicate that using gpgme may potentially improve this user experience (as outlined). > It's also pretty clear from the above that > gpgme is not the solution to the problem which is introduced by the > migration path to versions of gpg which are not declared stable by the > gpg project, away from a gpg version which is not obsoleted by the gpg > project (2.0, maybe 1). While I agree that gpgme does not solve all problems with this migration, it solves some problems with it (see above and my other email) and offers other advantages as the official API of GnuPG. Still it is the call of people more familiar with git usage and use cases to see if they have a good reason to deviate from the common case and use a non-official way of automating crypto operations with GnuPG. > And, really, key store migration was the only "major" thing we had to > think about to cope with gpg 2.1+ - it affected the test suite setup. > > So, once 2.2 is out and stable and mainstream and we don't risk > migrating users' secret key store sneakily I'm all for defaulting to > gpg2, and then is a good time for us to look into gpgme. The key store migration is mainly an orthogonal issue and the problems will happen with or without using gpgme. As GnuPG 2.2 is under way, it makes sense to look into an improved support now, to be ready if GnuPG 2.2 becomes more widely used. It probably will get a lot more usage this year because of the ECC support (and some other reasons). Thanks all for your thoughts, using GnuPG, considering gpgme and in general for maintaining git as Free Software! :) Best Regards, Bernhard -- www.intevation.de/~bernhard (CEO) +49 541 33 508 3-3 Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998 Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-17 9:56 ` Bernhard E. Reiter @ 2017-03-22 17:15 ` Werner Koch 2017-03-22 18:46 ` Peter Lebbing 2017-03-23 7:29 ` Bernhard E. Reiter 0 siblings, 2 replies; 19+ messages in thread From: Werner Koch @ 2017-03-22 17:15 UTC (permalink / raw) To: Bernhard E. Reiter Cc: Michael J Gruber, Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 2636 bytes --] On Fri, 17 Mar 2017 10:56, bernhard.reiter@intevation.de said: > As the command line is not a stable API to GnuPG, there were changes and there > will be changes to the command line over several versions. It maybe in the That is not true. The command line interface has been stable for the last 19 years. We only removed a left over PGP-2 backward compatibility in 2.1 (-kvv). I doubt that this has even been noticed. >> Unfortunately, gpgme does not solve the interoperability problems >> between gpg (1, classic, stable maint mode) or gpg2.0 (stable) and >> gpg2.1 (modern) key stores, and gpg2.2 (modern, stable) is not released >> yet. It actually does. For the tasks git uses gpg you should not notice a difference in gpgme between any of these versions. BTW, 2.1 was realeased more than 2 years ago and 2.0 will reach EOL in 9 months. > That is right, I also believe gpgme does not solve all interoperability > problems. I guess it solves some, but I would do more research to come up The main arguments pro GPGME are - There is generic stable API and ABI which did not changes since 0.4.1 (2003-06-06) when we decided to redesign the API. - Iff verification of signatures needs a speedup we can do this inside GPGME and GnuPG without the GPGME user noticing that. Technically we will then run gpg as co-process controlled by GPGME; for gpgsm (S/MIME) this is already done but there has not yet been a need to do this also for gpg. Git could be the trigger to implement that. - The GPGME API would allow to provide a rich and stable output with the pretty print format. AFAICS the current %G* format characters are a bit limited and require that scripts need to look at the key for further information. > The key store migration is mainly an orthogonal issue and the problems will > happen with or without using gpgme. As GnuPG 2.2 is under way, it makes sense Please note that 2.2 is more of a marketing term than a change. 2.1 is already in use and will be the standard gpg version in several distros, including Debian. Interoperability with 1.4 is a bit cumbersome if you often add new keys. However, "gpg --export | gpg1 --import" is not too complicated. Be aware that ECC keys will be used more and more and they are not supported by gpg1. Further we are currently defining a v5 key format to be prepared for weaknesses in the SHA-1 based fingerprint. It is very unlikely that a v6 key format will be added to gpg1. Salam-Shalom, Werner -- Die Gedanken sind frei. Ausnahmen regelt ein Bundesgesetz. [-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-22 17:15 ` Werner Koch @ 2017-03-22 18:46 ` Peter Lebbing 2017-03-23 6:52 ` Werner Koch 2017-03-23 7:29 ` Bernhard E. Reiter 1 sibling, 1 reply; 19+ messages in thread From: Peter Lebbing @ 2017-03-22 18:46 UTC (permalink / raw) To: Bernhard E. Reiter, Michael J Gruber, Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 1662 bytes --] On 22/03/17 18:15, Werner Koch wrote: > It actually does. For the tasks git uses gpg you should not notice a > difference in gpgme between any of these versions. Bernhard wrote "interoperability problems between [...] key stores". I'm under the impression you are actually answering the question "can GPGME be used in the same way regardless of the GnuPG version" instead? > Interoperability with 1.4 is a bit cumbersome if you often add new keys. > However, "gpg --export | gpg1 --import" is not too complicated. This presumes that 1) Keys are only updated on the 2.1 side 2) Keys are not deleted 3) Secret keys are never changed right? 1) is trivially solvable. 2) is trickier, but can be done. 3) is because GnuPG 1.4 cannot update a secret key at all. Adding a new subkey fails with: gpg: key DCDFDFA4: already in secret keyring gpg: Total number processed: 1 gpg: secret keys read: 1 gpg: secret keys unchanged: 1 You could delete before re-importing, but what if the key on the 1.4 side is actually the newer one? You'd lose all changes. Worst case: updates of a single key on both sides. But that is perhaps pushing the limit of sane use. Perhaps not, perhaps the user likes to use two different frontends which use different GnuPG versions as their backend. Luckily, expiration extensions are picked up by just transferring the public key part of a secret key, so that does work. Peter. -- I use the GNU Privacy Guard (GnuPG) in combination with Enigmail. You can send me encrypted mail if you want some privacy. My key is available at <http://digitalbrains.com/2012/openpgp-key-peter> [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-22 18:46 ` Peter Lebbing @ 2017-03-23 6:52 ` Werner Koch 0 siblings, 0 replies; 19+ messages in thread From: Werner Koch @ 2017-03-23 6:52 UTC (permalink / raw) To: Peter Lebbing Cc: Bernhard E. Reiter, Michael J Gruber, Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 625 bytes --] On Wed, 22 Mar 2017 19:46, peter@digitalbrains.com said: > under the impression you are actually answering the question "can GPGME > be used in the same way regardless of the GnuPG version" instead? Right. > 3) is because GnuPG 1.4 cannot update a secret key at all. Adding a new > subkey fails with: Indeed, I was not anymore aware of this limitation in versions < 2.1. There are even more conflicts when PGP-2 keys (to me the only valid reason to use gpg1 along with gpg2) or ECC keys need to be considered. Shalom-Salam, Werner -- Die Gedanken sind frei. Ausnahmen regelt ein Bundesgesetz. [-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-22 17:15 ` Werner Koch 2017-03-22 18:46 ` Peter Lebbing @ 2017-03-23 7:29 ` Bernhard E. Reiter 2017-03-23 10:56 ` Werner Koch 1 sibling, 1 reply; 19+ messages in thread From: Bernhard E. Reiter @ 2017-03-23 7:29 UTC (permalink / raw) To: Werner Koch Cc: Michael J Gruber, Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 1349 bytes --] Am Mittwoch 22 März 2017 18:15:36 schrieb Werner Koch: > On Fri, 17 Mar 2017 10:56, bernhard.reiter@intevation.de said: > > As the command line is not a stable API to GnuPG, there were changes and > > there will be changes to the command line over several versions. It maybe > > in the > > That is not true. The command line interface has been stable for the > last 19 years. We only removed a left over PGP-2 backward compatibility > in 2.1 (-kvv). I doubt that this has even been noticed. I was under the impression (and I do remember you telling me a few times) that the output of the command line interaction did change a lot over times and using applications had issues. I've experienced a few of those over the last 17 years. :) Still I would have to dig up the examples. You seem to refer more to the existence and principal meaning of command line options, which is very stable because of your efforts, which I've mentioned before. Contents of the resulting input and output strings has changed though and gpgme does this parsing for a using application in a more stable way. Best Regards, Bernhard -- www.intevation.de/~bernhard (CEO) +49 541 33 508 3-3 Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998 Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-23 7:29 ` Bernhard E. Reiter @ 2017-03-23 10:56 ` Werner Koch 0 siblings, 0 replies; 19+ messages in thread From: Werner Koch @ 2017-03-23 10:56 UTC (permalink / raw) To: Bernhard E. Reiter Cc: Michael J Gruber, Ævar Arnfjörð Bjarmason, Git Mailing List, gnupg-devel, Lukas Puehringer, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 539 bytes --] On Thu, 23 Mar 2017 08:29, bernhard.reiter@intevation.de said: > I was under the impression (and I do remember you telling me a few times) > that the output of the command line interaction did change a lot over times > and using applications had issues. I've experienced a few of those over the Sure, but that is only for human consumption and not for scripts. gpg introduced its --status-fd machine interface 19 years ago. Shalom-Salam, Werner -- Die Gedanken sind frei. Ausnahmen regelt ein Bundesgesetz. [-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason 2017-03-13 10:14 ` Michael J Gruber @ 2017-03-13 10:30 ` Bernhard Reiter 1 sibling, 0 replies; 19+ messages in thread From: Bernhard Reiter @ 2017-03-13 10:30 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason Cc: Git Mailing List, gnupg-devel, Lukas Puehringer, Michael J Gruber, Linus Torvalds [-- Attachment #1: Type: text/plain, Size: 3870 bytes --] Am Freitag 10 März 2017 15:23:27 schrieb Ævar Arnfjörð Bjarmason: > On Fri, Mar 10, 2017 at 11:00 AM, Bernhard E. Reiter wrote > > please consider using libgpgme interfacing to GnuPG, because the gpg > > command-line interface is not considered an official API to GnuPG by the > > GnuPG-devs and thus potentially unstable. > > == Usability problem with `gpg2` vs `gpg` > > > > My use case today was signing and git by default found the `gpg` binary > > by default and the command failed. I've mentioned this as one example for a possible advantage using libgpgme when interfacing with GnuPG. > > The reason is that I have `gpg2` > > installed and most applications use it right away. So git failed signing > > because the .gnupg configuration of the user was not ready for the old > > `gpg` which is still installed on Debian GNU/Linux for purposes of the > > operating system. If git would have used libgpgme, gpgme would have > > choosen the most uptodate version of `gpg` available (or configured) > > without me intervening via gpg.program. Now because of this problem you > > could adding a check for `gpg2` and fallback to `gpg`, but even better > > would be to move to libgpgme. >:) > > I'm on Debian but haven't had these issues. What's your gpg & gpg2 > --version & Debian release? And what in particular failed? If you use options in your configuration that only gpg2 understands, gpg(1) will barf. For example the following lines in ~/.gnupg/gpg.conf debug-level basic log-file socket:///home/bern/.gnupg/log-socket will lead to LANG=C gpg -K gpg: /powerhome/bern/.gnupg/gpg.conf:102: argument not expected gpg: /powerhome/bern/.gnupg/gpg.conf:103: invalid option where gpg2 works as expected. As a number of application already uses gpg2 (via libgpgme or not), this may go unnoticed for a while. So when I've started to sign with git on this machine I ran into the problem (current Jessie default versions): dpkg -s gnupg | grep ^Version #Version: 1.4.18-7+deb8u3 dpkg -s gnupg2 | grep ^Version #Version: 2.0.26-6+deb8u1 Workarounds are: * Use a different config for gpg2 and gpg, e.g. ~/.gnupg/gpg.conf-2 (https://www.gnupg.org/documentation/manuals/gnupg-2.0/Invoking-GPG.html ) * or set gpg.program for git to gpg2. > And what git version was this? I see we've had a couple of workarounds > for gpg2, in particular Linus's v2.8.4-1-gb624a3e67f, but if you have > v2.10.0 or later that won't fix whatever issue you had. dpkg -s git | grep ^Version Version: 1:2.1.4-2.1+deb8u2 (I've checked the most current master source to see that git still calls gpg.program, otherwise followed advise on https://git-scm.com/community to send reports and questions to the list.) > Using the library sounds good, but a shorter-term immediate fix would > be to figure out what bug you encountered in our use of the > command-line version, and see if we've fixed that already or not. > Regardless of what we do with a gpg library in the future some distros > might want to backport such a small patch if we can come up with it. I guess a good simple approach would be to try "gpg2" first and then fall back to "gpg" or "gpgv" in case only these version are available. (Here is a report that puts forward using gpgv in some situations https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852684 ) As there are other subtle potential issues with directly calling a gpg binary, using libgpgme by default probably has other advantages as well. And if there are important functions missing the GnuPG-devs would like to hear about them. Regards, Bernhard -- www.intevation.de/~bernhard +49 541 33 508 3-3 Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998 Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason @ 2017-03-10 18:54 ` Linus Torvalds 2017-03-10 20:26 ` Theodore Ts'o 2017-03-13 11:14 ` Bernhard E. Reiter 2017-03-11 0:10 ` brian m. carlson 2 siblings, 2 replies; 19+ messages in thread From: Linus Torvalds @ 2017-03-10 18:54 UTC (permalink / raw) To: Bernhard E. Reiter; +Cc: Git Mailing List, gnupg-devel On Fri, Mar 10, 2017 at 2:00 AM, Bernhard E. Reiter <bernhard.reiter@intevation.de> wrote: > > git uses an pipe-and-exec approach to running a GnuPG binary > as writen in the documentation [1]: > > gpg.program > Use this custom program instead of "gpg" found on $PATH when making > or verifying a PGP signature. The program must support the same > command-line interface as GPG > > please consider using libgpgme interfacing to GnuPG, because the gpg > command-line interface is not considered an official API to GnuPG by the > GnuPG-devs and thus potentially unstable. Quite frankly, I will NAK this just based on previous bad experiences with using "helpful" libraries. Maybe you can lay my worries to rest, but the problems with libraries in this context tend to be - hard to personalize. At least right now, we just allow people to set their gpg binary. I'm betting that the library would pick whatever random preferred version, and in the process possibly screwed us. Example: what if somebody is actually using another pgp implementation entirely for some reason, and is just scripting around it? Or maybe he's using the regular gnupg, but using different keys for different projects (using "--homedir"). That's trivial with the current model. How trivial is that with a library? - existing configuration This is the main problem I've seen in the past. Using the "ssh" _program_ is easy. You add your keys, your config files, whatever, and it "just works" (or rather, you fight it once and it definitely doesn't "just" work, but then you copy your .ssh directory around for the rest of your and forget how it ever worked, but it does). Using "libssh2" is an exercise in futility, and you have to do a crazy amount of stupid "look up keys" and simple configuration in your .ssh/config (like per-host keys, hostname swizzling etc) just don't pick up the configurations you already did for the program. - UI For things like gpg, the UI is traditionally horrible. But there tends to be various things like password agents that help with caching passwords and/or add a graphical UI to get the password when necessary. - library versioning. I don't know why, but I've never *ever* met a library developer who realized that libraries were all about stable API's, and the library users don't want to fight different versions. And to make matters worse, the different versions (particularly if you end up having to use a development version due to bugs or required features etc) are always made horribly bad to even detect at built-time automatically with simple #ifdef etc, so now you have to do autoconf crap etc. Now, it may be that the pgpme library "just works" across architectures and handles all of the above situations as gracefully as the external program does. In that case - but _ONLY_ in that case - would a switch-over to the library possibly be a good thing. I'd be pleasantly surprised. But I *would* be surprised, because every time I've seen that "library vs program" model, I've seen the above issues. In fact, we have those exact issues very much in git itself too. Yes, I've used libgit2 (for subsurface). It's a pain in the arse to do *exactly* the above kinds of things, and the thing is, that isn't git-specific. So I'm very down on using external libraries unless they are stable and have no need for configuration etc. Things like zlib is fine - there just isn't much to configure outside of the "just how hard do you want me to try to compress". Nobody has a ".zlib/config" file that you need to worry about accessing etc. Of course, maybe pgpme is a world first, and actually does read your .gnupg/config file trivially, and has all the gpg agent integration that it picks up automatically, and allows various per-user configurations, and all my worries are bogus. But that would literally be the first time I've ever seen that. Linus ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-10 18:54 ` Linus Torvalds @ 2017-03-10 20:26 ` Theodore Ts'o 2017-03-13 11:14 ` Bernhard E. Reiter 1 sibling, 0 replies; 19+ messages in thread From: Theodore Ts'o @ 2017-03-10 20:26 UTC (permalink / raw) To: Linus Torvalds; +Cc: Bernhard E. Reiter, Git Mailing List, gnupg-devel On Fri, Mar 10, 2017 at 10:54:19AM -0800, Linus Torvalds wrote: > - library versioning. > > I don't know why, but I've never *ever* met a library developer who > realized that libraries were all about stable API's, and the library > users don't want to fight different versions. Actually, you have. (Raises hand :-) libext2fs has a stable API *and* ABI. We add new functions instead of changing function parameters (so ext2fs_block_iterate2() is implemented in terms of ext2fs_block_iterate3(), and so on). And structures have magic numbers that have served as versioning signal. This is actually not rocket science. If you've met anyone who's programmed for Multics, they did something similar. And of course, that's why we have the wait3(2) and wait(4) system calls. I do have to agree with your general point, that most developers tend to be *incredibly* sloppy with their interfaces. That being said, not all library developers are as bad as GNOME. :-) - Ted ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-10 18:54 ` Linus Torvalds 2017-03-10 20:26 ` Theodore Ts'o @ 2017-03-13 11:14 ` Bernhard E. Reiter 2017-03-13 12:53 ` Jeff King 1 sibling, 1 reply; 19+ messages in thread From: Bernhard E. Reiter @ 2017-03-13 11:14 UTC (permalink / raw) To: Linus Torvalds; +Cc: Git Mailing List, gnupg-devel [-- Attachment #1: Type: text/plain, Size: 6185 bytes --] Am Freitag 10 März 2017 19:54:19 schrieb Linus Torvalds: > On Fri, Mar 10, 2017 at 2:00 AM, Bernhard E. Reiter > > please consider using libgpgme interfacing to GnuPG, because the gpg > > command-line interface is not considered an official API to GnuPG by the > > GnuPG-devs and thus potentially unstable. > > Quite frankly, I will NAK this just based on previous bad experiences > with using "helpful" libraries. > > Maybe you can lay my worries to rest, but the problems with libraries > in this context tend to be As gpgme is not just a helpful library, but the official API to GnuPG, it is well supported by the GnuPG-Initiative itself and stable. Still there could be problems and of course in some situations the disadvantages outweigh the advantages. On the other hand we have seen a number of systematic problems with "just calling gpg" that libgpgme tries to provide a solution to. So it is too early to say that libgpgme would be right choice for git to me, but it should be seriously considered. Grateful that you have written down some of your concern, let me try give you some pointers. > - hard to personalize. > > At least right now, we just allow people to set their gpg binary. > I'm betting that the library would pick whatever random preferred > version, and in the process possibly screwed us. > > Example: what if somebody is actually using another pgp > implementation entirely for some reason, and is just scripting around > it? > Or maybe he's using the regular gnupg, but using different keys for > different projects (using "--homedir"). That's trivial with the > current model. How trivial is that with a library? https://www.gnupg.org/documentation/manuals/gpgme/Engine-Configuration.html " You can change the configuration of a backend engine, and thus change the executable program and configuration directory to be used. You can make these changes the default or set them for some contexts individually. " Using a completely different OpenPGP implementation maybe a potential use case for keeping a configuration option around. I did not deeply examine what git really needs. Usually a different implementation will have quite a different command line interface, so it may require substaintial work to come up with a wrapper about that other OpenPGP implementation to provide the same command line interface as GnuPG. > - existing configuration > > This is the main problem I've seen in the past. Using the "ssh" > _program_ is easy. You add your keys, your config files, whatever, and > it "just works" (or rather, you fight it once and it definitely > doesn't "just" work, but then you copy your .ssh directory around for > the rest of your and forget how it ever worked, but it does). gpgme via gpg uses the existing configuration files (which you can also read and modify with gpgconf for implementiong GUIs). > - UI > > For things like gpg, the UI is traditionally horrible. But there > tends to be various things like password agents that help with caching > passwords and/or add a graphical UI to get the password when > necessary. As the gpg binary itself speaks to gpg-agent, this is fully integrated when used via gpgme. Our GPA and Kleopatra GUIs work fine with gpgme and others as well, because they call come together in the deeper engine functions of GnuPG. > - library versioning. > > I don't know why, but I've never *ever* met a library developer who > realized that libraries were all about stable API's, and the library > users don't want to fight different versions. In my experience Werner (the lead GnuPG developers) is quite reasonable about keeping APIs stable (he often goes out of his way to keep even the command line version stable, maybe he shouldn't do that to the command line options so you are more motivated to go to this official API gpgme. >:) ) > And to make matters worse, the different versions (particularly if > you end up having to use a development version due to bugs or required > features etc) are always made horribly bad to even detect at > built-time automatically with simple #ifdef etc, so now you have to do > autoconf crap etc. https://www.gnupg.org/documentation/manuals/gpgme/Library-Version-Check.html " The function gpgme_check_version has four purposes. It can be used to retrieve the version number of the library. In addition it can verify that the version number is higher than a certain required version number. In either case, the function initializes some sub-systems, and for this reason alone it must be invoked early in your program, before you make use of the other functions in GPGME. The last purpose is to run selftests. As a side effect for W32 based systems, the socket layer will get initialized. " > Now, it may be that the pgpme library "just works" across > architectures and handles all of the above situations as gracefully as > the external program does. In that case - but _ONLY_ in that case - > would a switch-over to the library possibly be a good thing. At least gpgme aims to fulfill these goals (and is used on many architectures). > I'd be pleasantly surprised. But I *would* be surprised, because every > time I've seen that "library vs program" model, I've seen the above > issues. Your concerns are understandable, I've seen similiar problems with "library vs program" and the unix tools box approach gives a number of lessons on how to losely couple components. Thanks again for taking the time and writing them down. I've given you some pointers why gpgme indeed could be different and may be an improvement for git (or other applications). I guess one of the next steps would be for someone to look for specific points or try gpgme for git purposes. Me and gnupg-devel@ are happy to take your questions or get feedback. Regards, Bernhard -- www.intevation.de/~bernhard (CEO) +49 541 33 508 3-3 Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998 Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-13 11:14 ` Bernhard E. Reiter @ 2017-03-13 12:53 ` Jeff King 0 siblings, 0 replies; 19+ messages in thread From: Jeff King @ 2017-03-13 12:53 UTC (permalink / raw) To: Bernhard E. Reiter; +Cc: Linus Torvalds, Git Mailing List, gnupg-devel On Mon, Mar 13, 2017 at 12:14:17PM +0100, Bernhard E. Reiter wrote: > Using a completely different OpenPGP implementation maybe a potential use case > for keeping a configuration option around. I did not deeply examine what git > really needs. Usually a different implementation will have quite a different > command line interface, so it may require substaintial work to come up with a > wrapper about that other OpenPGP implementation to provide the same command > line interface as GnuPG. The existing config option will have to stay, period. If we take it away, then we'd be breaking existing setups that do anything more exotic than point it at a particular gpg binary. So I think the possible patches in this area are: - adding gpgme support (probably as an optional build-time dependency) along with a config option to use it instead of gpg.program (or possibly just "gpg.program = gpgme" to trigger it). - defaulting to gpgme (when it's compiled in) instead of gpg.program The first one by itself shouldn't hurt anybody, aside from carrying some extra code around. We'd have to see how the first one goes before considering the second. If we hit a point where people have problems and the advice is "just set gpg.program to gpgme", then that's a good sign that the default should be flipped. -Peff ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason 2017-03-10 18:54 ` Linus Torvalds @ 2017-03-11 0:10 ` brian m. carlson 2017-03-13 12:29 ` Bernhard E. Reiter 2017-03-13 19:48 ` Christian Neukirchen 2 siblings, 2 replies; 19+ messages in thread From: brian m. carlson @ 2017-03-11 0:10 UTC (permalink / raw) To: Bernhard E. Reiter; +Cc: git, gnupg-devel [-- Attachment #1: Type: text/plain, Size: 3122 bytes --] On Fri, Mar 10, 2017 at 11:00:07AM +0100, Bernhard E. Reiter wrote: > My use case today was signing and git by default found the `gpg` binary by > default and the command failed. The reason is that I have `gpg2` installed > and most applications use it right away. So git failed signing because > the .gnupg configuration of the user was not ready for the old `gpg` which is > still installed on Debian GNU/Linux for purposes of the operating system. If > git would have used libgpgme, gpgme would have choosen the most uptodate > version of `gpg` available (or configured) without me intervening via > gpg.program. Now because of this problem you could adding a check for `gpg2` > and fallback to `gpg`, but even better would be to move to libgpgme. >:) There are a couple potential problems I see with this approach. First, I'd want to know whether gpgme supports gpgsm, which I know some people use to sign commits and tags. Another issue is what happens to the git verify-* --raw output. Some people want the ability to script signature verification. This can be really important when you have automated systems verifying tags and commits. For example, running the following commands, we can determine that Junio signs his tags with SHA-1 (algorithm 2), while I sign my commits with SHA-512 (algorithm 10). genre ok % git verify-tag --raw v2.12.0 2>&1 | grep VALIDSIG [GNUPG:] VALIDSIG E1F036B1FEE7221FC778ECEFB0B5E88696AFE6CB 2017-02-24 1487962205 0 4 0 1 2 00 96E07AF25771955980DAD10020D04E5A713660A7 genre ok % git verify-commit --raw object-id-part10 2>&1 | grep VALIDSIG [GNUPG:] VALIDSIG 5FC3A781776B26DF87F70C37BF535D811F52F68B 2017-03-06 1488760639 0 4 0 1 10 00 88ACE9B29196305BA9947552F1BA225C0223B187 There's literally no other way to get this information at the moment (which is why I added the --raw option). A gpgme implementation would need to expose this same information, at which point, we might as well have used gpg directly. This is not an idle consideration; we have automated systems at work that update software automatically and submit it for human review, including verifying signatures and hashes. This saves hundreds of hours of staff time and results in better security. Because the amount of the gpg API we actually use is very small, a user who wants to use a custom signature program (say, OpenBSD's signify), can actually write a simple wrapper that mimics it and use that instead. Finally, I work on a development system where work is done both as an unprivileged user and as root. Because I use the same socket for both, GnuPG screams bloody murder that the permissions are wrong. I know this is secure in my scenario, but without a custom wrapper, I have to deal with GnuPG polluting my terminal every time I sign a commit or a tag. A gpgme implementation would need to honor the same wrapper script or otherwise not scream to the terminal. -- brian m. carlson / brian with sandals: Houston, Texas, US +1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only OpenPGP: https://keybase.io/bk2204 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 868 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-11 0:10 ` brian m. carlson @ 2017-03-13 12:29 ` Bernhard E. Reiter 2017-03-13 19:48 ` Christian Neukirchen 1 sibling, 0 replies; 19+ messages in thread From: Bernhard E. Reiter @ 2017-03-13 12:29 UTC (permalink / raw) To: brian m. carlson; +Cc: git, gnupg-devel [-- Attachment #1: Type: text/plain, Size: 3510 bytes --] Am Samstag 11 März 2017 01:10:31 schrieb brian m. carlson: > On Fri, Mar 10, 2017 at 11:00:07AM +0100, Bernhard E. Reiter wrote: > > but even better would be to move to libgpgme. >:) > > There are a couple potential problems I see with this approach. First, > I'd want to know whether gpgme supports gpgsm, which I know some people > use to sign commits and tags. Yes, gpgme supports gpgsm. https://www.gnupg.org/documentation/manuals/gpgme/Cryptographic-Message-Syntax.html "CMS is implemented by GpgSM, the S/MIME implementation for GnuPG." > Another issue is what happens to the git verify-* --raw output. Some > people want the ability to script signature verification. This can be > really important when you have automated systems verifying tags and > commits. > > For example, running the following commands, we can determine that Junio > signs his tags with SHA-1 (algorithm 2), while I sign my commits with > SHA-512 (algorithm 10). > > genre ok % git verify-tag --raw v2.12.0 2>&1 | grep VALIDSIG > [GNUPG:] VALIDSIG E1F036B1FEE7221FC778ECEFB0B5E88696AFE6CB 2017-02-24 > 1487962205 0 4 0 1 2 00 96E07AF25771955980DAD10020D04E5A713660A7 genre ok % > git verify-commit --raw object-id-part10 2>&1 | grep VALIDSIG [GNUPG:] > VALIDSIG 5FC3A781776B26DF87F70C37BF535D811F52F68B 2017-03-06 1488760639 0 4 > 0 1 10 00 88ACE9B29196305BA9947552F1BA225C0223B187 > > There's literally no other way to get this information at the moment > (which is why I added the --raw option). A gpgme implementation would > need to expose this same information, at which point, we might as well > have used gpg directly. I'm quite optimistic that the use case can be covered implementing the git --raw option using gpgme, but I don't know the best way right away. (I also wonder what would happen if someone manages to put in two or more signatures in the object you are verifying.) > Because the amount of the gpg API we actually use is very small, a user > who wants to use a custom signature program (say, OpenBSD's signify), > can actually write a simple wrapper that mimics it and use that instead. I agree that this can be a use case against using libgpgme. > Finally, I work on a development system where work is done both as an > unprivileged user and as root. Because I use the same socket for both, > GnuPG screams bloody murder that the permissions are wrong. I know this > is secure in my scenario, but without a custom wrapper, I have to deal > with GnuPG polluting my terminal every time I sign a commit or a tag. A > gpgme implementation would need to honor the same wrapper script or > otherwise not scream to the terminal. I don't understand the scenario well enought to advise. Using the same socket sounds strange at the onset. In general libgpgme allows someone to use other binaries (see my answer to Linus) and offers quite a number of possibilities. However there may be special cases that are not covered as good as using everything raw and manually. Libgpgme has to be this way and offering some standard default otherwise it would use some of its merits. On the other hand the code doing special things or calling gpg directly can have defects as well which is a significant drawback. Regards, Bernhard -- www.intevation.de/~bernhard (CEO) +49 541 33 508 3-3 Intevation GmbH, Osnabrück, Germany; Amtsgericht Osnabrück, HRB 18998 Owned and run by Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Stable GnuPG interface, git should use GPGME 2017-03-11 0:10 ` brian m. carlson 2017-03-13 12:29 ` Bernhard E. Reiter @ 2017-03-13 19:48 ` Christian Neukirchen 1 sibling, 0 replies; 19+ messages in thread From: Christian Neukirchen @ 2017-03-13 19:48 UTC (permalink / raw) To: git "brian m. carlson" <sandals@crustytoothpaste.net> writes: > Because the amount of the gpg API we actually use is very small, a user > who wants to use a custom signature program (say, OpenBSD's signify), > can actually write a simple wrapper that mimics it and use that instead. FWIW, I did this, and it was quite some effort to figure out the actual API that is needed: http://chneukirchen.org/dotfiles/bin/git-signify -- Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2017-03-23 11:03 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-03-10 10:00 Stable GnuPG interface, git should use GPGME Bernhard E. Reiter 2017-03-10 14:23 ` Ævar Arnfjörð Bjarmason 2017-03-13 10:14 ` Michael J Gruber 2017-03-13 12:49 ` Bernhard E. Reiter 2017-03-14 10:39 ` Michael J Gruber 2017-03-17 9:56 ` Bernhard E. Reiter 2017-03-22 17:15 ` Werner Koch 2017-03-22 18:46 ` Peter Lebbing 2017-03-23 6:52 ` Werner Koch 2017-03-23 7:29 ` Bernhard E. Reiter 2017-03-23 10:56 ` Werner Koch 2017-03-13 10:30 ` Bernhard Reiter 2017-03-10 18:54 ` Linus Torvalds 2017-03-10 20:26 ` Theodore Ts'o 2017-03-13 11:14 ` Bernhard E. Reiter 2017-03-13 12:53 ` Jeff King 2017-03-11 0:10 ` brian m. carlson 2017-03-13 12:29 ` Bernhard E. Reiter 2017-03-13 19:48 ` Christian Neukirchen
Code repositories for project(s) associated with this public inbox https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).