GPG Commit Signing Project

GPG Commit Signing Project

[Jimit Bhalavat]

Hello Junio,

I am Jimit Bhalavat, and as you know I have been selected by The Linux Foundation to work on the GPG Commit Signing Project. My mentor is David Huseby. I have spent the last few weeks working on a Technical Design Document which specifies how the commit signing works in git and what functions are called. I have done the same for verifying signatures, commit tags, and verifying tags. The aim of this project is that we want to focus entirely on refactoring the signing infrastructure to make it easy for users to support their choice of signing tool while preserving full backwards compatibility with the way things work today. We are NOT at all trying to inject any "emerging technology" into Git, just merely loosening the binding between Git and GPG enough so that users have more choice with signing tools.

The detailed technical design proposal is below: 

What changes did patches make last summer?: The project’s goal was to abstract the current GPG interface into a more ‘generic’ API that supports the existing OpenPGP and X.509 functionality while creating the ability to write additional signature drivers. Currently git supports signing/verifying commits and tags using GPG only. The goal of the project was to make the git signing interface compatible with external signing tools. The main sections of the project were updating the user configuration and the command handling when a signing or verifying operation occurs. The prototype of Ibrahim’s patches has already implemented configuration aliases to ensure that the new approach is backwards compatible with previous configuration. The patches also worked towards improving the documentation for clear transition, along with maintaining backward compatibility. 

Why were the patches from last summer rejected?: The patches from last summer opted for a config based approach and a tool-agnostic signing interface. The main goal of the project was to abstract the current GPG interface into a more ‘generic’ API that supports the existing OpenPGP and X.509 functionality while creating the ability to write additional signature drivers. The project proposal had already implemented configuration aliases to ensure that the new approach is backwards compatible with previous configuration. The other reasons include: the drivers for external signing tools were still written in C and the patches were not more for a configuration based interface. The patches implement updated user configuration to define signing tools and implement a tool-agnostic signing interface in C code. It also added the possibility to use bash helper scripts to drive additional tools in case the default interface doesn't work as intended.

Where are we going next?/What we are trying to achieve?: The main goal of this year’s project is instead of having C code for each new signing tool, we will try define a new  standardized pipe forking interface that takes the values from the config file. The config file will contain the following values: which executable to run, what parameters to pass, what regular expressions to use for detecting signature type and pulling the signature out of commits and tags. Part of the patches from last summer update the config format to support any number of different signing tools. If the config file does not have any values using the new signing section format, it defaults to the GPG executable, GPG regular expressions, and the GPG parameters. This ensures that we maintain backward compatibility and preserve existing behaviour for users who have not adopted the new config values. We are going to deprecate things, but old options will not stop working.

The proposal:

	1. We aim to achieve that instead of having C code that calls all the signing tools, we will try to achieve that only one piece of code will do the standardized pipe forking and take the values from the config file. The config file will contain the following values: which executable to run, what parameters to pass, what regular expressions to use for detecting signature type and pulling the signature out of commits and tags.
	2. The current patch uses a set of function pointers, but this project aims to use a struct with the configuration data for the detected signature type such as PGP, x509, minisig, etc.
		a. When signing, the signature type either comes from the command-line switch or from the signing default as specified in the config file. The config file has different parameters: which executable to run, what parameters to pass, what regular expressions to use for detecting signature type and pulling the signature out of commits and tags. The goal is to try and call the signing tools from one piece of code instead multiple C code. If the config file does not have any values, it defaults to the GPG executable, GPG regular expressions, and the GPG parameters.
		b. During the verification process, the signature type is detected by using regular expressions from the config file. The regular expression searches for metadata for a matching signature, and the first one that matches, that format is returned and used for the verification. This design also opens up the possibility of having multiple signatures on a commit/tag.
		c. Once format is determined, a signing_tool struct is initialized for the given format. The struct contains all of the config data for the given tool (e.g. program to run, regex’s, command line switches, etc).
	3. The project also aims to improve the signing-interface.c file to work from the signing_tool struct which we created above which contains all of the configuration data for the given tool such as what program to run, regular expressions, and command line switches. The flow of the code should look like this:
		a. Pass in a signing tool struct to sign commit
		b. Instead of calling function pointers to the signing-tool-specific “drivers”, it will instead do what the existing code does; however the executable and command line switches all come from the config/defaults instead of being hard coded..
	4. Create a struct for signer_identity:
		a. This struct will be helpful to load signer_identity from config (e.g. the existing signingkey config item) or command line.
		b. It stores the signing-tool-specific signer identity string/identifier, so that it can be passed to the signing tool.
	5. All of these changes will make sure that we maintain backwards compatibility. We are aiming to deprecate things a little but old options will not stop working. We will preserve existing behavior. The goal is to try and call the signing tools from one piece of code instead multiple C code. 

Thank you for your time and we appreciate you looking into it. We are open to having conversations and work with you to make a final proposal which we can use to move forward. 

Thank you once again.

Best,
Jimit Bhalavat.

Re: GPG Commit Signing Project

[Junio C Hamano]

Jimit Bhalavat <jimit@rams.colostate.edu> writes:

> Hello Junio,

Sorry, I read the message twice but I am not sure what you expect
out of me.  We usually do not evaluate a proposal only from its
"design document" in the sense that we won't say neither "such a
proposal is great" or "we won't accept such a proposal" until we
actually see patches.  It is a good idea to have a written design as
it may help reviewers when they review patch submission.

> 	5. All of these changes will make sure that we maintain
> 	backwards compatibility. We are aiming to deprecate things a
> 	little but old options will not stop working. We will
> 	preserve existing behavior.

That is always a requirement by our project so it goes without
saying ;-) but it is good to see the promise written down.

Re: GPG Commit Signing Project

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 3688 bytes --]

On 2020-08-18 at 18:43:45, Jimit Bhalavat wrote:
> The proposal:
> 
> 	1. We aim to achieve that instead of having C code that calls all the signing tools, we will try to achieve that only one piece of code will do the standardized pipe forking and take the values from the config file. The config file will contain the following values: which executable to run, what parameters to pass, what regular expressions to use for detecting signature type and pulling the signature out of commits and tags.

Assuming the config file is the standard one and not a custom config
file, this seems fine.

> 	2. The current patch uses a set of function pointers, but this project aims to use a struct with the configuration data for the detected signature type such as PGP, x509, minisig, etc.
> 		a. When signing, the signature type either comes from the command-line switch or from the signing default as specified in the config file. The config file has different parameters: which executable to run, what parameters to pass, what regular expressions to use for detecting signature type and pulling the signature out of commits and tags. The goal is to try and call the signing tools from one piece of code instead multiple C code. If the config file does not have any values, it defaults to the GPG executable, GPG regular expressions, and the GPG parameters.
> 		b. During the verification process, the signature type is detected by using regular expressions from the config file. The regular expression searches for metadata for a matching signature, and the first one that matches, that format is returned and used for the verification. This design also opens up the possibility of having multiple signatures on a commit/tag.

Having multiple signatures is tricky, but I'm willing to reserve
judgement until I see this.  I'm not opposed in principle.

> 		c. Once format is determined, a signing_tool struct is initialized for the given format. The struct contains all of the config data for the given tool (e.g. program to run, regex’s, command line switches, etc).
> 	3. The project also aims to improve the signing-interface.c file to work from the signing_tool struct which we created above which contains all of the configuration data for the given tool such as what program to run, regular expressions, and command line switches. The flow of the code should look like this:
> 		a. Pass in a signing tool struct to sign commit
> 		b. Instead of calling function pointers to the signing-tool-specific “drivers”, it will instead do what the existing code does; however the executable and command line switches all come from the config/defaults instead of being hard coded..
> 	4. Create a struct for signer_identity:
> 		a. This struct will be helpful to load signer_identity from config (e.g. the existing signingkey config item) or command line.
> 		b. It stores the signing-tool-specific signer identity string/identifier, so that it can be passed to the signing tool.
> 	5. All of these changes will make sure that we maintain backwards compatibility. We are aiming to deprecate things a little but old options will not stop working. We will preserve existing behavior. The goal is to try and call the signing tools from one piece of code instead multiple C code.

As Junio said, we prefer to look at patches rather than a proposal, but
I so far am happy with what you've proposed.  Assuming your code is of
good quality, I can see this being a viable design and I'd be happy to
see these changes come in.

I expect some conflicts with some future SHA-256 work I'm doing, but
that's my problem, not yours.
-- 
brian m. carlson: Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

GPG Commit Signing Project

[Jimit Bhalavat]

Hello Junio,

When we last spoke, David Huseby, my mentor for Hyperledger Git Commit Signing Project, proposed an outline of what we are trying to achieve. We have come to the conclusion that me and him are going to spend the rest of the summer analyzing the format signing infrastructure and will have proposals and questions as we go. I am also going to write a technical design document which will help us analyze the problem. I realize this is a larger long-term project that I think deserves a topic and how would I set a topic up? 

I am also going to be thoroughly reading and understanding the patches from last summer which will help me guide through the format. If you are not the right person to be in contact with, please put me in touch with the right person, or if you don’t mind, please forward this email to them so that we can be in touch. 

I really appreciate all your help, thank you once again, and I will have proposals and questions as we move forward. 

Thank you.

Be well and stay safe,
Jimit Bhalavat

Re: GPG Commit Signing Project

[Junio C Hamano]

Jimit Bhalavat <jimit@rams.colostate.edu> writes:

> When we last spoke, David Huseby, my mentor for Hyperledger Git
> Commit Signing Project, proposed an outline of what we are trying
> to achieve. We have come to the conclusion that me and him are
> going to spend the rest of the summer analyzing the format signing
> infrastructure and will have proposals and questions as we go.

I thought the conclusion was that there is nothing to do on the git
side as your project would wrap the non-GPG signing tool of your
choice to have an external interface that is similar to how GPG
looks to git and that has the added benefit that the product of your
project to interact with tools other than git [*1*].  So...

> I am also going to write a technical design document which will
> help us analyze the problem. I realize this is a larger long-term
> project that I think deserves a topic and how would I set a topic
> up?

... your longer-term plan may very well deserve a detailed technical
design document, but I am not sure what help our project can give
(other than obvious things like saying "good luck").

Sorry for not being able to help all that much.




[Reference]

*1* https://lore.kernel.org/git/xmqqd0642p3q.fsf@gitster.c.googlers.com/
 

GPG Commit Signing Project

[Jimit Bhalavat]

Good Afternoon, 

Hope you are doing well. As you know, I have been selected by The Linux Foundation to work on the Commit Signing Project. We are trying again to make the signing subsystem generic so that we can use other signing tools while preserving backward compatibility with existing .gitconfigs and setups. The code changes are pretty significant. 

Do you think we should create a topic under the pu branch? 

The below link shows some of the topics currently in pu:

https://github.com/git/git/commits/pu

Thank you once again. I really appreciate all your help and guidelines. 

Be well and Stay safe,
Jimit Bhalavat.

GPG Commit Signing Project

[Jimit Bhalavat]

Good Afternoon, 

I am Jimit Bhalavat, and I am a Junior at Colorado State University and my major is Computer Science. Recently, I accepted to work on Hyperledger Git Commit Signing Project through The Linux Foundation and my mentor is David Huseby. I am writing to you in order to ask you if you are the maintainer for the GPG Signing Project? 

Which branches are for refactoring/new features in the GPG Commit Signing Project?

Thank you so much. Have a great rest of your day.

Best,
Jimit Bhalavat.

Re: GPG Commit Signing Project

[Junio C Hamano]

Jimit Bhalavat <jimit@rams.colostate.edu> writes:

> I am Jimit Bhalavat, and I am a Junior at Colorado State
> University and my major is Computer Science. Recently, I accepted
> to work on Hyperledger Git Commit Signing Project through The
> Linux Foundation and my mentor is David Huseby. I am writing to
> you in order to ask you if you are the maintainer for the GPG
> Signing Project?
>
> Which branches are for refactoring/new features in the GPG Commit
> Signing Project?

The fact that I know almost nothing about "Hyperledger Git Commit
Signing Project" (other than the search term returns some hits from
the search engines [*1*]) makes me suspect that whatever branch I
control is not suitable to contribute to that project, which does
not have much to do with the Git project, where this mailing list is
its home for.  Perhaps ask your mentor first?


[Reference]

*1* https://wiki.hyperledger.org/display/INTERN/Git+signing+with+DIDs

Re: GPG Commit Signing Project

[dwh]

On 10.06.2020 12:35, Junio C Hamano wrote:
>The fact that I know almost nothing about "Hyperledger Git Commit
>Signing Project" (other than the search term returns some hits from
>the search engines [*1*]) makes me suspect that whatever branch I
>control is not suitable to contribute to that project, which does
>not have much to do with the Git project, where this mailing list is
>its home for.  Perhaps ask your mentor first?

Hello Junio,

I thought I should jump in here and introduce myself and give Jimit
a little help. My name is Dave Huseby and I'm Jimit's mentor. I'm also
the Security Maven for the Hyperledger Project. Jimit was selected for
our Summer 2020 mentorship project to work on our ongoing efforts to
support alternative signing tools in Git. Last summer a series of
patches were submitted by Ibrahim and it was not accepted, although
we did get some good feedback.

The feedback from the Git community was that the refactor of the
signing system organized the signing-tool-specific C code into
"drivers" for each signing tool instead of being configuration based.
See Brian's comment here:

https://public-inbox.org/git/20190826231543.GD11334@genre.crustytoothpaste.net/

Ibrahim's mentorship ended with him sending a new proposal for a config
based approach to solve this problem here:

https://public-inbox.org/git/R3X1WzWH0sgOh85GuUmXwsTC6CPKysi4TRzN_BPecDVGr__ET2-mitZ2DZA0_bpKkzLRtnTtoomIWxZtL52_1XkihYBVBAuWMpSdwoboixY=@pm.me/T/#u

I now think even that proposal is overly complicated. I think the
easiest solution is to simply standardize the existing pipe-fork
interface as the way GPG talks to all signing tools. For signing tools
that have different command line interfaces than GPG, we can create
adapter scripts. Tools that want to be compatible can adapt.

I'll outline a new proposal in a follow up email.

Cheers!
Dave

Re: GPG Commit Signing Project

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1110 bytes --]

On 2020-06-12 at 01:55:56, dwh@linuxprogrammer.org wrote:
> I now think even that proposal is overly complicated. I think the
> easiest solution is to simply standardize the existing pipe-fork
> interface as the way GPG talks to all signing tools. For signing tools
> that have different command line interfaces than GPG, we can create
> adapter scripts. Tools that want to be compatible can adapt.

This becomes pretty tricky because Git parses OpenPGP headers in a
variety of places (e.g., at the end of tags).  If your proposal is to
wrap new formats in a fake OpenPGP format, like some existing tools do,
then that would be viable, but otherwise you're going to require either
Git to know about your signing format specifically (which is not a
sustainable approach) or some sort of configuration framework like has
been previously discussed.

If you're going to wrap things in a fake OpenPGP format, then you don't
actually need to send any patches to Git at all; you can simply set
gpg.program and continue.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: GPG Commit Signing Project

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> On 2020-06-12 at 01:55:56, dwh@linuxprogrammer.org wrote:
>> I now think even that proposal is overly complicated. I think the
>> easiest solution is to simply standardize the existing pipe-fork
>> interface as the way GPG talks to all signing tools. For signing tools
>> that have different command line interfaces than GPG, we can create
>> adapter scripts. Tools that want to be compatible can adapt.
>
> This becomes pretty tricky because Git parses OpenPGP headers in a
> variety of places (e.g., at the end of tags).  If your proposal is to
> wrap new formats in a fake OpenPGP format, like some existing tools do,
> then that would be viable, but otherwise you're going to require either
> Git to know about your signing format specifically (which is not a
> sustainable approach) or some sort of configuration framework like has
> been previously discussed.
>
> If you're going to wrap things in a fake OpenPGP format, then you don't
> actually need to send any patches to Git at all; you can simply set
> gpg.program and continue.

True enough ;-)  Thanks for a concise summary of the situation.

GPG Commit Signing Project

[Jimit Bhalavat]

Hello, 

Hope you are doing good. Which branches are for refactoring/new features in the GPG Commit Signing Project?

Thank you so much. 

Best,
Jimit Bhalavat.