Proposal: server-advertised config options

Proposal: server-advertised config options

[Drew DeVault]

The basic idea is that the server could advertise some config options
which it recommends the client sets for a given repo after a fetch. Some
possible use-cases for this feature include setting:

- format.subjectPrefix to 'PATCH my-project'
- sendemail.to to the mailing list address
- push.pushOption to recommended push options

Upon cloning, each recommended config option would be displayed to the
user, and they would be prompted ([Y/n]) to agree to set that value in
the config file for that repository. Additionally, there would be a
config option which white-lists a list of config options which are
automatically agreed to without prompting, and each config option would
have one of the following states:

- accept-silent: set the option without printing a message
- accept-verbose: set the option and display a message
- prompt: prompt the user to set this config option
- reject-silent: silently refuse to set this config option
- reject-verbose: refuse to set this config option and display a message

We would default to reject-verbose for all unknown config options and
include a set of defaults which specifies the appropriate trust level
for various useful benign options (such as the examples above).

The implementation would involve advertising config-advertisement in the
fetch protocol. Both the client and server would have to agree to use
it. If the server supports it but the client does not (in the case of an
old client), it could fall back to printing the list of recommended
options to stderr.

To choose which config options to advertise, a new option would be
introduced (uploadpack.advertiseOptions) for example, which has a list
of .git/config options from the remote repository to forward to the
client.

This would be a lot of work so I'd like to float it for discussion
before getting started. What do you guys think?

Re: Proposal: server-advertised config options

[Christian Couder]

On Mon, Sep 7, 2020 at 7:06 PM Drew DeVault <sir@cmpwn.com> wrote:
>
> The basic idea is that the server could advertise some config options
> which it recommends the client sets for a given repo after a fetch. Some
> possible use-cases for this feature include setting:
>
> - format.subjectPrefix to 'PATCH my-project'
> - sendemail.to to the mailing list address
> - push.pushOption to recommended push options

It could be useful to suggest promisor/partial clone remote config options too.

> Upon cloning, each recommended config option would be displayed to the
> user, and they would be prompted ([Y/n]) to agree to set that value in
> the config file for that repository.

Maybe the default should be "N" instead of "Y" for more security. Also
when not using a terminal, it should do nothing by default too.

> Additionally, there would be a
> config option which white-lists a list of config options which are
> automatically agreed to without prompting,

This might be dangerous if this option can also be proposed by the
server, as it could first propose a big list of white listed options
to the client.

> and each config option would
> have one of the following states:
>
> - accept-silent: set the option without printing a message
> - accept-verbose: set the option and display a message
> - prompt: prompt the user to set this config option
> - reject-silent: silently refuse to set this config option
> - reject-verbose: refuse to set this config option and display a message
>
> We would default to reject-verbose for all unknown config options and
> include a set of defaults which specifies the appropriate trust level
> for various useful benign options (such as the examples above).
>
> The implementation would involve advertising config-advertisement in the
> fetch protocol. Both the client and server would have to agree to use
> it. If the server supports it but the client does not (in the case of an
> old client), it could fall back to printing the list of recommended
> options to stderr.
>
> To choose which config options to advertise, a new option would be
> introduced (uploadpack.advertiseOptions) for example, which has a list
> of .git/config options from the remote repository to forward to the
> client.
>
> This would be a lot of work so I'd like to float it for discussion
> before getting started. What do you guys think?

My opinion is that you might not want to start working on all the
above at once. It might be better to start small and safe while
leaving the door open to further improvements.

Re: Proposal: server-advertised config options

[Drew DeVault]

On Mon Sep 7, 2020 at 2:49 PM EDT, Christian Couder wrote:
> > Upon cloning, each recommended config option would be displayed to the
> > user, and they would be prompted ([Y/n]) to agree to set that value in
> > the config file for that repository.
>
> Maybe the default should be "N" instead of "Y" for more security. Also
> when not using a terminal, it should do nothing by default too.

Ack, ack.

> > Additionally, there would be a config option which white-lists a
> > list of config options which are automatically agreed to without
> > prompting,
>
> This might be dangerous if this option can also be proposed by the
> server, as it could first propose a big list of white listed options
> to the client.

Aye, I think we'd prevent the server from advertising that option
period, as a hard-coded restriction.

> My opinion is that you might not want to start working on all the
> above at once. It might be better to start small and safe while
> leaving the door open to further improvements.

While this work could easily be (and ought to be) broken up into small
commits which introduce it one piece at a time, I'm not sure that any
subset of the pieces is *shippable*. Do you have a suggestion for how it
could be broken up into small, shippable pieces?

Re: Proposal: server-advertised config options

[Junio C Hamano]

"Drew DeVault" <sir@cmpwn.com> writes:

> The basic idea is that the server could advertise some config options
> which it recommends the client sets for a given repo after a fetch. Some
> possible use-cases for this feature include setting:
>
> - format.subjectPrefix to 'PATCH my-project'
> - sendemail.to to the mailing list address
> - push.pushOption to recommended push options
>
> Upon cloning, each recommended config option would be displayed to the
> user, and they would be prompted ([Y/n]) to agree to set that value in
> the config file for that repository. Additionally, there would be a
> config option which white-lists a list of config options which are
> automatically agreed to without prompting, and each config option would
> have one of the following states:
>
> - accept-silent: set the option without printing a message
> - accept-verbose: set the option and display a message
> - prompt: prompt the user to set this config option
> - reject-silent: silently refuse to set this config option
> - reject-verbose: refuse to set this config option and display a message
>
> We would default to reject-verbose for all unknown config options and
> include a set of defaults which specifies the appropriate trust level
> for various useful benign options (such as the examples above).
>
> The implementation would involve advertising config-advertisement in the
> fetch protocol. Both the client and server would have to agree to use
> it. If the server supports it but the client does not (in the case of an
> old client), it could fall back to printing the list of recommended
> options to stderr.
>
> To choose which config options to advertise, a new option would be
> introduced (uploadpack.advertiseOptions) for example, which has a list
> of .git/config options from the remote repository to forward to the
> client.
>
> This would be a lot of work so I'd like to float it for discussion
> before getting started. What do you guys think?

Assuming I am among the guys (do you solicit opinions from gals, by
the way?), here are a few unconnected random thoughts.

I do not want to see this as a "server" thing.  All the examples are
"per project preference" and I do agree it would be nice to have a
standardised way for projects to communicate their preference to
their participants.  Regardless of the hosting site I clone and
fetch my project from, I'd want to see it communicated consistently
to them.

Which means that it must not be a patch to the "server" component to
what responds to your "git fetch" and "git clone" (i.e. upload-pack)
as some hosting sites do not even use upload-pack.

Also, I do not want to see this as a "git" thing and I mean it in
two ways.  

In addition to your examples of "per project preference", there are
projects' coding style guides, etc., that we do not enforce as git
config at all, e.g. how wide your editors TAB and single level of
indentation should be.  It will unnecessarily narrow your view to
assume that the kind of "per project preference" you convey from the
project to its participants need to be the Git configuration and
nothing else.

And this should not be a "git the SCM" thing.  If you download and
extract a release tarball and write a patch on top of it, you should
be able to learn what the project convention of what the "[PATCH]"
prefix looks like and what the mailing list address is, even though
you did not clone with "git".

All of the above leads to a design to have a common convention
widely shared among projects to express project preferences over
different kind of tools, among which Git is one of them, and store
it in a known location in the projects' trees.  Most importantly,
there must not be any Git protocol extension for doing this kind of
thing.  

Don't limit the user's choice in either of these two ways.  The
preferences for tools other than Git should be sharable with the
same ease as preference for Git, and the preference should be
sharable with the same ease to those who use Git and those who
don't.

Perhaps have a .project-preference/ directory at the root level of
the project tree, talk with other SCM vendors and editor vendors to
design what kind of information are recorded in that directory and
how, and write a script to work on that to map the project
preference information to git configuration while other SCM vendors
and editor vendors write their scripts to help their users to map
the project preference information to the configuration files that
their tools use.  Then you can either write a wrapper around "git
clone" to first run "git clone" and then run these scripts you
prepare to process the contents of the .project-preference
directory, or perhaps trigger these scripts from the post-clone hook.

Re: Proposal: server-advertised config options

[Drew DeVault]

On Mon Sep 7, 2020 at 2:51 PM EDT, Junio C Hamano wrote:
> Assuming I am among the guys (do you solicit opinions from gals, by
> the way?), here are a few unconnected random thoughts.

Guys is gender netural where I'm from, of course guys and gals and
anyone else are invited to comment :)

> I do not want to see this as a "server" thing. All the examples are
> "per project preference" and I do agree it would be nice to have a
> standardised way for projects to communicate their preference to
> their participants. Regardless of the hosting site I clone and
> fetch my project from, I'd want to see it communicated consistently
> to them.

The server I have in mind (git.sr.ht) is a little bit different in that
most of those examples I gave would be configured automatically on the
server side. My server software knows where your mailing list is, for
example. My goal is to try and make this as hands-off and "it just
works" as possible.

> In addition to your examples of "per project preference", there are
> projects' coding style guides, etc., that we do not enforce as git
> config at all, e.g. how wide your editors TAB and single level of
> indentation should be. It will unnecessarily narrow your view to
> assume that the kind of "per project preference" you convey from the
> project to its participants need to be the Git configuration and
> nothing else.

I think there's a difference between preferences regarding the contents
of the repository (e.g. style guide), and preferences regarding the
administration and usage of the repository itself (e.g. this feature I'm
proposing). I think the argument for integration with git is much
stronger for the latter.

> All of the above leads to a design to have a common convention
> widely shared among projects to express project preferences over
> different kind of tools, among which Git is one of them, and store
> it in a known location in the projects' trees. Most importantly,
> there must not be any Git protocol extension for doing this kind of
> thing.

Storing a file in your project tree to handle this configuration would
eliminate the "hands off" feature I was aiming for. We also have a
policy which forbids our software from making any automated changes to
the contents of your git repository - we just don't consider it
appropriate in the domain of our server software's responsibilities.

Some sort of common config file for this purpose, in-tree, would perhaps
be useful, but it would remove a lot of the value-add that I'm seeking
to provide. I already posess most of the necessary information
server-side and I can offer it to clients without any explicit
involvement from the project maintainers.

Also, the conventions for tooling-related files in-tree like this is
currently very disorganized within the ecosystem. Between .editorconfig,
.gitattributes, .github/funding.yml, a dozen CI systems, and who knows
what else, there's no common consensus on where to put files like this
or what they should look like. I think that securing consensus for this
would involve reaching out to these projects, and the scope of that
effort and the necessary follow-up developments and compatibility
planning on behalf of these projects would be...  astonishingly large.

And, ultimately, even with a common configuration, we'd end up having to
add vendor-specific extensions, for example to support the example of
push options given in the initial mail.

So, in summary, based on your suggestions this proposal could grow
10x-100x in scope and lose no small degree of the desired utility. Maybe
yours is a worthwhile idea, but it poorly solves the particular problem
I set out to solve and I lack the time/motivation to work on that
approach.

Re: Proposal: server-advertised config options

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 3996 bytes --]

On 2020-09-07 at 19:23:15, Drew DeVault wrote:
> On Mon Sep 7, 2020 at 2:51 PM EDT, Junio C Hamano wrote:
> > I do not want to see this as a "server" thing. All the examples are
> > "per project preference" and I do agree it would be nice to have a
> > standardised way for projects to communicate their preference to
> > their participants. Regardless of the hosting site I clone and
> > fetch my project from, I'd want to see it communicated consistently
> > to them.
> 
> The server I have in mind (git.sr.ht) is a little bit different in that
> most of those examples I gave would be configured automatically on the
> server side. My server software knows where your mailing list is, for
> example. My goal is to try and make this as hands-off and "it just
> works" as possible.

The Git security model doesn't permit untrusted config options, so I
think it's risky to add support for config options from the server
side.  We need to consider not only advanced users who are going to be
able to make a good decision here, but novice users who are struggling
to understand how Git works and are prone to social engineering.  Just
because your server is not malicious does not mean that others aren't.

In addition, if I'm cloning a repository just to build it, I don't want
to be prompted to set those configuration options at all.  My experience
in Git hosting is that clones and fetches far, far outnumber pushes, so
adding a prompting feature adds a bunch of impediment with little gain
for the vast majority of users.

> > All of the above leads to a design to have a common convention
> > widely shared among projects to express project preferences over
> > different kind of tools, among which Git is one of them, and store
> > it in a known location in the projects' trees. Most importantly,
> > there must not be any Git protocol extension for doing this kind of
> > thing.
> 
> Storing a file in your project tree to handle this configuration would
> eliminate the "hands off" feature I was aiming for. We also have a
> policy which forbids our software from making any automated changes to
> the contents of your git repository - we just don't consider it
> appropriate in the domain of our server software's responsibilities.

That doesn't mean you can't provide a downloadable shell script that
people could check into their repositories to configure this for the
user.  That's the typical way that projects that use standardized hooks
work, for example, and it lets the user decide whether they want to
configure these things (by running the script) or not (by not running
it).  Users who are not interested in becoming contributors need not
ever be bothered with it at all.

It doesn't automatically "just work", but it also lets projects decide
for themselves what their settings should be.  Just because a site
offers, say, mailing lists, doesn't mean that folks will want to use
those mailing lists.  For example, the Go language repository is hosted
on GitHub, but uses Gerrit for code review, not GitHub pull requests.

> Also, the conventions for tooling-related files in-tree like this is
> currently very disorganized within the ecosystem. Between .editorconfig,
> .gitattributes, .github/funding.yml, a dozen CI systems, and who knows
> what else, there's no common consensus on where to put files like this
> or what they should look like. I think that securing consensus for this
> would involve reaching out to these projects, and the scope of that
> effort and the necessary follow-up developments and compatibility
> planning on behalf of these projects would be...  astonishingly large.

You can try to standardize all repository dotfiles, or you can just
provide a configuration file and documentation and let people adopt it
as you go, which is how most of these work.  If your design is
desirable, people will adopt it and spread it across projects.
-- 
brian m. carlson: Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: Proposal: server-advertised config options

[Drew DeVault]

On Mon Sep 7, 2020 at 4:52 PM EDT, brian m. carlson wrote:
> The Git security model doesn't permit untrusted config options, so I
> think it's risky to add support for config options from the server
> side. We need to consider not only advanced users who are going to be
> able to make a good decision here, but novice users who are struggling
> to understand how Git works and are prone to social engineering. Just
> because your server is not malicious does not mean that others aren't.

Hm. If we view some of these use-cases as legitimate, maybe it would
make sense to only permit a hard-coded list of config options to be
advertised, removing all other configurability to avoid letting users
configure themselves into a compromised system (potentially via social
engineering). For example, project policies like sendmail.to and config
options in the server's domain like push options would be in-scope, but
something like core.editor would be out of scope.

> In addition, if I'm cloning a repository just to build it, I don't want
> to be prompted to set those configuration options at all. My experience
> in Git hosting is that clones and fetches far, far outnumber pushes, so
> adding a prompting feature adds a bunch of impediment with little gain
> for the vast majority of users.

That alternative approach would also eliminate the need for prompting.

> That doesn't mean you can't provide a downloadable shell script that
> people could check into their repositories to configure this for the
> user.

Yeah, but that also eliminates most of the convenience factor and is
even more of a risk.

> It doesn't automatically "just work", but it also lets projects decide
> for themselves what their settings should be. Just because a site
> offers, say, mailing lists, doesn't mean that folks will want to use
> those mailing lists. For example, the Go language repository is hosted
> on GitHub, but uses Gerrit for code review, not GitHub pull requests.

Note that in the case of sourcehut, mailing lists are optional. It's not
like GitHub where you can't turn off pull requests and have to deal with
rejecting them manually or set up a bot to auto-close them or something.

Re: Proposal: server-advertised config options

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2942 bytes --]

On 2020-09-08 at 14:14:47, Drew DeVault wrote:
> On Mon Sep 7, 2020 at 4:52 PM EDT, brian m. carlson wrote:
> > The Git security model doesn't permit untrusted config options, so I
> > think it's risky to add support for config options from the server
> > side. We need to consider not only advanced users who are going to be
> > able to make a good decision here, but novice users who are struggling
> > to understand how Git works and are prone to social engineering. Just
> > because your server is not malicious does not mean that others aren't.
> 
> Hm. If we view some of these use-cases as legitimate, maybe it would
> make sense to only permit a hard-coded list of config options to be
> advertised, removing all other configurability to avoid letting users
> configure themselves into a compromised system (potentially via social
> engineering). For example, project policies like sendmail.to and config
> options in the server's domain like push options would be in-scope, but
> something like core.editor would be out of scope.

sendmail.to would potentially leak information if you cloned a
repository from one server but intended to use it internally to send
patches elsewhere, especially if you'd already set that option globally.
I'm pretty sure that other scenarios exist where even mundane config
options can be leveraged as an attack vector.

That's the problem: we as the defenders have to come up with every
possible scenario to allow a trusted config item, and an attacker just
has to come up with one scenario to exploit it.  We've seen security
problems in the past with .gitmodules multiple times because it's
server-sent config (in the repository).  Vim has also seen multiple CVEs
from untrusted configuration (modelines).  It's just a minefield waiting
to happen.

As a result, I'm very much opposed to allowing any config options to be
copied from the server.

> > In addition, if I'm cloning a repository just to build it, I don't want
> > to be prompted to set those configuration options at all. My experience
> > in Git hosting is that clones and fetches far, far outnumber pushes, so
> > adding a prompting feature adds a bunch of impediment with little gain
> > for the vast majority of users.
> 
> That alternative approach would also eliminate the need for prompting.

Are you suggesting that we just accept the server's settings without
prompting?  I certainly don't think that's a viable approach.

> > That doesn't mean you can't provide a downloadable shell script that
> > people could check into their repositories to configure this for the
> > user.
> 
> Yeah, but that also eliminates most of the convenience factor and is
> even more of a risk.

It's also the standard approach that every project uses already, and it
has the benefit that the user can inspect it at their leisure before
running it.
-- 
brian m. carlson: Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: Proposal: server-advertised config options

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> ...  We've seen security
> problems in the past with .gitmodules multiple times because it's
> server-sent config (in the repository).

Exactly.  For some reason some people still seem to think that it is
wasteful that we force users to approve/reject copying from .gitmodules
to .git/config and push to read and use settings directly from the
former---I think it is a huge mistake.

> As a result, I'm very much opposed to allowing any config options to be
> copied from the server.

+1.

> It's also the standard approach that every project uses already, and it
> has the benefit that the user can inspect it at their leisure before
> running it.

+100 ;-)