[PATCH] Contribute a fairly paranoid update hook

[PATCH] Contribute a fairly paranoid update hook

[Shawn O. Pearce]

I'm using a variant of this update hook in a corporate environment
where we perform some validations of the commits and tags that
are being pushed.  The model is a "central repository" type setup,
where users are given access to push to specific branches within
the shared central repository.  In this particular installation we
run a specially patched git-receive-pack in setuid mode via SSH,
allowing all writes into the repository as the repository owner,
but only if this hook blesses it.

One of the major checks we perform with this hook is that the
'committer' line of a commit, or the 'tagger' line of a new annotated
tag actually correlates to the UNIX user who is performing the push.
Users can falsify these lines on their local repositories, but
the central repository that management trusts will reject all such
forgery attempts.  Of course 'author' lines are still allowed to
be any value, as sometimes changes do come from other individuals.

Another nice feature of this hook is the access control lists for
all repositories on the system can also be stored and tracked in
a supporting Git repository, which can also be access controlled
by itself.  This allows full auditing of who-had-what-when-and-why,
thanks to git-blame's data mining capabilities.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---

 I have often mentioned this hook on the list, and some people
 have been interested in at least looking at it, or using it as a
 foundation for their own hooks.  I think it is a reasonable thing
 for contrib/hooks, so here it is...  ;-)

 contrib/hooks/update-paranoid |  284 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 284 insertions(+), 0 deletions(-)
 create mode 100644 contrib/hooks/update-paranoid

diff --git a/contrib/hooks/update-paranoid b/contrib/hooks/update-paranoid
new file mode 100644
index 0000000..5ee1835
--- /dev/null
+++ b/contrib/hooks/update-paranoid
@@ -0,0 +1,284 @@
+#!/usr/bin/perl
+
+use strict;
+use File::Spec;
+
+$ENV{PATH}     = '/opt/git/bin';
+my $acl_git    = '/vcs/acls.git';
+my $acl_branch = 'refs/heads/master';
+my $debug      = 0;
+
+=doc
+Invoked as: update refname old-sha1 new-sha1
+
+This script is run by git-receive-pack once for each ref that the
+client is trying to modify.  If we exit with a non-zero exit value
+then the update for that particular ref is denied, but updates for
+other refs in the same run of receive-pack may still be allowed.
+
+We are run after the objects have been uploaded, but before the
+ref is actually modified.  We take advantage of that fact when we
+look for "new" commits and tags (the new objects won't show up in
+`rev-list --all`).
+
+This script loads and parses the content of the config file
+"users/$this_user.acl" from the $acl_branch commit of $acl_git ODB.
+The acl file is a git-config style file, but uses a slightly more
+restricted syntax as the Perl parser contained within this script
+is not nearly as permissive as git-config.
+
+Example:
+
+  [user]
+    committer = John Doe <john.doe@example.com>
+    committer = John R. Doe <john.doe@example.com>
+
+  [repository "acls"]
+    allow = heads/master
+    allow = CDUR for heads/jd/
+    allow = C    for ^tags/v\\d+$
+
+For all new commit or tag objects the committer (or tagger) line
+within the object must exactly match one of the user.committer
+values listed in the acl file ("HEAD:users/$this_user.acl").
+
+For a branch to be modified an allow line within the matching
+repository section must be matched for both the refname and the
+opcode.
+
+Repository sections are matched on the basename of the repository
+(after removing the .git suffix).
+
+The opcode abbrevations are:
+
+  C: create new ref
+  D: delete existing ref
+  U: fast-forward existing ref (no commit loss)
+  R: rewind/rebase existing ref (commit loss)
+
+if no opcodes are listed before the "for" keyword then "U" (for
+fast-forward update only) is assumed as this is the most common
+usage.
+
+Refnames are matched by always assuming a prefix of "refs/".
+This hook forbids pushing or deleting anything not under "refs/".
+
+Refnames that start with ^ are Perl regular expressions, and the ^
+is kept as part of the regexp.  \\ is needed to get just one \, so
+\\d expands to \d in Perl.  The 3rd allow line above is an example.
+
+Refnames that don't start with ^ but that end with / are prefix
+matches (2nd allow line above); all other refnames are strict
+equality matches (1st allow line).
+
+Anything pushed to "heads/" (ok, really "refs/heads/") must be
+a commit.  Tags are not permitted here.
+
+Anything pushed to "tags/" (err, really "refs/tags/") must be an
+annotated tag.  Commits, blobs, trees, etc. are not permitted here.
+Annotated tag signatures aren't checked, nor are they required.
+
+The special subrepository of 'info/new-commit-check' can
+be created and used to allow users to push new commits and
+tags from another local repository to this one, even if they
+aren't the committer/tagger of those objects.  In a nut shell
+the info/new-commit-check directory is a Git repository whose
+objects/info/alternates file lists this repository and all other
+possible sources, and whose refs subdirectory contains symlinks
+to this repository's refs subdirectory, and to all other possible
+sources refs subdirectories.  Yes, this means that you cannot
+use packed-refs in those repositories as they won't be resolved
+correctly.
+
+=cut
+
+my $git_dir = $ENV{GIT_DIR};
+my $new_commit_check = "$git_dir/info/new-commit-check";
+my $ref = $ARGV[0];
+my $old = $ARGV[1];
+my $new = $ARGV[2];
+my $new_type;
+my ($this_user) = getpwuid $<; # REAL_USER_ID
+my $repository_name;
+my %user_committer;
+my @allow_rules;
+
+sub deny ($) {
+	print STDERR "-Deny-    $_[0]\n" if $debug;
+	print STDERR "\ndenied: $_[0]\n\n";
+	exit 1;
+}
+
+sub grant ($) {
+	print STDERR "-Grant-   $_[0]\n" if $debug;
+	exit 0;
+}
+
+sub info ($) {
+	print STDERR "-Info-    $_[0]\n" if $debug;
+}
+
+sub parse_config ($$) {
+	my ($data, $fn) = @_;
+	info "Loading $fn";
+	open(I,'-|','git',"--git-dir=$acl_git",'cat-file','blob',$fn);
+	my $section = '';
+	while (<I>) {
+		chomp;
+		if (/^\s*$/ || /^\s*#/) {
+		} elsif (/^\[([a-z]+)\]$/i) {
+			$section = $1;
+		} elsif (/^\[([a-z]+)\s+"(.*)"\]$/i) {
+			$section = "$1.$2";
+		} elsif (/^\s*([a-z][a-z0-9]+)\s*=\s*(.*?)\s*$/i) {
+			push @{$data->{"$section.$1"}}, $2;
+		} else {
+			deny "bad config file line $. in $fn";
+		}
+	}
+	close I;
+}
+
+sub all_new_committers () {
+	local $ENV{GIT_DIR} = $git_dir;
+	$ENV{GIT_DIR} = $new_commit_check if -d $new_commit_check;
+
+	info "Getting committers of new commits.";
+	my %used;
+	open(T,'-|','git','rev-list','--pretty=raw',$new,'--not','--all');
+	while (<T>) {
+		next unless s/^committer //;
+		chop;
+		s/>.*$/>/;
+		info "Found $_." unless $used{$_}++;
+	}
+	close T;
+	info "No new commits." unless %used;
+	keys %used;
+}
+
+sub all_new_taggers () {
+	my %exists;
+	open(T,'-|','git','for-each-ref','--format=%(objectname)','refs/tags');
+	while (<T>) {
+		chop;
+		$exists{$_} = 1;
+	}
+	close T;
+
+	info "Getting taggers of new tags.";
+	my %used;
+	my $obj = $new;
+	my $obj_type = $new_type;
+	while ($obj_type eq 'tag') {
+		last if $exists{$obj};
+		$obj_type = '';
+		open(T,'-|','git','cat-file','tag',$obj);
+		while (<T>) {
+			chop;
+			if (/^object ([a-z0-9]{40})$/) {
+				$obj = $1;
+			} elsif (/^type (.+)$/) {
+				$obj_type = $1;
+			} elsif (s/^tagger //) {
+				s/>.*$/>/;
+				info "Found $_." unless $used{$_}++;
+				last;
+			}
+		}
+		close T;
+	}
+	info "No new tags." unless %used;
+	keys %used;
+}
+
+sub check_committers (@) {
+	my @bad;
+	foreach (@_) { push @bad, $_ unless $user_committer{$_}; }
+	if (@bad) {
+		print STDERR "\n";
+		print STDERR "You are not $_.\n" foreach (sort @bad);
+		deny "You cannot push changes not committed by you.";
+	}
+}
+
+sub git_value (@) {
+	open(T,'-|','git',@_); local $_ = <T>; chop; close T;
+	$_;
+}
+
+deny "No GIT_DIR inherited from caller" unless $git_dir;
+deny "Need a ref name" unless $ref;
+deny "Refusing funny ref $ref" unless $ref =~ s,^refs/,,;
+deny "Bad old value $old" unless $old =~ /^[a-z0-9]{40}$/;
+deny "Bad new value $new" unless $new =~ /^[a-z0-9]{40}$/;
+deny "Cannot determine who you are." unless $this_user;
+
+$repository_name = File::Spec->rel2abs($git_dir);
+$repository_name =~ m,/([^/]+)(?:\.git|/\.git)$,;
+$repository_name = $1;
+info "Updating in '$repository_name'.";
+
+my $op;
+if    ($old =~ /^0{40}$/) { $op = 'C'; }
+elsif ($new =~ /^0{40}$/) { $op = 'D'; }
+else                      { $op = 'R'; }
+
+# This is really an update (fast-forward) if the
+# merge base of $old and $new is $old.
+#
+$op = 'U' if ($op eq 'R'
+	&& $ref =~ m,^heads/,
+	&& $old eq git_value('merge-base',$old,$new));
+
+# Load the user's ACL file.
+{
+	my %data = ('user.committer' => []);
+	parse_config(\%data, "$acl_branch:users/$this_user.acl");
+	%user_committer = map {$_ => $_} @{$data{'user.committer'}};
+	my $rules = $data{"repository.$repository_name.allow"} || [];
+	foreach (@$rules) {
+		if (/^([CDRU ]+)\s+for\s+([^\s]+)$/) {
+			my $ops = $1;
+			my $ref = $2;
+			$ops =~ s/ //g;
+			$ref =~ s/\\\\/\\/g;
+			push @allow_rules, [$ops, $ref];
+		} elsif (/^for\s+([^\s]+)$/) {
+			# Mentioned, but nothing granted?
+		} elsif (/^[^\s]+$/) {
+			s/\\\\/\\/g;
+			push @allow_rules, ['U', $_];
+		}
+	}
+}
+
+if ($op ne 'D') {
+	$new_type = git_value('cat-file','-t',$new);
+
+	if ($ref =~ m,^heads/,) {
+		deny "$ref must be a commit." unless $new_type eq 'commit';
+	} elsif ($ref =~ m,^tags/,) {
+		deny "$ref must be an annotated tag." unless $new_type eq 'tag';
+	}
+
+	check_committers (all_new_committers);
+	check_committers (all_new_taggers) if $new_type eq 'tag';
+}
+
+info "$this_user wants $op for $ref";
+foreach my $acl_entry (@allow_rules) {
+	my ($acl_ops, $acl_n) = @$acl_entry;
+	next unless $acl_ops =~ /^[CDRU]+$/; # Uhh.... shouldn't happen.
+	next unless $acl_n;
+	next unless $op =~ /^[$acl_ops]$/;
+
+	grant "Allowed by: $acl_ops for $acl_n"
+	if (
+	   ($acl_n eq $ref)
+	|| ($acl_n =~ m,/$, && substr($ref,0,length $acl_n) eq $acl_n)
+	|| ($acl_n =~ m,^\^, && $ref =~ m:$acl_n:)
+	);
+}
+close A;
+deny "You are not permitted to $op $ref";
-- 
1.5.1.1.135.gf948

Re: [PATCH] Contribute a fairly paranoid update hook

[Junio C Hamano]

"Shawn O. Pearce" <spearce@spearce.org> writes:

> +$ENV{PATH}     = '/opt/git/bin';

This caught my attention ;-).

Re: [PATCH] Contribute a fairly paranoid update hook

[Shawn O. Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> "Shawn O. Pearce" <spearce@spearce.org> writes:
> 
> > +$ENV{PATH}     = '/opt/git/bin';
> 
> This caught my attention ;-).

What, is replacing PATH in the hook really all that paranoid?
Its a perfectly reasonable thing to do.  ;-)

Someday I'll post the patch to receive-pack that I use to run
it setuid.  That thing is also fairly paranoid.  I won't submit
it for application to git.git though, as I don't think it really
belongs in the main receive-pack code.  So it might be a patch file
for contrib, or just an article in the mailing list archives that
people can link to.

I also recently hacked some semi-useful features into gitweb.cgi.
I need to finish those for our local use, and then see if I can
extract them enough to make them publicly available.  gitweb.cgi is
probably going to become a cornerstone of our main development and
testing workflow, especially for managers and testers who don't want
(or really need) full repositories on their deskstops.

-- 
Shawn.

Re: [PATCH] Contribute a fairly paranoid update hook

[Junio C Hamano]

"Shawn O. Pearce" <spearce@spearce.org> writes:

> Junio C Hamano <junkio@cox.net> wrote:
>> "Shawn O. Pearce" <spearce@spearce.org> writes:
>> 
>> > +$ENV{PATH}     = '/opt/git/bin';
>> 
>> This caught my attention ;-).
>
> What, is replacing PATH in the hook really all that paranoid?
> Its a perfectly reasonable thing to do.  ;-)

I 100% agree it is the right thing to to make sure PATH is a
known good value.  What caught my attention is /opt/git/bin
is the *ONLY* place you need to look at, not even /usr/bin!

Re: [PATCH] Contribute a fairly paranoid update hook

[Jakub Narebski]

Shawn O. Pearce wrote:

> I also recently hacked some semi-useful features into gitweb.cgi.
> I need to finish those for our local use, and then see if I can
> extract them enough to make them publicly available.  gitweb.cgi is
> probably going to become a cornerstone of our main development and
> testing workflow, especially for managers and testers who don't want
> (or really need) full repositories on their deskstops.

If I can ask: what features? 

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: [PATCH] Contribute a fairly paranoid update hook

[Shawn O. Pearce]

Jakub Narebski <jnareb@gmail.com> wrote:
> Shawn O. Pearce wrote:
> 
> > I also recently hacked some semi-useful features into gitweb.cgi.
...
> If I can ask: what features? 

I added a "pending" action and link in the top navigation bar.
This page shows all of the topic branches that:

 - match a particular prefix string
   (e.g. "heads/test/pending/");
 - has commits not reachable through refs matching another prefix
   (e.g. "heads/test/approved/")
 - has commits not reachable through another specific ref
   (e.g. "heads/test/current").

These branches are sorted by name and show under their title a
shortlog of the "new commits" that aren't reachable by the latter
two sets of refs.

Sometimes branches fully contain another; these are identified under
both branches.  When this happens commits are only shown under one
of the two listings.

Here's a rough ASCII approximation of the pending page:

  fix1:

    Also bundled within:
    
      * set-of-fixes

    Shawn Pearce      don't be stupid, delete temp files

  fix2:

    Also bundled within:

      * set-of-fixes

    Shawn Pearce      don't leak printer handles

  set-of-fixes:

    Includes topics:

      * fix1:
      * fix2:

    Stewie Griffin    Merge 'tiny-fix1'
    Stewie Griffin    Merge 'tiny-fix2'
    Glenn Quagmire    Improve performance by sucking less

Each topic branch also has an "approve" link, which uses an "approve"
action to copy the branch (err, create a new branch with update-ref)
into the "heads/test/approved/" namespace.  This causes them to
fall out of the "pending" page when it reloads.

The approved page shows topics matching the "heads/test/approved/"
page, but that have commits not in "heads/test/current".  It looks
exactly like the pending page above, but has an "unapprove" link
to delete the ref from the "heads/test/approved/" directory.

Why?

Managers and testers want to choose the topic branches they will be
including into the next release, and they don't make that decision
until just before they ask the build manager to start creating
the build.  This gitweb interface lets them see the topics that
are ready for building, and make their selections.

The build manager can then sweep the entire "heads/test/approved/"
directory and merge everything not already merged into the current
branch.  Of course if a merge fails, or a compile fails, the build
manager stops and throws the mess back at the development team.

Bad merges or build failures happen less often than you might expect,
as we do have fairly good isolation between our topic branches.
During some periods of the year we don't think twice about using the
octopus driver to merge topics, and will merge 30+ topics at a clip
in just a minute or two, because they are all unrelated and isolated
changes that management has asked to go into a single release.

Most of the above code is actually fairly well isolated in
gitweb.cgi.  It turns out it was quite easy to add what I needed,
and to reuse a lot of the existing code.  I'll probably be able to
clean it up well enough to submit a patch.  Might take a few weeks.

-- 
Shawn.