Server-side preventing some files from being overwritten

Server-side preventing some files from being overwritten

[Thorsten Glaser]

Hi *,

is there a way, for example with some sort of pre-receive hook,
to prevent some files from being overwritten by a push?

Use case:

In some project, we use Flyway to manage the database schema,
and Jenkins to automatically build master’s HEAD after each
push (and install it, thus activating the schema files almost
immediately). Now, I wish to prevent coworkers from changing
anything in the SQL subdirectory that has ever been pushed,
forcing them to push new SQL files with ALTER statements instead.
(Yes, I am aware this will likely make me even less liked. No,
this is not an issue.)

As for the comparison, only the changes between the previous
HEAD of master and the new HEAD of master after the push would
have been accepted need to be taken into account; any intermediate
commits, merges, etc. are no problem (because Jenkins does not
build them, and because, once a push fails, the developer will
have to add a commit reverting their change and moving it to
another file on top, I’m no friend of rewriting).

File matching would be “any files under a certain subdirectory”,
nothing fancier than that.

I’ve tried a web search (with two different search engines) for
“git prevent pushed files from modification”, but this seems to
only show people who want to ignore local changes or somesuch…

I’ve asked in IRC, but with no answer for hours I thought that
maybe this was the better place to ask for it.

Thanks in advance,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

Re: Server-side preventing some files from being overwritten

[Stefan Beller]

On Thu, Jul 14, 2016 at 8:31 AM, Thorsten Glaser <t.glaser@tarent.de> wrote:
> Hi *,
>
> is there a way, for example with some sort of pre-receive hook,
> to prevent some files from being overwritten by a push?

pre-receive hooks are a thing!

 pre-receive
       This hook is invoked by git-receive-pack on the remote
repository, which happens when a git push is done on a local
repository. Just before starting to update refs on the remote
repository,
       the pre-receive hook is invoked. Its exit status determines the
success or failure of the update.

       This hook executes once for the receive operation. It takes no
arguments, but for each ref to be updated it receives on standard
input a line of the format:

           <old-value> SP <new-value> SP <ref-name> LF

       where <old-value> is the old object name stored in the ref,
<new-value> is the new object name to be stored in the ref and
<ref-name> is the full name of the ref. When creating a new ref,
       <old-value> is 40 0.

       If the hook exits with non-zero status, none of the refs will
be updated. If the hook exits with zero, updating of individual refs
can still be prevented by the update hook.

       Both standard output and standard error output are forwarded to
git send-pack on the other end, so you can simply echo messages for
the user.


As you have access to the old and new value, just check them out to
/tmp and inspect the files?
(That is slower than optimal I'd assume, so to get it faster you could
go roughly like

    git ls-tree -r <oldvalue> | grep <certain dir>  >old_interest
    git ls-tree -r <newvalue> | grep <certain dir> >new_interest
    if diff old_interest new_interest
    then
        echo "There is a difference in  <certain dir>
        exit 1
    fi



>
> Use case:
>
> In some project, we use Flyway to manage the database schema,
> and Jenkins to automatically build master’s HEAD after each
> push (and install it, thus activating the schema files almost
> immediately). Now, I wish to prevent coworkers from changing
> anything in the SQL subdirectory that has ever been pushed,
> forcing them to push new SQL files with ALTER statements instead.
> (Yes, I am aware this will likely make me even less liked. No,
> this is not an issue.)
>
> As for the comparison, only the changes between the previous
> HEAD of master and the new HEAD of master after the push would
> have been accepted need to be taken into account; any intermediate
> commits, merges, etc. are no problem (because Jenkins does not
> build them, and because, once a push fails, the developer will
> have to add a commit reverting their change and moving it to
> another file on top, I’m no friend of rewriting).
>
> File matching would be “any files under a certain subdirectory”,
> nothing fancier than that.
>
> I’ve tried a web search (with two different search engines) for
> “git prevent pushed files from modification”, but this seems to
> only show people who want to ignore local changes or somesuch…
>
> I’ve asked in IRC, but with no answer for hours I thought that
> maybe this was the better place to ask for it.
>
> Thanks in advance,
> //mirabilos
> --
> tarent solutions GmbH
> Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
> Tel: +49 228 54881-393 • Fax: +49 228 54881-235
> HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
> Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Server-side preventing some files from being overwritten

[Thorsten Glaser]

On Thu, 14 Jul 2016, Stefan Beller wrote:

> go roughly like

Thanks, that did the trick!

Although I’m ordinarily loath to write GNU bash scripts, this
helps avoiding temporary files. This works:

-----cutting here may damage your screen surface-----
#!/bin/bash
export LC_ALL=C
subdir=x/y
while IFS=' ' read -r old new name; do
	test x"$name" = x"refs/heads/master" || continue
	if test x"0" != x"$(comm -23z \
	    <(git ls-tree -r -z "$old" "$subdir" | sort -z) \
	    <(git ls-tree -r -z "$new" "$subdir" | sort -z) | wc -c)"; then
		echo >&2 'Untouchable files touched, commit rejected!'
		exit 1
	fi
done
exit 0
-----cutting here may damage your screen surface-----

Of course, set “subdir” in line 3 correctly, and GNU coreutils
are required for the NUL line termination, which is not an issue
here. (BSD has “-R ''” for sort(1), for example.)

bye,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

Re: Server-side preventing some files from being overwritten

[Junio C Hamano]

Thorsten Glaser <t.glaser@tarent.de> writes:

> Although I’m ordinarily loath to write GNU bash scripts, this
> helps avoiding temporary files. This works:
>
> -----cutting here may damage your screen surface-----
> #!/bin/bash
> export LC_ALL=C
> subdir=x/y
> while IFS=' ' read -r old new name; do
> 	test x"$name" = x"refs/heads/master" || continue
> 	if test x"0" != x"$(comm -23z \
> 	    <(git ls-tree -r -z "$old" "$subdir" | sort -z) \
> 	    <(git ls-tree -r -z "$new" "$subdir" | sort -z) | wc -c)"; then
> 		echo >&2 'Untouchable files touched, commit rejected!'
> 		exit 1
> 	fi

Can't this become simpler, e.g.

	if ! git diff-tree --quiet "$old" "$new" -- "$subdir"
	then
		echo >&2 "Ooh, $subdir is touched"
		exit 1
	fi

> done
> exit 0
> -----cutting here may damage your screen surface-----
>
> Of course, set “subdir” in line 3 correctly, and GNU coreutils
> are required for the NUL line termination, which is not an issue
> here. (BSD has “-R ''” for sort(1), for example.)
>
> bye,
> //mirabilos

Re: Server-side preventing some files from being overwritten

[Junio C Hamano]

On Thu, Jul 14, 2016 at 11:27 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Thorsten Glaser <t.glaser@tarent.de> writes:
>
>>       if test x"0" != x"$(comm -23z \
>>           <(git ls-tree -r -z "$old" "$subdir" | sort -z) \
>>           <(git ls-tree -r -z "$new" "$subdir" | sort -z) | wc -c)"; then
>>               echo >&2 'Untouchable files touched, commit rejected!'
>>               exit 1
>>       fi
>
> Can't this become simpler, e.g.
>
>         if ! git diff-tree --quiet "$old" "$new" -- "$subdir"
>         then
>                 echo >&2 "Ooh, $subdir is touched"
>                 exit 1
>         fi

Ehh, you need to tell diff-tree to recurse, i.e. "diff-tree -r".

Re: Server-side preventing some files from being overwritten

[Thorsten Glaser]

On Thu, 14 Jul 2016, Junio C Hamano wrote:

> Can't this become simpler, e.g.
> 
> 	if ! git diff-tree --quiet "$old" "$new" -- "$subdir"

Thought about diff-tree, but additions are permitted,
and diffing the actual file content has overhead too.

Just counting the number of object hashes removed from
the old tree (recursed) works out just fine.

bye,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

Re: Server-side preventing some files from being overwritten

[Stefan Beller]

On Thu, Jul 14, 2016 at 11:44 AM, Junio C Hamano <gitster@pobox.com> wrote:
> On Thu, Jul 14, 2016 at 11:27 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Thorsten Glaser <t.glaser@tarent.de> writes:
>>
>>>       if test x"0" != x"$(comm -23z \
>>>           <(git ls-tree -r -z "$old" "$subdir" | sort -z) \
>>>           <(git ls-tree -r -z "$new" "$subdir" | sort -z) | wc -c)"; then
>>>               echo >&2 'Untouchable files touched, commit rejected!'
>>>               exit 1
>>>       fi
>>
>> Can't this become simpler, e.g.
>>
>>         if ! git diff-tree --quiet "$old" "$new" -- "$subdir"
>>         then
>>                 echo >&2 "Ooh, $subdir is touched"

No need to go for >&2 here, as it makes no difference to
the client.

>>                 exit 1
>>         fi
>
> Ehh, you need to tell diff-tree to recurse, i.e. "diff-tree -r".

Re: Server-side preventing some files from being overwritten

[Junio C Hamano]

Thorsten Glaser <t.glaser@tarent.de> writes:

> On Thu, 14 Jul 2016, Junio C Hamano wrote:
>
>> Can't this become simpler, e.g.
>> 
>> 	if ! git diff-tree --quiet "$old" "$new" -- "$subdir"
>
> Thought about diff-tree, but additions are permitted,

	diff-tree -r --diff-filter=a "$old" "$new" -- "$subdir" 

or something like that?

> and diffing the actual file content has overhead too.

It is why exactly "diff-tree" is desired here, as you do not have to
diff actual file at all.

Re: Server-side preventing some files from being overwritten

[Ævar Arnfjörð Bjarmason]

On Thu, Jul 14, 2016 at 8:44 PM, Junio C Hamano <gitster@pobox.com> wrote:
> On Thu, Jul 14, 2016 at 11:27 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Thorsten Glaser <t.glaser@tarent.de> writes:
>>
>>>       if test x"0" != x"$(comm -23z \
>>>           <(git ls-tree -r -z "$old" "$subdir" | sort -z) \
>>>           <(git ls-tree -r -z "$new" "$subdir" | sort -z) | wc -c)"; then
>>>               echo >&2 'Untouchable files touched, commit rejected!'
>>>               exit 1
>>>       fi
>>
>> Can't this become simpler, e.g.
>>
>>         if ! git diff-tree --quiet "$old" "$new" -- "$subdir"
>>         then
>>                 echo >&2 "Ooh, $subdir is touched"
>>                 exit 1
>>         fi
>
> Ehh, you need to tell diff-tree to recurse, i.e. "diff-tree -r".

Note that although this is literally what Thorsten is asking for, I
think it's worth noting for the list explicitly that all these
examples that do "diff $old..$new" will *not* prevent your repository
from having *commits* that touch those files, but they will prevent
you from having *pushes* where the end state is a net change in those
files.

I.e. it allows pushing a series which is a series of two commits which:

  1. Change the forbidden file(s)
  2. Undo changes to the forbidden file(s)

This *can* be critically important or not matter at all depending on
your use case, i.e. does it matter that disallowed and potentially
malicious changes come up in "git bisect", or will you ever be rolling
out anything but the latest tip of the branch you're testing in
production?

If the answer to either of those is "yes" you need something that does
a "git log --stat $old..$new" and parses out if *any* of the commits
make changes to those files.

See e.g. my https://github.com/avar/pre-receive-reject-binaries for
one example of that, although it rejects binaries you could easily
modify it to check the filename(s) instead.

Re: Server-side preventing some files from being overwritten

[Thorsten Glaser]

On Fri, 15 Jul 2016, Ævar Arnfjörð Bjarmason wrote:

> I.e. it allows pushing a series which is a series of two commits which:
> 
>   1. Change the forbidden file(s)
>   2. Undo changes to the forbidden file(s)

That’s precisely allowed.

bye,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

Re: Server-side preventing some files from being overwritten

[Jakub Narębski]

W dniu 2016-07-14 o 18:57, Stefan Beller pisze:
> On Thu, Jul 14, 2016 at 8:31 AM, Thorsten Glaser <t.glaser@tarent.de> wrote:
>> Hi *,
>>
>> is there a way, for example with some sort of pre-receive hook,
>> to prevent some files from being overwritten by a push?
> 
> pre-receive hooks are a thing!

Or you can use third-party tools with support for such cases, such
as Gitolite.  VREFs allow to restrict pushes by the filename, see
http://gitolite.com/gitolite/vref.html#NAME (though I am not sure
if it checks only the end state, or if it checks every commit
that was pushed).

Best,
-- 
Jakub Narębski

Re: Server-side preventing some files from being overwritten

[Thorsten Glaser]

On Thu, 21 Jul 2016, Jakub Narębski wrote:

> Or you can use third-party tools with support for such cases, such

Not if the repository is already managed by some other tool
(FusionForge, in this case).

Thanks anyway,
//mirabilos
-- 
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg