git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* Git server side "pre-receive" hook to create new repositories
@ 2019-03-09 10:46 Dimitri Joukoff
  2019-03-19  7:06 ` Jeff King
  0 siblings, 1 reply; 5+ messages in thread
From: Dimitri Joukoff @ 2019-03-09 10:46 UTC (permalink / raw)
  To: git

Some time ago, Feb 2018, I put in a proposal at Swift.org , that would
allow the Swift Package Manager to clone packages from the Internet to a
local git server and use the local version without requiring updates to
the Swift package:
https://forums.swift.org/t/proposal-to-enable-spm-to-use-a-local-repository-mirror/10291/5
.  At the time, I also raised an issue at
https://github.com/github/platform-samples/issues/171 asking for the
feature described below.  However, today I realised that that was not
the right place to make such a request, and hence I've written this email.

The above thread contains a solution to the local mirror problem whose
only drawback was that new repositories had to be MANUALLY initialised
bare on the local server.  In all other respects, in my opinion, the
solution worked perfectly and saved me downloading huge amounts of data
each time I performed a distribution clean on the project or switched to
another project that used the same package.  The solution published at
Swift.org used a Ubuntu 16.04.2 server running Git 2.16.2, and the
client was a Macbook Pro running MacOS with its default git.

Thus, this feature request is asking that the 'pre-receive' hook
triggers when someone tries to push to a repository regardless of
whether the repository exists.  This would allow automatic creation of
new repositories and smooth the work-flow described above.  If the
semantics of the existing 'pre-receive' hook are such that it would not
be suitable for such a purpose, then an alternative way of providing the
call-back ability would be implemented.

Best regards,
Dimitri.



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Git server side "pre-receive" hook to create new repositories
  2019-03-09 10:46 Git server side "pre-receive" hook to create new repositories Dimitri Joukoff
@ 2019-03-19  7:06 ` Jeff King
  2019-03-19  9:24   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 5+ messages in thread
From: Jeff King @ 2019-03-19  7:06 UTC (permalink / raw)
  To: Dimitri Joukoff; +Cc: git

On Sat, Mar 09, 2019 at 10:46:09AM +0000, Dimitri Joukoff wrote:

> Thus, this feature request is asking that the 'pre-receive' hook
> triggers when someone tries to push to a repository regardless of
> whether the repository exists.  This would allow automatic creation of
> new repositories and smooth the work-flow described above.  If the
> semantics of the existing 'pre-receive' hook are such that it would not
> be suitable for such a purpose, then an alternative way of providing the
> call-back ability would be implemented.

The pre-receive hook is a bit too late for this. It runs after the
server has told the client what it has in the repo, the client decides
what to push, and the server has received the pack. So receive-pack
would have to know about this and fake having an empty repository. And
then figure out where to store the incoming packfile, since we have no
repo.

So I think it would have to be another hook that runs before the rest of
receive-pack. I.e., a system-level config option that says "if you are
asked to accept a push for a repo and it doesn't exist, run this instead
and then run as usual".

It does feel a little error-prone, though, if the client does not
positively say "I want you to create this if it doesn't exist".
Otherwise if I do "git push server:my-misspelled-repo.git", the result
is going to be rather confusing. And retro-fitting that into the
receive-pack protocol is going to be tricky.

It would be much easier to have a separate endpoint for the client to
say "please make this repo if it doesn't exist". And then just run that
before doing the push.

For an unrestricted client connecting over ssh, we already have that:
you can just run "ssh $host git init /path/to/repo". There isn't a
similar thing that can be done over HTTP, though.

-Peff

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Git server side "pre-receive" hook to create new repositories
  2019-03-19  7:06 ` Jeff King
@ 2019-03-19  9:24   ` Ævar Arnfjörð Bjarmason
  2019-03-19 12:16     ` Dimitri Joukoff
  0 siblings, 1 reply; 5+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2019-03-19  9:24 UTC (permalink / raw)
  To: Jeff King; +Cc: Dimitri Joukoff, git


On Tue, Mar 19 2019, Jeff King wrote:

> On Sat, Mar 09, 2019 at 10:46:09AM +0000, Dimitri Joukoff wrote:
>
>> Thus, this feature request is asking that the 'pre-receive' hook
>> triggers when someone tries to push to a repository regardless of
>> whether the repository exists.  This would allow automatic creation of
>> new repositories and smooth the work-flow described above.  If the
>> semantics of the existing 'pre-receive' hook are such that it would not
>> be suitable for such a purpose, then an alternative way of providing the
>> call-back ability would be implemented.
>
> The pre-receive hook is a bit too late for this. It runs after the
> server has told the client what it has in the repo, the client decides
> what to push, and the server has received the pack. So receive-pack
> would have to know about this and fake having an empty repository. And
> then figure out where to store the incoming packfile, since we have no
> repo.
>
> So I think it would have to be another hook that runs before the rest of
> receive-pack. I.e., a system-level config option that says "if you are
> asked to accept a push for a repo and it doesn't exist, run this instead
> and then run as usual".
>
> It does feel a little error-prone, though, if the client does not
> positively say "I want you to create this if it doesn't exist".
> Otherwise if I do "git push server:my-misspelled-repo.git", the result
> is going to be rather confusing. And retro-fitting that into the
> receive-pack protocol is going to be tricky.
>
> It would be much easier to have a separate endpoint for the client to
> say "please make this repo if it doesn't exist". And then just run that
> before doing the push.
>
> For an unrestricted client connecting over ssh, we already have that:
> you can just run "ssh $host git init /path/to/repo". There isn't a
> similar thing that can be done over HTTP, though.

Sounds simpler to just change the user's login shell to a wrapper that
checks if the repo exists, and if not create it before proceeding.

The same with http(s). I.e. in whatever webserver that's now pointing to
git-http-backend as a script point to the same wrapper script.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Git server side "pre-receive" hook to create new repositories
  2019-03-19  9:24   ` Ævar Arnfjörð Bjarmason
@ 2019-03-19 12:16     ` Dimitri Joukoff
  2019-03-21  9:28       ` Sitaram Chamarty
  0 siblings, 1 reply; 5+ messages in thread
From: Dimitri Joukoff @ 2019-03-19 12:16 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Jeff King; +Cc: git

Thanks for the feedback.

Whilst I understand the options that you've both proposed, the intent
was to enable the Swift Package Manager to mirror repositories in a
transparent way.  I'll look into whether these options can be
implemented inside the SPM.

Cheers,
Dimitri.


On 19/3/19 7:24 pm, Ævar Arnfjörð Bjarmason wrote:
> On Tue, Mar 19 2019, Jeff King wrote:
>
>> On Sat, Mar 09, 2019 at 10:46:09AM +0000, Dimitri Joukoff wrote:
>>
>>> Thus, this feature request is asking that the 'pre-receive' hook
>>> triggers when someone tries to push to a repository regardless of
>>> whether the repository exists.  This would allow automatic creation of
>>> new repositories and smooth the work-flow described above.  If the
>>> semantics of the existing 'pre-receive' hook are such that it would not
>>> be suitable for such a purpose, then an alternative way of providing the
>>> call-back ability would be implemented.
>> The pre-receive hook is a bit too late for this. It runs after the
>> server has told the client what it has in the repo, the client decides
>> what to push, and the server has received the pack. So receive-pack
>> would have to know about this and fake having an empty repository. And
>> then figure out where to store the incoming packfile, since we have no
>> repo.
>>
>> So I think it would have to be another hook that runs before the rest of
>> receive-pack. I.e., a system-level config option that says "if you are
>> asked to accept a push for a repo and it doesn't exist, run this instead
>> and then run as usual".
>>
>> It does feel a little error-prone, though, if the client does not
>> positively say "I want you to create this if it doesn't exist".
>> Otherwise if I do "git push server:my-misspelled-repo.git", the result
>> is going to be rather confusing. And retro-fitting that into the
>> receive-pack protocol is going to be tricky.
>>
>> It would be much easier to have a separate endpoint for the client to
>> say "please make this repo if it doesn't exist". And then just run that
>> before doing the push.
>>
>> For an unrestricted client connecting over ssh, we already have that:
>> you can just run "ssh $host git init /path/to/repo". There isn't a
>> similar thing that can be done over HTTP, though.
> Sounds simpler to just change the user's login shell to a wrapper that
> checks if the repo exists, and if not create it before proceeding.
>
> The same with http(s). I.e. in whatever webserver that's now pointing to
> git-http-backend as a script point to the same wrapper script.
>
>




^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Git server side "pre-receive" hook to create new repositories
  2019-03-19 12:16     ` Dimitri Joukoff
@ 2019-03-21  9:28       ` Sitaram Chamarty
  0 siblings, 0 replies; 5+ messages in thread
From: Sitaram Chamarty @ 2019-03-21  9:28 UTC (permalink / raw)
  To: Dimitri Joukoff, Ævar Arnfjörð Bjarmason, Jeff King; +Cc: git



On 19/03/2019 17.46, Dimitri Joukoff wrote:
> Thanks for the feedback.
> 
> Whilst I understand the options that you've both proposed, the intent
> was to enable the Swift Package Manager to mirror repositories in a
> transparent way.  I'll look into whether these options can be
> implemented inside the SPM.

I don't know the Swift Package Manager, but looking at the preceding
discussion reminded that gitolite allows this.

I'm not saying that someone should install gitolite just to get this
feature, but rather that *any* script that traps the incoming request
can do this.  Almost trivial if the request is coming in via ssh,
actually.  Just set the shell to a script that looks at and parses
SSH_ORIGINAL_COMMAND and do whatever is needed before passing control to
git proper.

sitaram

> 
> Cheers,
> Dimitri.
> 
> 
> On 19/3/19 7:24 pm, Ævar Arnfjörð Bjarmason wrote:
>> On Tue, Mar 19 2019, Jeff King wrote:
>>
>>> On Sat, Mar 09, 2019 at 10:46:09AM +0000, Dimitri Joukoff wrote:
>>>
>>>> Thus, this feature request is asking that the 'pre-receive' hook
>>>> triggers when someone tries to push to a repository regardless of
>>>> whether the repository exists.  This would allow automatic creation of
>>>> new repositories and smooth the work-flow described above.  If the
>>>> semantics of the existing 'pre-receive' hook are such that it would not
>>>> be suitable for such a purpose, then an alternative way of providing the
>>>> call-back ability would be implemented.
>>> The pre-receive hook is a bit too late for this. It runs after the
>>> server has told the client what it has in the repo, the client decides
>>> what to push, and the server has received the pack. So receive-pack
>>> would have to know about this and fake having an empty repository. And
>>> then figure out where to store the incoming packfile, since we have no
>>> repo.
>>>
>>> So I think it would have to be another hook that runs before the rest of
>>> receive-pack. I.e., a system-level config option that says "if you are
>>> asked to accept a push for a repo and it doesn't exist, run this instead
>>> and then run as usual".
>>>
>>> It does feel a little error-prone, though, if the client does not
>>> positively say "I want you to create this if it doesn't exist".
>>> Otherwise if I do "git push server:my-misspelled-repo.git", the result
>>> is going to be rather confusing. And retro-fitting that into the
>>> receive-pack protocol is going to be tricky.
>>>
>>> It would be much easier to have a separate endpoint for the client to
>>> say "please make this repo if it doesn't exist". And then just run that
>>> before doing the push.
>>>
>>> For an unrestricted client connecting over ssh, we already have that:
>>> you can just run "ssh $host git init /path/to/repo". There isn't a
>>> similar thing that can be done over HTTP, though.
>> Sounds simpler to just change the user's login shell to a wrapper that
>> checks if the repo exists, and if not create it before proceeding.
>>
>> The same with http(s). I.e. in whatever webserver that's now pointing to
>> git-http-backend as a script point to the same wrapper script.
>>
>>
> 
> 
> 

-- 
sitaram

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2019-03-21  9:28 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-09 10:46 Git server side "pre-receive" hook to create new repositories Dimitri Joukoff
2019-03-19  7:06 ` Jeff King
2019-03-19  9:24   ` Ævar Arnfjörð Bjarmason
2019-03-19 12:16     ` Dimitri Joukoff
2019-03-21  9:28       ` Sitaram Chamarty

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git