user/dev discussion of public-inbox itself
 help / color / Atom feed
* RFE: default hooks for git repositories
@ 2019-03-06 17:03 Konstantin Ryabitsev
  2019-03-07  2:42 ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Konstantin Ryabitsev @ 2019-03-06 17:03 UTC (permalink / raw)
  To: meta

[-- Attachment #1: Type: text/plain, Size: 486 bytes --]

Hello:

Since V2 creates git shards automatically, I suggest there should be a 
way to tell it to install some set of default hooks in all new git repos 
it creates. The easiest would be by using a skel dir and specifying it 
in PI_CONFIG:

[publicinbox]
  githooksskel=/usr/share/public-inbox/hooks

We currently do it via a cronjob, but unless we run it very frequently, 
there's a delay between when a new shard is created and when we add the 
hooks to it.

Best,
-K

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-03-06 17:03 RFE: default hooks for git repositories Konstantin Ryabitsev
@ 2019-03-07  2:42 ` Eric Wong
  2019-03-08 23:01   ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2019-03-07  2:42 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> Hello:
> 
> Since V2 creates git shards automatically, I suggest there should be a way
> to tell it to install some set of default hooks in all new git repos it
> creates. The easiest would be by using a skel dir and specifying it in
> PI_CONFIG:
> 
> [publicinbox]
>  githooksskel=/usr/share/public-inbox/hooks

Seems reasonable.  Though "gitInitTemplateDir" would probably be
easier-to-implement and cover hooks.  This should let us avoid
creating needless "branches" directories and "info/exclude"
files, as well.

Curious, what hooks do you use?  I've never used any.

> We currently do it via a cronjob, but unless we run it very frequently,
> there's a delay between when a new shard is created and when we add the
> hooks to it.

I've also been thinking of a way to get mirrors to automatically
know about new epochs, too.  Maybe submodules can be used...


Note: I prefer the "epoch" term for the $N.git dirs over "shard"
      to disambiguate it from the Xapian partitioning we do.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-03-07  2:42 ` Eric Wong
@ 2019-03-08 23:01   ` Eric Wong
  2019-03-18 23:11     ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2019-03-08 23:01 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Eric Wong <e@80x24.org> wrote:
> Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > Hello:
> > 
> > Since V2 creates git shards automatically, I suggest there should be a way
> > to tell it to install some set of default hooks in all new git repos it
> > creates. The easiest would be by using a skel dir and specifying it in
> > PI_CONFIG:
> > 
> > [publicinbox]
> >  githooksskel=/usr/share/public-inbox/hooks
> 
> Seems reasonable.  Though "gitInitTemplateDir" would probably be
> easier-to-implement and cover hooks.  This should let us avoid
> creating needless "branches" directories and "info/exclude"
> files, as well.
> 
> Curious, what hooks do you use?  I've never used any.

Actually, is there a reason the git-native "init.templateDir"
configuration variable isn't sufficient?

i.e. something about public-inbox git repos should be set/unset
which is fundamentally incompatible with code-oriented git repos?

I've been trying to use normal git defaults and directives as
possible to minimize the learning curve.


I've started working on a dirty patch to implement this feature;
but it's rather intrusive and I'm not sure I like it...

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-03-08 23:01   ` Eric Wong
@ 2019-03-18 23:11     ` Eric Wong
  2019-04-01 15:41       ` Konstantin Ryabitsev
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2019-03-18 23:11 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Eric Wong <e@80x24.org> wrote:
> Actually, is there a reason the git-native "init.templateDir"
> configuration variable isn't sufficient?

Ping?

> i.e. something about public-inbox git repos should be set/unset
> which is fundamentally incompatible with code-oriented git repos?
> 
> I've been trying to use normal git defaults and directives as
> possible to minimize the learning curve.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-03-18 23:11     ` Eric Wong
@ 2019-04-01 15:41       ` Konstantin Ryabitsev
  2019-04-01 18:04         ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Konstantin Ryabitsev @ 2019-04-01 15:41 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

On Mon, Mar 18, 2019 at 11:11:52PM +0000, Eric Wong wrote:
>Eric Wong <e@80x24.org> wrote:
>> Actually, is there a reason the git-native "init.templateDir"
>> configuration variable isn't sufficient?
>
>Ping?

Sorry, Eric, I'm close to declaring email bankruptcy. :) I am okay with 
any solution that lets us add hooks. To explain why we need hooks in 
public-inbox repos, it's mostly for the ease of mirroring -- I want a 
post-update hook to update the grokmirror [1] manifest so that our 
mirroring script knows when there are updates and also when there are 
new archives or repository epochs added.

So, I defer to you entirely when it comes to how to do it. :)


[1] https://github.com/mricon/grokmirror/

-K

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-04-01 15:41       ` Konstantin Ryabitsev
@ 2019-04-01 18:04         ` Eric Wong
  2019-04-05 17:43           ` Konstantin Ryabitsev
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2019-04-01 18:04 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Mon, Mar 18, 2019 at 11:11:52PM +0000, Eric Wong wrote:
> > Eric Wong <e@80x24.org> wrote:
> > > Actually, is there a reason the git-native "init.templateDir"
> > > configuration variable isn't sufficient?
> > 
> > Ping?
> 
> Sorry, Eric, I'm close to declaring email bankruptcy. :) I am okay with any
> solution that lets us add hooks. To explain why we need hooks in
> public-inbox repos, it's mostly for the ease of mirroring -- I want a
> post-update hook to update the grokmirror [1] manifest so that our mirroring
> script knows when there are updates and also when there are new archives or
> repository epochs added.
> 
> So, I defer to you entirely when it comes to how to do it. :)

Yes, putting the hook in the init.templateDir should get you
what you want.  It would work the same for newly-created code
repositories, too.

> [1] https://github.com/mricon/grokmirror/

I should look into that for my own mirrors, even :)

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-04-01 18:04         ` Eric Wong
@ 2019-04-05 17:43           ` Konstantin Ryabitsev
  2019-04-05 20:07             ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Konstantin Ryabitsev @ 2019-04-05 17:43 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

On Mon, Apr 01, 2019 at 06:04:03PM +0000, Eric Wong wrote:
>> Sorry, Eric, I'm close to declaring email bankruptcy. :) I am okay 
>> with any
>> solution that lets us add hooks. To explain why we need hooks in
>> public-inbox repos, it's mostly for the ease of mirroring -- I want a
>> post-update hook to update the grokmirror [1] manifest so that our mirroring
>> script knows when there are updates and also when there are new archives or
>> repository epochs added.
>>
>> So, I defer to you entirely when it comes to how to do it. :)
>
>Yes, putting the hook in the init.templateDir should get you
>what you want.  It would work the same for newly-created code
>repositories, too.

So, I just realized that, at least when mail is delivered via 
public-inbox-mda, none of the hooks are ever triggered. I've tried with 
post-update, post-receive, post-commit, pre-commit, etc. Am I doing 
something wrong, or is this because the operation is being done on a low 
enough level that the hooks aren't triggered as they would with a 
regular git operation?

-K

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-04-05 17:43           ` Konstantin Ryabitsev
@ 2019-04-05 20:07             ` Eric Wong
  2019-04-06  4:52               ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2019-04-05 20:07 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> So, I just realized that, at least when mail is delivered via
> public-inbox-mda, none of the hooks are ever triggered. I've tried with
> post-update, post-receive, post-commit, pre-commit, etc. Am I doing
> something wrong, or is this because the operation is being done on a low
> enough level that the hooks aren't triggered as they would with a regular
> git operation?

Hmm... Perhaps git-fast-import does not call hooks?
That might need to be brought up with the git list.

(only online for a few minutes, gotta run out again)

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: RFE: default hooks for git repositories
  2019-04-05 20:07             ` Eric Wong
@ 2019-04-06  4:52               ` Eric Wong
  2019-04-15 18:29                 ` [PATCH] TODO: add an item for hooks support Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2019-04-06  4:52 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Eric Wong <e@80x24.org> wrote:
> Hmm... Perhaps git-fast-import does not call hooks?

Yeah, I remember; low-level git stuff doesn't do hooks.

So we might want to implement our own...
(won't be online much the next few days)

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH] TODO: add an item for hooks support
  2019-04-06  4:52               ` Eric Wong
@ 2019-04-15 18:29                 ` Eric Wong
  2019-04-29  0:33                   ` update on grokmirror integration [was: [PATCH] TODO: add an item for hooks support] Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2019-04-15 18:29 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Eric Wong <e@80x24.org> wrote:
> Yeah, I remember; low-level git stuff doesn't do hooks.
> 
> So we might want to implement our own...

-------8<------
Subject: [PATCH] TODO: add an item for hooks support

and add a note for grokmirror support/integration, too
---
 TODO | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/TODO b/TODO
index ef77d1c..7a3bb6b 100644
--- a/TODO
+++ b/TODO
@@ -98,3 +98,8 @@ all need to be considered for everything we introduce)
 
 * imperfect scraper importers for obfuscated list archives
   (e.g. obfuscated Mailman stuff, Google Groups, etc...)
+
+* support hooks, since low-level git-fast-import does not run them
+  https://public-inbox.org/meta/20190405174329.GA21472@chatter.qube.local/
+
+* investigate native grokmirror support/integration
-- 
EW

^ permalink raw reply	[flat|nested] 11+ messages in thread

* update on grokmirror integration [was: [PATCH] TODO: add an item for hooks support]
  2019-04-15 18:29                 ` [PATCH] TODO: add an item for hooks support Eric Wong
@ 2019-04-29  0:33                   ` Eric Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2019-04-29  0:33 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Eric Wong <e@80x24.org> wrote:
> +* investigate native grokmirror support/integration

GitPython in Debian 9.x is busted with sorted packed-refs;
but the GitPython in Debian testing is fine and the next stable
release is just around the corner.

Hoping the Debian packager can put in an updated grokmirror, soon:

	https://bugs.debian.org/927271

But I think existing PSGI/Plack with Plack::Middleware::ConditionalGET
static file support is sufficient on the HTTP-side, and
our Import.pm should probably call $GIT_DIR/hooks/post-update...

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, back to index

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-06 17:03 RFE: default hooks for git repositories Konstantin Ryabitsev
2019-03-07  2:42 ` Eric Wong
2019-03-08 23:01   ` Eric Wong
2019-03-18 23:11     ` Eric Wong
2019-04-01 15:41       ` Konstantin Ryabitsev
2019-04-01 18:04         ` Eric Wong
2019-04-05 17:43           ` Konstantin Ryabitsev
2019-04-05 20:07             ` Eric Wong
2019-04-06  4:52               ` Eric Wong
2019-04-15 18:29                 ` [PATCH] TODO: add an item for hooks support Eric Wong
2019-04-29  0:33                   ` update on grokmirror integration [was: [PATCH] TODO: add an item for hooks support] Eric Wong

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror http://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.org/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox