git repack and dumb protocols

git repack and dumb protocols

[Andreas Klöckner]

[-- Attachment #1: Type: text/plain, Size: 1332 bytes --]

Hi there,

I've posted a git repository to the web that is an rsync copy of my 
development archive's .git directory, with update-server-info run on it. Now 
if I understand correctly, this contains a single file for every 
commit/file/whatever else I ever made, which is a paint over HTTP. I have then 
recently learned about the wonder that is git repack, which I would love to 
use on this repository, were it not for this comment in its manpage:

8< ----------------------------------------------------------------------
Especially useful when packing a repository that is used for private 
development and there is no need to worry about people fetching via dumb 
protocols from it. 8< 
----------------------------------------------------------------------

It says this relating to the '-a' option, but it makes me wonder if I'll break 
copies of the repo that other people have pulled if I use 'repack'? What are 
the possible interactions and things to keep in mind between repack and dumb 
protocols?

In any case, I would like to suggest that the rather ominous-sounding text in 
the manpage be replaced with something more concrete, i.e. "If you repack an 
archive that has people fetching via dumb protocols from it, X, Y, and Z will 
happen."

Thanks,
Andreas

(please cc--not subscribed)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[PATCH] git-pack.txt: Clarify implications of -a for dumb protocols

[Michael J Gruber]

The current text make some users feel uneasy, worrying whether
'-a' could lead to corrupt repositories. Clarify that '-a'
may lead to performance issues only for dumb protocols.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
Is that clear enough but concise?

Applies to maint.

 Documentation/git-repack.txt |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index aaa8852..8af5505 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -32,7 +32,10 @@ OPTIONS
 	pack everything referenced into a single pack.
 	Especially useful when packing a repository that is used
 	for private development and there is no need to worry
-	about people fetching via dumb protocols from it.  Use
+	about people fetching via dumb protocols from it. 
+	(Over dumb protocols, one would have to fetch the whole new
+	pack in order to get any contained object, no matter how
+	many other objects in that pack exist locally already.) Use
 	with '-d'.  This will clean up the objects that `git prune`
 	leaves behind, but `git fsck --full` shows as
 	dangling.
-- 
1.6.3.1.268.g94d6d1

[PATCHv1+eps] git-repack.txt: Clarify implications of -a for dumb protocols

[Michael J Gruber]

The current text makes some users feel uneasy, worrying whether
'-a' could lead to corrupt repositories. Clarify that '-a'
may lead to performance issues only for dumb protocols.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
Is that clear enough but concise?

Applies to maint.

[Sorry for the resend, I managed to mistype the subject.]

 Documentation/git-repack.txt |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index aaa8852..8af5505 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -32,7 +32,10 @@ OPTIONS
 	pack everything referenced into a single pack.
 	Especially useful when packing a repository that is used
 	for private development and there is no need to worry
-	about people fetching via dumb protocols from it.  Use
+	about people fetching via dumb protocols from it. 
+	(Over dumb protocols, one would have to fetch the whole new
+	pack in order to get any contained object, no matter how
+	many other objects in that pack exist locally already.) Use
 	with '-d'.  This will clean up the objects that `git prune`
 	leaves behind, but `git fsck --full` shows as
 	dangling.
-- 
1.6.3.1.268.g94d6d1

Re: [PATCHv1+eps] git-repack.txt: Clarify implications of -a for dumb protocols

[Andreas Klöckner]

[-- Attachment #1: Type: text/plain, Size: 1377 bytes --]

Thanks, that's much better!

Andreas

On Donnerstag 04 Juni 2009, Michael J Gruber wrote:
> The current text makes some users feel uneasy, worrying whether
> '-a' could lead to corrupt repositories. Clarify that '-a'
> may lead to performance issues only for dumb protocols.
>
> Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
> ---
> Is that clear enough but concise?
>
> Applies to maint.
>
> [Sorry for the resend, I managed to mistype the subject.]
>
>  Documentation/git-repack.txt |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
>
> diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
> index aaa8852..8af5505 100644
> --- a/Documentation/git-repack.txt
> +++ b/Documentation/git-repack.txt
> @@ -32,7 +32,10 @@ OPTIONS
>  	pack everything referenced into a single pack.
>  	Especially useful when packing a repository that is used
>  	for private development and there is no need to worry
> -	about people fetching via dumb protocols from it.  Use
> +	about people fetching via dumb protocols from it.
> +	(Over dumb protocols, one would have to fetch the whole new
> +	pack in order to get any contained object, no matter how
> +	many other objects in that pack exist locally already.) Use
>  	with '-d'.  This will clean up the objects that `git prune`
>  	leaves behind, but `git fsck --full` shows as
>  	dangling.



[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCHv1+eps] git-repack.txt: Clarify implications of -a for dumb protocols

[Stephen Boyd]

On Thu, Jun 4, 2009 at 4:34 AM, Michael J
Gruber<git@drmicha.warpmail.net> wrote:
>        pack everything referenced into a single pack.
>        Especially useful when packing a repository that is used
>        for private development and there is no need to worry
> -       about people fetching via dumb protocols from it.  Use
> +       about people fetching via dumb protocols from it.
> +       (Over dumb protocols, one would have to fetch the whole new
> +       pack in order to get any contained object, no matter how
> +       many other objects in that pack exist locally already.) Use
>        with '-d'.  This will clean up the objects that `git prune`
>        leaves behind, but `git fsck --full` shows as
>        dangling.

Instead of adding this, why don't we just remove the part about dumb
protocols? So just say that it's especially useful for private
development. Then, if we really want to keep this performance note
just add it after the paragraph in one of those "+" things (What are
those called?)

Re: [PATCHv1+eps] git-repack.txt: Clarify implications of -a for dumb protocols

[Michael J Gruber]

Stephen Boyd venit, vidit, dixit 04.06.2009 21:46:
> On Thu, Jun 4, 2009 at 4:34 AM, Michael J
> Gruber<git@drmicha.warpmail.net> wrote:
>>        pack everything referenced into a single pack.
>>        Especially useful when packing a repository that is used
>>        for private development and there is no need to worry
>> -       about people fetching via dumb protocols from it.  Use
>> +       about people fetching via dumb protocols from it.
>> +       (Over dumb protocols, one would have to fetch the whole new
>> +       pack in order to get any contained object, no matter how
>> +       many other objects in that pack exist locally already.) Use
>>        with '-d'.  This will clean up the objects that `git prune`
>>        leaves behind, but `git fsck --full` shows as
>>        dangling.
> 
> Instead of adding this, why don't we just remove the part about dumb
> protocols? So just say that it's especially useful for private
> development. Then, if we really want to keep this performance note
> just add it after the paragraph in one of those "+" things (What are
> those called?)

...because "repack -a" hurts *badly* on even midsized repos when dumb
protocols are used. On the other hand, using it isn't that much of an
improvement in terms of pack size, unless you repack manually very often
and, thus, produce many small packs. So, "gc --auto" users should be
fine in any case (dumb or not - the protocol, that is...) as long as
they don't use "-a".

Michael

Re: [PATCHv1+eps] git-repack.txt: Clarify implications of -a for dumb protocols

[Stephen Boyd]

On Fri, Jun 5, 2009 at 1:16 AM, Michael J
Gruber<git@drmicha.warpmail.net> wrote:
> Stephen Boyd venit, vidit, dixit 04.06.2009 21:46:
>> Instead of adding this, why don't we just remove the part about dumb
>> protocols? So just say that it's especially useful for private
>> development. Then, if we really want to keep this performance note
>> just add it after the paragraph in one of those "+" things (What are
>> those called?)
>
> ...because "repack -a" hurts *badly* on even midsized repos when dumb
> protocols are used. On the other hand, using it isn't that much of an
> improvement in terms of pack size, unless you repack manually very often
> and, thus, produce many small packs. So, "gc --auto" users should be
> fine in any case (dumb or not - the protocol, that is...) as long as
> they don't use "-a".

Maybe I spoke wrongly. I'm suggesting something like

       Especially useful when packing a repository that is used
       for private development.  Use with '-d'.  This will clean up
       the objects that `git prune` leaves behind, but
       `git fsck --full` shows as dangling.
+
Note: Users fetching over dumb protocols will have to fetch the
whole pack in order to get any contained object, no matter how
many other objects in that pack they already have locally.

I think this Note part probably needs some work though.

[PATCH] git-repack.txt: Clarify implications of -a for dumb protocols

[Michael J Gruber]

The current text makes some users feel uneasy, worrying whether
'-a' could lead to corrupt repositories. Clarify that '-a'
may lead to performance issues only for dumb protocols.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
Helped-by: Stephen Boyd <bebarino@gmail.com>

 Documentation/git-repack.txt |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index aaa8852..c9257a1 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -31,11 +31,14 @@ OPTIONS
 	Instead of incrementally packing the unpacked objects,
 	pack everything referenced into a single pack.
 	Especially useful when packing a repository that is used
-	for private development and there is no need to worry
-	about people fetching via dumb protocols from it.  Use
+	for private development. Use
 	with '-d'.  This will clean up the objects that `git prune`
 	leaves behind, but `git fsck --full` shows as
 	dangling.
++
+Note that users fetching over dumb protocols will have to fetch the
+whole new pack in order to get any contained object, no matter how many
+other objects in that pack they already have locally.
 
 -A::
 	Same as `-a`, unless '-d' is used.  Then any unreachable
-- 
1.6.3.2.278.gb6431.dirty

Re: git repack and dumb protocols

[Peter Harris]

On Wed, Jun 3, 2009 at 7:50 PM, Andreas Klöckner wrote:
>
> 8< ----------------------------------------------------------------------
> Especially useful when packing a repository that is used for private
> development and there is no need to worry about people fetching via dumb
> protocols from it. 8<
> ----------------------------------------------------------------------
>
> It says this relating to the '-a' option, but it makes me wonder if I'll break
> copies of the repo that other people have pulled if I use 'repack'? What are
> the possible interactions and things to keep in mind between repack and dumb
> protocols?

It won't break any copies, but it may cause excessive network use.

Dumb protocols can only fetch or not-fetch the entire pack. So if
someone hasn't fetched since the last push, the next time they do a
"git pull", they will retrieve the entire repository. Even if they
only need a single small commit.

As you can imagine, people using dumb protocols over slow connections
may become somewhat annoyed with the admin that often does a repack
-a.

You do want to repack (usually without -a -- but see also ".keep")
periodically, though. Working out a schedule depends on the activity
level of the repository, and is left as an exercise for the reader.

> In any case, I would like to suggest that the rather ominous-sounding text in
> the manpage be replaced with something more concrete, i.e. "If you repack an
> archive that has people fetching via dumb protocols from it, X, Y, and Z will
> happen."

Patches welcome.

Peter Harris