obnoxious CLI complaints

obnoxious CLI complaints

[Brendan Miller]

Here are a bunch of really basic usability issues I have with git:

1. cloning from a new empty repo fails, and so do a lot of other
operations. This adds unnecessary steps to setting up a new shared
repo.

2. git --bare init. The flag goes before the operation unlike every other flag?

3. It's not obvious whether operations work on the working
directory/the "index"/the repository
e.g. get reset --soft, --mixed, --hard. git diff --cached

4. The index is inconsistently referred to as too many different
things (cache, index, staging area) and only the last one makes any
intuitive sense to a new user. This is partially a CLI issue, and
partially a documentation issue, but both add up to cause confusion.

5. Most commands require lots of flags, and don't have reasonable
defaults. e.g. archive.

git archive --format=tar --prefix=myproject/ HEAD | gzip >myproject.tar.gz

Should just be:
git archive
run from the root of the repo.

This is what I want to do 90% of the time, so it should just have the
proper defaults, and not make me look at the man page every time I
want to use it.

6. Where is the bug tracker? If people users can't find the bug
tracker, they can't report issues, and obnoxious bugs go unfixed, or
people have to whine on the mailing list. There should be a nice big
link on the front page of git-scm.com. A bug tracker is really the
only way for the vast majority of a community that use a tool can give
feedback on the problems the tool has.

7. Man pages: It's nice we have them, but we shouldn't need them to do
basic stuff. I rarely had to look at the man pages using svn, but
every single time I use git I have to dig into these things. Frankly,
I have better things to do than RTFM.

8. There's no obvious way to make a remote your default push pull
location without editing the git config file. Why not just something
like

git remote setdefault origin

or even

git remote add --default origin http://somegiturl.org/

This come up in the use case where I:
1. set up a remote bare repo
2. push from my local repo, and thence forth want to keep local and
remote in sink.

Right now I have to modify .git/config to do this.

It's ok to have kind of a weak UI on a new tool, when people are busy
adding basic functionality. However, at this point git already has way
more features than most of the competition, and the needless
complexity of the CLI is the biggest issue in day to day use.

Brendan

Re: obnoxious CLI complaints

[Jakub Narebski]

Brendan Miller <catphive@catphive.net> writes:

> Here are a bunch of really basic usability issues I have with git:

First question: which git version do you use?
 
> 1. cloning from a new empty repo fails, and so do a lot of other
> operations. This adds unnecessary steps to setting up a new shared
> repo.

I think it works in modern git (where modern might mean 'master',
i.e. not yet released version)

> 
> 2. git --bare init. The flag goes before the operation unlike every
> other flag?

"git init --bare" works in fit version 1.6.4.2

In "git --bare init" the --bare option is to 'git' wrapper, not to
git-init command (see git(1)):

  --bare Treat the repository as a bare repository. If GIT_DIR
         environment is not set, it is set to the current working
         directory.

> 
> 3. It's not obvious whether operations work on the working
> directory/the "index"/the repository
> e.g. get reset --soft, --mixed, --hard. git diff --cached

There is "git diff --staged" synonym for "git diff --cached"

> 
> 4. The index is inconsistently referred to as too many different
> things (cache, index, staging area) and only the last one makes any
> intuitive sense to a new user. This is partially a CLI issue, and
> partially a documentation issue, but both add up to cause confusion.

Usage of '--index' and '--cached' in CLI is consistent, and different.
See gitcli(7) manpage.

Some of those inconsistences are historical remainings (I think we got
rid of 'dircache' and 'ent').  Do you offer to do a cleanup?

> 
> 5. Most commands require lots of flags, and don't have reasonable
> defaults. e.g. archive.
> 
> git archive --format=tar --prefix=myproject/ HEAD | gzip >myproject.tar.gz
> 
> Should just be:
> git archive
> run from the root of the repo.

I'd rather not have "git archive" work without specifying tree-ish.
As for having to do compression using separate program: do one thing
and do it well is UNIX philosophy, and Git is UNIX-y tool.

BTW. git-archive _has_ default format type (and empty prefix by
default).

> 
> This is what I want to do 90% of the time, so it should just have the
> proper defaults, and not make me look at the man page every time I
> want to use it.

You learn those idioms.

> 
> 6. Where is the bug tracker? If people users can't find the bug
> tracker, they can't report issues, and obnoxious bugs go unfixed, or
> people have to whine on the mailing list. There should be a nice big
> link on the front page of git-scm.com. A bug tracker is really the
> only way for the vast majority of a community that use a tool can give
> feedback on the problems the tool has.

Do you offer to maintain and manage such bug tracker?  I mean here
taking care of duplicated bugs, tracking which bugs are resolved and
which are not, checking if bug is reproductible, etc.  Do you?
Unmaintained bugtracker is worse than useless.

Using mailing list for bug reports and for patches is time-honored
workflow, which works rather well for smaller projects such as Git.
Note that git mailing list is free for all; you don't need to
subscribe to send, and you can watch it via many archives and gateways
(like GMane).

> 
> 7. Man pages: It's nice we have them, but we shouldn't need them to do
> basic stuff. I rarely had to look at the man pages using svn, but
> every single time I use git I have to dig into these things. Frankly,
> I have better things to do than RTFM.

Learn.  If you learn the philosophy behind git design, you would have
much easier understanding and remembering git.

There is "Git User's Manual", "The Git Community Book", "Pro Git" and
many other references.

> 
> 8. There's no obvious way to make a remote your default push pull
> location without editing the git config file. Why not just something
> like

origin is default, I think.

> 
> git remote setdefault origin
> 
> or even
> 
> git remote add --default origin http://somegiturl.org/
> 
> This come up in the use case where I:
> 1. set up a remote bare repo
> 2. push from my local repo, and thence forth want to keep local and
> remote in sink.
> 
> Right now I have to modify .git/config to do this.

And?

> It's ok to have kind of a weak UI on a new tool, when people are busy
> adding basic functionality. However, at this point git already has way
> more features than most of the competition, and the needless
> complexity of the CLI is the biggest issue in day to day use.

Creating good UI is not easy, especially if you are limited by
backward compatibility.

-- 
Jakub Narebski

Git User's Survey 2009: http://tinyurl.com/GitSurvey2009

Re: obnoxious CLI complaints

[Wincent Colaiuta]

El 09/09/2009, a las 23:54, Jakub Narebski escribió:

> Brendan Miller <catphive@catphive.net> writes:
>
>> 5. Most commands require lots of flags, and don't have reasonable
>> defaults. e.g. archive.
>>
>> git archive --format=tar --prefix=myproject/ HEAD | gzip  
>> >myproject.tar.gz
>>
>> Should just be:
>> git archive
>> run from the root of the repo.
>
> I'd rather not have "git archive" work without specifying tree-ish.

Why, out of interest? I would've thought that HEAD would be a pretty  
good default, although I confess that I have never used "git archive"  
without specifying a particular signed tag.

Cheers,
Wincent

Re: obnoxious CLI complaints

[Jakub Narebski]

Dnia czwartek 10. września 2009 00:06, Wincent Colaiuta napisał:
> El 09/09/2009, a las 23:54, Jakub Narebski escribió:
>> Brendan Miller <catphive@catphive.net> writes:
>>
>>> 5. Most commands require lots of flags, and don't have reasonable
>>> defaults. e.g. archive.
>>>
>>> $ git archive --format=tar --prefix=myproject/ HEAD | 
>>> > gzip myproject.tar.gz 
>>>
>>> Should just be:
>>> git archive
>>> run from the root of the repo.
>>
>> I'd rather not have "git archive" work without specifying tree-ish.
> 
> Why, out of interest? I would've thought that HEAD would be a pretty  
> good default, although I confess that I have never used "git archive"  
> without specifying a particular signed tag.

First, it would be consistent with how ordinary archivers such as tar
or zip are used, where you have to specify list of files to archive
(in our case this list is HEAD).  Second, I'd rather not accidentally
dump binary to terminal: "git archive [HEAD]" dumps archive to standard
output.

-- 
Jakub Narebski
Poland

Re: obnoxious CLI complaints

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> First, it would be consistent with how ordinary archivers such as tar
> or zip are used, where you have to specify list of files to archive
> (in our case this list is HEAD).  Second, I'd rather not accidentally
> dump binary to terminal: "git archive [HEAD]" dumps archive to standard
> output.

So does "cat".  I do not agree with your second point.

While I somewhat see the similarity argument, your first point, I am not
sure if it is relevant.  It is not like "tar or zip allows us to say what
files to archive, but git-archive doesn't and it always archives HEAD";
you are saying "they require us to specify, so should we".

But I do not see a strong reason not to default to HEAD.  The case that
would make difference would be to differentiate among

	$ git archive HEAD TAIL
        $ git archive HEAD -- TAIL
        $ git archive -- HEAD TAIL

i.e. what if you happen to have a tracked content called HEAD.  I didn't
check the current command line parser in git-archive understands the "--"
convention for that, but it is not a rocket science to add it if it
doesn't.

Re: obnoxious CLI complaints

[René Scharfe]

Junio C Hamano schrieb:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
>> First, it would be consistent with how ordinary archivers such as tar
>> or zip are used, where you have to specify list of files to archive
>> (in our case this list is HEAD).  Second, I'd rather not accidentally
>> dump binary to terminal: "git archive [HEAD]" dumps archive to standard
>> output.
> 
> So does "cat".  I do not agree with your second point.
> 
> While I somewhat see the similarity argument, your first point, I am not
> sure if it is relevant.  It is not like "tar or zip allows us to say what
> files to archive, but git-archive doesn't and it always archives HEAD";
> you are saying "they require us to specify, so should we".
> 
> But I do not see a strong reason not to default to HEAD.  The case that
> would make difference would be to differentiate among
> 
> 	$ git archive HEAD TAIL
>         $ git archive HEAD -- TAIL
>         $ git archive -- HEAD TAIL
> 
> i.e. what if you happen to have a tracked content called HEAD.  I didn't
> check the current command line parser in git-archive understands the "--"
> convention for that, but it is not a rocket science to add it if it
> doesn't.

Currently it doesn't.  An attempt to implement it is below (tests and
documentation update missing).

I wonder if we want to make treeless calls to archive the worktree (or the
index) instead of HEAD, similar to git grep, though.  Not that I remember
someone requesting such a thing, but I'm already slightly surprised about
archive being used to tar up HEAD in any case -- I imagined it would mostly
be used to make releases of tagged versions.

---
 archive.c |   34 ++++++++++++++++++++++++----------
 1 files changed, 24 insertions(+), 10 deletions(-)

diff --git a/archive.c b/archive.c
index 0bca9ca..04fa6a5 100644
--- a/archive.c
+++ b/archive.c
@@ -214,18 +214,32 @@ static void parse_pathspec_arg(const char **pathspec,
 	ar_args->pathspec = get_pathspec(ar_args->base, pathspec);
 }
 
-static void parse_treeish_arg(const char **argv,
-		struct archiver_args *ar_args, const char *prefix)
+static int parse_treeish_arg(int argc, const char **argv,
+			     struct archiver_args *ar_args, const char *prefix)
 {
-	const char *name = argv[0];
+	const char *name = "HEAD";
 	const unsigned char *commit_sha1;
 	time_t archive_time;
 	struct tree *tree;
 	const struct commit *commit;
 	unsigned char sha1[20];
 
+	if (argc > 0) {
+		int consume = 1;
+
+		if (strcmp(argv[0], "--")) {
+			name = argv[0];
+			if (argc > 1 && !strcmp(argv[1], "--"))
+				consume++;
+		}
+
+		argc -= consume;
+		memmove(argv, argv + consume, argc * sizeof(*argv));
+		argv[argc] = NULL;
+	}
+
 	if (get_sha1(name, sha1))
-		die("Not a valid object name");
+		die("Not a valid object name: %s", name);
 
 	commit = lookup_commit_reference_gently(sha1, 1);
 	if (commit) {
@@ -256,6 +270,8 @@ static void parse_treeish_arg(const char **argv,
 	ar_args->commit_sha1 = commit_sha1;
 	ar_args->commit = commit;
 	ar_args->time = archive_time;
+
+	return argc;
 }
 
 #define OPT__COMPR(s, v, h, p) \
@@ -309,7 +325,8 @@ static int parse_archive_args(int argc, const char **argv,
 		OPT_END()
 	};
 
-	argc = parse_options(argc, argv, NULL, opts, archive_usage, 0);
+	argc = parse_options(argc, argv, NULL, opts, archive_usage,
+			     PARSE_OPT_KEEP_DASHDASH);
 
 	if (remote)
 		die("Unexpected option --remote");
@@ -327,9 +344,6 @@ static int parse_archive_args(int argc, const char **argv,
 		exit(0);
 	}
 
-	/* We need at least one parameter -- tree-ish */
-	if (argc < 1)
-		usage_with_options(archive_usage, opts);
 	*ar = lookup_archiver(format);
 	if (!*ar)
 		die("Unknown archive format '%s'", format);
@@ -361,8 +375,8 @@ int write_archive(int argc, const char **argv, const char *prefix,
 	if (setup_prefix && prefix == NULL)
 		prefix = setup_git_directory();
 
-	parse_treeish_arg(argv, &args, prefix);
-	parse_pathspec_arg(argv + 1, &args);
+	argc = parse_treeish_arg(argc, argv, &args, prefix);
+	parse_pathspec_arg(argv, &args);
 
 	git_config(git_default_config, NULL);
 
-- 
1.6.5.rc0

Re: obnoxious CLI complaints

[Björn Steinbrink]

On 2009.09.10 11:53:23 -0700, Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> > First, it would be consistent with how ordinary archivers such as tar
> > or zip are used, where you have to specify list of files to archive
> > (in our case this list is HEAD).  Second, I'd rather not accidentally
> > dump binary to terminal: "git archive [HEAD]" dumps archive to standard
> > output.
> 
> So does "cat".  I do not agree with your second point.

"cat $some_binary" does, not just "cat". I guess Jakub's point was that
a command without arguments shouldn't just put some binary crap onto
your screen. Of course, "git archive HEAD" still does that, but I kind
of he where he's coming from, being one of those that tends to run
"$some_command" without arguments, just to see if it shows me some sort
of short help.

Björn

Re: obnoxious CLI complaints

[John Tapsell]

2009/9/10 Jakub Narebski <jnareb@gmail.com>:
> Dnia czwartek 10. września 2009 00:06, Wincent Colaiuta napisał:
>> El 09/09/2009, a las 23:54, Jakub Narebski escribió:
>>> Brendan Miller <catphive@catphive.net> writes:
>>>
>>>> 5. Most commands require lots of flags, and don't have reasonable
>>>> defaults. e.g. archive.
>>>>
>>>> $ git archive --format=tar --prefix=myproject/ HEAD |
>>>> > gzip myproject.tar.gz
>>>>
>>>> Should just be:
>>>> git archive
>>>> run from the root of the repo.
>>>
>>> I'd rather not have "git archive" work without specifying tree-ish.
>>
>> Why, out of interest? I would've thought that HEAD would be a pretty
>> good default, although I confess that I have never used "git archive"
>> without specifying a particular signed tag.
>
> First, it would be consistent with how ordinary archivers such as tar
> or zip are used, where you have to specify list of files to archive
> (in our case this list is HEAD).  Second, I'd rather not accidentally
> dump binary to terminal: "git archive [HEAD]" dumps archive to standard
> output.

That could be fixed by outputting to a file.  git format-patch outputs
to a file, so why wouldn't git achieve?

John

Re: obnoxious CLI complaints

[Sverre Rabbelier]

Heya,

On Thu, Sep 10, 2009 at 21:46, John Tapsell <johnflux@gmail.com> wrote:
> That could be fixed by outputting to a file.  git format-patch outputs
> to a file, so why wouldn't git achieve?

Because git format-patch works on a per-patch basis, and patches
inherently have a 'name' (the first line of the commit message), an
entire repository does not, so one would have to resort to arbitrary
names such as 'archive.tar.gz' or such.

-- 
Cheers,

Sverre Rabbelier

Re: obnoxious CLI complaints

[Jakub Narebski]

Dnia czwartek 10. września 2009 21:46, John Tapsell napisał:
> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:

> > First, it would be consistent with how ordinary archivers such as tar
> > or zip are used, where you have to specify list of files to archive
> > (in our case this list is HEAD).  Second, I'd rather not accidentally
> > dump binary to terminal: "git archive [HEAD]" dumps archive to standard
> > output.
> 
> That could be fixed by outputting to a file.  git format-patch outputs
> to a file, so why wouldn't git achieve?

"git format-patch" outputs to files because it generates _multiple_
files; generating single patch is special case.  Also git-format-patch
can generate file names from patch (commit) subject; it is not the case
for "git archive" (what name should it use?).

-- 
Jakub Narebski
Poland

Re: obnoxious CLI complaints

[John Tapsell]

2009/9/10 Jakub Narebski <jnareb@gmail.com>:
> Dnia czwartek 10. września 2009 21:46, John Tapsell napisał:
>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>
>> > First, it would be consistent with how ordinary archivers such as tar
>> > or zip are used, where you have to specify list of files to archive
>> > (in our case this list is HEAD).  Second, I'd rather not accidentally
>> > dump binary to terminal: "git archive [HEAD]" dumps archive to standard
>> > output.
>>
>> That could be fixed by outputting to a file.  git format-patch outputs
>> to a file, so why wouldn't git achieve?
>
> "git format-patch" outputs to files because it generates _multiple_
> files; generating single patch is special case.  Also git-format-patch
> can generate file names from patch (commit) subject; it is not the case
> for "git archive" (what name should it use?).

What if it used the current (or topleve) directory name?  Wouldn't
that work in most cases?  For cases it doesn't work, the user can just
rename or specify the output name, so it would be no worse than the
current case.

John

Re: obnoxious CLI complaints

[Junio C Hamano]

John Tapsell <johnflux@gmail.com> writes:

> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>> Dnia czwartek 10. września 2009 21:46, John Tapsell napisał:
>>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>>
>>> > First, it would be consistent with how ordinary archivers such as tar
>>> > or zip are used, where you have to specify list of files to archive
>>> > (in our case this list is HEAD).  Second, I'd rather not accidentally
>>> > dump binary to terminal: "git archive [HEAD]" dumps archive to standard
>>> > output.
>>>
>>> That could be fixed by outputting to a file.  git format-patch outputs
>>> to a file, so why wouldn't git achieve?
>>
>> "git format-patch" outputs to files because it generates _multiple_
>> files; generating single patch is special case.  Also git-format-patch
>> can generate file names from patch (commit) subject; it is not the case
>> for "git archive" (what name should it use?).
>
> What if it used the current (or topleve) directory name?  Wouldn't
> that work in most cases?

Following along the same line of reasoning, it would work in most cases if
the output is literally named "archive.tar".  If it is not the name the
user wants, the user can "mv" afterwards, or give an explicit filename.

What it does _not_ allow is to send the output to a downstream command for
postprocessing without introducing some magic token to say "standard
output" (e.g. "git archive -f - | (cd ../foo && tar xf -)").

If the default is to write to the standard output, we won't have all of
these issues.  People who want a file can name the file by

	git archive >my.file.tar

and people who want to pipe (which is 99% of the use pattern for me) can
say

	git archive | down stream commands.

Oh, wait.

That is exactly what we have, so what's the point of continuing this
discussion any further?  Can we just stop?

Re: obnoxious CLI complaints

[demerphq]

2009/9/11 Junio C Hamano <gitster@pobox.com>:
> John Tapsell <johnflux@gmail.com> writes:
>
>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>>> Dnia czwartek 10. września 2009 21:46, John Tapsell napisał:
>>>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>>>
>>>> > First, it would be consistent with how ordinary archivers such as tar
>>>> > or zip are used, where you have to specify list of files to archive
>>>> > (in our case this list is HEAD).  Second, I'd rather not accidentally
>>>> > dump binary to terminal: "git archive [HEAD]" dumps archive to standard
>>>> > output.
>>>>
>>>> That could be fixed by outputting to a file.  git format-patch outputs
>>>> to a file, so why wouldn't git achieve?
>>>
>>> "git format-patch" outputs to files because it generates _multiple_
>>> files; generating single patch is special case.  Also git-format-patch
>>> can generate file names from patch (commit) subject; it is not the case
>>> for "git archive" (what name should it use?).
>>
>> What if it used the current (or topleve) directory name?  Wouldn't
>> that work in most cases?
>
> Following along the same line of reasoning, it would work in most cases if
> the output is literally named "archive.tar".  If it is not the name the
> user wants, the user can "mv" afterwards, or give an explicit filename.

Why not $sha1.tar?

> What it does _not_ allow is to send the output to a downstream command for
> postprocessing without introducing some magic token to say "standard
> output" (e.g. "git archive -f - | (cd ../foo && tar xf -)").
>
> If the default is to write to the standard output, we won't have all of
> these issues.  People who want a file can name the file by
>
>        git archive >my.file.tar
>
> and people who want to pipe (which is 99% of the use pattern for me) can
> say
>
>        git archive | down stream commands.
>
> Oh, wait.
>
> That is exactly what we have, so what's the point of continuing this
> discussion any further?  Can we just stop?

Is it portable to assume that piping is always in binmode? From a
portability POV i could imagine piping being a problem in this
respect, and might be why tar provides a way to output to a file and
not just to a handle. For example ISTR that on windows piping is by
default in text mode. I think its not a showstopper there as you can
change it, but still, from a portability point of view you might not
want to depend on piping.

cheers,
Yves


-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: obnoxious CLI complaints

[Junio C Hamano]

demerphq <demerphq@gmail.com> writes:

> 2009/9/11 Junio C Hamano <gitster@pobox.com>:
>> John Tapsell <johnflux@gmail.com> writes:
>>
>>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>>>> Dnia czwartek 10. września 2009 21:46, John Tapsell napisał:
>>>>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>>>>
>>>>> > First, it would be consistent with how ordinary archivers such as tar
>>>>> > or zip are used, where you have to specify list of files to archive
>>>>> > (in our case this list is HEAD).  Second, I'd rather not accidentally
>>>>> > dump binary to terminal: "git archive [HEAD]" dumps archive to standard
>>>>> > output.
>>>>>
>>>>> That could be fixed by outputting to a file.  git format-patch outputs
>>>>> to a file, so why wouldn't git achieve?
>>>>
>>>> "git format-patch" outputs to files because it generates _multiple_
>>>> files; generating single patch is special case.  Also git-format-patch
>>>> can generate file names from patch (commit) subject; it is not the case
>>>> for "git archive" (what name should it use?).
>>>
>>> What if it used the current (or topleve) directory name?  Wouldn't
>>> that work in most cases?
>>
>> Following along the same line of reasoning, it would work in most cases if
>> the output is literally named "archive.tar".  If it is not the name the
>> user wants, the user can "mv" afterwards, or give an explicit filename.
>
> Why not $sha1.tar?

Why not $(basename $(dirname $(pwd)))-$(date).tar instead?

See?  archive.tar is as good a compromise (so is a.out from cc).

> Is it portable to assume that piping is always in binmode? From a
> portability POV i could imagine piping being a problem in this
> respect, and might be why tar provides a way to output to a file and
> not just to a handle. For example ISTR that on windows piping is by
> default in text mode. I think its not a showstopper there as you can
> change it, but still, from a portability point of view you might not
> want to depend on piping.

Windows is not a showstopper to me ;-).

But seriously, I am glad that you brought up about a potential issue with
pipe.  There is one fairly important reason that it is better to say

	GZIP=-9 tar zcf here.tar.gz .

than to say

	tar cf - . | gzip -9 >here.tar.gz

but it has nothing to do with binmode.  The reason is error detection.

For exactly the same reason, if we can say

	git archive -9 --output-file=here.tar.gz HEAD

it is much better than having to say

	git archive HEAD | gzip -9 >here.tar.gz

In other words, I am not opposed to supporting a "--output-file here.tar"
at all.  I just do not want it to be mandatory.  I think that it is an
ugly kludge to force people to work it around with "-f /dev/stdout".

Oh wait.

That is exactly what we have, so what's the point of continuing this
discussion any further?  Can we just _really_ stop this time, please?

Re: obnoxious CLI complaints

[John Tapsell]

2009/9/11 Junio C Hamano <gitster@pobox.com>:
> John Tapsell <johnflux@gmail.com> writes:
>
>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>>> Dnia czwartek 10. września 2009 21:46, John Tapsell napisał:
>>>> 2009/9/10 Jakub Narebski <jnareb@gmail.com>:
>>>
>>>> > First, it would be consistent with how ordinary archivers such as tar
>>>> > or zip are used, where you have to specify list of files to archive
>>>> > (in our case this list is HEAD).  Second, I'd rather not accidentally
>>>> > dump binary to terminal: "git archive [HEAD]" dumps archive to standard
>>>> > output.
>>>>
>>>> That could be fixed by outputting to a file.  git format-patch outputs
>>>> to a file, so why wouldn't git achieve?
>>>
>>> "git format-patch" outputs to files because it generates _multiple_
>>> files; generating single patch is special case.  Also git-format-patch
>>> can generate file names from patch (commit) subject; it is not the case
>>> for "git archive" (what name should it use?).
>>
>> What if it used the current (or topleve) directory name?  Wouldn't
>> that work in most cases?
>
> Following along the same line of reasoning, it would work in most cases if
> the output is literally named "archive.tar".  If it is not the name the
> user wants, the user can "mv" afterwards, or give an explicit filename.

That would also work.  Like how gcc uses "a.out" as the default filename


> What it does _not_ allow is to send the output to a downstream command for
> postprocessing without introducing some magic token to say "standard
> output" (e.g. "git archive -f - | (cd ../foo && tar xf -)").

Right, so what's wrong with the magic token?  There's plenty of precedence.


> If the default is to write to the standard output, we won't have all of
> these issues.

These are issues?

>  People who want a file can name the file by
>
>        git archive >my.file.tar

I thought you didn't like this because then you dump binary to the
console by default ?

> and people who want to pipe (which is 99% of the use pattern for me) can
> say
>
>        git archive | down stream commands.

Why would it be so bad to do:

git archive -f - | down stream commands

?

This is the most logical way forward.  It keeps the command simple for
simple use cases (make an archive - "git archive")  but easily
scalable for more complex use cases (add a  "-f -" if you want to do
magical things)

John

Re: obnoxious CLI complaints

[Junio C Hamano]

John Tapsell <johnflux@gmail.com> writes:

>>        git archive >my.file.tar
>
> I thought you didn't like this because then you dump binary to the
> console by default ?

No. That objection came from Jakub and I said it did not make sense to me.

> Why would it be so bad to do:
>
> git archive -f - | down stream commands

Are you seriously asking "why" after I said this?

>> and people who want to pipe (which is 99% of the use pattern for me) can
>> say

If you really need it to be spelled out,...

Because I have to say "-f -" 99% of the time without no good reason.

Re: obnoxious CLI complaints

[Brendan Miller]

On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> Brendan Miller <catphive@catphive.net> writes:
> First question: which git version do you use?

It sounds like a bunch of things have been fixed in yet to be released
versions. That's great.

>>
>> This is what I want to do 90% of the time, so it should just have the
>> proper defaults, and not make me look at the man page every time I
>> want to use it.
>
> You learn those idioms.

I guess. Is that a good thing? Is the goal of interface design to make
it difficult so I need to learn a lot of things, or easy so I can
remain blissfully ignorant but still do what I want?

>
>>
>> 6. Where is the bug tracker? If people users can't find the bug
>> tracker, they can't report issues, and obnoxious bugs go unfixed, or
>> people have to whine on the mailing list. There should be a nice big
>> link on the front page of git-scm.com. A bug tracker is really the
>> only way for the vast majority of a community that use a tool can give
>> feedback on the problems the tool has.
>
> Do you offer to maintain and manage such bug tracker?  I mean here
> taking care of duplicated bugs, tracking which bugs are resolved and
> which are not, checking if bug is reproductible, etc.  Do you?
> Unmaintained bugtracker is worse than useless.
>
> Using mailing list for bug reports and for patches is time-honored
> workflow, which works rather well for smaller projects such as Git.
> Note that git mailing list is free for all; you don't need to
> subscribe to send, and you can watch it via many archives and gateways
> (like GMane).

Bug trackers are a hassle, believe me, I know... but I think they
contribute to the overall quality of the product if used effectively.
Mailing lists seem like a good way to forget about bugs after people
have given up on getting developers to fix them.

>
>>
>> 7. Man pages: It's nice we have them, but we shouldn't need them to do
>> basic stuff. I rarely had to look at the man pages using svn, but
>> every single time I use git I have to dig into these things. Frankly,
>> I have better things to do than RTFM.
>
> Learn.  If you learn the philosophy behind git design, you would have
> much easier understanding and remembering git.

I think what you mean by philosophy is the underlying data structures,
which are discussed in the manual (how many apps do you have that do
that!). I have read that. However, that one needs to understand
underlying data structure is just one more hurdle to understanding
git.

If I use GCC, do I need to know that it has a recursive descent
parser? That it is implemented with a garbage collector? No. I just
need to know that I give it C, and it gives me a binary.

Example:
gcc main.c

Think about all the defaults that are specified here! I don't
explicitly tell it how to find libc.so or what path the dynamic linker
is at. I don't even really need to tell it which operation it is
performing, i.e. generating a binary, .o, .so, .os, .a, etc because it
has a smart default.

This an order of magnitude more complex than any git operation in
terms of implementation, but it is dead simple from the users
perspective.

>
> There is "Git User's Manual", "The Git Community Book", "Pro Git" and
> many other references.

Yeah, I've been reading them. I'm saying that the docs are a crutch.
RTFM is the problem not the solution. It makes the user do more work
to avoid fixing usability issues.

A CLI has some inherent limitations in that it doesn't have big
labeled buttons to press. However, that doesn't mean it has to be hard
to use. I think a lot of the strength of the linux CLI is that most of
the utilities have actually pretty well thought out interfaces that
have been refined over time. That one's that aren't like that... well,
no one uses them.

I'm not saying that a unixy approach is wrong, but that most unix
utilities are much easier to use than git, and that git needs
improvement on this front.

Brendan

Re: obnoxious CLI complaints

[Todd Zullinger]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

Brendan Miller wrote:
> On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>> Brendan Miller <catphive@catphive.net> writes:
>> First question: which git version do you use?
>
> It sounds like a bunch of things have been fixed in yet to be released
> versions. That's great.

If I am not mistaken, support for cloning an empty repo was available
in 1.6.2, which was released in March.

-- 
Todd        OpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Drugs may lead to nowhere, but at least it's the scenic route.


[-- Attachment #2: Type: application/pgp-signature, Size: 542 bytes --]

Re: obnoxious CLI complaints

[Jakub Narebski]

On Thu, 10 Sep 2009, Brendan Miller wrote:
> On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:

>> Brendan Miller <catphive@catphive.net> writes:
>> First question: which git version do you use?
> 
> It sounds like a bunch of things have been fixed in yet to be released
> versions. That's great.

Both "git init --bare" and cloning empty repository are in _released_
versions.
 
>>> This is what I want to do 90% of the time, so it should just have the
>>> proper defaults, and not make me look at the man page every time I
>>> want to use it.
>>
>> You learn those idioms.
> 
> I guess. Is that a good thing? Is the goal of interface design to make
> it difficult so I need to learn a lot of things, or easy so I can
> remain blissfully ignorant but still do what I want?

There are at least two issues:
1. What you want to do 90% of time might be not what other people want
   to do 90% of time.
2. Some thing are due to git philosophy (like e.g. "git commit -a").
 
As to git-archive example: first, git-archive _has_ default format, and
it is tar.  Second, compression is better left to separate program, but
I guess we can follow GNU tar example and add equivalents of -Z/-z/-j
and --use-compress-program options when using --output=<file>.  Third,
when using e.g. tar you have to specify files to compress, so I don't
know why you complain that git-archive requires equivalent, HEAD in your
example.

>>> 6. Where is the bug tracker? If people users can't find the bug
>>> tracker, they can't report issues, and obnoxious bugs go unfixed, or
>>> people have to whine on the mailing list. There should be a nice big
>>> link on the front page of git-scm.com. A bug tracker is really the
>>> only way for the vast majority of a community that use a tool can give
>>> feedback on the problems the tool has.
>>
>> Do you offer to maintain and manage such bug tracker?  I mean here
>> taking care of duplicated bugs, tracking which bugs are resolved and
>> which are not, checking if bug is reproductible, etc.  Do you?
>> Unmaintained bugtracker is worse than useless.
>>
>> Using mailing list for bug reports and for patches is time-honored
>> workflow, which works rather well for smaller projects such as Git.
>> Note that git mailing list is free for all; you don't need to
>> subscribe to send, and you can watch it via many archives and gateways
>> (like GMane).
> 
> Bug trackers are a hassle, believe me, I know... but I think they
> contribute to the overall quality of the product if used effectively.
> Mailing lists seem like a good way to forget about bugs after people
> have given up on getting developers to fix them.

This is a good way to separate important from unimportant bug reports
and feature requests ;-)

>>> 7. Man pages: It's nice we have them, but we shouldn't need them to do
>>> basic stuff. I rarely had to look at the man pages using svn, but
>>> every single time I use git I have to dig into these things. Frankly,
>>> I have better things to do than RTFM.
>>
>> Learn.  If you learn the philosophy behind git design, you would have
>> much easier understanding and remembering git.
> 
> I think what you mean by philosophy is the underlying data structures,
> which are discussed in the manual (how many apps do you have that do
> that!). I have read that. However, that one needs to understand
> underlying data structure is just one more hurdle to understanding
> git.

No, I didn't mean understanding underlying data structures, but 
understanding philosophy: graph of revisions; tracking contents; refs
as pointers to graph of revisions; the trifecta of working area, 
the index and repository, etc.

> If I use GCC, do I need to know that it has a recursive descent
> parser? That it is implemented with a garbage collector? No. I just
> need to know that I give it C, and it gives me a binary.

Better example would be using "make".  You need to understand 'make'
philosophy to write all but most simple of Makefiles.

> 
> Example:
> gcc main.c
> 
> Think about all the defaults that are specified here! I don't
> explicitly tell it how to find libc.so or what path the dynamic linker
> is at. I don't even really need to tell it which operation it is
> performing, i.e. generating a binary, .o, .so, .os, .a, etc because it
> has a smart default.

And if you are smart, you never use this form, but "gcc -o main main.c".
And you have to specify '-lm' if you use math routines.  Not that simple,
isn't it?

> This an order of magnitude more complex than any git operation in
> terms of implementation, but it is dead simple from the users
> perspective.

When git (or the concept of DVCS) is as old as gcc (or C compiler) is
now, then we can talk.
 
>> There is "Git User's Manual", "The Git Community Book", "Pro Git" and
>> many other references.
> 
> Yeah, I've been reading them. I'm saying that the docs are a crutch.
> RTFM is the problem not the solution. It makes the user do more work
> to avoid fixing usability issues.

When the tool is more complicated (like DVCS), you can't use it in all
but simplest cases without understanding it.

> A CLI has some inherent limitations in that it doesn't have big
> labeled buttons to press. However, that doesn't mean it has to be hard
> to use. I think a lot of the strength of the linux CLI is that most of
> the utilities have actually pretty well thought out interfaces that
> have been refined over time. That one's that aren't like that... well,
> no one uses them.
> 
> I'm not saying that a unixy approach is wrong, but that most unix
> utilities are much easier to use than git, and that git needs
> improvement on this front.

I'm not saying that git doesn't need UI (and documentation) improvements.
But first, your attitude is a bit grating, and second, your examples
are not it.

On the other hand there is inherent problems that serious git 
contributors use git for a long time and are used to (and perhaps even
attached to) git UI warts, and newbies which start to use git not always
can distinguish between things that can be changed and things that cannot
be changed.

-- 
Jakub Narebski
Poland

Re: obnoxious CLI complaints

[Eric Schaefer]

2009/9/10 Jakub Narebski <jnareb@gmail.com>:
> This is a good way to separate important from unimportant bug reports
> and feature requests ;-)

"Unimportant bug reports"? Interesting concept... ;-)

BTW: A bug tracker has the advantage that bugs don't fall on the
floor. They can be postponed for later fixing but you will not forget
them.
But you are right about feature request. You would either have to have
a rigorous policy of dropping bogus or unwanted (unimportant) requests
or you go with the mailing list approach to keep the pile from
stinking. ;-)

Eric

Re: obnoxious CLI complaints

[Sverre Rabbelier]

Heya,

On Thu, Sep 10, 2009 at 20:18, Eric Schaefer
<eric.schaefer@ericschaefer.org> wrote:
> "Unimportant bug reports"? Interesting concept... ;-)

Sure, if there's a bug in feature foo, but it only happens when
invoking it with some rarely used argument, and only on Solaris
platforms, it is probably not worth spending time on it if the
original reporter does not even have the time to stick around and aid
in resolving the issue. It's probably better to spend that precious
time on other bug fixes, or features instead.

-- 
Cheers,

Sverre Rabbelier

Re: obnoxious CLI complaints

[René Scharfe]

Jakub Narebski schrieb:
> [...] Second, compression is better left to separate program, but
> I guess we can follow GNU tar example and add equivalents of -Z/-z/-j
> and --use-compress-program options when using --output=<file>. [...]

Compression only makes sense for the tar format, so I think it's better
exposed by new formats and not by generic options.

For compress and bzip2 we'd need to call the external archiver, similar
to a pager.  Interesting idea.

For gzip, we can use the zlib helper functions, since we're linking
against it anyway.  I mention this because the following patch has been
laying around here for a while, collecting dust because it was a feature
waiting for a requester.

Using zlib directly avoids the overhead of a pipe and of buffering the
output for blocked writes; surprisingly (to me), it isn't any faster.
I didn't make any tuning efforts, yet, though.  Anyway, here it is:

---
 Documentation/git-archive.txt |    2 +-
 archive-tar.c                 |   76 ++++++++++++++++++++++++++++++++++++----
 archive.c                     |   43 ++++++++++++++++++++++-
 archive.h                     |    1 +
 t/t5000-tar-tree.sh           |   30 ++++++++++++++++
 5 files changed, 141 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
index 92444dd..2935246 100644
--- a/Documentation/git-archive.txt
+++ b/Documentation/git-archive.txt
@@ -34,7 +34,7 @@ OPTIONS
 -------
 
 --format=<fmt>::
-	Format of the resulting archive: 'tar' or 'zip'.  The default
+	Format of the resulting archive: 'tar', 'tar.gz' or 'zip'.  The default
 	is 'tar'.
 
 -l::
diff --git a/archive-tar.c b/archive-tar.c
index cee06ce..f22304d 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -58,18 +58,78 @@ static void write_blocked(const void *data, unsigned long size)
 	write_if_needed();
 }
 
+static void gzwrite_or_die(gzFile *gzfile, const void *buf, size_t count)
+{
+	const int chunk = 1 << 30; /* Big arbitrary value that fits into int. */
+	const char *p = buf;
+
+	while (count > 0) {
+		unsigned int to_write = (count < chunk) ? count : chunk;
+		int written = gzwrite(gzfile, p, to_write);
+		if (written <= 0) {
+			int err;
+			const char *msg = gzerror(gzfile, &err);
+			if (err != Z_ERRNO)
+				die("zlib error: %s", msg);
+			if (errno == EAGAIN || errno == EINTR)
+				continue;
+			if (errno == EPIPE)
+				exit(0);
+			die_errno("write error");
+		}
+		count -= written;
+		p += written;
+	}
+}
+
+/*
+ * Writes directly through zlib and pads with NUL bytes to multiples of
+ * RECORDSIZE.  Updates offset because the length of the trailer depends
+ * on it.
+ */
+static void write_to_tgz(struct archiver_args *args, const void *data,
+			 unsigned long size)
+{
+	unsigned long tail = size % RECORDSIZE;
+	gzwrite_or_die(args->gzfile, data, size);
+	if (tail) {
+		tail = RECORDSIZE - tail;
+		if (gzseek(args->gzfile, tail, SEEK_CUR) == -1)
+			die("zlib error while seeking.");
+	}
+	offset = (offset + size + tail) % BLOCKSIZE;
+}
+
+static void write_to_archive(struct archiver_args *args, const void *data,
+			     unsigned long size)
+{
+	if (args->gzfile)
+		write_to_tgz(args, data, size);
+	else
+		write_blocked(data, size);
+}
+
 /*
  * The end of tar archives is marked by 2*512 nul bytes and after that
  * follows the rest of the block (if any).
  */
-static void write_trailer(void)
+static void write_trailer(struct archiver_args *args)
 {
 	int tail = BLOCKSIZE - offset;
-	memset(block + offset, 0, tail);
-	write_or_die(1, block, BLOCKSIZE);
-	if (tail < 2 * RECORDSIZE) {
-		memset(block, 0, offset);
+	if (args->gzfile) {
+		if (tail < 2 * RECORDSIZE)
+			tail += BLOCKSIZE;
+		if (gzseek(args->gzfile, tail - 1, SEEK_CUR) == -1)
+			die("zlib error while seeking.");
+		if (gzputc(args->gzfile, '\0') == -1)
+			die("zlib error while writing a NUL byte.");
+	} else {
+		memset(block + offset, 0, tail);
 		write_or_die(1, block, BLOCKSIZE);
+		if (tail < 2 * RECORDSIZE) {
+			memset(block, 0, offset);
+			write_or_die(1, block, BLOCKSIZE);
+		}
 	}
 }
 
@@ -201,9 +261,9 @@ static int write_tar_entry(struct archiver_args *args,
 			return err;
 	}
 	strbuf_release(&ext_header);
-	write_blocked(&header, sizeof(header));
+	write_to_archive(args, &header, sizeof(header));
 	if (S_ISREG(mode) && buffer && size > 0)
-		write_blocked(buffer, size);
+		write_to_archive(args, buffer, size);
 	return err;
 }
 
@@ -245,6 +305,6 @@ int write_tar_archive(struct archiver_args *args)
 	if (!err)
 		err = write_archive_entries(args, write_tar_entry);
 	if (!err)
-		write_trailer();
+		write_trailer(args);
 	return err;
 }
diff --git a/archive.c b/archive.c
index 0bca9ca..8809f51 100644
--- a/archive.c
+++ b/archive.c
@@ -15,6 +15,8 @@ static char const * const archive_usage[] = {
 };
 
 #define USES_ZLIB_COMPRESSION 1
+#define USES_GZIP_COMPRESSION 2
+#define USES_COMPRESSION (USES_ZLIB_COMPRESSION | USES_GZIP_COMPRESSION)
 
 static const struct archiver {
 	const char *name;
@@ -22,6 +24,7 @@ static const struct archiver {
 	unsigned int flags;
 } archivers[] = {
 	{ "tar", write_tar_archive },
+	{ "tar.gz", write_tar_archive, USES_GZIP_COMPRESSION },
 	{ "zip", write_zip_archive, USES_ZLIB_COMPRESSION },
 };
 
@@ -336,7 +339,7 @@ static int parse_archive_args(int argc, const char **argv,
 
 	args->compression_level = Z_DEFAULT_COMPRESSION;
 	if (compression_level != -1) {
-		if ((*ar)->flags & USES_ZLIB_COMPRESSION)
+		if ((*ar)->flags & USES_COMPRESSION)
 			args->compression_level = compression_level;
 		else {
 			die("Argument not supported for format '%s': -%d",
@@ -351,11 +354,38 @@ static int parse_archive_args(int argc, const char **argv,
 	return argc;
 }
 
+static void archive_gzfile_open(struct archiver_args *args)
+{
+	char mode[] = "wbX";
+	if (args->compression_level == Z_DEFAULT_COMPRESSION)
+		mode[2] = '\0';
+	else
+		mode[2] = '0' + args->compression_level;
+	args->gzfile = gzdopen(xdup(1), mode);
+	if (!args->gzfile)
+		die("zlib error: out of memory.");
+}
+
+static void archive_gzfile_close(struct archiver_args *args)
+{
+	int err = gzclose(args->gzfile);
+	switch (err) {
+	case Z_OK:
+		break;
+	case Z_ERRNO:
+		die_errno("zlib error");
+	default:
+		die("zlib error %d while closing.", err);
+	}
+	args->gzfile = NULL;
+}
+
 int write_archive(int argc, const char **argv, const char *prefix,
 		int setup_prefix)
 {
 	const struct archiver *ar = NULL;
 	struct archiver_args args;
+	int err;
 
 	argc = parse_archive_args(argc, argv, &ar, &args);
 	if (setup_prefix && prefix == NULL)
@@ -366,5 +396,14 @@ int write_archive(int argc, const char **argv, const char *prefix,
 
 	git_config(git_default_config, NULL);
 
-	return ar->write_archive(&args);
+	args.gzfile = NULL;
+	if (ar->flags & USES_GZIP_COMPRESSION)
+		archive_gzfile_open(&args);
+
+	err = ar->write_archive(&args);
+
+	if (!err && (ar->flags & USES_GZIP_COMPRESSION))
+		archive_gzfile_close(&args);
+
+	return err;
 }
diff --git a/archive.h b/archive.h
index 038ac35..638a7ba 100644
--- a/archive.h
+++ b/archive.h
@@ -12,6 +12,7 @@ struct archiver_args {
 	unsigned int verbose : 1;
 	unsigned int worktree_attributes : 1;
 	int compression_level;
+	gzFile gzfile;
 };
 
 typedef int (*write_archive_fn_t)(struct archiver_args *);
diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh
index 5f84b18..4094c18 100755
--- a/t/t5000-tar-tree.sh
+++ b/t/t5000-tar-tree.sh
@@ -26,6 +26,8 @@ commit id embedding:
 
 . ./test-lib.sh
 UNZIP=${UNZIP:-unzip}
+GZIP=${GZIP:-gzip}
+GUNZIP=${GUNZIP:-$GZIP -d}
 
 SUBSTFORMAT=%H%n
 
@@ -79,6 +81,15 @@ test_expect_success \
     'git tar-tree HEAD >b2.tar'
 
 test_expect_success \
+    'git archive --format=tar.gz' \
+    'git archive --format=tar.gz HEAD >bz.tar.gz'
+
+test_expect_success \
+    'git archive --format=tar.gz with --output' \
+    'git archive --format=tar.gz --output=bz1.tar.gz HEAD &&
+     test_cmp bz.tar.gz bz1.tar.gz'
+
+test_expect_success \
     'git archive vs. git tar-tree' \
     'test_cmp b.tar b2.tar'
 
@@ -146,7 +157,9 @@ test_expect_success \
     'cp .git/info/attributes .git/info/attributes.before &&
      echo "substfile?" export-subst >>.git/info/attributes &&
      git archive HEAD >f.tar &&
+     git archive --format=tar.gz HEAD >fz.tar.gz &&
      git archive --prefix=prefix/ HEAD >g.tar &&
+     git archive --format=tar.gz --prefix=prefix/ HEAD >gz.tar.gz &&
      mv .git/info/attributes.before .git/info/attributes'
 
 test_expect_success \
@@ -173,6 +186,23 @@ test_expect_success \
       test_cmp a/substfile2 g/prefix/a/substfile2
 '
 
+$GUNZIP -h >/dev/null 2>&1
+if [ $? -eq 127 ]; then
+	echo "Skipping tar.gz expansion tests, because gunzip was not found"
+else
+	test_expect_success \
+		'expand *.tar.gz' \
+		'$GUNZIP bz.tar.gz &&
+		 $GUNZIP fz.tar.gz &&
+		 $GUNZIP gz.tar.gz'
+
+	test_expect_success \
+		'compare files created by formats tar and tar.gz' \
+		'test_cmp b.tar bz.tar &&
+		 test_cmp f.tar fz.tar &&
+		 test_cmp g.tar gz.tar'
+fi
+
 test_expect_success \
     'git archive --format=zip' \
     'git archive --format=zip HEAD >d.zip'
-- 
1.6.5.rc0

Re: obnoxious CLI complaints

[Linus Torvalds]

On Fri, 11 Sep 2009, René Scharfe wrote:
> 
> Using zlib directly avoids the overhead of a pipe and of buffering the
> output for blocked writes; surprisingly (to me), it isn't any faster.

In fact, it should be slower.

On SMP, you're quite likely better off using the pipe, and compressing on 
another CPU. Of course, it's usually the case that the compression is _so_ 
much slower than generating the tar-file (especially for the hot-cache 
case) that it doesn't matter or the pipe overhead is even a slowdown.

But especially if generating the tar-file has some delays in it 
(cold-cache object lookup, whatever), the "compress in separate process" 
is likely simply better, because you can compress while the other process 
is looking up data for the tar.

				Linus

Re: obnoxious CLI complaints

[René Scharfe]

Am 11.09.2009 16:47, schrieb Linus Torvalds:
>
>
> On Fri, 11 Sep 2009, René Scharfe wrote:
>>
>> Using zlib directly avoids the overhead of a pipe and of buffering the
>> output for blocked writes; surprisingly (to me), it isn't any faster.
>
> In fact, it should be slower.
>
> On SMP, you're quite likely better off using the pipe, and compressing on
> another CPU. Of course, it's usually the case that the compression is _so_
> much slower than generating the tar-file (especially for the hot-cache
> case) that it doesn't matter or the pipe overhead is even a slowdown.
>
> But especially if generating the tar-file has some delays in it
> (cold-cache object lookup, whatever), the "compress in separate process"
> is likely simply better, because you can compress while the other process
> is looking up data for the tar.

Yes, that makes sense and can be seen here (quad core, Fedora 11, best
of five consecutive runs, Linux kernel repo):

	# git v1.6.5-rc0
	$ time git archive --format=tar v2.6.31 | gzip -6 >/dev/null

	real	0m16.591s
	user	0m19.769s
	sys	0m0.474s

	# git v1.6.5-rc0 + patch
	$ time ../git/git archive --format=tar.gz -6 v2.6.31 >/dev/null

	real	0m20.390s
	user	0m20.299s
	sys	0m0.088s

User time is quite similar, real time is lower when using a pipe.

But what has bugged me since I added zip support is this result:

	# git v1.6.5-rc0
	$ time git archive --format=zip -6 v2.6.31 >/dev/null

	real	0m16.471s
	user	0m16.340s
	sys	0m0.128s

I'd have expected this to be the slowest case, because it's compressing
all files separately, i.e. it needs to create and flush the compression
context lots of times instead of only once as in the two cases above.
And it's sequential and uses zlib, just like the tar.gz format.  I
suspect the convenience function gzwrite() adds this overhead.


Oh, I just discovered pigz (http://zlib.net/pigz/), a parallel gzip:

	# git v1.6.5-rc0, pigz 2.1.5
	$ time git archive --format=tar v2.6.31 | pigz -6 >/dev/null

	real	0m6.251s
	user	0m21.383s
	sys	0m0.547s

So pipes win. :)  Still need to investigate why zip is as (relatively)
fast as it is, though.

René

Re: obnoxious CLI complaints

[Linus Torvalds]

On Sat, 12 Sep 2009, René Scharfe wrote:
> 
> But what has bugged me since I added zip support is this result:
> 
> 	# git v1.6.5-rc0
> 	$ time git archive --format=zip -6 v2.6.31 >/dev/null
> 
> 	real	0m16.471s
> 	user	0m16.340s
> 	sys	0m0.128s
> 
> I'd have expected this to be the slowest case, because it's compressing
> all files separately, i.e. it needs to create and flush the compression
> context lots of times instead of only once as in the two cases above.

Oh no, I think it's easily explained.

Compressing many small files really is often cheaper than compressing one 
large one.

With lots of small files, you end up being very limited in the 
search-space, so the compression decisions get simpler. Compression in 
general is not O(n), it's some non-linear factor, often something like 
O(n**2).

Of course, all compression libraries have an upper bound on the 
non-linearity (often expressed as a "window size"), so a particular 
compression algorithm may end up being close to O(n) (with a huge 
constant). But that upper bound will only kick in for large files, small 
files that fit entirely into the compression window will still see the 
underlying O(n**2) or whatever.

But I have no actual numbers to back up the above blathering. But feel 
free to try to compress 10 small files and compare it to compressing one 
file that is as big as the sum. I bet you'll see it.

			Linus

Re: obnoxious CLI complaints

[Dmitry Potapov]

On Wed, Sep 09, 2009 at 05:09:31PM -0700, Brendan Miller wrote:
> On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> > Brendan Miller <catphive@catphive.net> writes:
> >>
> >> This is what I want to do 90% of the time, so it should just have the
> >> proper defaults, and not make me look at the man page every time I
> >> want to use it.
> >
> > You learn those idioms.
> 
> I guess. Is that a good thing?

In general, yes, because most of them exist for a good reason.

> Is the goal of interface design to make
> it difficult so I need to learn a lot of things, or easy so I can
> remain blissfully ignorant but still do what I want?

Neither. You cannot get what unless you have specified what you want,
and for that you have to learn how to say that. Having good defaults is
very important, but the problem with choosing them is that people have
different preferences about them. For instance, you wanted the default
prefix for git-archive to be $myproject. For me, a good default would be
either $tag_name, or $myproject-$tag_name, or empty (as it is now!). So,
what you propose is *never* a good default for me. Moreover, changing
any default will cause a lot of pain for other people who use Git now.
Besides, writing something like --prefix='' is very ugly. So, the
current default makes perfect sense.

> >>
> >> 7. Man pages: It's nice we have them, but we shouldn't need them to do
> >> basic stuff. I rarely had to look at the man pages using svn, but
> >> every single time I use git I have to dig into these things. Frankly,
> >> I have better things to do than RTFM.
> >
> > Learn.  If you learn the philosophy behind git design, you would have
> > much easier understanding and remembering git.
> 
> I think what you mean by philosophy is the underlying data structures,
> which are discussed in the manual 

I think I have read them a lot, but I do not remember any underlying
data structures described in them. What are you reading?

> If I use GCC, do I need to know that it has a recursive descent
> parser? That it is implemented with a garbage collector? No. I just
> need to know that I give it C, and it gives me a binary.
> 
> Example:
> gcc main.c

The fallacy in your logic is that you compare two completely different
things thinking about them as almost identical. In case of GCC, the file
contains a program written in the C language. If you do not learn the C
language, you will not be able to use GCC. On the other hand, for Git
any file is just bytes, but the command-line interface describes what
you want to do with them. There is no single action that would make
sense in all cases, you have to specify what you want. Even with simple
tools like sed or awk, you have to learn something before you can use
them.

> >
> > There is "Git User's Manual", "The Git Community Book", "Pro Git" and
> > many other references.
> 
> Yeah, I've been reading them. I'm saying that the docs are a crutch.
> RTFM is the problem not the solution. It makes the user do more work
> to avoid fixing usability issues.

A usability issue exists when a person knows how to do that, but it is
inconvenient or error-prone; or when a learning curve is too steep.
But when someone cannot use, let's say, a compiler, because he or she
refuses to read to learn the language, it is not a usability issue.


Dmitry

Re: obnoxious CLI complaints

[John Tapsell]

2009/9/12 Dmitry Potapov <dpotapov@gmail.com>:
> On Wed, Sep 09, 2009 at 05:09:31PM -0700, Brendan Miller wrote:
>> On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>> > Brendan Miller <catphive@catphive.net> writes:
>> >>
>> >> This is what I want to do 90% of the time, so it should just have the
>> >> proper defaults, and not make me look at the man page every time I
>> >> want to use it.
>> >
>> > You learn those idioms.
>>
>> I guess. Is that a good thing?
>
> In general, yes, because most of them exist for a good reason.
>
>> Is the goal of interface design to make
>> it difficult so I need to learn a lot of things, or easy so I can
>> remain blissfully ignorant but still do what I want?
>
> Neither. You cannot get what unless you have specified what you want,
> and for that you have to learn how to say that. Having good defaults is
> very important, but the problem with choosing them is that people have
> different preferences about them. For instance, you wanted the default
> prefix for git-archive to be $myproject. For me, a good default would be
> either $tag_name, or $myproject-$tag_name, or empty (as it is now!). So,
> what you propose is *never* a good default for me. Moreover, changing
> any default will cause a lot of pain for other people who use Git now.
> Besides, writing something like --prefix='' is very ugly. So, the
> current default makes perfect sense.

Ah, great logic.  You can't find a default that will suit everyone,
therefore don't bother.

>> Yeah, I've been reading them. I'm saying that the docs are a crutch.
>> RTFM is the problem not the solution. It makes the user do more work
>> to avoid fixing usability issues.
>
> A usability issue exists when a person knows how to do that, but it is
> inconvenient or error-prone; or when a learning curve is too steep.
> But when someone cannot use, let's say, a compiler, because he or she
> refuses to read to learn the language, it is not a usability issue.

It's a usability issue when it doesn't just do the right thing in the
majority of cases and lets you specify what you want it to do in the
rest of the cases.

John

Re: obnoxious CLI complaints

[Dmitry Potapov]

On Sat, Sep 12, 2009 at 09:32:09PM +0300, John Tapsell wrote:
> 2009/9/12 Dmitry Potapov <dpotapov@gmail.com>:
> > On Wed, Sep 09, 2009 at 05:09:31PM -0700, Brendan Miller wrote:
> >> On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> >> > Brendan Miller <catphive@catphive.net> writes:
> >> >>
> >> Is the goal of interface design to make
> >> it difficult so I need to learn a lot of things, or easy so I can
> >> remain blissfully ignorant but still do what I want?
> >
> > Neither. You cannot get what unless you have specified what you want,
> > and for that you have to learn how to say that. Having good defaults is
> > very important, but the problem with choosing them is that people have
> > different preferences about them. For instance, you wanted the default
> > prefix for git-archive to be $myproject. For me, a good default would be
> > either $tag_name, or $myproject-$tag_name, or empty (as it is now!). So,
> > what you propose is *never* a good default for me. Moreover, changing
> > any default will cause a lot of pain for other people who use Git now.
> > Besides, writing something like --prefix='' is very ugly. So, the
> > current default makes perfect sense.
> 
> Ah, great logic.  You can't find a default that will suit everyone,
> therefore don't bother.

I did not say "don't bother". On contrary, I said that defaults are very
important, but, in this case, the current default makes far more sense
that what was proposed by Brendan.

> 
> >> Yeah, I've been reading them. I'm saying that the docs are a crutch.
> >> RTFM is the problem not the solution. It makes the user do more work
> >> to avoid fixing usability issues.
> >
> > A usability issue exists when a person knows how to do that, but it is
> > inconvenient or error-prone; or when a learning curve is too steep.
> > But when someone cannot use, let's say, a compiler, because he or she
> > refuses to read to learn the language, it is not a usability issue.
> 
> It's a usability issue when it doesn't just do the right thing in the
> majority of cases and lets you specify what you want it to do in the
> rest of the cases.

It does the right thing for me, and not just in most cases, it does so
in _all_ cases, because it does exactly it is told to do. And it is a
very important characteristics for any VCS, otherwise you can mess up
things easily. What is also good about Git is that it does not require
much keystrokes to do even rather complex stuff. And many defaults and
commands are configurable, so you can adjust it to your workflow. So,
I am not sure what your problem is.


Dmitry

Re: obnoxious CLI complaints

[John Tapsell]

2009/9/13 Dmitry Potapov <dpotapov@gmail.com>:
> On Sat, Sep 12, 2009 at 09:32:09PM +0300, John Tapsell wrote:
>> 2009/9/12 Dmitry Potapov <dpotapov@gmail.com>:
>> > On Wed, Sep 09, 2009 at 05:09:31PM -0700, Brendan Miller wrote:
>> >> On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>> >> > Brendan Miller <catphive@catphive.net> writes:
>> >> >>
>> >> Is the goal of interface design to make
>> >> it difficult so I need to learn a lot of things, or easy so I can
>> >> remain blissfully ignorant but still do what I want?
>> >
>> > Neither. You cannot get what unless you have specified what you want,
>> > and for that you have to learn how to say that. Having good defaults is
>> > very important, but the problem with choosing them is that people have
>> > different preferences about them. For instance, you wanted the default
>> > prefix for git-archive to be $myproject. For me, a good default would be
>> > either $tag_name, or $myproject-$tag_name, or empty (as it is now!). So,
>> > what you propose is *never* a good default for me. Moreover, changing
>> > any default will cause a lot of pain for other people who use Git now.
>> > Besides, writing something like --prefix='' is very ugly. So, the
>> > current default makes perfect sense.
>>
>> Ah, great logic.  You can't find a default that will suit everyone,
>> therefore don't bother.
>
> I did not say "don't bother". On contrary, I said that defaults are very
> important, but, in this case, the current default makes far more sense
> that what was proposed by Brendan.
>
>>
>> >> Yeah, I've been reading them. I'm saying that the docs are a crutch.
>> >> RTFM is the problem not the solution. It makes the user do more work
>> >> to avoid fixing usability issues.
>> >
>> > A usability issue exists when a person knows how to do that, but it is
>> > inconvenient or error-prone; or when a learning curve is too steep.
>> > But when someone cannot use, let's say, a compiler, because he or she
>> > refuses to read to learn the language, it is not a usability issue.
>>
>> It's a usability issue when it doesn't just do the right thing in the
>> majority of cases and lets you specify what you want it to do in the
>> rest of the cases.
>
> It does the right thing for me, and not just in most cases, it does so
> in _all_ cases, because it does exactly it is told to do. And it is a
> very important characteristics for any VCS, otherwise you can mess up
> things easily. What is also good about Git is that it does not require
> much keystrokes to do even rather complex stuff. And many defaults and
> commands are configurable, so you can adjust it to your workflow. So,
> I am not sure what your problem is.

Because I wouldn't call this just a few keystrokes to do the common case:

    git archive --format=tar --prefix=HEAD/ HEAD | gzip > head.tar.gz

I honestly don't understand the backlash against Brenden's point that
this could be made a bit simpler.

John

Re: obnoxious CLI complaints

[A Large Angry SCM]

John Tapsell wrote:
> 2009/9/13 Dmitry Potapov <dpotapov@gmail.com>:
>> On Sat, Sep 12, 2009 at 09:32:09PM +0300, John Tapsell wrote:
>>> 2009/9/12 Dmitry Potapov <dpotapov@gmail.com>:
>>>> On Wed, Sep 09, 2009 at 05:09:31PM -0700, Brendan Miller wrote:
>>>>> On Wed, Sep 9, 2009 at 2:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>>>>>> Brendan Miller <catphive@catphive.net> writes:
>>>>> Is the goal of interface design to make
>>>>> it difficult so I need to learn a lot of things, or easy so I can
>>>>> remain blissfully ignorant but still do what I want?
>>>> Neither. You cannot get what unless you have specified what you want,
>>>> and for that you have to learn how to say that. Having good defaults is
>>>> very important, but the problem with choosing them is that people have
>>>> different preferences about them. For instance, you wanted the default
>>>> prefix for git-archive to be $myproject. For me, a good default would be
>>>> either $tag_name, or $myproject-$tag_name, or empty (as it is now!). So,
>>>> what you propose is *never* a good default for me. Moreover, changing
>>>> any default will cause a lot of pain for other people who use Git now.
>>>> Besides, writing something like --prefix='' is very ugly. So, the
>>>> current default makes perfect sense.
>>> Ah, great logic.  You can't find a default that will suit everyone,
>>> therefore don't bother.
>> I did not say "don't bother". On contrary, I said that defaults are very
>> important, but, in this case, the current default makes far more sense
>> that what was proposed by Brendan.
>>
>>>>> Yeah, I've been reading them. I'm saying that the docs are a crutch.
>>>>> RTFM is the problem not the solution. It makes the user do more work
>>>>> to avoid fixing usability issues.
>>>> A usability issue exists when a person knows how to do that, but it is
>>>> inconvenient or error-prone; or when a learning curve is too steep.
>>>> But when someone cannot use, let's say, a compiler, because he or she
>>>> refuses to read to learn the language, it is not a usability issue.
>>> It's a usability issue when it doesn't just do the right thing in the
>>> majority of cases and lets you specify what you want it to do in the
>>> rest of the cases.
>> It does the right thing for me, and not just in most cases, it does so
>> in _all_ cases, because it does exactly it is told to do. And it is a
>> very important characteristics for any VCS, otherwise you can mess up
>> things easily. What is also good about Git is that it does not require
>> much keystrokes to do even rather complex stuff. And many defaults and
>> commands are configurable, so you can adjust it to your workflow. So,
>> I am not sure what your problem is.
> 
> Because I wouldn't call this just a few keystrokes to do the common case:
> 
>     git archive --format=tar --prefix=HEAD/ HEAD | gzip > head.tar.gz
> 
> I honestly don't understand the backlash against Brenden's point that
> this could be made a bit simpler.

Be made simpler for whom? The first rule of defaults is that they are 
never correct.

And, to every one, if you don;t think like the way <SOME_PROGRAM> has 
it's defaults set and the developer(s) don't agree to change the default 
for _everyone_ to what _you_ like, you can still use your shell's alias 
facility to fix the situation for your own use case.

To channel Junio, why are we still having this discussion?

Re: obnoxious CLI complaints

[Dmitry Potapov]

On Sun, Sep 13, 2009 at 01:21:43AM +0300, John Tapsell wrote:
> 
> Because I wouldn't call this just a few keystrokes to do the common case:
> 
>     git archive --format=tar --prefix=HEAD/ HEAD | gzip > head.tar.gz
> 
> I honestly don't understand the backlash against Brenden's point that
> this could be made a bit simpler.

You do not have to specify '--format=tar', because it is default. The
prefix name is a matter of one's preferences. Brenden wanted it to be
$myproject, while I have used three different versions. Now, you suggest
some other. IMHO, having it empty by default makes much more sense when
there is no obvious value on what most would agree. Finally, 'HEAD' is
required, because we do not want 'git archive' being run without any
parameter to write a binary file to the terminal. (Yes, it is foolish to
run command that you do not know to see what it does, but some people do
that, and we want all commands to be safe). BTW, I wonder whether use of
HEAD is really common with git-archive. Normally, you would archive a
tagged release, and then it is better to use the tag name to be sure
that you have archived the right thing.


Dmitry

Re: obnoxious CLI complaints

[John Tapsell]

2009/9/13 Dmitry Potapov <dpotapov@gmail.com>:
> On Sun, Sep 13, 2009 at 01:21:43AM +0300, John Tapsell wrote:
>>
>> Because I wouldn't call this just a few keystrokes to do the common case:
>>
>>     git archive --format=tar --prefix=HEAD/ HEAD | gzip > head.tar.gz
>>
>> I honestly don't understand the backlash against Brenden's point that
>> this could be made a bit simpler.
>
> You do not have to specify '--format=tar', because it is default. The
> prefix name is a matter of one's preferences. Brenden wanted it to be
> $myproject, while I have used three different versions. Now, you suggest
> some other. IMHO, having it empty by default makes much more sense when
> there is no obvious value on what most would agree. Finally, 'HEAD' is
> required, because we do not want 'git archive' being run without any
> parameter to write a binary file to the terminal. (Yes, it is foolish to
> run command that you do not know to see what it does, but some people do
> that, and we want all commands to be safe). BTW, I wonder whether use of
> HEAD is really common with git-archive. Normally, you would archive a
> tagged release, and then it is better to use the tag name to be sure
> that you have archived the right thing.

Ah, the manpage examples specifically give the --format=tar though.

Why not have  --format=tgz  then or something?  Or better yet, give
the filename on the command line and detect the format from the file
extension.

Re: obnoxious CLI complaints

[Junio C Hamano]

John Tapsell <johnflux@gmail.com> writes:

> Ah, the manpage examples specifically give the --format=tar though.

So what?

> Why not have  --format=tgz  then or something?  Or better yet, give
> the filename on the command line and detect the format from the file
> extension.

That is an interesting enhancement and sounds like a useful feature.

[PATCH 1/2] git-archive: add '-o' as a alias for '--output'

[Dmitry Potapov]

The '-o' option is commonly used in many tools to specify the output file.
Typing '--output' every time is a bit too long to be a practical alternative
to redirecting output. But specifying the output name has the advantage of
making possible to guess the desired output format by filename extension.

Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
---

PS I resend this patch because I forgot to include the git mailing list when
I sent it before. Sorry for inconvinience...

 Documentation/git-archive.txt |    3 ++-
 archive.c                     |    2 +-
 builtin-archive.c             |    2 +-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
index 92444dd..f7a3b95 100644
--- a/Documentation/git-archive.txt
+++ b/Documentation/git-archive.txt
@@ -10,7 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git archive' [--format=<fmt>] [--list] [--prefix=<prefix>/] [<extra>]
-	      [--output=<file>] [--worktree-attributes]
+	      [-o | --output=<file>] [--worktree-attributes]
 	      [--remote=<repo> [--exec=<git-upload-archive>]] <tree-ish>
 	      [path...]
 
@@ -48,6 +48,7 @@ OPTIONS
 --prefix=<prefix>/::
 	Prepend <prefix>/ to each filename in the archive.
 
+-o::
 --output=<file>::
 	Write the archive to <file> instead of stdout.
 
diff --git a/archive.c b/archive.c
index 0bca9ca..73b8e8a 100644
--- a/archive.c
+++ b/archive.c
@@ -283,7 +283,7 @@ static int parse_archive_args(int argc, const char **argv,
 		OPT_STRING(0, "format", &format, "fmt", "archive format"),
 		OPT_STRING(0, "prefix", &base, "prefix",
 			"prepend prefix to each pathname in the archive"),
-		OPT_STRING(0, "output", &output, "file",
+		OPT_STRING('o', "output", &output, "file",
 			"write the archive to this file"),
 		OPT_BOOLEAN(0, "worktree-attributes", &worktree_attributes,
 			"read .gitattributes in working directory"),
diff --git a/builtin-archive.c b/builtin-archive.c
index f9a4bea..565314b 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -71,7 +71,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
 	const char *output = NULL;
 	const char *remote = NULL;
 	struct option local_opts[] = {
-		OPT_STRING(0, "output", &output, "file",
+		OPT_STRING('o', "output", &output, "file",
 			"write the archive to this file"),
 		OPT_STRING(0, "remote", &remote, "repo",
 			"retrieve the archive from remote repository <repo>"),
-- 
1.6.4

[PATCH 2/2] teach git-archive to auto detect the output format

[Dmitry Potapov]

When I type something like this:
  git archive -o my-v2.0.zip v2.0
it is almost certainly that I want to create a zip archive, and not
a tar file.

This patch teaches git-archive to auto detect the output format from the
file name. Currently, only '.zip' is supported. If the auto detect failed,
the tar format is used as default. The auto detect is not used when the
output format is specified explicitly.

Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
---

On Sat, Sep 12, 2009 at 07:47:21PM -0700, Junio C Hamano wrote:
> John Tapsell <johnflux@gmail.com> writes:
>_
> > Why not have  --format=tgz  then or something?  Or better yet, give
> > the filename on the command line and detect the format from the file
> > extension.
>_
> That is an interesting enhancement and sounds like a useful feature.

Here is my first attempt to implement that. I have not added 'tgz' yet,
but only auto detect the format from the output file name.

PS I resend this patch because I forgot to include the git mailing list when
I sent it before. Sorry for inconvinience...

 Documentation/git-archive.txt |   10 +++++++++-
 builtin-archive.c             |   25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
index f7a3b95..c6fb21c 100644
--- a/Documentation/git-archive.txt
+++ b/Documentation/git-archive.txt
@@ -35,7 +35,9 @@ OPTIONS
 
 --format=<fmt>::
 	Format of the resulting archive: 'tar' or 'zip'.  The default
-	is 'tar'.
+	is 'tar', unless the output file is specified, and it has a known
+	extension (such as '.zip') then the default for the output format
+	will be determined by this extension.
 
 -l::
 --list::
@@ -130,6 +132,12 @@ git archive --format=zip --prefix=git-docs/ HEAD:Documentation/ > git-1.4.0-docs
 	Put everything in the current head's Documentation/ directory
 	into 'git-1.4.0-docs.zip', with the prefix 'git-docs/'.
 
+git archive -o latest.zip HEAD::
+
+	Create a Zip archive that contains the contents of the latest
+	commit on the current branch. Note that the output format is
+	specified implicitly by the extension of the output file.
+
 
 SEE ALSO
 --------
diff --git a/builtin-archive.c b/builtin-archive.c
index 565314b..878c6b2 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -60,6 +60,17 @@ static int run_remote_archiver(int argc, const char **argv,
 	return !!rv;
 }
 
+static const char* format_from_name(const char *filename)
+{
+	const char *ext = strrchr(filename, '.');
+	if (!ext)
+		return NULL;
+	ext++;
+	if (!strcasecmp(ext, "zip"))
+		return "zip";
+	return NULL;
+}
+
 #define PARSE_OPT_KEEP_ALL ( PARSE_OPT_KEEP_DASHDASH | 	\
 			     PARSE_OPT_KEEP_ARGV0 | 	\
 			     PARSE_OPT_KEEP_UNKNOWN |	\
@@ -70,6 +81,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
 	const char *exec = "git-upload-archive";
 	const char *output = NULL;
 	const char *remote = NULL;
+	const char *format = NULL;
 	struct option local_opts[] = {
 		OPT_STRING('o', "output", &output, "file",
 			"write the archive to this file"),
@@ -77,14 +89,27 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
 			"retrieve the archive from remote repository <repo>"),
 		OPT_STRING(0, "exec", &exec, "cmd",
 			"path to the remote git-upload-archive command"),
+		OPT_STRING(0, "format", &format, "fmt", "archive format"),
 		OPT_END()
 	};
+	char fmt_opt[32];
 
 	argc = parse_options(argc, argv, prefix, local_opts, NULL,
 			     PARSE_OPT_KEEP_ALL);
 
 	if (output)
+	{
 		create_output_file(output);
+		if (!format)
+			format = format_from_name(output);
+	}
+
+	if (format)
+	{
+		sprintf(fmt_opt, "--format=%s", format);
+		argv[argc++] = fmt_opt;
+		argv[argc] = NULL;
+	}
 
 	if (remote)
 		return run_remote_archiver(argc, argv, remote, exec);
-- 
1.6.4

Re: [PATCH 2/2] teach git-archive to auto detect the output format

[Junio C Hamano]

Dmitry Potapov <dpotapov@gmail.com> writes:

> diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
> index f7a3b95..c6fb21c 100644
> --- a/Documentation/git-archive.txt
> +++ b/Documentation/git-archive.txt
> @@ -35,7 +35,9 @@ OPTIONS
>  
>  --format=<fmt>::
>  	Format of the resulting archive: 'tar' or 'zip'.  The default
> -	is 'tar'.
> +	is 'tar', unless the output file is specified, and it has a known
> +	extension (such as '.zip') then the default for the output format
> +	will be determined by this extension.

Once it is _determined_, then it is not the default anymore.

	If this option is not given, and the output file is specified, the
	format is inferred from the filename if possible (e.g. writing to
	"foo.zip" makes the output to be in the zip format).  Otherwise
	the output format is `tar`.

> @@ -130,6 +132,12 @@ git archive --format=zip
>  	Put everything in the current head's Documentation/ directory
>  	into 'git-1.4.0-docs.zip', with the prefix 'git-docs/'.
>  
> +git archive -o latest.zip HEAD::
> +
> +	Create a Zip archive that contains the contents of the latest
> +	commit on the current branch. Note that the output format is
> +	specified implicitly by the extension of the output file.
> +

Perhaps "s/specified implicitly/inferred/" but that is a minor point.

> diff --git a/builtin-archive.c b/builtin-archive.c
> index 565314b..878c6b2 100644
> --- a/builtin-archive.c
> +++ b/builtin-archive.c
> @@ -77,14 +89,27 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
>  			"retrieve the archive from remote repository <repo>"),
>  		OPT_STRING(0, "exec", &exec, "cmd",
>  			"path to the remote git-upload-archive command"),
> +		OPT_STRING(0, "format", &format, "fmt", "archive format"),
>  		OPT_END()
>  	};
> +	char fmt_opt[32];
>  
>  	argc = parse_options(argc, argv, prefix, local_opts, NULL,
>  			     PARSE_OPT_KEEP_ALL);
>  
>  	if (output)
> +	{

On the same line, i.e. "if (output) {".

>  		create_output_file(output);
> +		if (!format)
> +			format = format_from_name(output);
> +	}
> +
> +	if (format)
> +	{

On the same line, i.e. "if (format) {".

> +		sprintf(fmt_opt, "--format=%s", format);
> +		argv[argc++] = fmt_opt;
> +		argv[argc] = NULL;

Did you make sure you are allowed to write into argv[] and the array is
large enough?  You probably need to make a copy of the array.

Otherwise, the idea feels sound.

[PATCH v2 2/2] teach git-archive to auto detect the output format

[Dmitry Potapov]

When I type something like this:
  git archive -o my-v2.0.zip v2.0
it is almost certainly that I want to create a zip archive, and not
a tar file.

This patch teaches git-archive to auto detect the output format from the
file name. Currently, only '.zip' is supported. If the auto detect failed,
the tar format is used as before. The auto detect is not used when the
output format is specified explicitly.

Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
---

I have corrected all remarks except this:

On Sun, Sep 13, 2009 at 11:52:56AM -0700, Junio C Hamano wrote:
> > +		sprintf(fmt_opt, "--format=%s", format);
> > +		argv[argc++] = fmt_opt;
> > +		argv[argc] = NULL;
> 
> Did you make sure you are allowed to write into argv[] and the array is
> large enough?  You probably need to make a copy of the array.

Either --output or --format option was used before, and this option is
extracted from argv[] by parse_options(). So it should be space for at
least one argument in argv.


 Documentation/git-archive.txt |   13 +++++++++++--
 builtin-archive.c             |   25 ++++++++++++++++++++++++-
 2 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
index 1917f2e..3d1c1e7 100644
--- a/Documentation/git-archive.txt
+++ b/Documentation/git-archive.txt
@@ -34,8 +34,11 @@ OPTIONS
 -------
 
 --format=<fmt>::
-	Format of the resulting archive: 'tar' or 'zip'.  The default
-	is 'tar'.
+	Format of the resulting archive: 'tar' or 'zip'. If this option
+	is not given, and the output file is specified, the format is
+	inferred from the filename if possible (e.g. writing to "foo.zip"
+	makes the output to be in the zip format). Otherwise the output
+	format is `tar`.
 
 -l::
 --list::
@@ -130,6 +133,12 @@ git archive --format=zip --prefix=git-docs/ HEAD:Documentation/ > git-1.4.0-docs
 	Put everything in the current head's Documentation/ directory
 	into 'git-1.4.0-docs.zip', with the prefix 'git-docs/'.
 
+git archive -o latest.zip HEAD::
+
+	Create a Zip archive that contains the contents of the latest
+	commit on the current branch. Note that the output format is
+	inferred by the extension of the output file.
+
 
 SEE ALSO
 --------
diff --git a/builtin-archive.c b/builtin-archive.c
index 565314b..6efba6f 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -60,6 +60,17 @@ static int run_remote_archiver(int argc, const char **argv,
 	return !!rv;
 }
 
+static const char* format_from_name(const char *filename)
+{
+	const char *ext = strrchr(filename, '.');
+	if (!ext)
+		return NULL;
+	ext++;
+	if (!strcasecmp(ext, "zip"))
+		return "zip";
+	return NULL;
+}
+
 #define PARSE_OPT_KEEP_ALL ( PARSE_OPT_KEEP_DASHDASH | 	\
 			     PARSE_OPT_KEEP_ARGV0 | 	\
 			     PARSE_OPT_KEEP_UNKNOWN |	\
@@ -70,6 +81,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
 	const char *exec = "git-upload-archive";
 	const char *output = NULL;
 	const char *remote = NULL;
+	const char *format = NULL;
 	struct option local_opts[] = {
 		OPT_STRING('o', "output", &output, "file",
 			"write the archive to this file"),
@@ -77,14 +89,25 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
 			"retrieve the archive from remote repository <repo>"),
 		OPT_STRING(0, "exec", &exec, "cmd",
 			"path to the remote git-upload-archive command"),
+		OPT_STRING(0, "format", &format, "fmt", "archive format"),
 		OPT_END()
 	};
+	char fmt_opt[32];
 
 	argc = parse_options(argc, argv, prefix, local_opts, NULL,
 			     PARSE_OPT_KEEP_ALL);
 
-	if (output)
+	if (output) {
 		create_output_file(output);
+		if (!format)
+			format = format_from_name(output);
+	}
+
+	if (format) {
+		sprintf(fmt_opt, "--format=%s", format);
+		argv[argc++] = fmt_opt;
+		argv[argc] = NULL;
+	}
 
 	if (remote)
 		return run_remote_archiver(argc, argv, remote, exec);
-- 
1.6.5.rc1.2.g6bb993

Re: [PATCH v2 2/2] teach git-archive to auto detect the output format

[Junio C Hamano]

Dmitry Potapov <dpotapov@gmail.com> writes:

> On Sun, Sep 13, 2009 at 11:52:56AM -0700, Junio C Hamano wrote:
>> > +		sprintf(fmt_opt, "--format=%s", format);
>> > +		argv[argc++] = fmt_opt;
>> > +		argv[argc] = NULL;
>> 
> Either --output or --format option was used before, and this option is
> extracted from argv[] by parse_options(). So it should be space for at
> least one argument in argv.

To my taste, that is a unwarranted (on the borderline) assumption of what
parse_options() does, but I'll let it pass with some additional comment to
warn readers of the code.

Applied.

Re: [PATCH 1/2] git-archive: add '-o' as a alias for '--output'

[Junio C Hamano]

Dmitry Potapov <dpotapov@gmail.com> writes:

> The '-o' option is commonly used in many tools to specify the output file.
> Typing '--output' every time is a bit too long to be a practical alternative
> to redirecting output. But specifying the output name has the advantage of
> making possible to guess the desired output format by filename extension.
>
> Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
> ---
> ...
> diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
> index 92444dd..f7a3b95 100644
> --- a/Documentation/git-archive.txt
> +++ b/Documentation/git-archive.txt
> @@ -48,6 +48,7 @@ OPTIONS
>  --prefix=<prefix>/::
>  	Prepend <prefix>/ to each filename in the archive.
>  
> +-o::
>  --output=<file>::
>  	Write the archive to <file> instead of stdout.

I think this patch is very reasonable, except for this hunk, which would
want to say "-o <file>::" instead.  I'll see if there are comments from
others and if there is none, apply this patch with that minor tweak.

Thanks.

[PATCH v2 1/2] git-archive: add '-o' as a alias for '--output'

[Dmitry Potapov]

The '-o' option is commonly used in many tools to specify the output file.
Typing '--output' every time is a bit too long to be a practical alternative
to redirecting output. But specifying the output name has the advantage of
making possible to guess the desired output format by filename extension.

Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
---

On Sun, Sep 13, 2009 at 11:34:43AM -0700, Junio C Hamano wrote:
> I think this patch is very reasonable, except for this hunk, which would
> want to say "-o <file>::" instead.

Corrected.

 Documentation/git-archive.txt |    3 ++-
 archive.c                     |    2 +-
 builtin-archive.c             |    2 +-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
index 92444dd..1917f2e 100644
--- a/Documentation/git-archive.txt
+++ b/Documentation/git-archive.txt
@@ -10,7 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git archive' [--format=<fmt>] [--list] [--prefix=<prefix>/] [<extra>]
-	      [--output=<file>] [--worktree-attributes]
+	      [-o | --output=<file>] [--worktree-attributes]
 	      [--remote=<repo> [--exec=<git-upload-archive>]] <tree-ish>
 	      [path...]
 
@@ -48,6 +48,7 @@ OPTIONS
 --prefix=<prefix>/::
 	Prepend <prefix>/ to each filename in the archive.
 
+-o <file>::
 --output=<file>::
 	Write the archive to <file> instead of stdout.
 
diff --git a/archive.c b/archive.c
index 0bca9ca..73b8e8a 100644
--- a/archive.c
+++ b/archive.c
@@ -283,7 +283,7 @@ static int parse_archive_args(int argc, const char **argv,
 		OPT_STRING(0, "format", &format, "fmt", "archive format"),
 		OPT_STRING(0, "prefix", &base, "prefix",
 			"prepend prefix to each pathname in the archive"),
-		OPT_STRING(0, "output", &output, "file",
+		OPT_STRING('o', "output", &output, "file",
 			"write the archive to this file"),
 		OPT_BOOLEAN(0, "worktree-attributes", &worktree_attributes,
 			"read .gitattributes in working directory"),
diff --git a/builtin-archive.c b/builtin-archive.c
index f9a4bea..565314b 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c
@@ -71,7 +71,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
 	const char *output = NULL;
 	const char *remote = NULL;
 	struct option local_opts[] = {
-		OPT_STRING(0, "output", &output, "file",
+		OPT_STRING('o', "output", &output, "file",
 			"write the archive to this file"),
 		OPT_STRING(0, "remote", &remote, "repo",
 			"retrieve the archive from remote repository <repo>"),
-- 
1.6.5.rc1.2.g6bb993

Re: obnoxious CLI complaints

[Brendan Miller]

On Sun, Sep 13, 2009 at 11:47 AM, Junio C Hamano <gitster@pobox.com> wrote:
> John Tapsell <johnflux@gmail.com> writes:
>
>> Ah, the manpage examples specifically give the --format=tar though.
>
> So what?

That looks like a manpage bug. In the version I have 1.6.4 the format
for archive is given like this:

 git archive --format=<fmt> [--list] [--prefix=<prefix>/] [<extra>]
                         [--output=<file>] [--worktree-attributes]
                         [--remote=<repo> [--exec=<git-upload-archive>]] <tree-i
sh>
                         [path...]

So --format isn't marked as optional. Later in the manpage it mentions
tar as the default, but that contradicts this, and the examples use
--format=tar, so it's easy to miss.

>
>> Why not have  --format=tgz  then or something?  Or better yet, give
>> the filename on the command line and detect the format from the file
>> extension.
>
> That is an interesting enhancement and sounds like a useful feature.
>
I do like that idea.

git archive --output=myarchive.tar.gz HEAD is a bit more
straightforward, and still lets people pipe in the old way if they
want to.

I think someone mentioned we're already linking the requisite library?
Otherwise, you can always open up a pipe programmatically within git.
Don't you guys do something like that for ssh? I seem to recall it
complaining that ssh couldn't be found on windows, but maybe it was
just the library.

Prefix could be myarchive. I guess some people have more specific
requirements, but I usually just want it to be *sometime* so it
doesn't spew out tons of files into the directory I decompress it
into.

Brendan

Re: obnoxious CLI complaints

[Junio C Hamano]

Brendan Miller <catphive@catphive.net> writes:

> On Sun, Sep 13, 2009 at 11:47 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> John Tapsell <johnflux@gmail.com> writes:
>>
>>> Ah, the manpage examples specifically give the --format=tar though.
>>
>> So what?
>
> That looks like a manpage bug. In the version I have 1.6.4 the format
> for archive is given like this:
>
>  git archive --format=<fmt> [--list] [--prefix=<prefix>/] [<extra>]
>                          [--output=<file>] [--worktree-attributes]
>                          [--remote=<repo> [--exec=<git-upload-archive>]] <tree-i
> sh>
>                          [path...]
>
> So --format isn't marked as optional.

Ahh, that would indeed be a documentation bug.

82d97da (Documentation: git-archive: mark --format as optional in summary,
2009-08-27) fixed it already, so upcoming 1.6.5 should have that fix.

Thanks.

Re: obnoxious CLI complaints

[Sverre Rabbelier]

Heya,

On Wed, Sep 9, 2009 at 23:27, Brendan Miller<catphive@catphive.net> wrote:
> 1. cloning from a new empty repo fails, and so do a lot of other
> operations. This adds unnecessary steps to setting up a new shared
> repo.

Actually, it works in recent git.

> 2. git --bare init. The flag goes before the operation unlike every other flag?

Because unlike most flags it applies to anything git does, e.g. 'git
--bare status' to treat the current directory as a git repository
rather than looking for a .git directory.

> 4. The index is inconsistently referred to as too many different
> things (cache, index, staging area) and only the last one makes any
> intuitive sense to a new user. This is partially a CLI issue, and
> partially a documentation issue, but both add up to cause confusion.
>
> 5. Most commands require lots of flags, and don't have reasonable
> defaults. e.g. archive.

Most git commands need no more than two flags, and the defaults are
reasonable to me; of course everybody has different needs, git
aliasses make it easy to overcome this.

> This is what I want to do 90% of the time, so it should just have the
> proper defaults, and not make me look at the man page every time I
> want to use it.

Again, make an alias if you use it a lot; it might be what _you_ want
to do, but as you can see from the plethora of other examples, it is
perhaps not what everybody else wants to do 90% of the time.

> 6. Where is the bug tracker? If people users can't find the bug
> tracker, they can't report issues, and obnoxious bugs go unfixed, or
> people have to whine on the mailing list. There should be a nice big
> link on the front page of git-scm.com. A bug tracker is really the
> only way for the vast majority of a community that use a tool can give
> feedback on the problems the tool has.

Nope, that's not how things work for us. We _want_ people to 'whine'
on the mailing list, so that if they really don't care that much about
something, it is "dropped on the floor" (because the thread becomes
stale) after a while, and we move on. Bug trackers are notorious for
growing in size very fast, and becoming cluttered (I have experience
with this as well, and am very pleased with how the git's mailing list
approach solves this).

> 7. Man pages: It's nice we have them, but we shouldn't need them to do
> basic stuff. I rarely had to look at the man pages using svn, but
> every single time I use git I have to dig into these things. Frankly,
> I have better things to do than RTFM.

Honestly, you never had to consult the help for svn? Perhaps the help
was not in a man page, but in some obscure website online; I for one
welcome our on-disk information bringing overlords.

> 8. There's no obvious way to make a remote your default push pull
> location without editing the git config file. Why not just something
> like

> It's ok to have kind of a weak UI on a new tool, when people are busy
> adding basic functionality. However, at this point git already has way
> more features than most of the competition, and the needless
> complexity of the CLI is the biggest issue in day to day use.

If you don't need all that functionality, why not just use one of them
fancy GUI's out there?

-- 
Cheers,

Sverre Rabbelier

Re: obnoxious CLI complaints

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

On Wed, Sep 09, 2009 at 02:27:56PM -0700, Brendan Miller wrote:
> 5. Most commands require lots of flags, and don't have reasonable
> defaults. e.g. archive.
> 
> git archive --format=tar --prefix=myproject/ HEAD | gzip >myproject.tar.gz
> 
> Should just be:
> git archive
> run from the root of the repo.

You can't, because "myproject" cannot be guessed most of the time.

If you really want to automatize it, it's a 2 liner shell script, or
alternatively you can add that to your .gitconfig:

[alias]
    archive-gz=!git archive --prefix="$(basename "$(dirname "$(readlink -m "$(git rev-parse --git-dir)")")")"/ HEAD | gzip -c

It will use the name of the directory containing your .git/ directory as a
prefix, and compress it using gzip to stdout.

git archive-gz > myproject.tar.gz will do what you want.

See, that's the point about git having so many flags. You only need to
look the man page _once_ (despite what you pretend): the one time you
need to write your convenient wrapper around git commands that suits
your exact needs.


-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: obnoxious CLI complaints

[Björn Steinbrink]

On 2009.09.09 14:27:56 -0700, Brendan Miller wrote:
> 8. There's no obvious way to make a remote your default push pull
> location without editing the git config file. Why not just something
> like
> 
> git remote setdefault origin

Because "git remote" is the wrong tool. The default remote for
fetch/push is configured per branch head, not globally.

> or even
> 
> git remote add --default origin http://somegiturl.org/

Because that would likely mean changing all the existing branch head
configurations, quite possibly making them invalid, because the
branch.<name>.merge value suddenly makes no sense anymore.

> This come up in the use case where I:
> 1. set up a remote bare repo
> 2. push from my local repo, and thence forth want to keep local and
> remote in sink.

Maybe you want something like this?
http://thread.gmane.org/gmane.comp.version-control.git/107671

> Right now I have to modify .git/config to do this.

You can also do:
git config branch.<name>.remote <remote>

which does the config editing for you.

And maybe:
git config branch.<name>.merge <ref>

if you also want just "git pull" to work, or use push.default = tracking.


There's also still my old "retrack" hack, which John Wiegley extended:
http://github.com/jwiegley/git-scripts/blob/6ba3184d7b9f6dae3d10379a6bac29a01ceef190/git-retrack

That's a bit more comfortable, but probably doesn't work for a lot of
"non-standard" setups.

Björn

Re: obnoxious CLI complaints

[Matthieu Moy]

Björn Steinbrink <B.Steinbrink@gmx.de> writes:

> On 2009.09.09 14:27:56 -0700, Brendan Miller wrote:
>> 8. There's no obvious way to make a remote your default push pull
>> location without editing the git config file. Why not just something
>> like
>> 
>> git remote setdefault origin
>
> Because "git remote" is the wrong tool. The default remote for
> fetch/push is configured per branch head, not globally.

(the --track option of git branch and git checkout can help).

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/