Re: [RFC/PATCH (WIP)] gitweb: Use Test::WWW::Mechanize::CGI to test gitweb output

[Lea Wiemann <lewiemann@gmail.com>]

Jakub Narebski wrote:
> This test requires [PATCH] t/test-lib.sh: add test_external and test_external_without_stderr

For everyone's convenience, here's the raw patch: 
http://article.gmane.org/gmane.comp.version-control.git/83415/raw
(When repo.or.cz is up again you can also locate a version with fixed 
whitespace in my repo: <http://repo.or.cz/w/git/gitweb-caching.git>)

> This test uses Perl module Test::WWW::Mechanize::CGI to check
> gitweb output, using HTML::Lint (if present) to validate HTML.

Awesome, thank you so much for making a start here!  Here are some quick 
comments.

> NOTE: Currently test_external_without_stderr fails because when trying
> to access URL for non-existent commit gitweb writes to STDERR; it is
> not necessarily a bug because it is not written to web server logs

Without having looked at the cause of that, I think that gitweb should 
not be writing stuff to stderr unless an internal or serious error 
occurs; in particular trying to access invalid commits shouldn't cause 
messages on stderr, only to log files if at all.

That said, as long as it isn't fixed, here's my workaround to 
temporarily discard stderr (from my t/t9710/test.pl):

our $old_stderr;
sub discard_stderr {
         open our $old_stderr, ">&", STDERR or die "cannot save STDERR";
         close STDERR;
}
sub restore_stderr {
         open STDERR, ">&", $old_stderr or die "cannot restore STDERR";
}

It works on Unix, but I don't know about other platforms.

> +cat >gitweb_config.perl <<EOF
> [...]
> +our \$GIT = "git";

t9500 seems to be doing the same(?) thing, but this somehow doesn't work 
with your t9503 test:

$ git     # no git in PATH to make sure it picks up the right git binary
bash: git: command not found
$ ./t9500-gitweb-standalone-no-errors.sh | grep passed
* passed all 75 test(s)
$ ./t9503-gitweb-Mechanize.sh -v
[...]
	gitweb.perl: Can't exec "git": No such file or directory at 
/home/lea/source/git/fresh-git/gitweb/gitweb.perl line 380.

-- Lea