Re: [RFC/WIP/PATCH v2] gitweb: add test suite with Test::WWW::Mechanize::CGI

[Jakub Narebski <jnareb@gmail.com>]

Lea Wiemann wrote:

> Changed since v1:

> - Use test_external instead of test_external_without_stderr so that
>   the test doesn't fail if there is output on stderr.
> 
>   Discarding and restoring STDERR (proposed in a recent message)
>   doesn't work with Mechanize, as Mechanize somehow seems to save the
>   STDERR handle and reuse it, so subsequent calls to Mechanize keep
>   using the discarded STDERR.  (The discard/restore functions work
>   fine in other test modules, it's really just Mechanize here.)

Actually I think it is CGI module itself catching and redirecting
STDERR from Perl to logfile, and WWW::Mechanize::CGI having to catch
and redirect all stderr of invoked application, but I'm not sure.


What should we think about now, I guess, is
1.) Should we put all tests in one file, or should they be split
    according to what do they test?  For example separate tests
    for correct 4xx and 5xx responses for requests for non-exiting
    objects, and for not permitted views.

2.) What invariants should we test, beside obvious HTML validation
    using HTML::Lint (by the way, is there some Perl module for RSS,
    Atom and OPML validation?).  Checking for example if all items
    are listed in a 'tree' view, or if all inner links (#link) are
    valid would be a good start... pity that Mechanize modules don't
    have very good documetation.

3.) What invariants you want to test for your caching efforts, e.g.
    checking (not necessary with TWM::CGI) if cached output matches
    non-cached, with exception of marking output as cached?

-- 
Jakub Narebski
Poland