From: Jakub Narebski <jnareb@gmail.com>
To: Lea Wiemann <lewiemann@gmail.com>
Cc: git@vger.kernel.org
Subject: Re: [RFC/WIP/PATCH v2] gitweb: add test suite with Test::WWW::Mechanize::CGI
Date: Mon, 16 Jun 2008 02:40:17 +0200 [thread overview]
Message-ID: <200806160240.18539.jnareb@gmail.com> (raw)
In-Reply-To: <4855634C.8060606@gmail.com>
Lea Wiemann wrote:
> Jakub Narebski wrote:
> >
> > 1.) Should we put all tests in one file, or should they be split
>
> I'd suggest we leave it in a single file until test execution time
> becomes an issue. Then (when it has become too large) we'll be able to
> figure out good boundaries along which to split the test suite.
I wanted to split tests mainly not because of performance, but because
of making it easier to maintain. Although perhaps single driver-test,
and do()-ing or require()-ing sub-files would be enough.
> > 2.) What invariants should we test [...] Checking for example if all items
> > are listed in a 'tree' view, or if all inner links (#link) are
> > valid would be a good start...
>
> Yup; completeness of item lists is especially relevant for paginated
> output. Also check for the presence and validity of links (like
> "parent" links, etc.), and for the presence of certain elements (like
> the file modes in the tree view).
For example if "next" (and like) views really lead to next page.
> Also, with a $ENV{LONG_GIT_TEST} variable or so, we could automatically
> validate all links for each page we're checking -- it takes a long time,
> but it's still way more efficient than exhaustive spidering of the whole
> site.
Good idea. I would examine how it is done in other tests.
> > (by the way, is there some Perl module for RSS, Atom and OPML validation?)
>
> I can't find anything on Google right now,
I usually search CPAN first, not Google...
> but piping them into external
> validators might be just as fine. Also, since those formats are
> generated using print statements (which is really error-prone for XML
> formats), I'd say that a good start would be to check for XML validity.
We can use Test::XML / Test::XML::Valid / Test::XML::Simple for being
well-formed XML. If RSS / Atom / OPML have good DTD / XML Schema /
/ Relax-NG schema / Sablotron rules, they could be checked using that
from Perl.
> > 3.) What invariants you want to test for your caching efforts, e.g.
> > checking if cached output matches non-cached
>
> How about this:
>
> 1. Run the Mechanize tests (and possibly also the existing t9500 tests)
> *without* caching, recording the URL's and contents of all pages the
> test suite accesses.
>
> 2. Get all those URL's again *with* caching (from a cold cache), and
> assert that the output is identical.
How would you ensure cold cache?
> 3. Get all those URL's again *with* caching (from a warm cache), and
> assert that the output is identical.
Well, it might be identical, but it also might have "cached output"
somewhere in the output.
> Perhaps also assert that no call
> to the git binary is made (i.e. everything has actually been cached).
> (Of course we might need options for the production site to not cache
> certain things, but let's defer this discussion.)
Or at least (if we don't cache everything, and that could be good idea)
to check if there are less git binary calls.
--
Jakub Narebski
Poland
next prev parent reply other threads:[~2008-06-16 0:41 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-06-14 12:47 [RFC/PATCH (WIP)] gitweb: Use Test::WWW::Mechanize::CGI to test gitweb output Jakub Narebski
2008-06-14 14:40 ` Lea Wiemann
2008-06-14 18:07 ` Jakub Narebski
2008-06-14 18:31 ` Lea Wiemann
2008-06-14 18:59 ` Jakub Narebski
2008-06-14 21:12 ` Lea Wiemann
2008-06-15 8:36 ` Jakub Narebski
2008-06-14 18:18 ` Lea Wiemann
2008-06-14 18:31 ` Jakub Narebski
2008-06-14 23:57 ` [RFC/WIP/PATCH v2] gitweb: add test suite with Test::WWW::Mechanize::CGI Lea Wiemann
2008-06-15 18:01 ` Jakub Narebski
2008-06-15 18:45 ` Lea Wiemann
2008-06-16 0:40 ` Jakub Narebski [this message]
2008-06-16 9:10 ` Lea Wiemann
2008-06-16 20:15 ` Jakub Narebski
2008-06-20 3:18 ` [WIP/PATCH v3] " Lea Wiemann
2008-06-20 12:08 ` Jakub Narebski
2008-06-20 13:49 ` Lea Wiemann
2008-06-20 18:03 ` Jakub Narebski
2008-06-20 22:04 ` Lea Wiemann
2008-06-20 22:18 ` [WIP/PATCH v4] " Lea Wiemann
2008-06-23 0:45 ` [PATCH v5] " Lea Wiemann
2008-06-23 1:14 ` [PATCH v6] " Lea Wiemann
2008-06-23 2:30 ` Junio C Hamano
2008-06-23 7:00 ` Lea Wiemann
2008-06-23 13:31 ` Jakub Narebski
2008-06-23 17:57 ` Lea Wiemann
2008-06-23 22:18 ` Jakub Narebski
2008-06-24 2:01 ` Lea Wiemann
2008-06-24 2:18 ` [PATCH v7] " Lea Wiemann
2008-06-26 13:47 ` [PATCH] " Lea Wiemann
2008-06-26 13:48 ` [PATCH v8] " Lea Wiemann
2008-06-29 22:47 ` Jakub Narebski
2008-06-29 23:39 ` Lea Wiemann
2008-06-29 23:56 ` Jakub Narebski
2008-06-30 0:30 ` Lea Wiemann
2008-06-30 21:55 ` Jakub Narebski
[not found] ` <48681EC8.8000606@gmail.com>
2008-06-30 22:01 ` Jakub Narebski
2008-06-24 4:20 ` [PATCH v6] " Junio C Hamano
2008-06-24 8:37 ` Lea Wiemann
2008-06-24 9:23 ` Jakub Narebski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=200806160240.18539.jnareb@gmail.com \
--to=jnareb@gmail.com \
--cc=git@vger.kernel.org \
--cc=lewiemann@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).