
quvi test suite
===============

Prerequisites
-------------

  * quvi 0.4.1+
     - The test suite uses the new options introduced in quvi 0.4.1,
       e.g. "-vq" instead of (now depr.) "-q", etc.

  * Perl (and prove <http://search.cpan.org/perldoc?prove>)

  * Perl modules (that are not among the 5.12.2 core modules)
     - Test::Deep <http://search.cpan.org/perldoc/Test::Deep>
     - JSON::XS <http://search.cpan.org/perldoc?JSON::XS>

Notes
-----

  * Running "make check" can be a lengthy process
    - The tests are disabled by default
    - Run configure with --with-tests to enable them
    - The tests require an internet connection

  * To run individual tests, run prove with the -q and -b options,
    see examples below

  * quvi command reads the ~/.quvirc, keep this in mind when
    you use the test suite

  * If you report bugs detected by the test suite, please send the log
    (omit whatever is irrelevant to prevent clutter):

      script -c "make check" foo.log

Running
-------

  cd $top_srcdir ; mkdir tmp ; cd tmp ;
  ../configure --with-tests && make check (or "make test")

Similar, but without "make check":

  (assuming you have build quvi in $top_srcdir/tmp)
  cd $top_srcdir/tests
  prove -l t/ :: -q path/to/quvi/command -b ../share

Running: Individual tests
-------------------------

  prove -l t/nosupport.t :: -q /path/to/quvi/command -b ../share

Running: Custom tests
---------------------

See the $top_srcdir/tests/t/d/ directory. These tests are not run by the
test suite when you type "make check". You may find these tests useful
when you are, for example, developing a new webscript.

  (assuming you have build quvi in $top_srcdir/tmp and the lua
  scripts reside in $top_srcdir/share/)
  cd $top_srcdir/tests
  prove -l t/d/url.t :: -q /path/to/quvi/command -b ../share \
        -u $test_url

Arguments to tests
------------------

"It is possible to supply arguments to tests. To do so separate
them from prove's own arguments with the arisdottle, '::'.
For example:

  prove -v t/mytest.t :: --url http://example.com

would run t/mytest.t with the options '--url http://example.com'.
When running multiple tests they will each receive the same
arguments." -- prove(1)

The test suite supports the following options:

  -q,--quvi-path arg            Path to quvi(1) command
  -b,--libquvi-scriptsdir arg   Path to dir containing the libquvi-scripts
  -o,--quvi-opts arg            Additional options to quvi(1) command
  -j,--json-file arg            Verify received data against local JSON file
  -J,--dump-json                Print quvi generated JSON to stderr
  -i,--ignore arg               Ignore arg in JSON comparison
  -d,--data-root arg            Path to directory containing the (test) data/
  -v,--valgrind-path arg        Path to valgrind(1) command

Note that the test suite presumes that quvi command is either found in
the $PATH or -q is used to define the path to the command.

If quvi cannot find the lua scripts, use the --libquvi-scriptsdir option,
e.g. "-b $top_srcdir/share/"

You can pass additional options to quvi, e.g:

  prove t/foo.t :: -o "-vq -e-r"

The test suite uses the bundled JSON files for the verification process.
You have the option of using --json-file with the "Custom tests" (see
above), e.g.:

  prove -l t/d/url.t :: \
    -q /path/to/quvi/command -b ../share \
    -j data/format/default/youtube.json

Use the --ignore option if you need to ignore any of the values in the
JSON, e.g.:

  prove t/foo.t :: -i content_type
Or
  prove t/foo.t :: -i content_type,page_title,host

The test suite ignores some of the values (e.g. media URL,
thumbnail URL) by default.

Environment
-----------

Set TEST_VERBOSE to amp up the verbosity level of the tests. If you
run the tests manually with prove(1), use -v with prove command
instead. TEST_VERBOSE is only picked by the "make check".

Set TEST_SKIP if you'd like to skip any of the test groups, e.g.:

  export TEST_SKIP=mem,format_default

You can run the "find-tests.sh" script found in the top source directory
to get a list of the test names that can be used with TEST_SKIP.

Test suite data
---------------

The $top_srcdir/data/ contains the JSON files that the test suite uses.

  $top_srcdir/tests/data/resolve/

    Resolve shortened, redirect etc. For example, shortened.t and
    redirect.t use these JSONs.

  $top_srcdir/tests/data/format/default/

    JSONs produced by either "quvi $test_url" or "quvi $test_url -f
    default". Each website should have at least one of these.

  $top_srcdir/tests/data/format/default/nsfw/

    Data for NSFW scripts. Ignored unless configure is run with the
    --with-nsfw switch.

  $top_srcdir/tests/data/format/default/ignore/

    Read the README in this directory.

  $top_srcdir/tests/data/format/other/

    JSONs for those websites that support more than just the
    'default format'. One JSON for each format.

  $top_srcdir/tests/data/format/other/ignore/

    Read the README in this directory.
