MUTAT - Developer information

MUTAT is a simple cgi script to demonstrate possible uses of EARL, and also use of RDF to configure an application. It is designed to help perform Quality Assurance evaluation tasks.

Using it

You can go straight to the demonstration page of the testing tool which has been updated to be more user-friendly.

You can also make a URI that calls it directly, such as http://www.w3.org/QA/Tools/MUTAT/bin/mutat?script_status=1&TEST_url=http://www.w3.org/QA/Tools/MUTAT/wcag.n3

About MUTAT

QA experts have come up with a variety of solutions to handle entry and retrieval of the data they need - sometimes little more than a paper and pencil. The idea behind MUTAT was to demonstrate automating the process for human-centered testing, and output the results in EARL. The script presents the user with a series of questions: first a page of fill-in information on their identity and testing environment, and then the actual tests. The tests can be formatted either as a single page of text-based questions and links or a succession of framed HTML pages, with form inputs for the results. Both the initial information questions and the test questions are configured using an RDF file. There is also a feature that allows filtering of test questions.

The script uses the Ginger Alliance's RDF n3 parser. @@Is that still a sensible decision?

Getting the Source Code

the code is available read-only, using the CVS server at W3C.

Set the CVSROOT parameter: CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public
with a unix shell (e.g bash) type :
me@myhost:~$ export CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public.
Or see the documentation for your CVS client.
login :
me@myhost:~$ cvs login
(password:anonymous).
checkout the module perl/modules/W3C/MUTAT.
with a unix shell (e.g bash) type :
me@myhost:~$ cvs checkout perl/modules/W3C/MUTAT

You can also browse the corresponding CVS repository with your favorite browser.

TODO

Update to new version of EARL as implemented by Hera, AccVerify, axforms.
Document process for updating EARL versions in case the working group restarts.
Restart discussion on creation and use of a public EARL store
Accept XML RDF. For instance it should be able to read the description of WCAG at http://www.w3.org/2001/sw/Europe/200305/axforms/earlInst.rdf at least.
Error-check on user input
Allow for completion of partial reports (should be possible with working EARL store)
Document how to describe a test set more clearly

The files @@incomplete

mutat: A perl file that runs the mutat thing. Asks for a test file if called with no parameters. There are parameters to make it ask for info like tool etc (needs to be fixed to work for WCAG example), or to run the tests themselves.
EARL.pm: Some PERL that allows for the creation of XML/RDF, and some specific stuff for EARL. Needs commenting and documenting. Not sure that it is a great thing to stick with - guess it's fine for PERL programmers.
RDF/Notation3/*: The RDF N3 parser from the Ginger Alliance. It probably makes sense to replace this stuff over the medium term with an RDF/XML parser such as Redland.
RDFcloud.pm: some stuff for collecting RDF assertions.
RDFtest.pm: some stuff for querying RDF.
create: A PERL script that helps you to create a new set of tests. Unfortunately the key step, of adding an actual test case, seems not to work at the moment. And it is not clear to me how it interprets the other stuff.

The text file of tests

The text file is now in RDF. Namespace can be found here. Right now the script only accepts files in n3, but there are tools to convert between formats. The easiest way to figure out what's going on is to look at the example file wcag.n3 above for examples of all the features. There is also a script to help automate creation of test files. @@Check out script, documentation.

Features to play with:

Add your own properties for the earl:Assertor or earl:TestSubject (change the domain to the proper value)
```
:thingy a rdf:Property;
rdfs:label "myThingy";
rdfs:domain earl:TestSubject; rdfs:range rdfs:Literal .
```
Add default values to any property (sorry about the re-declaration)
```
earl:date a rdf:Property;
rdf:value "Wed 13 Mar 2002" .
```
Add a limitless number of possible groupings to each TestCase to filter which bits you would like to see - declare like below and make up whatever values you'd like
```
:priority a td:GroupProperty;
rdfs:label "priority" .
```

Declare testcases with any properties you'd like to be copied into the final report, even ones that aren't used by my script

:test01 a earl:TestCase ;
rdfs:label "Test A";
:priority "C";
earl:id <HTTP: ATest.html Something www.w3.org> ;
td:text "This statement is false." .
:test02 a earl:TestCase ;
rdfs:label "Test Frog";
:priority "Argh";
earl:id <HTTP: somewhere some.domain> ;
td:text "My feet are green." .

Make up your own choices for results to override earl's passes/fails

:C a td:ResultProperty; rdfs:label "Complete";
earl:validity earl:Pass;
earl:confidence earl:High .

ChangeLog

2005-01-11: Started a new test set for SpecGL. Fixed the WCAG set to ask for a page being tested. Tested create script and noted that it can't add tests.
2005-01-03: reordered dates. Split off a page to start using it nicely from this developer info. Started commenting the code.
2002-07-25 / 2002-07-26: finally moving the stuff to web/CVS space of the QA activity at W3C.
2002-04-22: Recovered lost page from Google cache (yay!). Changed EARL output to reflect change proposed in w3c-wai-er-ig a month ago.
2002-04: Created script to automate creation of n3 test files
2002-03-12: Changed test format again... hopefully it's legal. Added ability to read beginning questions in from earl spec, td spec, and test file. Added ability to set default values with above. Added ability to override earl's existing ResultProperties. Added ability to carry all test information on so it can be printed in final earl report. Added ability to create your own grouping properties for tests so they can be filtered on any number of criteria. Abstracted a lot of stuff away from mutat script and into odd data structures to holddata.
2002-03-04: Changed test format again in attempt to make it more earl-like, added "group" property for tests that allows them to be filtered, added ability to post to annotea database (but it'll give an error if you do), added ability to post proper incomplete tests and added a quick-out button for the frames view so you don't have to do all tests until the end
2002-02-27: Made perl module to simplify EARL creation, added ability to make comments
2002-02-25: Changed test question format to n3 (and use someone's n3 RDF parser to parse), changed script's data structure for tests and then added object wrapper with pretty access methods
2002-02-18: Added access to Jim Ley's EARL database.
2002-02-15: Made source code public
2002-02-13: Created!