Automatically generated directories for individual tasty tests

This is a hack practical trick for creating directories based on test names using the Haskell test framework tasty, as well as accessing the test names inside your tasty tests themselves.

You can find the full code on GitHub

Last week I worked on a test suite that printed a lot of logs to stdout. This didn’t play well with tasty as the test results were interleaved with whatever was printed out on stdout. I decided to redirect the logs to files but I wanted those logs to be easily retrievable in case of a test failure. The easiest way to do so was to create a directory for each test, based on the test’s name.

The tasty framework unfortunately does not allow you to read the TestTree structure, thereby making it impossible to know the current test’s name. Fortunately tasty has a flexible option infrastructure which we can leverage to track the names used in the tree, from the root up to the leaf – i.e. the full test name.

First we’ll create a type that contains the names encountered on a particular path in the tree and make it a tasty option (that is, create an IsOption instance):

Note: in practice you do not want to export TastyNames as this should only be used internally. You don’t want your users to actually set TastyNames on the command line, for instance.

Then we can provide a substitute to tasty’s testGroup that updates the TastyNames option for the children of the node:

Finally we provide helper functions that provide the test’s name as an argument to the test itself – or better, create a directory for the test:

This can be used together as follows:

And that’s how you can easily access tasty test names inside the test themselves and create unique directories for your logs! How you incorporate this in your test suite is up to you. One simple way is to use the .Extended pattern and create new modules – Test.Tasty.Extended and Test.Tasty.HUnit.Extended – which re-export most functions from the original modules, but also the tweaked testGroup and the helper functions testCaseWithNames and testCaseWithDir. Also you may want to add another IsOption for setting the base directory in which the logs are created – this one actually specifiable by the user.

Here’s the full code: