Tuesday, June 24, 2008

MbUnit with NAnt support for TeamCity 3.x

This week I've switched to TeamCity 3.1 after using CruiseControl.NET for a while. CC.NET is ok, but feels like something built on an off day to get automated builds working. TeamCity on the other hand feels much more mature. It's a lot easier to configure (no more XML fiddling for basic setup!). The UI is smooth and easy to work with. It's just a friendly tool.

No MbUnit support
TeamCity 3.1 does however have one big drawback for me; it only supports NUnit unit testing on .Net. There is no support for MbUnit, my current unit testing framework of choice. Fortunately, TeamCity can be extended quite easily. So, I was exprecting someone had already found a solution for this shortcoming. After some googling around I quickly found there is no easy solution available yet. I've tried some stuff like writing an XSL for the XML generated by MbUnit, but that did not yield any usable results.

The fix
TeamCity will monitor the build log and capture all sorts of information from it (described here), including test results. This is the easiest way to relay information from a build to TeamCity. MbUnit tests are executed by a custom NAnt task that comes with MbUnit (MbUnit.Tasks assembly). Since MbUnit is open source the source code is available and I can expand the task to do as I want. So, I've added support for an additional report type for TeamCity. Instead of logging to a file, it uses the build log to directly report results to TeamCity.

Not quite perfect
Unfortunatly, due to the way the MbUnit test runner works it's not possible to log test results in real-time as TeamCity expects. The results are only available after all tests in the task have completed. At that time the results are exported. This means timing information is not valid. The test results however are quite detailed, including console output, error output and a stack trace.

Using the task in a build
To use the new MbUnit task in a NAnt build script there are 3 options:

  1. If you copied the MbUnit.Tasks.dll into the NAnt bin folder, replace it with Alanta.MbUnit.Tasks.dll.
  2. Load the tasks from your build script: <loadtasks assembly="path-to-custom-tasks/Alanta.MbUnit.Tasks.dll">
  3. Load the tasks from the command line.
I use NAnt build scripts to build from the command line as well as for automated builds. So, I've defined the report type for MbUnit in a property: <property name="mbunit-report-type" value="Html" overwrite="false" /> ... <target name="test"> ... <mbunit types="${mbunit-report-type}" report-output-directory="TestResults" halt-on-failure="false"> <assemblies> <include name="MyProject.*.dll" /> </assemblies> </mbunit> </target>In TeamCity I can now set the mbunit-report-type property on the command line ( on the Runner tab ):
-D:mbunit-report-type=html;teamcity

This will generate both an html report and output the test results to the log file for TeamCity.

Update This solution is now part of the NAnt-Extensions project. You can download it here.