Fork me on GitHub

Friday, December 5, 2008

Test Drivers, Frameworks and Harness

Edited on 20 March 2009 - Added the last paragraph about JUnit

Wanted write about this for a long time. People tend to ask me if Watir is a framework or does Watir read Excel or how to connect to database from Watir and questions similar to that. This is not a Watir specific post but some understanding I have about these three important components of any automated testing. I wrote this because I feel that people tend to ask these questions because they don't understand the function of these three components. So here it goes...

Test Drivers - They drive the application. If you are connecting directly to domain or service layer you will not have an external one most probably, the API itself will be the driver. But if you driving tests through GUI interface or may be a remote service, then you will have a GUI driver or something like a webservice driver to drive the application. Libraries like Watir, Selenium, WebDriver or SOAP4R (to drive webservices through SOAP) fall into this category. They are not testing tools but you can very well build one on top of them.

Test frameworks - Scaffolding to start your testing. Nobody will start writing tests from scratch every time.Framework acts as the base infrastructure so that we don't need to care much about it. It will have things which can be reused across our tests but not specific to application mostly like connection to database, writing log or reading and writing to Excel. Normally we start with a generic framework and as we write tests we tend to build abstractions of our domain and slowly project specific abstractions and functions grow into the framework.

For example in JUnit, TestCase class is a framework part. It gives you assertions. Similarly Taza framework for Watir gives you generators, integrates with Rspec for assertions and reporting (It is a framework with frameworks inside ;)).

Test Harness - They are used to execute the tests we write. Like JUnit test runner or RSpec test runner. Most of the time harness will be coupled with a framework but it is possible to create a generic harness not coupled to a framework as long as the framework follows some kind of contract that harness enforces. Like in JUnit any method annotated as test will be picked up by the test runner to be executed as test.

What I have written is a view I developed when working with these components over years. You may have a different perspective. If so let me know. Your comments and views are welcome.

One more thing... A common question I get is why I use JUnit or RSpec which are meant for unit testing. Junit and RSpec are test frameworks. You can run pretty much any code inside them. It is incidental that they are used for unit testing because they are simple and light weight.

1 comment:

Lisa said...

Thank you for this post, you have made terminology which is confusing to me much more clear.