Fork me on GitHub

Wednesday, September 24, 2008

Are table structured tests good for regression testing?

I have been seeing cases lately where people try to use Fitnesse or Fit like infrastructures or a custom excel based fixture where in the tests are written in the form of tables. The reason behind this may be they have seen the success of using Fit or Fitnesse and want to extrapolate it everywhere. The problem I have with this approach is whether this table structure is enough for meeting the conditions needed for regression testing?

I am big fan of Fit and Fitnesse and advocate, use it as a way to write executable specifications. They are really good when you work with the customer to etch concrete examples about the business domain conditions. But using them as a framework for regression testing is something I am skeptical about. I prefer to use a language like Ruby or Python or Java to write my tests using drivers like watir or webdriver or schnell. The reason behind this is more than acceptance tests, regression tests cover much more conditions in form of data driven tests as well need constructs for iterating, conditional constructs, variable storage and passing and reusing code by modules. Having these constructs will help us simplify the code we write. As well regression tests don't have very close customer involvement as acceptance tests do and in this case for a tester or an automation engineer it is easier to use a language than a table structure.

This doesn't mean that we can't do or have these things in a table driven test. But for this we need to put an extra infrastructure in place which we have to maintain as well these constructs will not look elegant or natural in a table.

Implementing the regression tests in a developer language also doesn't mean that they will be just raw code which have the user actions flow driven using watir or webdriver or schnell. These tests are also written in a way that they reflect the domain language by building proper abstractions may be as an internal domain specific language. But these abstractions need not be structured as a table as explained in the above case.

This is from my opinion and experience. Different people will have varied experiences and thoughts. I would be happy if you can share your thoughts on this as it will help to understand this problem more or may be in a different angle...

No comments: