Fork me on GitHub

Wednesday, July 15, 2009

Setup and teardown are evil for a unit test

I have been writing different types of test for quiet a while now. TDDing with unit tests, writing acceptance tests with FitNesse and tests with Watir. Except for FitNesse for writing other kinds of tests, I prefer to use some kind of xunit framework as it gives me most of infrastructure I need already. I may build my own test framework on top of this if needed but that is a different story.

Now the problem comes to set up and tear down methods in unit tests. Mark that these apply for unit tests and may not be directly useful for other kinds of tests.

By definition set up method is used to create the infrastructure needed for the test to run and tear down is used clear any hangover state from the test run. So if these methods are useful why are they evil for a unit test?

Here are the problems I find with a set up and tear down methods
  • Typically people use set up and tear down as places to refactor the common part which needs to run before and after each test method. But the problem for me is I need to scroll up and down to set up and tear down when I am reading the test. I cannot the get complete picture of the test unless I see these methods every time. I hope the test class is not so long that there is need to scroll up and down but I also feel that the meaning of the test should not be fragmented.
  • Also more importantly using a set up method will hide the details and complexity of dependencies being created to make the SUT work as this information is not directly visible in related test method which will look simple. Too many dependency may be a candidate for refactoring.
  • Tear down is typically used for clearing the hangover from the tests. The objects created in the test methods in a unit test should not so heavy that we need a separate tear down method to clear them from memory. The garbage collector should be good enough to take care of them. But more importantly predominant use of tear down in a unit test may mean that some state created by the test or SUT (Bad. I don't think I need to talk about importance of side effect free code :). Either way it is important to refactor them to become side effect free.
Lesson - Don't use set up and tear down in a unit test . They normally end up hiding the intent of the test and may be a signal for code smell like side effects.

So where would I use these? I will definitely use set up and tear down in end to end tests which I write in Watir or API level tests which create some heavy resource. I will use these methods to create the Build Operate Check Clear pattern which I discussed in one of the previous posts.

If you feel that you need to factor out some set up things to somewhere, create a set up method and explicitly call it in each test so that it becomes explicit of what dependencies we are setting up for that test method.

The approach and thoughts in this post may be a bit extreme. But they havyou as well e helped me and hopefully to make tests more expressive and code cleaner :). Let me know your thoughts...


nfma said...

Though in my current project I only use setup and teardown methods in end-to-end tests, I now face a little problem, or incoherence. The problem lies with side-effect free methods, which I use extensively. They kind of go against the "tell, don't ask" principle. So, I'm codinng in a OO language but end up using too many functional concepts. Is this good or bad? I don't know, yet...

Sai Venkatakrishnan said...

I feel writing side effect free functions is good. It is influenced by functional programming. But the basic tenet of OO is also to write independent entities.

Not sure how you violate tell don't ask (An example would be handy right now :) because if you have side effect free functions you may not have a lot of state hanging around exposed. You ask a function to do work for you and it returns the result... And that seems to be what tell don't ask suggests.

roeeoz said...

I basically feel functions, objects and code reuse are evil.
I strongly recommend to write long function with many repeating lines.

Sai Venkatakrishnan said...

Welcome to the world of Cobol.. You will receive your IBM 704 machine in a few days..