Fork me on GitHub

Thursday, September 17, 2009

Functional Test suite in C# or Java... Why?

Every time I see a functional test suite being developed in a static language, I get to the same question. Why? I feel odd when I hear people that the development language is C# or Java and so the functional test suite testing the web application or REST interface also needs to be in the same language. I understand and accept the reason for unit test as TDD is a design activity and it is better to be in the same language. But functional tests? Why?

I also feel sorry for people in the team who spend time mangling a static language to build test suites and framework to make the whole thing work. Also the slowness of the feedback cycle as most popular static languages are compiled and so functional test suite also needs to go through the same cycle.

So here are my reasons to choose a cool funky dynamic language to write your application level test suite.
  • Easier development cycle. Typically dynamic languages have more essence to ceremony ratio. They are lightweight. They don't have the type noise which static languages have as we don't need that kind of information in our functional tests a lot. Also dynamic languages being less ceremonious is easier to pick up and start working.
  • No wastage in compiling time. Most of the dynamic languages (Actually all I know) are not compiled and so donot have that long compile cycle. Typically functional test suites don't run as an application and donot have the same considerations as them. For functional tests, the quicker they run better the feedback cycle.
  • Easier to build abstractions. It is easier to build abstractions like frameworks, DSLs or Fluent Interfaces in dynamic languages. Less noise, more essence and better abstractions :)
  • Better drivers and libraries - Almost all the drivers I know are in dynamic languages. Watir, Selenium, WebDriver (Use with JRuby or python :), Celerity. Also you have frameworks like Rspec, Cucumber and Robot framework already available to build your test suite better. You have HttpDrivers, XmlParsers, JSonParsers or whatever infrastructure or libraries you need are available as well.
  • Easier to experiment with. Dynamic languages are lightweight to work with. They also typically come with a REPL which is a good place to learn and experiment. If you want to know how to play with a REST based service, fire up the REPL. Import a Http library and start sending requests.
  • They have a great community to work with. Difficult to substantiate but I can challenge that Ruby and Python community are way ahead in catching up with new stuff than .Net or Java. Also if you see Github and find the major languages of open source projects hosted there :).
May be there are more reasons. If you can think of any add them to the comments to discuss.

If you are a developer or a tester in a team (I don't differentiate between both of these roles :), do yourself and your team a favor. If your development itself is a dynamic language, you lucky dog :). Else choose a suitable language of choice to write functional or end to end tests. Don't get into the dogma of using the same language as development language.


Stephan.Schmidt said...


"Easier development cycle. Typically dynamic languages have more essence to ceremony ratio."


"Easier to build abstractions. It is easier to build abstractions like frameworks, DSLs or Fluent Interfaces in dynamic languages."


"[...] Selenium [...] "


"They also typically come with a REPL which is a good place to learn and experiment."

As does Scala.

"They have a great community to work with."

As has Scala.

Could be a better informed post.


Sai Venkatakrishnan said...

I have used scala and find it interesting. Even though it is a natural choice for me over Java, I don't think I will choose it over Ruby or Python. Again it is my choice.

I had scala in mind when I wrote this... But it was a deliberate choice not to include them. :)

Scala makes it easier to build abstractions, there is type inference but still there is some kind of type noise inside which I need to worry about.

Please mind that I am not against scala. I like the language. It is just that I haven't worked a lot on it. Probably I think your comment adds to this post that you can use scala as well...

One more thing. The reason for this post is to contradict the position of a lot of teams that they need to choose the language to test the application as the development language.

Will Sargent said...

I think this blog post is not very well informed.

The biggest factor in dev cycle is the people and process involved, not the language. The amount of typing or ceremony is more about the framework, not the core language.

Building abstractions is the same amount of work in any language, as far as I've seen. I've used parsers like ANTLR and yacc and they're no more complex than the DSL setup you have to do in Ruby.

Selenium works fine with Java.

I won't comment on the community or REPL, but as far as I can see, the language itself has little to do with the ease of developing a functional test suite. Setting up a good test framework (jUnit 4, DBUnit, and a mocking framework) may take work to set up, but I can tell you from experience than you have to do exactly the same work in Ruby.

Sai Venkatakrishnan said...

Ok.. People called me crazy when I used DI in functional test but using Antlr... Please read the post that it is about creating functional test of web application or something of similar. Am not against static languages or scala or groovy (other than the fact that I haven't mentioned them in the post).. It is just that I don't want to invest so much when I am writing functional tests.

And there are my views formed after working on both the sides. Feel free to contradict but don't make the assumption that I am not well informed :).

Chris McMahon said...

What odd comments!

The critical phrase in the post is "testing the web application or REST interface".

When testing such public interfaces, all the typing and discipline of building the interface is over.

Now we need to be very flexible and exercise these interfaces in the most interesting ways we can think of.

If you get a new idea for a test, it is a simple matter in a dynamic language to make the idea real. A strongly typed language requires many, many levels of abstraction in place to make such a thing possible.