Fork me on GitHub

Thursday, July 31, 2008

Ruby: A simple tracer code to track method calls

I was working on a small piece of code and there was need for a tracer code which can find the method being called, in which order and the arguments passed into the method and the value the method returned. The methods were public instance methods and I did not need an heavy or fully functional tracer. So I spent sometime to write a small tracer for myself and give myself a small exercise in metaprogramming. Below the code which came out of it... There may be better ways to do it and I am learning them as I go along my journey of software development and testing code... Pretty long journey I suppose :).

module Tracer
def self.included(klass)
instance_methods = klass.public_instance_methods(false)
instance_methods.each do method
def self.hook_method(klass, method)
klass.class_eval do
alias_method "old_#{method}", "#{method}"
define_method(method) do *args
puts "#{method} called" + " with #{args.join(',')}" unless args.nil?
value = self.send("old_#{method}",*args)
puts "#{method} returned #{value}" unless value.nil?
variables = []
self.instance_variables.each do variable
variables << "#{variable} = #{self.instance_variable_get(variable)}"
puts "instance variables - #{variables.join(',')}"
return value

class Person
def greet(name)
@name = name
@greeting = "hello"
return "#{@greeting} #{@name}"
include Tracer

Monday, July 28, 2008

Tools in Agile projects

Last week I read a paper written by Kent Beck for Microsoft about the usage and impact of tools on Agile development project. On reading that paper I realize that how ubiquitous tools have become in Agile development that we don't realize them as a tool any more. Tools used agile projects are not heavy and bloated but tools which are lightweight, which can support frequent change and tool s which can support transition across different activities in short time. These tools will not hinder the developer from doing the core work but still assist him effectively to do the day to day work.

Agile projects use simple tools which when seen alone without the corresponding process and principle can seem extremely simple. Junit when standing alone is extremely simple which could be written by any good developer, but when combined with Test Driven Development it gives the developer the power to feel confident about the code he has written. Continuous Integration has no meaning alone without tests, but in short iterations with tests and verifications, it gives us the health of the project as a whole on an hourly basis.

All the tools used in Agile are like that for example cards, junit, burn down charts, open source tools. Tools chosen and available in common area, tools which can be put together easily and also changed easily. But when combined together and used with the process they give an enormous power to guide a complex and dynamic project towards success. Using bloated commercial tools in Agile project are a sure recipe for disaster as these tools are change and transition resistant and can't be redesigned for a different purpose than the one for which they are built. Tools which are simple to use and allow change and quick transition are the ones which we need to seek. They may have disadvantages but they also help us to stay focused on the job we have at hand rather than spending time fighting with tool for its configuration.

Sunday, July 27, 2008

Announcing Schnell release 0.2.0

It has been a busy week for me. Despite that I managed to pull together the next release of schnell. This release has a lot of new features and lots of refactoring done. There is still lot of work to be done but I believe in moving step by step or as in Agile release by release to the goal.
The latest release shows that you must be able to run around 1000 tests in a minute roughly (All the developers are staring at me for not letting them go to a coffee break during the build time :0).

New features in this release

1) Addition of collections like images, links, text_fields, buttons, etc...

2) Addition of non control elements like span, div, pre, etc...

3) Addition of text area (oh my god!! forgot to add hidden.. No issues will be there in next release)

Major refactorings are done in this release. But still a lots more to be done in so I have added a Todo file to keep track. As well I am planning to add it to the issues also to make it effective. Also a few non project issues, as the code base is growing deployment is becoming an issues. So I am planning to automate it and may be share my experience with others.

Planned for next release

1) Support for nested elements (frame, div, span, area...)

2) Support for tables, rows, cells, elements inside tables.

3) Support for hidden (Left out in this release :( )

4) Support for xpath for identification needs to be worked on.

5) Lots of refactoring especially in locator... (I feel some design pattern probaby Strategy may fit there.. Any suggestions?)

6) One known defect -- I found it my self :) (rdoc is displaying twice for each file at the same time.. No time to fix now, so in backlog. Issues added)

Lots and lots more to be worked on... Happy hacking. Expect the next release in next few weeks

Friday, July 25, 2008

Thoughts on truthfulness as agile value

I read this nice post in infoQ... actually I had to dig it from somewhere as it was already lost from my reading list. It discusses truthfulness as an agile value. I have a lot of respect for agile methodology as it is the only methodology which considers values as one of the main ingredients of software development (Others being practices and principles). Practices like TDD, Continuous integration don’t make meaning when they stand alone. They need to be based on values. Values decide why we do something and why we don’t. We have daily standup meeting as a practice because we value communication, we use test driven development and continuous integration because we value immediate feedback. But values and practices are two distinct things. Because we value communication we can’t add writing 1000 page document as practice when the message can be conveyed face to face in a meeting. To avoid such unnecessary practices and help us to add new practices when we feel something is lacking we have principles. Kent Beck gives an analogy of Principles acting as a bridge between Values and Practices.

Now coming to this post actual message, do we need truthfulness as an agile value? If we see the agile methodology the practices like Onsite customer, daily standup meeting, continuous integration, collective ownership, planning game all have truthfulness as an implicit in itself. If we are not truthful about the current status of the project, the planning game and whole team will be a failure as well we can’t have collective ownership. If we are not truthful about the estimates release planning and iteration planning won’t get done properly. If we don’t value truthfulness, we can’t work in pair as we have to admit if we make a mistake and also have to accept if the other person has a better design decision than us. But making truthfulness as an explicit value helps us to keep it in context when we make decisions.

People may argue that truthfulness is needed for every project development project whether we follow agile or not. So are the other values of Agile (Simplicity, Feedback, Communication, and Courage). But keeping these values explicit helps us to keep them at front when we make decisions and add practices make them aligned with the basic principles we have. If we have them implicit we may not take notice of them as we go along using the methodology.

In a complex process like software development it is easy to loose perspective of our values when we are working. In my opinion having the values and principles as explicit as possible is always beneficial for us so that next time we make a decision we will take all these into consideration. Truthfulness has to be always in agile to make the methodology work successful but we can make it explicit to help people use it as a guiding value in their day to day work.

Friday, July 18, 2008

Oh my God!! Estimation do change

It is sad to see so many projects get shocking news that their schedule is not according to the one planned and estimated. I work as a consultant at on site in the client locations. At times I study the feasibility of automating an application and at the end normally the manager who interacts with the client from my side urges me to estimate the project accurately so that after we get the project it won’t lead to any issues after the project starts. There are a lot of things fundamentally wrong in that opinion. The first is software development project is a dynamic system and with so many unknown factors in a project it is never possible to predict anything remotely accurate. The second is the only accurate thing we know about a plan for a project is that it is bound to change. And finally and most important of all is estimation is a prediction into future. As far as I know there isn’t anybody who can predict into future accurately unless you have extremely high levels of ESP.

Are there ways in which we can make estimations accurate? What are the mistakes we are doing currently? I write this in relation to testing projects as I work mainly on them but they are general principles of estimation. As I can get my analogy easily from the environment I am currently in I am using it. But feel free to switch it to any other scenario with similar circumstances.

When we look at the current projects, the estimation is done together for all the requirements as one big task and never seem to updated at the later stage of the project. Typically they come to months of effort or worse (to the delight of the manager) to years. At a later stage of the project changing estimation is considered as a sin in a lot of companies as it will reduce your reputation in the eyes of customer. This is quiet understandable but it has to be taken into account that estimations are vision into future. They are bound to be inaccurate if we want to do it once and follow the plan till the end. As I have been working on these for sometime I try my experiments and gather my thoughts on what work and what won't. I write my observations here. These are in no way exhaustive but a small set of practices I found useful.

1) Do estimations in short iterations. if you try to estimate for longer time the mistakes we make add up together to make our estimate more inaccurate. When working in iterations, make concrete for current iteration which is starting, plan for next iteration, anything after that is a vision. It is not a problem to have a vision of what should happen but don't expect it to work directly as planned.

2) It is okay to create a release plan by taking all the functionalities into consideration but it should be noted that release plan is bound to change. After the project starts, across the iterations the project current status should be used to change the release plan for the better rather than keeping it constant and forcing people to work by it.

3) It is better to estimate for small amount of work at a time. The more work you take in hand the longer it will take to complete, longer time period will lead to inaccurate estimations. We will be back to square one where we started with a bulky estimation.

4) Estimations are typically done by some consultant and then handed on to the project manager who will use a team of developers to complete the work. Unfortunately this won’t work most of the time for the reason that there are differences in the way each and every developer works. It will be better to get the developer together and ask them to estimate as they know their work ability. Forcing developers to choose what you think as best won’t be a good choice. Discuss with them and if you don’t feel comfortable you can ask for a justification.

5) Estimating done in iterations will also help you to understand the speed of the project as well the pain issues which are hindering it. Using the previous iteration’s experience can help you to estimate better for the next iteration. You can also use this to update the release plan to make it more realistic for the moment.

Don't strive for using more accurate algorithm, estimation is always a guess. Putting a lot of effort in making it a bit more accurate won't be very useful in real time for the effort put in the process of calculating it.

A lot of these points correspond to some of the practices in agile development. But I tried these methods when I faced problems with estimation when I was working at onsite with a client and incidentally understood that they are also used in Agile. The woes I faced when we were fighting over an estimation because it was too unrealistic made me to rethink the way in which estimation has to be approached and the above points are the result of it. I am not advocating you to follow any agile methodology even though it will be better and I am big fan and follower of it. But these methods can be used without implementing agile and still work together to make your estimates accurate. Finally I urge you to understand and approach the estimation in a humane way rather than thinking that it as a mere part of a big process.

Monday, July 14, 2008

Integrate early and soon...Please....

I have been talking with various leads and managers across my practice to understand what seems to be the reason behind the delay in the delivery of projects. I work in a testing major area as a developer in test and I just wanted to understand why there seems to be an ever increasing number of delivery failures and delays? Well there are lots of reasons

1) Not following iterative model

2) Not working closely and getting immediate feedback from the clients.

3) Too much emphasis on unnecessary and bulky process instead of delivering value

4) Too much emphasis on fancy frameworks

4) But an oft missed point is final dash to integration.

It is normal for me to work in a testing alone project, where the regression test scripts for an application are built by the team. This post is for teams who work on such projects though the general principle also applies to the whole development community.

Whenever I meet a lead of a team confident of his project success, the first question I ask them is how often they integrate and run their scripts. Their answer always seems to be we have a well defined process, a time tested framework and coding standards. Let the boys concentrate on coding now. We don’t need to worry about the integration now. We will do it before we deliver. But every time I see them struggling with the code base to glue it together to make sense out of it. Haven’t we all learned the lessons that even if we have the best process in the world, differences however subtle can disrupt it and make us scurry to find the cause? Haven’t we learned that if we don’t do something we can’t be confident of it?

Even if we follow all the best practices (Personally I don’t believe practices are context dependent) in the world in the team, there are differences in way which each person would work. As well without constant feedback that what we are doing individually works together as a single entity, there is no way we can march forward towards deadline. Integrating early and often is the only option we have in projects and it need not be only development teams but also teams creating testing code. It eliminates the doubt and risk of hassles during the delivery as well helps us to deliver on demand. Whenever during the day a piece of task is complete, integrate with the existing testing code base and run it against the application. Most of the existing continuous integration servers help you to do the task. Even if you work on purely automated testing projects which I do sometime, opt for continuous integration to see if the scripts or code you are writing can work together and not just as individual modules in developer’s machine. Using continuous integration also helps us to run the scripts in a standardized environment rather than individual ones. It also helps to catch the failures that may happen when someone’s code broke another person’s work which won’t be possible in the last minute integration.

There would be an overhead of an integration machine and build scripts which the testing team would normally not prefer. But it comes at the advantage of giving us feedback about the project’s health, eliminating the big bang integration at delivery and helping us to eliminate the errors and differences as soon as possible. When you make integration a non event, it will be surprising to find how easy it is to deliver the code without hassles.

If you want to know more about continuous integration, you can get more info from the article by Martin Fowler

Also there is a book Continuous Integration, Improving software quality and reducing risk by Paul Duvall.

Saturday, July 12, 2008

Announcing release of schnell

Tests are considered a valuable feedback in any product development. Faster the feedback, better the product quality. Typically the major tools in the market or in open source space which test the UI for functional and regression testing are dreaded in an agile project because of their slow speed.

Schnell is born out of need for fast automation tools. Built around jruby and htmlunit, it uses watir like API for scripting. The installation guide, user guide as well as examples are available in project site schnell-jruby.

Initial benchmarks show that the test suite with 54 tests, 340 assertions takes around 4 seconds to run. Watir, while running the same suite runs at around 150 seconds (Not bad :)). I wonder what the commercial tools like QTP, Winrunner take to run the same.

Checkout the project and let me know your feedback. Hope it will be useful for you.
If you face any problems, please log them in issues, I will try to work it as soon as possible.

Finally, more volunteers are needed to improve the code base and add new functionality. Some major refactorings and new features are planned for the upcoming releases. So if you want to join, drop me a mail. Happy hacking

Wednesday, July 2, 2008

What are unit tests anyway?

Well I hadn't posted for a long time... It is almost 3 AM in Shanghai and I am still not asleep. What keeps me awake is the question "What are unit tests anyway?" :).

I am an ardent fan of test first programming and doing XP for past 1 1/2 years. I am writing this for people who are new to XP or TDD. These concepts are already there for long time and I am elucidating here to show my understanding.

Well lets take off.. Unit tests are

1) Obviously tests and also source of feedback, safety net, confidence builder, saviour of the project and many other gallant titles to follow. I am not going to concentrate on this because they are already covered a lot.

2) When you are testing a class they provide the client's view of usage of the class or the interaction of the class with other classes (Being mocks). This is more interesting one. It helps us in two different ways.

One is it shows if the class, its members, the parameters glue together properly. In other words by using the class in a unit test we can see if we have structured the class properly. Some of the things like proper naming of classes and methods come out during this.

The other way is it shows the distribution of responsibility in a class. This is a difficult part of object orientation. When we see a method being called on a object we see that the method is fulfilling one of responsibility. It also make us think that whether the object should have that responsibility. Do we need to move that to a new class or to an existing one?

These questions lead to a lot of refactoring and then again unit tests help us to make them safer. The second point leads us to the Test Driven Design. The unit tests are actually shaping the classes and driving their design. Thinking about the responsibility at early stage in unit tests also leads us better responsibility distribution, to Single Responsibility principle and all such good things.

I am big fan of tests and welcome tests whenever possible. But the point of tests driving the design makes the test first a more favourable one for me.

Well there are a lot of good things about unit tests.. But with sleep fast approaching I think I have written about the most important ones.

If you want to join me or contra me please add it to the comments.