I really like the three ways he says of driving the design and testing the app
- Doing heavy amounts of unit testing
- Using white boards and charts to draw, discuss and distill the knowledge
- Use large amounts of manual exploratory testing
But I have reservations about a few statements he makes about automating acceptance tests and following are my views on that.
I think the name ATDD is bad or it doesn't convey its actual purpose. I don't know how the practice of ATDD started but I never think it as an extrapolation of TDD as Marick mentions. I see TDD, ATDD and ET (Exploratory Tests) as different things which help us in three different ways. There may be some overlap in the function of all three but when compared to the primary purpose I think this overlap is minimal. ATDD is not an extrapolation of TDD as TDD is not about unit testing.
For me TDD drives how classes collaborate,which class takes up which responsibility. Exploratory testing is completely about using the power of human minds to creatively explore and test the application (Not just application. You can explore design, architecture, pretty much anything). Acceptance Test Driven Development for me is articulating and crystallizing the domain knowledge we need for our application.
For me these three methods are different and they serve different purposes. It is incidental that TDD and ATDD are also used as tests but that is a secondary effect. And I feel that we need all three kinds of tests in a project.
ATDD drives design of your application domain not majorly in a technical point of view as TDD does. It helps more in concentrating on the domain, DDD and ubiquitous language. It helps in making the domain rules clear and captures the knowledge as an executable verification mechanism. Doing this helps in preventing defects (which is much more powerful) than finding one at a later stage. So expecting acceptance tests to drive design from a technical stand point may not be most profitable one. They do help in separation of layer and a few other design points but they help more in domain based design.
It is possible to do this by drawing diagrams on a white board and discussing with domain expert and customer. We do this in ATDD also but after we discuss where do keep this knowledge? How do we stop the domain ambiguity creep? I think it is good to crystallize the knowledge as test more than keeping them as text in wikis (For me English is always ambiguous so I prefer tests).
Another important question is, after we develop a complex business logic how do we regress it at a later stage if we want to know it is working fine. None wants to draw these drawings while developing a feature and throw it away after we develop. We rarely develop software where we don't revisit what we have written a few iterations back. How do we regress? Unit tests help you but how do we know if the domain logic and business rules are working fine. Doing ET for this is not a great way to approach the problem. I like to explore features which are being developed and not do the same feature exploratory testing again and again. ATDD which are also tests help us in this regressing.
About Acceptance tests being fragile and cost more in maintenance. It may be because we are approaching it wrong. TDD costs more if we approach it wrong. The problem may be we are trying to write the tests from a layer of app from which we shouldn't be doing it. I never prefer to write acceptance tests driving from UI as I feel we are using a wrong layer(I do have some end to end UI tests in Watir). And as Lisa Crispin said acceptance tests also need care to be alive and working all the time giving us valuable feedback.
So from my point of we need all three forms of tests as they take up three different kinds of roles. I also wish we can change the name ATDD as people may misunderstand that just because it has a TDD embedded in it, it may be an extrapolation of it. I like the names Executable Specification or Examples.
Conclusion I feel ATDD does add value provided you set the right expectations for it and approach it right. Don't expect it to help you in a way it is not meant to be or may be you are looking for something else.
Last but not least if you want to know how to use acceptance tests to achieve fast feedback please attend Elizabeth Hendrickson's Acceptance Test Driven Development workshop. She has a wonderful way of demonstrating this and I learned a lot from her.
Needless to say I always welcome your views and thoughts. If you want to discuss more please add it as comments :)