When Automation Goes To Hell….Plus, a Pipe Dream

In the January 2008 issue of Software Test and Performance (available as a PDF), the head of Parasoft, a QA software company, explains when automation efforts can go to pieces.


Once QA has the new version in hand, they try to run the old regression test suite against the new version of the application. It runs, but an overwhelming number of test case failures are reported because the code has changed so much.

At that point, QA often thinks, “Instead of trying to modify these test cases or test scripts for the new version, we [sic] might as well go ahead and test it by hand because it’s the same amount of work, and even if I update it now, I’ll still have to update it all over again for the next version.” So they [sic] end up testing by hand, and typically come to the conclusion that automation is overrated.

Perhaps he did mean to apply QA was schizophrenic in referring to itself in two different tenses in its own thoughts, neither of which matched the singular antecedent “QA.”

Regardless, his point mirrors one that I alluded to in Sometimes, Automated Testing Is Folly. That is, sometimes the maintenance hit for an automated testing suite outweighs its perceived benefits.

However, Adam Kolawa offers this solution:

One is that team leaders must allocate sufficient budget and time for regression test suite development and maintenance. I’ve found that the best results are achieved when there’s roughly a 50/50 distribution of effort between writing code that represents the functionality of the application and writing code that verifies that functionality. [Emphasis mine]

I’ll pause here until such time as you pick yourself from the floor in shock or in laughter.

Perhaps that is a good mix, and perhaps it does make sense, but good luck on convincing your team stakeholders and pursestring and hourstring holders that you need one QA for every developer.

If you find that promised land of coffee and doughnuts, send me an application.

Comments are closed.


wordpress visitors