QA Hates You

Scary Tester does a good bit of putting a positive spin on when it’s best to do automated testing:

Automated tests are suitable for the following purposes:
-    Regression testing for a stable system that will be run on a regular basis
-    Fast data creation in test systems where the database must be wiped on a regular basis

Automated tests are NOT suitable for the following purposes:
-    Testing new functionality – this should be done manually before automated tests are created
-    Regression testing systems that are expected to have significant user interface changes. Large changes to the user interface require a lot of maintenance for automated tests.

You know, testers make these arguments over and over again, but I’ve gone into a number of places to talk about starting QA efforts on major product lines or to work on smaller (160 hour) projects where the principal involved wants automated testing.  Usually on an evolving product and with only one QA person.  Try as I might to dissuade them, they go out and find someone willing to bill them less fruitful hours of QA work because that’s what the client wants.  And the client/employer gets it: an automated effort of some sort, a low defect count (because the QA person spent hours selecting/writing/maintaining automated scripts instead of testing.

But Scary Tester’s and my commentaries fall on sympathetic ears.  Meanwhile, Baseline magazine will run a bunch of ads from software companies selling automated testing software and amid a splashy article about how automated tests can do the work of 20 monkey testers.

I think I’m repeating myself, aren’t I?

In the January 2008 issue of Software Test and Performance (available as a PDF), the head of Parasoft, a QA software company, explains when automation efforts can go to pieces.

(more…)

This month (pdf) in Software Test & Performance, editor Edward J. Correia again takes on automated software testing. The intro paragraph led me to believe he might have become a joyous skeptic, like us:

Why, for instance, do we build software to test other software? This question has never before occurred to me, nor does it parallel such mysteries as people who are financially wealthy but short on values. But it does bear some discussion.

Does he then contemplate the possibility that trusting software to test software is something like telling criminals to police themselves? Nah, he just marvels at the beauty of it. As he should, since the automated software companies are the ones buying the ads in his magazine.

However, we at QAHatesYou.com disagree with his conclusion:

Software is very good at automating things. So when automated testing is the need, why not use the best tool for the job? For the practice of automating software testing, the best tool happens to be more software. Sometimes the best tool is staring you right in the face.

Here at QAHatesYou.com, we have found in our experience that the following are sometimes better solutions, especially when tailored to limited budgets:

• Zombies. All you need are a recurring maintenance budget, i.e., brains. You can certainly find some unused brains on your development team anyway. So raise some dead and show them which keys to push, and wallah! Automated software testing using the undead.
• Steam piston driven software appliances. All you need is a machine shop, some wrenches, and boiling water to build complex steam-driven keyboard punchers. Mouse-handling and pointing-and-clicking are less accurate, so you’ll have to work around that. Also, remember to calibrate the finger-rods correctly, or they will punch right through the keyboard instead of efficiently delivering the keyclick you want.
• Monkeys. Just kidding. We use all our monkeys for new functionality testing.

Automated software testing is really only possible through the use of software, which comes with its own hazards which I’ll go into some other time.