The Myth of the Automatic Automated Benefit

Scary Tester does a good bit of putting a positive spin on when it’s best to do automated testing:

Automated tests are suitable for the following purposes:
–    Regression testing for a stable system that will be run on a regular basis
–    Fast data creation in test systems where the database must be wiped on a regular basis

Automated tests are NOT suitable for the following purposes:
–    Testing new functionality – this should be done manually before automated tests are created
–    Regression testing systems that are expected to have significant user interface changes. Large changes to the user interface require a lot of maintenance for automated tests.

You know, testers make these arguments over and over again, but I’ve gone into a number of places to talk about starting QA efforts on major product lines or to work on smaller (160 hour) projects where the principal involved wants automated testing.  Usually on an evolving product and with only one QA person.  Try as I might to dissuade them, they go out and find someone willing to bill them less fruitful hours of QA work because that’s what the client wants.  And the client/employer gets it: an automated effort of some sort, a low defect count (because the QA person spent hours selecting/writing/maintaining automated scripts instead of testing.

But Scary Tester’s and my commentaries fall on sympathetic ears.  Meanwhile, Baseline magazine will run a bunch of ads from software companies selling automated testing software and amid a splashy article about how automated tests can do the work of 20 monkey testers.

I think I’m repeating myself, aren’t I?

No Responses to “The Myth of the Automatic Automated Benefit”

  1. scarytester Says:

    It’s really difficult to explain this concept to managers without getting the “why don’t you want to do your job, you lazy tester?” kind of response. A lot of managers seem to want automated tests simply because somebody told them that it’s what all the cool companies are doing these days. Then they want all tests 100% automated.

    Drives me batty because it’s so hard to explain to people who know very little about testing or automation. I keep trying to think of an appropriate metaphor for it but there just doesn’t seem to be one. It’s even worse when the person concerned thinks that testing is a completely clueless job where you just follow exact test steps like a drone all day. If you think about it, it’s pretty insulting when someone implies that the work you do can be replaced by a couple of Selenium scripts.

    But yes, this kind of rant is exactly why I wrote the article in a more textbook-style positive way, because ranting generally just gets you labeled as “negative”, whereas stating facts like a 6th grade science project sometimes makes people listen. Which is a shame, because ranting is way more fun.

wordpress visitors