Exploratory Testing During Load Testing

In Connecticut, some exploratory testing types found and exploited a software flaw in lottery terminals:

An investigator for the Connecticut Lottery determined that terminal operators could slow down their lottery machines by requesting a number of database reports or by entering several requests for lottery game tickets. While those reports were being processed, the operator could enter sales for 5 Card Cash tickets. Before the tickets would print, however, the operator could see on a screen if the tickets were instant winners. If tickets were not winners, the operator could cancel the sale before the tickets printed.

It’s a condition that only occurred while the system was under processing load.

Which is why, whenever I get to do some load testing, I also like to call up the application under test and run through some basic smoke tests with it. You can find different places where resources are not available or where the load times can lead to unintended consequences–like allowing the user to click a button that renders but is hidden when the page fully loads. Or to act on data that the user should not be able to act on, as the lottery terminal displays.

Of course, you can do something like this through some network-throttling tools, but that will only really handle client-side slowdowns and problems, not necessarily issues with the server and infrastructure.

Also, it’s a way to get one more user’s worth of load on the system, and given our load testing budget most of the time, that can be a 5% increase over the 20 virtual users we have licenses for.

Comments are closed.


wordpress visitors