Archive for the ‘Clean Tricks’ Category

The Draw Test

Tuesday, November 26th, 2013 by The Director

In North American football, the offense sometimes runs a draw play, which is a delayed handoff that gets the moving parts of the defense to act before the offense turns the ball over to its running back:

The idea behind a draw play is to attack aggressive, pass-rushing defenses by “drawing” them downfield. This creates larger gaps between defenders and thereby allows the offense to effectively run the ball. Draw plays are often run out of the shotgun formation, but can also be run when the quarterback is under center. These types of draw plays are sometimes referred to as “delayed handoffs”. The running back will most often run straight downfield through the “A-Gap” (the space between the center and the offensive guard), although there are more complicated variations.

Now, in our deadline-driven software development world, we often find ourselves typing and clicking as fast as we can to get as much test coverage as we can in our allotted time. In a way, we’re playing to software that’s trained to be aggressive linebackers, running toward our software’s operations as fast as possible. Kinda like I started charging into this metaphor too quickly and find myself off-balance, unable to elegantly segue to my point.

Within applications, you can sometimes find defects by delaying actions. If your software integrates with other software or services asynchronously, the expectations of the code might not match how fast–or slow–a user might interact with the software.

For example, a shopping cart piece of e-commerce with distinct units might mark those units as Hold, Wait For Payment if a user adds it to the shopping cart and starts the payment process. If the hold has a timeout associated with it–an expectation that, if the user does not complete the payment within five minutes (or two minutes, or twenty minutes) that the payment has been abandoned and the system should remove the hold. However, in the case of paying with PayPal, the PayPal window itself can linger for that duration and more, ready to accept your login and confirmation of payment, at which time PayPal will send a payment received message to your system. For an item whose hold might have been relinquished and might not be available any more. How will you know unless you hold onto that ball for a a couple of seconds and let all the integrated software make their plays?

Another example: Financial software that uses two stages of authentication. The first time you log in from a browser, it prompts you for your user name and then directs you to a security question; after you answer that correctly, it prompts you for your password. If you log in and don’t do anything for ten minutes, your session times out. But what happens if it takes you longer than ten minutes to answer your security question? What happens if you answer the security question but don’t enter a password in those ten minutes?

The draw test is a valid line of testing because your users don’t live to fill out your forms. In their real lives, they’re called to meetings, go to lunch, minimize the browser window when a co-worker comes in, or otherwise temporarily delay their activity in a fashion that might put their next actions beyond your software’s expectations. If you take a little time, you can identify the places where your system might allow this and to make sure your software handles delayed action on your user’s part elegantly.

Somebody’s Mama Made The Never Happen

Friday, September 20th, 2013 by The Director

Long-named US woman celebrates government climb-down:

A US woman has won a battle to have her full name put on her driving licence.

Janice “Lokelani” Keihanaikukauakahihuliheekahaunaele’s name is so long – containing 36 letters and 19 syllables – that it would not fit on the documentation.

What is the lesson here?

The obvious lesson, of course, is that you need to consider names other than Mike Smith when building your applications. Of course, we’re pretty much past that now given our more worldly software development culture (people who build and alter government computer systems not included, of course).

But you’ve got to test those long strings going into the system, and you’ve got to test those long strings coming out of the system.

If you’re displaying someone’s name, does the name lie over other design elements on the page or, urk, the mobile app? If it’s printing out somewhere, is there room on the page or in the PDF for it?

You’ll never know until you try. Your developers will never know until you try, either, because they won’t try it themselves.

(Link via Tweet.)

Decoding the Credit Card Number

Monday, January 9th, 2012 by The Director

If you work with credit card processing, this graphic will explain how the credit card is derived.

You can use it to better create test data or whatnot.

Your Placeholder Text Leaks Out

Friday, March 25th, 2011 by The Director

As careful as you are, well, no, strike that: as careful as the non-QA elements in your organization are, some of your test data and placeholder text are going to leak out into the real world.

You have to choices: Encourage your careless team members (and your QA staff) to use relatively innocuous placeholders, or expose your users to lorem ipsum ad nauseum.

The Worst I Ever Got Was A Shower Caddy

Friday, January 7th, 2011 by The Director

Other testers recount their experiences with finding themselves in test data found in production:

  • James Bach has a vanity license plate with TESTER on it and got a ticket as a result. Word is that this used to happen a lot with plates with NO PLATE or NONE on them, too, since the cops wrote this on tickets given to those cars and HQ added an extra fine for not having a plate.
  • As an overcorrection, Eusebiu Blindu (aka testalways) found that his hosting provider blocks the keyword test in certain circumstances.

As you can well guess, I insist upon testing in production whenever possible, and when I was working for an interactive agency, I entered the contests as Brian Tester with an e-mail address with the interactive agency’s domain and with the agency’s address. As a result, I, I mean Brian Tester won the consolation prizes in several of the CPG client’s giveaways. This meant coupon packs and the occasional free sample. The biggest prize I ever got was a shower caddy, a piece of plastic you could doublestick-tape to your shower to hold the client’s product bottles and useless for anything else.

Ha, ha!

These amusing anecdotes aside, what can one do to try to prevent this?

  • If possible, remove your test data. If you can delete your test record as part of the test, do it. In many cases, though, you don’t have a delete function or direct access to the database to delete.
  • Standardize your test data for production. Use the same surname or address whenever you test, for example. This provides a key to use to identify and scrub that test data whenever your organization needs to do that test data. You could use your company’s address, for example. If you can’t find a single field you can be sure is 100% unique and that users will not actually input, use a composite built of several fields.
  • Understand where the data goes. A lot of times, other companies and organizations are getting dumps of your application’s data for other purposes, such as contest administration. Even if your team knows to scrub for your data, these other organizations might not. And suddenly, you’re on the top 10 most wanted list, Bob Tester. Sorry.
  • Communicate with data consumers. Once you know where the data is going, you can tell them about the standard form for your test data and warn them to scrub it or handle it.
  • Document your test data. As part of any data schema or spec document that should get passed onto other data consumers, especially the ones about whom you won’t know or with whom you will not communicate, include details about your test data standard. Include it in code comments. Stick that note everywhere as a warning.

Following those steps, if you can, might help prevent the problems that happen when some program takes your test data seriously.

Banish Lorem Ipsum

Tuesday, September 21st, 2010 by The Director

In case you’re looking for a little Hamlet test of your own or need fill copy for a design, you can use Fillerati to create long strings from various literary texts.

(Link seen in the Twitterverse. Sorry, I forgot who betweeted it.)

Testing E-mail Addresses The Right Way

Wednesday, March 25th, 2009 by The Director

Joe Strazzere offers a list of tests for e-mail addresses.

Go, learn, and do. Don’t make me find them for you.

Designers Lose One Excuse

Tuesday, July 15th, 2008 by The Director

As you know, readers, I’m a fan of paying attention to the status bar in Internet Explorer, ever vigilant for the icon that indicates a scripting error on the page. The QA people at GoDaddy explain how you can do the same in Safari 3.x, almost, which means your team’s designers and hipster developer have one less excuse not to notice these things before you do.

Of course, it does not address the reason, which is the inherent self-righteous sloth these people exhibit, but that’s another matter entirely.

Reminder: Test Month Fields with September

Monday, April 28th, 2008 by The Director

A posting at DailyWTF reminds us that we should always test month edit boxes with September, especially when the user can enter 09:

See, Javascript supports octal numbers. Any number starting with a zero is octal, even if it can’t be an actual octal number. In certain languages, like Perl, trying to use a non-octal number as an octal number results in an error. In other languages, like Javascript, it silently fails.

This problem can also occur when the user selects the month from a drop-down list and converts it to a number for database purposes, so you should always test with September at least once in your testing.

Keeping Your Test Data Clean

Monday, September 24th, 2007 by The Director

There are many things one can learn fromĀ  this story:

Symantec Corp.’s early-warning system gave its enterprise customers a brief scare late Friday when it erroneously sent an alert that said an Internet-crippling attack was in progress.

However, the one I want to tease out is this lesson: always keep your junk data clean and readable and clearly defined as test data.

(more…)

Opening the Raincoat on Flash Version Testing

Wednesday, September 5th, 2007 by The Director

The cool kids like to use Adobe Flash to build rich Web applications because if you build it in Flash, you don’t have to worry about compatibility with Web browsers. Oh, who am I kidding, they like to use Flash because it offers a bunch of effects out of the box and it does some right purty things with animation and pictures. It allows them to put forms and data submission stuff right into the flash really fast, too, often without any data validation, but that’s another story.

No, today’s lamentation is about how those cool kids build Flash animations or applications using the latest version of Adobe Flash love to use the latest gimcracks and whatnot and how people who have a previous version of the Flash player installed with their browsers sometimes don’t get to see and marvel at the genius of your design team. Sometimes, they will see nothing at all, which is very, very bad. And you know, some browsers/computers ship without Flash installed in the default browser, right? So what happens when that user tries to hit the page, he gets no message and no warning, which means he thinks the site is defective. Because it is.

(more…)

Screenshots: A Primer

Tuesday, July 17th, 2007 by The Director

Screenshots are the QAer’s best friend, gentle reader. Appended to a defect report or printed on a color plotter at 20″ by 30″ so you can wave them two-handedly in front of the developer, they can prove conclusively that something very wrong happened with the application. Because otherwise the developers will fix something and claim it was never broken in the first place, or the humidity within the office will change, altering the flow of electrons through the network cable, producing different results when you try to recreate it, or something. It’s always something.

So, at any rate, a good screenshot adds a lot of value to a defect report or other communication.

(more…)