QA Hates You

I was trying to use the online interface for one of my credit cards and wasn’t doing anything too naughty, I thought, when I got the Java Null Pointer Exception:

Click for full size

I only use this application once in a great while (this was my second try with it), and I only spent a couple minutes with it as a user, not as a tester. Yet, here it is, an asplosion.

Am I just talented, or does the world of applications just suck that badly?

Whenever I’m planning or estimating for a financial application of any sort, I always plan for weeks of testing. Then, I don’t get called back. No doubt development shops the world over can find lower bidders (or none at all!)

And here you go.

Jeez, I spend so much time trying to think of clever QA songs that I ignore the obvious Iron Maiden.

I should get more of them into the QAHY rotation for you since they’re sure in rotation in the QA lab.

Personally, I am a font of trivia. The guy who wins a lot at Trivial Pursuit. The guy who drops unbelievable factoids into conversation. And I’m just the sort of guy who sees that trivia as an important part of software testing.

Trivia, little known bits of knowledge, represents a particularly squirrelly test condition for a lot of applications.

Take, for example, this Forbes application, a map of the United States by county.

Ignore the obvious error that crops up:

Click for full size

Well, if you’re testing the Web site, log it, but it’s not germane to our discussion (although it does further illustrate how Web sites run with errors obvious to testers even though one suspects someone looked at the Web site before launch).

Now, if you’re a resident of the St. Louis, Missouri, area, you might know that the city of St. Louis back in the late 19th century decided that it didn’t want to waste its tax dollars on the farmland around it and officially removed itself from the county. As a result, St. Louis County does not include St. Louis City, and in the beginning of the 21st century, the city wants access to the tax revenue of the metropolitan area and every couple of years tells the county that it would like to reconcile.

Here’s the county represented:

Click for full size

That looks like it runs completely to the Mississippi River. Have they forgotten the city?

Kudos to them: they have not. Mouse around enough and you’ll find:

Click for full size

Kudos to the developers of this application for knowing about the split. I would made the designer capitalize City to be consistent with other labels in the app, though. Note that the lines on the map are not supposed to be from the city of St. Louis but from a previous click on some county in Ohio.

Sure, you might say that this city/county split is basic knowledge that should be tested, and it is basic knowledge to most people in the government of St. Louis. But to someone in Massachusetts, it’s trivia. So the more regional basic knowledge you know from outside your region, the better.

What sort of trivia is best for testing? Probably not old baseball players and their averages. But geographical trivia and date-based trivia could be handy. If your software is international in audience, any insight into those facts of that foreign nation–trivia to most, but daily life to your users– could give you avenues of testing.

This last calls into mind what Mark A. said about a recent post about a children’s book and Mother’s Day:

The calendar is fine as long as the book takes place in either Paraguay (May 15) or Costa Rica (August 15), at least according to Wikipedia.

If you don’t have a stock of trivia accumulated already, Wikipedia is a good place to peruse. So is City-Data.com.

What your developers know can hurt them, and you can be the one swinging that hurtin’ outlier trivia.

From A Few Figs From Thistles:

SAFE upon the solid rock the ugly houses stand:
Come and see my shining palace built upon the sand!

Here’s an uplifting song about the software development industry to start your week:

Ted Nugent is my project manager!

An e-mail from Software Test Professionals:

Click for full size

I get a lot of e-mails with action links that want me to leer more.

And, hey, while you’re at it, click on over and nominate me for a software testing professional luminary. I like to light fires, so I illuminate a lot. Indirectly.

If you’re anything like me, you use e-mail addresses for testing purposes. I make up nonexistent addresses for user creation and use one or more existing e-mail addresses that I receive in my inbox for tests where I need to review the resulting e-mail, such as a tell-a-friend e-mail or a form that elicits an automated response such as a customer service ticket.

But what happens if you put in the return e-mail address of your company’s newsletter?

In certain circumstances, when your organization composes and compiles those e-mails on its own, you might find that entering the newsletter return address in one of your organization’s other automatic e-mail generating applications will trigger an e-mail to your entire newsletter list or some other e-mail, such as an open relay response.

It’s a damn dirty trick, and you should try it on your organization before someone else does.

As a rule, your organization should make sure that the user cannot enter those sorts of e-mail addresses, but it should allow you to test using individual e-mail addresses internally.

I guess it’s hard to capture hours of dodging acrimony and ignoring defects on video.

The best part of this is that the United States Department of Labor wants you to make a video like this and send it in. You could win $1000!

I would, but the special effects budget would exceed $1000. Blood spatter packets aren’t cheap, you know.

A Rolling Stone gathers no moss.

As a reminder, friends, neither does one you throw through a developer’s glass ego.

Testy Redhead said on Twitter:

Never stop testing in production. You have to test the cloud, not just your code. Ken Johnston #bsadt

You know, that’s my motto, too. At my last full-time posting, I set my browser home pages to client sites so that I had to look at the sites by default every time I opened my browser. I ran through the sites every promotion with a basic set of regression tests. And I hounded the developers to make minor bug fixes, the stuff marked low or typo in the defect tracker, so that a low priority did not automatically mean it was resolved (wink, wink) as won’t fix without, you know, marking it so and leading to the firestorm of righteous QA indignation.

However.

We could do that because we had a maintenance budget for these clients/sites/applications. Granted, we blew through the maintenance retainer because those retainers were set based on the assumption that the maintenance contract was pure profit after having a network admin make sure the machines were patched once every couple of months.

The point is, when your organization writes up its maintenance contracts, you should push to include as much QA and bug fixing time as you can to make the testing in production and the bug fixing palatable to the organization.

Otherwise, if the client writes a check and you’re done at product launch, you can test all you want and find a bunch of problems, but the rest of your organization will toussle your hair, chuck you on the chin, and ignore the whole business.

Bug Forces Google To Drop Homepage Art:

Due to a bug, Google on Thursday cut short plans to have rotating artwork on the search engine’s normally stark homepage for 24 hours.

Starting at just after midnight Eastern time, Google traded its usual white background for rotating photographs of the works of Dale Chihuly, Jeff Koons, Tom Otterness, Polly Apfelbaum, Kengo Kuma, Tord Boontje, and others.

The idea was to promote a personalization feature launched last week that let users choose their own background from photos on their computer, stored on Google’s Picasa photo service, or from a free Picasa photo gallery set up by Google.

However, after less than 14 hours, Google decided to go back to its traditional homepage because of a “bug,” Marissa Mayer, VP of search product and user experience at Google, said on Twitter.

As Joe always says, maybe they should have test more.

I have an old timey cellular phone. It’s not in a bag, but it doesn’t have a camera, Web browsing capability, or the ability to receive pictures.

Which means when someone sends me a picture, I get a message that I should go to a computer and follow the instructions to see my pix or my flix:

Click for full size

All right, I got to a computer and enter that URL, all prepared to follow instructions and, frankly, fearing what picture might await:

Click for full size

And I press ENTER to find:

Click for full size

Great. They changed the Web site, but not the automated text message.

It’s a lot to keep in mind, every reference to every URL your site has used. Instead of keeping track that way, wouldn’t it be easier to just use redirects in this case? That way, your non-iPhone using customers (which would be all of them, hey, Verizon?) can still use your system.

After 30 minutes on the phone with tech support, I could navigate through the busy and slow Verizon Web site to find a picture of….. lilies. Lilies I planted at my former and yet unsold home in St. Louis, Missouri. The lilies are doing very well. Better than Verizon’s Web site.

You know, gentle reader, that I like to click the View as Web page link that I see in e-mail campaigns to see how they bollix it up.

You know, by including the link to see the Web page representing an e-mail as a Web page or including unsubscribe links when the static Web page that all viewers who click through see. Or to fail catastrophically like the guys at The Web Corner do with their Ace Hardware e-mails.

Here’s my e-mail as displayed as a Web page:

Click for full size

Huh, it’s got my e-mail address in it, and I don’t see my e-mail address in the querystring. What I do see is a couple of numbers. And if I change one of those numbers….

Click for full size

Hey, that ain’t me.

Okay, guys, what lessons have we been shown today that they will not learn this time, either?

• Maybe you should check to see if someone has rights to see something if they alter the querystring by changing an obvious parameter.
• Test your view as Web page option in your e-mails to see if maybe you’ve messed it up.

That’s a provocative title, and I’m not really going to make any serious arguments pro or con.

Instead, let me point you to this story where an upgrade to military software gone awry knocked out some of the United States military’s GPS capabilities:

A problem that rendered as many as 10,000 U.S. military GPS receivers useless for days is a warning to safeguard a system that enemies would love to disrupt, a defense expert says.

The Air Force has not said how many weapons, planes or other systems were affected or whether any were in use in Iraq or Afghanistan. But the problem, blamed on incompatible software, highlights the military’s reliance on the Global Positioning System and the need to protect technology that has become essential for protecting troops, tracking vehicles and targeting weapons.

“Everything that moves uses it,” said John Pike, director of Globalsecurity.org, which tracks military and homeland security news. “It is so central to the American style of war that you just couldn’t leave home without it.”

The problem occurred when new software was installed in ground control systems for GPS satellites on Jan. 11, the Air Force said.

Officials said between 8,000 at 10,000 receivers could have been affected, out of more than 800,000 in use across the military.

In a series of e-mails to The Associated Press, the Air Force initially blamed a contractor for defective software in the affected receivers but later said it was a compatibility issue rather than a defect. The Air Force didn’t immediately respond to a request for clarification.

The Air Force said it hadn’t tested the affected receivers before installing the new software in the ground control system.

I used to think that the military and healthcare organizations were two shining spots of absolute quality when it came to software. Yea, even I had illusions that are dropping away.

(Link seen on Instapundit.)

A bit of filk for you this morning. Filk, of course, is science fiction and fantasy folk music. This song combines two of the luminaries in the field, Julia Ecklar and Leslie Fish, putting Rudyard Kipling’s “Hymn to Breaking Strain” to music.

Have I used this before? If so, consider this a regression post.

You know, if your feature isn’t going to work right, maybe you shouldn’t use it at all.

Case in point: I received an online computer form, as in a real form and not an Internet form, from my real estate agent because apparently my real estate agent thinks this is a better idea than e-mailing me a PDF. At any rate, if you view the site in IE, it looks like this:

Click for full size

No problem, right? Ossum! Now try it in Firefox, and by “try it,” I mean look in it in Firefox by default because you’re in the tech world and don’t use IE as a browser except to test. Now:

Click for full size

Hey, it’s our old friend NaN.

Do you suppose this application is peeing all over itself because I’m have professional Adobe Acrobat installed on my machine or because it just sux?

Either way, that makes an interesting test case for auto-detection of plugins: What if the user has the professionals’ version installed? What, you don’t think a professional would dare visit your site?