QA Hates You

Gimlet passes along the news story "Chrysler recalling over 280k minivans because airbags may deploy on wrong side:

Chrysler has issued a recall for some 2013 Town & Country, Dodge Grand Caravan and Ram C/V Tradesman vans built between May 10, 2012 and June 7, 2013. These vehicles may have a software error that would cause the wrong side (opposite side) airbags to deploy in a crash. With this defect, a left-side impact would cause the right-side airbag to deploy, etc.

You know, I have a lot of respect for embedded systems testers. You and I got to worry about browser and device compatibility, but we get to try these things in a number of real-world situations given that our ‘real world’ is the Web and computer systems.

When you’re testing out the embedded systems, it’s mostly testing tools and simulations. It’s not like those guys get to pop off the airbags in cars a hundred times a day and sometimes more when they’re trying to recreate an issue. Poor lads.

WebTV, now MSNTV, is going away, and not just in a press release:

Microsoft said that its MSN TV service will be closing down at the end of September, in a post on its Web site and in an email to users.

It’s not that anyone was testing compatibility of Web sites any more for it, but its users were still calling the help desks of consumer products companies whose Web sites did not support their preferred browsing method.

But before you cheer too loudly, consider how many of these people will be upgrading to recycled and donated PCs running Windows Me and IE 6. The answer will be…. more than you’d like to think.

(Link via.)

Six Sigma Wine.

No, really. I mean, no, really, it exists. But, personally speaking, I don’t so much decant as debox.

Leonard Cohen, “Everybody Knows”, featuring scenes from the film Pump Up The Volume (although the version on the soundtrack is a cover by Concrete Blonde):

One might suspect the film influenced your humble Director a lot as a young man. And one might be right.

Your designers probably want the latest cutting edge gizmos and paradigms in your applications. What do your users want? The familiar and the things they have already learned to use:

Ford Motor Co. is going back to buttons and knobs.

Punished by third-party quality reports because of the difficulty of using its touch-screen multimedia system, called MyFord Touch, the auto maker will reprise tuning and volume knobs for the radio as it redesigns existing models, a top Ford executive said.

It is a reversal for Ford, which has been a first-mover with installing mobile-phone-based technologies, voice recognition and touch screens in its vehicles. The systems have been a big selling point for Ford with its vehicles, but also have dragged down its reputation for quality.

. . . .

One of the things that bothered customers was the inability to quickly change the channel or volume on the radio through familiar knobs, he said. As Ford redesigns its vehicles, the flat control panels with add more buttons and knobs[SIC] and[SIC] the main screen will become simpler.

Users have other goals in mind with your software than building a portfolio to take to their next job interviews. In a lot of cases, they want to do the same things in the same ways they’ve always done things.

Granted, you do have to balance the same old way of doing things with any process improvement that your software brings, but far too often, design changes come from the minds of the designers and frustrate the users to no end.

Ever wonder why fast food restaurants and many less expensive department stores have very similar layouts to others in the chain? It’s because customers whose goals are getting a quick lunch or grabbing the essentials quickly, looking for the things they need (which includes the bathrooms and concerns as to whether they pay the server or a cash register by the door) mars the simplicity they crave.

Likewise, if your designers had their way, each freaking McDonalds would have a different layout, some with gerbil tubes between the floors so users could climb between the registers and the condiments stand. Because it would be different. Because it would satisfy their needs as architects.

Some stores and restaurants are destinations qua as eating experiences or shopping experiences. But your software rarely is. Nobody’s running your software to enjoy your software.

So make it easy for them to do what they do with your software, and don’t add weird or unknown design elements just to do that because your users might not appreciate it. Even on the ancillary functions, like changing the radio or clicking a button.

Man, do I think the whole “putting the control label in the control” thing is a bad design pattern for many reasons.

First and foremost, when you start typing or sometimes set focus to the control, the label disappears, and you won’t know what you’re supposed to type into the edit box. Oh, you know, your designers know, your developers know. But will your users know and/or remember?

Second, there’s what do you do with the label when the user saves the form? Does it get saved? What happens when the user clears the data from the control? Does it display again? Most of the time, at least on the rough cuts, this gets mangled. 60% of the time, it gets bollixed every time.

Third, there’s the matter of sizing and spacing the text in the edit box, which our friends at the Custom Showing Service illustrate:

In these fellows’ defense, it’s not just the label in the control that’s sized badly; even the information you type in the edit boxes is trimmed at the top and the bottom to the point of illegibility. I just wanted to use this excuse to lament the label-in-control design pattern.

In these days of super custom designs monkeying with the very nature of controls themselves, altering fonts in the controls, you’ve got to watch out for problems like this. Even if they’re just problems like this:

In the old days, we didn’t have these problems. Labels were outside the control like they should be, and the default fonts fit in the default controls. I’d say “get off my lawn,” but I’m not sure I’ve left enough room for the tail on that g.

I’m a big proponent of capturing and documenting your interface style guide that captures the common rules and spellings to use in your organization’s interfaces and communications to make it seamless for the user and to make it look like you know what you’re doing outside of C#, Ruby, and Java.

For example, canceled and cancelled are both in the dictionary. But they should not both be in your dialog boxes.

Come on, even the music business has a style guide these days:

Roll over William Strunk, and tell E.B. White the news. The music business now has its own grammar guide that might have had the “Elements of Style” authors singing the blues.

The song titles “In da House,” “Kill ‘Em ‘n Grill ‘Em” and “It’s fo’ Realz,” for instance, all get thumbs up in the music industry’s newly issued Style Guide as examples of proper capitalization.

“Intentionally misspelled words must respect the same title casing rules” as those spelled normally, says the guide.

Released last month at the National Association of Recording Merchandisers’ annual convention, rock ‘n’ roll’s new rule book aims to establish basic rules for data entry in the fragmenting world of music, where some folks have become a little too creative for their own good—at least when it comes to spelling, grammar and description.

Come on, be like Too Short. Your deadlines already are. Hey-yo!

You might think that my recent trip to San Francisco changed my life because I keep droning on and on about it, but it’s just that social media glurging is the twenty-first century equivalent of showing you slides I took (And here is a hill in the Marin Highlands. And here is another hill in the Marin Highlands.)

In the dense urban environment, I enjoyed riding in cabs, something I can’t do in the lower density environs of my native habitat. It’s just like a roller coaster, except that the route is unplanned and it carries actual risk of death at the hands of a street car.

You know what a San Francisco taxi driver is just like? Your user. And you should test your applications just like you were taking a pair of white knuckled tourists from the airport to the Sunset neighborhood. How do you do that? Funny, I have a blog post all about it.

Ignore the warnings.

If you’re brave enough, look at the dashboard of your cab. You’ll see that any number of warning lights are on. If you saw those Warning lights on your automobile, you’d limp it to the shop as soon as possible. However, in a cab, the check engine, low tire pressure, no windshield wiper fluid, flames in engine compartment lighs all these mean nothing to the driver. It’s not his car, it’s his workplace for twelve hours a day, and it’s not his problem.

Much like your software is for your professional users.

Your professional user is going to try to get from the start of the process your software tries to make easier for the user to the end. Given the demands of his or her job and what he or she is trying to do, your user might ignore flags and warnings and even error messages if possible. That is, any little warnings your software throws up, the user might try to save the data anyway, even with a weak password. Even with a six digit zip code. Even when step one fails and the user tries step two.

Sometimes, we might be tempted to see a warning or error and stop testing that avenue, instead waiting for problems we encounter to be fixed. But users might not stop when they see that error and await a patch six weeks (or six hours) from now. Test through the warnings and errors and see if there’s something beyond to log that might be as bad or worse. Also, there is a slight chance that those bugs won’t get fixed before your user gets to take it for a spin.

Hit the gas before the light turns green.

Let’s face it, today’s modern Web applications, steeped in divs and AJAX, parts of pages display before other parts, and sometimes controls display before the JavaScript controlling the actions of those controls or links loads. If you use automated testing, you know to account for this by putting in special statements to wait for the page or control to load or the element or div to be visible before interacting with it.

But.

Time is money, and your San Francisco hack is going to anticipate a light’s turning green and at the very least will let off the brake when he thinks the light is going to turn (if not actually jam on the accelerator. He wants to get you to your destination and himself to a tip as fast as possible.

So your user is going to interact with your Web site. If you put the button or link the user needs on the screen, he’s going to click it even if the rest of the page and code are still streaming through the tubes and pipes.

You’re probably already doing a bit of this as you try to run through as many scenarios as possible in the thirty minutes you have to test the Web site, but whenever you encounter a problem because you’re moving faster than the application, the application has the problem, not you. Log the problems, and don’t let your peers tell you that it will work in production or that it’s not a problem. It most certainly is.

Punch it, Margaret.

I know I’ve gone on about this before here and elsewhere, but your load testing tools have to punch it when the light turns green.

Look at how your hack drives. He floors it before the traffic signal changes, and his foot is on the accelerator until he’s about to collide with some stopped innocent bystander, at which time he’s on the brakes just hard enough to avert catastrophe.

Aside from your load testing strategy, you can punch it in manual testing. If one click of a button is good, isn’t one hundred clicks of a button better? If the maximum length of a string is 32 characters, enter 32 characters. Not to test whether you can enter 32 characters in a particular form, but to see what happens elsewhere when the string is the allowed maximum. And so on.

Test the application not only beyond its limits, but to its limits. And beyond.

Find the shortcuts and back ways, and test them.

Your cabbie knows if there’s a line of cars at a light on Pine, he can turn down Dashiell Hammett Street to try his luck on Bush. Your users, too, are going to find those little shortcuts that aren’t on the main menu and that aren’t on the sticky notes or improvised, anticipated workflow around which your developers will write the code. Users are going to find those back alleys in your modules and are going to exploit them to try to get their work done.

Since you’re spending your time, amid from the endless meetings and putting your head on your desk and weeping in soundless shudders, in the application, you’ve got to figure out those shortcuts the users will find and test them out, too.

Cruise the application once in a while.

Cab drivers can get their next fare by sleeping in their cabs until they’re dispatched (the Missouri method), they can wait in line at a cab stand at the airport or a hotel, or they can find defects, I mean customers, driving the city streets (sometimes on the way to a nap or a stand).

So you can find defects outside of your normal duties, aside from your test cases or user stories, just by cruising through your application once in a while, taking a look with fresh eyes for defects waving their hands in the air. Take a couple minutes before the meeting when you can’t start a bigger task to just hit the site. You’ll be surprised what you might find.

However, unlike a San Francisco cabbie, these are the best tips you’re going to get for testing, so make the best of them.

Phil Collins from ::cough, cough:: thirty years ago with “I Don’t Care Any More”:

The key to a long, happy career in QA is to know when to care and when not to care. Criticism, office politics, pleas for mercy? Don’t care. Missing a comma with a noun of direct address in the welcome email? CARE!

Way, way back for Quiet Riot’s “Bang Your Head” this morning:

As some of you might be aware, I spent last weekend in the San Francisco area. What did I do? Things worthy of QA, of course, and here’s what you, too, can do while visiting San Francisco and capturing a bit of the flavor of the area worthy of QA.

Eat at NaN

A restaurant called NaN? That’s just screaming for QA’s dining dollars.

Oh, but the actual QA world in San Francisco let us down. It’s NaR (not a restaurant). So no Korean fusion for you or me.

Visit Sonoma Valley for the Wonderful Mobile Testing

The Sonoma Valley, partly known for its wines, also has low to no mobile data connectivity. Did you know there’s an icon on your iPhone that indicates just cellular service? I do now. Now, what happens to my app under test when I am in a tastefully comported Faraday Cage, sampling Cabernet and adding records to the DB?

Also, there are some wineries around, which gives you the proper answer for any “Why would the user do that?” questions you might get: “It was the third winery on the tour.”

Take a Bay Cruise to Find the Null in Geolocation Testing

In at least one spot on San Francisco Bay, I discovered I was (null) miles away from Sausalito. Can you find that spot?

Granted, that was an error in geolocation in this particular app, but what happens in your mobile application when you’re on a boat?

Never pass up an opportunity to try the mobile app out in unexpected circumstances (What street address do I get when I’m flying over the building in a helicopter tour?).

Although strange and extreme, they might uncover conditions and application behaviors that would occur in less extreme circumstances.

Traverse the Golden Gate Bridge in the Fog and the Wind in an Open Car or Bus

Brothers and sisters who are not in San Francisco: Just because San Francisco is in California, that does not mean it is warm. Your stereotypes only apply to southern California.

Crossing the Golden Gate bridge in the morning in a convertible or one of those open-topped tour buses allows you to prove your toughness and endurance, and it provides a handy metaphor for the software development life cycle. You’re in a fog, you can’t see anything, suddenly something large looms over you, then it passes.

Additionally, crossing the same way on the southbound, windward side of the bridge is a particularly moving experience that will bring tears to your eyes and proves to be an endurance test not unlike a lessons learned meeting.

Walk the City. Then Take A Cab

Walking around San Francisco (or any other geographically bound and dense city) allows you to get up close to things and to view the emergence of the architecture and whatnot slowly.

Then take a cab and see how those familiar with the city and who have a particular goal in mind use the city streets and alleys to accomplish their goal.

It’s different, but they’re both San Francisco.

That’s a bit of a stretch and overt metaphor for testing, but I needed a fifth item because the title says five.

Cher, "Gypsies, Tramps, and Thieves":

Who are, by the way, working today even though it’s a holiday in the United States. Paid time off is for suckers. So we tell ourselves.

Killswitch Engage, "In Due Time":

So I took This BBC News grammar test, and I got a perfect score, natch:

And a bug, natch.

Come on, kids. One simply must check numbers greater than one digit (or more) to make sure they fit inside your little design elements.

Some days, I feel like the Dowager Countess, sitting here marking pointed remarks as I watch the old order of people caring about this stuff falling apart.

Stone Sour with “Absolute Zero”

Good day!

Tim McGraw, “The Cowboy in Me”

The urge to run, the restlessness
The heart of stone I sometimes get
The things I’ve done for foolish pride
The me that’s never satisfied
The face that’s in the mirror when I don’t like what I see
I guess that’s just the cowboy in me

It sounds a lot like the typical workday in software testing.

Ever get asked why a user would do that? Of course you do. You’ve already been asked that today.

Here’s a little story for you about why the ball player trying to steal third base ended up on first base:

This guy stole second. Then he tried to steal third but somehow wound up on first. Then he got thrown out trying to steal second again. All in a span of five pitches.

The result, as far as we’re concerned:

The part where a runner on second base finishes the next play on first base? It’s not possible to score that without crashing every computer in America.

“There’s no way to do that,” longtime official scorer and SABR historian David Vincent said Saturday. “Not covered in the rules. A runner on second base going to first base? That’s impossible.”

Now obviously it’s not “impossible,” because it really happened. But tell that to the computer programmers of America.

“All the computer software — none of it will handle that,” Vincent said. “You don’t run the bases [from] second to first. Any software that processes play-by-play won’t accept that.”

So because it’s theoretically impossible, the official box score of this game listed Segura as having been thrown out stealing third — even though he slid into second. Huh?

“That’s because the play-by-play listed him as staying at second base [because it couldn't compute that he was actually on first],” Vincent said. “So then he had to be caught stealing third. But that never happened. So that has to get changed.”

Right. But that’s not all. The official box score and play-by-play also said that Braun got caught stealing second.

So why would a user do that? Because the user could do that. And just because someone has not done that does not mean someone will not do that in a strange set of circumstances you cannot anticipate now.

(Link via tweet linking to this article.)

In This Moment, “Comanche”:

In This Moment is one of those bands where you might mistake the band’s name for the song title. But I assure you In This Moment is the band’s name in this case.

A little story to talk about some of the ways our inattention can lead us to miss some mistakes.

For years, I’d heard about an awesome British television program that, although it aired on PBS and had not reached popularity among mainstream Americans certainly got its discussion amongst the tech crowd which tends to favor BBC programs that air on PBS. The show?

Downtown Abbey

Or so I thought. I had seen discussion of the program on the Internet, as I mentioned, and and I watched the introduction to it seven times, and I saw it on a Roku menu many times. I even started a Twitter hashtag called #DowntownAbbeySpoilers to demonstrate my particular brand of wit and to flaunt my ignorance before the Internet. It was only in the seventh episode, which includes a scene at the Downton train station clearly labeled as such, did I discover my error.

I was chuffed, to say the least.

I’d read the word as Downtown at some point, maybe a misspelling or a bad pattern matching on my part, and every time I saw the title, my brain filled that Downtown instead of Downton. Every blooming time.

I started planning this post in my head, I encountered a misspelling in the application I’m testing. Although I’ve hit the screen a number of times, I’d missed the misspelling every time I saw the page previously until just yesterday, when I happened to look at the extraneous letter in the middle of the word. Other times, my eyes had skipped right over the defect like a smooth stone over still water.

The way to avoid these types of oversights, as I always have said, is to slow down and to look at everything with fresh eyes as often as possible. Take some time to read every word and to test every control in different combinations. Focusing on the higher-level functions of an application is very important, but focusing on individual parts of it at a very atomic level will find a number of problems that your users will eventually find if you don’t (and fix) first.

Now, onto the second error: trusting the context.

My UK readers and many of the Downton Abbey fans here in the US and elsewhere might be chortling about what I said above: I was chuffed, to say the least.

It sounds like a combination of chagrined and maybe huffed, doesn’t it? You’re forgiven if you assumed it meant something like that from context. But chuffed means pleased. I was not pleased at all.

This kind of oversight comes when you trust the context of something, generally in terms of application behavior. When you click that button, of course it does that. It’s not crazy enough out of whack to be an obvious bug, so you let it go. This happens a lot in two cases: if you’re testing something in an industry or vertical in which you’re not already steeped, like counting DIDs for a telecommunications company’s order scheduling software. If the number of DIDs doesn’t match the number of telephones and never has, will you notice it as long as you can save the order and make sure it displays properly on the technicians’ tablets? Maybe not.

The other place where you can get into that sort of trouble is after you’ve tested an application for a long enough period of time that its evolution gets a little mixed in your mind. Did that used to work? Did it used to do something different? Should it do something other than what it does?

To try to minimize these style oversights, don’t trust the context. Try to learn why the context is the way it is. If something seems strange, ask. And if the person you asks says, “Just because,” “Trust me,” or “It’s always been that way,” open your defect tracker. Because QA has always been that way, just because. Trust me.

So the lessons are, again, slow down and learn as much as you can while you’re testing something to test it most effectively, and not to trust yourself or your context because you’ve been wrong before. Or at least I have. AND I AM NOT CHUFFED ABOUT IT.

QA shares a lot of qualities with the popular conception of the diva. We’re always demanding attention, and when we have attention, we make demands.

Researchers who have studied divas in the wild recognize two kinds: healthy divas, who aren’t unhealthy divas, and unhealthy divas, who are the divas who’ve built the popular conception of diva.

Regardless, the Wall Street Journal recently featured an article on the difference:

Divas, by definition, are high-performing, high-maintenance narcissists. Some are needy, demanding, negative—and talk almost incessantly about themselves. Researchers say these are “unhealthy divas” and the source of their narcissism usually is low self-esteem: They are constantly trying to pump themselves up.

Yet, believe it or not, researchers say some divas are healthy. They adore the limelight and work hard to be always front and center—but they are willing to make room for others. They are spirited, fun and positive. Because they assume everyone around them is interested in them, they share a lot of themselves—and in this way bring people together. They have the ability to help others enjoy things that aren’t normally enjoyable, whether it’s a long line at the store, an office meeting or dinner with the boss.

Within the article, you’ll find a number of differences delineated between unhealthy tantrums and expecting and settling for nothing less than the best because, baby, you deserve it (and so do your users).

Also, to prove its worth the the QA community, please note the Van Halen Brown M&Ms story appears, so you know it’s got to be True (much as things on this blog are, as I mentioned the story last year).