QA Hates You

Motorcyclist survives lightning strike while riding on freeway:

A motorcyclist riding on Interstate 5 survived a lightning strike Thursday as a tumultuous day of weather saw thunderstorms and rain roll through Washington on both sides of the Cascade Mountains.

The biker was riding through Chehalis in western Washington when the lightning hit Thursday morning, Washington State Patrol Trooper William Finn said.

Remember, remote chances of a bad thing occurring are not the same as no chance, and you need to assess your risk accordingly. Individually, people have a very remote chance of winning the lottery, yet someone wins the lottery every couple of weeks.

Make sure when assessing your risk, calculate in the severity of the problem and the number of users mashing the keys in addition to the remoteness of the possibility of that happening.

The first step on software testing appearing in Diagnostic and Statistical Manual of Mental Disorders 6: a scientific-sounding name for the QA mindset: negative dispositional attitude:

New research has uncovered the reason why some people seem to dislike everything while others seem to like everything. Apparently, it’s all part of our individual personality — a dimension that researchers have coined “dispositional attitude.”

People with a positive dispositional attitude have a strong tendency to like things, whereas people with a negative dispositional attitude have a strong tendency to dislike things, according to research published in the Journal of Personality and Social Psychology. The journal article, “Attitudes without objects: Evidence for a dispositional attitude, its measurement, and its consequences,” was written by Justin Hepler, University of Illinois at Urbana-Champaign, and Dolores Albarracín, Ph.D., the Martin Fishbein Chair of Communication and Professor of Psychology at Penn.

Realism is an illness.

I was alerted via Tweet about a software test engineer getting noticed for his pipes:

It took only 45 minutes to change Michael Wansley’s life. The Seattle-based singer, known by his stage name Wanz, has found himself at the top of the charts with his bass-heavy credit on Macklemore and Ryan Lewis’ “Thrift Shop,” which has now enjoyed two weeks at No. 1 on the Hot 100.

. . . .

Now, the former software test engineer is taking a shot at solo stardom. Having toured the country with Macklemore and Ryan Lewis over the past few months and performed with them on “The Ellen Show,” Wanz is building his brand on social networking sites and recording a six-track EP that he hopes to release before he joins the pair on their Australian trek next month.

Here’s the song the article talks about, “Thrift Shop”. I think it’s about quality assurance budgets somehow.

Warning: Hip-hop language. Might not be safe for your workplace.

If this had happened to Amazon, Google, or another company geeks love, you would have heard about it already: Computer Problems Leave Goods Stranded at New York Port:

Computer problems at one of the East Coast’s biggest ports have snarled the flow of cargo across the Northeast for weeks, delaying the delivery of consumer goods needed for back-to-school sales and the start of the holiday shopping season.

The problems at the Port of New York and New Jersey began in June, when Maher Terminals LLC, one of the world’s largest handlers of shipping containers, launched a new computer operating system, according to shipping, trucking, retail-industry and government officials.

. . . .

Maher Terminals and the operating system’s maker, Navis LLC, a division of Finland-based Cargotec Corp., said in a joint news release last week that “real-time interactions between the various system components deployed in the container yard were not operating as designed.” As a temporary solution, certain automated components of the system were scaled back, the companies said. They didn’t reveal the source of the problems.

Rest assured, it worked on a developer’s machine.

Meanwhile, even though it did not impact geeks directly, this bug had huge impact all down the logistical downstream.

An article entitled An Important Life Lesson from Blackjack and Baseball: You gain more by not being stupid than you do by being smart has this brief takeaway:

The moral: You gain more by not being stupid than you do by being smart. Smart gets neutralized by other smart people. Stupid does not.

The gist: In some competitions, you can lose on purpose, but you can only try to win on purpose, so making the smart moves won’t necessary lead you to success and winning. But making stupid moves can and will thwart you.

In software development or startups, it’s easy to fall into that bad habit of pursing something new, neat, or smart and forgetting to take care of the little things like user experience and bounding your edit boxes.

That’s where QA comes in: We’re trying to box out some of the stupid.

Firefox 23 nixes support for outdated blink HTML tag:

Mozilla announced on Tuesday that Firefox 23, the latest version of its browser, will not support the HTML tag blink.

I would tell you to enjoy it while you can, but it’s already gone, and you never noticed it.

I’ve always used the blink tag to test whether edit boxes appropriately strip HTML formatting because it was a nice, obvious way to see if that failed. Oh, well, I still have the h1 tag, I suppose.

UPDATE: This post was originally entitled Fortunately, It Still Works in IE 6 until someone pointed out it does not work in IE 6. Deep down I knew that, but I was too quick with the quippy headline. Thanks, Jen. I have updated the headline with an alternate quip.

Xerox scanners/photocopiers replacing 6s with 8s? In some cases, apparently so:

In this article I present in which way scanners / copiers of the Xerox WorkCentre Line randomly alter written numbers in pages that are scanned. This is not an OCR problem (as we switched off OCR on purpose), it is a lot worse – patches of the pixel data are randomly replaced in a very subtle and dangerous way: The scanned images look correct at first glance, even though numbers may actually be incorrect. Without a fuss, this may cause scenarios like:

1. Incorrect invoices
2. Construction plans with incorrect numbers (as will be shown later in the article) even though they look right
3. Other incorrect construction plans, for example for bridges (danger of life may be the result!)
4. Incorrect metering of medicine, even worse, I think.

Who knew you had to test your printed outputs? I did.

Back in 2007, a developer caused a delay in a very large software launch for an Easter egg or a prank:

Microsoft reps have told Next-Gen the real reason for the Halo 2 Vista delay: partial nudity–specifically someone’s backside in the game’s map editor, the Entertainment Software Rating Board added today.

. . . .

Friday morning, however, the ESRB e-mailed Next-Gen a statement that said a “photograph of an individual showing his bare backside” is accessible within Halo 2 Vista’s exclusive map editor tool. Apparently, it pops up when an “.ass” error occurs….

Hijinks that cost the company millions.

Moral of the story: Always check your software for .ass errors.

Earl Wiener was a former military pilot who became professor of management science and industrial engineering at the University of Miami and conducted a lot of studies on how automation in the cockpit affects the pilots.

At some point, he put together a pithy list of things about automation and going more to fly-by-wire systems. These are known as Wiener’s Laws:

(Note: Nos. 1-16 intentionally left blank)

17. Every device creates its own opportunity for human error.

18. Exotic devices create exotic problems.

19. Digital devices tune out small errors while creating opportunities for large errors.

20. Complacency? Don’t worry about it.

21. In aviation, there is no problem so great or so complex that it cannot be blamed on the pilot.

22. There is no simple solution out there waiting to be discovered, so don’t waste your time searching for it.

23. Invention is the mother of necessity.

24. If at first you don’t succeed… try a new system or a different approach.

25. Some problems have no solution. If you encounter one of these, you can always convene a committee to revise some checklist.

26. In God we trust. Everything else must be brought into your scan.

27. It takes an airplane to bring out the worst in a pilot.

28. Any pilot who can be replaced by a computer should be.

29. Whenever you solve a problem you usually create one. You can only hope that the one you created is less critical than the one you eliminated.

30. You can never be too rich or too thin (Duchess of Windsor) or too careful what you put into a digital flight guidance system (Wiener).

31. Today’s nifty, voluntary system is tomorrow’s F.A.R. [Federal Aviation Regulation]

Because it’s got the word automation right in it, you’re probably looking at it in terms of test automation, but computer software itself is an automation of other processes, so the lessons therein apply more broadly.

You can read more about Wiener in this Aviation Week archive, and I’ll daresay we can learn a lot.

You damn kids think everything about the 1980s was the cool things you revel in calling retro. But this is not always so. The early parts of the eighties included pieces of the 1970s that grimly hung on. Like Discoification of everything.

Like this bit from Meco, “The Empire Strikes Back (Medley)”:

This particular effort reached #18 on the Billboard charts. Which is not as well as his rendition of “Star Wars Theme/Cantina Band”. Which reached #1.

I hope I haven’t triggered too many flashbacks amongst you old timers.

Aussie restaurateur Paul Mathis invents new letter of the Alphabet:

WOULDN’T it be easier if the word “the” was just simply a letter?

Well at least one person seems to think so.

Aussie restaurateur, Paul Mathis has invented a new letter of the alphabet to replace the word “the” because he thinks it is more efficient.

The letter looks like the Cyrillic letter ‘Ћ’. If an upper case T and a lower case h were to have a typographic baby, this is what it would look like.

I’ll wait for the unicode character before I take this seriously.

But how would your app handle a new letter of the alphabet or a new glyph of some sort? How closely do you pay attention to these things?

Gimlet passes along the news story "Chrysler recalling over 280k minivans because airbags may deploy on wrong side:

Chrysler has issued a recall for some 2013 Town & Country, Dodge Grand Caravan and Ram C/V Tradesman vans built between May 10, 2012 and June 7, 2013. These vehicles may have a software error that would cause the wrong side (opposite side) airbags to deploy in a crash. With this defect, a left-side impact would cause the right-side airbag to deploy, etc.

You know, I have a lot of respect for embedded systems testers. You and I got to worry about browser and device compatibility, but we get to try these things in a number of real-world situations given that our ‘real world’ is the Web and computer systems.

When you’re testing out the embedded systems, it’s mostly testing tools and simulations. It’s not like those guys get to pop off the airbags in cars a hundred times a day and sometimes more when they’re trying to recreate an issue. Poor lads.

WebTV, now MSNTV, is going away, and not just in a press release:

Microsoft said that its MSN TV service will be closing down at the end of September, in a post on its Web site and in an email to users.

It’s not that anyone was testing compatibility of Web sites any more for it, but its users were still calling the help desks of consumer products companies whose Web sites did not support their preferred browsing method.

But before you cheer too loudly, consider how many of these people will be upgrading to recycled and donated PCs running Windows Me and IE 6. The answer will be…. more than you’d like to think.

(Link via.)

Six Sigma Wine.

No, really. I mean, no, really, it exists. But, personally speaking, I don’t so much decant as debox.

Leonard Cohen, “Everybody Knows”, featuring scenes from the film Pump Up The Volume (although the version on the soundtrack is a cover by Concrete Blonde):

One might suspect the film influenced your humble Director a lot as a young man. And one might be right.

Your designers probably want the latest cutting edge gizmos and paradigms in your applications. What do your users want? The familiar and the things they have already learned to use:

Ford Motor Co. is going back to buttons and knobs.

Punished by third-party quality reports because of the difficulty of using its touch-screen multimedia system, called MyFord Touch, the auto maker will reprise tuning and volume knobs for the radio as it redesigns existing models, a top Ford executive said.

It is a reversal for Ford, which has been a first-mover with installing mobile-phone-based technologies, voice recognition and touch screens in its vehicles. The systems have been a big selling point for Ford with its vehicles, but also have dragged down its reputation for quality.

. . . .

One of the things that bothered customers was the inability to quickly change the channel or volume on the radio through familiar knobs, he said. As Ford redesigns its vehicles, the flat control panels with add more buttons and knobs[SIC] and[SIC] the main screen will become simpler.

Users have other goals in mind with your software than building a portfolio to take to their next job interviews. In a lot of cases, they want to do the same things in the same ways they’ve always done things.

Granted, you do have to balance the same old way of doing things with any process improvement that your software brings, but far too often, design changes come from the minds of the designers and frustrate the users to no end.

Ever wonder why fast food restaurants and many less expensive department stores have very similar layouts to others in the chain? It’s because customers whose goals are getting a quick lunch or grabbing the essentials quickly, looking for the things they need (which includes the bathrooms and concerns as to whether they pay the server or a cash register by the door) mars the simplicity they crave.

Likewise, if your designers had their way, each freaking McDonalds would have a different layout, some with gerbil tubes between the floors so users could climb between the registers and the condiments stand. Because it would be different. Because it would satisfy their needs as architects.

Some stores and restaurants are destinations qua as eating experiences or shopping experiences. But your software rarely is. Nobody’s running your software to enjoy your software.

So make it easy for them to do what they do with your software, and don’t add weird or unknown design elements just to do that because your users might not appreciate it. Even on the ancillary functions, like changing the radio or clicking a button.

Man, do I think the whole “putting the control label in the control” thing is a bad design pattern for many reasons.

First and foremost, when you start typing or sometimes set focus to the control, the label disappears, and you won’t know what you’re supposed to type into the edit box. Oh, you know, your designers know, your developers know. But will your users know and/or remember?

Second, there’s what do you do with the label when the user saves the form? Does it get saved? What happens when the user clears the data from the control? Does it display again? Most of the time, at least on the rough cuts, this gets mangled. 60% of the time, it gets bollixed every time.

Third, there’s the matter of sizing and spacing the text in the edit box, which our friends at the Custom Showing Service illustrate:

In these fellows’ defense, it’s not just the label in the control that’s sized badly; even the information you type in the edit boxes is trimmed at the top and the bottom to the point of illegibility. I just wanted to use this excuse to lament the label-in-control design pattern.

In these days of super custom designs monkeying with the very nature of controls themselves, altering fonts in the controls, you’ve got to watch out for problems like this. Even if they’re just problems like this:

In the old days, we didn’t have these problems. Labels were outside the control like they should be, and the default fonts fit in the default controls. I’d say “get off my lawn,” but I’m not sure I’ve left enough room for the tail on that g.

I’m a big proponent of capturing and documenting your interface style guide that captures the common rules and spellings to use in your organization’s interfaces and communications to make it seamless for the user and to make it look like you know what you’re doing outside of C#, Ruby, and Java.

For example, canceled and cancelled are both in the dictionary. But they should not both be in your dialog boxes.

Come on, even the music business has a style guide these days:

Roll over William Strunk, and tell E.B. White the news. The music business now has its own grammar guide that might have had the “Elements of Style” authors singing the blues.

The song titles “In da House,” “Kill ‘Em ‘n Grill ‘Em” and “It’s fo’ Realz,” for instance, all get thumbs up in the music industry’s newly issued Style Guide as examples of proper capitalization.

“Intentionally misspelled words must respect the same title casing rules” as those spelled normally, says the guide.

Released last month at the National Association of Recording Merchandisers’ annual convention, rock ‘n’ roll’s new rule book aims to establish basic rules for data entry in the fragmenting world of music, where some folks have become a little too creative for their own good—at least when it comes to spelling, grammar and description.

Come on, be like Too Short. Your deadlines already are. Hey-yo!

You might think that my recent trip to San Francisco changed my life because I keep droning on and on about it, but it’s just that social media glurging is the twenty-first century equivalent of showing you slides I took (And here is a hill in the Marin Highlands. And here is another hill in the Marin Highlands.)

In the dense urban environment, I enjoyed riding in cabs, something I can’t do in the lower density environs of my native habitat. It’s just like a roller coaster, except that the route is unplanned and it carries actual risk of death at the hands of a street car.

You know what a San Francisco taxi driver is just like? Your user. And you should test your applications just like you were taking a pair of white knuckled tourists from the airport to the Sunset neighborhood. How do you do that? Funny, I have a blog post all about it.

Ignore the warnings.

If you’re brave enough, look at the dashboard of your cab. You’ll see that any number of warning lights are on. If you saw those Warning lights on your automobile, you’d limp it to the shop as soon as possible. However, in a cab, the check engine, low tire pressure, no windshield wiper fluid, flames in engine compartment lighs all these mean nothing to the driver. It’s not his car, it’s his workplace for twelve hours a day, and it’s not his problem.

Much like your software is for your professional users.

Your professional user is going to try to get from the start of the process your software tries to make easier for the user to the end. Given the demands of his or her job and what he or she is trying to do, your user might ignore flags and warnings and even error messages if possible. That is, any little warnings your software throws up, the user might try to save the data anyway, even with a weak password. Even with a six digit zip code. Even when step one fails and the user tries step two.

Sometimes, we might be tempted to see a warning or error and stop testing that avenue, instead waiting for problems we encounter to be fixed. But users might not stop when they see that error and await a patch six weeks (or six hours) from now. Test through the warnings and errors and see if there’s something beyond to log that might be as bad or worse. Also, there is a slight chance that those bugs won’t get fixed before your user gets to take it for a spin.

Hit the gas before the light turns green.

Let’s face it, today’s modern Web applications, steeped in divs and AJAX, parts of pages display before other parts, and sometimes controls display before the JavaScript controlling the actions of those controls or links loads. If you use automated testing, you know to account for this by putting in special statements to wait for the page or control to load or the element or div to be visible before interacting with it.

But.

Time is money, and your San Francisco hack is going to anticipate a light’s turning green and at the very least will let off the brake when he thinks the light is going to turn (if not actually jam on the accelerator. He wants to get you to your destination and himself to a tip as fast as possible.

So your user is going to interact with your Web site. If you put the button or link the user needs on the screen, he’s going to click it even if the rest of the page and code are still streaming through the tubes and pipes.

You’re probably already doing a bit of this as you try to run through as many scenarios as possible in the thirty minutes you have to test the Web site, but whenever you encounter a problem because you’re moving faster than the application, the application has the problem, not you. Log the problems, and don’t let your peers tell you that it will work in production or that it’s not a problem. It most certainly is.

Punch it, Margaret.

I know I’ve gone on about this before here and elsewhere, but your load testing tools have to punch it when the light turns green.

Look at how your hack drives. He floors it before the traffic signal changes, and his foot is on the accelerator until he’s about to collide with some stopped innocent bystander, at which time he’s on the brakes just hard enough to avert catastrophe.

Aside from your load testing strategy, you can punch it in manual testing. If one click of a button is good, isn’t one hundred clicks of a button better? If the maximum length of a string is 32 characters, enter 32 characters. Not to test whether you can enter 32 characters in a particular form, but to see what happens elsewhere when the string is the allowed maximum. And so on.

Test the application not only beyond its limits, but to its limits. And beyond.

Find the shortcuts and back ways, and test them.

Your cabbie knows if there’s a line of cars at a light on Pine, he can turn down Dashiell Hammett Street to try his luck on Bush. Your users, too, are going to find those little shortcuts that aren’t on the main menu and that aren’t on the sticky notes or improvised, anticipated workflow around which your developers will write the code. Users are going to find those back alleys in your modules and are going to exploit them to try to get their work done.

Since you’re spending your time, amid from the endless meetings and putting your head on your desk and weeping in soundless shudders, in the application, you’ve got to figure out those shortcuts the users will find and test them out, too.

Cruise the application once in a while.

Cab drivers can get their next fare by sleeping in their cabs until they’re dispatched (the Missouri method), they can wait in line at a cab stand at the airport or a hotel, or they can find defects, I mean customers, driving the city streets (sometimes on the way to a nap or a stand).

So you can find defects outside of your normal duties, aside from your test cases or user stories, just by cruising through your application once in a while, taking a look with fresh eyes for defects waving their hands in the air. Take a couple minutes before the meeting when you can’t start a bigger task to just hit the site. You’ll be surprised what you might find.

However, unlike a San Francisco cabbie, these are the best tips you’re going to get for testing, so make the best of them.

Phil Collins from ::cough, cough:: thirty years ago with “I Don’t Care Any More”:

The key to a long, happy career in QA is to know when to care and when not to care. Criticism, office politics, pleas for mercy? Don’t care. Missing a comma with a noun of direct address in the welcome email? CARE!