Archive for the ‘Miscellany’ Category

QAers Gonna QA

Tuesday, September 3rd, 2013 by The Director

The first step on software testing appearing in Diagnostic and Statistical Manual of Mental Disorders 6: a scientific-sounding name for the QA mindset: negative dispositional attitude:

New research has uncovered the reason why some people seem to dislike everything while others seem to like everything. Apparently, it’s all part of our individual personality — a dimension that researchers have coined “dispositional attitude.”

People with a positive dispositional attitude have a strong tendency to like things, whereas people with a negative dispositional attitude have a strong tendency to dislike things, according to research published in the Journal of Personality and Social Psychology. The journal article, “Attitudes without objects: Evidence for a dispositional attitude, its measurement, and its consequences,” was written by Justin Hepler, University of Illinois at Urbana-Champaign, and Dolores Albarracín, Ph.D., the Martin Fishbein Chair of Communication and Professor of Psychology at Penn.

Realism is an illness.

It’s Not A Geek Company, So You Didn’t Hear About The Problem

Tuesday, August 27th, 2013 by The Director

If this had happened to Amazon, Google, or another company geeks love, you would have heard about it already: Computer Problems Leave Goods Stranded at New York Port:

Computer problems at one of the East Coast’s biggest ports have snarled the flow of cargo across the Northeast for weeks, delaying the delivery of consumer goods needed for back-to-school sales and the start of the holiday shopping season.

The problems at the Port of New York and New Jersey began in June, when Maher Terminals LLC, one of the world’s largest handlers of shipping containers, launched a new computer operating system, according to shipping, trucking, retail-industry and government officials.

. . . .

Maher Terminals and the operating system’s maker, Navis LLC, a division of Finland-based Cargotec Corp., said in a joint news release last week that “real-time interactions between the various system components deployed in the container yard were not operating as designed.” As a temporary solution, certain automated components of the system were scaled back, the companies said. They didn’t reveal the source of the problems.

Rest assured, it worked on a developer’s machine.

Meanwhile, even though it did not impact geeks directly, this bug had huge impact all down the logistical downstream.

Blackjack, Baseball, Software, and Startups

Friday, August 9th, 2013 by The Director

An article entitled An Important Life Lesson from Blackjack and Baseball: You gain more by not being stupid than you do by being smart has this brief takeaway:

The moral: You gain more by not being stupid than you do by being smart. Smart gets neutralized by other smart people. Stupid does not.

The gist: In some competitions, you can lose on purpose, but you can only try to win on purpose, so making the smart moves won’t necessary lead you to success and winning. But making stupid moves can and will thwart you.

In software development or startups, it’s easy to fall into that bad habit of pursing something new, neat, or smart and forgetting to take care of the little things like user experience and bounding your edit boxes.

That’s where QA comes in: We’re trying to box out some of the stupid.

Next, They Come For Comic Sans

Thursday, August 8th, 2013 by The Director

Firefox 23 nixes support for outdated blink HTML tag:

Mozilla announced on Tuesday that Firefox 23, the latest version of its browser, will not support the HTML tag blink.

I would tell you to enjoy it while you can, but it’s already gone, and you never noticed it.

I’ve always used the blink tag to test whether edit boxes appropriately strip HTML formatting because it was a nice, obvious way to see if that failed. Oh, well, I still have the h1 tag, I suppose.

UPDATE: This post was originally entitled Fortunately, It Still Works in IE 6 until someone pointed out it does not work in IE 6. Deep down I knew that, but I was too quick with the quippy headline. Thanks, Jen. I have updated the headline with an alternate quip.

OU612

Tuesday, August 6th, 2013 by The Director

Xerox scanners/photocopiers replacing 6s with 8s? In some cases, apparently so:

In this article I present in which way scanners / copiers of the Xerox WorkCentre Line randomly alter written numbers in pages that are scanned. This is not an OCR problem (as we switched off OCR on purpose), it is a lot worse – patches of the pixel data are randomly replaced in a very subtle and dangerous way: The scanned images look correct at first glance, even though numbers may actually be incorrect. Without a fuss, this may cause scenarios like:

  1. Incorrect invoices
  2. Construction plans with incorrect numbers (as will be shown later in the article) even though they look right
  3. Other incorrect construction plans, for example for bridges (danger of life may be the result!)
  4. Incorrect metering of medicine, even worse, I think.

Who knew you had to test your printed outputs? I did.

Wiener’s Laws of Aircraft Automation

Wednesday, July 31st, 2013 by The Director

Earl Wiener was a former military pilot who became professor of management science and industrial engineering at the University of Miami and conducted a lot of studies on how automation in the cockpit affects the pilots.

At some point, he put together a pithy list of things about automation and going more to fly-by-wire systems. These are known as Wiener’s Laws:

(Note: Nos. 1-16 intentionally left blank)

17. Every device creates its own opportunity for human error.

18. Exotic devices create exotic problems.

19. Digital devices tune out small errors while creating opportunities for large errors.

20. Complacency? Don’t worry about it.

21. In aviation, there is no problem so great or so complex that it cannot be blamed on the pilot.

22. There is no simple solution out there waiting to be discovered, so don’t waste your time searching for it.

23. Invention is the mother of necessity.

24. If at first you don’t succeed… try a new system or a different approach.

25. Some problems have no solution. If you encounter one of these, you can always convene a committee to revise some checklist.

26. In God we trust. Everything else must be brought into your scan.

27. It takes an airplane to bring out the worst in a pilot.

28. Any pilot who can be replaced by a computer should be.

29. Whenever you solve a problem you usually create one. You can only hope that the one you created is less critical than the one you eliminated.

30. You can never be too rich or too thin (Duchess of Windsor) or too careful what you put into a digital flight guidance system (Wiener).

31. Today’s nifty, voluntary system is tomorrow’s F.A.R. [Federal Aviation Regulation]

Because it’s got the word automation right in it, you’re probably looking at it in terms of test automation, but computer software itself is an automation of other processes, so the lessons therein apply more broadly.

You can read more about Wiener in this Aviation Week archive, and I’ll daresay we can learn a lot.

Updating Your Tests Might Be A Bit Premature

Friday, July 12th, 2013 by The Director

Aussie restaurateur Paul Mathis invents new letter of the Alphabet:

WOULDN’T it be easier if the word “the” was just simply a letter?

Well at least one person seems to think so.

Aussie restaurateur, Paul Mathis has invented a new letter of the alphabet to replace the word “the” because he thinks it is more efficient.

The letter looks like the Cyrillic letter ‘Ћ’. If an upper case T and a lower case h were to have a typographic baby, this is what it would look like.

I’ll wait for the unicode character before I take this seriously.

But how would your app handle a new letter of the alphabet or a new glyph of some sort? How closely do you pay attention to these things?

Off By 180 Degrees Error

Thursday, July 11th, 2013 by The Director

Gimlet passes along the news story "Chrysler recalling over 280k minivans because airbags may deploy on wrong side:

Chrysler has issued a recall for some 2013 Town & Country, Dodge Grand Caravan and Ram C/V Tradesman vans built between May 10, 2012 and June 7, 2013. These vehicles may have a software error that would cause the wrong side (opposite side) airbags to deploy in a crash. With this defect, a left-side impact would cause the right-side airbag to deploy, etc.

You know, I have a lot of respect for embedded systems testers. You and I got to worry about browser and device compatibility, but we get to try these things in a number of real-world situations given that our ‘real world’ is the Web and computer systems.

When you’re testing out the embedded systems, it’s mostly testing tools and simulations. It’s not like those guys get to pop off the airbags in cars a hundred times a day and sometimes more when they’re trying to recreate an issue. Poor lads.

I Guess You Can Remove One Item From Your Compatibility Testing Matrix

Wednesday, July 10th, 2013 by The Director

WebTV, now MSNTV, is going away, and not just in a press release:

Microsoft said that its MSN TV service will be closing down at the end of September, in a post on its Web site and in an email to users.

It’s not that anyone was testing compatibility of Web sites any more for it, but its users were still calling the help desks of consumer products companies whose Web sites did not support their preferred browsing method.

But before you cheer too loudly, consider how many of these people will be upgrading to recycled and donated PCs running Windows Me and IE 6. The answer will be…. more than you’d like to think.

(Link via.)

What QA Decants

Tuesday, July 9th, 2013 by The Director

Six Sigma Wine.

No, really. I mean, no, really, it exists. But, personally speaking, I don’t so much decant as debox.

Speaking of Too Short

Thursday, June 20th, 2013 by The Director

Man, do I think the whole “putting the control label in the control” thing is a bad design pattern for many reasons.

First and foremost, when you start typing or sometimes set focus to the control, the label disappears, and you won’t know what you’re supposed to type into the edit box. Oh, you know, your designers know, your developers know. But will your users know and/or remember?

Second, there’s what do you do with the label when the user saves the form? Does it get saved? What happens when the user clears the data from the control? Does it display again? Most of the time, at least on the rough cuts, this gets mangled. 60% of the time, it gets bollixed every time.

Third, there’s the matter of sizing and spacing the text in the edit box, which our friends at the Custom Showing Service illustrate:

CSS fail

In these fellows’ defense, it’s not just the label in the control that’s sized badly; even the information you type in the edit boxes is trimmed at the top and the bottom to the point of illegibility. I just wanted to use this excuse to lament the label-in-control design pattern.

In these days of super custom designs monkeying with the very nature of controls themselves, altering fonts in the controls, you’ve got to watch out for problems like this. Even if they’re just problems like this:

Just take a little off the bottom

In the old days, we didn’t have these problems. Labels were outside the control like they should be, and the default fonts fit in the default controls. I’d say “get off my lawn,” but I’m not sure I’ve left enough room for the tail on that g.

If The Rappers Do It, Why Don’t You?

Wednesday, June 19th, 2013 by The Director

I’m a big proponent of capturing and documenting your interface style guide that captures the common rules and spellings to use in your organization’s interfaces and communications to make it seamless for the user and to make it look like you know what you’re doing outside of C#, Ruby, and Java.

For example, canceled and cancelled are both in the dictionary. But they should not both be in your dialog boxes.

Come on, even the music business has a style guide these days:

Roll over William Strunk, and tell E.B. White the news. The music business now has its own grammar guide that might have had the “Elements of Style” authors singing the blues.

The song titles “In da House,” “Kill ‘Em ‘n Grill ‘Em” and “It’s fo’ Realz,” for instance, all get thumbs up in the music industry’s newly issued Style Guide as examples of proper capitalization.

“Intentionally misspelled words must respect the same title casing rules” as those spelled normally, says the guide.

Released last month at the National Association of Recording Merchandisers’ annual convention, rock ‘n’ roll’s new rule book aims to establish basic rules for data entry in the fragmenting world of music, where some folks have become a little too creative for their own good—at least when it comes to spelling, grammar and description.

Come on, be like Too Short. Your deadlines already are. Hey-yo!

Test It Like A San Francisco Hack

Tuesday, June 11th, 2013 by The Director

You might think that my recent trip to San Francisco changed my life because I keep droning on and on about it, but it’s just that social media glurging is the twenty-first century equivalent of showing you slides I took (And here is a hill in the Marin Highlands. And here is another hill in the Marin Highlands.)

In the dense urban environment, I enjoyed riding in cabs, something I can’t do in the lower density environs of my native habitat. It’s just like a roller coaster, except that the route is unplanned and it carries actual risk of death at the hands of a street car.

You know what a San Francisco taxi driver is just like? Your user. And you should test your applications just like you were taking a pair of white knuckled tourists from the airport to the Sunset neighborhood. How do you do that? Funny, I have a blog post all about it.

Ignore the warnings.

If you’re brave enough, look at the dashboard of your cab. You’ll see that any number of warning lights are on. If you saw those Warning lights on your automobile, you’d limp it to the shop as soon as possible. However, in a cab, the check engine, low tire pressure, no windshield wiper fluid, flames in engine compartment lighs all these mean nothing to the driver. It’s not his car, it’s his workplace for twelve hours a day, and it’s not his problem.

Much like your software is for your professional users.

Your professional user is going to try to get from the start of the process your software tries to make easier for the user to the end. Given the demands of his or her job and what he or she is trying to do, your user might ignore flags and warnings and even error messages if possible. That is, any little warnings your software throws up, the user might try to save the data anyway, even with a weak password. Even with a six digit zip code. Even when step one fails and the user tries step two.

Sometimes, we might be tempted to see a warning or error and stop testing that avenue, instead waiting for problems we encounter to be fixed. But users might not stop when they see that error and await a patch six weeks (or six hours) from now. Test through the warnings and errors and see if there’s something beyond to log that might be as bad or worse. Also, there is a slight chance that those bugs won’t get fixed before your user gets to take it for a spin.

Hit the gas before the light turns green.

Let’s face it, today’s modern Web applications, steeped in divs and AJAX, parts of pages display before other parts, and sometimes controls display before the JavaScript controlling the actions of those controls or links loads. If you use automated testing, you know to account for this by putting in special statements to wait for the page or control to load or the element or div to be visible before interacting with it.

But.

Time is money, and your San Francisco hack is going to anticipate a light’s turning green and at the very least will let off the brake when he thinks the light is going to turn (if not actually jam on the accelerator. He wants to get you to your destination and himself to a tip as fast as possible.

So your user is going to interact with your Web site. If you put the button or link the user needs on the screen, he’s going to click it even if the rest of the page and code are still streaming through the tubes and pipes.

You’re probably already doing a bit of this as you try to run through as many scenarios as possible in the thirty minutes you have to test the Web site, but whenever you encounter a problem because you’re moving faster than the application, the application has the problem, not you. Log the problems, and don’t let your peers tell you that it will work in production or that it’s not a problem. It most certainly is.

Punch it, Margaret.

I know I’ve gone on about this before here and elsewhere, but your load testing tools have to punch it when the light turns green.

Look at how your hack drives. He floors it before the traffic signal changes, and his foot is on the accelerator until he’s about to collide with some stopped innocent bystander, at which time he’s on the brakes just hard enough to avert catastrophe.

Aside from your load testing strategy, you can punch it in manual testing. If one click of a button is good, isn’t one hundred clicks of a button better? If the maximum length of a string is 32 characters, enter 32 characters. Not to test whether you can enter 32 characters in a particular form, but to see what happens elsewhere when the string is the allowed maximum. And so on.

Test the application not only beyond its limits, but to its limits. And beyond.

Find the shortcuts and back ways, and test them.

Your cabbie knows if there’s a line of cars at a light on Pine, he can turn down Dashiell Hammett Street to try his luck on Bush. Your users, too, are going to find those little shortcuts that aren’t on the main menu and that aren’t on the sticky notes or improvised, anticipated workflow around which your developers will write the code. Users are going to find those back alleys in your modules and are going to exploit them to try to get their work done.

Since you’re spending your time, amid from the endless meetings and putting your head on your desk and weeping in soundless shudders, in the application, you’ve got to figure out those shortcuts the users will find and test them out, too.

Cruise the application once in a while.

Cab drivers can get their next fare by sleeping in their cabs until they’re dispatched (the Missouri method), they can wait in line at a cab stand at the airport or a hotel, or they can find defects, I mean customers, driving the city streets (sometimes on the way to a nap or a stand).

So you can find defects outside of your normal duties, aside from your test cases or user stories, just by cruising through your application once in a while, taking a look with fresh eyes for defects waving their hands in the air. Take a couple minutes before the meeting when you can’t start a bigger task to just hit the site. You’ll be surprised what you might find.

 

However, unlike a San Francisco cabbie, these are the best tips you’re going to get for testing, so make the best of them.

Five QA Things To Do In San Francisco

Friday, May 31st, 2013 by The Director

As some of you might be aware, I spent last weekend in the San Francisco area. What did I do? Things worthy of QA, of course, and here’s what you, too, can do while visiting San Francisco and capturing a bit of the flavor of the area worthy of QA.

Eat at NaN

NaN restaurant

A restaurant called NaN? That’s just screaming for QA’s dining dollars.

Oh, but the actual QA world in San Francisco let us down. It’s NaR (not a restaurant). So no Korean fusion for you or me.

Visit Sonoma Valley for the Wonderful Mobile Testing

The Sonoma Valley, partly known for its wines, also has low to no mobile data connectivity. Did you know there’s an icon on your iPhone that indicates just cellular service? I do now. Now, what happens to my app under test when I am in a tastefully comported Faraday Cage, sampling Cabernet and adding records to the DB?

Also, there are some wineries around, which gives you the proper answer for any “Why would the user do that?” questions you might get: “It was the third winery on the tour.”

Take a Bay Cruise to Find the Null in Geolocation Testing

In at least one spot on San Francisco Bay, I discovered I was (null) miles away from Sausalito. Can you find that spot?

Granted, that was an error in geolocation in this particular app, but what happens in your mobile application when you’re on a boat?

Never pass up an opportunity to try the mobile app out in unexpected circumstances (What street address do I get when I’m flying over the building in a helicopter tour?).

Although strange and extreme, they might uncover conditions and application behaviors that would occur in less extreme circumstances.

Traverse the Golden Gate Bridge in the Fog and the Wind in an Open Car or Bus

Brothers and sisters who are not in San Francisco: Just because San Francisco is in California, that does not mean it is warm. Your stereotypes only apply to southern California.

Crossing the Golden Gate bridge in the morning in a convertible or one of those open-topped tour buses allows you to prove your toughness and endurance, and it provides a handy metaphor for the software development life cycle. You’re in a fog, you can’t see anything, suddenly something large looms over you, then it passes.

Additionally, crossing the same way on the southbound, windward side of the bridge is a particularly moving experience that will bring tears to your eyes and proves to be an endurance test not unlike a lessons learned meeting.

Walk the City. Then Take A Cab

Walking around San Francisco (or any other geographically bound and dense city) allows you to get up close to things and to view the emergence of the architecture and whatnot slowly.

Then take a cab and see how those familiar with the city and who have a particular goal in mind use the city streets and alleys to accomplish their goal.

It’s different, but they’re both San Francisco.

That’s a bit of a stretch and overt metaphor for testing, but I needed a fifth item because the title says five.

Why Would a Baseball Player Do That?

Tuesday, April 23rd, 2013 by The Director

Ever get asked why a user would do that? Of course you do. You’ve already been asked that today.

Here’s a little story for you about why the ball player trying to steal third base ended up on first base:

This guy stole second. Then he tried to steal third but somehow wound up on first. Then he got thrown out trying to steal second again. All in a span of five pitches.

The result, as far as we’re concerned:

The part where a runner on second base finishes the next play on first base? It’s not possible to score that without crashing every computer in America.

“There’s no way to do that,” longtime official scorer and SABR historian David Vincent said Saturday. “Not covered in the rules. A runner on second base going to first base? That’s impossible.”

Now obviously it’s not “impossible,” because it really happened. But tell that to the computer programmers of America.

“All the computer software — none of it will handle that,” Vincent said. “You don’t run the bases [from] second to first. Any software that processes play-by-play won’t accept that.”

So because it’s theoretically impossible, the official box score of this game listed Segura as having been thrown out stealing third — even though he slid into second. Huh?

“That’s because the play-by-play listed him as staying at second base [because it couldn't compute that he was actually on first],” Vincent said. “So then he had to be caught stealing third. But that never happened. So that has to get changed.”

Right. But that’s not all. The official box score and play-by-play also said that Braun got caught stealing second.

So why would a user do that? Because the user could do that. And just because someone has not done that does not mean someone will not do that in a strange set of circumstances you cannot anticipate now.

(Link via tweet linking to this article.)

Re-Cognize to Avoid Mental Errors

Wednesday, April 17th, 2013 by The Director

A little story to talk about some of the ways our inattention can lead us to miss some mistakes.

For years, I’d heard about an awesome British television program that, although it aired on PBS and had not reached popularity among mainstream Americans certainly got its discussion amongst the tech crowd which tends to favor BBC programs that air on PBS. The show?

Downtown Abbey

Or so I thought. I had seen discussion of the program on the Internet, as I mentioned, and and I watched the introduction to it seven times, and I saw it on a Roku menu many times. I even started a Twitter hashtag called #DowntownAbbeySpoilers to demonstrate my particular brand of wit and to flaunt my ignorance before the Internet. It was only in the seventh episode, which includes a scene at the Downton train station clearly labeled as such, did I discover my error.

Downton Abbey, not Downtown Abbey

I was chuffed, to say the least.

I’d read the word as Downtown at some point, maybe a misspelling or a bad pattern matching on my part, and every time I saw the title, my brain filled that Downtown instead of Downton. Every blooming time.

I started planning this post in my head, I encountered a misspelling in the application I’m testing. Although I’ve hit the screen a number of times, I’d missed the misspelling every time I saw the page previously until just yesterday, when I happened to look at the extraneous letter in the middle of the word. Other times, my eyes had skipped right over the defect like a smooth stone over still water.

The way to avoid these types of oversights, as I always have said, is to slow down and to look at everything with fresh eyes as often as possible. Take some time to read every word and to test every control in different combinations. Focusing on the higher-level functions of an application is very important, but focusing on individual parts of it at a very atomic level will find a number of problems that your users will eventually find if you don’t (and fix) first.

Now, onto the second error: trusting the context.

My UK readers and many of the Downton Abbey fans here in the US and elsewhere might be chortling about what I said above: I was chuffed, to say the least.

It sounds like a combination of chagrined and maybe huffed, doesn’t it? You’re forgiven if you assumed it meant something like that from context. But chuffed means pleased. I was not pleased at all.

This kind of oversight comes when you trust the context of something, generally in terms of application behavior. When you click that button, of course it does that. It’s not crazy enough out of whack to be an obvious bug, so you let it go. This happens a lot in two cases: if you’re testing something in an industry or vertical in which you’re not already steeped, like counting DIDs for a telecommunications company’s order scheduling software. If the number of DIDs doesn’t match the number of telephones and never has, will you notice it as long as you can save the order and make sure it displays properly on the technicians’ tablets? Maybe not.

The other place where you can get into that sort of trouble is after you’ve tested an application for a long enough period of time that its evolution gets a little mixed in your mind. Did that used to work? Did it used to do something different? Should it do something other than what it does?

To try to minimize these style oversights, don’t trust the context. Try to learn why the context is the way it is. If something seems strange, ask. And if the person you asks says, “Just because,” “Trust me,” or “It’s always been that way,” open your defect tracker. Because QA has always been that way, just because. Trust me.

So the lessons are, again, slow down and learn as much as you can while you’re testing something to test it most effectively, and not to trust yourself or your context because you’ve been wrong before. Or at least I have. AND I AM NOT CHUFFED ABOUT IT.

The QA Diva: Tips and Tricks

Wednesday, April 17th, 2013 by The Director

QA shares a lot of qualities with the popular conception of the diva. We’re always demanding attention, and when we have attention, we make demands.

Researchers who have studied divas in the wild recognize two kinds: healthy divas, who aren’t unhealthy divas, and unhealthy divas, who are the divas who’ve built the popular conception of diva.

Regardless, the Wall Street Journal recently featured an article on the difference:

Divas, by definition, are high-performing, high-maintenance narcissists. Some are needy, demanding, negative—and talk almost incessantly about themselves. Researchers say these are “unhealthy divas” and the source of their narcissism usually is low self-esteem: They are constantly trying to pump themselves up.

Yet, believe it or not, researchers say some divas are healthy. They adore the limelight and work hard to be always front and center—but they are willing to make room for others. They are spirited, fun and positive. Because they assume everyone around them is interested in them, they share a lot of themselves—and in this way bring people together. They have the ability to help others enjoy things that aren’t normally enjoyable, whether it’s a long line at the store, an office meeting or dinner with the boss.

Within the article, you’ll find a number of differences delineated between unhealthy tantrums and expecting and settling for nothing less than the best because, baby, you deserve it (and so do your users).

Also, to prove its worth the the QA community, please note the Van Halen Brown M&Ms story appears, so you know it’s got to be True (much as things on this blog are, as I mentioned the story last year).

It’s How I Got Started

Tuesday, April 16th, 2013 by The Director

I’m a little behind in my newspaper reading, as you will see. But this Non Sequitur cartoon from last week might describe how one comes to become a software tester, or how one becomes a software testing consultant:

Something To Perk You Up

Wednesday, April 10th, 2013 by The Director

Screenshots of Despair.

Via tweet:

A Line To Borrow

Tuesday, March 26th, 2013 by The Director

To begin with, the thing was so antipodally at variance with the whole chain of horrors preceding it–the change of mood from stark terror to cool complacency and even exultation was so unheralded, lightning-like, and complete!

–H.P. Lovecraft, “The Whisperer in the Darkness”

New career goal: I’m going to start a defect with To begin with, the thing was so antipodally at variance with the whole chain of horrors preceding it at least once.