QA Hates You

Ever get asked why a user would do that? Of course you do. You’ve already been asked that today.

Here’s a little story for you about why the ball player trying to steal third base ended up on first base:

This guy stole second. Then he tried to steal third but somehow wound up on first. Then he got thrown out trying to steal second again. All in a span of five pitches.

The result, as far as we’re concerned:

The part where a runner on second base finishes the next play on first base? It’s not possible to score that without crashing every computer in America.

“There’s no way to do that,” longtime official scorer and SABR historian David Vincent said Saturday. “Not covered in the rules. A runner on second base going to first base? That’s impossible.”

Now obviously it’s not “impossible,” because it really happened. But tell that to the computer programmers of America.

“All the computer software — none of it will handle that,” Vincent said. “You don’t run the bases [from] second to first. Any software that processes play-by-play won’t accept that.”

So because it’s theoretically impossible, the official box score of this game listed Segura as having been thrown out stealing third — even though he slid into second. Huh?

“That’s because the play-by-play listed him as staying at second base [because it couldn't compute that he was actually on first],” Vincent said. “So then he had to be caught stealing third. But that never happened. So that has to get changed.”

Right. But that’s not all. The official box score and play-by-play also said that Braun got caught stealing second.

So why would a user do that? Because the user could do that. And just because someone has not done that does not mean someone will not do that in a strange set of circumstances you cannot anticipate now.

(Link via tweet linking to this article.)

A little story to talk about some of the ways our inattention can lead us to miss some mistakes.

For years, I’d heard about an awesome British television program that, although it aired on PBS and had not reached popularity among mainstream Americans certainly got its discussion amongst the tech crowd which tends to favor BBC programs that air on PBS. The show?

Downtown Abbey

Or so I thought. I had seen discussion of the program on the Internet, as I mentioned, and and I watched the introduction to it seven times, and I saw it on a Roku menu many times. I even started a Twitter hashtag called #DowntownAbbeySpoilers to demonstrate my particular brand of wit and to flaunt my ignorance before the Internet. It was only in the seventh episode, which includes a scene at the Downton train station clearly labeled as such, did I discover my error.

I was chuffed, to say the least.

I’d read the word as Downtown at some point, maybe a misspelling or a bad pattern matching on my part, and every time I saw the title, my brain filled that Downtown instead of Downton. Every blooming time.

I started planning this post in my head, I encountered a misspelling in the application I’m testing. Although I’ve hit the screen a number of times, I’d missed the misspelling every time I saw the page previously until just yesterday, when I happened to look at the extraneous letter in the middle of the word. Other times, my eyes had skipped right over the defect like a smooth stone over still water.

The way to avoid these types of oversights, as I always have said, is to slow down and to look at everything with fresh eyes as often as possible. Take some time to read every word and to test every control in different combinations. Focusing on the higher-level functions of an application is very important, but focusing on individual parts of it at a very atomic level will find a number of problems that your users will eventually find if you don’t (and fix) first.

Now, onto the second error: trusting the context.

My UK readers and many of the Downton Abbey fans here in the US and elsewhere might be chortling about what I said above: I was chuffed, to say the least.

It sounds like a combination of chagrined and maybe huffed, doesn’t it? You’re forgiven if you assumed it meant something like that from context. But chuffed means pleased. I was not pleased at all.

This kind of oversight comes when you trust the context of something, generally in terms of application behavior. When you click that button, of course it does that. It’s not crazy enough out of whack to be an obvious bug, so you let it go. This happens a lot in two cases: if you’re testing something in an industry or vertical in which you’re not already steeped, like counting DIDs for a telecommunications company’s order scheduling software. If the number of DIDs doesn’t match the number of telephones and never has, will you notice it as long as you can save the order and make sure it displays properly on the technicians’ tablets? Maybe not.

The other place where you can get into that sort of trouble is after you’ve tested an application for a long enough period of time that its evolution gets a little mixed in your mind. Did that used to work? Did it used to do something different? Should it do something other than what it does?

To try to minimize these style oversights, don’t trust the context. Try to learn why the context is the way it is. If something seems strange, ask. And if the person you asks says, “Just because,” “Trust me,” or “It’s always been that way,” open your defect tracker. Because QA has always been that way, just because. Trust me.

So the lessons are, again, slow down and learn as much as you can while you’re testing something to test it most effectively, and not to trust yourself or your context because you’ve been wrong before. Or at least I have. AND I AM NOT CHUFFED ABOUT IT.

QA shares a lot of qualities with the popular conception of the diva. We’re always demanding attention, and when we have attention, we make demands.

Researchers who have studied divas in the wild recognize two kinds: healthy divas, who aren’t unhealthy divas, and unhealthy divas, who are the divas who’ve built the popular conception of diva.

Regardless, the Wall Street Journal recently featured an article on the difference:

Divas, by definition, are high-performing, high-maintenance narcissists. Some are needy, demanding, negative—and talk almost incessantly about themselves. Researchers say these are “unhealthy divas” and the source of their narcissism usually is low self-esteem: They are constantly trying to pump themselves up.

Yet, believe it or not, researchers say some divas are healthy. They adore the limelight and work hard to be always front and center—but they are willing to make room for others. They are spirited, fun and positive. Because they assume everyone around them is interested in them, they share a lot of themselves—and in this way bring people together. They have the ability to help others enjoy things that aren’t normally enjoyable, whether it’s a long line at the store, an office meeting or dinner with the boss.

Within the article, you’ll find a number of differences delineated between unhealthy tantrums and expecting and settling for nothing less than the best because, baby, you deserve it (and so do your users).

Also, to prove its worth the the QA community, please note the Van Halen Brown M&Ms story appears, so you know it’s got to be True (much as things on this blog are, as I mentioned the story last year).

I’m a little behind in my newspaper reading, as you will see. But this Non Sequitur cartoon from last week might describe how one comes to become a software tester, or how one becomes a software testing consultant:

Screenshots of Despair.

Via tweet:

Screenshots of despairscreenshotsofdespair.tumblr.com

— Jonathan Korman (@miniver) April 9, 2013

To begin with, the thing was so antipodally at variance with the whole chain of horrors preceding it–the change of mood from stark terror to cool complacency and even exultation was so unheralded, lightning-like, and complete!

–H.P. Lovecraft, “The Whisperer in the Darkness”

New career goal: I’m going to start a defect with To begin with, the thing was so antipodally at variance with the whole chain of horrors preceding it at least once.

He wants us all to know about this Tumblr of animated gifs called DBA Reactions.

Thank you, that is all.

Everybody has focused on the fact that the new CEO of Yahoo! has ended telecommuting in the company, another soul-crusher occurred at Harley-Davidson: they banned rock and roll music:

Hundreds of Harley-Davidson employees learned through a memo last week that their radios and music being piped onto the factory floor would be kaput by Wednesday — part of a continuous effort to improve safety.

No headphones. No headbanging. No rock ‘;n’ roll.

Just the sound of motorcycles being made. It’s the sweet sound of productivity for a Fortune 500 firm whose earnings have made a comeback since an organization-wide restructuring began in 2009.

But it wasn’t one incident in particular that made them “stop the music,” as singer Rihanna says.

“It’s a distraction,” said Maripat Blankenheim, director of external communications for Harley. “It’s really important for people – no matter what they do – to be focused on what they do.”

The memo, authored by John Dansby II, vice president for manufacturing, reflects that mantra.

“As you are aware, it is imperative that we improve our safety and first-time quality performance,” he writes. “Too many distractions and potential hazards still exist in the workplace that impact our performance every day.”

Jeez, Louise, what will people at Yahoo! do if they cannot blare AC/DC while they test?

You know, when I worked in a plant, we didn’t get radios, and I had to compensate by tacking up a poem to memorize while the Didde-Glaser ran. That experience pacing and talking to myself provided me with valuable experience I apply to QA every day.

Mocking up some fake addresses and using “101 Tester Way”?

Bear in mind these are actual addresses:

101 Tester Lane, McEwen, TN 37101:

View Larger Map

101 Tester Street, Clinton, AR 72031:

View Larger Map

These samples will give you the satisfaction of having Test right in the data and they work for map integration purposes.

Strangely enough, 101 Tester Street is not that far down the highway from my secure mountain redoubt.

There’s a White House petition to make the United States change from imperial units to metric units of measurement.

I got two words for you: Gimli Glider, a plane that almost fell out of the sky during the Canadian switchover to the metric system:

At the time of the incident, Canada was converting to the metric system. As part of this process, the new 767s being acquired by Air Canada were the first to be calibrated for metric units (litres and kilograms) instead of customary units (gallons and pounds). All other aircraft were still operating with Imperial units (gallons and pounds). For the trip to Edmonton, the pilot calculated a fuel requirement of 22,300 kilograms (49,000 lb). A dripstick check indicated that there were 7,682 litres (1,690 imp gal; 2,029 US gal) already in the tanks. To calculate how much more fuel had to be added, the crew needed to convert the quantity in the tanks to a weight, subtract that figure from 22,300 kg and convert the result back into a volume. In previous times, this task would have been completed by a flight engineer, but the 767 was the first of a new generation of airliners that flew without a flight engineer and flew only with a pilot and co-pilot.

A litre of jet fuel weighs 0.803 kg, so the correct calculation was:

7682 L × 0.803 kg/L = 6169 kg
22300 kg − 6169 kg = 16131 kg
16131 kg ÷ (0.803 kg/L) = 20088 L of fuel to be transferred

Between the ground crew and pilots, however, they arrived at an incorrect conversion factor of 1.77, the weight of a litre of fuel in pounds. This was the conversion factor provided on the refueller’s paperwork and which had always been used for the airline’s imperial-calibrated fleet. Their calculation produced:

7682 L × 1.77 kg/L = 13597 kg
22300 kg − 13597 kg = 8703 kg
8703 kg ÷ (1.77 kg/L) = 4916 L of fuel to be transferred

Instead of 22,300 kg of fuel, they had 22,300 pounds on board — a little over 10,000 kg, or less than half the amount required to reach their destination. Knowing the problems with the FQIS, Captain Pearson double-checked their calculations but was given the same incorrect conversion factor and inevitably came up with the same erroneous figures.

You can read the whole story of the Gimli Glider in the excellent book Freefall: 41,000 feet & Out of Fuel, which I read on a plane trip to Florida a couple years ago.

No, I have five words for you. The other three are Mars Climate Orbiter:

On November 10, 1999, the Mars Climate Orbiter Mishap Investigation Board released a Phase I report, detailing the suspected issues encountered with the loss of the spacecraft. Previously, on September 8, 1999, Trajectory Correction Maneuver-4 was computed and then executed on September 15, 1999. It was intended to place the spacecraft at an optimal position for an orbital insertion maneuver that would bring the spacecraft around Mars at an altitude of 226 kilometers on September 23, 1999. However, during the week between TCM-4 and the orbital insertion maneuver, the navigation team indicated the altitude may be much lower than intended at 150 to 170 kilometers. Twenty-four hours prior to orbital insertion, calculations placed the orbiter at an altitude of 110 kilometers; 80 kilometers is the minimum altitude that Mars Climate Orbiter was thought to be capable of surviving during this maneuver. Final calculations placed the spacecraft in a trajectory that would have taken the orbiter within 57 kilometers of the surface where the spacecraft likely disintegrated because of atmospheric stresses. The primary cause of this discrepancy was engineering error. Specifically, the flight system software on the Mars Climate Orbiter was written to take thrust instructions using the metric unit newtons (N), while the software on the ground that generated those instructions used the Imperial measure pound-force (lbf). This error has since been known as the metric mixup and has been carefully avoided in all missions since by NASA.

Now, imagine every computer system you know being rewritten to use or convert from imperial measurements to metric measurements. All the Why1K bugs in critical embedded systems. Forget flying. Forget driving. I’ll be walking.

You might be thinking, Gee, Director, aren’t you hitting the luddite thing a little hard these days? Well, I just got a smart phone and a Roku box. It’s an instinctive pushback against my entrance to the 21st century.

(Petition link via tweet.)

An interesting story about the Internet as seen in North Korea:

There’s a curious quirk on every official North Korean website. A piece of programming that must be included in each page’s code.

Its function is straightforward but important. Whenever leader Kim Jong-un is mentioned, his name is automatically displayed ever so slightly bigger than the text around it. Not by much, but just enough to make it stand out.

The article also details the various home-brewed operating systems and whatnot in North Korea.

You know, I think QA’s a pretty high-pressure gig since I’m very eager to make the software perfect before the alpha release even, but I’m not putting my life on the line testing it. I can’t imagine what it’s like to work QA in North Korea.

(Link seen here.)

Noah Sussman has two lists about how developers often bungle time in their applications:

I have repeatedly been confounded to discover just how many mistakes in both test and application code stem from misunderstandings or misconceptions about time. By this I mean both the interesting way in which computers handle time, and the fundamental gotchas inherent in how we humans have constructed our calendar — daylight savings being just the tip of the iceberg.

Read his lists: Falsehoods programmers believe about time and More falsehoods programmers believe about time; “wisdom of the crowd” edition.

Then go ye forth and see if your developers got any of them right.

(Link courtesy Gimlet.)

So I was thinking about going to see the film Wreck-It Ralph to see if I could spot any other testers in the audience betraying the cause and rooting for a defect found in production. I thought about ordering tickets online because of the post-Thanksgiving crowds here in the United States, so I visited the theater’s Web site.

And decided not to.

You see, the first screen allows me to choose a date I want to go to the cinema, and the first choice in the drop-down list is yesterday:

As you can imagine, when confronted immediately by a defect, I did not dare enter my credit card information.

If you’re from St. Louis, Missouri, you might hear whispers in your head saying WTFenberg. WTFenberg. WTFenberg. If you’re not, you probably need some explanation.

Wired has an interesting story from the world of manufacturing: Why Products Fail.

It deals with the fact that entropy will lead to the eventual demise of even the most finely crafted and engineered items. The laws of mother nature, the variability in the repeated processing of materials, and other things work against absolute perfection.

Of course, in the IT world, failure emerges a lot more quickly given the nature of the “engineering” (that is, the cobbling together of Internet examples to barely solve poorly understood problems) coupled with “natural laws” — the physical and technological environment from Web browser versions to commonplace architectures–that change every six months or so leads to far less success, and little fection, much less perfection.

To maintain our sanity, though, we really do have to recognize that things will break down. We just have to keep agitating and pushing to make sure that that eventual failure is more isolated and harder to get to than a couple keystrokes combined with a mouse click.

Ladies and gentlemen, the IBM Jargon and General Computing Dictionary circa 1990.

You can bet your bottom dollar that I’m going to bring some of that slang back. By myself if I have to, but I bet some of you will join me.

I wish I was reading stuff like that on the computer networks in 1990. Instead, I was busy reading the MJ-12 cover-up in raw text files. Just think, if I had wasted my time on the former instead of the latter, I coulda made something of my life.

Thanks be to Gimlet for the pointer.

But it’s who I want to be: The H.P. Lovecraft Institute of Software Design.

Working with Someone You Don’t Like?

Summary: It’s all in your head as you resent your own shortcomings and project them onto QA. Which is not perfect, but is better than you.

Over at Dice TV, Cat Miller identifies some of the things software engineers don’t learn in school:

I agree with many of the things she presents, but I’d also like to point out what other thing computer science students don’t learn in school that’s important, but often overlooked: Domain knowledge.

It’s not the same as listening to your customer, because your customer and/or user has one perspective on domain knowledge, and it might not be deep nor broad. The customer might have just started the job last week and is trying to tell you how to write software to assist him with a job that he doesn’t know how to do himself.

Can you write computer software without domain knowledge? Yes. Heck, my whole schtick is that I don’t need a lot of domain knowledge when testing an application because software fails in predictable places because it’s software, irrespective of industry.

But I do pick up some domain knowledge from a little research to help supplement my understanding of how the user will use the software and to try to predict some patterns of behavior that real users will attempt when confronted with this strange piece of programming during the workday.

I’m not keeping it very secret, am I? Because I want the developers to pay attention to it, too. And project managers. And client engagement vice presidents. Try to glean a little about the problems the software is trying to solve instead of just solutions presented to you to code. Because where your assumptions and their assumptions differ, danger lies.

Luxury Cars Struggle In New Crash Tests:

Last year the National Highway Traffic Safety Administration (NHTSA) instituted more rigorous crash-testing procedures that made perfect five-star ratings tougher to come by. This year the Insurance Institute for Highway Safety (IIHS) is doing the same, and the first round of results finds many popular luxury cars to be lacking. Only three out of the 11 model-year 2012 cars tested under the Institute’s new so-called “small overlap frontal crash test” earned what would be considered passing ratings.

. . . .

In the IIHS’s new frontal test, cars are smashed at 40 mph with only 25 percent of a car’s front end on the driver side striking a five-foot tall rigid barrier. The Institute says the test is designed to replicate what happens when the front corner of a car strikes another vehicle or an object like a tree or utility pole in more of glancing blow, rather than a full-on or offset frontal collision.

This is another industry’s example of happy path testing. The full frontal crash test has thus far yielded results that made car makers design their cars to meet that test, head-on if you will, and to account for what happens in that test. That sounds an awful lot like what happens in happy path testing or the bare minimum “testing” that occurs in demos, internal and external, where someone shows someone else exactly how your developers have expected the user will use the software. Maybe there’s even some validation testing to make sure that, if a collision occurs, it’s handled.

But the real damage, and the real value of QA, is in the oblique testing, where QA doesn’t do what the user is supposed to do, but also does something that the developer doesn’t expect a user to do even when the user isn’t doing what he’s supposed to do.

The trick, of course, is to think that way and to see the possibilities of just tangentally using the software incorrectly or working just a little bit out of order to elicit a response that you can film into a compelling PDA for the benefits of not commuting or you can enter into the defect tracker.

It’s also a keen insight into why QA is never happy nor confidant in a product; sure, it might have passed all the tests (Ha! Just kidding! It just passed enough tests to satisfy the low standards of the project managers), but QA knows it did not think of all the tests, and out there, some user is going to find the right combination of events and the right angle and speed to hit the application to build his own catastrophe. If only I could think of that use case.

Report: Remote Workers More Engaged, Committed:

If your employees come into the office each day, it’s natural to think that they’re engaged and well-connected with one another. But that’s a misperception, according to a recent blog post from the Harvard Business Review.

Scott Edinger, founder of Tampa-based Edinger Consulting Group, wrote that the physical proximity of an office gives the illusion that co-workers are communicative and working together efficiently. The opposite is true, however. Remote workers are actually more engaged and committed to their team, Edinger wrote.

One reason for this, Edinger pointed out, could be that members of virtual teams feel the physical distance between them makes interactions more valuable.

When people do not sit at adjacent desks, they try hard to connect with one another and maximize what little time they do spend speaking with one another.

He wrote: “What’s more, because they have to make an effort to make contact, these leaders can be much more concentrated in their attention to each person and tend to be more conscious of the way they express their authority.”

Working remotely does focus one’s attention and effort to the task at hand and to make sure that when you’re “on the clock,” you’re actually putting forth effort toward the company goals. You’re not as focused on making it until the end of the day as you are getting the tasks done.

I worked at a posting where we went from a remote office setup to an office space, and a lot of the communication remained through IM or email for tracking purposes or because we were too lazy to get up from our desks. As far as communication goes, the only real benefit of being in the office is being able to trap the developer who’s been avoiding you in a cubicle and make sure that he is, in fact, going to hear you out (as will the whole office by that time).

There are distinct benefits to allowing remote workers, especially in IT. I think it will continue, and this article helps bolster the arguments for it.