QA Hates You

Hey, is that another Facebook bug?

Last night, in an embarrassing glitch for Facebook that raises questions about privacy on the site, some users of the social-networking service began getting hundreds of personal messages that weren’t intended for them.

A WSJ.com editor, Zach Seward, tipped Digits off to the apparent glitch after his Facebook inbox was flooded with messages ranging from the mundane to the truly private.

“I am sorry for letting my jealousy and worry get the best of me,” reads one of the emails.

Oh, boy, Facebook is really in the race to obsolescence here. Misdirected personal messages? That’s going to undermine user trust and confidence.

And once one of the other up-and-coming social networking solutions hits a tipping point, Facebook is going to lose users in droves. You can’t keep burning people with poor quality and expect to be an ongoing concern no matter how many Farmville players you have.

Sobe’s new contest misses it by that much:

Click for full size

“In the 10 Ring,” he explained to his foreign readers, refers to hitting a bullseye when target shooting.  Sobe missed the frame slightly here when dropping their Flash application onto their Facebook page.

Want to know what else they did wrong with the contest entry?

• They put the rules in a pop-up window which Internet Explorer blocks since it’s coming from a Flash application.  Instead, they could have put it in a panel in the Flash application.
• They ask for a phone number, but they don’t tell you what a proper phone number is.  You get to try and err.
• They don’t enable the Submit button on the Tell a Friend form unless you enter valid e-mail addresses.  The other steps, though, enable the Next/Submit buttons before the user has filled out the form.

Well, it was good enough to separate Sobe from its interactive budget.  Carry on, then!

The State of Missouri in the United States has uncovered a computer error in its favor:

 Missouri acknowledged Monday that it reported inflated numbers of food stamp recipients to the federal government, calling into question millions of dollars of bonuses paid to the state for running one of the nation’s top-flight programs.

The Department of Social Services said a computer programming error has consistently exaggerated the figures submitted since September 2002.

You know, the most cynical amongst us would claim that computer errors that “accidentally” would give preferential treatment to the organization using or writing the software would get qualified as lower priority defects and would be allowed to run as long as possible.

However, the teams I’ve worked on probably wouldn’t allow that sort of thing to occur.  But maybe I’m not cynical enough.

I imagine that’s what happened here: Programmer slip-up produces critical bug, Microsoft admits:

“Look at the two array references to ValidateRoutines[] near the end,” said Michael Howard, principal security program manager in Microsoft’s security engineering and communications group, referring to a code snippet he showed in a post to the Security Development Lifecycle (SDL) blog. “The array index to both is the wrong variable: pHeader->Command should be pWI->Command.”

One can imagine that developer, cranking along bopping his head to some Shakira, coming to the place to insert a variable name in the code, typing a couple letters and then letting the IDE autofill the wrong variable name.  A code review could catch that, I suppose, if you were diligent enough, but who has time for code reviews when there are deadlines approaching and no time to squander on anything but Rock Band Beatles in the lounge?

Like most everyone else, Microsoft has run into problems with localizing its Web site from English to Polish.  For example, Engadget finds an instance where someone altered, badly, a stock photo instead of getting a new one.

Poking around the site, I found a couple additional flaws.

(more…)

Found on a box:

Uh huh.

You know, if you’re ever called upon to test label printers, scanners, or any of the various and sundry other peripherals your software will use or interact with, you get those printers, scanners, or peripherals and test on them.  Don’t trust the developers and whatever emulators they might have concocted.  Then, build a complete set of test cases for the apparatus.  Then, run those test cases.  Finally, make developers fix the problems you find.

Sure, it’s not as sexy as the main module of your shipping software or the Web interface or the other fun things developers want to do.  However, it is the basic thing your software needs to do.  It’s what your users expect it to do.  And it better do it.

An embedded video not only fails to play, but it fails to show me one error message, I think:

I guess Error 1 of 2 was lost in the mail.

Or is Error #2035 the Error 1 of 2?

Regardless, I think it’s poor juju to show troubleshooting to a user.  It makes an error that is incomprehensible even incomprehensibler.   Show a user-friendly error message, for crying out loud.

As Joe would say, maybe they should have tested more:

Dozens of “temporarily out of stock” signs dot the shelves of some state liquor stores, and store managers say they’re not sure when their complete product line will again be available.

State officials blame the difficulties on a glitch in a new software system that controls the movement of 18,000 cases of liquor a day through the state’s distribution center on East Marginal Way South in Seattle.

The one thing that could possibly upset QA more than a defect is a defect that keeps QA from its liquor.

(Story seen here.)

Joe Strazzere quaffs a tall Starbugs:

For a couple of days in May, coffee giant Starbucks ended up double charging about 1 million of their U.S. and Canadian customers.

• More than 1 million transactions at 7,000 Starbucks locations on May 22 and May 23 were double billed
• The one-million Starbuck transactions involved both credit and debit cards
• The POS charges—and the receipts given to customers—were perfectly in order. The problems kicked in hours later, in the settlement processing area
• Starbucks has declined to say how the glitch happened
• Starbucks has declined to say how they plan to prevent it from happening again
• Starbucks says they’ve had some customers call whose accounts were not fixed
• Any customers with questions should contact the company’s customer relations hotline at 1-800-23-LATTE.

It’s best you not think how precursorily most monetary transactions are tested between the commerce system and the clearinghouse.  I’ve delved into this on occasion, and the process of making that particular sausage doesn’t bother me, but the high number of rat hairs that are acceptable does.

One of the biggest problems with Java is that it doesn’t apparently by default repaint quickly enough.  If you get it thinking about other things, it forgets to update its interface properly.

Take for example this Java applet I found via Knowing .NET.  You go there,  enter a URL, and it displays a word analysis of a Web page, like this one for QAHY:

Click for full size

Now, click Open in Window to pop it into a new window.  The applet beneath remains.  Now, drag that new window over the original applet, and it doesn’t repaint itself while the window is open:

Click for full size

I’ve seen this many times in desktop environments, too.  Sadly, Java seems unprimed to repaint smoothly without some attention from its developers.  Regardless, in non-Java environments, this unrepainting tends to occur when the application is biting off more than it can chew of the processor or something’s going horribly wrong, and users will pass that particular sentiment over to your Java application that might be otherwise working fine.

And if the users think something’s screwed up, they’ll chew up your support time or not use the product at all, which seems to me less ideal for your organization than just inserting the code to handle this better.

It’s a simple Flash widget, with simple actions.  When the user mouses over a word or the line beside the word, the panel to the right displays the associated word and its poll values and the word itself displays in bold font.  Simple, right?  Apparently not simple enough to get it right.

Here’s what the Flash widget looks like:

Click for full size

Notice that if you mouseover the line beside Confident or the corresponding percentage, it displays the panel to the right correctly.  However, the word Confident in the table flickers between bold and normal font.

How many test cases would you need to make sure this worked right?  I mean, mouse over each, check the data, check the altered display state.  It takes a couple of minutes to do it, a couple more minutes if you’re wise enough to check it in other versions of Flash and without Flash installed (to check the page layout).

However, the people behind this page did not test it adequately or at all.  And they bollixed something simple.

Yeah, I know, it’s a bloated bit of processes bogging down my PC at all times, but I’ve got a brand loyalty to Norton products going back more than a decade.  He said, justifying finding the problem he notes below.

So it runs one of its automated update processes and pops this little monologue box up on the screen:

So I foolishly simply click the X at the corner of the dialog box designed to dismiss the dialog box; however, this is a monologue box.  Suddenly, all of my application windows start closing.  Symantex has chosen to rewrite a commonly understood piece of GUI standard behavior and has turned this close button to an Apply button.  I supposed if I’d lost some actual data instead of just a collection of tabs I’d opened in an Internet browser, I’d be a little more upset.

I’ve seen a pile of that sort of thinking in actual design, not just in stupid bugs.  Designers and developers build out a GUI where the moving parts don’t behave like other applications or Web sites.  They think it makes a nice little stamp of their individuality upon it, much like they think the Apple stickers on the back of their cars identify them as individual thinkers.  But in the case of building applications and sites useable by normal people, it falls flat.

Listen, guys, I could come into the next meeting spouting hipster beatnik slang straight out of a 1950s coffee shop, complete with bongos, and give a status report that will make me feel individual but won’t do any good to anyone else.  That’s what you’re doing when you do incomprehensible or unexpected things with controls or buttons.  You make the user feel foolish or worse make a mistake of varying cost.

So, for Pete’s sake, do it like everyone else is doing it.  And me, I’m going to continue giving my status reports in gritty 1990s urban slang.  For the authenticity.

Don’t forget that the garbage you put into the system might come out somewhere else, badly.

Here’s an example, courtesy of the Milwaukee Journal-Sentinel’s Weather Cam which is supposed to show you how wonderful the intersection of Kilbourn and 3rd street looks when covered with a foot or more of snow.

Well, if it wasn’t currently displaying the default admin sort of thing that tells the Web masters at jsonline.com that the media player is installed and ready for action, that is:

Click for full size

I’ll leave it for the reader to speculate whether the genius behind this story embedded a live camera feed for a temporary camera placement, resulting in the admin screen when the paper’s personnel moved the camera.

Instead, let’s talk a bit about the media player and what you should check if you were to test it.

(more…)

Step 1: Have an unnamed Product Update screen display by the system tray on product startup:

Given how long the newer version I have (Paint Shop Pro Photo X2) takes to load, I often click its icon and then go back about my business so it can show its splash screen for 30 seconds while I work in an active application.

Step 2: When I click for More information, show me yet another screen that doesn’t tell me any information.

Click for full size

Well, what can it hurt? By this time I’ve figured it’s probably PSP since I can sometimes get to this screen and sometimes I can get to PSP. So I start the download, hoping I’m not getting Weatherbug 2009.

But here’s the thing: in the middle of the download, I decided to lament to it to you guys, and I clicked Cancel to stop it so I could get those lovely screenshots for you. The Updater dispelled the progress bar window, but it left Paint Shop Pro in a modal form so I could not actually get to it. I had to kill it from the taskbar.

Eventuallly, though, after I mucked around with those things enough, I got to the installer, and it finally, finally identified the product:

Click for full size

Corel should have branded each and every of the preceding screens, but for some reason did not. Poor form, Peter. Now, maybe some day they can un-screw-up a great product that they had to tart up and, more inanely, change keyboard shortcuts that had been part of the product for a freakin’ decade. Yeah, when I get wed to a product, I get wed to a product and its custom shortcuts.

This is the sort of defect I really hate.  Not that it’s broken, not that it’s a simple fix, but this is the sort of defect that you spend more time arguing about than it would take fixing.

The application is a little flow charting piece called RF Flow.  I recommend it for building handy flowcharts to graphically illustrate the processes that everyone in your organization ignores.  However, right after you first install or reinstall version 5 (four years old now), the recent file list includes empty numbers since you haven’t actually opened 9 files yet:

Click for full size

It would be simple enough to not populate the menu with recent filenames where the recent filename==null.

However.

In a real development environment, the sequence of this would be:

1. QA logs the defect.
2. Project manager, who’s been happy path testing and whatnot, reviews defect and says he cannot recreate it.
3. QA explains that it only happens when the recent file list is empty.
4. Developer says it won’t be a problem once user has opened 9 files, so  it’s not worth fixing since there’s a workaround.
5. QA says, come on, it’s a single conditional.  We could fix it ourselves but we don’t fix problems.
6. Developer comes back from lunch at the Thai place and says, but how many people will see this problem?
7. QA responds, “How many customers will we have?”
8. Developer recommends that on first launch, application should open 9 sample files maybe.  He’ll wait for someone in the Training/Documentation department to create them.  As soon as the company creates a Training/Documentation department.
9. Quibbling continues until launch date approaches.  As this small thing is not a critical defect, it does not stop the launch.
10. Developer attends launch party; QA waits in its lair, plotting against or hexing the developer who spent several hours over several weeks dodging a far smaller amount of work.

From this crucible, Known Issues Lists emerge.

And hey, RF Flow is an easy tool for flowcharting and cheaper than Microsoft Vizio.

The developers of the Circuit City store locator fail logic.

Here’s the situation.  You’re a user in tiny Grantwood Village, a mostly forgotten municipality in St. Louis County, Missouri, who wants to go to Circuit City because….well, okay, maybe it is an outrageous use case, but it fails:

1. Go to the Circuit City home page.
2. Click the Store Locator link at top.
3. Store Locator displays:
   
   In the City edit box, type grantwood village.
4. From the State drop-down list, select Missouri.
5. Click Find.
6. Uh oh.  According to Circuit City, Grantwood Village does not exist:
   Much to the chagrin of Grantwood Village.Well, then, type the zip code of Lakeshire, Missouri (63121) into the Zip code edit box.  Funny, though, Lakeshire is even smaller than Grantwood Village, as it’s essentially a small subdivision with a post office.
7. Click Find again.
8. The application acts as though the zip code is invalid:
   

This occurs whether you click the Find button underneath the Zip code edit box or underneath the City/State combination.  Don’t get me started about the design wisdom of putting two controls on a form that do the same thing.  You cannot convince me of its utility, and I disbelieve in your value of symmetry.

In this form, if the application detects a value in the latter, it ignores the former, period.  So it does sort of handle Or (you need to enter something in one or the other), it does not handle both (And) correctly.  Even though someone will probably encounter the situation of entering data in both forms.

And, when you’re feeling particularly nasty (which is to say, every day of the week), remember to try 87894.  This is an invalid zip code, and if your application doesn’t handle nonexistent zip codes (not merely strings that are not five numbers) or relies on a Web service call or whatnot to an application that does not handle nonexistent zip codes, hilarity ensues.

The company formerly known as Diebold but changed its name when Diebold became a bad word due to poor quality in electronic voting machines admits that its voting machines might have lost a couple votes here and there:

 A voting system used in 34 states contains a critical programming error that can cause votes to be dropped while being electronically transferred from memory cards to a central tallying point, the manufacturer acknowledges.

The problem was identified after complaints from Ohio elections officials following the March primary there, but the logic error that is the root of the problem has been part of the software for 10 years, said Chris Riggall, a spokesman for Premier Election Solutions, formerly known as Diebold.

Acknowledged, but not before running through the gamut of excuses:

As recently as May, Premier said the problem was not of its making but stemmed from anti-virus software that Ohio had installed on its machines. It also briefly said the mistakes could have come from human mistakes.

But the important thing, I suppose, is that Diebold met its deadlines and had the faulty software up and running in time for the elections and in time to cash the government contract checks.

A poster to the ComputerWorld Shark Bait forum discovers why 214-748-3647 is such a popular phone number.

The answer, of course, is lack of quality assurance.

Target’s Find a Store feature offers limited usability for its Get Directions feature, since it obviously lacks insight into inconsequential details like street names:

Click for full size

That’s a mapping utility of dubious usability. Now, if you wanted to go Southern, I guess they could insert landmarks properly:

1. Y’all go down to the old Phillips station that’s now a used tire shop and turn RIGHT (West).
2. Go over the crick and on the little dirt road by the magnolia tree where Bud Walters crashed his bike and turn RIGHT (North).
3. When you get to the pole where Jeb’s scarecrow used to be, turn RIGHT (North)….

Nah.

Instead, we’ll launch this application with a couple of known issues.

Son of a glitch:

Paying bills online is fast, efficient, and you save the price of a postage stamp.

But how safe are you? Every company with an online payment system says you needn’t worry about privacy or security.

Leigh McDowell believed it.

She’s a paralegal in O’Fallon, Ill., and pays her $60 monthly cable TV bill to Charter Communications online on her home computer. She has paid electronically without any problem for five years.

Tuesday morning, she entered her regular logon and password, but got the Charter account of a woman in Kingsport, Tenn., instead.

It showed the woman’s full name, address, phone number, security code number, her cable TV service (the “Big Value Package,” with Digital Sports View), her high-speed Internet service, and her Charter telephone service (she paid $1.79 for one directory assistance call) — and her bill for $237.16.

“No stamps, no check, no hassle!” Charter promises on the woman’s electronic bill.

McDowell was horrified that she had somehow gotten into a stranger’s account. She quickly logged off. Besides, she still had her own bill to pay.

She tried again to log on to her account. This time she arrived at the Charter account of a woman in Slidell, La.

McDowell logged off and tried again, this time arriving at the Charter account of a woman in Covington, Ga.

McDowell says she did this 20 times, each time getting the account of a different Charter customer. She couldn’t see any connection between the names or addresses, although she did note that many of the accounts listed overdue bills.

“I just kept entering my name and password, and every time it gets another account,” McDowell said. “I could see everything, just as if I was that person.”

McDowell is so upset, she says, she won’t pay online again.

Here’s a quick question for you: Do you, as a QA person or other IT person, use online bill paying? I sure do not; I have worked with too many billing systems and have seen what kind of issue passes as an acceptable risk for a deployment to production. Your chances of winning the lottery don’t seem to be that far lesser than encountering a billing error. Some of which come with special “prizes” of their own in the form of extraneous, incorrect debits that can cause your bank’s overdraft charges to kick in, thank you very much.

I’ll stick with the chance of human error or data entry problems that come with writing a check, thanks. I can forgive human error, not computer error that’s engineered into a software system.

(Yes, I know that’s human error, but in too many cases, it’s human error that’s overlooked and unrepaired for a variety of rationalizations, at which point it goes beyond error into willful malfeasance.)