QA Hates You

Hey, as a reminder, the QAHatesYou.com store is open. You can order a lot of good QA swag there.

Let them know where you stand.

A couple weeks back (which means it’s now available on the Web), Larry O’Brien covered user testing to show that development shops could actually, you know, see how users work with a software tool.

Nut grafs:

There’s a part of me that loves user testing; the same part that enjoys visiting ThereIFixedIt.com and watching videos of people destroying their trucks by felling trees on top of them. I take comfort in believing, however briefly, that there are people more foolish than myself. My source code often makes me despair of my own sentience, but I like to tell myself that I wouldn’t need to be prompted to use a button marked “search.”

My major reaction to user testing, though, is often resignation. User testing invariably reveals distasteful work—layouts that need total reworking, dead-end navigation paths, and corner cases that the library API doesn’t cover. The interesting algorithmic stuff that you developed while surrounded by a stack of heavily annotated journal articles and intently pored over on a statement-by-statement basis? That stuff works fine.

Unfortunately, the temptation to skip user testing is often encouraged by clients. While experienced developers know that user testing will lead to a certain amount of dismay, inexperienced clients dismiss user testing because they’re invariably overconfident. I don’t think many are as bad as my above-mentioned client, who was confident that word would spread like wildfire that one could cycle colors by directly clicking on the product. More generally, the problem is the opposite: Clients have seen so many mockups and prototypes and test versions that they cannot see it with fresh eyes.

Keep in mind, QA, in all the situations where your organization is too cheap to provide user testing, you have to be the eyes of the user. Designers love their cutting edge design, but applications should not make only as much sense as a Christo or Serra installation. Applications should behave more or less like all the other applications in the whole world regardless for how much disdain the developers have for the bourgeois sensibilities of users. And so on and so forth.

But read O’Brien’s piece as usual.

An author identifies some of the ways Starbucks order taking and processing mirrors software:

I just returned from a 2 week trip to Japan. One of the more familiar sights was the ridiculous number of Starbucks coffee shops, especially around Shinjuku and Roppongi. While waiting for my “Hotto Cocoa” I started to think about how Starbucks processes drink orders. Starbucks, like most other businesses is primarily interested in maximizing throughput of orders. More orders equals more revenue. As a result they use asynchronous processing.

That’s all well and good, but we here at QAHY want to extend the metaphor further.

Ways Starbucks is like a development team:

• They’re very expensive for what you get.
• Anything besides the basic product costs extra.
• The customer is the only QA.
• Those who participate believe their in a superior class, but the rest of us use the word clique.

Ways Starbucks is not like your development team:

• Starbucks produces the same thing over and over again, so they get good at it, unlike your development team who build slightly different things using new technologies they want to learn on the fly. It’s like having every barrista be a trainee every day.
• Barristas won’t try to adjust a drink if the order changes in midstream. They rightfully throw it out and start again and sometimes charge you again. A development team will just try to remix the existing espresso, sugar, and flavor shot so that you get hot chocolate at the end instead of a triple Venti cappuccino.
• Starbucks gets its money up front and doesn’t have to wheedle and do just one more thing to get its paycheck.
• You like to see a Starbucks product or representative first thing in the morning.

And in closing, I’d like to point out that management from your software development team could probably run a Starbucks, but not vice versa.

Feel free to add your own below.

(Link seen on Megan McArdle via Instapundit. That’s enough chain of custody to introduce it as evidence, werd.)

I was reviewing this user interface designed in an old technology, when suddenly I was struck by the infeasibility of the maxlengths assigned to the fields:

Click for full size

That’s a maximum e-mail address of 20 characters. Chop 4 or 5 off for the domain extension, the dot, and the @ sign and suddenly you’re limited to a username and a domain name of 15-16 characters. If you’re not at MSN or AOL and an early adopter at that, you’d better hope that they call you on the phone if you’ve won.

Okay, aside from that, what’s the other problem on the page?

Based on a tweet this morning lamenting the dearth of proper test plan sample documents on the Internet, I put together a sample document in PDF format that you can use when putting together your own test plans.

You can view that sample here.

I hope that my regular readers and especially those of you who got here by a Google search find it useful for your testing documentation.

This does not represent a good practice of synchronizing your application data with the real world:

Twice we had ordered a pizza with extra-large pepperoni. Twice it had arrived without the extra large. See, the order is calibrated to hit everyone’s preferences, and my daughter will only have pepperoni, so: her half has pepperoni and extra-large pepperoni. But twice the pizza has arrived without.

“I’m going to call them,” I said.

“No, Dad, don’t! It’s okay! Don’t make a fuss about it.”

“Honey, a manager would want to know these things.”

So I called, and explained, and the manager asked if I ordered online. I said that I did, modern-type person that I was. That’s the problem. Extra-large has been discontinued, but it’s still on the online menu. Can you tell me what the printout on the bottom of the box said? I noted that it had elided the extra-large issue altogether. So the problem wasn’t on their end. [Emphasis added.]

This isn’t a simple change made on the fly, either. It’s a menu change determined probably by a national pizza chain’s HQ and telegraphed to its franchisees by semaphore or something. Somehow the change managed to dodge the people responsible for the Web storefront.

Forget keeping your application data synched with the other online data. Your application has to keep up with the real world, too. A lot of IT teams and vendors can rationalize not keeping up if the customer doesn’t keep up, but you need to make it easy to change and to grab your client/internal stakeholders by the lapels to ensure they keep you up to date.

James Lileks, from whose blog I took the anecdote, is a patient customer and does not blame his local pizza shop. However, another client will quit a brand for that sort of thing. Especially if that client is in QA.

So the local newspaper, the Springfield News Leader, has a little problem with its site. When you view an article that spans multiple pages, the page numbers or the next page link does not work in Firefox. This is quite the reverse of most software, which typically works in Firefox or Chrome but bombs out in Internet Explorer and the developers sniff about the hoi polloi and their attachment to the dominant Internet browser.

So I go to the send feedback form, fill out a defect report, and….

Click for full size

That’s what I get, I suppose.

In a stunning turn of events, the problems remain unfixed.

India has given itself a new symbol for its currency, the rupee:

More information at Wikipedia, including the nugget that this will not replace the existing Unicode character ₨which, as I understand it, will henceforth represent Ru Paul.

So how will your application like that? I can’t wait to find out.

Here’s a pinch of L7 to start your work week:

Now shove.

So I stopped by the Branson (Missouri) Regional Airport recently, and I spotted this kiosk:

Click for full size

It offers the user the opportunity to enter some sort of contest to go to Nashville. It’s obviously a Web browser in kiosk mode, but this one has a full keyboard with a trackball and two mouse buttons. Uh oh.

So I click the Contest Rules link at the bottom and get the contest rules, which has a naked link at the top that takes you back to the form. But hover over the link and right click and…. Uh oh.

Click for full size

What happens if I open that in a new window? Hello, Internet!

Click for full size

So a user has complete access to the Internet. Go where you want. Get all the malware you want. I didn’t try to see if a regular download and install worked, but I would not doubt it. What happen if I ALT+TAB?

Click for full size

Lookie there! Lookie there! It’s the command line. A little CTRL+C action and I have access to issue commands to the machine and maybe even the network.

So is that Cat-5 cable running out of the back of the box connected to the airport network itself or a dedicated safe portal to the Internet? Given what we’ve seen here, what do you think?

If you’re ever called to check out a kiosk application, not only should you run through the form the kiosk will host, but you should get a kiosk itself and run it through its paces and look outside the confines of the application to look for security pitfalls.

You need to check out the user interface action. This kiosk gives the user all the normal tools that users need for full input opportunity to the Internet. Some kiosks only have touchpads or touchscreens. Here are a couple of things to think about:

• Know your keyboard shortcuts. Most people don’t know these keyboard shortcuts, but they do things to your active window (even your kiosked browser). What can you do with that?
• Know your internal browser behavior. I remember seeing a kiosk with only a touchscreen that offered the Web sites of a building’s residents. Within a touchscreen environment, you would think you’re limited to navigating through links in the browser window. You would be wrong. mailto: links trigger the helper application associated with e-mail. What can you do when you try that?
• What happens when you unplug the machine and plug it back in? It reboots, probably, affording you the ability to go into alternate bootup scenarios and whatnot. Should your user have access to that? Probably not.

To begin vetting kiosks, you need to think outside the terms of your application and think in terms of the technologies that encapsulate it. The better you understand those and can identify the ways users could interact with the whole kiosk, the better you can prevent them from doing so inappropriately.

A bit of Winger to start the week.

In a side not, I am working on my screenplay for Bill and Ted’s Gnarly Career wherein our heroes become IT professionals and meet the archetypes we work with every day.

The Dark Side of Perfectionism Revealed:

Perfectionists, by definition, strive for the best, trying to ace exams, be meticulous at their jobs, and raise perfect children. So one might assume this drive for the ideal translates over to their health as well, with perfectionist being models for physical and mental well-being.

But new research is revealing the trait can bring both profits and perils.

Though perfection is an impossible goal, striving for it can be a boon for one’s health, causing one to stick to exercise programs to a tee, say, or follow a strict regimen for treating chronic illnesses like type 2 diabetes. But the same lofty goals can mean added mental pressure when mistakes are made and the resistance to asking for help from others in fear of revealing one’s true, imperfect self.

In fact studies show the personality trait of perfectionism is linked to poor physical health and an increased risk of death.

Well, I am not in it for my health. I’m in it for the respect and love of my fellow software professionals.

Which means I’ll die young and lonely, but that’s better than dying old and lonely.

(Link seen at Troglopundit.)

Isarian sends along another episode of a media player that displays NaN when it doesn’t know when a program will end:

Click for full size

Come on, guys, is it so hard to think up or implement a conditional that says IF this is a live stream THEN do not show the length of the clip?

Geez, I think I just accidentally did a little freelance software architecture or something. It hurt. No wonder the actual people tasked with the responsibility avoid it, too.

(Previously on QAHY….)

Hotmail displays a bit of helpful troubleshooting information:

Click for full size

I guess that would be helpful if I wanted to know which machine to remote into and up the timeout or rebalance the load balancer. However, for a user, this isn’t very helpful at all.

The phrasing on this survey question stopped me in my tracks:

Click for full size

The label says you should use the number 5 for the one you use the most and the number 1 for the one you use the least. Then it gives you 6 items.

Really, I think the survey writers want you to rate the frequency, but they use superlatives instead of comparatives here, so logically you cannot rank them from the least to the most without leaving one out.

At least not to my orderly mind. Maybe Joe has a solution. His mind is less highly developed, which is why he would not be taking a survey about the Green Bay Packers in the first place.

e-week offers a slideshow of current software development slang.

Personally, I take the number of definitions that relate to bugs and lack of quality as a metric indicating how bad software code is in the industry today.

As if I needed further evidence than actually using software on a daily basis, not including the stuff I test.

First guy in the office, and now you have to wait for the coffee to brew? Here, QAHY has you covered with a little verbal caffeine:

Plus it will give you a little brain teaser as you point out all the misspellings in the lyrics. On one hand, I don’t think English is the first language of the guy who put this together. On the other hand, it’s not the first language of your offshore team who’s responsible for ad libbing the error messages in your application, either.

And remember, Mötley Crüe is a brand name, so it’s supposed to be misspelled.

Apparently, Google’s use of the Beta tag isn’t keeping it out of a lawsuit:

In a frivolous lawsuit that ranks right up there with people suing McDonald’s for burning themselves drinking hot coffee, a woman is suing Google because she claims Google Maps directions led her to a highway, where she was struck by a car.

Sure, the lawsuit probably won’t go anywhere. But Google’s going to have to spend precious coin responding to the papers and whatnot. If the lawsuit gets some traction, Google’s going to have to settle or whatnot.

Keep that in mind whenever your organization pushes something “quick and dirty” live, or when it takes shortcuts with the initial development and promises to fix the shortcomings, bugs, and oversights with later release which will be under the same sort of pressure and process for the new features as was the initial development (that is, the new features are slapped on barely tested and with no bug fixing or revisiting clunky workflow), or when it hides behind a “beta” tag hoping that only smart users will use the application according to the happy path but that might find users are not as clever at sussing out the happy path as are the dev team.

You’re buying a Daily Pick 5 ticket with that philosophy. Sure, your number probably won’t come up, but you’re playing the drawing every day, and if it comes up, you’re going to lose enough to make you wish you had done things a little differently every day.

Well, not for the Finnish readers, since they pause their Nightwish CDs to play whatever I throw up here. Here’s Nightwish playing “Slay the Dreamer” live.

Now go out there and metaphorically slay the dreamer in your office, the project manager who thinks the application will ship with no level 1 or 2 defects on Thursday.

When you have an intersection of public money – software project – no testing, you’re achieving a singularity of gonnagobadly.

Latest example: Queensland (Australia) Health’s new payroll system.

Look at this recipe of success with its pinch of simpleton’s drive for simplicity and its dash of “deadline-at-any-cost.”

Among the board’s decisions was to change the definition of severity one and severity two defects so the project could pass exit criteria.

During testing, the board decided not to undertake a full parallel pay run test because of the size and complexity of the task.

In January, the testing company suggested either the rollout be delayed until a full system and integration test was done, or the board accept that untested scenarios might not go to plan.

The board chose to accept that risk over delaying the rollout.

You can guess how well that went given it’s in CIO magazine.

On the plus side, it was only $24 million over budget, only 60%. Here in the United States, you don’t get enough code written for government projects with a mere 60% overrun to test at all.

(Link seen on Cartoontester’s Twitter feed.)