Archive for December, 2008

Borrowed Maxim

Wednesday, December 31st, 2008 by The Director

From a post by Michele Smith:

A good plan, violently executed now, is better than a perfect plan next week. (General George Patton)

As a reminder, in QA you get style points if you actually shell-shock a dev team.

A QA Anthem

Tuesday, December 30th, 2008 by The Director

Here’s a little suggestion of a theme to share with your team as you hold your daily two-minute hate before laying into the software:

It sure does get the mind into the right place, doesn’t it?

Guilty Until Proven Innocent

Monday, December 29th, 2008 by The Director

As you know, ungentle reader, I favor making your Web site check the user’s Web browser to make sure it supports JavaScript, cookies, Flash, and all the assorted plugins that your designers and developers fawn over at any given time. However, I’m not in favor of the methodology used by Miller Genuine Draft 64’s Web site.

When you initially hit the site, it flashes on the screen an error message while the site runs its checks:

Instinctive accusation?  Did QA actually build this site?
Click for full size

After a moment, the page displays and does an age check that all liquor sites do:

The screen if you're set up and lucky enough to get redirected
Click for full size

This visible error message displays on the screen when the error condition does not exist long enough for the user to read it. As a matter of fact, if the redirection or redisplay fails, this error message will remain the only communication CoorsSaabMiller has with you. That is, this message can display to users who use their default browser settings and don’t know how to turn on JavaScript (which is already on) or cookies (which are already on).

There are right ways and wrong ways to do this. Preparing and displaying an error message just in case isn’t the right way.

That Ain’t Me

Friday, December 26th, 2008 by The Director

Joel on Software depicts his Director of Quality.

I would be more likely depicted attacking Pam with a waffle iron.

Thank you, that is all.

Garbage In, Garbage Out (Somewhere)

Tuesday, December 23rd, 2008 by The Director

Don’t forget that the garbage you put into the system might come out somewhere else, badly.

Here’s an example, courtesy of the Milwaukee Journal-Sentinel‘s Weather Cam which is supposed to show you how wonderful the intersection of Kilbourn and 3rd street looks when covered with a foot or more of snow.

Well, if it wasn’t currently displaying the default admin sort of thing that tells the Web masters at jsonline.com that the media player is installed and ready for action, that is:

'S no cam, all right.
Click for full size

I’ll leave it for the reader to speculate whether the genius behind this story embedded a live camera feed for a temporary camera placement, resulting in the admin screen when the paper’s personnel moved the camera.

Instead, let’s talk a bit about the media player and what you should check if you were to test it.

(more…)

Axiomatic != Automatic

Tuesday, December 23rd, 2008 by The Director

I only state the obvious because it’s often ignored: Trusting the developers is not a quality assurance process.

Annoys As Designed

Monday, December 22nd, 2008 by The Director

An idea whose time should go: Flash video of talking people who overlay the Web site you’re trying to read wherein the people try to sell you what the Web site is selling or, worse, an advertisement overlaying a newspaper’s Web site that’s trying to sell you something from the banner ad. Personally, I find these ads intrusive and distracting, and they’ll drive me from a Web site faster than anything else, including animated gifs with hippos dancing.

But here’s one that provides a teachable moment about what to test in media players.

(more…)

A Failure of Forward Compatibility

Thursday, December 18th, 2008 by The Director

You know, everyone gives a lot of thought to backward compatibility when designing for the Web.  By “everyone,” I mean, “too few people,” and by “gives a lot of thought,” I mean “occasionally has the presence of mind to clearly state at the outset that the Web site won’t support AOL on Macintosh or WebTV.”  Hey, that’s a great thing.  Were I able to get my way and actually quite arbitrary, I’d say make everything work on the most recent version of Firefox only.  However, that’s not going to work for a majority of actual, you know, users or environments unless you’re designing for a captive, corporate-lockdown environment (where the only installed, supported browser is IE 5.5 on Windows NT because it was good enough for my father, dammit!)

However, if your organization is building Web sites or applications, particularly consumer-facing sorts of things, that you expect to have in production for a year or more, you’d better give some thought to forward compatibility.

(more…)

Metallica: The Essence of QA

Wednesday, December 17th, 2008 by The Director

Linda Wilkinson explains that Metallica expresses the essence of QA:

Thus the Metallica bass line. It’s like, 4 notes, repeated over and over and over and over and over and over and over and over and over and over.

By the end of a Metallica song, I’m a headbanger, all right. Banging my head against a wall until my brains leak out my ears, screaming. I hate anything excessively repetitive. That means I’ve never been fond of instrumental jazz. One instruments flogs the crap out of a theme, then another instrument picks it up and does the same thing. While I appreciate the skill of the musicians, the music itself gives me a headache.

In my experience, that is what the job of QA in a nutshell.

If you’re looking to fill out your QA playlist, remember my suggestions.

Maybe, by DOM, They Misspelled DOOM

Friday, December 12th, 2008 by The Director

A helpful error message offered by advertising delivery software on Salon.com:

You DOOM is already registered
Click for full size

Helpful if you’re a developer trying to use the software that makes the ads, I suppose.

Keep in mind that we in QA are better described as destructors, not constructors.

Bonus Salon huzzahs below.

(more…)

Curse of the Black Pearl Media

Thursday, December 11th, 2008 by The Director

As you know, I have a special place in my heart and on this blog for media companies whose own Web sites are, erm, less than optimal. For example, Black Pearl Media:

No title, no meta, just pretty, pretty Flash
Click for full size

Let’s see, what do we have?

  • No title tag, so it’s forever immortalized as Untitled Document.
  • No meta keywords or description, so its search engine results choose the client login and associated links as well as the privacy policy information as descriptions.
  • Reliance, probably, on an out-of-the-box content management thing that renders many pages with actual Times New Roman font.
  • Reliance on Flash to cover for the fact that the content management system relies on a really old technology (Cold Fusion).

And…..

A Challenge of Consultancy

Wednesday, December 10th, 2008 by The Director

Larry O’Brien makes light of consultants who milk assignments in his latest column for SD Times:

Dear Bob:

I have reviewed your outrageously padded consulting bill to Client X for last month. Well done! In these tough economic times, we here at Lucifer Consulting have to go that extra mile to make sure that our interests—not those of our customers—remain first and foremost in our minds.

As a consultant, I’m always a little concerned that any freelance QA effort will seem to be milking an assignment since any testing effort can expand to fill any and all available time offered for testing.

For example, if all planned test cases pass and you’ve got extra time, you can always come up with additional, more elaborate test cases based on knowledge you’ve gleaned learning the system under test. Or you can test with an additional browser. Or you can do some load testing. Or regression testing on new builds/deployments. Based on the number of defects found (a bunch, no doubt), even retesting issues can take hours, particularly if they require particular setup preparations (see more elaborate test cases above).

Ergo, I have to make sure clients understand how these things take the time and to make sure we’re clear on the priorities of billed time whenever we get into an engagement. You’ve got budget for 20 hours or 40 hours of testing, so here’s how I’ll attack it, and we’ll check in frequently to coordinate on what time we’ve got left and where to focus.

Consultants, or maybe only those with integrity, have to fight against this popular, and sometimes deserved, conception of consultants. But it’s a challenge of perception in a field where performance metrics and the quality of quality assurance can be fluid and hard to measure across projects.

Some days, I wonder why I left the print shop. You could count impressions of the rollers on paper, and you could see flaws in the printing rather easily.

The Beta Fallacy

Tuesday, December 9th, 2008 by The Director

Frank Kelly disparages releasing your software as “beta”:

In the past few years it’s become more and more acceptable to release “Beta” software to the public – almost as if it was a production release.

The reasons for that I believe are manifold but boil down to
1) Gain user feedback
2) Release early to gain “mindshare” and/or get first to market.
3) Your QA process or team isn’t that great (or your unwilling to spend money on them) and you’d rather have users do your testing for you.
4) You need to gain the confidence of your customers / investors in what you’re building.

You know which I suspect, hey? Yeah, number 3.

Of course, it could be #2, too; I recently talked to a company who built a lot of sites, rapidly, without QA, so that it could be first to market. If the sites got users, then the company would fix the sites up. For future efforts, the company was looking for an automated tester to be the solution. Which will lead me to a longer post one of these days.

Kelly ends up with a quote from this piece which explains why, culturally, IT is defining quality down and managing users’ expectations to IT’s benefit, not the users':

I’m mostly tired about the fact that it seems that we all have given up. Tired because . . . . in reality, it’s laziness and a poor job on the manufacturer part that we have accepted without questioning. Instead of calling foul play and refusing to participate, we keep buying.

We haven’t all given up. The remaining squeaky wheels get the ignored.

Pilot to F-Bombardier

Monday, December 8th, 2008 by The Director

From Inside Higher Education, a sad tale of what passes for technical writing instruction in the university:

“I will no longer tolerate,” the chair writes in his letter to my friend, “what can only be described as your insensitive, vulgar, and obscene language in the classroom.”

The colleague’s intent in a graduate-level, academic tech writing class (i.e., not a vocational training workshop) is not just to teach students how to type memos, but rather to challenge students to consider how they know what they know as tech writers. This can be achieved while they expand their knowledge of their field, which exists right in the oily hinge, right in the fishy craw of the intersection of higher education and the corporation. Given the mess such a collision must be, he and I agree, some form of institutional critique is vital, and this sort of three-dimensional, reflexive analysis can, over time, only make students better tech writers. To know your context is to know your work.

Leaving aside the useful things that technical writers should learn instead, such as how to freakin’ open the software they’re documenting (true story, and anecdotal but typical in my experience: after I moved from tech writing to QA, I went back to the technical writing department to answer a question my replacement had with the doc covering a particular screen and couple of controls. I looked at it and couldn’t remember what it meant, so I said, “Let’s open it up,” referring to the application she was documenting. “Open it up?” she asked. “The serials module,” I said. She didn’t have it installed and was writing the manual on it without actually, you know, having seen it in person).

But aside from my riffs on technical writing, it brings to mind a good question: when is it okay to swear at work if you’re QA?

Some of you, perhaps reflecting on the somewhat, erm, challenging tone of this Web log might think I spend most of my working hours in critiquing institutionally like a sailor. However, that is not the case. I mean, contemporary psychology of the younger generations (can you believe there are people actually working who don’t get Pink Floyd, Doctor Who, or Top Gun allusions?) and basic deformalization of the workplace has made it an environment where it’s not just the old boys occasionally letting go a curse or a swear word. At one of my previous postings, the most foul-mouthed amongst the crew was a young woman who barraged every meeting with the f-bomb.

Brothers and sisters of QA, we are not impassioned barbarians at the gates, lobbing dead animals over the walls. We’re cold, cool professionals working efficiently to deflate the overheated rhetoric and lofty marketing-as-project-planning speak with cold hard facts and failed tests. You’re not going to add an edge to your presentation with potty language. You’re just going to make yourself sound like a twit. Perhaps a contemporaneous twit, a twit among peer twits, but you’re not adding authenticity to your speech.

Besides, you have access to the dirtiest word in IT: NO. Use that, and you’ll shock enough people and ruin enough days without references to bodily by-products.

(Link seen on Instapundit.)

P.S., just for discussion purposes, the best Doctor of all: Colin Baker, #6. If you think differently, you don’t belong in QA.

Testing Is Not The Problem

Friday, December 5th, 2008 by The Director

A facile, misleading headline from ComputerWorld: Testing glitches delay launch of NASA’s $2B Mars super rover:

 Testing and hardware problems are pushing back the launch of NASA’s $2.3 billion Mars super rover from next fall to 2011, the space agency announced today.

Doug McCuistion, director of NASA’s Mars Exploration Program, said during a press conference this afternoon that the program has been held up by problems with both the hardware and the extensive testing needed to send a robotic machine to work on Mars. He noted that the issues might end up only delaying them for a matter of weeks, but even that small amount of time would push them past the window for a 2009 or 2010 launch.

No, sir, testing is not the problem.  Issues uncovered by testing and project plans that didn’t allow for adequate time to fix the issues uncovered by the testing is the problem.

But that’s just how management and the marketing team would like to spin it.  It’s not our rockstars in development and design.  It’s those meddling kids in QA causing the problems.

Assumptions: The Bane of Quality

Friday, December 5th, 2008 by The Director

Here in the United States, we have a saying that starts, “When you assume….”  Although it’s true, it’s more to our point to say, “When you assume, you make bad software.”

You see, assumptions are by definition unspoken, undocumented, and too frequently just plain different from one person to another, such as from business analyst to developer.  One will assume one thing, the other will assume something different, and the difference only comes to light some thousands or millions of dollars of development later.

Two cases in point:

VAT Cut Implications for Ecommerce Sites in the UK

In todays [sic] pre-budget report the Chancellor introduced a reduction in VAT rate from 17.5% to 15% to help kick start the economy and increase consumer spending.

VAT has been set at 17.5% since before ecommerce was invented and this latest change is going to cause a huge headache for ecommerce websites.

Some systems are likely to have VAT hard coded into the ecommerce systems but even advanced programs will still need to be adjusted to reflect the new rates.

Hard coding a tax rate?  How optimistic is that?  Well, maybe not so much optimism as expediency, and by “expediency,” I mean “willing to cut corners foolishly to maximize profit or at least minimize the loss on an underbid development project.”  (Thanks to PhilK for the link.)

Myth: Every system has Internet access

I work and live in rural Texas. Software purveyers are providing MISSION CRITICAL software which REQUIRES access to the Internet for some functionality. I did a field (literally, in a field) call on a system which works perfectly near the Interstate 35 corridor, but fails within two hours when further west. The problem? It tries to contact its host system to upload log files via the Internet and, because there is no telephone line attached (or even within a few miles), no WiFi and there are “no bars” for the cellular telephone card, it posts an error dialog and STOPS WORKING! Of course, this little “feature” or “requirement” does not appear in the documentation.

This reminds me of my days testing client/server applications. What if the client can’t reach the server? I’d ask.  That will never happen, the lead developer would say.  I’d send him a snarky remark in reply, but then the e-mail would go down and he wouldn’t get it.

Sometimes, these “oversights” do not even fall under an assumption; they fall under a that won’t happen or that will happen so infrequently that we’re not going to worry about it sort of reasoning.  But in many cases, it’s an assumption that the developer makes because nobody piped up and said otherwise.

That’s why, QA bretheren, you need to stick your head into those requirements discussions and make sure that you get everything, everything, spelled out.  Developers allowed to assume are developers allowed to improvise jazz riffs in the middle of your company’s symphony.  Okay, that metaphor oversells it, but that’s what you’ve got to do.

Because nobody else is going to do it.  They’ve all got Second Life characters to clothe, you know.

PointRoll Calls It A Creative Showcase; I Call It A Shooting Gallery

Thursday, December 4th, 2008 by The Director

PointRoll, a rich banner ad technology provider, offers a creative showcase page that lists some of its customers and what they’ve done with PointRoll’s technologies.  Since so many of you readers come from Internet searches for banner ad and banner ad testing, I’d like to use this bit as a bit of a jumping off point for some things you need to look for when testing banner ads.

For starters, all of the technologies that designers use to make banner ads come without the failsafes you find in IDEs and established development languages that prevent inattentive developers (I’m sorry, that’s redundant) from making basic interface, such as not accounting for standardish interface behaviors (Control+click to multiselect from a list, double-clicking, click and drag, and so on).  Flash and whatnot do not provide those sorts of built-in bumpers, so you have to try to click and drag elements of the ads to see if you can make a mockery of your client’s crafted image.

I wanted to throw that out there to start with before getting into specific uh-ohs I found in the PointRoll showcase.

(more…)

Cooking Your Own Dog Food

Wednesday, December 3rd, 2008 by The Director

An old expression in IT goes something along the lines of eating your own dog food, which means you use the very software you write every day.  Also, it goes to show that people who use that expression tend to think of their own work as grinding horsemeat and mixing in the right amount of ash to meet the minimum standards.  However, Joel Spolsky explains in his Inc. column for December that, as a manager, you need to be able to do more than report cost/benefit analyses and calculate ROI in 3000 words or more:

Anyway, on my first day of work for the sergeant major, I didn’t know what to expect. I was sure it was going to be horrible, a suspicion that seemed to be confirmed when he took me to the officers’ bathroom and told me I would be responsible for keeping it clean. And then he said something I didn’t anticipate.

“Here’s how you clean a toilet,” he said.

And he got down on his knees in front of the porcelain bowl — in his pressed-starched-spotless dress uniform — and scrubbed it with his bare hands until it shined.

To a 19-year-old assigned to clean toilets, which is almost by definition the worst possible job in the world, the sight of this high-ranking, 38-year-old, manicured, pampered disciplinary officer cleaning a toilet was a shock. And it completely reset my attitude. If he can clean a toilet, I can clean a toilet, I thought. There’s nothing wrong with cleaning toilets. My loyalty and inspiration from that moment on were unflagging. Now that’s leadership.

To be a good QA manager, you need to be able to actually perform software quality assurance tasks as well as or better than your underlings.  You should probably be one of the team’s star players, able to teach and inspire and lead by example.

A lot of QA management without this capability is right now sucking wind through puckered lips, but it’s true.  Come crunch time, you have to be able to get in there and contribute to what your team is doing with a late deliverable and a looming deadline.  Walking around with a checklist on a clipboard and you’re only making things worse.

Although you could walk around, like Spolsky, with a power tool.  That sort of energizes/terrifies people.

Bonus defect!  The automatic pager on Inc.com content management system fails.  Go to page two of the article and click either prev or 1 links to go back to the first page, get a free 404 error.

Poor Web Sites Are Not Funny!!!

Tuesday, December 2nd, 2008 by The Director

Wait a minute, where am I?  Oh, yeah, they’re awfully funny in a gallows humor sort of way.  The Onion is trying to mainstream the mirth a bit with a little bit about a poorly designed site:

“All of the teachers have their own profiles and everything,” Orman said while scrolling through the GIF-littered basic HTML design, credited to his former third-grade art teacher Mrs. Wolford. “And look, the cafeteria still serves pizza on Fridays. This is so crazy.” Orman reportedly attempted to sign the website’s guestbook several times, but was unable do to so because of an internal programming error.

Sadly, that punchline applies to a lot of professionally designed sites we visit every day, completely down to the programming errors in production, doesn’t it?

Every Year, Somebody’s Web Site Falls Down

Monday, December 1st, 2008 by The Director

This year’s winner in the Which Web Site Was Inadequately Load Tested Before Black Friday Sweepstakes is Sears.com:

Sears.com was inaccessible to U.S. shoppers for two hours on Friday in what was the most notable Web hiccup of the holiday gift-buying season’s official start.

Sadly, Sears.com also had problems last year, which indicates an institutional problem of some sort, not merely a technical glitch.