Archive for May, 2012

Something Else To Look At

Thursday, May 31st, 2012 by The Director

A reader sends in a pointer to her own site: F— Yeah QA on Tumblr.

“What The Director?” you ask. “You’re bleeping out naughty words on your blog?”

Well, yes, as a matter of fact, I am. I don’t tend to use naughty words that anyone else understands in the workplace, and I also have to recognize that some readers are at work, works that might not be happy that their QA staffs are reading a subversive little piece of work like this. So I try not to include words like —–, —–, or German words like ————-chtch———g to keep the Internet filters pure and clean.

QA Music: All of These Days

Monday, May 28th, 2012 by The Director

The Foo Fighters explain the QA Mindset in their song “These Days”:

Share that with your Project Manager Polly Anna.

The Pain in Obsoleted Functionality

Friday, May 25th, 2012 by The Director

Buzzfeed has a list of words you cannot use in Twitter.

Because of the censorship? No, because:

A majority of these commands aren’t things you’d otherwise use in a two-word tweet, and most only work through the SMS service — you can tweet “suggest something” just fine from a computer or app, for example. A couple years ago you could even type “accept [username]” and force anyone to follow you, a command that, like “get,” had been lying mostly dormant for years.

That is, old functionality hanging around awaiting someone to accidentally trigger it.

If your application is robust enough, you’ll run into the same sort of danger. If your application is robust enough, and your organization turns its QA staff and other staff over quickly enough that nobody knows what was going on five years ago, woo boy. You’re asking for it.

(Link via…oh, I forget, but thanks!)

The Computer Killed Them All

Thursday, May 24th, 2012 by The Director

An oldie but a goodie:

Court officials have figured out why Hartford residents were excluded from Federal grand jury pools over the past three years: The computer that selected names thought everyone in the city was dead.

Sure, that’s from the New York Times in 1982 1992, but rest assured, I’m not 20 years behind in my newspaper reading. But I am a month behind in my Stupid History 2012 desk calendar, which explains why I’m getting to the April 20th sheet today.

UPDATE: New commenter kbiel pointed out that I had the wrong year attributed to the article. I’ve corrected it immediately.

I Build My Walls out of Metallica

Thursday, May 24th, 2012 by The Director

Scientists and interior designers are starting to think that short cubicle walls and open floor plans are neat places to visit, but you wouldn’t want to work there:

Cubicle culture is already something of a punch line — how many ways can we find to annoy one another all day? — but lately the complaints are being heard by the right people, including managers and social scientists. Companies are redesigning offices, piping in special background noise to improve the acoustics and bringing in engineers to solve volume issues. “Sound masking” has become a buzz phrase.

Scientists, for their part, are measuring the unhappiness and the lower productivity of distracted workers. After surveying 65,000 people over the past decade in North America, Europe, Africa and Australia, researchers at the University of California, Berkeley, report that more than half of office workers are dissatisfied with the level of “speech privacy,” making it the leading complaint in offices everywhere.

“In general, people do not like the acoustics in open offices,” said John Goins, the leader of the survey conducted by Berkeley’s Center for the Built Environment. “The noisemakers aren’t so bothered by the lack of privacy, but most people are not happy, and designers are finally starting to pay attention to the problem.”

I prefer a QA lab with walls because it’s so much harder to get heavy metal to bounce off of the walls without the, you know, walls.

That, and QA requires a lot of focus, and people popping by all the time or even the fleeting glimpse out of the corner of my eye can be enough to make me fear I’ve missed something.

I See This Error Message Every Morning in My Closet

Wednesday, May 23rd, 2012 by The Director

So I’m trying to get to MSN’s Last Night on TV feature to find out if Donald Driver won Dancing with the Stars (not because I like dancing, mind you, but because I like the Green Bay Packers. For Pete’s sake, I even watched the episode of Criminal Minds that featured a speaking part for Greg Jennings.), and my IE browser is set to scream bloody murder if there’s any problem with the JavaScript.

So I get this reminder that the black background of my tie doesn’t exactly match the black slacks I’m wearing:

My style = null

Either that, or the Web site has problems.

Basing Your Compatibility Matrix on a Press Release, Redux

Wednesday, May 23rd, 2012 by The Director

I’ve said it’s dangerous to base your Web browser compatibility testing matrix on a press release.

But this story might have some use to you:

Google’s Chrome edged past Microsoft’s Internet Explorer (IE) last week to become the world’s most widely used browser, according to data from an Irish metric firm.

Chrome’s average usage share for the week of May 14-20 was 32.8%, said StatCounter, an analytics company that tracks browser and operating system trends. For the same week, IE’s share was 31.9%.

If you read the whole story and not just the headline, you’ll find that this metrics-providing firm used some data modeling to conclude as it did, and that other firms with other ideas about data models continue to come up with different results.

However, you can learn something from this:

  • It’s important to continually re-analyze your assumptions.
    If you thought it was important to test a browser last year, you might need to change your Web testing to accommodate the changing realities. I acknowledge this so much that I’m no longer mentioning testing in Netscape or AOL Explorer even though I still have those browsers installed in the lab.
     
  • Just because it’s a cool browser doesn’t mean you shouldn’t test in it. Or, more to the point, because it’s an IE and Firefox world in the popular culture consumer mindset world (in the popular culture, the world runs Safari. Inspect every television program, commercial, or print advertisement showing a Web page, and 97% of the time, you’ll see the Safari browser window around it, or I’m not a guy with an English degree just making statistics up). More to the point, it’s important to remember that sometimes you do need to test using the things your designers and developers think is cool. They’re not always wrong, just mostly.

SDTimes Has It In For Testing This Month

Tuesday, May 22nd, 2012 by The Director

SD Times magazine has it in for testing in the May 2012 issue.

First, in the article on Kanban (“Kanban: Is It In The Cards?“), when it comes time to illustrate a blocked task, we get a likely scapegoat:

Testing is the scapegoat again

Secondly, we get an Industry Watch column “Testing culture undergoes dramatic shift” that describes how testing is becoming more relevant by adapting itself to what the developers want or by turning into tech support people.

This culture shift in testing might help testers shed the image of being impediments to software releases instead of facilitators of quality software releases. “How do we work with the development team in this brave new world?” Sterling asked. “The role of testers in the cloud becomes a huge value-add. Testers triage the feedback data, and turn around to tell developers, ‘Your customers want features 5, 10 and 43.’

“All of a sudden,” he said, “your success in getting your next check is pinned on testers getting you the feedback data to deliver what those customers want.”

I agree with some of the sentiments in the article, but I’d like to say that it’s not that testing is changing to fit the needs of software development. I’d like to think quality assurance professionals are helping to change the whole kit-and-kaboodle to provide better quality.

As long as we’re actually making the developers do what we want and making them think it’s their idea, though, we’re golden.

QA Music: Bullet in My Hand

Monday, May 21st, 2012 by The Director

“A Bullet in My Hand” by Redlight Kings.

Caution: The song has fewer f-bombs in it than the first chapter of my novel John Donnelly’s Gold, but that is a nonzero number, so if you insist upon turning it up in your workplace, HR might like to have a word with you.

More Thoughts On Third Party Scripts

Friday, May 18th, 2012 by The Director

Joshua Bixby has an article about how third party scripts on your Web site can seriously hinder the Web site’s performance (Has your site’s third-party content gone rogue? Here’s how to regain control.)

In addition to the performance issues, you need to consider the following dangers and drawbacks of introducing third party code into your application or Web site:

  • You have no control over what they do.
    Sure, they tell you they do something, but that might not be all that they do. For example, a number of years back, I recall a Web site visit tracker that provided a “free” version and a paid version. A lot of people went with the “free” version, which not only provided rudimentary statistics on your Web site, but also served pop-under ads. By that time, most browsers allowed pop-up blocking, this was not always the case, and the host was making money on its users’ content. The provider of this free utility did mention it was going to do it in the terms of use, somewhere around the term that said you could not use the Web counter on Web sites discussing John Norman’s Gor books (no kidding). So not many people read it.
     
  • They can be an attack vector for malware.
    This is a corollary of the above point, but it’s worth noting in its own: Not even the third party vendors, especially ad delivery services, have control over what the code does. In many cases, that’s left to the person who buys the ad, and sometimes that’s a bad, bad man who wants to do bad, bad things to user computers and inserts attack code into ads that the third party code serves up. As a matter of fact, the last attack I know of on my client machine came not from a Web site discussing John Norman’s Gor books, but from the live stream page of KMOX radio, a CBS affiliate in St. Louis, where one of its ads tried a JavaScript exploit on me.
     
  • You have no control over quality of the third party code.
    No matter how much or how little you test your Web site or application, you can rest assured the third parties test their stuff less (even if that is, in fact, a negative number). Many of the JavaScript errors I see when careering around the corners of the Internet stem from missing objects associated with third party code. This might not adversely impact your Web site, but we don’t like to deal with might not as a plan of action in QA, do we?

I realize this is a repeat of what I have said early and often throughout the almost five (!) years of the blog, but the above article gave me an excuse to repeat it again.

(Link via Scott Barber tweet.)

While We’re on the Subject of Cartoons

Thursday, May 17th, 2012 by The Director

XKCD uncovers a bug that QA should always find:

(Thanks to the most beautiful developer I know.)

Book Report: Dear Valued Customer, You Are A Loser by Rick Broadhead (2004)

Thursday, May 17th, 2012 by The Director

Book coverYou know, reading horror books doesn’t keep me from sleeping like a baby (a colicky baby) at night. What do I read to give me chills and to keep me awake in the darkness, staring at the ceiling and contemplating dark things that might snatch my life away? Books like this book.

The subtitle of the book is And Over 100 Other Stories of Embarrassing and Funny Stories of Technology Gone Mad. It collects a number of humorous incidents where software or software-related processes have gone awry and made the papers, causing great embarrassment for the companies responsible. I wouldn’t call them epic fails, because in the 21st century, epic fails are fleeting. These are legendary failures still half-remembered and fully documented for posterity.

Reading through the book, one identifies some areas of risk to pay particular attention to if you’re trying to prevent your company’s failures from becoming the stuff of legend and Snopes articles. These include:

  • Preventing the leak of test data.
    In many of the stories, great fun happens when test data or placeholder material goes into production. Such as the titular “Dear customer, You are a loser.” email or the “Rich Bastard” test name in a mail merge. When you’re creating test data, don’t be clever or wry, since that might leak out. Play it straight. And for Pete’s sake, figure out how to purge it before it gets out there.
     
  • Review your CMS procedures.
    A lot of the news-stories-that-aren’t-real tales in this book come from instances where content authors somehow put their incomplete works into draft and they end up live on the Web site. This might be because the content management system has issues, or it might be because the content author has the ability to publish his or her own work and inadvertantly does so before the proper time. Sometimes, this happens without a CMS where code roll-ups get promoted with draft content. Regardless, you need to scrutinize those procedures to minimize the chances of this happening. As a bonus, one of the stories is about a journalist whose story gets promoted to the live site with disrespectful placeholders within it. Have I mentioned that’s a bad thing?
     
  • Not understanding practices of the users and building in problems through ignorance.
    Then there’s the story about the guy who got a license plate that said NO PLATE and ended up getting the tickets for every car in the state without a license plate. Or at least those where the police officers had written No plate for a license plate number. If you don’t know what your users’ habits are, you can walk right into a problem where their habits conflict with your software’s interface and abilities.
     
  • The impossible calculated numbers.
    The book is rife with the stories of impossible calculation results, such as the trillions of dollars in library fines, the bajillions of dollars in water meter charges, and so on. Does your software have a sanity check to flag outlying calculation results? If not, why not?

This book has a lot for the software quality professional to learn. It exposes patterns of failure we need to recognize and to account for in our testing and rolls up a whole lot of lessons learned meetings into a very browseable 300 or so pages.

Definitely recommended.

Books mentioned in this review:

Error 37, Where Are You?

Wednesday, May 16th, 2012 by The Director

Apparently, they’re on the Blizzard servers:

The Diablo 3 servers are at full capacity, preventing many from playing the game.

Players across the globe are reporting “Error 37″ when trying to log in following Diablo 3’s midnight launch in the UK at 11pm last night and, just hours ago, on the West Coast.

“Due to high concurrency the login servers are currently at full capacity,” Blizzard wrote on the Battle.net forum. “This may cause delays in the login process, account pages and web services.

The best part, or worst part, depending upon whether you’re a mere observer or a customer who plunked down $60 for the game: Blizzard actually warned they weren’t going to have enough server capacity to handle their user needs in a blog post last week. And didn’t accommodate the usage spike until it happened.

(Seen via Fred Beringer tweet. I’m not a fan of the video game series. It reminds me too much of my day-to-day work.)

Measuring and Improving Risk Intelligence

Wednesday, May 16th, 2012 by The Director

Here’s a book excerpt in the Wall Street Journal on improving your judgment of risk:

Most of us have to estimate probabilities every day. Whether as a trader betting on the price of a stock, a lawyer gauging a witness’s reliability or a doctor pondering the accuracy of a diagnosis, we spend much of our time—consciously or not—guessing about the future based on incomplete information. Unfortunately, decades of research indicate that humans are not very good at this. Most of us, for example, tend to vastly overestimate our chances of winning the lottery, while similarly underestimating the chances that we will get divorced.

Psychologists have tended to assume that such biases are universal and virtually impossible to avoid. But certain groups of people—such as meteorologists and professional gamblers—have managed to overcome these biases and are thus able to estimate probabilities much more accurately than the rest of us. Are they doing something the rest of us can learn? Can we improve our risk intelligence?

Sarah Lichtenstein, an expert in the field of decision science, points to several characteristics of groups that exhibit high intelligence with respect to risk. First, they tend to be comfortable assigning numerical probabilities to possible outcomes. Starting in 1965, for instance, U.S. National Weather Service forecasters have been required to say not just whether or not it will rain the next day, but how likely they think it is in percentage terms. Sure enough, when researchers measured the risk intelligence of American forecasters a decade later, they found that it ranked among the highest ever recorded, according to a study in the Journal of the Royal Statistical Society.

The excerpt says that you can improve your risk analysis abilities by getting immediate feedback. However, if you’re trying to answer the risk of deploying undertested software with the potential for hidden defects or if you’re estimating the chances of a discovered error occurring in the wild, that feedback might not be immediately available if the circumstances don’t occur until six months after the software is in use.

At any rate, it’s an article worth reviewing and maybe it’s worth getting the whole book Risk Intelligence: How to Live with Uncertainty.

Log a Defect on Captain Sulu

Tuesday, May 15th, 2012 by The Director

George Takei shared this photograph on Facebook:

With OR Without You, not With AND Without You

Class, who can tell me what’s wrong with this picture?

QA Makes Software Development More Like Sports

Tuesday, May 15th, 2012 by The Director

A Non Sequitor cartoon from April 9, 2012:

Non Sequitor by Wiley

Strangely enough, QA does just that.

And, yeah, I am a month behind on the local newspaper. I’m even further behind on the Wall Street Journal, which means when I try to catch up on them, it’s almost like living as Time in Piers Anthony’s Incarnations of Immortality series.

QA Music: Indestructible

Monday, May 14th, 2012 by The Director

“Indestructible” by Disturbed.

It sounds better really, really loud.

See Also

Friday, May 4th, 2012 by The Director

Appearing in the new ST & QA Magazine, it’s “When Users Collide“. (Registration required.)

Thus Spake the QAssandra

Thursday, May 3rd, 2012 by The Director

Computerworld reports IE ‘silent’ upgrade helps put newest browser on Windows: Stats show some Windows 7 and Vista users upgraded to IE9, but the new practice affected few XP users:

Microsoft’s decision late last year to switch on “silent” upgrades for Internet Explorer (IE) has moved some Windows users to newer versions, but has had little, if any, impact on the oldest editions, IE6 and IE7, according to usage statistics.

Being in QA means you get to say “I Told You So” an awful lot. But it never gets old.

A Concerning Metric

Tuesday, May 1st, 2012 by The Director

Hidden in this Forbes article about Amazon.com is a disturbing metric, particularly disturbing if you consider it in any detail. The metric:

Even the tiniest delay in loading a Web page isn’t trivial. Amazon has metrics showing that a 0.1 second delay in page rendering can translate into a 1% drop in customer activity.

Why is this particularly disturbing? Customers go to Amazon to buy. What is that slow page low time doing to your site’s visitors whose attachment and commitment to your site might be much lower?

By the way, you are doing your performance testing from outside the corporate network to get a feel for the load times on the actual Internet, aren’t you? I’d feel a little silly asking it, except I am a seasoned QA consultant. You might not be.