QA Hates You

Joel Spolsky of Fog Creek Software explains some of the thinking behind Evidence-Based Scheduling included in the new release of FogBugz.

Over the last year or so at Fog Creek we’ve been developing a system that’s so easy even our grouchiest developers are willing to go along with it. And as far as we can tell, it produces extremely reliable schedules. It’s called Evidence-Based Scheduling, or EBS. You gather evidence, mostly from historical timesheet data, that you feed back into your schedules. What you get is not just one ship date: you get a confidence distribution curve, showing the probability that you will ship on any given date.

Honestly, that’s what you ought to be doing if you’re taking a scientific approach. However, your organization and its schedule builders aren’t scientific, preferring instead to build timelines and effort estimates to fit external constraints, deadlines, or budgets instead of reality.

So, carry on with those unwritten tasks of covering your rump when failures occur.

Some developers think the words “Training Issue” are a parry five when it comes to defect resolution. The developers will mark the issue as resolved/won’t fix or some such nonsense, brush the crumbs of QA intransigence from their hands, and go back to voting for the Nintendo Wii over the XBox in an Internet poll.

I don’t know where the developers got the idea of this mythical training upon which they hope to rely to cover their deficient code; none of the places for which I’ve worked or contracted in this century have had actual training departments. Or much of a documentation department. So the developers who deploy the “training issue” defense really mean that a) This issue will be brought up in the 30 minute demo with the check signer where the check signer decides whether to sign that check or b) They really, really hope that a user doesn’t find it or c) “Training Issue” means “customer support issue.”
(more…)

If you want it quick and dirty, you’ll surely get it dirty.

You might have heard that the definition of insanity is doing the same thing over and over again and expecting to get a different result? Software and Web applications are much the same way, in a twisted fashion of their own. You’re insane in QA if you do the same thing over and over again and expect to get the same result.

That’s why QA has to check everything, all the time, over and over again. The simplest and most slam-dunk, we’ve done this a million times before things. The things everyone else takes for granted. For example, let’s look at a simple state combo box/drop-down list.

(more…)

Whenever I bring up performing data validation on information returned from the database to the application, they tell me it’s a waste of resources. However, I think Robin Harris might agree with me.

(more…)

Remember, computer users are not all geeks, nor are they godlike rockstar developers who live on Web logs, twitter, usenet, or whatever today’s cool means of intrageek chatter is. Here is your computer user:

Pensioners surfing the internet are spending more time online than their younger counterparts.

So-called “silver surfers” dedicate an average of 42 hours a month to the World Wide Web, compared with 37.9 hours among 18 to 24-year-olds.

Those are the computer users who need the bumpers and the training wheels and all the mechanisms within your Web sites/applications. If it’s good enough for those who quote Office Space or Hackers or Star Wars all day, it’s not good enough to ship. It has to be good enough for your grandmother who’s one of the 12 million people still dialing up through AOL.

Pardon me for harping on this again, but I spent several hours last night trying to explain client-server technology, again, to someone who asked me how to move image files (my term, not hers) from My Documents to My Pictures.

If you’re in the software industry and you actually, you know, run the software (which excludes most managers, client facing people, technical writers that produce the manuals, many developers, and some quality assurance professionals whose jobs rely upon creating pretty reports with neat statistics), you’ll probably encounter an issue, otherwise known as a defect or a bug, with your application or Web site. What should you do? If you ask a developer, he’ll probably tell you don’t do that. However, you should probably log an issue or otherwise report the problem so that your organization can earnestly act as though it’s going to fix the problem.

Many organizations create elaborate procedures to trace the defect’s accountability and to standardize defect report information. Many software packages ensure that users enter the same sorts of information for a defect report, but those expensive applications do nothing to audit the quality of the report that the users enter. Some organizations just use spreadsheets and misplaced e-mails in lieu of spending money or installing Bugzilla. Regardless of what technology your organization uses, the quality of the content within the defect report is more valuable than the most rigorous procedures in handling the defect report.

(more…)

Remember, Project Managers, No is an unlimited resource; you can use it as often as you like to deal with timeline compression and feature creep, and you will always have more for the next time.

And here’s some new information for everyone in direct contact with the client: that word that everyone’s always talking about is pronounced noʊ. Try to use it in conversation with the client sometime.

I’ve encountered the question in interviews before, “How do you test without requirements?” The SD Times offers an article entitled “Testing Without Requirements—Impossible?”

As crazy as it sounds, there are actually teams of software testers that perform their job without the slightest clue of what the application under test is supposed to do. How is this possible?

Jumping Jack Sprat, how do you test with requirements? I’ve heard mysterious talk about these mythical beings upon some pantheon of BusinessAnalympus somewhere, where the gods of projects have come together and thought about how things should go, but I’ve never seen them myself.

(more…)