QA Hates You

Well, no, ComputerWorld calls them 5 easy ways to commit career suicide, but I have found them to be effective techniques in establishing proper business relationships when you’re in QA. Particularly #3, Contradicting the boss in public.

I never inappropriately discharged a firearm in the presence of coworkers. I’m stuck on deciding which punchline to go with here, so you choose whichever you like best:

• In all cases, the discharge was appropriate.
• I wish I’d thought of it.

If you’re a grade A QA professional, you’ve managed to worm quality assurance into the entirety of your organization’s software development lifecycle (if you’re grade A+, you’ve actually broken out of the SDLC and have someone from the quality team proofreading corporate communications, werd). That means you get a seat at the table in the requirements gathering process along with some free-talking technical guy who’s really a sales guy with a cert or two, a business analyst if you’re lucky, and a customer relationship management yippy dog who jumps up and down agreeing with whatever the customer says and sometimes with something your company’s representatives say. However you got yourself into this predicament, best practice or not, you have to take care of QA in this meeting, and here’s what I do in that situation, particularly if I find myself in that situation disarmed.

(more…)

Joel Spolsky talks at length about Internet Explorer 8’s upcoming standards adherence and the coming Internet cataclysm because of it.

The piece is long and offers many lessons, but the best one is the recognition that even though you’ve launched a site or Web application that works with current browsers, you’d better test them again when new browsers come out, even if it’s only a quick run through sanity check.

I like to pretend I’m an old-school computer user, steeped in the command line world of old operating systems and the dark screen and keyboard unfettered by the need to use a mouse. I know all the operating system hot keys. I like the Tab key. I don’t like wasting the time to move my hand a couple inches to the left to inelegantly maneuver a collection of pixels on my screen so I can work. I know, with that much attempt at cred, you’d think I’d learn to touch type, but this article isn’t about my shortcomings. It’s about the shortcomings of interfaces that don’t allow you to interact with the application without the mouse.

(more…)

Project Manager Frank Kelly guesses 4 reasons it sucks to be in QA.

I’ll rebut with one good reason about being in QA: Schadenfreude, all day, every day.

A recent article shows how the workarounds put in place in software impedes people. This article, entitled “Apostrophes in names stir lot o’ trouble“, alludes to a number of characters that software designers/developers mishandle:

It can stop you from voting, destroy your dental appointments, make it difficult to rent a car or book a flight, even interfere with your college exams. More than 50 years into the Information Age, computers are still getting confused by the apostrophe. It’s a problem familiar to O’Connors, D’Angelos, N’Dours and D’Artagnans across America.

To avoid SQL injection attacks, which can use the apostrophe, a lot of times developers just prevent users from entering apostrophes into the edit boxes on Web forms.

That’s not the best solution, as the people in the article attest. You shouldn’t let your developers get away with it.

Ah. A Web application has stood up to your rigorous testing in the original test environment. Hey, in a strange quirk of fate, you’re living in that fantasyland where the application is then deployed to a staging server that resembles the structure of a production server so you can run through it again on a final build. The application passes with flying colors, which means it hasn’t bled to death in stage from your repeated thrusts. No, some project manager or customer lover comes in and says, “Ship it.” One of your technical co-workers deploys the application to production sometime in the middle of a night on a weekend or, more likely, 20 minutes before the client expects it.

You, tester, have just one run through it and MillerCoors time, right? Hold on, little pardner, not so fast. Particularly if your application’s production environment uses a load balancer.

(more…)

In the January 2008 issue of Software Test and Performance (available as a PDF), the head of Parasoft, a QA software company, explains when automation efforts can go to pieces.

(more…)

As you expect, gentle reader, even when it comes to checking the links on Web sites, I prefer manual testing, particularly at the onset of a Web site development project. That is, I do want to personally, with my own index finger, click every single link on every single page, including that repeated navigational menu bar that would never, ever change across the pages (the developers and designers say) and don’t tend to change except when they catastrophically fail, for no discernable reason, on a single page.

That’s not to say that automated link checking doesn’t have its place, because it does. The remainder of this piece talks about its place.

However.

(more…)

“Those who would give up Essential Quality to earn a little Temporary Profit, will see neither Quality nor Profit.”

Feel free to use that in your next status meeting, when someone is explaining why the team shouldn’t fix a large number of defects.

In the message from the editor in the November 2006 (pdf) issue of Software Test & Performance, Edward J. Correia expresses a basic belief in the magical potency of automated testing:

Repetitive tasks are bad enough—having to perform them repetitively is insane. If something can be automated, it should be. Even if it takes 10 times longer and costs a hundred times more than the original task itself, it pays in the long run to automate your tests.

I cannot tell you how many times I’ve gone into interviews and sales calls for projects where the other person across the table blurts out the benefits of automation and how the company wants to use automation to build a complex suite of automated tests to ensure 100% code coverage to run with nightly builds to ensure project success.

But the automated test evangelists, speaking in their tongues, are wrong. Sometimes automated testing is a bane.
(more…)

An article at Dark Reading explains Real Data in App Testing Poses Real Risks:

If you use real, live customer data in your testing and development of applications, you may want to think twice about the risks of exposing that data.

Organizations that use live data in their testing do so basically because it makes the testing more real-world and better puts the app through its paces. Trouble is, it also can expose sensitive data to engineering staff who normally wouldn’t have access to that data, as well as to consultants and other outside contractors working with your organization on the testing process.

But you don’t have to use the real thing in app testing and development: “It needs to be real enough, but it’s better if it’s not people’s confidential information,” says Gary McGraw, CTO of Cigital.

Still, it’s common practice among many organizations today. According to a new study from the Ponemon Institute, which was commissioned by Compuware, 69 percent of the over 800 IT professionals surveyed said they use live data for testing their applications, and 62 percent say they do so in their software development. Over 50 percent outsource their app testing, and of that group, 49 percent of them share live data with the outsourcing organization.

The article conflates using real data with using live data, but it’s really two different things, both of which comes with its own risks.

(more…)

Remember, when you try to whitewash a problem, use enough actual whitewash to adequately cover the problem. Otherwise, you’re merely applying an antique finish to the problem.

Sometimes, even if you have a bunch of tools, all problems still look like a nail if you really want to use a hammer.

Failure is cheap.

Really, even the journalists of SD Times ignore what QA might have been telling them:

It seems counterintuitive to think that the biggest time-sink in the application production life cycle would receive the least regard from development managers. However, a survey published by Forrester Consulting has revealed that this conundrum is the cold hard fact for many organizations.

The objective of Forrester’s “Problem Resolution Survey Results and Analysis” was to determine where developers and testers are spending their time and to learn what is automated and what is not. The biggest time drain, according to the managers, directors and executives who responded: investigating and resolving application problems.

According to Forrester, almost half of the respondents require more than an hour to document a problem, and a problem report uses six types of media on average. “That is interesting to us, given the large number of problems,” remarked Eldad Maniv, vice president of BMC’s Identify Software Business Unit.

The respondents spend almost three out of every 10 hours (29 percent) in various stages of troubleshooting: documenting, reproducing or testing. On the average, a problem takes six days or more to resolve, and one in four of the problems reported by a QA or test group are returned as irreproducible.

(more…)

A year and a half ago, Peter Coffee of eWeek wrote a column exhorting software designers to stick to convention when designing application interfaces:

The Web-browser actions of “back” and “reload,” along with a history of visited locations, have become widespread user interface metaphors. Browser-style buttons invite us to explore file systems, review online documents and access digital media—whether or not we’re on a network and whether we’re viewing resources that are local or remote. It figures that just as we reach the point where everyone knows how these things work, the content creation community is changing the rules.

Users have deep-rooted expectations about the semantics of “back” and “reload.” “Back” means “show me what I was looking at before I was looking at this.” “Reload” means “show me the present state of the content whose past state I’m seeing right now.” When designers don’t respect these conventions, users become confused and transactions are derailed.

He’s right, of course, and it goes without saying that in the absence of detailed requirements, testers will test applications to behave like all other applications.

So keep your nonconformist genius to the metal sculptures you make in your backyards  on the weekends.

As some of you know, I am quite the proponent of client-side validation for Web applications and Web sites. The main reasons are speed and load.
(more…)

Joel Spolsky of Fog Creek Software explains some of the thinking behind Evidence-Based Scheduling included in the new release of FogBugz.

Over the last year or so at Fog Creek we’ve been developing a system that’s so easy even our grouchiest developers are willing to go along with it. And as far as we can tell, it produces extremely reliable schedules. It’s called Evidence-Based Scheduling, or EBS. You gather evidence, mostly from historical timesheet data, that you feed back into your schedules. What you get is not just one ship date: you get a confidence distribution curve, showing the probability that you will ship on any given date.

Honestly, that’s what you ought to be doing if you’re taking a scientific approach. However, your organization and its schedule builders aren’t scientific, preferring instead to build timelines and effort estimates to fit external constraints, deadlines, or budgets instead of reality.

So, carry on with those unwritten tasks of covering your rump when failures occur.

Some developers think the words “Training Issue” are a parry five when it comes to defect resolution. The developers will mark the issue as resolved/won’t fix or some such nonsense, brush the crumbs of QA intransigence from their hands, and go back to voting for the Nintendo Wii over the XBox in an Internet poll.

I don’t know where the developers got the idea of this mythical training upon which they hope to rely to cover their deficient code; none of the places for which I’ve worked or contracted in this century have had actual training departments. Or much of a documentation department. So the developers who deploy the “training issue” defense really mean that a) This issue will be brought up in the 30 minute demo with the check signer where the check signer decides whether to sign that check or b) They really, really hope that a user doesn’t find it or c) “Training Issue” means “customer support issue.”
(more…)