QA Hates You

If you’re in the software industry and you actually, you know, run the software (which excludes most managers, client facing people, technical writers that produce the manuals, many developers, and some quality assurance professionals whose jobs rely upon creating pretty reports with neat statistics), you’ll probably encounter an issue, otherwise known as a defect or a bug, with your application or Web site. What should you do? If you ask a developer, he’ll probably tell you don’t do that. However, you should probably log an issue or otherwise report the problem so that your organization can earnestly act as though it’s going to fix the problem.

Many organizations create elaborate procedures to trace the defect’s accountability and to standardize defect report information. Many software packages ensure that users enter the same sorts of information for a defect report, but those expensive applications do nothing to audit the quality of the report that the users enter. Some organizations just use spreadsheets and misplaced e-mails in lieu of spending money or installing Bugzilla. Regardless of what technology your organization uses, the quality of the content within the defect report is more valuable than the most rigorous procedures in handling the defect report.

(more…)

Remember, Project Managers, No is an unlimited resource; you can use it as often as you like to deal with timeline compression and feature creep, and you will always have more for the next time.

And here’s some new information for everyone in direct contact with the client: that word that everyone’s always talking about is pronounced noʊ. Try to use it in conversation with the client sometime.

Over at Worse Than Failure, they’ve captured some error messages on marquees and message boards.

Although I haven’t seen any of these particular errata, I have found a number of ways to break kiosks that use Web pages to deliver their content. One of my favorites is to navigate the links to external sites provided that give local details and color and whatnot; within them, you can sometimes find the mailto: links that trigger Microsoft Outlook Express to open up or you can get access to forms and whatnot that you can manipulate with the tap of the screen.

Actually, there’s this kiosk down at a certain building in downtown St. Louis where you can double-tap a row on the screen of the building’s residents, triggering a stack trace. Seems the developer forgot to account for that action; after all, who double-clicks on a kiosk?

Ah, the worst of both worlds.

Gentle reader, I want to let you in on a little soft underbelly your Web sites and Web applications might have. The querystring.

As you might know, gentle reader, the querystring is that junk in the Address bar of the Web application. It includes the URL/pagename of the page your user is accessing, but it also can include parameter/value pairs that server-side applications process. That is, it’s a way that your developers can ignore inserting error-catching logic and show the world the stack traces they’re so proud of.

(more…)

New hacking technique exploits common programming error:

Researchers at Watchfire Inc. say they have discovered a reliable method for exploiting a common programming error, which until now had been considered simply a quality problem and not a security vulnerability.

Jonathan Afek and Adi Sharabani of Watchfire stumbled upon the method for remotely exploiting dangling pointers by chance while they were running the company’s AppScan software against a Web server. The server crashed in the middle of the scan and after some investigation, the pair found that a dangling pointer had been the culprit. This wasn’t a surprising result, given that these coding errors are well-known for causing crashes at odd times. But after some further experimentation, Afek and Sharabani found that they could cause the crash intentionally by sending a specially crafted URL to the server and began looking for a way to run their own code on the target machine.

Funny, most security holes exploit a common software problem: developers wrote it.

Suddenly, all of those issues marked not reproducible or low priority (fix when hell freezes over) would suddenly get new importance. Never mind that, developers, you better check Woot! right now, or you could miss a deal!

I, like many, have tried the Personal DNA test, and it revealed something about me that I had never known:

At least the title and alt attributes are consistently misspelled.

It’s bad enough to subject users to your software, but it’s another to hold them criminally liable for your bugs:

Prosecutors are considering criminal charges against casino gamblers who won big on a slot machine that had been installed with faulty software.

More than the Target usability lawsuit, developers are watching this legal proceeding because, in developer logic, if one is prosecuted for using buggy software, one will not use buggy software; that is, the software developers write will suddenly be bug-free under penalty of law.

CIO magazine offers an article entitled How to Spot a Failing Project; that begs for a punchline, such as The client is still paying the bills, unlike a failed project, where the client has stopped paying the bills.

I can’t really find too much fault for the actual information in the article that identifies symptoms, including a lot of overtime/resources throw at the project, a lack of communication among project members, missed milestones, and so on. However, these are mere symptoms that let you identify when something is off the rails, but doesn’t really explain what you should do–take the legal hit of breaching the contract? Buy some of the project management tools from the people interviewed for the story?

Probably the latter.

However, most of those steps apply to most projects, including the ones that don’t fail (and by “don’t fail,” we mean “end and the customer pays anyway”). So we thought we would provide with an additional set of signs you can use to spot a failing project:

• Another project manager commits seppuku.
• QA walks around snickering or smiling.
• The coffeepot in the communal pot has begun to have a Ritalin flavor.
• The account directors show up in a neck brace from rigorous nodding in agreement with irritated clients.
• Developers are seeking work visas for Tuvalu.

As commenter The QA Slayer pointed out in a preceding post, the comment text area was susceptible to the Hamlet test. Not so much that it dumped a stack trace, mind you, but in that it allowed the user to enter unbroken strings such as Hamlet, and the cascading style sheets and basic logic of WordPress didn’t have a mechanism for handling it gracefully; instead, the unbroken string would burst through the right side of the body image like an alien from Kane’s chest:

Click for larger (more…)

This is Web 2.0 at its finest:

Oh, we have, have we?

Somehow, I suspect if you had encountered the error, I would not have encountered the error. Or at least I would hope so.

It’s Web 2.0, though, because we encountered the error together, and I am invited to freely contribute to the community by submitting a bug report.

Screenshots are the QAer’s best friend, gentle reader. Appended to a defect report or printed on a color plotter at 20″ by 30″ so you can wave them two-handedly in front of the developer, they can prove conclusively that something very wrong happened with the application. Because otherwise the developers will fix something and claim it was never broken in the first place, or the humidity within the office will change, altering the flow of electrons through the network cable, producing different results when you try to recreate it, or something. It’s always something.

So, at any rate, a good screenshot adds a lot of value to a defect report or other communication.

(more…)

Software Test & Performance magazine calls for nominations for its 2007 TESTERS CHOICE AWARDS.

[sic]

I guess it’s an improvement. The first time I went to the Simpsonize Me Web site, I got a PHP error message. As you know, this is only a single line of text on the screen.

The next time, though, I reloaded the page, I got the Web site with only a JavaScript error:

Click for full size I guess that’s progress. I’d suppress the Flash context menu, too, but then again I am a stickler.

(Link seen on Outside the Beltway.)

UPDATE: It looks as though they failed to anticipate load and have had to throw up a temporary page until such time as they can accommodate the load. Double oops.

A commenter recently asked if I had heard the excuse, “It’s impossible to write idiot proof code because idiots are so damn inventive.” Well, I don’t hear that one very often because the excuse itself implies a flaw in the code, and developers rarely admit that outside, perhaps, the circle of other developers. Because THEY ARE LIKE THE GODS!

However, developers often justify their sloth for administrative applications. “Admins will use this,” they say, explaining why they’re not bothering to build a functioning user interface or enforcing any sort of data validation. As such, the tools in question often look like something that got an incomplete grade in a community college Introduction to Programming class. Web applications are missing images, fields and controls are laid out in the order in which the developer thought of them, and you can enter bad data directly that will have an immediate adverse impact on the application the adminster is adminstrating.

(more…)

So I was looking at the pictures on the Today Show’s Web site that someone was trying to use to blackmail Miss New Guernsey or Miss New Jersey or some beauty pageant participant/winner. I was doing this for research purposes, understand. And I noticed something special about the slide show viewer. Here’s one of the photographs:

You can click it to see it full size in a new browser window, gentle reader. I’ve even circled points of interest to simplify things if you’re a developer, although I couldn’t constrain my syllable count in explanation. You see that the querystring identifies this as page 7, but the iteration at the bottom says it’s image 1 of 10.So what happens if you click the Next button?

(more…)

Ladies and gentlemen, the infamous Hamlet Test, explained for your edification and as a tool for your arsenal. Some might call it a mechanism for Boundary Analysis, but it’s more than that. It’s just plain mean.

The use case, if you need one (oh, and how you’ll need one since “rock star” developers will tell you this would never happen in real life so he can, instead of writing flawless code, can get back to YouTubing): Back when I was a technical writer, I used to write the documentation by using software. Hey, I know, that’s an odd concept; most technical writers, if they exist for an organization, will take what the developers give them and put it into a serifed font and call it a day. Not me, I actually used the software, which also explains why I had the second highest defect count in the company, above most of the full time QA people, but that’s another story.

So there I am, swiping and pasting data from the application into my text editor (UltraEdit, don’t you know?) while I’m rearranging a user’s guide weighing in at about 250 pages. I’m reorganizing procedures and how-tos, building new chapter intros, and whatnot, and I’m swiping and pasting from a massive Microsoft Word document at the same time as I’m swiping and pasting shorter strings from the application.

You can see where this is going, right? A never-in-the-real-world situation occurs. Instead of pasting a short string into an edit box, I dumped an entire chapter of the user guide into it. And it took it.

(more…)

I’ve encountered the question in interviews before, “How do you test without requirements?” The SD Times offers an article entitled “Testing Without Requirements—Impossible?”

As crazy as it sounds, there are actually teams of software testers that perform their job without the slightest clue of what the application under test is supposed to do. How is this possible?

Jumping Jack Sprat, how do you test with requirements? I’ve heard mysterious talk about these mythical beings upon some pantheon of BusinessAnalympus somewhere, where the gods of projects have come together and thought about how things should go, but I’ve never seen them myself.

(more…)

Here is the ultimatum of our camp: what can be smashed, must be smashed; whatever survives a blow has value, whatever flies to smithereens is rubbish; in any case, smash right and left, it will and can do no harm.

Dmitry I. Pisarev

Buy it:

Or you’re on QA’s list.

After I read How to Break Software (which a quick Google check indicates I have not reviewed, gentle reader, but most of you wouldn’t have read it anyway), I bought the companion volumes. This book, which I bought off of Amazon.com at its retail price, disappointed me where How to Break Software did not.

Both books run off of a quick list of fault-model testing (a term I learned from the first book). I had a ball with the first book, laughing at seeing some of my favorite dirty tricks encapsulated in someone definitive’s book. This book, however, didn’t hold the same glee for me.

The first book dealt with a broad subject and offered some very concrete things to try to attack software. This second book deals with a similarly broad subject (security testing), but is more abstract. The attacks it discusses aren’t as narrow and easy to recreate; they’re more methods and abstract ideas to try rather than concrete shortcuts to finding issues. I know, there’s something to be said for a broad, ranging methodology, but the first book wasn’t that way, and I didn’t expect this one to be that way. Additionally, the book is sized similarly to the first, which doesn’t allow it to go into a lot of detail for each of the abstract things it talks about.

Finally, I don’t know that the book focuses enough on actual security attacks; rather, it focuses on attacks that could be construed as security breaches. However, in many cases, they’re not specifically security attacks, but rather regular tests that could, if applied to applications needing security, be security attacks.

Maybe that’s all security testing is, but this book wasn’t different enough from the first book to make me wonder if it wasn’t really a sequel given a better title.

On the other hand, it does come with a CD and a tool which looks to be pretty cool, if I could get some professional time to play with it.

So buy the first book, How to Break Software, and apply its attacks to secure software. Buy this book if you’re really into it or if the company is buying it for you.

Books mentioned in this review:

(Originally posted on Musings from Brian J. Noggle)