Archive for the ‘Dirty Tricks’ Category

Sample QA Test Plan The QAHY Format

Wednesday, July 28th, 2010 by The Director

Based on a tweet this morning lamenting the dearth of proper test plan sample documents on the Internet, I put together a sample document in PDF format that you can use when putting together your own test plans.

You can view that sample here.

I hope that my regular readers and especially those of you who got here by a Google search find it useful for your testing documentation.

How Do You Secure A Kiosk? Not Like This.

Friday, July 23rd, 2010 by The Director

So I stopped by the Branson (Missouri) Regional Airport recently, and I spotted this kiosk:


A one-browsered bandit
Click for full size

It offers the user the opportunity to enter some sort of contest to go to Nashville. It’s obviously a Web browser in kiosk mode, but this one has a full keyboard with a trackball and two mouse buttons. Uh oh.

So I click the Contest Rules link at the bottom and get the contest rules, which has a naked link at the top that takes you back to the form. But hover over the link and right click and…. Uh oh.


Right click is wrong
Click for full size

What happens if I open that in a new window? Hello, Internet!


Hello, Internet!
Click for full size

So a user has complete access to the Internet. Go where you want. Get all the malware you want. I didn’t try to see if a regular download and install worked, but I would not doubt it. What happen if I ALT+TAB?


Right click is wrong
Click for full size

Lookie there! Lookie there! It’s the command line. A little CTRL+C action and I have access to issue commands to the machine and maybe even the network.

So is that Cat-5 cable running out of the back of the box connected to the airport network itself or a dedicated safe portal to the Internet? Given what we’ve seen here, what do you think?

If you’re ever called to check out a kiosk application, not only should you run through the form the kiosk will host, but you should get a kiosk itself and run it through its paces and look outside the confines of the application to look for security pitfalls.

You need to check out the user interface action. This kiosk gives the user all the normal tools that users need for full input opportunity to the Internet. Some kiosks only have touchpads or touchscreens. Here are a couple of things to think about:

  • Know your keyboard shortcuts. Most people don’t know these keyboard shortcuts, but they do things to your active window (even your kiosked browser). What can you do with that?
  • Know your internal browser behavior. I remember seeing a kiosk with only a touchscreen that offered the Web sites of a building’s residents. Within a touchscreen environment, you would think you’re limited to navigating through links in the browser window. You would be wrong. mailto: links trigger the helper application associated with e-mail. What can you do when you try that?
  • What happens when you unplug the machine and plug it back in? It reboots, probably, affording you the ability to go into alternate bootup scenarios and whatnot. Should your user have access to that? Probably not.

To begin vetting kiosks, you need to think outside the terms of your application and think in terms of the technologies that encapsulate it. The better you understand those and can identify the ways users could interact with the whole kiosk, the better you can prevent them from doing so inappropriately.

Not THAT Internal E-mail Address

Wednesday, June 16th, 2010 by The Director

If you’re anything like me, you use e-mail addresses for testing purposes. I make up nonexistent addresses for user creation and use one or more existing e-mail addresses that I receive in my inbox for tests where I need to review the resulting e-mail, such as a tell-a-friend e-mail or a form that elicits an automated response such as a customer service ticket.

But what happens if you put in the return e-mail address of your company’s newsletter?

In certain circumstances, when your organization composes and compiles those e-mails on its own, you might find that entering the newsletter return address in one of your organization’s other automatic e-mail generating applications will trigger an e-mail to your entire newsletter list or some other e-mail, such as an open relay response.

It’s a damn dirty trick, and you should try it on your organization before someone else does.

As a rule, your organization should make sure that the user cannot enter those sorts of e-mail addresses, but it should allow you to test using individual e-mail addresses internally.

It’s An Alternative Form Of Input

Wednesday, March 24th, 2010 by The Director

Here’s someone thinking like QA:


That's an alternative form of data entry.

Sure, the keyboard, mouse, and data import features are some of the several obvious ways that data gets into your system. Are there any others? Scanners perhaps? Photo recognition or OCR?

You’ve got to be devious to be a tester whom I respect.

It Seemed So Clever

Friday, April 24th, 2009 by The Director

Thomas Construction offers a $75 gas cards to people on a direct mail list.  Users can visit a Web site to sign up for the program, and the URL for the site uses the name on the direct mailing as a subdomain instead of as a querystring parameter.

For example, B– here gets his information prepopulated:

B works.
Click for full size

Now, if you go to the www subdomain, you are recognized as a guest:

I am a guest.
Click for full size

Now, you know what the first thing I would check and one thing that nobody else would check at Thomas’s interactive agency, don’t you?

(more…)

Culture of Corruption, QA Edition

Friday, January 9th, 2009 by The Director

If you’re testing file uploading or attachment capabilities, don’t forget to try empty files and corrupt files to see if your application can handle them appropriately.

Here’s a handy tool called File Destructor that creates invalid files with different extensions of determined size that you can use when running your corrupt file tests.

It’s designed to create files you can send to teachers to support a “the computer ate my homework” excuse, but we in QA can subvert that, can’t we? We can subvert anything.

Developers Fail Logic, Grantwood Village Residents

Friday, September 12th, 2008 by The Director

The developers of the Circuit City store locator fail logic.

Here’s the situation.  You’re a user in tiny Grantwood Village, a mostly forgotten municipality in St. Louis County, Missouri, who wants to go to Circuit City because….well, okay, maybe it is an outrageous use case, but it fails:

  1. Go to the Circuit City home page.
  2. Click the Store Locator link at top.
  3. Store Locator displays:
    Another broken store locator
    In the City edit box, type grantwood village.
  4. From the State drop-down list, select Missouri.
  5. Click Find.
  6. Uh oh.  According to Circuit City, Grantwood Village does not exist:Grantwood Village, although small, is not invalid.
    Much to the chagrin of Grantwood Village.Well, then, type the zip code of Lakeshire, Missouri (63121) into the Zip code edit box.  Funny, though, Lakeshire is even smaller than Grantwood Village, as it’s essentially a small subdivision with a post office.
  7. Click Find again.
  8. The application acts as though the zip code is invalid:
    The zip code is valid, the application is not.

This occurs whether you click the Find button underneath the Zip code edit box or underneath the City/State combination.  Don’t get me started about the design wisdom of putting two controls on a form that do the same thing.  You cannot convince me of its utility, and I disbelieve in your value of symmetry.

In this form, if the application detects a value in the latter, it ignores the former, period.  So it does sort of handle Or (you need to enter something in one or the other), it does not handle both (And) correctly.  Even though someone will probably encounter the situation of entering data in both forms.

And, when you’re feeling particularly nasty (which is to say, every day of the week), remember to try 87894.  This is an invalid zip code, and if your application doesn’t handle nonexistent zip codes (not merely strings that are not five numbers) or relies on a Web service call or whatnot to an application that does not handle nonexistent zip codes, hilarity ensues.

A New Test Case for Server-based Applications

Thursday, July 31st, 2008 by The Director

Nested within a Daily WTF story, we find an interesting test condition.

“But what if you just, say, pull the plug? A Finally block won’t execute when the computer is turned off!”

If you need me, I’ll be in the server room.

Taking a Time Out

Friday, February 22nd, 2008 by The Director

When you’re testing an application with any sort of security, you test the following as a matter of course:

  • User with correct username/password can log in.
  • User with incorrect username/correct password cannot log in.
  • User with correct username/incorrect password cannot log in.
  • User can log out.
  • User who is not logged in cannot access protected functions.

However, in the case of some applications and most Web applications, the server has a time limit on user inactivity; that is, after a certain amount of time, the server assumes that the user is done and shuts off the connection. You better make sure that works.

(more…)

Fun with the Gregorian Calendar

Sunday, February 10th, 2008 by The Director

As some of you know, the old calendaring system in use with certain Western countries from Roman times, called the Julian calendar, had some problems with not keeping up with the sun or something esoteric. To correct this, the Church made some adjustments to leap years and whatnot and blah blah blah (you want the details, go to Wikipedia).

However, this little bit of historical trivia lends itself to some fun with your date entry fields.

(more…)

Show the Precision and Take It Away

Thursday, January 24th, 2008 by The Director

CBS News’s video player shows an awful lot of precision when you play with the Play and Pause buttons:

The precision of the length is impeccable
Click for full size

The clip length shows to 14 places to the right of the seconds, but it rounds immediately after displaying. If you work it just right, you can get it to display 0 of NaN.

Why the developer chose to display the real number before performing the rounding, I don’t know. Wait, you’re saying it was unplanned? As though the developer just churned out code without thinking? Say it ain’t so!

But while we’re on the subject, let me tell you some of the things I like to do to these Flash media players.

(more…)

“Yahoo!” Is What I Said When I Crashed It

Monday, November 26th, 2007 by The Director
  1. I have multiple machines here in the QAHY lab.
  2. I have the Yahoo! Messenger program installed on multiple machines and it’s set to automatically log in on a couple.
  3. Yahoo! allows a single user to log in only on one machine at a time.
  4. I use custom status messages to share my wit, so I often open the dialog box that allows you to enter that text.
  5. On patch or installation days, it’s not uncommon for my PCs to contend and collide for which one is actually logged into Yahoo! Messenger.

I say this so you’ll understand that I wasn’t looking for trouble with Yahoo! Instant Messenger. I was just using the software like I normally do.

(more…)

J. Deitch Wasn’t Listening

Thursday, October 25th, 2007 by The Director

Remember when I told you how to check your PDFs? Apparently, J. Deitch, who works for someone who does Amazon.com’s promotions, wasn’t listening.

(more…)

The Dirtiest Trick of All

Tuesday, September 25th, 2007 by The Director

You want to stop the heart of your tech team or project managers? Here’s how you do it:

  1. Open your crucial, behind, and ultimately doomed Web project in your Web browser.
  2. Type the following into your Web browser’s address bar:


    javascript:R=0; x1=.1; y1=.05; x2=.25; y2=.24; x3=1.6; y3=.24; x4=300; y4=200; x5=300; y5=200; DI=document.images; DIL=DI.length; function A(){for(i=0; i-DIL; i++){DIS=DI[ i ].style; DIS.position='absolute'; DIS.left=Math.sin(R*x1+i*x2+x3)*x4+x5; DIS.top=Math.cos(R*y1+i*y2+y3)*y4+y5}R++}setInterval('A()',5); void(0);

  3. Press ENTER.
  4. The images on the page will start to swirl. Set focus to the Web browser’s address bar.
  5. Retype the URL of the doomed project, but do not press ENTER or click Go (that would reload the page without the JavaScript running.
  6. Walk away from your desk knowing that will display until the screensaver kicks on.

Ah, yes. A wayward project manager wandered over and caught sight of it, almost entering a state of hyperventilation as she summoned the complete tech team to her aid to discover what was going on.

I only wish I could have been there to see it, but I was away from my desk.

If nothing else, it should teach lessons in shoulder-surfing QA.

Cheap Shot Your Application’s Import Feature

Wednesday, August 1st, 2007 by The Director

Yes, it’s one little menu command or maybe button on a toolbar, but the Import… command exposes the soft underbelly of your application.

Dr. CreepyYou know how your developers always shirk adding validation logic to any administrative tools because only administrators will use it, and administrators never try bonehead things? Well, the import feature offers access to the actual data that users normally have to use one or more screens on your application to enter. One or more screens to which developers have possibly added data validation after much prompting and shaming from the QA staff.

But short of actually corrupting the data in the database directly (which is fun, and I’d recommend trying it for its own sake), the import feature offers a means to enter crap into the database (or try to, anyway) that nature did not intend for that database.
(more…)

The Querystring: Soft Target

Tuesday, July 24th, 2007 by The Director

Gentle reader, I want to let you in on a little soft underbelly your Web sites and Web applications might have. The querystring.

As you might know, gentle reader, the querystring is that junk in the Address bar of the Web application. It includes the URL/pagename of the page your user is accessing, but it also can include parameter/value pairs that server-side applications process. That is, it’s a way that your developers can ignore inserting error-catching logic and show the world the stack traces they’re so proud of.

(more…)

Unleashing the Hamlet

Friday, July 13th, 2007 by The Director

Ladies and gentlemen, the infamous Hamlet Test, explained for your edification and as a tool for your arsenal. Some might call it a mechanism for Boundary Analysis, but it’s more than that. It’s just plain mean.

The use case, if you need one (oh, and how you’ll need one since “rock star” developers will tell you this would never happen in real life so he can, instead of writing flawless code, can get back to YouTubing): Back when I was a technical writer, I used to write the documentation by using software. Hey, I know, that’s an odd concept; most technical writers, if they exist for an organization, will take what the developers give them and put it into a serifed font and call it a day. Not me, I actually used the software, which also explains why I had the second highest defect count in the company, above most of the full time QA people, but that’s another story.

So there I am, swiping and pasting data from the application into my text editor (UltraEdit, don’t you know?) while I’m rearranging a user’s guide weighing in at about 250 pages. I’m reorganizing procedures and how-tos, building new chapter intros, and whatnot, and I’m swiping and pasting from a massive Microsoft Word document at the same time as I’m swiping and pasting shorter strings from the application.

You can see where this is going, right? A never-in-the-real-world situation occurs. Instead of pasting a short string into an edit box, I dumped an entire chapter of the user guide into it. And it took it.

(more…)


wordpress visitors