Archive for the ‘Dirty Tricks’ Category

To Be Honest, The Pikachu Suit Is In My Standard Test Plan, Too

Thursday, November 16th, 2017 by The Director

I might have mentioned it before, but I would not mind testing Amazon’s new cashier-less stores (Amazon’s Cashierless Store Is Almost Ready for Prime Time):

Employees have tried to fool the technology. One day, three enterprising Amazonians donned bright yellow Pikachu costumes and cruised around grabbing sandwiches, drinks and snacks. The algorithms nailed it, according to a person familiar with the situation, correctly identifying the employees and charging their Amazon accounts, even though they were obscured behind yellow polyester.

I mean, the Pikachu suit is an integral part of any well-designed test plan.

But I am not running those test cases today.

In the case of Amazon Go, though, the Pikachu suits are a showy bit of theatre, but the real tests would involve more elaborate and time-tested mechanisms for criminal shrinkage like false-bottomed boxes and whatnot. Also, you’d definitely want to try out false positives, where teams would pass the items around and see who gets charged or who gets wrongly charged.

Today’s Dirty Trick: URL Truncation

Friday, July 7th, 2017 by The Director

So I’m testing a Web application that sends a lot of different notification types to the users, including emails that include links to the items the user just posted on the site or things the users can do now on the site.

So instead of just clicking the link, I’m copying the link to the clipboard, and when I paste it into the address bar of the browser, I lop the last couple of characters off.

For example, if the URL in the email is:

https://(redacted)/posts/198992

I lop a bit off so it’s:

https://(redacted)/posts/1989

That should either display a post with that ID (if one exists AND the user logged in can see it) or an error message that says the post doesn’t exist.

The site should NOT spit up a Python error or an HTTP 500 error. I argue (and at length) that it should not display a generic 404 in this case, as that will make it look like there’s something wrong with your site instead of the URL it was given.

Instead of a simple problem with an invalid ID, you might find the truncated URL bollixes up some routing information (to make a long story short: Modern URLs include in the paths, separated by slashes, identifiers that tell the Web server what part of the code should handle the request). You might even want to specifically bollix the routing information to see what happens. For example, a URL like this:

https://(redacted)/users/edit/1099991/

Chop out some of the routing information:
https://(redacted)/users/edi

Where does that go? Who knows?

In any application that sends out URLs, you really have no idea how the user will handle that URL. They might click a link, they might swipe and paste, they might get a forwarded email where the URL is wrapped on two lines but the email program only makes the first part on the first line into a link the user can click. So your application has to account for and to handle elegantly URLs that are truncated.

So let it be truncated, so let it be done.

Here, Hold My Beer Place

Monday, June 12th, 2017 by The Director

Putting placeholder text in edit boxes in addition to (or, heaven forfend, instead of) labels became all the rage sometime recently (and, by recently, I’m using the old man’s yardstick of sometime in the last decade).

Placeholder text

Which leads to a simple test often overlooked:

What happens if I type that placeholder text into the edit box?

Now, ungentle reader, what should happen is that the string you type replaces the sample text. If your developers/designers are kludging the equivalent of a placeholder attribute into the control, you might end up typing at the end of the placeholder string which is a bit inconvenient for your users, particularly those who type without clicking on the edit box first (aka your keyboard-loving users).

Now, what happens when you submit?

Well, if the placeholder string fits within the constraints of the data string you can enter in the edit box, your application should accept it.

However, I’ve found situations where the placeholder text, when typed into the edit box, trigger validation messages because the validation logic looked for the placeholder text. This is less a problem when the placeholder text is “First name” but more a problem when the placeholder is “John”.

I got the idea for this post when I typed the placeholder text for an online import edit box that accepts a URL. The sample URL apparently resolves to a real Web site, but one which returns an HTTP 599 error due to a bad certificate (which led to a defect report about an unhelpful error message for HTTP 599 errors).

But typing the placeholder text into edit boxes can prove to be a test that occasionally bears bad fruit. Like any test.

No Comment

Wednesday, January 18th, 2017 by The Director

I’ve gotten a link to the Big List of Nasty Strings several times in the last couple of days, so it must be going around the social media again. I’ve already used it for a number of years as a second set of strings to test after my first line of strings (including Hamlet) if I have time.

But you know what the BLNS lacks? Code comment markers and other code keywords. Oh, yeah. I like to use these:

<!HTMLComment>

<!

// JavaScript Comment

""" Python Comment
comment that spans multiple lines"""

''' Python Comment
comment that spans multiple lines'''

""" Python Multiline comment end

''' Python Multiline comment end

/* Comment */
*/ Comment ended

?> end PHP Script

REM batch and Oracle comment

-- SQL Line comment

GO //start SQL Script

/// C# XML Tag Comments

' Visual Basic comments

<!--- Cold Fusion Comments

<% ASP Comment <% Response.End %>

<?php php.script(start)

Try those bad Oscars out in your edit boxes. Keep in mind, they might well go into the database without a problem, but as with any other string test, half of the test (and quite often much of the fun) comes when your Web or other application is called upon to display these values again.

For example, WordPress itself cannot handle HTML comments and the end PHP script line above; when I first reviewed this post, the complete text of the post did not display and much of the blog itself did not display (as PHP after the end PHP line did not work).

Fun Test: Hot Key Race Conditions

Tuesday, August 16th, 2016 by The Director

Did you know if you select multiple items in Mozilla Thunderbird and press Delete followed quickly by enter, Thunderbird deletes the messages and then opens multiple empty message windows?

You can often find unexpected behavior when you trigger two actions at once that the user would never do, such as this particular thing I always do.

In Web testing, you can do this using the Enter key to trigger one button while clicking another or by clicking multiple buttons in quick succession.

In mobile testing, you can do this by tapping two things at once or making two gestures at once. Or by Doing something and pressing the Home button or the Power button.

In desktop application testing, this can be by clicking a button while pressing a hot key or pressing multiple hot keys at once or in rapid succession.

Regardless, the application should always pause other input while taking an action and should always check to see if it has everything it needs to act on when starting an action. In this case, it would be an active, not deleted message.

In Other Words in Other Places

Wednesday, June 25th, 2014 by The Director

Now on StickyMinds: Picture Imperfect: Methods for Testing How Your App Handles Images.

It’s a list of dirty tricks but without the snark.

You Can’t Ignore My Strings of Electric Six in Foreign Languages

Monday, January 28th, 2013 by The Director

As a public service, I hereby provide you with the first verse of Electric Six’s “Synthesizer” in a dozen non-Roman alphabets for your testing pleasure. Remember, Hebrew, Arabic, and Urdu read and represent from the right to the left which might bollix your application if it tries to handle them.

Chinese traditional
合成器
您可以撼動它周圍
你可以去向上和向下
你可以失去你
但你不能無視我的化工技術

Chinese Simplified
合成器
您可以撼动它周围
你可以去向上和向下
你可以失去你
但你不能无视我的TECHNO

Arabic
المزج
يمكنك التخلص منه في جميع أنحاء
يمكنك الذهاب صعودا وهبوطا
يمكنك أن تفقد ما وجدت
ولكن لا يمكنك تجاهل تكنو بلدي

Hebrew
סינתיסייזר
אתה יכול לנער אותו בכל רחבי
אתה יכול לעלות ולרדת
אתה יכול לאבד את מה שמצאת
אבל אתה לא יכול להתעלם טכנו

Greek
Συνθεσάιζερ
Μπορείτε να ταρακουνήσει όλο
Μπορείτε να πάτε πάνω και κάτω
Μπορείτε να χάσετε ό, τι βρήκατε
Αλλά δεν μπορείτε να αγνοήσετε techno μου

Korean
합성기
당신은 주위를 흔들 수
당신은 가서 다운 수
당신이 발견 잃을 수
하지만 당신은 내 테크노를 무시 할 수 없습니다

Russian
синтезатор
Вы можете встряхнуть все вокруг
Вы можете идти вверх и вниз
Вы можете потерять то, что вы нашли
Но вы не можете игнорировать мои техно

Urdu
مرکب ساز
آپ کے ارد گرد ہلا کر سکتے ہیں
تم جاؤ اور نیچے کر سکتے ہیں
تم ہار جو آپ محسوس کر سکتے ہیں
لیکن تم نے میری تکنیکی کو نظر انداز نہیں کر سکتے

Thai
สังเคราะห์
คุณสามารถเขย่ามันทุกรอบ
คุณสามารถไปขึ้นและลง
คุณจะสูญเสียสิ่งที่คุณพบ
แต่คุณไม่สามารถละเลยเทคโนของฉัน

Japanese
シンセサイザー
あなたはすべてのまわりでそれを振ることができます
あなたが上に移動し、ダウンすることができます
あなたが見つけたものを失う可能性があります
しかし、あなたは私のテクノを無視することはできません

Kannada
ಸಂಯೋಜಕ
ನೀವು ಎಲ್ಲಾ ಸುಮಾರು ಅಲ್ಲಾಡಿಸಿ ಮಾಡಬಹುದು
ನೀವು ಹೋಗಿ ಡೌನ್ ಮಾಡಬಹುದು
ನೀವು ಪತ್ತೆ ಕಳೆದುಕೊಳ್ಳಬಹುದು
ಆದರೆ ನೀವು ನನ್ನ ಟೆಕ್ನೊ ನಿರ್ಲಕ್ಷಿಸಿ ಸಾಧ್ಯವಿಲ್ಲ

Tellugu
సింథిసైజర్
మీరు అన్ని చుట్టూ ఇది షేక్ చేయవచ్చు
మీరు అప్ వెళ్ళి డౌన్ చేయవచ్చు
మీరు దొరకలేదు ఏమి కోల్పోతారు
కానీ మీరు నా టెక్నో విస్మరించకూడదు

Also, for your listening pleasure, all of Electric Six’s “Syntesizer”:

Popular Vacation Destinations for Testers

Tuesday, December 4th, 2012 by The Director

You know what is a damn cheap trick? Entering an invalid value in a Zip Code field, especially if your system is trying to validate it.

Here are three Zip codes that do not exist for your testing pleasure:

87258
86705
97052

You’re welcome. Your developers will definitely not thank you.

Sometimes, Fast Eyes Aren’t Enough

Wednesday, July 25th, 2012 by The Director

I know, you’re saying, “The Director, you have the fastest eyes I’ve ever seen.” Well, probably not, since you might not have ever seen my eyes. But in my years as a printer, which meant I operated a Web printing press, which does not mean I did anything with the World Wide Web (the recruiters I talked to in the early part of the century looked crestfallen at that admission) but instead meant that I was responsible for the quality of printed material moving past me at 100 feet per minute or more. So I have a great skill at seeing problems with redirect pages and loading messages that most of the time only display for a fraction of a second.

But even I, ol’ The “Quick Eyes” Director, use a dirty trick to see what the user most likely will not: I take a screenshot of the loading message to check it.

You don’t have to get too fancy with it; when you see the screen, press Print Screen on the keyboard. That captures a bitmap of your screen to the clipboard, and you can paste it into Microsoft Paint or your preferred image editing software, and you can review it at your leisure. You have to press the key immediately when you see it, so it’s a test of your reflexes as well, and it might take a couple tries to get the screenshot. It helps if you say, “Big money, big money, no Whammies, STOP!” as you try it (so you can test like the other Michael Larson, word).

Note you can do the same thing on the Macintosh using Open Apple Command+SHIFT+3 or whatnot, but that’s a lot of synchronous button mashing, so it’s easier with a Windows machine.

Or if you have screen recording software that allows you to play the screen back at slow speed, you can review these messages very easily.

The point is, you can check the spelling, layout, and behavior of messages like this:

The elusive loading message

And you don’t even have to serve several years covered in spots of Reflex Blue ink to do it.

To Coin A Phrase

Thursday, March 29th, 2012 by The Director

Maybe he didn’t coin it, but Joe Strazzere talks about how QA needs to do some Crappy Path testing.

Leap Year Reminder

Tuesday, February 7th, 2012 by The Director

I draw your attention to this post from January 2009 about another type of test case to consider during leap year.

Not only do you have to accommodate the date of February 29, 2012, but you need to also check any calculations that count the days.

News You Can Use

Wednesday, January 11th, 2012 by The Director

There is a Unicode character and an HTML character for the skull and crossbones.

Please work it into your testing accordingly.

Just When You Think You’ve Tried All The Date/Time Test Cases

Thursday, December 29th, 2011 by The Director

The real world intercedes with something that would never happen in the real world:

THERE is no today in Samoa.

The tiny nation will jump forward in time as it crossed westward over the international dateline to align itself with its main trading partners throughout the region.

At the stroke of midnight on December 29, the time in Samoa will leap forward to December 31 – New Year’s Eve. For Samoa’s 186,000 citizens, Friday, December 30, 2011, will simply cease to exist.

I wonder how many automated processes melted down. Or are still going to melt down.

Remember to test all of your future applications that allow you to select a birthdate and country or a start/end date and country that this particular rule should exist.

Oh, man oh man, I can’t wait to log my first defect and start my first fight over it.

(Courtesy Trisherino.)

Double-click That Link

Tuesday, October 4th, 2011 by The Director

A pretty stock naughty thing to do when testing a Web application is to double-click a link instead of single-clicking it.

But, Director, what sort of madman would do such a thing?

  • Someone used to the desktop paradigm might do it just because he or she does not know not to (someone like Roberta).
  • Someone like me who doesn’t see any action immediately and wonders if he clicked the link or if he clicked while the cursor was not on the link.

Case in point: In WordPress, you can move an item to the trash by clicking the link labeled, appropriately, Trash:

The mouseover indicates the link is selected....When you click....

If you click the link, the page comes back with the item missing from the list and your trash incremented by 1.

If you double-click the link, though:

When you double-click, hilerrorty ensues.

Hilerrority ensues! The application deletes it and then tries to delete it again! This results in an unspecific error condition, but what would happen in your application?

Come on, guys, the user might double-click a link, and your Web application needs to take that into account and to handle it elegantly. More elegantly than a non-specific error message with no further navigation, certainly.

There’s Plenty Of Time To Procrastinate

Wednesday, January 19th, 2011 by The Director

Add the following date to your calendars and to your test cases: January 20, 2038:

The year 2038 problem (also known as Unix Millennium Bug, Y2K38, Y2.038K or S2G by analogy to the Y2K problem) may cause some computer software to fail before, in the year 2038 or after. The problem affects all software and systems that both store system time as a signed 32-bit integer, and interpret this number as the number of seconds since 00:00:00 UTC on Thursday, 1 January 1970. The furthest time that can be represented this way is 03:14:07 UTC on Tuesday, 19 January 2038. Times beyond this moment will “wrap around” and be stored internally as a negative number, which these systems will interpret as a date in 1901 rather than 2038. This is caused by Integer overflow.

In the end, all software shortcuts will out and will crash a moonplane.

A Nihilist’s Enumeration

Tuesday, October 19th, 2010 by The Director

An old Blockbuster envelope teaches us a valuable lesson about alternative methods of output:


I am one of nothing, too, but I'm not proud of it.
Click for full size

So what portions of your application come out of the printer? Does it work right? Does it look right? Is it correct?

It’s not enough that you make sure the print dialog comes up correctly. You need to make sure that the extras that are often added to the printed page display correctly. For example, some maps add details such as the location, some Web sites put their names on it, and some applications use formula. To ill effect in this case.

If you want to be a real rapscallion, see what happens if you print to a file or to a PDF driver of some sort. Because someone out there in the real world just might.

That’s Something You Can Hang Your App On

Wednesday, September 8th, 2010 by The Director

Friends, we’ve already covered file upload test cases, haven’t we?

Well, if you’re new here, let’s recap:

  • Large files: make sure the application can handle 1Gb or more or stops user from uploading them.
  • Empty files: make sure application can handle 0 kb files.
  • Invalid files: make sure application can handle files that are corrupted.
  • Wrong file types: make sure application can handle when you try to upload the wrong type of file
  • Long file names: make sure application can handle long file/path names.
  • Invalid file/path: make sure application can handle invalid locations if you’re allowed to type in file names and paths.

The little Space Your Face Flash ditty created by NASA hangs on the 2nd and 3rd of the bullets above:


Spacing your infinite reaches of space
Click for full size

Worse, it hangs with some cheaply produced space groove playing.

When One Becomes Two

Wednesday, August 25th, 2010 by The Director

So your designers have constrained the input length on your application so you cannot enter more characters than the database can handle. If the developers force the string into all caps, have I got a nasty little trick for you. Ladies and gentlemen, the German eszett:

Also, the eszett or scharfes S (ß) is used. It exists only in a lowercase version since it can never occur at the beginning of a word (there are a few loan words starting with an s followed by a z (e.g. Szegediner Krautfleisch but that is not the same as the eszett which counts as one letter).

In all caps it is converted to SS….

There’s a new unicode symbol for the capital version, but a lot of old applications will still force that into an SS. So a word like confuße might get uppercased to CONFUSSE, and if you set the string to the maxlength, uppercasing it will blow that up.

To be honest, I did discover this when I was working on an application for a German customer and I (and only I of a team of far more seasoned QA people than I at the time) sought out the German alphabet to learn its vagaries.

I just ruined a little of my mystique, didn’t I?

However, if your application might possibly be localized to German, you have my permission to use this. Use this new power only for good. Strangely, though, QA good means evil to everyone else, but that’s not our fault.

SQL Injection Cheat Sheet

Wednesday, August 18th, 2010 by The Director

Here’s a SQL Injection Cheat Sheet for you.

Remember to check your form fields for these bad dogs when you can.

(Link courtesy the Twitterverse.)

Sample QA Test Plan The QAHY Format

Wednesday, July 28th, 2010 by The Director

Based on a tweet this morning lamenting the dearth of proper test plan sample documents on the Internet, I put together a sample document in PDF format that you can use when putting together your own test plans.

You can view that sample here.

I hope that my regular readers and especially those of you who got here by a Google search find it useful for your testing documentation.


wordpress visitors