QA Hates You

I hit Dice.com to check out the local job market action, but instead of letting me go about my bidness, immediately the home page asks me to take a poll wherein I could win a Kindle. All the better for reading a $.99 copy of John Donnelly’s Gold, I think, so I click through to it.

And then I get to this particular bit of logical Möbius strip:

To clarify: The control is labeled What did you accomplish on Dice.com today? (Select all that apply)

However, not one of the checkboxes is labeled None of these.

So to continue to the next step, if you want to continue, you must lie. And remember, the entrance to this quiz is on page load of Dice.com. That is, before you have accomplished anything at all.

Me, I didn’t lie: I eventually checked Other and Specified I got a blog post out of it.

What’s the lesson here, lads and lasses? Read the labels of the controls you’re checking, and make sure they make sense and make sure any enforcement rules upon them make sense vis-à-vis that label text.

A CBS radio site throws up an error for my viewing pleasure. And check this out: It’s an old-timey syntax error:

What, not a missing resource? Not a null passed where a value is expected? This is particularly amateur work.

And it’s from a third party, again, fouling up the radio station’s Web site and doing who knows what. Brothers and sisters, I’ve had slightly more mature computers’ virtual memory swallowed up by these third party scripts running if I’ve left the Web site open overnight. Like it or not, this third party crap is a part of your Web site and its failures will reflect upon your company, not your ad server.

Ah, the old “insecure images on a secure Web page error”:

That one is classic. And ongoing on oh so many sites that try to use their insecure templates on their secured Web pages.

Story: Hold those caps: The average web page is now almost 1MB:

Mobile broadband caps might not be putting the hurt on most mobile subscribers yet, but data usage is going to keep creeping up. And that’s without people doing any more actual browsing.

The HTTP Archive charted the growth of the average web page and found that average web pages have grown from 726 KB a year ago to 965 KB now. The 33 percent jump is due in large part to more images and third-party scripts like ads and analytics. Javascript content, spurred on by the rise of HTML5, has grown over the last year by 44.7 percent, according to analysis by Royal Pingdom.

I remember when 1Mb hard drives cost $1000, you damn kids. Before you were born.

But it does clarify an interesting point: The time your Web page takes to load isn’t the only metric you should check when you’re performance testing your mobile applications or even your regular Web site if you expect mobile users will access it.

Although your leadership might say, That’s okay, we’re not designing for mobile users, if your bloated Web site is chewing up their allocated bandwidth, they’re going to find someone more streamlined.

(Link via Tweet retweeted by Fred Beringer.)

At a little after 7am this morning, I tweeted:

In about an hour, I’ll be participating in the crowd-sourced load testing of http://Packers.com http://bit.ly/ucICSO Good luck, everyone.

A couple of near-timeouts later and some really, really long load times, and I’m the proud owner of one share of Green Bay Packers stock. It’s fewer shares than I own of the startup where I used to work, but they’re worth about the same.

At any rate, two hours after I tweeted, the first news reports about the slow Web response time appeared:

On Twitter, fans were reporting that the it was difficult to access the website – www.packersowner.com – or the toll-free line: 855-8-GOPACK.

Murphy and other team officials said they anticipated there would be high demand. “Have patience,” Murphy told reporters at a news conference Tuesday morning. “Be patient.”

Who could have expected that this would have occurred? Anyone with dramatic Web launch experience.

UPDATE: More on that great tsunami here:

Sarah Johnson, 34, of Portage said it took her nearly 20 minutes to complete what should have been a 30-second process, but it was worth to wait.

…

The team received 1,600 orders in the first 11 minutes of the sale, said Packers president Mark Murphy, who had to reassure fans the Packers website was still working. Team spokesman Aaron Popkey said he did not have any sales data as of early Tuesday afternoon.

“It’s just a question of volume,” Murphy said. “Fans are excited about this opportunity. We just encourage fans to be patient.”

150 orders a minute. What, were they running it in development?

Content management systems, ya gotta love them. Unless, of course, you’re a professional quality assurance professional who likes to make sure that every i is dotted, t is crossed, and serial comma is twisted. Then CMS packages are scary. They allow just anyone to get in there and throw something up onto the Internet that the whole world can see and mock. I’ve often maintained that if you’re going to use CMS, you still need to have a two-person system at the very least. One to type it up and one to preview it.

To keep something like this from going where it will scare the innocent users:

As you can see, all the appearances of li have changed to p. Forensically speaking, we can ascertain that someone changed this from a bulleted list and used search-and-replace to do it.

Never, never, ever, do a blind search-and-replace on your text. And have someone else look at it before it’s scattered across the Internet.

In this case, it’s a JavaScript error:

In this case, the partner passing an invalid character did not cause a catastrophic failure; instead, it spit up a JavaScript error and probably screwed up some user tracking metrics somewhere.

But what happens when your trusted partners pass your applications crap? And what’s with this “trusted” business? Your organization’s partners have less QA than you do. You need to test every little thing they pass to your application or Web site and make sure they won’t blow you up. Additionally, you need to test what happens when you pass things over there to make sure your mistakes won’t blow you up. And test to see what happens if your trusted partner isn’t there when you need it.

Any shared application programming interfaces, any interfaces period, require proper and suitable testing because software can fat-finger data, too. Just more efficiently and faster than humans.

Stltoday.com, the Internet site of the St. Louis Post-Dispatch, lives up to its name this morning, as it serves up fatal errors for St. Louis Cardinals fans seeking heart:

Trying to allocate 42 bytes led to this fatal error.

Although configuration of the server and the processes in the ops department are not the purview normally of QA, as the normal harbinger of doom, you can always ask, “What happens if we run out of server memory?” and “What happens if the database logs fill up?” Load testing might find these, but if you’re testing in a staged environment that gets cleaned out after every test run, you’re going not going to find out what happens when the crud accumulates in the pipes through daily usage. But it will be your fault for not finding it in performance testing.

So ask the unthinkable questions just to get someone thinking about them.

A do-it-yourself email communication shows a few flaws:

You know what the most basic bit of quality assurance is? Having someone else look at it.

I know some smaller companies don’t have great budgets for proofreaders, testers, or professionals of any stripe to handle their client communication needs. Or maybe they have a marketing intern on staff. Whatever. Your organization should never send something out, particularly a formal communication, without someone other than the writer/designer reviewing it.

It wouldn’t be a bad idea to send yourself the email before you blast it to the subscription list, since the missing text and extraneous nonbreaking-space-missing its-semicolon might not display in the WYSAWTS (What You See Ain’t What They See) editor that assembles your text, your logo, your custom text, and your address into the final product.

Come on, have someone else look at it. It’s what the professionals do.

Twitter added the common social media widget that allows you to Tell a Friend by sending invitation emails to people you know. They put a lot of thought into the design, as you can see by the fact that the instructions are nearly invisible against the default background color:

It’s nice of Twitter to hide the spoiler and compel you to highlight the test to read it.

SPOILER ALERT! You can send to multiple email addresses by separating them with commas.

A pretty stock naughty thing to do when testing a Web application is to double-click a link instead of single-clicking it.

But, Director, what sort of madman would do such a thing?

• Someone used to the desktop paradigm might do it just because he or she does not know not to (someone like Roberta).
• Someone like me who doesn’t see any action immediately and wonders if he clicked the link or if he clicked while the cursor was not on the link.

Case in point: In WordPress, you can move an item to the trash by clicking the link labeled, appropriately, Trash:

If you click the link, the page comes back with the item missing from the list and your trash incremented by 1.

If you double-click the link, though:

Hilerrority ensues! The application deletes it and then tries to delete it again! This results in an unspecific error condition, but what would happen in your application?

Come on, guys, the user might double-click a link, and your Web application needs to take that into account and to handle it elegantly. More elegantly than a non-specific error message with no further navigation, certainly.

As I was planning to leave a clever and pointed comment, I encountered the following captcha:

You’ll notice it’s a math problem where the first number is given as a number, and the second number displays as a word. So the answer should be….?

If your clever designers and developers build forms with fields whose expected format is not clear, you should make them identify the expected format so the user doesn’t click Submit and have to wait for the application to choke on the incorrect format. This is especially true if the resulting error message does not include the correct format. The user isn’t filling in your form because he like to solve puzzles.

Some of the field elements where this can be problematic:

• Phone number: ###-###-###? (###)###-###? #########?
• Postal/ZIP code: #####? #####-####? Open string to accommodate British or Canadian codes?
• Dates: ##/##/##? ##/##/####? ####/##/##? ##-##-####?

And so on.

You can avoid these, of course, by putting in separate fields for each of the parts of the whole, whether drop-down lists or edit boxes. But if they won’t suffer you that, they should darn well make it obvious what the user should enter.

Apparently, Bit.ly has its mind stuck back in the time of the American Civil War or something:

Either that or they’re buying into the divided nation thing, where one nation is rich and the other is poor or one America is teal and the other America is mauve.

Or maybe they just forgot to include “Other/Unknown” in the graphic representation.

Sitemeter’s support page has an incident report form with only a single priority level:

Why bother including it if there’s only one choice?

They grafted a third party package onto the Web site and did not suppress the field or they have a system beyond that which is customer-facing with other priorities but did not suppress the field on the customer-facing site. Either way, you can guess what I think they should have done.

Suppress a field where the user has no choice.

It’s a test that I passed, but Lysol’s Web team failed.

Go to the Lysol home page, type 4 in 1 in the search form, click Search, and….

Oopsy daisy. Good to see they have a quality assurance team that checks things in production to ensure this does not happen.

By the way, don’t click it hoping to find a $1,000,000 coupon. It leads to a 404. You want $1,000,000, you buy a lottery ticket like the rest of us.

Yeah, I know, comma splice in the headline. Deal with it.

So I’m flipping through the Saturday Night Live Web pages on NBC.com, trying to figure out when the new season starts, but that’s apparently a closely guarded secret. Please, NBC urges me, watch some highly dated topical humor from October 2010 instead. Christine O’Donnell jokes will be eternally funny!

So I start clicking the next > the slideshow of Recap clips in the middle of the page, and it says:

err: undefined does sound a little like a British non-lexical vocable speech disfluency, kinda like the machine is saying, “Uh….oops.” However, it’s two bad things in one: an error condition and an undefined error condition.

Within a content management system (CMS) environment, which I assume NBC.com uses, they have a way for non-technical people to enter content into a slideshow presentation like this, complete with images and links and whatnot. Some Web sites are built with a base set and expanded or replaced as needed. How to test for them?

In the beginning and in smaller starting sites or with content rotators that will not change content, you can simply manually test to ensure the content and images are there and correct.

With a larger site that’s going to be out there long term with a bunch of interns mucking around in the CMS, you should try to get some automated testing running through it that clicks through the elements in an individual content rotator and looks not so much for grammar errors or incongruities between the text, image, and links (if any), but instead focuses on making sure that each click brings a complete new slide with image and not an error or missing content. Then, you should schedule that automated test to run regularly, nightly if you can make it, and send you results that will highlight when someone has bollixed it up accidentally.

And a note about this particular error: it does not show in Internet Explorer, but in Firefox, it overlays the screen as seen above. The most basic things, such as link checking, should be part of your browser compatibility tests especially if they involve anything more complicated than links using the <a href=""> HTML tag. JavaScript has its ways of making differences known amongst the varying browsers.

My first 417 Server Error, I think:

Here’s a checklist of them all. I can mark this one off.

This error occurred on a page-initiated reload, which makes it double awesome.

Take a look at this image; the original is infinitely larger than the thumbnail:

As big as an image scaled infinityx seems, any mathematician knows there are at least two numbers larger than infinity.

And when Google discovers them, it will become self-aware.

I don’t know why, but when I conduct this search on one of my machines, the result is:

I, too, am one of nothing.

Strangely enough, this works on another of my machines and yields search results. I don’t know if it’s browser incompatibility or some sort of load balancing issue, where on machine is bound to a Web server that is working and the other is bound to a Web server that is not. However, it is broken.

I’ve spoken on many occasions about the pitfalls of putting the labels within the text boxes in a data collection form. I know, the designers are all about the “real estate” as though they’re subdividing oceanfront lots in New Jersey, but they’re really doing it because that’s how the other cool designers are doing it this year.

So allow me to show you a pitfall of this as it happens in Google+.

If you want to add links to your personal profile, you look at this little bit on the right sidebar:

Ha! As if Google didn’t already know where I am on the Web, where I was on the Web, and where I was last Thursday at 10:32 am when their satellite passed over. At any rate, you click and you get this little control:

All right, I want to add a link, so I click that link and:

I get a control box with focus set to it, so the cursor replaces any text that had been in it. Like the label.

Being a seasoned computer veteran, I immediately typed my URL into that box, which failed.

When I changed focus on another try, it became clear that the form’s idea of what I was supposed to put there differed from my own:

Ah! Not the URL, but a free text label to apply to the link. That is, the clickable text to display.

But I had no idea. But real estate was saved!

As I mentioned on Twitter, I’m currently up to 3 defects I’ve found with Google+ acting only in the capacity of a user. What’s your score so far?