Archive for August, 2012

Working Your Obliques

Thursday, August 30th, 2012 by The Director

Luxury Cars Struggle In New Crash Tests:

Last year the National Highway Traffic Safety Administration (NHTSA) instituted more rigorous crash-testing procedures that made perfect five-star ratings tougher to come by. This year the Insurance Institute for Highway Safety (IIHS) is doing the same, and the first round of results finds many popular luxury cars to be lacking. Only three out of the 11 model-year 2012 cars tested under the Institute’s new so-called “small overlap frontal crash test” earned what would be considered passing ratings.

. . . .

In the IIHS’s new frontal test, cars are smashed at 40 mph with only 25 percent of a car’s front end on the driver side striking a five-foot tall rigid barrier. The Institute says the test is designed to replicate what happens when the front corner of a car strikes another vehicle or an object like a tree or utility pole in more of glancing blow, rather than a full-on or offset frontal collision.

This is another industry’s example of happy path testing. The full frontal crash test has thus far yielded results that made car makers design their cars to meet that test, head-on if you will, and to account for what happens in that test. That sounds an awful lot like what happens in happy path testing or the bare minimum “testing” that occurs in demos, internal and external, where someone shows someone else exactly how your developers have expected the user will use the software. Maybe there’s even some validation testing to make sure that, if a collision occurs, it’s handled.

But the real damage, and the real value of QA, is in the oblique testing, where QA doesn’t do what the user is supposed to do, but also does something that the developer doesn’t expect a user to do even when the user isn’t doing what he’s supposed to do.

The trick, of course, is to think that way and to see the possibilities of just tangentally using the software incorrectly or working just a little bit out of order to elicit a response that you can film into a compelling PDA for the benefits of not commuting or you can enter into the defect tracker.

It’s also a keen insight into why QA is never happy nor confidant in a product; sure, it might have passed all the tests (Ha! Just kidding! It just passed enough tests to satisfy the low standards of the project managers), but QA knows it did not think of all the tests, and out there, some user is going to find the right combination of events and the right angle and speed to hit the application to build his own catastrophe. If only I could think of that use case.

Remote Workers Not As Distant

Wednesday, August 29th, 2012 by The Director

Report: Remote Workers More Engaged, Committed:

If your employees come into the office each day, it’s natural to think that they’re engaged and well-connected with one another. But that’s a misperception, according to a recent blog post from the Harvard Business Review.

Scott Edinger, founder of Tampa-based Edinger Consulting Group, wrote that the physical proximity of an office gives the illusion that co-workers are communicative and working together efficiently. The opposite is true, however. Remote workers are actually more engaged and committed to their team, Edinger wrote.

One reason for this, Edinger pointed out, could be that members of virtual teams feel the physical distance between them makes interactions more valuable.

When people do not sit at adjacent desks, they try hard to connect with one another and maximize what little time they do spend speaking with one another.

He wrote: “What’s more, because they have to make an effort to make contact, these leaders can be much more concentrated in their attention to each person and tend to be more conscious of the way they express their authority.”

Working remotely does focus one’s attention and effort to the task at hand and to make sure that when you’re “on the clock,” you’re actually putting forth effort toward the company goals. You’re not as focused on making it until the end of the day as you are getting the tasks done.

I worked at a posting where we went from a remote office setup to an office space, and a lot of the communication remained through IM or email for tracking purposes or because we were too lazy to get up from our desks. As far as communication goes, the only real benefit of being in the office is being able to trap the developer who’s been avoiding you in a cubicle and make sure that he is, in fact, going to hear you out (as will the whole office by that time).

There are distinct benefits to allowing remote workers, especially in IT. I think it will continue, and this article helps bolster the arguments for it.

Only the Elder Sign Offers More Protection

Tuesday, August 28th, 2012 by The Director

This captcha will protect against bots particularly well.

Now with foreign characters not on your keyboard!

Also, users who don’t have an e with an acute accent on their keyboards.

I guess it’s to test how bad you really, really want to create a new account.

Hey, let me digress here: Why doesn’t the Elder Sign actually have a Unicode symbol or character for it?

The Elder Sign

Aren’t geeks in charge of the official Unicode spec, or what?

The Safest Development Job in the World

Monday, August 27th, 2012 by The Director

I don’t normally do job postings here at QAHY, but my wife is looking for a new flunky, so here are the details.

Senior Programmer Analyst

Exits, Inc. is seeking a software developer/architect/requirements analyst to assist with ongoing and expanding business needs with its export compliance software and services sold to major and small customers alike. The work is varied – meetings, coding, design, requirements, and legacy application translation to a new system. Early work will be code centric.

Five to 10 years’ experience coding on systems projects in a combination of .NET and ASP Classic. Experience with large, complex web applications. Business rule/requirements definition experience.

Technical Skillset
C# (preferably 4.0), .NET services, VBScript, XML, IIS, SQL Server, HTML, MVC design pattern, JavaScript, cookies, I/O, various data transport protocols. XML Schema.

Preferred Qualifications
Experience defining business rules with customers and working as a vendor with large project teams. Experience translating customer requirements into system design and identifying and communicating requirements contradictions to the team. Data integration experience. Sharepoint knowledge and/or experience. Regular expressions experience. AS2/sFTP protocol experience. ASP Classic experience. Extensive business writing experience.

This position is a six-month contract for a telecommuting worker. A long-term to open-ended extension is likely. Some travel may be necessary, so applicants should have easy access to an airport. Please reply with preferred hourly rate, contact information, and a list of qualifications addressing these requirements to: Applicants must be eligible to work in the United States of America.

Why is it the safest developer job in the world? Because I’m not allowed to test my wife’s code any more.

Hey, wait a minute. If I write code for your wife, that won’t be your wife’s code. Could you test my code? You’re right. IT’S A TRAP! But one where you can work from home and make some money for that abuse.

Book Report: Winning through Intimidation by Robert J. Ringer (1974)

Monday, August 27th, 2012 by The Director

Book coverI’ll admit, I picked up this book because it had the word intimidation right in the title. Hey, who in QA wouldn’t want to be more intimidating? And maybe to win for once?

The book was a best seller for the self-published author in the 1970s and has been re-released this century with a different title (To Be or Not To Be Intimidated). So somebody found the lessons within it to be worthwhile. A lot of somebodies.

The book explains the author’s approach to real estate sales and a bit of philosophy extrapolated from the behaviors that made him a successful real estate salesman dealing in the sale of large apartment complexes across the country. The author invokes Ayn Rand, author of Atlas Shrugged and The Fountainhead, in the first sentence, saying that he titled the book using the off-putting word Intimidation because its meaning was slanted to the perjorative, much as Rand did when she entitled a book (based on her essay entitled) “The Virtue of Selfishness”. So you get an idea from an outset what sort of take the author is going to have on life.

As it stands, Ringer’s freewheeling view of business is pretty cut-throat, zero-sum. But by intimidation, he means, ultimately, setting yourself up in a strong posture to deal with adversity. So you don’t have to think you’re going to be someone who comes out of the book with the tools to become some sort of cut-throat jerk looking to take advantage of easy marks to get something out of this book. If you’re a contractor out there on your own hustling for deals and contracts, you’ll get something out of how he implements his philosophy.

One of his basic theories of success is called Theory of Sustenance of a Positive Attitude through the Assumption of a Negative Result. That is, to keep your positive attitude in a position where failure abounds (in his case property sales), you need to recognize and even plan that most of your efforts will fail. That way, you’re not surprised nor discouraged when they do. It’s an interesting twist on giving yourself a mindset for startups or whatnot; when you read about failures of successful people, they always brush them off. Perhaps it’s a native mindset in those particular people, or perhaps it’s a mindset groomed through advice much like Ringer’s here.

Additionally, Ringer presents three bases for the successful execution of intimidation: A good, professional image; a good legal framework for the work done; and successful execution of the work. Although we’re not dealing with signing up to represent properties for sale, you could apply the tenets to looking for, securing, and finishing consulting work in any stripe, including IT contracting.

One important lesson, also encapsulated in the book, is that the goal of any contract or project is not the successful completion of the project (in Ringer’s world, the closing of a sale). The goal of the project is getting paid for doing the work. It’s a lesson he reiterates throughout the book, and it’s a lesson that many people in the IT world should remember. In a world where a lot of companies start up with the goal of getting users and eyeballs (still) without actual thought to “monetization,” this lesson bears repeating and drumming.

I enjoyed the book, and it gave me a different perspective that I can apply to the industry from the experience of another. It’s key, though, to remember that the business world is not quite as cynical nor zero sum as the author here presents it, where other business people are out to take your chips and cut your hands off at the wrists, but it’s probably not a bad idea to plan as though they are. Because some are.

Books mentioned in this review:


Tuesday, August 21st, 2012 by The Director

Fabusend is an email provider handles not large, mass emails, but small, individual emails. It (apparently) adds in letterhead images and whatnot for senders and allows the recipient to click through if the images are blocked in the client email program.

But if you click through a little late, you get all kinds of FUBAR.

There’s the JavaScript error:

Fabusend JavaScript error

If you’re looking at it in Firefox, you’ll see the tables are misaligned:

Fabusend JavaScript error

Now, you’re saying, Who’s going to see that?

Someone who stores important emails and then tries to click through them. Someone like me. And not “like me” in the QA sense, “like me” in the user sense.

It’s such a little thing!

You know, you’re right, it is going to be seen by only a few people. But it’s also not something that requires refactoring the whole application, ainna? Take a couple minutes, fix the little error page up, would you?

Your goal should be a quality application across the board, not just quality for most of the people, most of the time.

It Pays To Build Your QA Vocabulary Power

Monday, August 20th, 2012 by The Director

Use this one in conversation:

glich – (n.) A software bug thought to have been killed, but due to some eldritch dark developer magic or incompetence, rises again more powerful and destructive in an attempt to achieve some form of immortality.

Allowing a space in the password was a glich that caused many users to fail at resetting their passwords after the security breach.

The Intersection of Boobs and Boobs

Monday, August 13th, 2012 by The Director

What happens if you don’t check your Web site’s alt text and filenames when you create or upload content to a CMS? Bad, bad things (link safe for work).

Ten women in the St. Louis area have sued their doctor after learning that before-and-after pictures of their breast augmentation surgeries could be found online through a simple search of their names.

. . . .

The photos are widely used as a marketing tool to promote doctors’ work. They are not publicly labeled with names. But if patients’ names aren’t removed from the computerized picture file information, they can be displayed with the images during an Internet search.

You think you might be exempt if you’re just a CMS provider and a Web host? Think again.

In court filings, Koo’s lawyers blame MedNet, saying it failed to maintain Koo’s site in a “competent and professional manner.”

“Whatever Dr. Koo did do or didn’t do,” said one of her lawyers, Jonathan Ries, she had “no intention” of linking pictures and names.

MedNet’s response blames Koo, saying the company did not post, control or influence the content. It also claims legal immunity under the Communications Decency Act, which protects websites from suits over postings by third parties.

Already, you can puzzle out how this happened, or how this could have happened. A civilian, and by that I mean a semi-ignorant user, just uploaded image files from the hard drive without thinking about the filenames, and away you go. The CMS picks up the image name to use as the alt text, and suddenly you’re showing the world JaneDoubleDoeAfter to the world.

What can you do, QA? Remember to check the alt/title text carefully whenever you can, and pay attention to file names and URLs.

Because that stuff doesn’t matter until it does, really expensive-like and attorney-chocked.

UPDATE: And upon further reflection, you know what holds true of the people who made these costly errors? They were all probably badmins.

The Civilians Are Noticing

Friday, August 10th, 2012 by The Director

The Atlantic is noticing what we QAssandras have been prophesying forever: Software Runs the World: How Scared Should We Be That So Much of It Is So Bad?

What do most people think of when they think of software? A decade ago, probably Microsoft Word and Excel. Today, it’s more likely to be Gmail, Twitter, or Angry Birds. But the software that does the heavy lifting for the global economy isn’t the apps on your smartphone. It’s the huge, creaky applications that run Walmart’s supply chain or United’s reservation system or a Toyota production line.

And perhaps the most mission-critical of all mission-critical applications are the ones that underpin the securities markets where a large share of the world’s wealth is locked up. Those systems have been in the news a lot recently, and not for good reasons. In March, BATS, an electronic exchange, pulled its IPO because of problems with its own trading systems. During the Facebook IPO in May, NASDAQ was unable to confirm orders for hours. The giant Swiss bank UBS lost more than $350 million that day when its systems kept re-sending buy orders, eventually adding up to 40 million shares that it would later sell at a loss. Then last week Knight Capital — which handled 11 percent of all U. S. stock trading this year — lost $440 million when its systems accidentally bought too much stock that it had to unload at a loss.* (Earlier this year, a bad risk management model was also fingered in JP Morgan’s $N billion trading loss, where N = an ever-escalating digit.)

The underlying problem here is that most software is not very good. Writing good software is hard. There are thousands of opportunities to make mistakes. More importantly, it’s difficult if not impossible to anticipate all the situations that a software program will be faced with, especially when–as was the case for both UBS and Knight–it is interacting with other software programs that are not under your control. It’s difficult to test software properly if you don’t know all the use cases that it’s going to have to support.


Greatest Hits List

Tuesday, August 7th, 2012 by The Director

Wikipedia has a rather incomplete list of software bugs that’s amusing. It’s not comprehensive, and it’s not very educational, but it’s a good reminder of what can happen if you aren’t paying attention.

Mind Traps That Aren’t Deadly, But Are Disruptive

Monday, August 6th, 2012 by The Director

Psychology Today has an interesting article called “Deadly Mind Traps: Simple cognitive errors can have disastrous consequences—unless you know how to watch out for them” that describes a number of, well, cognitive errors that have caused people to make mistakes that lead to their doom.

Although we’re not really working with literal doom on a daily basis in software development (although, increasingly, the errors we introduce, uncover, and/or do not discover in software do lead to literal doom), the same mental errors can lead to problems within software development projects and within the quality assurance within them.

So let’s look at the traps listed in Psychology Today:


Hall [a mountaineer who summitted Mount Everest but died on the descent] fell victim to a simple but insidious cognitive error common to many types of high-pressure undertakings. I call it “redlining.” Anytime we plan a mission that requires us to set a safety parameter, there’s a risk that in the heat of the moment we’ll be tempted to overstep it. Divers see an interesting wreck or coral formation just beyond the maximum limit of their dive tables. Airplane pilots descend through clouds to their minimum safe altitude, fail to see the runway, and decide to go just a little bit lower.

You see a lot of this sort of behavior on timelines, where close to milestones or the end of cycles, someone introduces changes or whatnot. Sometimes you see this when someone wants to use some unproven, misunderstood piece of technology to solve a problem, but the learning curve proves too steep to get things done on time, or someone has to kludge a solution to meet the need at the last minute.

You know how to fix this? Understand and adhere to limits.

The Domino Effect

Similar tragedies play out time and again when people try to rescue companions. A teen jumps from a dangerous waterfall and disappears; his buddies follow, one after the other, until they all drown. A firefighter goes into a burning building to rescue a comrade; another goes in after him, then another.

In each case, the domino effect results from a deep-seated emotion: the need to help others. Altruism offers an evolutionary advantage but can compel us to throw our lives away for little purpose. “In stressful situations, you see a failure in the working memory, which is involved in inhibiting impulses,” says Sian Beilock, a psychology professor at the University of Chicago. “People lose the ability to think about the long-term consequences of their actions.”

You see this in software development in many cases: When something’s going badly, the management throws more people at it, and they do something badly, only less efficiently (see also Parkinson’s Law). Or a company chases features that other companies put into their software or chase some small bit that competitors or the industry press follows.

You see this, too, when too much effort is put into one channel of testing, where one expends a lot of effort in automated testing and continues pursuing that goal to the exclusion of other, more relevant exploratory testing or whatnot.

To keep from being another domino to fall, the article warns you not to leap instinctively into the manure machinery (truly, that’s what it suggests), but that you take a step back and consider alternative actions if you see cascading failure before you. The saying attributed to Einstein, that insanity is doing the same thing over and over again and expecting different results, applies. So does your mother’s “If everyone else jumped off the bridge, would you jump off the bridge, too?”

When you see something fail, don’t try it again. Try it again, differently.

Situational Blindness

As GPS units and satellite navigation apps have flourished over the past few years, there’s been a spate of similar cases, in which travelers follow their devices blindly and wind up getting badly lost. In each case, the underlying mistake is not merely technological but perceptual: the failure to remain aware of one’s environment, what aviation psychologists call situational awareness, or SA. People have always had difficulties maintaining SA, psychologists say, but the proliferation of electronics, and our blind faith that it will keep us safe, has led to an epidemic of absentmindedness.

“A big element in SA is paying attention to cues,” says Jason Kring, president of The Society for Human Performance in Extreme Environments. “If you’re focusing just on that GPS unit, and you see that little icon moving down the road, and say to yourself, OK, I know where I am, technically, that can be a big problem, because you’re not looking at the world passing by your windshield.”

Full situational awareness requires incorporating outside information into a model of your environment, and using that model to predict how the situation might change.

You can lose situational awareness in many ways in software development and testing. You can narrow your focus only to the problems in your niche of the process, your set of features, or what have you. You focus on the trees, and you can’t see the Christmas tree lot.

So to combat this, you need to pay attention to higher level considerations. You need to know as much as you can about the industry, the customers, and your organization as you can, and to factor as much as you can into your decisions as to how to proceed. If you know what your customers, that is, your users, really need and really do over the course of the day, you can weight your testing to cover those needs more than the features your business analyst forced onto the software because your customer’s VP, hired from outside the industry, wanted them.

Double or nothing

The runaway-balloon problem is a manifestation of our irrational assessment of risks and rewards. As Daniel Kahneman and Amos Tversky first pointed out back in 1979, we tend to avoid risk when contemplating potential gains but seek risk to avoid losses. For instance, if you offer people a choice between a certain loss of $1,000 and a 50-50 chance of losing $2,500, the majority will opt for the riskier option, to avoid a definite financial hit. From the perspective of someone dangling 20 feet in the air, the gamble that they might be able to ride the gondola safely back down to the ground seems preferable to a guaranteed pair of broken legs. But in the moment, they can’t factor in the price they’ll pay if they lose.

We see this in software development when someone takes a flier because he or she thinks he or she has nothing to lose. A developer doesn’t like the way the code works or encounters some difficulty with it, so he spends all weekend rewriting several weeks worth of several developers’ work before the Monday client meeting instead of just putting a little rouge on the hog and acting from the insight gleaned at the client meeting. Or a company rushes headlong into a move to a new technology or new cloud strategy against the reluctance of its clients. And so on.

To combat this risky frame of mind, you’ve got to, again, establish limits and procedures to make sure that people know when they can take risks on their own–and when they can’t.

Bending the map

Such errors of overconfidence are due to a phenomenon psychologists call confirmation bias. “When trying to solve a problem or troubleshoot a problem, we get fixated on a specific option or hypothesis,” explains Kring, “and ignore contradictory evidence and other information that could help us make a better decision.”

Confirmation bias is very common in our field. Technological fanboys think that their favored language or framework is the best, and that all evidence points in that direction. The whole sales, marketing, and company magazine teams live just to provide this confirmation bias about how well your organization is doing and how everything is going swimmingly.

To combat confirmation bias, you’ve got to learn to distrust yourself. You’re not that smart. The way you think something is going isn’t the way it is going. Well, maybe it is. You’re not even lucky enough to get it wrong 100% of the time. When you encounter a new piece of evidence or information, don’t just try to fit it into your gestalt or dismiss it. You’d better figure out why that little strange thing happened, and what it can mean for your whole endeavor.

(Link seen in Reader’s Digest, where the things are given in a different order, perhaps to better track Internet content thieves. And, yes, I do read Reader’s Digest. I am an old man.)

Steps to Recreate an Amazon Bug

Friday, August 3rd, 2012 by The Director

QA begins its Christmas shopping early because QA expects something to go wrong. And QA is not only rewarded in this instance when something goes wrong, but also with an Amazon bug.

When a user orders an item with a supplementary item and then orders the item without the supplementary item, the user receives an email that says user added supplementary item to the cart but did not purchase.

To recreate:

  1. Search for and find an expensive, out-of-left field gift for someone who might or might not read the defect tracker.
  2. Add the gift to the shopping cart.
  3. Click one of the “People who bought this also bought that” links.
  4. The new item’s page displays. Click to add it to the shopping cart.
  5. Click Check out.
  6. Complete the checkout page.
  7. Thank you page displays. Confirmation email arrives. When the main product arrives, find a manufacturing defect immediately upon opening the box. Return the item for a refund, not an exchange, because this item is only ordered through Amazon and is fulfilled by someone else.
  8. When refund is credited, return to My Account section of Amazon and find recent orders.
  9. Click through the refunded item to view the item page again.
  10. Add item to cart.
  11. Proceed to checkout.
  12. Complete checkout.
  13. Thank you page displays. Confirmation email displays. However, over night, an email is also generated and sent overnight that says We noticed that you added one or more items to your Shopping Cart, but didn't continue to Checkout. When you're ready to buy, simply visit your Shopping Cart to complete your order.
  14. Go to Amazon account/cart. Note it is empty.

Apparently, because I clicked through the recent orders, Amazon still links the two items from the order. Should not generate this email when the same item is ordered by itself in a new order.

Sorry I couldn’t be more specific. On the other hand, you might surmise that I’m getting you more than one thing for Christmas, gentle reader, and I only wish I could be there with you on Christmas morning to share your disappointment.

As Good A QA Test Plan Template As Any

Thursday, August 2nd, 2012 by The Director

Robert Rogers’ 28 Rules of Ranging.

Surprisingly, there is a lot of overlap in what QA does.

wordpress visitors