QA Hates You

Fozzy, “Judas”:

Now go out there and betray some friendly software for a couple bits of silver.

“Indestructible” by Disturbed.

I know, the song is almost as old as this blog is. That doesn’t make it any less relevant.

Apparently, I did something unexpected at GitHub: I logged out.

No user would ever do that!

Note the presence of the new hotness, the interrobang (‽) character, a cartoonish mash-up of the question mark and the exclamation point that is becoming popular on error pages. I just made a developer take it off one of the sites I work on because it’s currently a developer and technology field-centric thing, and to a consumer-level user, it looks like poor kerning.

Someday, it might be more mainstream, but that is not today.

Unlike, say, logging out, which should be supported without error.

When signing up for yet another Slack (and adding to the possibility that you’ll end up saying something to the wrong client or team), one sees this message regarding passwords:

However, its denial to the contrary, Slack thinks
password, 123456 or abcdef.
is a great password.

You don’t have to be a complete smart aleck to work in this industry, but you do need to be a complete smart aleck to thrive.

The Atlantic Monthly has a piece entitled “The Coming Software Apocalypse” that starts out with some examples of computer problems akin to what I post here:

There were six hours during the night of April 10, 2014, when the entire population of Washington State had no 911 service. People who called for help got a busy signal. One Seattle woman dialed 911 at least 37 times while a stranger was trying to break into her house. When he finally crawled into her living room through a window, she picked up a kitchen knife. The man fled.

The 911 outage, at the time the largest ever reported, was traced to software running on a server in Englewood, Colorado. Operated by a systems provider named Intrado, the server kept a running counter of how many calls it had routed to 911 dispatchers around the country. Intrado programmers had set a threshold for how high the counter could go. They picked a number in the millions.

Shortly before midnight on April 10, the counter exceeded that number, resulting in chaos. Because the counter was used to generating a unique identifier for each call, new calls were rejected. And because the programmers hadn’t anticipated the problem, they hadn’t created alarms to call attention to it. Nobody knew what was happening. Dispatch centers in Washington, California, Florida, the Carolinas, and Minnesota, serving 11 million Americans, struggled to make sense of reports that callers were getting busy signals. It took until morning to realize that Intrado’s software in Englewood was responsible, and that the fix was to change a single number.

Not long ago, emergency calls were handled locally. Outages were small and easily diagnosed and fixed. The rise of cellphones and the promise of new capabilities—what if you could text 911? or send videos to the dispatcher?—drove the development of a more complex system that relied on the internet. For the first time, there could be such a thing as a national 911 outage. There have now been four in as many years.

It’s been said that software is “eating the world.” More and more, critical systems that were once controlled mechanically, or by people, are coming to depend on code. This was perhaps never clearer than in the summer of 2015, when on a single day, United Airlines grounded its fleet because of a problem with its departure-management system; trading was suspended on the New York Stock Exchange after an upgrade; the front page of The Wall Street Journal’s website crashed; and Seattle’s 911 system went down again, this time because a different router failed. The simultaneous failure of so many software systems smelled at first of a coordinated cyberattack. Almost more frightening was the realization, late in the day, that it was just a coincidence.

Okay, I agree with a lot of the premise of the article. But I know that the author is not a computer expert of any stripe when we get to this passage:

Since the 1980s, the way programmers work and the tools they use have changed remarkably little.

Well, that’s a remarkably daft statement. I wrote a bit of code in the 1980s (for pay once, but I was young and I needed the money). What has changed since then?

• IDEs.
• Object-oriented programming.
• Never mind, let’s go back to functional programming again.
• Client-server architecture.
• Web-based software.
• IDEs and other scaffolding mechanism building a bunch of code you don’t understand or need automatically.
• Inserting open-source libraries and dependencies in your code for everything.
• Distributed architectures where different machines handle different bits of your code.
• Cutting and pasting from Stack Overflow.

And so on and so on.

The rest of the article seems to be a white paper for business object-based development. Which is totally a new thing that will change everything. Except that it’s not new; it’s as old at least as Versata, a company I invested in around the turn of the century and that was founded in 1989.

You know why this never takes off? Because the code making the pretty replacement for the code is code itself and an abstraction of the type that this article claims is the problem.

You know what the real problem is?

Computer programming rarely, and even rarelier now, gets to a mature and proven technology. If you’ve been in the business for any number of years, you’ve seen technology stacks come and go along with the various frameworks, architectures, programming languages, and development methodologies. Every couple of years, they rise and fall, and projects, products, and features get started, kludged on using, or completely rebuilt in the new languages and frameworks. Then, a couple years later, something else comes up and something gets started anew.

I know this reads a little bit like Old Man Yells At The Cloud, but there’s a lot of institutional knowledge lost when these ebbs and flows occur. Nobody’s gotten node.js right yet, but don’t worry, there’ll be something new in two years to take its place, and all of our defects can be washed clean and rebuilt in the new hotness.

The article compares software architecture to old timey physical engineering, but it draws the wrong lessons. Instead of trying to make programming more visual like things in the physical world are, we need to ensure that the ‘best practices’ are learned and applied as universally as possible, and to slow down so we can learn what they are and to work with them and with mature technologies to create things that work.

Instead, companies will continue to chase the newest technologies and languages and minimum viable products as fast as they can with the result that computer science is less like science and more like Dungeons and Dragons Wild Magick rules.

Snakeskin (n)
Evidence of a defect that is not the defect itself.

Have you ever seen something that you know is wrong, but you’re not sure what made it wrong so you’re a little hesitant to log it (but you log it anyway because, hey, someone else might see it, and it’s best to know something is wrong)? That’s a snakeskin.

If you’re in the southern United States, feel free to call this a cicada shell.

Currently on heavy rotation deep in my darkened QA lair is this song by Rise Against, “The Violence”:

Clearly, I’ve been clicking the replay button too much to do any real writing lately.

“It is only the (truly) virtuous man, who can love, or who can hate, others.” –Confucius

You better believe that’s going on my resume in the Awards and Achievements section: Truly virtuous, awarded by the Chinese sage. Although anyone familiar with Confucius reading my resume would instantly throw it into the discard pile, because he or she would know Confucius did not think of himself as particularly sagacious.

An Internet of Things software update renders things even more insecure by making door locks work incorrectly:

The failure occurred last Monday when LockState mistakenly sent some 6i lock models a firmware update developed for 7i locks. The update left earlier 6i models unable to be locked and no longer able to receive over-the-air updates. LockState Marketing Manager John Cargile told Ars that the failure hit about 500 locks. The company is offering affected customers one of two options: (1) return the back portion of the lock to LockState so the firmware can be updated, with a turnaround time of about five to seven days, or (2) request a replacement interior lock, with a turnaround time of about 14 to 18 days. In the meantime, customers can use a physical key to unlock doors. (Like most hotel rooms, the doors automatically lock each time they’re closed.)

I haven’t really tested any IoT things yet, and sometimes I wonder if anyone has.

When preparing to check my brand engagement this morning (that is, check my Twitter feed, but to be a consultant, you have to have a brand and you have to continually propose to companies, as I understand it), I encountered an unfortunate problem:

A Spidey protocol error.

I blame Venom.

A control that says (Internal Use):

What do you suppose that means? I, a member of the concert-ticket-buying public, should not choose it?

Yer dang right I wrote in that office use only spot.

Fortunately for me, the tickets did arrive in the mail.

Mind your application’s labels, brothers and sisters, and ensure they contain relevant, helpful information for the user.

As some of you might know, I’m particularly lazy. When my hands are on the keyboard, I don’t like to reach all the way over to that mouse to do something, so I pay especial attention to hot keys and tab stops. And apparently, I am one of the few people to do so.

If I have a couple minutes and want to vex the developers, I start checking the tab stops in the applications I test.

Here’s what I like to check:

• Navigational elements (links and headings that expand divs) have tab stops as do the form controls (buttons, check boxes, radio buttons, text areas, and the like.
• Make sure when navigational elements and controls appear, such as if the user expands a form or chooses an option that displays dependent controls, that the new items have tab stops.
• The tabs stop order makes sense. Sometimes IDEs assign tab stops as the controls are added to a designer. Sometimes, developers add controls after the initial form design that don’t get tab stops or get tab stops at the end of the tab order instead of in the place where the form displays on the page.
• Ensure hidden controls do not appear in the tab order. Just this morning, I found a tab stop for a hidden button and, using the space bar to click the button, I triggered an inappropriate action.
• Controls have visible focus indicators. Although I’m savvy enough to look to the status bar to see if focus is set to a link, your users are not super users, so make sure your application gives them a hand by showing them where the focus is. And note the effects of the focus indicator: make sure it’s not resizing the control with focus and bumping everything else.
• Tab order is reversed if you go backwards. SHIFT+TAB should take you in reverse order. Make sure it does.
• Tabbing doesn’t get trapped in third-party controls. Things like calendar date pickers and stuff might come from outside your organization. Make sure when your developers use them that the keyboard user does not get stuck in the controls with no way out.

Remember the tab stop and keep it wholly for my benefit.

After century of dispute, the German alphabet just got a new character:

Have you ever been typing in German in a blaze of BLOCK CAPITAL anger, but been stopped short by the inability to write the next letter of the word SCHEI…? Help is finally at hand.

At the end of June, the German Spelling Council decided to add a capital ß (Eszett) to the language, bringing to an end a debate that had raged on in the world of German orthography since the 19th century.

Now, instead of using SS to capitalize the Eszett, Germans should use ẞ.

Oh, boy, what will this do to your legacy data?

On the other hand, it will render this old test obsolete.

(Link via.)

Say, doesn’t the icon for Private/Incognito browsing sort of look like a bra?

As I once said:

Yeah, QA, in the meetings and in the defects, you have to go there. Remember, the Internet is place full of miscreants, miscontents, and people who will, in fact, go there.

Hopefully, your co-workers will recognize that you’re just being professional. So just be professional about it, but do bring these sorts of things up.

WELCOME TO OUR STARTUP WHERE EVERYONE IS 23 YEARS OLD BECAUSE WE BELIEVE OLD PEOPLE ARE VISUALLY DISPLEASING AND OUT OF IDEAS:

As you can probably tell by looking around, every employee at our startup is 23 years old. On the morning of your 24th birthday, the barcode on your employee ID stops working and you can no longer enter our building. We do this to ensure our company has a ceaseless, youthful energy. We believe old people are displeasing to look at and also, bad at ideas.

The startups I’ve worked at and with have been started by people over 30.

But I’ve talked with a number of places where I wondered if my, erm, years of experience might not have been dissuasive.

So I’m testing a Web application that sends a lot of different notification types to the users, including emails that include links to the items the user just posted on the site or things the users can do now on the site.

So instead of just clicking the link, I’m copying the link to the clipboard, and when I paste it into the address bar of the browser, I lop the last couple of characters off.

For example, if the URL in the email is:

https://(redacted)/posts/198992

I lop a bit off so it’s:

https://(redacted)/posts/1989

That should either display a post with that ID (if one exists AND the user logged in can see it) or an error message that says the post doesn’t exist.

The site should NOT spit up a Python error or an HTTP 500 error. I argue (and at length) that it should not display a generic 404 in this case, as that will make it look like there’s something wrong with your site instead of the URL it was given.

Instead of a simple problem with an invalid ID, you might find the truncated URL bollixes up some routing information (to make a long story short: Modern URLs include in the paths, separated by slashes, identifiers that tell the Web server what part of the code should handle the request). You might even want to specifically bollix the routing information to see what happens. For example, a URL like this:

https://(redacted)/users/edit/1099991/

Chop out some of the routing information:
https://(redacted)/users/edi

Where does that go? Who knows?

In any application that sends out URLs, you really have no idea how the user will handle that URL. They might click a link, they might swipe and paste, they might get a forwarded email where the URL is wrapped on two lines but the email program only makes the first part on the first line into a link the user can click. So your application has to account for and to handle elegantly URLs that are truncated.

So let it be truncated, so let it be done.

Data glitch sets tech company stock prices at $123.47

A stock market data error this evening set an undetermined number of companies listed on the Nasdaq exchange to a share price of $123.47, sending some tech companies’ stock prices crashing and others’ soaring. In a statement obtained by the Financial Times, Nasdaq said the culprit was “improper use of test data” that was picked up by third party financial data providers. The exchange said it was “working with third party vendors to resolve this matter.”

I hope none of you gentle readers turned in resignation letters based on sudden ephemeral wealth.

And I hope you work with your devops guys to help make sure they scrub test data appropriately before promoting to production. Although they might rankle at it, your scrutinizing gaze upon their procedures and processes can sometimes help to find problems or to spot places to improve. Bloody heck, in the olden days, testers worked on Extract/Transform/Load, data warehouse, and conversions between expensive software packages. Just because your company does the same thing every week or every night doesn’t mean QA involvement should be less.

Has this been helpful?

Cue the Meghan Trainor, again.

I wonder if I could do nothing but posts about CAPTCHAs and what they can teach us; after all, this is my second one recently (see also.)

But here’s another one.

You see, it says Select all squares with street signs, but there are no street signs in the image.

Which made me think of all the forms that ask us to put something into edit boxes other than what the labels describe.

Do your labels all give proper patterns for data entry? Ask for the right thing? Are your end users doing strange workarounds and using data elements to contain different things than expected?

Is your application or your customer support team telling the user to lie to it to make the application work right?

That’s a problem, you know.

Of course you know. But make sure everyone else knows, too.

Putting placeholder text in edit boxes in addition to (or, heaven forfend, instead of) labels became all the rage sometime recently (and, by recently, I’m using the old man’s yardstick of sometime in the last decade).

Which leads to a simple test often overlooked:

What happens if I type that placeholder text into the edit box?

Now, ungentle reader, what should happen is that the string you type replaces the sample text. If your developers/designers are kludging the equivalent of a placeholder attribute into the control, you might end up typing at the end of the placeholder string which is a bit inconvenient for your users, particularly those who type without clicking on the edit box first (aka your keyboard-loving users).

Now, what happens when you submit?

Well, if the placeholder string fits within the constraints of the data string you can enter in the edit box, your application should accept it.

However, I’ve found situations where the placeholder text, when typed into the edit box, trigger validation messages because the validation logic looked for the placeholder text. This is less a problem when the placeholder text is “First name” but more a problem when the placeholder is “John”.

I got the idea for this post when I typed the placeholder text for an online import edit box that accepts a URL. The sample URL apparently resolves to a real Web site, but one which returns an HTTP 599 error due to a bad certificate (which led to a defect report about an unhelpful error message for HTTP 599 errors).

But typing the placeholder text into edit boxes can prove to be a test that occasionally bears bad fruit. Like any test.