Archive for the ‘Tools’ Category

Two Minute QAte: Automated Link Checkers

Friday, April 22nd, 2011 by The Director

Do you think your automated link checker performs a comprehensive review of your Web site? Think again. This episode of Two Minute QAte explores some of the limitations of automated link checking tools.

HTML E-mail Comparison

Tuesday, June 30th, 2009 by The Director

Email-standards.org offers a handy comparison of how different e-mail clients handle HTML e-mails.

Still, you should check on your own, but this would be a handy thing to pass to your designers just in case they are trainable.

Culture of Corruption, QA Edition

Friday, January 9th, 2009 by The Director

If you’re testing file uploading or attachment capabilities, don’t forget to try empty files and corrupt files to see if your application can handle them appropriately.

Here’s a handy tool called File Destructor that creates invalid files with different extensions of determined size that you can use when running your corrupt file tests.

It’s designed to create files you can send to teachers to support a “the computer ate my homework” excuse, but we in QA can subvert that, can’t we? We can subvert anything.

How To Make Your Product Update Look Suspicious, Courtesy of Corel

Thursday, October 30th, 2008 by The Director

Step 1: Have an unnamed Product Update screen display by the system tray on product startup:

Something's calling for you

Given how long the newer version I have (Paint Shop Pro Photo X2) takes to load, I often click its icon and then go back about my business so it can show its splash screen for 30 seconds while I work in an active application.

Step 2: When I click for More information, show me yet another screen that doesn’t tell me any information.

Something's downloading
Click for full size

Well, what can it hurt? By this time I’ve figured it’s probably PSP since I can sometimes get to this screen and sometimes I can get to PSP. So I start the download, hoping I’m not getting Weatherbug 2009.

But here’s the thing: in the middle of the download, I decided to lament to it to you guys, and I clicked Cancel to stop it so I could get those lovely screenshots for you. The Updater dispelled the progress bar window, but it left Paint Shop Pro in a modal form so I could not actually get to it. I had to kill it from the taskbar.

Eventuallly, though, after I mucked around with those things enough, I got to the installer, and it finally, finally identified the product:

Progress or congress
Click for full size

 

Corel should have branded each and every of the preceding screens, but for some reason did not. Poor form, Peter. Now, maybe some day they can un-screw-up a great product that they had to tart up and, more inanely, change keyboard shortcuts that had been part of the product for a freakin’ decade. Yeah, when I get wed to a product, I get wed to a product and its custom shortcuts.

For Want Of A Conditional

Wednesday, October 29th, 2008 by The Director

This is the sort of defect I really hate.  Not that it’s broken, not that it’s a simple fix, but this is the sort of defect that you spend more time arguing about than it would take fixing.

The application is a little flow charting piece called RF Flow.  I recommend it for building handy flowcharts to graphically illustrate the processes that everyone in your organization ignores.  However, right after you first install or reinstall version 5 (four years old now), the recent file list includes empty numbers since you haven’t actually opened 9 files yet:

Someday, this problem will solve itself
Click for full size

It would be simple enough to not populate the menu with recent filenames where the recent filename==null.

However.

In a real development environment, the sequence of this would be:

  1. QA logs the defect.
  2. Project manager, who’s been happy path testing and whatnot, reviews defect and says he cannot recreate it.
  3. QA explains that it only happens when the recent file list is empty.
  4. Developer says it won’t be a problem once user has opened 9 files, so  it’s not worth fixing since there’s a workaround.
  5. QA says, come on, it’s a single conditional.  We could fix it ourselves but we don’t fix problems.
  6. Developer comes back from lunch at the Thai place and says, but how many people will see this problem?
  7. QA responds, “How many customers will we have?”
  8. Developer recommends that on first launch, application should open 9 sample files maybe.  He’ll wait for someone in the Training/Documentation department to create them.  As soon as the company creates a Training/Documentation department.
  9. Quibbling continues until launch date approaches.  As this small thing is not a critical defect, it does not stop the launch.
  10. Developer attends launch party; QA waits in its lair, plotting against or hexing the developer who spent several hours over several weeks dodging a far smaller amount of work.

From this crucible, Known Issues Lists emerge.

And hey, RF Flow is an easy tool for flowcharting and cheaper than Microsoft Vizio.

QA Didn’t Even Have To Physically Convince Deziner

Friday, October 24th, 2008 by The Director

Deziner Folio says, “Do some IE homework!

He then proceeds to provide a list of handy tools you developers and designers can use as you create sites that should work with the dominant Web browser, but often don’t because you’re too bound to Safari or Firefox (“It works on my machine!”)

Here at QAHY, we’ll still use our preferred tool to test Web sites for IE compatibility: freakin’ IE.

The Limitations of BrowserShots

Tuesday, August 5th, 2008 by The Director

We’ve all been there, or maybe I’ve just been there enough for all of us.  The designers/developers talk about the QA timeline and budget, and beneath the insignificant times for manual testing, regression testing but at least contained in the test fantasy, unlike performance testing which never appears for small projects, we get to the browser compatibility portion of the program. Instead of a rigorous run through of the site or application in various Web browsers on Web platforms, you get 15 minutes to run it through BrowserShots.org.

I have a particular look for that moment in meetings.  I lower my brow, raise an eyebrow, tighten my mouth, and squint just a little.  This look indicates that I’ve reevaluated my assessment, and I’m checking the incurable wing box on the papers for your commitment.

BrowserShots.org is a handy little tool that will take a single page from your Web site, submit it to a distributed set of computers so that it can access your Web page in a varied set of browsers and platforms, and display a set of images of what that page looks like in those browsers.  You know, it’s a handy tool for a designer who wants to see how CSS and templates display across browsers, but it’s not a complete set of browser compatability testing, no matter how much your project manager and other stakeholders wish it.

Here’s a handy bulleted list of some shortcomings of BrowserShots and what it cannot do that real QA, or at least the intern you can spare on it, can handle.

  • Mouse over effects.  It just takes a screenshot, remember.  That means that any mouseover images won’t show as broken images (and brother, I see this often enough to want to mouse over every menu item on every page).  Additionally, it won’t tell you that your a:hover style is Times New Roman 34 point when viewed in Safari.  Although I’ve not seen that particular combination, I see plenty of instances where it throws off the layout.  The images of the browser don’t show you that, but mousing over a link in a Web browser will.
  • Animation.  The screenshot takes a quick shot, so it won’t show any animation you’ve got going on.  You’ll get a single bit of it.
  • Anything rendered through plugins.  You won’t get to see if that animation works or that MP3 plays.  Additionally, you won’t get to see how many plugins cause problems since you’ll just get the image.
  • Pages requiring login.  You can see how the login form looks, but it doesn’t have a facility to show you pages requiring login.
  • In-place content rotation.  That is, if the application is supposed to show multiple rotating things, such as touts or callouts, in a position on the page, you’re going to see the one that loads the first time the browser loads.
  • Polls.  You’ll see the poll question, but not how the results display after the user votes.
  • Refreshed information. If something changes after x amount of time, you only get the first screenshot, so you won’t see any changes on the page.
  • Forms.  So many gotchas in browser compatibility come from how forms behave, but you only get to look at them, not test their validation.
  • Showing layouts of hidden bits in forms.  If you’ve got a drop-down list, you won’t see how the items within the list are rendered (that is, whether the list is wide enough).  Also, you won’t see the alignment of the entry into text boxes (off center sometimes; try it).
  • Showing any of those show/hide divs. Designers are so fond of these devices now.  They like to click links and show/hide content on the page, but you will never see how that looks because BrowserShots shows only the page’s appearance on page load.
  • Whether tracking works.  You need to run through the site to make sure tracking works as expected.  BrowserShots shows the page.  Period.  Additionally, tracking links and redirects may work oddly in BrowserShots.
  • Some screenshots taken before site renders if it’s a slow site.  If your site is full of slow-loading awesomeness, some of the screenshots that display on BrowserShots.org will show the site as its rendering, not its final look.  Granted, that indicates problems anyway, but your designers will look at it and tell you that the site would have rendered correctly given enough time, even when this might not be the case.I mean, let’s look at one of our favorite sites, StlToday.com, as it appears in BrowserShots:Slow to load, so it doesn't display.
                                       Click for full size.

    Seriously, what does that show you that’s useful?
  • Page behavior with browser resizing.  Most of the screenshots come from full-size browser windows.  You have no insight into what happens when the user sees it in windowed mode.  Does the sidebar, set to an absolute right position in the CSS, overlay the content page when the window is less than 800 pixels wide?  Hey, who knows?
  • Scrolling concerns.  You can tell in the screenshots if you’d need to scroll right to see the page or down, but it’s not as obvious–or annoying–as it would be if you had to do it in an actual browser.  If you did, you’d log an issue, but BrowserShots makes this easy to overlook.

You can work around some of these by running your site through BrowserShots.org over and over again and for each individual page, but come on, eventually that will be as time consuming as just running through the site in different browsers.  Less cool, maybe, and it feels like work instead of submitting your URL and reading Slashdot until the pictures show up.

You know, I’ve got nothing against BrowserShots and as I’ve indicated, I think it’s got its place.  However, it is not a complete browser compatability regimen, and if your organization insists on using it as such, well, you’re going to have some well-deserved problems.

A Spell Checker for Code

Saturday, April 26th, 2008 by The Director

If you’re not compelling your developers to spell check their code, you’re falling down on the job.

ComponentOne is offering one that works with Microsoft Visual Studio. It runs through comments, HTML, and string constants.

Quote from the story of the product announcement:

Billy Hollis, an author and Microsoft “regional director”—one of a number of volunteers recognized by Microsoft’s Developer Platform evangelism group for technical expertise—suggested that developers should use a spell checker to improve the perceived quality of their work.

“The only way users can judge quality is [by] what they see,” Hollis explained. “If they see misspelled words, many will assume they are seeing shoddy work.” He believes that many developers depend on testers to find spelling problems that appear in the user interface and submit the spelling errors as bug reports.

Yes, indeed. And if you’re expecting QA to find all the words in all of the messages, you’re expecting more out of QA than I do.

That Cannot Be Bad News

Saturday, April 26th, 2008 by The Director

Oracle is picking up Empirix’s e-Test Suite.

The product can only get better. Of course, if anyone had picked it up, it could only get better. As you might know, gentle reader, I remain singularly unimpressed with the product.

That Could Come In Handy

Friday, April 25th, 2008 by The Director

Joe Strazzere points to a tool that looks handy: BareTail.

Some of my happiest years in testing occurred as I sat in a dark computer lab, watching a bank of monitors run automated scripts while a main monitor (19″! At the time, it was worth an exclamation point!) used the command tail to display a scrolling list of the latest entry to the test logs.

Were those happy, halcyon days because I got to work in a darkened computer lab? Because I was young and still optimistic? Or because I had the wonderful tail command?

It’s hard to say, but I did move on to other things and DOS/Windows environments without access to the joy of the command line tail, and I did become a bitter, cynical, distrusting quality assurance professional you readers all know and, well, read.

However, BareTail looks to provide the same functionality as tail with some additional Windows bells and whistles. Whereas I’ve sometimes thought of writing my own utility to do this, it looks as though I’m spared that effort. I’ve downloaded my copy and cannot wait to try it out.

Cue the renascence music, and watch for a smile on my face. Professionally, I haven’t had one for years (save for dark mirth and gallows humor), but it might happen. Might.

Tips for Using Automated Link Checking Software

Tuesday, January 22nd, 2008 by The Director

As you expect, gentle reader, even when it comes to checking the links on Web sites, I prefer manual testing, particularly at the onset of a Web site development project. That is, I do want to personally, with my own index finger, click every single link on every single page, including that repeated navigational menu bar that would never, ever change across the pages (the developers and designers say) and don’t tend to change except when they catastrophically fail, for no discernable reason, on a single page.

That’s not to say that automated link checking doesn’t have its place, because it does. The remainder of this piece talks about its place.

However.

(more…)

Evidence-Based Scheduling in FogBugz

Wednesday, October 31st, 2007 by The Director

Joel Spolsky of Fog Creek Software explains some of the thinking behind Evidence-Based Scheduling included in the new release of FogBugz.

Over the last year or so at Fog Creek we’ve been developing a system that’s so easy even our grouchiest developers are willing to go along with it. And as far as we can tell, it produces extremely reliable schedules. It’s called Evidence-Based Scheduling, or EBS. You gather evidence, mostly from historical timesheet data, that you feed back into your schedules. What you get is not just one ship date: you get a confidence distribution curve, showing the probability that you will ship on any given date.

Honestly, that’s what you ought to be doing if you’re taking a scientific approach. However, your organization and its schedule builders aren’t scientific, preferring instead to build timelines and effort estimates to fit external constraints, deadlines, or budgets instead of reality.

So, carry on with those unwritten tasks of covering your rump when failures occur.

Sending An Application To Do A Man’s Work

Monday, October 15th, 2007 by The Director

Throughout much of the IT world, the developers and the people who love them want technology to solve everything for them, to be everything to them. Unfortunately, we in QA spend most of our days steeped in the myriad ways technology fails without remorse on its part and often without remorse on the part of the negligent nabobs who created it.

So you can understand why I’m not an early adopter to the latest gee-whizzery that uses faulty algorithms to supplant fallible people. So when I saw several ads for WhiteSmoke, a product that’s supposed to review and improve your written work, you might think I would be tempted to go to its Web site and review it for grammar. Brother, you know me too well.

(more…)

Microsoft Warstrike Load Testing Software

Thursday, September 6th, 2007 by The Director

Well, it’s not really Microsoft Warstrike, it’s Microsoft Web Application Stress Tool (WAST, which as you old Bard’s Tale fans know was the keystroke combination for the Warstrike conjuror’s spell, good for 4-16 points of damage on a group and a pretty potent weapon). Now, Microsoft Web Application Stress Tool lies buried in the Microsoft Downloads section (here). Screenshot:

Microsoft Web Application Stress Tester
Click for full size

It’s an old download (ca. 2002), but it looks to work with IE 7. So if you’re looking for a free rudimentary load-style tester, here’s something.

Tools of the Trade: Paint Shop Pro

Wednesday, August 8th, 2007 by The Director

The following is not a compensated post; I’m merely extolling the virtues of a piece of software I found useful.

As I’ve mentioned, falsifying taking screenshots is a good means to capture details for defect reports. Your basic Windows install comes with Microsoft Paint, which is a mechanism you can use to save and manipulate your images, but it’s very clunky, with rudimentary tools and only the ability to have one file open at once.

Some people use Microsoft Word or PowerPoint for their picture editing ability and save their screenshots as documents or slide presentations, but some of our outsourced friends might not have Microsoft Office on their workstations. Remember, you want to save those screenshots as an image format so the developers can ignore the obvious that’s presented by an image editor or a simple Web browser.

I’ve used Paint Shop Pro since version 7 (which I still have installed on my main workstation, since there’s nothing I’ve needed since 2001. Jasc and then Corel have come out with newer versions every couple of years, and they’re still priced under $70 a seat (unlike other, more expensive graphics editors). Like UltraEdit, I’ve spread it across several of my employers.

Paint Shop Pro has a pretty good set of tools for circling or highlighting issues on screenshots, for adding text for emphasis, and for altering Web 2.0 user submissions to give cute little doggies red demon eyes to match your QA soul. You can do all of these at once because you can have more than one file open at a time.

So if you haven’t considered a graphics editor, consider this one. It costs under a hundred, so you’re not exactly breaking the budget on it, either.

Fuzzyoumang

Friday, August 3rd, 2007 by The Director

The Mozilla Foundation plans to give away its own security tools, including a fuzzer:

Mozilla Corp. will release some of its homegrown security tools to the open-source community, the company’s head of security said Wednesday, starting with a “fuzzer” it uses to pin down JavaScript bugs in Firefox.

The JavaScript fuzzer, said Window Snyder, Mozilla’s security chief since last September, will be handed over tomorrow morning, following a presentation at Black Hat, the two-day security conference that opened today in Las Vegas.

“We’re announcing that we’ll be sharing our tools with the community,” said Snyder, “and releasing the JavaScript fuzzer then.” Other tools, she said, would follow, including fuzzers that stress-test the HTTP and FTP protocols. Those two, however, are not ready to offer up to outsiders, largely because Mozilla wants to wrap up talks with other browser vendors before they do.

So if you haven’t been fuzzing your applications, you’re running out of excuses.

Tools of the Trade: UltraEdit

Thursday, August 2nd, 2007 by The Director

The following is not a compensated post; I’m merely extolling the virtues of a piece of software I found useful.

I have been a fan of UltraEdit probably for about a decade; I cannot actually remember the last time I had a PC onto which I did not install this robust little text editor.

Notepad and Wordpad come free with Windows and most workstations in offices come with Microsoft Word these days, so you might not see the value in adding another program to your desktop. However, UltraEdit adds a number of features not currently available in these other applications, including:

  • Syntax highlighting, which shows keywords in a number of languages in another color to make the files more easily readable. The list of languages and the keywords themselves are extensible, so you can make your own as needed (I once made keyword files for MOL and SDF chemical data files).
  • Tabbed view, which means I can have a bunch of files or empty files open and can work on them at once, seeing their contents at a glance. Neither Notepad or Wordpad offer the multiple files thing, and Microsoft Word allows multiple files, but not a tabbed view.
  • Find or search/replace in files, which lets you dig through all the files in a folder to find what you need.

At only $50 a throw, I install it on every PC I own, as I mentioned, and I’ve also introduced it to every workplace where I’ve been in the last decade. Friends, let me tell you that this includes a rather surreal experience I had with my last Corporate employer.
(more…)


wordpress visitors