The Limitations of BrowserShots

We’ve all been there, or maybe I’ve just been there enough for all of us.  The designers/developers talk about the QA timeline and budget, and beneath the insignificant times for manual testing, regression testing but at least contained in the test fantasy, unlike performance testing which never appears for small projects, we get to the browser compatibility portion of the program. Instead of a rigorous run through of the site or application in various Web browsers on Web platforms, you get 15 minutes to run it through BrowserShots.org.

I have a particular look for that moment in meetings.  I lower my brow, raise an eyebrow, tighten my mouth, and squint just a little.  This look indicates that I’ve reevaluated my assessment, and I’m checking the incurable wing box on the papers for your commitment.

BrowserShots.org is a handy little tool that will take a single page from your Web site, submit it to a distributed set of computers so that it can access your Web page in a varied set of browsers and platforms, and display a set of images of what that page looks like in those browsers.  You know, it’s a handy tool for a designer who wants to see how CSS and templates display across browsers, but it’s not a complete set of browser compatability testing, no matter how much your project manager and other stakeholders wish it.

Here’s a handy bulleted list of some shortcomings of BrowserShots and what it cannot do that real QA, or at least the intern you can spare on it, can handle.

  • Mouse over effects.  It just takes a screenshot, remember.  That means that any mouseover images won’t show as broken images (and brother, I see this often enough to want to mouse over every menu item on every page).  Additionally, it won’t tell you that your a:hover style is Times New Roman 34 point when viewed in Safari.  Although I’ve not seen that particular combination, I see plenty of instances where it throws off the layout.  The images of the browser don’t show you that, but mousing over a link in a Web browser will.
  • Animation.  The screenshot takes a quick shot, so it won’t show any animation you’ve got going on.  You’ll get a single bit of it.
  • Anything rendered through plugins.  You won’t get to see if that animation works or that MP3 plays.  Additionally, you won’t get to see how many plugins cause problems since you’ll just get the image.
  • Pages requiring login.  You can see how the login form looks, but it doesn’t have a facility to show you pages requiring login.
  • In-place content rotation.  That is, if the application is supposed to show multiple rotating things, such as touts or callouts, in a position on the page, you’re going to see the one that loads the first time the browser loads.
  • Polls.  You’ll see the poll question, but not how the results display after the user votes.
  • Refreshed information. If something changes after x amount of time, you only get the first screenshot, so you won’t see any changes on the page.
  • Forms.  So many gotchas in browser compatibility come from how forms behave, but you only get to look at them, not test their validation.
  • Showing layouts of hidden bits in forms.  If you’ve got a drop-down list, you won’t see how the items within the list are rendered (that is, whether the list is wide enough).  Also, you won’t see the alignment of the entry into text boxes (off center sometimes; try it).
  • Showing any of those show/hide divs. Designers are so fond of these devices now.  They like to click links and show/hide content on the page, but you will never see how that looks because BrowserShots shows only the page’s appearance on page load.
  • Whether tracking works.  You need to run through the site to make sure tracking works as expected.  BrowserShots shows the page.  Period.  Additionally, tracking links and redirects may work oddly in BrowserShots.
  • Some screenshots taken before site renders if it’s a slow site.  If your site is full of slow-loading awesomeness, some of the screenshots that display on BrowserShots.org will show the site as its rendering, not its final look.  Granted, that indicates problems anyway, but your designers will look at it and tell you that the site would have rendered correctly given enough time, even when this might not be the case.I mean, let’s look at one of our favorite sites, StlToday.com, as it appears in BrowserShots:Slow to load, so it doesn't display.
                                       Click for full size.

    Seriously, what does that show you that’s useful?
  • Page behavior with browser resizing.  Most of the screenshots come from full-size browser windows.  You have no insight into what happens when the user sees it in windowed mode.  Does the sidebar, set to an absolute right position in the CSS, overlay the content page when the window is less than 800 pixels wide?  Hey, who knows?
  • Scrolling concerns.  You can tell in the screenshots if you’d need to scroll right to see the page or down, but it’s not as obvious–or annoying–as it would be if you had to do it in an actual browser.  If you did, you’d log an issue, but BrowserShots makes this easy to overlook.

You can work around some of these by running your site through BrowserShots.org over and over again and for each individual page, but come on, eventually that will be as time consuming as just running through the site in different browsers.  Less cool, maybe, and it feels like work instead of submitting your URL and reading Slashdot until the pictures show up.

You know, I’ve got nothing against BrowserShots and as I’ve indicated, I think it’s got its place.  However, it is not a complete browser compatability regimen, and if your organization insists on using it as such, well, you’re going to have some well-deserved problems.

Comments are closed.


wordpress visitors