Well, you’ve found an issue, ungentle tester; now what? Well, we’ll assume you have some sort of mechanism in place to track that issue, but the software mechanism (usually called a defect tracker, but sometimes known as the developer’s junk e-mail box) only serves to provide a technological means to support a process that handles these issues. That is, your organization needs to have an effective idea of what to do when QA starts identifying what the developers have done wrong and how to make sure that the issues are addressed correctly. That is, the developers are brow-beaten into actually opening up their little script editors/Eclipse/IDEs and making things right.
This post, then, will discuss various ineffective defect processes I’ve seen and how issue resolution should work.
Here’s a very common defect/issue resolution process, very popular in the world:
This process is only the second-most popular though, since the “No QA at all” process which uses customers/users as the entities whose discovery of issues yields the result of “Nothing.”Obviously, we can see the fundamental flaws in that process.I’ve seen this novel process in place in more than one project/product, unfortunately:
Click for full sizeThat is, QA goes through all the trouble of finding and logging issues, but the developers sit on the issues until a deadline has passed, at which time they can just bulk close the issues. I actually did work once at a company that rearchitected and rewrote its enterprise level software annually, at which time all issues in the defect tracker were closed summarily.Sometimes, though, you have developers who do respond to issues logged, and the process mutates into something like this:
Click for full sizeNotice what QA does when it uncovers an issue; it
evaluates the issue to see if it’s a real issue or a misunderstanding of requirements. Then, QA tries to
recreate the issue on its own to make sure it can reproduce the issue. Because the developers won’t put any effort into research, because if it’s not on YouTube, a bug doesn’t exist for the highly-touted technical rock stars. Then, QA logs the issue and assigns it to a developer in charge of the project or functional area.But, Director, shouldn’t QA assign it to a manager or project manager? Well, you
could do it that way, but QA is going to find a large number of issues, no doubt, and the management official in charge of vetting the issues will serve as a bottleneck. I’ve always favored keeping the lines of communication open between peers in different teams. Ergo, trust your QA member to assign it to a developer. Of course, trusting the developer would be a mistake, because as we can see in this sample process, the developer will go through any and all of your defect tracker’s resolution statuses to try to get out of fixing the issue.This particular flow chart shows this process going onto infinity, but the QA/Developer “It’s an issue”/”No, it’s not”/”Is too”/”Is not” thing will last until the deadline, at which point the software will ship with a set of known issues that your organization could have fixed.
Now, you can expect a certain amount of give-and-let-die between development and QA regarding issues. Instead of letting these disputes spin into infinity, you need to have some diamonds and arrows in place so that someone has the ultimate authority to adjure. Of course, I would prefer that QA have the ultimate say, but most people in the technology field would disagree. Well, a fie on them. However, you should probably not leave it to someone in the development chain of command unless you’re comfortable with the following process:
Click for full sizeTo be honest, I didn’t work at the place this process describes (and I tell you, friends, I
could not), but a QAmrade did. Apparently, when a tester found an issue, he or she then had to present it to the developer and show the developer; if the developer agreed that it was an issue, QA could log it in the defect tracker. That probably looked really good on the defect reports.No, instead of a developer or a QA person acting as that role of arbiter, someone facing the client/user should decide the issue’s priority. This usually means a project manager or a customer manager person, perhaps someone at the help desk since that’s who’s going to answer the phones and explain why
this happened.The following process shows a semi-ideal process that involves an arbiter who reviews disputes between QA and developers regarding issues and can ultimately decide whether to fix an issue. This works when the arbiter is quality-minded and has a good view of the whole project/product and an issue’s relationship to the whole. If the arbiter is a developer-groupie, you might as well forget it; the process will just look like the one above with the “arbiter” instead of a developer waving his or her hand Jedi-like, trying to convince reasonable people that these are not the bugs you’re looking for.
No, the semi-ideal process looks like this:
Click for full sizeIn this case, QA finds/recreates/logs the issue, and the developer tries to recreate it. If the developer has trouble recreating it,
he asks QA for help recreating it. This flies in the face of convention, where developers who cannot understand issues simply mark them as non-reproducible. Once a developer defensively claims it’s not an issue, then you bring in the arbiter who can determine if it’s an issue or not. Hey, maybe you even have the developer and QA converse to see if QA still thinks the issue exists in light of development’s rationalizing insight. Then, the arbiter talks it over and makes a decision. QA can appeal, but ultimately, a decision ends the resolve/reopen marathon dev and QA could have in the defect tracker otherwise.I call this the semi-ideal process, because the ideal solution would involve cattle prods. But it allows for rapid defect responses for those times when the developers cannot justify their glitches and for keeping the process moving when you reach impasses.
This entry was posted
on Wednesday, August 22nd, 2007 at 3:11 pm by The Director and is filed under Process.
You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.