To a lot of people, maybe even you, the sewer is akin to the cloud. That is, you flush the toilet and some magic happens and at the end, clean water flows into a stream. To a lot of organizations, that describes their process:

1. Assignment
2. ?
3. Profit!

However, as anyone in the wastewater treatment industry or anyone who has spent days proofreading and editing copy related to the wastewater industry knows, the sewer is more complicated than that. And the sewer can offer valuable lessons to streamline your organization’s software development process.

There is one part of the process that is your responsibility.
Here in the state of Missouri, the part of the wastewater treatment system that connects your house to the main, the sewer lateral, is your responsibility. If your effluent backs up into your basement because an unexpected tree root poked into the clay pipe, you need to get it fixed. If you need to put in some PVC to replace dysfunctional wastewater removal on your property, you need to fix it.

Basically, at some point, individuals are responsible for their portions of the process. A failure at this level does not mean the whole process has broken down. Some individuals in some organizations are prone to blaming their own faults and failures on the process and hope to invalidate the entire process thusly. A good process identifies individual responsibilities and allows for some alteration and improvements by individuals so long as the effluent reaches the main.

The sewer uses simple principles.
Most of your sewer systems are gravity-fed. That means the basic structure uses the easiest, most efficient conveyance to carry the effluent from individual nodes in its network to the next stage. There’s even a saying, at least in the blue collar world, about solid waste matter rolling downhill. Your process should use as much natural energy and flow to get things along. Natural tasks and natural checkpoints. Smaller pipes should empty into larger pipes. And so on.

Sure, some sewer systems do make use of STEP (Electric Pump) that use pumps to force water to run uphill or grinders (think garbage disposers) to chop particles in effluent into smaller pieces to force it into smaller pipes. A lot of organizations use these kinds of processes, which requires more energy, more maintenance, and more replacement of moving parts. Basically, this doubles your project manager’s migraine abatement efforts.

The sewer takes into account the local topography.
Every sewer district serves a different region with a different topography. The hills, creeks, mountains, and valleys of each differ. Ergo, the actual sewer itself must account for all the differences. The sewer lines have to go around obstructions and seek out the easiest grades to accommodate the flow of the effluent.

Similarly, individual organizations have different corporate cultures and personalities to accommodate. As hard as it is to imagine, organizations are not only full of roles and job titles, but they also contain diverse personalities. The exposure of the corporate culture to those personalities ultimately influences the organization itself, so sometimes traces of personalities remain after the individuals are gone. Why do we do it that way? Because Andrew did it that way. Sometimes, people won’t even know Andrew did it that way, just that they were taught to do it that way.

When putting in place a process, one might want to toss that way and implement this way, which one might think is the right way. If they feel more comfortable doing it that way, they might continue to do it that way regardless. Sometimes, one should know which people and groups are least likely to change and to account for it.

The point of this cutesy metaphor is this: Distinct organizations have distinctions. Before trying to put a process in place, you need to understand something of the organization so that you build a process that reflects the realities, habits, and tendencies of the organization. Otherwise, you’re building a process that invents them, and that invention is illusion.

Sewers are not replicated across districts.
The basic concepts of wastewater treatment remain a sort of constant, excepting the annual changes in regulations and standards with which sewer districts must comply. However, the fine details of individual sewer districts are based on their local topography (see above). If you try to apply the Metropolitan Sewer District blueprint to Boone County, you’ll find that the individual nodes do not align with the individual businesses and homes to service nor does it account for the hills and creeks of the more rural region.

Likewise, if someone at your organization reads a book about LEAN principles at Toyota and decides to lay your organization on that particular Procrustean bed, well, someone has to step up as Theseus. Don’t let them chop too many limbs off of what your organization does currently to account for the Platonic process whose perfect ideas you want to realize.

A sewer evolves.
Although it might seem that the roadsides feature a constant stream of construction equipment digging, boring, and burying things, your sewer district is not tearing out the existing sewer system and replacing it annually. Sadly, some organizations treat their processes or software tools to manage those processes just that way: they tear it out regularly when it does not work and try again.

Once you put in a process, any process, you should consider making evolutionary changes to the process as you learn how it works and how your organization works with it. This can streamline and improve operations and remove those square corners where waste material can hang up. Additionally, your organization will not require reeducation from scratch annually, and you might avoid another round of meetings where team or group leaders sit around and make the same boxes on the whiteboard again but put different words in them.

Sometimes, you don’t need a whole sewer system.
In many cases, in a whole lot of the country where the houses are not concentrated within a couple dozen feet from each other and from the street, houses and businesses don’t use sewer systems at all. They rely on septic systems or (shudder) open lagoons for their wastewater. It’s just not economically feasible to connect all these discrete locations into a centralized facility.

Look at your organization and the tasks and projects that it works on. Not everything needs to hook into the central process. A single user-reported bug? A small client copy change? The addition of a single piece of copy to the CMS? These tasks don’t need a complete design review with code review and a couple hours of meetings.

That’s not to say they don’t need some process. Even individual mobile homes need some wastewater treatment, even if it’s just a septic tank or an outhouse. But your organization needs to recognize and, as needed, implement alternatives to the process.

Conclusion
When you started this article, admit it: you thought it was a goofy conceit. But when you wash your hands or shower, that water goes down the drain and through a set of pipes, mains, and channels for its wastewater treatment. You want your project to easily, magically flow like wastewater from your tasks to successful project completion. Too many times, organizations create processes out of the abstract and out of pure science, and the processes’ application fails. With a little understanding of the concrete, of the topography, and of the “natural laws” of your organization, you can better construct your processes to work easily and automatically, like a wastewater treatment system.

One challenge for those in IT organizations is not to become so enamored of your own process that you lose sight of serving the business first and foremost. I saw this happen early on in my IT career after a boss went to see Edwards Deming talk about Total Quality Management.

….

Of course, it didn’t work that way. Certain things lent themselves to the TQM process and some didn’t. In particular, there are always those little exceptions, the small jobs that IT teams need to do without running them through whatever process you’ve chosen, because if you did run them through your process, you’d either miss the window of useful opportunity or you’d turn that small job into a big, expensive one.

We longed for some sort of 80/20 rule. We would happily use the TQM ritual for most of what we did, if only my boss would let us quickly deal with the innovation-oriented 20% of tasks that didn’t fit that model. It never happened, and rather than getting behind his TQM philosophy, we and business side peers rebelled, to the point where my boss lost his job.

The problem was that we were being asked to serve the process rather than serve our constituents in the business. Today, I see the same thing happening, often in organizations that are heavily committed to ITIL or some other definitional methodology.

Oh, I recognize that mindset. I saw it aplenty after sitting through process meetings where we discovered previously unknown gems of process, such as When the designers finish the design, they send it to QA, and QA looks it over before it goes to the dev team! Gems of process that hide in plain sight, but which we had to glean and polish in hours of process meetings.

Then, when we finalized the document and everyone signed in blood, the customer facing people, LOB in Wittmann’s vernacular, found that 20% in 100% of the cases that follow. “The client needs this now!” “We’re behind, so we just gave it to the development team with the misspelling of the client products on the buttons and everything, to dev so they could put something quick and dirty up,” et cetera and et cetera.

Then the personnel would switch (let’s just say the turnover at an interactive agency isn’t as bad as a Burger King, but probably worse than a Wendy’s), and we’d be off to make more flowcharts of proper process to ignore.

As I’ve said before, a proper process takes into account that the weasels in your organization are going to try to subvert or ignore the process whenever possible and to create alternative paths for those cases. If the client returns items x days/hours late or the x days/hours/minutes before or after the end of the milestone/stage, shorten the process by this. But only in those cases.

Otherwise your LOB customer-loving, over-promising people will continue to argue that the current effort should not be constrained by process just this once.

What do weight-loss plans and process-improvement programs such as Six Sigma and “lean manufacturing” have in common?

They typically start off well, generating excitement and great progress, but all too often fail to have a lasting impact as participants gradually lose motivation and fall back into old habits.

I’ve sat through enough process meetings to know it’s true. We’d sit in a conference room, chart out a nice flowchart eventually of an ideal situation, and then when the meeting broke up and actual projects started, everyone would do what they always did in the first place. Which is ignore the process.

When you’re trying to improve the process in an organization, you have to take into account the specific organization and the major players within the organization. An outside consultant beaming down solutions from planet Six Sigma is not going to know enough about the industry and about the people within the company. They will offer procrustean solutions that the teams will easily ignore or forget.

The best process improvement can come from within if you have a long enough relationship with other people in the organization to know their strengths and their weaknesses. To improve the process, you need to account for the people who will try to make it work and to accommodate them. You’ll want to capture the best of the things they do and you’ll want to make sure the process handles all of their weaknesses and shortcuts, too, so when the fit hits the shan, the process handles that, too.

A process under glass, framed on the wall, isn’t the goal. Doing things the right way is.

The only difference is that, in a real interactive project, the words would not have been spelled correctly, and sometimes, for no known reason, the design team would use an old, discarded version as the base upon which to make the latest client revisions.

(Link courtesy an interactive project manager in New York.)

You might be tempted to keep your eyes on the tabletop and hope no one knows you’re there and no one calls on you (no one will, of course, because they don’t know why you’re there and, if you’re doing your job effectively, they don’t like you anyway). You might be tempted to make snarky comments and deploy the normal gallows humor, frightening the client. Well, do so, but sparringly. Mostly, though, you’re there to do two things.

One, you need to shoot for the moon on features. You have to ask the client pretty much if he or she wants the application to wash his dog for him. The more features you throw out, the more chances the client can say, “Yes, I want that,” and work that into the estimate, or the client can say, “No, that doesn’t make sense,” or whatnot. Because the client’s going to take a look at whatever your organization puts together at some milestone after the company has expended pizza budget on getting something mostly working by the milestone, and the client will then say, “You know what else I want it to do?” At that time, the yippy customer service person will agree to anything just to get his or her commission, and the timelines and budget might take a hit, especially if the client said he wanted it to wash his puppy in the first place, but also wanted it to wash his golf balls. A little change directive won’t account for major refactoring, especially since the client will want to have secure login next time so that his business partner can wash his or her dog or bowling balls.

You need to get out ahead of that and try to think of how many features and enhancements and alternative workflows you can think of to get them down on paper before you begin.

Two, you need to make sure that the requirements account not only for happy path use cases, but also that they capture how the application should behave when the user steps off of the happy path. Most of your defects are going to come from this area, particularly if you leave it to the developers to come up with their own ad hoc solutions and assumptions. Face it, you’re in QA. You’re the best one in the room for imagining and creating disasters. Create them here, get them on paper, and then make sure they’re handled according to plan. Otherwise, when you do the black magic you do so well in the testing cycle, the developers will have to send down to the gift shop for more chewing gum to implement their fixes–if they bother to at all.

So there you have it. Swing for the fences in requirements gathering. Sure, it makes more work when the client loves half the ideas, but the client was going to come up with those on his or her own halfway through the project. And then the client will like a couple more of them, and you can point out where he or she said no when they were proposed, and fixing it will add cost to the project.

Sure, some of this is the work of business analysts, but I’ve worked plenty of places where they fell down on the job. So get yourself to A rating and get into those requirement meetings. At best, it will save you and your organization time, effort, and refactoring. At least, you might get lunch and the opportunity to frighten a real live client.

This post, then, will discuss various ineffective defect processes I’ve seen and how issue resolution should work.

Here’s a very common defect/issue resolution process, very popular in the world:

Click for full size

Click for full size

Now, you can expect a certain amount of give-and-let-die between development and QA regarding issues. Instead of letting these disputes spin into infinity, you need to have some diamonds and arrows in place so that someone has the ultimate authority to adjure. Of course, I would prefer that QA have the ultimate say, but most people in the technology field would disagree. Well, a fie on them. However, you should probably not leave it to someone in the development chain of command unless you’re comfortable with the following process:

Click for full size

No, the semi-ideal process looks like this:

Click for full size