Directory of All Essays

Sunday, February 18, 2007

Rockets, Cars and Gardens: Visualizing waterfall, agile and stage gate

The further I dig into new product development practices the more I crave a simple way of helping folks new to the concepts visualize them quickly. In that spirit, I've assembled a little pictorial journey through the intriguing landscapes of waterfall, agile, portfolio management and stage gate. For fun, there is also a description of how you can apply portfolio management techniques to individual agile project as a technique for additionally reducing the design risk.

The concepts illustrated here are readily applicable to most software projects including games, websites and of course full desktop applications.

Software development as a learning exercise
All the diagrams start with two premises:
  • Knowledge is critical to success: A large number of software projects fail because we build software that has little real world value. If you dig into the reasons, even teams with great production skills claim they focused on the wrong issue or they really didn’t understand what the customer actually wanted. When you build the wrong software, you end up with useless code, loss of political support and eternal development cycles. If only the team had reliable knowledge of what to build, they could deliver value early.
  • Most product teams start with very little practical knowledge: Knowledge is great, but we start projects with hardly any. We don’t know exactly what our customers want. We don’t know how our design will work when it runs smack into the complexities of the real world. We don’t know what new opportunities and constraints will emerge in the future. Even when we are experts from a related domain, the best we typically have are theorems, guesses and opinions.
Product development is about rapidly producing targeted solutions that generate maximum customer value. In order to do this efficiently, our teams need to be learning teams that are constantly gathering concrete domain knowledge about customer needs. A team that learns the quirks of its customers, code, and business rapidly will often out perform teams operating without this knowledge. With this concept as a filter, let’s take a look at some typical development scenarios and how they are affect by team learning.

Typical product development of a single product
In most traditional waterfall models, teams gather requirements, develop the product and then test it in order to see if they implemented the spec correctly. Only after they release do they gain any insight into what the customer actually desired. The metaphor that is often used is that of launching rocket towards a destination. You get to aim it once and then you fire it off and pray that you hit the target. All well and good, except A) this is a rocket that burns money for fuel and B) when it misses its target, entire teams (and sometimes entire companies) get the ax.

Now, not all is lost. If your company has enough money to burn, they can try again. There is a good chance the team learned quite a bit about what their customers actually desired by failing in the marketplace, so the next moonshot has a better chance of landing closer to some actual customer needs.

The old joke is “If you want to build a good piece of software, rewrite it four or five times.” Eventually, if you keep trying long enough, you’ll hit a target market that wants your product. Some very big companies have made a staggering amount of money following this dogged pursuit of success. They fail again and again and again, but their few successes fund the further pursuit of new growth businesses. The learning is quite expensive but very valuable.

Room for improvement
This pattern of eventual success through multiple failures has been well established in all sorts of new industries. A few smart companies have identified two basic strategies for speeding up the time it takes to identify a new successful market.
  • Standard portfolio model: Fail in parallel. By trying lots of different options all at once, a few will succeed.
  • Iterative model: Fail sooner. Try something simple, test it to see it works and repeat.
Standard Portfolio Model
If the waterfall process is like firing a rocket at a target, the portfolio model is like firing a swarm of rockets and hoping one hits. The company greenlights a large number of projects, funds them fully and hopes that one of them will blossom into a success. In return for spending more money, you rarely have to wait until version 3.0 to observe success.

This model tends to be highly useful when the target market is poorly understood or ephemeral in nature. For example, in the music industry, when user tastes change every month, it doesn’t make sense to spend three years attempting to evolve an individual album to fit the market. It also tends to be useful when the cost of development is low and potential payoff is high. The incremental cost of green lighting one more project can be easily offset by the incredible amount of money a successful record produces.

The big benefit of the portfolio model is that it allows you to try completely different options simultaneously. Companies are often put in situations where more than one idea has potential and they lack the information to choose which ones to fund. If they were forced to choose only one, they would often miss out on creating multiple product lines. Instead of putting all your eggs in one basket, a portfolio model spreads the risk and increased the chance of multiple innovative breakthroughs.

The Achilles heel of the traditional portfolio model is cost. Smaller companies have difficulty funding one project, never mind ten or twenty.

The iterative model
Smaller teams have learned to maximize their learning opportunities by building lots of opportunity for rapid feedback into their process. The agile software development movement is the poster child here, but many of the same lessons are found throughout lean manufacturing, kaizen and other successful practices used by product development companies across a broad spectrum of industries.

The common metaphor is that of driving a car as opposed to launching a rocket. At every possible opportunity, you are looking ahead and adjusting the team’s trajectory in order to steer towards. Each change may seem subtle, but due to all the rapid cumulative adjustments, the team hone in on their targets quite efficiently.

It ends up being okay that the team makes little mistakes. If they veer off a little bit to the left, that's fine. They rapidly learn it was a bad idea and correct their efforts. The short feedback cycles ensures the big mistakes happen far less often.

Instead of taking 12 to 18 months to create and evaluate a new concept, they build and put new version in front of users every 2 to 4 weeks. They also work in high bandwidth environments where all the team members are close together and close to the customer. Team members converge on and build concensus around good ideas over a period of hours, not months. Teams become experts through intense hands-on problem solving and testing. This ends up building products much more likely to serve real needs than those imagined in Platonic specs by ivory tower experts.

Agile development is favored by small start up teams because the techniques greatly reduce the risk of an individual team failing. If you only have room for one shot at the target, you might as well steer your way to success using lots of rich information instead of launching blindly into the unknown. Long term, agile processes delivering more value sooner, with lower overall risk.

An agile project is intensely focused. In the rush of completing only high priority features, many alternative concepts never get the chance to be explored. Customers, a rather vague and argumentative bunch at best, are required to speak with one voice in the name of reducing noise and thrash for the team. For many teams struggling just to get software out the door, these traits are a godsend. The downside is that there is little concept of strategic portfolio management.

Agile projects can be a bit like a hill climbing algorithm. They will steer towards the closest customer success story, but may ignore a bigger opportunity further away.

Stage Gate: Cross breeding portolio models and iterative development
We can do better than either pure agile or pure portfolio development. The stage gate process borrows a little bit from both iterative and portfolio techniques. You are still launching multiple products, but then you use iterative development techniques to guide each project towards success. You kill projects that fail to meet your goals.

Imagine that we are gardeners. We seed a diverse portfolio of projects, some high risk, so low. As we add team members and customer feedback, the projects begin to flourish, winding their way up towards the sunlight of launch. The core team development practices can be agile since we want to encourage short cycles of rapid feedback. The best ones start to blossom and ripen with obvious customer value. However, some projects are sickly and seem unlikely to produce much of anything.

As gardeners, it is also our job to groom the garden. The weak projects are trimmed back and turned into mulch that helps nourish the most successful projects as well as create a bed for seeding even more promising ideas. The best projects are harvested and sent off to customers, rich with ripe features, bursting with value. We are constantly seeding new projects to keep our portfolio balanced.

We’ve added three items to the traditional agile model.
  • Diverse seed teams: The first is the concept of multiple teams heading in different directions with different goals. There isn’t a single customer, but multiple customers driving multiple potential success strategies.
  • Kill Gates: The second item is the concept of gates every few iterations that kill poorly performing projects. It is better to return those resources back to existing projects than to continue investing. You can think of the success criteria that drives the gates as a set of reasonable constraints. The team is allowed to do whatever it desires as long as it meets the outlined constraints that are measured at each gate.
  • Concept banks: The third is the concept bank where old ideas are stored for possible recycling. You never know when the remnants of an old idea will nourish a strong new project.

You may start out with dozens of projects, but in the end you’ll only launch a few into the market. The combination of trying lots of directions with very small low cost agile teams dramatically increases your opportunities to learn compared to either pure iterative or pure portfolio techniques.

You spend dramatically less than you would in a pure portfolio model since you don’t have the heavy cost of finishing and deploying the bulk of unsuccessful market explorations. All that waste can go back into either exploring new options or building out projects that show success.

You have a higher success rate than a purely iterative model since you are exploring multiple hills instead of climbing a single hill and hoping that it is the highest one around. This recognizes that not all successes are created equal and it can be easy to get stuck in rut serving a niche. By keeping your options open, you can always shift resources around if a truly great opportunity emerges.

One downside is that cost is a little higher than a pure iterative model. You need to devote a portion of your overall efforts to generating and exploring new opportunities. This is typically more than offset by the earlier access to successful products. Another downside is that some long term projects (such as say…a fusion reactor) are culled before they show their promise. This can be handled through careful tailoring of gates so that break through innovations are properly weighted.

The Harvest Process: Applying a stage-gate model to a single product
A single product can benefit greatly from the ideas in the stage gate model. A single product is composed of a variety of separate features, scenarios and audiences. We can treat each chunk of customer value as a separate project and bundle of high value projects as a product. The basic model is the same as the stage gate process outlined above. However instead of spinning off multiple products, your individual teams tend to be focused on critical workflows or user scenarios for a single product.

Returning to the garden metaphor, as you nurture these areas of value, you should always have projects that are ripe for release. Publishing can be as simple as creating a public release that turns off all the experimental projects, leaving only your best features to show the public. I think of this as harvesting a ripe crop for your favorite customers.

As time passes, the little ideas that you seeded earlier will mature and produce a new crop of high value features. The process just keeps repeating, season after season, release after release.

The cliché in our business is that ‘ideas are cheap’. The shame of the matter is that some of the best ideas are left to rot because they don’t come from the right people and the production schedule has no room for creative thinking. The Harvest process builds innovation in at the lowest level possible. Everyone is encourage to come up with ideas and there is a clear and official process by which those ideas can become reality.

Making it work
In order to pull this off you need some basics in place. There is a substantial expense to running multiple projects and constantly integrating their efforts. With the right people, the right process and the right values in place, it is a very achievable goal.
  • Multiple, small agile teams: A single monolithic team has difficulty guiding a multitude of projects. Smaller teams can champion their area more easily.
  • Highly refactored and regression tested modular code: You need to be able to change one element without breaking the rest of the projects.
  • Innovation at the edges: Teams need to be able to innovate in interesting directions without constantly referring back to the ‘master planning committee’
  • Kill gates: If you don’t understand success and kill projects that don’t meet the bar, you’ll end up with a cancerous mass of bizarre, unhealthy features that have grown in random directions. Quality kill gates that actually kill projects are critical to success.
  • Shared design: You still need to present a unified face to the customer. Each team is under additional constraints to share visual styling and minimize conceptual complexity for the product as a whole. These constraints are built into the kill gates.
  • Customer feedback: At every stage, you need to be testing your products with real customers. You should be able to quickly deploy a working product to your customers every day. None of these ideas can succeed without people using working versions of your software and the team being wiling and able to learn from the customers’ reactions.

In the end, any of the techniques outlined above have a chance of working. You need a driven group of people with the right circumstances in place. With the right amount of random luck and persistence, even a waterfall project can change the world. For some fat cats working in bloated large companies, this is enough. However, if failure has a personal cost, you’ll want to look into the alternative techniques such as agile, stage gate, or harvest. Over the long term you’ll notice less overall portfolio risk and your customers will start enjoying your superior, well informed product designs.

If you take away one thought today, it should be that learning is fundamental to software development. We do not live in a Newtonian world where huge mechanical brains can predict the future. Instead, we operate in a constantly shifting organic universe of cultural trends, customer urges, fluctuating cash flow and skeptical sponsors. In order to deliver great products on time, we need to learn from our environment and build a response rapidly and precisely. Portfolio technique increase learning by trying more options. Agile techniques increase learning by shortening the feedback cycle. Build both of these key concepts into your teams and you’ll find that they make smarter decisions and deliver more value to your customers.

Take care

References and resources

Original pretty pictures
I was inspired by these image of agile development when I began making diagrams.

Small initial teams? Try 20% of one person.
With Harvest and Stage Gate, initial teams can be very small. For example, you could devote 20% of one person’s time for the initial concept stage. That doesn’t sound like much, but if you have every single person in the company devoting 20% of their time to new ideas, you’ll be seeding the equivalent of hundreds of new projects every year for no additional headcount. Google is the golden child of this concept right now, but 3M has been doing it successfully for decades.

Put those hundreds of ideas through a simple checklist that looks for potential customer value. Spin off the best ideas into tiny teams of a couple people and give them an iteration or two to come up with something cool (and testable.) Repeat the process several times and you rapidly gain some surprisingly innovative products .

Iterative techniques
Agile Overview:

Stage gate
Winning at New Products: Accelerating the process from idea to launch, 3rd edition Robert G. Cooper, copyright 2001

Project Horseshoe Report on building games that sell
Many of the ideas in this essay were heavily influenced by discussion inside the Play Dough group at Project Horseshoe 2006.

Labels: , , , , ,

Read more!

Sunday, January 21, 2007

Project Horseshoe Report: Building Innovative Games that Sell

The 2006 Project Horseshoe reports are out! In the place of my irregularly scheduled essay, I'd like to point you toward the collaborative report that I've been working on with our appropriately named "PlayDough." We dug deeply into the issues around funding innovative games and ended up producing 18 pages of revolutionary white paper goodness.

Click here to read the report. Now for a bit of commentary on the topic of innovation and business.

We are bad at designing profitable games

If you've read this site for some time, you'll notice that I keep returning to the themes of measuring success and rapid feedback cycles so that you can make necessary changes early. Unfortunately, it became very clear that most players in the game industry do not apply these lessons to the business of designing games. It isn't purely a publisher issue, nor is it purely a developer issue, but the overall issues are rather clear:
  • We don't know what makes a successful game.
  • We don't measure our games regularly from the customer perspective to see if we are going down a successful path.
  • We don't act on information that our projects will be market failures.

A Solution
You might expect a good deal of venting at one of these designer retreats, but everyone was far more interested in fixing the problem. The game industry isn't the first to run into these issues and the solutions we found used in other industries are surprising palatable. Our group ended up exploring the adaptation of the Stage Gate new product development process to game develpment.

Stage Gate is a well known options-based portfolio management technique that focuses on improving product launch success rates. It is particular appealing to the game industry because it allows companies to try out lots of different ideas ranging from highly innovative project to product line extensions and then guide the most successful ones toward a market launch. Hundreds of companies in other industries use this model and it has been proven time and time again to dramatically improve success rates, reduce design and execution risk and also reduce time to market. These are all good things. :-)

Low hanging fruit
My basic belief is that innovative products are a highly desirable and profitable component of any publisher's portfolio. However, in order to make this obvious to all the egos involved, you need a process that provides hard data and a way to mitigate risk/fear. The Stage Gate process does this with surprising efficiency.

It is also obvious that a competitive market eventually punishes stupidity. Ignoring new markets while releasing products with a poor chance of success only works until someone figures out that there are better ways.
The first party story has some obvious suspects and I know at least one forward thinking 3rd party publisher has already started rolling a version of the stage gate process out across their teams. It will be fascinating to watch the techniques we talk about in the paper inevitably take hold over the next decade.

If you work in any organization that deals with multiple retail game titles, you are likely to save millions of dollars by taking the lessons of this report to heart.
Perhaps that is a big claim, but one that is well justified by the historical results from other industries. The opportunity to become the savior hero in your organization is immense.

Kudos go out to everyone in the PlayDough group. We had the perspectives of independents, studio presidents, and publishers represented. Everyone is looking for solutions and everyone who worked on the report feels quite hopeful about the future. It is a wonderful thing to be able to point to business best practices that hold the promise of dramatically improving the industry as a whole.

take care

Labels: , , ,

Read more!