The 2008 Project Horseshoe reports are up! We wrote about how to diagram multiplayer games using skill atoms. Truly a brilliant weekend. The discussion was quite wide ranging and as a result the write up became a bit...long. However, the results should spark a few brain cells. Let me know what you think! :-)
I’m writing this on the long flight back from Project Horseshoe 2008. The last bittersweet night, we stayed up till five AM playing games and talking about games. The conversation shifted from the slow death of games as we knew them, to fresh games that will change the world, to the little tips we use to thrive each day. There is something distinctly surreal about chatting quietly with such an intimate knowledgeable group during the wee hours of the morning, there on a lonely porch in the uncharted depths of Texas. And yes, there were indeed baby racoons.
This year, I took a risk. If you’ve been following this blog for a bit, you know that I’ve been working on skill atom techniques for modeling gameplay. I’ve written about it. I’ve used it myself. There has even been a talk or two. Yet, aside from a few furtive emails with other happy heretics, I’ve never had a chance to do the following:
Explain the model to a crowd of natural skeptics, working designers who have been successfully building games for years.
Get them to tear it apart.
The cautionary tale of the secret paint formula I’m reminded of a story that Norman Rockwell used to tell. He once became good friends with a fellow painter who was famous for his rendering of luminescent, sensual skin tones. The painter used a secret formula for his paint and he guarded it jealously from potential imitators. When the painter died, he willed his greatest gift, his secret paint formula to Rockwell.
Rockwell excitedly tried out the formula, but ultimately found it disappointing. The paint was too slick and difficult to control, so he gave up on it and instead fell back on his own preferred techniques. The real secret had never been the paint formula. It was just one little piece of the painter’s vast organic, highly individual process. The real secret was the intuitive wisdom that comes from making a thousand paintings. Sadly, such a thing is not transferable to others. When he died, his specific way of creating paintings died with him.
Are skill atoms the same thing as the secret paint formula? Are they a glossy coat of theoretical hand waving that only works for the people who invented it? Many people I’ve talked with see ‘game grammar’ as nothing more than a time wasting intellectualization of a fundamentally intuitive activity. I went into the weekend with this thought very much at the forefront of my mind.
Why stop there? If all we had done was validate or invalidate the skill atom model for simple games, it would have been a useful weekend. But by god, this is Project Horseshoe and people are nothing if not psychotically ambitious. To up the ante, our group decided to apply skill atoms to multiplayer games. I’ve never done this.
How do you model a deeply psychological behavior like bluffing? Gifting? Competition? Collaboration? Goodness! I didn’t have a lot of answers prepared for this topic and honestly expected that the skill atom model would immediately collapse under the weight of all the crazy things that happen as soon as you add two or more players to a game design. All it would have taken is one smart designer to raise a single counter example and my fragile model would burst apart, defeated by reality.
Some questions that I had included:
Could we even begin to talk about multiplayer with skill atoms? The alternative is that this is a model that is limited to only single player experiences. That would be like coming up with a model of physics that worked for one ball in a vacuum, but wasn’t useful for something useful like say…building bridges.
Would the system scale to complex systems? Often when you use a diagramming technique (like UML or state diagrams) to understand real world projects, the resulting diagrams becomes so convoluted that the model does more to confuse than to illuminate.
Would the system be useful to designers during every day work? It is much easier to come up with a academic system of analyzing games that works best if you are an ivory tower dweller who can devote hundreds of hours to breaking down each interaction into pretty diagrams filled with obscure invented lingo. However, I’m looking for utilitarian tools that can be applied in that critical 10-minute gap between playing a prototype and deciding what to try next.
Can this system be taught to other designers? Like the secret paint formula, most game models I run across are only useful to their inventors. If I can’t observe other designers applying the model successfully without my intervention there is something horribly wrong with the approach.
We ripped the skill atoms apart. We analyzed multiplayer M.U.L.E. We looked at charades and then took on football and buffing in MMOs. We used skill atoms to prototype a new multiplayer game about gifting using a bag of plastic Indians. At some point, not so long from now, our group will come out with a report. In that report, we’ll be blunt about what we found. What worked? What was flawed? The results are fascinating.
Our team’s report will be one of several reports to come out of Project Horseshoe by groups of game designers just as crazy and inspired as we were. If any one of these reports starts gaining momentum, the world of gaming as we know will change. It turns out that moving our industry forward isn’t about complaining. It is about getting smart people together where they have the time and the space to think. Grab a beer (Aventinus Double Bock, no less), join the mind meld and use the vast pool of centuries (!) of game design experience to come up with real solutions. Then follow up again and again and again.
In that spirit, I can't wait to share our final report with everyone.
Time for some much needed sleep, chock full of dreams. Danc.
PS: Warm kudos to George, Linda and Teresa for putting Project Horseshoe on. It is obviously a labor of love and is utterly unique compared to the other events and conferences I’ve attended. If you ever get an invite, don’t hesitate to go.
Gamasutra was kind enough to post the report our group produced this year at Project Horseshoe. If you are interested in a brief glimpse into the thoughts of veteran game designers, it is worth a read. The topic our group chose was roughly related to 'story in games'. However, it became quite clear that the group was more interested in discussing how games are able to support powerful new experiences.
Story, in the traditional sense, was reduced into a rather small and telling role. The best stories, we concluded, are the ones told by each player as they share their amazing gaming experience with friends and family. In this game-centric view of media, "story" is but the afterglow, a processing step in our understanding of past grand events. Games are about causing the birth of the primary event, that glorious moment where the player actually experiences those grand events first hand.
And what is game design? Game design is the process of building devious systems that are fully aware of human foibles, quirks and desires. These systems click and whir with manipulative intent. The best ones encourage human beings to reach out and experience, of their own free will, something new, meaningful and real. Game design as a distinctly unique and powerful tool set I got the glimpse that designers are finally coming to terms with games as their own unique creative medium. I'm not quite sure how to put this into words, so bear with me. :-) The examples we discuss passionately were from games, not movies, theater or books. The language and metaphors were distinctly born from the theories of play and human psychology. Folks were discussing how to build the next world changing work of art in a rich, detailed vocabulary that despite borrowing liberally from other fields, was distinctly unique to games. In fact, there was little talk of cut scenes or protagonists or narrative in general.
The group quickly bypassed such dorm room topics as a waste of our limited time. It would have been as useless as a gathering of early Jazz musicians debating how the aesthetics of Shakespearean plays might advance their new art. Drawing such parallels might certainly be academically intriguing, but it has turned out to be generally orthogonal to the task at hand of making great games.
I've always believed that the real philosophy, language and tools of understanding and building great games will comes from messy development trenches, not the ivory towers or posh hills of Hollywood. Artists, whose lives crumble or soar based off the success of their vision, have far more incentive to push for new techniques that work. The language of games grows like slang in the ghetto, rough, occasionally incoherent to outsiders, but still immensely evocative and effective. When such a language makes the leap to the mainstream, the world changes.
Project Horseshoe 2007 slides: Smashing the game design atom
Here are my slides (with talking notes) from Project Horseshoe. I blazed through this in about 30 minutes since dinner was waiting and there is nothing more ornery than a crew of wild haired game designers in complete glucose crash. See if you can spot the source of the infamous '8mm' meme that stalked the conference.
Since it was Halloween yesterday, let's start with a tale of horror. Not so long ago there was an experienced team, working with a known platform, and a known engine. They had just scored a popular girl friendly license valued at roughly $160 million. Their game had the potential to hit as big as Pokemon or Nintendogs.
The designer ignored all this. You see, he had always wanted to make a Zelda clone...one with the critical element that has always been missing from all Zelda games: Hardcore jumping puzzles. The designer thought, "Nintendo is smart, but how could they have missed such an obvious improvement?" Sure the license was targeted at tween girls, but tweens like big swords don't they? This was the game he personally had always wanted to play.
The team were contractually obligated to go along with the design. It had been green lighted. There were milestones tied to the completion of each voluminous chapter of the tome like design script. If the team missed the milestones, the penalties were extreme. So they crunched in happy little silos of artists, level designers and programmers, all in accordance to a strict production schedule. It was the only possible way to get all the specified content done in time for the looming holiday ship date.
Finally, as the end drew near, they sent it off to testing. Early reports come back that the jumps in the first few levels were rather clumsy. The designer relied on his gut and sent forth an email containing a new set of parameters that were intended to polish the jump mechanics.
Eventually, a month later, someone got around to playing the last few levels. Uh oh. They relied heavily on laboriously constructed jumping puzzles tuned to the old jump mechanics. The last few levels of the game were massively broken.
It was too late to fix all the early levels so they entered into a death march to rework the last few levels. Lacking time or resources, they busted their budget hiring extra crew to take up the extra workload. The game was still delayed by several months.
Surprise, surprise, the end result wasn’t a very good game. It received miserable scores, but even worse, the core audience who would have bought it on the license alone was completely alienated by the design decisions. They wanted to befriend and grow cute little animals. They didn't want to die repeatedly while being attacked by spiky monsters while scrambling through military obstacle courses.
When the licensee pulled out of the sequel, the team collapsed. The human cost was immense. Home were lost, families relocated, many were so burnt out on game development they left the industry permanently, their passion crushed.
There were a lot of problems in this tale, but the primary one was a blatant failure of the game design process at almost every level. The game designer really didn’t know what he was doing. He thought he was writing a movie script. He thought he was making a game for himself. He had no idea that the game systems he was dabbling in were deeply interconnected.
Most game design that occurs isn’t much better off. It is a combination of craft (such as cloning Zelda) and intuition (such as when he hoped that tweaking the jumping mechanics would fix all his problems.) There is no science here. No predictability.
We have the ability to do so much better. We can create a science of game design.
If we want to modernize game design and move beyond the land of craft and intuition, we need to face and conquer four great challenges. These challenges will define game design for at least the next twenty years and perhaps longer.
Modeling our players: What do our player really want?
Generating new games systems: How do we create new mechanics that serve those player needs?
Metrics: How and what do we test to make sure we are doing the right thing?
Results: How do we get the results quickly and iterate on them so we can evolve the game quickly?
If we solve these issues and start spreading the resulting practices across the industry, horror stories like the one I just told will become mostly a thing of the past. The ultimate promise of a deep pragmatic science of game design is better game, happier teams and fewer human tragedies.
We are starting to see a smattering of theorists and teams with money to burn tackling these problems. They are creating systems that save their butts from expensive design mistakes. This is damned exciting.
You’ve got the Halo folks tracking heat maps of where players die. Valve has been relying on metrics for years. Nintendo builds early player tests and kleenex tester right into their dev process.
On the game systems side you’ve got Raph’s game grammar.
We are starting to rely on real data to model players moods and reactions with Chris Bateman and Nicole Lazarro’s work.
The work is still at the stage where most pragmatic folks think of these systems as the domain of eccentrics. Yet, each isolated advance reminds me of the turn of the century when physics was being cracked. Brilliant theorists. Great experiments. World changing results.
All these systems are being developed in parallel. You can measure things, but you don’t know what you are supposed to measure. You can write about game grammar, but it never is anything more than a loosely applied system of egghead analysis.
Maybe, just maybe, we can come up with a unified system that tries to answer multiple challenges simultaneously. The connections are all there. We just need to put them together.
In my Seattle laboratory, I'd been working on one attempt. It mixes game grammar, player models and measurement systems into one delightfully unified game design process. I’ve got 10 minutes left to share it with you. Think I can do it?
I started with a player model. Let's assume for a moment that players are naturally inclined to wander about, sucking up tidbits of info in the hope of learning interesting new skills.
From the player model we can construct an atomic feedback loop that describes how they learn all the new skills. This basic atomic loop includes all the fundamental pieces of a game. We are taking the deconstructed analytic elements described in so many books and tying them back together into a functional system.
You’ve got your game system, that black box in the center of the loops.
You’ve got your player input
You've got feedback to the player
You have the the players cognitive model of the game.
We’ve reduced the scope to a single atom in order to making things managable and Press button, character jumps. That’s a skill atom.
Once we have a skill atom we can say interesting things about how the player interacts with it. First, skill atoms have states.
The player can figure out how to jump. They can exercise that skill by jumping on things. That is mastery. We can record this.
The player can fail to figure out how to jump. They never touch the button again. That’s early burnout.
They can get bored with jumping and ever jump again. That is late burnout. We can measure this as well.
Skill atoms are chained together. You can visualize them as a directed graph. Later stage atoms depend heavily on early stage atoms.
Want to kill a Koopa? You need to jump on him. Better hope you mastered the jump skill. We can now represent that classic relationship created by Miyamoto ages ago in a visual model. The theory is slowly catching up with the experimentalists.
You can turn these little chains into big chains that describe the entire game. Here’s a skill chain of Tetris.
Skill chains are remarkably flexible and rather easy to apply to almost any game. You look for the actions the user is performing, the skills they are learning and the positive / negative feedback you’ve built into the game. Any game with explicit rewards can be diagrammed.
There are probably a goodly number of you rolling your eyes at this point. You can create pretty diagrams to analyze anything. Here we've got someone who has created a very lovely and describing diagram of a penguin defecating. This is not a helpful diagram.
We ultimately need pragmatic everyday tools, not egghead analytics. The primary reason we create skill chains is to help solve two of our outstanding challenges:
Get real results quickly
Choose the right metrics so we aren't wading through huge quantities of data.
Skill chains can be used to create a rapid, iterative test driven game design process.
If we really rapid feedback, let’s build the feedback system into the game from the very beginning. Skip the giant paper tome phase. Start with a playable system that gives you meaningful reports.
The nice thing about skill atoms is that they eminently testable. When you write code that is going to be put in front of player, define your skill atoms. Its the same conceptual ideas behind writing unit tests.
You have a test framework.
You write the tests when you write game logic.
You run the test suite when you run the game logic.
You get a clean simple report when someone plays the game.
When you write your game systems, you can instrument each and every atom. It is a relatively inexpensive process.
You labels the rewards
You label the actions
You know when and atom is touched. You know when it is inactive. All those, states, burnout, inactive, etc you can record.
Remember burnout? The next time someone plays the game, we can visualize burnout directly on our skill chain diagram. You see instantly what atoms folks are missing. Here is someone failing to figure out how to complete a single line in Tetris.
You can also look at the data in terms of feedback channels and activity graphs.
Either way you get quick, easy to decipher feedback.
Instead of having a team that creates customized visualizations tailored to your game, you can use a more generalized system.
Instead of sorting through dozens or hundreds of badly organized logs, you can see in a glance where problems are occurring.
This requires a change in your development methodology. You want people to play your game as early as possible and as often as possible. Luckily automated testing of skill atoms reduces the cost substantially compared to traditional manual tests.
Anytime that anyone, anywhere in the world runs you game, you get valuable play balancing information.
Build up a database of a thousand players and release your daily builds to three people a day for every single day of your dev cycle.
In this day of web 2.0 and connected consoles this is now a broadly accessible practice.
Once you have rapid, daily feedback in place, you can use the resulting reports to evolve your design iteratively. All this analytical game grammar silliness becomes a foundational feedback system.
We can regression test game designs now.
We can fix busted skill atoms and see how things improve the next day.
What happen when we refactor our designs to make them more testable? I have no idea, but it excites me.
Imagine if a system like this had been in place when the 'designer' in our horror story made his jumping tweaks. The dashboard would have gone dark almost instantly with burnout spreading across the screen.
The systems I've described today are just the beginning; rough sketch of the future, if you will.
Our player models are primitive.
Our metrics can advance dramatically in their sophistication. We are just starting to tap into biometrics
Our player testing systems are still expensive to run.
There are amazing new games waiting to be designed and evolved into stunning experiences.
The great challenges are still out there. Both the theory and practice of our science is still very being born. Sometimes I wonder, "Who is going to take game design to the next level?"
I love this picture. 1927 5th Solvay conference. Einstein, Bohr, Curie. 17 out of 29 attendees went on to win the Nobel prize.
The first conference was in 1911, almost a hundred years ago. Einstein was the youngest present. Who is the youngest person here? These quirky, brilliant people revolutionized our understanding of physics. Without their work, we wouldn’t have semi-conductors, computers or video games. They were theorists and experimenters not so different than what we have in our industry today. A small group of eggheads changed the world.
I look out at this group and I see the same potential. We’ve got the brains. We’ve got the time.
Let’s make this an amazing weekend."
take care Danc.
PS: There was one more group photo shown immediately after the Solvay photo. It however, has been redacted due to national security concerns.
First, isolate the group I just stumbled back from Project Horseshoe, the amazingly intense game design think tank set in the deep wilderness of Texas. Ah, Texas. Armadillos sporting leprosy and overhanging trees dangling with dozens of wiggling arachnids were amongst the more colorful wildlife lurking on the isolated ranch where we stayed. There is nothing more endearing than having a conversation about virtual gods interrupted by a friendly eight-legged fiend swaying two inches in front of your nose. Did you know that wolf spiders merely cause agonizing pain and swelling? We absentmindedly brushed the curious observers aside and got back to the intense task of solving the world’s biggest game design problems. No time for spiders.
If you’ve ever wandered the halls of GDC and managed to hook up for a few golden moments of great design conversation, you understand the purest essence of Project Horseshoe. It is exactly like that, except the mind melds last for two and a half days and the intensity is magnified by an order of magnitude. Sleep is deferred, scotch consumed, and electric arcs of thought crackle. I always come away with a feeling of inevitability: “We shall change the world.”
The quality of people in a small team matters Key to all this was the quality of the people. With the esoteric topic of game design, you might expect perhaps 1 out of every 100 industry folks and perhaps 1 out of every 100,000 players to be able to have a coherent conversation. Every single chat here went deep. Concepts that typically require a few hours (or days) of build up, were grokked in minutes and then leapfrogged. All told, the various attendees, hailing from a spread of publisher, AAA, casual and academic backgrounds have already created experiences that were integral to the lives of millions of players. I've worked with some good eggs in my life, but never have I been around such concentrated brilliance.
Passion is also critical And it was obvious this crew will inevitably create or influence the creation of a hundred million more experiences in the future. Over and over I heard the passionate sentiment that we are at the very beginning of games as revolutionary and fundamentally positive social force. To have such idealism and talent focused in the same room for a once-in-a-lifetime event was downright intoxicating. It was like witnessing the birth of quantum physics or the computer industry or modern cinema.
Include a few elements from left field I had the honor of giving the talk that kicked off the geeky portion of the event on Thursday. George “The Fatman” Sanger introduced me by saying he told his wife he found me on the internet. Note: Craigs List ads do in fact work. "Slinky SWGD seeks walks in park." I’ll post up my slides shortly. I must admit that I was repeatedly surprised and humbled to find out that so many of my fellow attendees read this little blog. (Hay, everyone!)
Work towards are a clear, pragmatic goal As is the yearly tradition, the working groups at Project Horseshoe will release their reports over the coming months. It would be impossible to capture everything discussed, but at least a few big ideas will surface. Practical, practical. Otherwise, it is just a bunch of egos gusting hot air. And that isn't the point.
What a rush. I’ve got another dozen fresh essays crammed into my bursting noggin. Life is absolutely glorious.
Rockets, Cars and Gardens: Visualizing waterfall, agile and stage gate
The further I dig into new product development practices the more I crave a simple way of helping folks new to the concepts visualize them quickly. In that spirit, I've assembled a little pictorial journey through the intriguing landscapes of waterfall, agile, portfolio management and stage gate. For fun, there is also a description of how you can apply portfolio management techniques to individual agile project as a technique for additionally reducing the design risk.
The concepts illustrated here are readily applicable to most software projects including games, websites and of course full desktop applications.
Software development as a learning exercise All the diagrams start with two premises:
Knowledge is critical to success: A large number of software projects fail because we build software that has little real world value. If you dig into the reasons, even teams with great production skills claim they focused on the wrong issue or they really didn’t understand what the customer actually wanted. When you build the wrong software, you end up with useless code, loss of political support and eternal development cycles. If only the team had reliable knowledge of what to build, they could deliver value early.
Most product teams start with very little practical knowledge: Knowledge is great, but we start projects with hardly any. We don’t know exactly what our customers want. We don’t know how our design will work when it runs smack into the complexities of the real world. We don’t know what new opportunities and constraints will emerge in the future. Even when we are experts from a related domain, the best we typically have are theorems, guesses and opinions.
Product development is about rapidly producing targeted solutions that generate maximum customer value. In order to do this efficiently, our teams need to be learning teams that are constantly gathering concrete domain knowledge about customer needs. A team that learns the quirks of its customers, code, and business rapidly will often out perform teams operating without this knowledge. With this concept as a filter, let’s take a look at some typical development scenarios and how they are affect by team learning.
Typical product development of a single product In most traditional waterfall models, teams gather requirements, develop the product and then test it in order to see if they implemented the spec correctly. Only after they release do they gain any insight into what the customer actually desired. The metaphor that is often used is that of launching rocket towards a destination. You get to aim it once and then you fire it off and pray that you hit the target. All well and good, except A) this is a rocket that burns money for fuel and B) when it misses its target, entire teams (and sometimes entire companies) get the ax.
Now, not all is lost. If your company has enough money to burn, they can try again. There is a good chance the team learned quite a bit about what their customers actually desired by failing in the marketplace, so the next moonshot has a better chance of landing closer to some actual customer needs.
The old joke is “If you want to build a good piece of software, rewrite it four or five times.” Eventually, if you keep trying long enough, you’ll hit a target market that wants your product. Some very big companies have made a staggering amount of money following this dogged pursuit of success. They fail again and again and again, but their few successes fund the further pursuit of new growth businesses. The learning is quite expensive but very valuable.
Room for improvement This pattern of eventual success through multiple failures has been well established in all sorts of new industries. A few smart companies have identified two basic strategies for speeding up the time it takes to identify a new successful market.
Standard portfolio model: Fail in parallel. By trying lots of different options all at once, a few will succeed.
Iterative model: Fail sooner. Try something simple, test it to see it works and repeat.
Standard Portfolio Model If the waterfall process is like firing a rocket at a target, the portfolio model is like firing a swarm of rockets and hoping one hits. The company greenlights a large number of projects, funds them fully and hopes that one of them will blossom into a success. In return for spending more money, you rarely have to wait until version 3.0 to observe success.
This model tends to be highly useful when the target market is poorly understood or ephemeral in nature. For example, in the music industry, when user tastes change every month, it doesn’t make sense to spend three years attempting to evolve an individual album to fit the market. It also tends to be useful when the cost of development is low and potential payoff is high. The incremental cost of green lighting one more project can be easily offset by the incredible amount of money a successful record produces.
The big benefit of the portfolio model is that it allows you to try completely different options simultaneously. Companies are often put in situations where more than one idea has potential and they lack the information to choose which ones to fund. If they were forced to choose only one, they would often miss out on creating multiple product lines. Instead of putting all your eggs in one basket, a portfolio model spreads the risk and increased the chance of multiple innovative breakthroughs.
The Achilles heel of the traditional portfolio model is cost. Smaller companies have difficulty funding one project, never mind ten or twenty.
The iterative model Smaller teams have learned to maximize their learning opportunities by building lots of opportunity for rapid feedback into their process. The agile software development movement is the poster child here, but many of the same lessons are found throughout lean manufacturing, kaizen and other successful practices used by product development companies across a broad spectrum of industries.
The common metaphor is that of driving a car as opposed to launching a rocket. At every possible opportunity, you are looking ahead and adjusting the team’s trajectory in order to steer towards. Each change may seem subtle, but due to all the rapid cumulative adjustments, the team hone in on their targets quite efficiently.
It ends up being okay that the team makes little mistakes. If they veer off a little bit to the left, that's fine. They rapidly learn it was a bad idea and correct their efforts. The short feedback cycles ensures the big mistakes happen far less often.
Instead of taking 12 to 18 months to create and evaluate a new concept, they build and put new version in front of users every 2 to 4 weeks. They also work in high bandwidth environments where all the team members are close together and close to the customer. Team members converge on and build concensus around good ideas over a period of hours, not months. Teams become experts through intense hands-on problem solving and testing. This ends up building products much more likely to serve real needs than those imagined in Platonic specs by ivory tower experts.
Agile development is favored by small start up teams because the techniques greatly reduce the risk of an individual team failing. If you only have room for one shot at the target, you might as well steer your way to success using lots of rich information instead of launching blindly into the unknown. Long term, agile processes delivering more value sooner, with lower overall risk.
An agile project is intensely focused. In the rush of completing only high priority features, many alternative concepts never get the chance to be explored. Customers, a rather vague and argumentative bunch at best, are required to speak with one voice in the name of reducing noise and thrash for the team. For many teams struggling just to get software out the door, these traits are a godsend. The downside is that there is little concept of strategic portfolio management.
Agile projects can be a bit like a hill climbing algorithm. They will steer towards the closest customer success story, but may ignore a bigger opportunity further away.
Stage Gate: Cross breeding portolio models and iterative development We can do better than either pure agile or pure portfolio development. The stage gate process borrows a little bit from both iterative and portfolio techniques. You are still launching multiple products, but then you use iterative development techniques to guide each project towards success. You kill projects that fail to meet your goals.
Imagine that we are gardeners. We seed a diverse portfolio of projects, some high risk, so low. As we add team members and customer feedback, the projects begin to flourish, winding their way up towards the sunlight of launch. The core team development practices can be agile since we want to encourage short cycles of rapid feedback. The best ones start to blossom and ripen with obvious customer value. However, some projects are sickly and seem unlikely to produce much of anything.
As gardeners, it is also our job to groom the garden. The weak projects are trimmed back and turned into mulch that helps nourish the most successful projects as well as create a bed for seeding even more promising ideas. The best projects are harvested and sent off to customers, rich with ripe features, bursting with value. We are constantly seeding new projects to keep our portfolio balanced.
We’ve added three items to the traditional agile model.
Diverse seed teams: The first is the concept of multiple teams heading in different directions with different goals. There isn’t a single customer, but multiple customers driving multiple potential success strategies.
Kill Gates: The second item is the concept of gates every few iterations that kill poorly performing projects. It is better to return those resources back to existing projects than to continue investing. You can think of the success criteria that drives the gates as a set of reasonable constraints. The team is allowed to do whatever it desires as long as it meets the outlined constraints that are measured at each gate.
Concept banks: The third is the concept bank where old ideas are stored for possible recycling. You never know when the remnants of an old idea will nourish a strong new project.
You may start out with dozens of projects, but in the end you’ll only launch a few into the market. The combination of trying lots of directions with very small low cost agile teams dramatically increases your opportunities to learn compared to either pure iterative or pure portfolio techniques.
You spend dramatically less than you would in a pure portfolio model since you don’t have the heavy cost of finishing and deploying the bulk of unsuccessful market explorations. All that waste can go back into either exploring new options or building out projects that show success.
You have a higher success rate than a purely iterative model since you are exploring multiple hills instead of climbing a single hill and hoping that it is the highest one around. This recognizes that not all successes are created equal and it can be easy to get stuck in rut serving a niche. By keeping your options open, you can always shift resources around if a truly great opportunity emerges.
One downside is that cost is a little higher than a pure iterative model. You need to devote a portion of your overall efforts to generating and exploring new opportunities. This is typically more than offset by the earlier access to successful products. Another downside is that some long term projects (such as say…a fusion reactor) are culled before they show their promise. This can be handled through careful tailoring of gates so that break through innovations are properly weighted.
The Harvest Process: Applying a stage-gate model to a single product A single product can benefit greatly from the ideas in the stage gate model. A single product is composed of a variety of separate features, scenarios and audiences. We can treat each chunk of customer value as a separate project and bundle of high value projects as a product. The basic model is the same as the stage gate process outlined above. However instead of spinning off multiple products, your individual teams tend to be focused on critical workflows or user scenarios for a single product.
Returning to the garden metaphor, as you nurture these areas of value, you should always have projects that are ripe for release. Publishing can be as simple as creating a public release that turns off all the experimental projects, leaving only your best features to show the public. I think of this as harvesting a ripe crop for your favorite customers.
As time passes, the little ideas that you seeded earlier will mature and produce a new crop of high value features. The process just keeps repeating, season after season, release after release.
The cliché in our business is that ‘ideas are cheap’. The shame of the matter is that some of the best ideas are left to rot because they don’t come from the right people and the production schedule has no room for creative thinking. The Harvest process builds innovation in at the lowest level possible. Everyone is encourage to come up with ideas and there is a clear and official process by which those ideas can become reality. Making it work In order to pull this off you need some basics in place. There is a substantial expense to running multiple projects and constantly integrating their efforts. With the right people, the right process and the right values in place, it is a very achievable goal.
Multiple, small agile teams: A single monolithic team has difficulty guiding a multitude of projects. Smaller teams can champion their area more easily.
Highly refactored and regression tested modular code: You need to be able to change one element without breaking the rest of the projects.
Innovation at the edges: Teams need to be able to innovate in interesting directions without constantly referring back to the ‘master planning committee’
Kill gates: If you don’t understand success and kill projects that don’t meet the bar, you’ll end up with a cancerous mass of bizarre, unhealthy features that have grown in random directions. Quality kill gates that actually kill projects are critical to success.
Shared design: You still need to present a unified face to the customer. Each team is under additional constraints to share visual styling and minimize conceptual complexity for the product as a whole. These constraints are built into the kill gates.
Customer feedback: At every stage, you need to be testing your products with real customers. You should be able to quickly deploy a working product to your customers every day. None of these ideas can succeed without people using working versions of your software and the team being wiling and able to learn from the customers’ reactions.
Conclusion In the end, any of the techniques outlined above have a chance of working. You need a driven group of people with the right circumstances in place. With the right amount of random luck and persistence, even a waterfall project can change the world. For some fat cats working in bloated large companies, this is enough. However, if failure has a personal cost, you’ll want to look into the alternative techniques such as agile, stage gate, or harvest. Over the long term you’ll notice less overall portfolio risk and your customers will start enjoying your superior, well informed product designs.
If you take away one thought today, it should be that learning is fundamental to software development. We do not live in a Newtonian world where huge mechanical brains can predict the future. Instead, we operate in a constantly shifting organic universe of cultural trends, customer urges, fluctuating cash flow and skeptical sponsors. In order to deliver great products on time, we need to learn from our environment and build a response rapidly and precisely. Portfolio technique increase learning by trying more options. Agile techniques increase learning by shortening the feedback cycle. Build both of these key concepts into your teams and you’ll find that they make smarter decisions and deliver more value to your customers.
Small initial teams? Try 20% of one person. With Harvest and Stage Gate, initial teams can be very small. For example, you could devote 20% of one person’s time for the initial concept stage. That doesn’t sound like much, but if you have every single person in the company devoting 20% of their time to new ideas, you’ll be seeding the equivalent of hundreds of new projects every year for no additional headcount. Google is the golden child of this concept right now, but 3M has been doing it successfully for decades.
Put those hundreds of ideas through a simple checklist that looks for potential customer value. Spin off the best ideas into tiny teams of a couple people and give them an iteration or two to come up with something cool (and testable.) Repeat the process several times and you rapidly gain some surprisingly innovative products .
Project Horseshoe Report: Building Innovative Games that Sell
The 2006 Project Horseshoe reports are out! In the place of my irregularly scheduled essay, I'd like to point you toward the collaborative report that I've been working on with our appropriately named "PlayDough." We dug deeply into the issues around funding innovative games and ended up producing 18 pages of revolutionary white paper goodness.
Click here to read the report. Now for a bit of commentary on the topic of innovation and business.
We are bad at designing profitable games
If you've read this site for some time, you'll notice that I keep returning to the themes of measuring success and rapid feedback cycles so that you can make necessary changes early. Unfortunately, it became very clear that most players in the game industry do not apply these lessons to the business of designing games. It isn't purely a publisher issue, nor is it purely a developer issue, but the overall issues are rather clear:
We don't know what makes a successful game.
We don't measure our games regularly from the customer perspective to see if we are going down a successful path.
We don't act on information that our projects will be market failures.
A Solution You might expect a good deal of venting at one of these designer retreats, but everyone was far more interested in fixing the problem. The game industry isn't the first to run into these issues and the solutions we found used in other industries are surprising palatable. Our group ended up exploring the adaptation of the Stage Gate new product development process to game develpment.
Stage Gate is a well known options-based portfolio management technique that focuses on improving product launch success rates. It is particular appealing to the game industry because it allows companies to try out lots of different ideas ranging from highly innovative project to product line extensions and then guide the most successful ones toward a market launch. Hundreds of companies in other industries use this model and it has been proven time and time again to dramatically improve success rates, reduce design and execution risk and also reduce time to market. These are all good things. :-)
Low hanging fruit My basic belief is that innovative products are a highly desirable and profitable component of any publisher's portfolio. However, in order to make this obvious to all the egos involved, you need a process that provides hard data and a way to mitigate risk/fear. The Stage Gate process does this with surprising efficiency.
It is also obvious that a competitive market eventually punishes stupidity. Ignoring new markets while releasing products with a poor chance of success only works until someone figures out that there are better ways. The first party story has some obvious suspects and I know at least one forward thinking 3rd party publisher has already started rolling a version of the stage gate process out across their teams. It will be fascinating to watch the techniques we talk about in the paper inevitably take hold over the next decade.
If you work in any organization that deals with multiple retail game titles, you are likely to save millions of dollars by taking the lessons of this report to heart. Perhaps that is a big claim, but one that is well justified by the historical results from other industries. The opportunity to become the savior hero in your organization is immense.
Kudos go out to everyone in the PlayDough group. We had the perspectives of independents, studio presidents, and publishers represented. Everyone is looking for solutions and everyone who worked on the report feels quite hopeful about the future. It is a wonderful thing to be able to point to business best practices that hold the promise of dramatically improving the industry as a whole.
I attended a unicorn convention in Texas this past weekend.
I always tell people that the role of 'game designer' is really a mirage left over from the days when there were one person teams and we needed a term to described the person who mashed together the messy creative gunk inside a game. Nowadays, we have specialists: Level designers, producers, creative directors, AI programmers, UI coders, writers, artists and more. Few companies seem to really hire the old definition of 'game designer'. Instead you are posthumously awarded the title when no one knows exactly what the heck you do any longer.
The Fatman and his wonderful wife managed to round up a herd of actual sparkling game designers, the impossible sort that think deeply about the big picture of how game systems work. The assembled brain trust was easily responsible for hundreds of games, many of which are legendary or at least ballsy beyond compare. For three days, they gathered at a remote Austin ranch and talked intensely about game design.
The goal: To solve the biggest problems facing game design over the next five years. I’ve cobbled together a few essays from the event that will eventually get posted on this website. More importantly, there will be no doubt stunning papers from the working groups posted later on down the road. Sparks were flying. And hay. Don’t forget the hay.
More than anything, it gave me faith that if you just get the brightest people of our industry off their isolated islands and give them a chance to talk, amazing ideas are inevitable. Experience shared is multiplied, not diminished.
Take care Danc.
PS: Despite all the magnificent displays of mental virility (complete with glowing horns and stamping hooves) there was a surprising lack of supple young virgins (male or female). I can only imagine that this will change once word gets out about the event.
I've been a game designer, pixel artist, painter, tools designer, product manager and marketing guy. I got my first job while in college working on a shooter called Tyrian at a little company called Epic Megagames. These days, I'm designing games deep in the forests of the North West.
I remain, to this day, not a chickadee plucker. Despite the rumors.