The Case Against Data Driven Design
Matt posted in The Industry on September 14th, 2006
Data driven design is a hot topic among the industry. But what does it mean? I couldn’t find a good definition of it on the web, so I’ll give you my definition. Data driven design is where most or all the elements of design are out of the code and exposed to the designers. The main difference in my mind is the division of labor. Data driven design has programmers doing less and leaving it up to designers/artists to implement. Having programmed gameplay for games both with and without data driven systems, I would say that data driven design is actually a worse way to create games even though many people feel that it’s the “right” way. In looking for a good definition, I stumbled on other blog posts about data driven design. On Kyle Wilson’s Blog, he says that the game engine shouldn’t contain a single line of game code and should be completely data driven. On Dan Cook’s blog, I found an interesting post about data driven development. I’m not sure he has the same definition of data driven design as I do, but he asserts that it’s a good way to keep down costs, but you can’t have any real gameplay innovations because that would be too risky. I would argue that using data driven design, you could innovate gameplay by giving the designers more control over the behaviors of in game characters possibly by using a scripting language. But, let me explain why I think data driven design is a bad idea.
Problems with Data Driven Design
Data driven design is about exposing as many aspects of development to the designers (and to some extent, the artists) to lessen the burden on programmers and to reduce the back and forth collaboration between programmers and the rest of the team. Control can be exposed in many ways. It can be exposed through scripting languages. Common choices for scripting languages are Lua, Ruby, Python or a custom language. There are several problems with scripting languages. They can be slow since they are ordinarily interpreted. Custom hooks need to go into the engine to expose the functionality that should be scriptable. If the scripting language is compiled, a compiler has to be implemented, which isn’t that difficult for most software developers, and I’m sure there are many open source projects available for common scripting languages, but some form of a debugger needs to be implemented, which is going to be a lot more difficult to find if you are doing console development because it’s on a unique architecture and needs to be debugged remotely. If you don’t have a debugger, debugging assembly generated by a compiler is going to be very difficult, and there’s always a lot of gameplay bugs, so this slows down development considerably. So, I imagine most game developers that use scripting languages choose to interpret them. Interpreting solves some of these problems, but debugging an interpreted scripting language is also tricky if you don’t have a debugger for it. You can step through the interpretation engine, but it’s tricky to see what is going on since you are seeing the execution of the interpreter rather than the actual code. Some interpreters that are optimized are very difficult to debug. The worst problem is when there’s a bug in your compiler or interpreter.
On top of this, designers are coding. Designers usually don’t have a background in computer science, and although scripting languages are pretty easy, they aren’t trivial to learn, nor will a good designer necessarily be good at them. Imagine a situation where you hire someone and find out he is great at level designing, but can’t write scripts to save his life. Do you lay him off? Do you force one of your other level designers to do scripting? Maybe they aren’t very good at it either. Now you have someone who is very good at “design” at your studio, but is terrible at scripting, so it may be hard to find a position for them. Oftentimes, people who don’t have a lot of experience programming can make a lot of mistakes. Sure they’ll get better with more experience, but they might not be cut out for coding. On top of this, they usually aren’t equipped with either the tools or the know how to fix their own bugs, so the regular programmers are forced to do it, and the bugs can be a real pain for programmers to find as I mentioned in the previous paragraph.
Many studios don’t give the designers scripting languages. They create an editor that is either using plug ins for a 3D modeling package or create their own stand alone application. The designers use this to create links between widgets that determine the behavior of the objects in game. They may have some widgets to do logical operations, counters, conditionals, etc in order to design complex behavior. The widgets themselves may have a varying amount of parameters that the designers can control. I think this is still a bad idea. For one thing, using a lot of widgets to essentially form a scripting language doesn’t work out too well either. Let’s say when the player slays the Fearless Three and a Half Headed Sea Dragon of Krandell you want to award him the Dongle of Incontinency Prevention if he/she hasn’t already bought it at the store. Otherwise, you want to award the Armulet of Ambivalence. Although it sounds easy, this can end up being very complicated. When the dragon sends itself the destroyed event, designers may send the player an event querying whether or not the player has the dongle. They could pass another widget as a parameter, so the player would send the false event to that widget. That widget can have a link to send the dongle a give to player event when it gets the false event. That same widget can have a link to send the armulet the give to player event when it gets the true event. Now, what happens if the player uses a temp save option and turns off the console? We’d want to store that the player has the dongle. Now what happens if the player turns the game on dies before hitting the checkpoint? The player will have to fight the dragon again, so they shouldn’t have the dongle. How would a designer express this with links and game widgets? A problem such as inventory would most likely be solved in code with an inventory system, but regardless of how many systems are in place, there are always situations where it is tricky for a designer to handle all the special cases that arise. Because of this, bugs can occur, and it is difficult for either a designer or a programmer to solve it. Designers don’t normally have the tools so they can see the logic that they have created in the game, and it’s difficult for programmers to follow a complicated web of links in order to track down where it’s actually going wrong.
Even though many game studios create widgets that designers can control, on short development titles, the widgets usually require quite a bit of tuning and tweaking. As a game widget evolves and improves over the development cycle, it will have new or modified parameters. If designers are setting this up, the parameters must be exposed to them, so some sort of documentation needs to pass the information along. Maintaining a wiki or some other documentation about how things work can be tricky if you don’t have a full time technical writer on staff. Programmers aren’t really known for their writing skills, so your descriptions in the wiki may not be clear. The worst thing that can happen is not when a designer doesn’t understand it, but rather when a designer thinks he/she understands it and builds off those false assumptions. This can turn out poorly because programmers will be spending their time documenting (and not be very happy about it), and designers may not even get the information that they are trying to communicate.
But Sony Santa Monica Did It!
Now some readers may have read gamasutra’s excellent coverage on Tim Moss’s talk about the development of God of War. It describes the very data driven approach that the engineers took on God of War. You may be thinking that God of War was a very successful game, and they used data driven design, so why shouldn’t we? What people may forget when they look at God of War was that it took 3 years to develop. Even though it had very few programmers, it had a very large development team for a PS2 game. I’ve heard rumors that there was a lot of turnover on the project because it was very difficult to work on. Maybe data driven design ends up being more efficient when you are developing a game for that amount of time, but few titles are afforded the luxury of a three or more year development cycle. In my experience, companies want to get a game out every year assuming they already have technology for the platform (which Sony of Santa Monica did since they released Kinetica, but they may have started from scratch). Obviously, the transition to new consoles (and new technology) is always difficult and time consuming, but God of War didn’t have to overcome that hurtle.
If Not Data Driven Design, then What?
If you have 3 years to make a game, then maybe data driven design is a good way to develop your title. If you don’t, the people who are going to be the best at coding your gameplay logic will be a programmer. Minigames, AI, player control, and any other things that change from game to game are best left in the capable hands of programmers. The people who are best at designing effects for real time rendering are also programmers. Unless you have a very technically savvy artist, a creative programmer will be able to create an effect that looks as good and gets better run-time performance, which is crucial to have a good looking game. Now, I’m not saying that artists can’t design particle effects, but usually the actual implementation should be left up to a programmer unless you have a technical artist. You also need to hire talented gameplay programmers that have a real feel for design. Ideally, the gameplay programmers are given a rough sketch of the design then free reign to create something great from it, and feedback after they are finished. Because the programmers themselves are able to tune and tweak either the effects or gameplay, it is much faster than the constant back and forth between the departments. Rarely will programmers be twiddling their thumbs waiting for someone to get out of a meeting before they know if something is the right direction or not. Gameplay programmers are empowered to pick the direction that they feel is right, and go down it. If it doesn’t work out, then the designer and programmer can work out a new direction to try. Gameplay is very difficult to specify. No designer is good enough to write a spec that is both detailed enough to include everything and is guaranteed to be fun. There are always a ton of unknowns in design – many decisions can only be made when the game is playable. In fact, that’s why it’s often bad to overspecify design at the beginning. At iterative approach is usually much more successful. Oftentimes, designers try to write a document with all the details upfront only to find out that there are some core aspects that need changing. It’s much better to start with a general idea, see what’s fun and then refine the design from there. If a company has talented gameplay programmers, they can help flesh out and improve the design with very little feedback from designers. This saves time for both the designers and gameplay programmers, which helps speed up the development cycle.
As team sizes increase, the more communication dependencies become a problem with development times. Fred Brooks’ seminal work on software development, The Mythical Man-Month, discusses the negative ramifications communication can have on a project’s development time. If the person in charge of implementing it (in this game the gameplay programmer) is the driving force behind making that feature fun, then iteration time is greatly reduced because there is little communication overhead. This is somewhat similar to the surgical team that Harlan Mills proposed for software development, except with even less overhead. The gameplay programmer is the surgeon – the driving force for implementing the feature. The designers are more like the hospital directors – they give feedback and direct the surgeon AFTER the surgeon completes the surgery. This means the gameplay programmer has free reign to make the feature fun, and only shows it to the design department for feedback after he/she is satisfied with the results. Too often when gameplay programmers take liberties with the design, designers can feel threatened. Game development should be collaborative and with talented gameplay programmers, they will know how to tune and tweak a game.
Although I talked at great length about the problems with data driven design, there are also problems with this development methodology. However, I think these problems are far outweighed by the benefit of having things programmer driven rather than data driven. Good gameplay programmers are hard to find especially now with team sizes increasing because of next gen. I agree, but I would argue that good designers are hard to find also. A studio that has less designers and more good gameplay programmers will have a higher productivity for the reasons I specified above. Another problem might be that programmers typically cost more than a designer with similar experience. That’s also true, but if it speeds up development, then it’s probably worth it. Slower development is expensive especially if you miss your release date like EA did with the Superman Returns game. Instead of coming out along side the movie, the game will be released at the same time as the DVD. I can’t guess exactly how much that cost EA, but I have a feeling they would have made a lot more money if the game were out alongside the movie – not to mention that the game would have been cheaper to make because it would have been in development for less time.
To sum it all up, designers should do just that: design. Let them design, and let the programmers implement. It takes programmers with a good gameplay sense, and yes, they may be hard to find, but the results are that you can develop a game faster and probably less expensively too. If you have a very long development cycle creating a lot of tools and making it data driven may pay off, but most games are on a one to two year development cycle, so I think in most cases, data driven design is a bad approach to game development.