Throughout the ‘AI & Games’ series, we typically place an emphasis on AI agents that control non-player characters (NPCs) within games. This is often the manner in which they are adopted, given that we need a level of intelligence to to be seen from our in-game characters. However, it is important to appreciate that agents do not need to be interpreted so literally. We as humans often associate AI applications with doing something physical – though that physical behaviour can takes place within a virtual space. This is typified by observing AI applications for robotics, where we see a complex behaviour being constructed using humanoid components. In the case of games, the AI code is attached to an avatar and we observe this intelligence through the behaviours it exhibits within the game world.
The reality is that agents can be utilised in a variety of ways. Arguably one of the more interesting challenges is resource management. AI systems are adopted for problems such as power station management and logistics, where it is not part of the day-to-day physical activity, but instead is a component of a larger and more complex system.
As is evident by the title of this article, we are going to discuss the notion of ‘director’ systems in video games. In these instances, we are not discussing the AI system that controls each non-player character, but directs how the game unfolds for the player.
The Agent as the Director
Director systems are agents in exactly the same way as any other agent within games: they adhere to the simple principles of being intelligent and rational. This is achieved by assessing the current state of the world, notably the progress and performance of the player and then making changes to the world that fit within the context of play.
In some respects the challenge of the director is to keep the game in-tune with the player: given we want the experience maintain the players interest and challenge. It’s a tough problem to get right, since should the director not be following the player effectively, then the player may either:
- Lose their interest and quit the game due to a lack of challenge.
- Become increasingly frustrated at the continued difficulty and quit the game!
Left 4 Dead
Left4Dead is a four-player co-operative game developed by Turtle Rock Studios in conjunction with Valve Corporation. The game is a first-person shooter where players are tasked with escaping a zombie outbreak somewhere in Pensylvania. The game is split into several campaigns, each containing a handful of levels where the task is to ensure your team make it from the start to goal location. A key element of the gameplay is that each player is expected to watch each others backs: providing covering fire, saving them being pinned down by zombies or providing health supplies to keep one another alive.
Note: Turtle Rock at the time of development were in fact owned by Valve. Since then they have moved on and developed Evolve, something of a spiritual successor to Left4Dead.
Progress through the environment is impeded primarily through the zombie horde. Zombies can appear sluggish and not pay a great amount of attention to you. However they can easily build up speed and attack rather ferociously. This issue is compounded when groups of zombies attack the player in large numbers (10 maybe 20 zombies). In addition to the regular zombies that can be found throughout each map, there are a number of ‘special infected’ that can attack and restrict the player in different ways.
- Hunter: a vicious zombie that lunges at the player. If the lunge connects, then the player is pinned down as the hunter continues its attack. Once pinned, a players can only escape if another player kills the hunter.
- Smoker: can drag a player into a corner by using its tongue. Once it has dragged the player it can then attack them while they are tied up. Once again another player must intervene to save the trapped character.
- Boomer: an overweight and bloated zombie that vomits a bile that attracts zombie hordes at any players that are coated with it. In addition, players must strike from a distance, given that a boomer explodes and splashes bile within its proximity upon death.
In addition, there are also special infected introduced in the sequel Left4Dead 2:
- Jockey: a dwarf zombie that jumps upon a players shoulders and drags them in random directions. Players cannot regain controller until another character kills the Jockey.
- Spitter: spits acid on the ground that can injure nearby players.
- Charger: uses a club-like arm to attack players. If a player is hit, the charger will push them until it reaches an obstacle and will pummel the player until killed by another character.
Finally, there are two special infected that could be considered ‘bosses’, given their tremendous attack power.
- The Tank: a large, brute-like infected whose punches can fatally injure players. The Tank is fast, can take a significant amount of damage and can throw boulders as projectiles. A tank can easily eliminate an entire team of players if they are not careful.
- The Witch: a deadly infected type that appears in the form of a young woman, albeit now with red eyes, sharp teeth and talon-like claws. The witch is unique in that it can be avoided altogether, given that it is initially passive to the players movements. However, should players move too close to the Witch or shine a light directly at her for too long, she will become startled and attack the player that is responsible for awakening her. If she is successful at attacking the instigating player, they are immediately downed and require revival by another player.
Each of these zombie characters, be it the horde or the special infected appear throughout the game at different times. As we will now discuss, the director is largely reliant upon placing these characters in the game at runtime by assessing player performance.
The AI Director
The director found within both Left4Dead games is focussed on maintaining pace: having an intensity that peaks and troughs throughout play. This intensity fluctuates both in periodicity and strength, meaning that the frequency and intensity of zombie attacks can change throughout a given level. The director is designed to keep the players on edge throughout play: ensuring not only that co-operative gameplay is maintained but also that players continue to progress towards the end goal.
The core functionality described above is achieved by ensuring that players only receive enough room to breathe when it is deemed necessary. This provides opportunities to reload weapons, change weapon load-outs or heal wounds either upon your character or someone else in the team. Attacks from zombies will tail off and provide some respite. The subsequent ferocity of future attacks is then dictated by the pace at which players continue to proceed through the level. If players are not progressing fast enough, then the intensity of future attacks may be increased to ensure players proceed. In addition, players are expected to stay together as a group and the director will focus attacks on isolated players if they deviate from the team.
In order to maintain the pace of gameplay, there has to be some method by which to record it. This can naturally be a rather hard thing to judge mathematically, given we often perceive it qualitatively as a by-product of our gameplay experience.
Left4Dead closely monitors the performance of each player in order to determine their ‘skill’. There are many factors that could be employed to determine a players skill value: their overall health, zombie kills, damage ratio (received vs delivered), health packs used, shot accuracy, ammunition consumption rate and many more. These factors can help determine just how skillful a player actually is.
Conversely, Valve also use a similar approach to determine how stressed a player is during a bout of play. This stress level helps determine the level of intensity that the player is currently facing. This stress level is based on whether the player is being attacked by zombies. The more zombies that are attacking, the more the intensity increases. In conjunction, the more the player kills zombies within close proximity, the more their stress level increases. This allows for players to snipe zombies without any impact on their stress level. Finally, if a character is incapacitated by a special infected, such as a charger, smoker or hunter, then there stress level reaches its maximum value.
Left4Dead models both the skill and stress levels of each player with the aim to determine:
- When the next major attack should occur.
- Where it should attack from.
- How it should attack.
And perhaps most interestingly…
- Who it should attack.
While this all sounds very aggressive, the system also uses this information to know when to back off and let the game run its course. The director wants to ensure that while the game is challenging, it is not perceived as unfair. This is achieved by maintaining spawn limits on all types of zombies that can appear on the map. Later in this article we will explain what these limitations are and how they fluctuate during gameplay.
Knowing when to attack is driven by the way in which the director manages pace. The game oscillates through periods of gameplay that are built around three key phases:
- The Build-Up: The game begins to gradually increase the intensity of play. At first, the game only has a couple of wandering zombies spawn in. As time passes, more mobs of zombies appear and special infected begin to pop-up more frequently and attacking survivors. While this is happening, the director assesses the stress levels of the players. As their stress increases, the director gradually increases the intensity of enemy spawns.
- The Peak: Once all players have attained maximum stress levels, the director acknowledges that the games intensity (and subsequently the pace) has reached a peak. It is on the verge of boiling over and needs to calm the game down. As such, no more wanderers, mobs or special infected will be placed into the map.
- Relax: Once the players have wiped out the infected, the game allows for a period of respite: allowing the team to heal and share resources. Typically the rest period will last 30-45 seconds. However, it can be interrupted if players start moving again too quickly.
There are two exceptions to the directors behaviour. In some instances, the game will turn the director off entirely. This occurs when the game wishes to focus on a narrative sequence. In addition, it may be to allow the game to focus on a crescendo. The crescendo events or finales allow the director to be a lot more punishing, as many of the spawning limitations are relaxed: allowing the system to throw a significantly larger number of zombies at the player.
While the primary threat comes from the variety of special infected that attack throughout a level. The director keeps the player busy by adding basic zombies into the level. Each level starts blank, with no enemies added until the director begins to populate the world. The world is then populated by standard zombies, either as individual wanderers or as mobs.
Wanderers are individual zombies that appear in the map in front of the players. Wanderers are typically spawned along the main path that takes you to the exit. However, some will be placed off the main path, in order to keep players on-edge as they wander off the beaten path. Interestingly, some will even spawn in locations that are not reachable by the player. While this adds nothing to gameplay, it maintains the illusion that the zombie infestation is pervasive and that there are no ‘safe’ areas in the local environment.
Large groups of zombies that attack the player at once. These mobs spawn either behind or to the side of players, ensuring their subsequent attack comes as a surprise. Depending on the perceived skill level and difficulty setting, mobs can be anywhere from 10 to 30 zombies in size and have a spawn delay between 1 and 4 minutes. Of course in more intense difficulty settings, this can result in a significant number of zombies appearing at once. As a result, the director will only spawn zombies provided there are less than 30 in the scene. Meaning that sometimes we have more than 30 in the map, but no further can be added until this value has dropped below the ‘zombie cap’.
The special zombies are more constrained, given that they are more powerful and present a greater threat to the team. Unlike the common infected, there are significantly less permitted on the map at any one time. Typically only three special infected are allowed to be on the screen at the same time. In addition, only one of each type can appear in the map at any given time. This ensures that there is an element of balance in gameplay, since certain characters such as the Smoker and Charger can pull or shove individual players away from the group.
The respawn timer for special infected is typically 45 seconds and will be inserted provided there is space. When space is available the respawn location is based within proximity of survivors. Ensuring that the special infected will focus on a particular player.
Unlike all other zombie types, the boss infected – the Tank and the Witch – will only spawn in certain locations that have been predefined by Valve’s level designers. These ‘hotspots’ appear typically on the main path of the game. Every time the a new instance of a map is created, the director will pick a handful of these locations to be used for Tank and Witch spawns.
When a Tank is spawned, no other special infected can enter the game and have to wait until the player defeats them. The game will set the Tank on the players once line-of-sight has been established. In fact, the director will force the Tank on the players regardless. This is enforced given that it is possible for players to avoid a Tank on the map.
Perhaps more interestingly, the spawning of boss zombies is constrained significantly to only occur once or twice in one map. However, the constraint on Tank spawns can be lifted during crescendos: resulting in multiple Tanks appearing in the final moments of play.
Managing the Scene
Outside of the larger points of how the director manages spawns, it is also responsible for one or two other interesting elements of play which we quickly summarise here.
As difficulty levels increase, the spawn rates are designed to increase as well. As such, while the zombies do not get stronger, the periods of respite are smaller as individual zombies pop up much faster. In addition, the rate at which mobs respawn is increased, meaning that the stress/intensity level is peaking and falling a lot more frequently.
Sometimes the director will forcibly remove wanderer zombies from the map if they are preventing it from spawning a mob. As such, you can sometimes predict a mob attack given the number of wanderers has reduced virtually to zero.
Items are spawned in using a collection of pre-determined locations. While the director is mostly focussed on ensuring similar items do not spawn too close together, it will periodically change pain pills into med kits if your team is not performing well.
Left4Dead 2 experiments with the notion that the maps can be dynamic. This only occurs on certain maps and the instances are very specific. If the system thinks that players are performing too well, it will attempt to interrupt them by changing the map slightly.
Left4Dead has its own audio manager that works in conjunction with the director. Each level has its own unique sounds and music that are indicative of the build-up, peak and relax phases of the director. If you pay attention, you can actually figure out when an attack is imminent.
What we have seen throughout this piece is that Left4Dead presented something novel: a game that tailors the experience based upon what is happening in real-time. This is achieved by modelling the players experience and understanding how much of a challenge that the player faces. This notion of managing pace helps maintain the interest and longevity of the series, given that no two playthroughs are the same. This is undoubtedly LeftDead‘s biggest strength: in taking the time and effort to understand how people are performing, the game uses this information to ‘punish’ the player accordingly. This dynamic experience allows for a lot of emergent gameplay and unique experiences, to the point that every game tells an exciting and often funny story to relay to friends. The director merely sets the scene, it’s how you perform that makes it all the more interesting.