The AI of Horizon Zero Dawn
A number of complex systems are required to bring 'the machines' to life.
AI and Games is a YouTube series made possible thanks to crowdfunding on Patreon as well as right here with paid subscriptions on Substack.
Support the show to have your name in video credits, contribute to future episode topics, watch content in early access and receive exclusive supporters-only content
If you'd like to work with us on your own games projects, please check out our consulting services provided at AI and Games.
Horizon Zero Dawn stands as one of the most critically acclaimed of Sony’s roster of Playstation 4 exclusives. As the hunter Aloy, players venture across the post-apocalyptic landscapes of the future to uncover the mysteries of her past and how the world fell in the years before. Humanities fall from grace has led to the rise of ‘the machines’ – robots of varying shapes and sizes that that now run free across the lands. These robotic animals are intelligent, coordinated and lethal: requiring quick thinking to survive and even greater preparation and planning to successfully destroy.
In episode, we’re exploring the gameplay and decision making systems that bring these creatures to life and power the core gameplay experience. We’ll be looking at not only how they make decisions to scavenge, to hunt and to survive but how they coordinate themselves in the wider world either individually or as part of a herds. Plus I explore how the game creates an ecosystem of machines that is balanced and managed across the many territories and regions of the new world.
Warning: Story spoilers for Horizon Zero Dawn.
The Ecology of Horizon Zero Dawn
The robotic wildlife of Horizon Zero Dawn dominate the landscape and hold control of territory throughout this open-world experience. Each machine carries a unique aesthetic and structure that reflects their function and purpose within the narrative of the game. As players work their way through the campaign, we come to learn of the Faro Plague – an army of rogue war-machines that rendered the world asunder – alongside the race of machines constructed by the AI known as GAIA that are designed to restore the planet to its natural glory and are largely inspired by long-extinct species.
As a result, the function of a given machine often holds the secrets to their weakness, players can use their Focus to scan local machines to ascertain not only their type, but vulnerabilities as well as weak points. In addition to the weak spots, a machine’s function within the narrative ultimately influences their in-game behaviour, this includes:
Acquisition and scavenger robots such as the Strider and Grazer. These worker class machines are typically found in open grass lands working the lands and while they are relatively docile and often flee confrontation they are nonetheless deadly when provoked.
Recon machines such as Watchers and Longlegs that scout the region for threats and alert nearby machines such that they can fight or flee.
Transport machines such as the Behemoth and Shell-Walker that gather up resources from the acquisition robots making them valuable targets. They are slower, more cumbersome and typically defended by more nimble units.
And lastly combat machines such as Sawtooths and Stalkers that are primarily there to attack the player and other nearby threats.
There are 28 unique machines that can be found roaming the wild lands, each with their own distinct passive and aggressive behaviours and players need to learn how best to approach them. While the greatest threats on the path to Meridian – such as the Stormbird and Thunderjaw – roam the wilds on their own, players can still run afoul of less lethal machines that operate in herds. It’s common to see the smaller robots such as Watchers, Striders, Scrappers and Lancehorns co-existing: supporting one another as they achieve their base functions. Over time these herds increase in size and diversity as players continue to explore the world with the likes of Shell-Walker convoys, packs of Tramplers and Sawtooths protecting nearby Grazers and Lancehorns.
In order to pull this off, the game requires numerous AI systems that dictate the behaviour of each robot, how they behave within a herd, the collective intelligence of the herds themselves but also balance the herds across the open world for a more varied experience. The developers behind Horizon Zero Dawn, Guerilla Games, came to this project lacking many of the AI tools required in order to bring this experience to life.
Having previously worked on their Killzone franchise, their toolchain was largely focussed to accommodate first person shooter and none of the design challenges that faced them for a large open-world experience. As such, during the course of development they built newAI and gameplay systems and expanded existing ones within the studio to bring this experience to life and over the next two blog posts I’m going to explore the following:
An agent hierarchy system that dictates how each machine behaves both individually, but also as part of a pack or herd (Beij 2017).
The expansion of the existing planning and execution system originally built for Killzone that allows each machine to make decisions (Berteling 2018).
A set of sensor systems that enables the machines to quickly and effectively detect threats and investigate when necessary (Beij 2017)
A dynamic and intelligent navigation system that provides interesting movement patterns for both ground robots that respects the world geometry and varying machine size (Beij 2017, Berteling 2018)
A completely separate navigation system that catered for air-based machines such as Glinthawks and Stormbirds (Josemans, 2017).
An animation system that respects each machines composition and ensures they move and attack in an intelligent way (Berteling 2018).
Last but not least, a system known as The Collective that manages every single machine that is active in the game world and how they are distributed across herds (Beij 2017)
So let’s begin by exploring how each AI character is built to make decisions and how the game manages herd behaviour.
Agent Hierarchy
Each machine is represented in the world as a separate non-player-character AI called an ‘agent’, but perhaps more interestingly, so is each herd. Horizon Zero Dawn adopts a hierarchy whereby a machine can exist entirely on its own as an agent or as a child of a group agent – which is used to store information about herds.
A group agent doesn’t physically exist in the game world, but it’s responsible for coordinating all agents within it. Each individual agent has its own set of sensors that enables it to hear noises closeby or spot threats in proximity. But in addition, a group agent has a blackboard system that stores in-game information a machine might want to use. This not only allows the game to keep performance costs down by storing information each individual might try to calculate on its own – like safe areas to stand in or active patrol routes – but more importantly, agents in a group can share information between each other. Hence when a recon machine spots the player, each machine can make a decision on how to respond to that information. Hence you might see acquisition robots in the herd such as grazers opt to run away while nearby combat machines move in for the kill.
Now the actual AI system used in Horizon Zero Dawn is called a hierarchical task network planner. Unlike the planning systems used in games such as F.E.A.R. that planned one action after the other, HTN planning generates plans comprised of action macros. Each macro contains multiple actions in a set sequence. This is ideal for game development given designers can put together macros of good behaviour they expect to see in-game. I’ve previously explored how HTN planning was used in Transformers: Fall of Cybertron, but it’s worth noting that Guerilla have been used HTN planning since Killzone 2 and as such this wasn’t built overnight, but has gradually matured over years of projects within the studio.
The HTN planner is used in two capacities by machines: individual agents can ask for a plan of action to resolve a specific task, such as finding new areas to visit, attacking a human, or fleeing to safety. Meanwhile group agents can run plans that dictate what goals individuals agents should be assigned, allows groups to actively pass information between one another within a hierarchy and potentially form new subgroups or destroy existing ones. In short, each individual machine is still largely responsible for their own behaviour and addressing the goals assigned to them and uses the planner to make decisions. Meanwhile, the groups keep individuals coordinated, re-group them based upon what’s happening in the world and – perhaps critically – assign roles to each machine such that they know what their job is to support the herd.
The Collective
Now with all of these machines being spawned into the world either individually or in groups, how does the game know what machines to spawn in the open world and where? There are designated sites across the game map where we can expect specific machines to appear either individually for machines like Thunderjaws and Stormbirds or in flocks for the likes of Striders, Grazers and Glinthawks. However, the rest of the game-world is full of herds comprised of different machines. So how does the game know when spawning in a specific machine where to place it in the world and what groups to assign it to based on the needs of the worlds design?
This is achieved courtesy of a system known as ‘The Collective’. The collective is the supergroup: all machine groups and individual machines exist in the collective. It manages the spawning of all machines into the world, keeps track of whether they’re in a group and handles their movement between groups and manages the machine ecosystem, such that performance isn’t impacted overall.
Individual machines in the world can request to join an existing open-world group if they’re on their own. This is useful feature given many machines are left isolated after the rest of the previous group either fled or have been destroyed. So the collective can recycle isolated machines to join other groups. To do this, each machine has a passport! Their passport stores facts about that machine, such as its level and machine type and the collective uses this to determine whether it will fit the requirements of another group, only moving across if it satisfies the groups needs.
Herd Example
Now let’s explore how herds operate and how group agents execute as players interact with them. Herds are typically placed into the world in a relaxed state where they’re not under threat and simply hanging out in a specific region of the map. Any given herd is typically a mixture of recon, acquisition and combat machines, with acquisition robots in the centre, whilst combat and recon patrol the outside to defend them.
When a herd is placed into the world, a collection of groups within that hierarchy request a role. This role assigns all machines it’s controlling with behaviours that reflect their purpose. This can include patrolling for threats, searching nearby disturbances, attacking enemies or simply scavenging resources in the world. These roles can shift as time passes and events force the herd to change its behaviour and structure. There is typically a limit on how many machines can fill a particular role within a herd such that their make-up isn’t unbalanced.
A machines role dictates the goals that the an individual agent will seek to complete. Hence with a relaxed herd, the acquisition machines will use the HTN planner and data stored within the group to find a local area to mine. Meanwhile recon and combat machines will generate patrol paths to follow. Patrol paths are auto-generated based on the local geometry, avoiding awkward terrain and – more importantly – avoids passing through long grass and other vegetation the player might use to stealthily approach the herd. However, in order to give players a fighting chance, many patrol paths will deliberately pass close to stealth vegetation such that you can place traps or attempt to hack them more easily.
But of course it doesn’t always stay this way; as players start to attack machines their behaviour will shift accordingly. For machines that aren’t part of herds, this comes down to the specific machine class. Only a few will actually run away when provoke directly, with the vast majority of machines attacking the player, with even acquisition machines often attacking the player if they’re on their own.
However, should the player attack a herd, the groups within hierarchy react in kind, but only if the herd has been alerted. As mentioned earlier, the group agent enables data to be shared between each machine, but it doesn’t update on every tick. As a result, the herds don’t have a hive intelligence and it is possible for the player to kill a machine without alerting the rest of that group. However, should this be spotted or if the machine under attack survives, it will notify the group that they’re under attack.
At this point, the hierarchy within the herd shifts, groups of machines within the herd request a new role and priority for execution of the corresponding behaviour. The group comprised of acquisition machines assume the flee role and run away together, while the groups comprised of combat and recon machines re-balance themselves with their roles prioritising each character that is attacking the herd. While in many cases this is just the player on their own, this enables the system to scale for story missions where the player is part of a group or for other dynamic events in the world where other human NPCs are attacking herds.
While at this point things have gone horribly wrong, it still gives players plenty of chances to turn the situation around, as both the grouping systems and AI combat systems work to create an experience that is both fun and challenging. This is achieved courtesy of an AI combat system that attempts to structure combat such that it isn’t chaotic or broken.
The combat groups that formed have different roles and priorities that are balanced out, hence the larger machines in a herd will be spread across the groups where possible, with smaller machines lined up to support them. In addition, groups select machines to attack the player based not just on whether the HTN planner thinks an it can execute a specific behaviour, but also whether that attack is going to be interesting. If all machines simply attacked the player at once, then you would die a horrible death pretty quickly. As such the combat systems use an approach similar to that found in Halo 3 where it balances which enemy attacks you based on how interesting that attack will be in the context of combat.
This requires an action selection utility function that is calculates how interesting it would be for that machine to attack. This is based on the current machines state, whether the player is aware of that machine in proximity, how close it is to the player and the amount of damage it has received and dished out. This becomes a precondition of the HTN planning system, and aims not only to provide some challenge, but also to inject some variety into the combat.
Perhaps more critically, when the HTN planner is asked to generate actions for attack groups, while an attack is being selected the other machines in the group are also being given movement behaviours to circle the player and wait their turn. This is done deliberately to create opportunities for the player to exploit, either attacking a passive machine or countering the one that just attacked you. In time you’ll slowly be able to wear them down, largely because the combat system is deliberately leaving itself open to attack.
Now that we have explored this complex agent hierarchy system where each machine can make decisions about how to behave using the HTN planner, we're going to look more closely at the systems that each individual machine can utilise as part of their core behaviour. This includes the likes of sensors systems, navigation for both land and air machines and, how execution of AI behaviours is heavily tied into the animation systems to give each machine an unsettling yet realistic behaviour.
Sensors and Movement
So first up, let's take a look at the sensors used by machines to trigger AI responses as well as how animation ties into the overall execution of their behaviour. There are a significant number of unique sensors that a machine can use: visual sensors such as the Watchers eye, to radar and proximity sensors on Longlegs, aural sensors that hear explosions at distance to stone throws close by as well as the ability to sense players colliding with them directly. Each machine has a collection of these sensors calibrated with their own sensitivity values, making it easier to sneak up on say a Watcher or Grazer, but a lot more challenging to catch a Stalker unawares.
Now a traditional sensor system might just enable for events to occur around an AI and they either 'see'/'hear' it or nothing, but the sensor systems are actually a lot more nuanced. This is achieved through information packets that are attached to objects that can cause stimulus in one of the machines sensors: this includes the player, other NPCs, rocks, fired arrows, other machines and wildlife. This data tells the receiver - the machine that sensed something - information about what it is that it detected and it's state. Hence machines as well as human NPCs can tell the difference between a dead body lying in front of it and an arrow that whizzed past their head and missed them, but it also helps ensure that things such as the player hiding in long grass or behind trees cannot be seen, given they recognise they're not visible in that state.
Each AI character - machine and human alike - can handle and interpret sensory data in a different way, so certain information might be ignored by some characters whereas others react to it promptly. In fact, depending on the strength of a sensor in a given machine, it can actually reduce the data from the sensory event that they can read. This helps manage the emergent properties of the game such each character type responds to information in their own unique way.
Now should an AI make a decision based on this sensory data, we've still got to make sure it looks realistic. So when a machine has decided to make any action, such as moving to a new location, investigating a disturbance or attacking the player, there is still the issue of animating it during execution such that it looks as realistic as possible. The animation of these machines is a big challenge, given that they need to be able to look like their animal inspirations, but also have a distinct machine-like behaviour at times. This requires both the navigation and combat systems to pay attention to the distance the machine is going to traverse and animation toolchain that adjusts root bones of animations and warp them to suit based on the perceived distance and time it needs.
This ensures that regardless of how far the machine is moving and how fast it is doing so, it can start the animation, move into the main part of the behaviour and then blend it correctly at the right time. This is important for things such as running to points, where the machine not only needs to slow down and stop at the right point, but it's even more relevant in combat. Many of the animations used for attacks have two distinct sequences: there's the wind-up, which telegraphs the attack, followed by the big finish where the damage is dealt. Horizon Zero Dawn uses a similar method to that discussed in my case study on the AI of DOOM, where the the system controls the current locomotion of the machine, blends movement or attack animations to suit at specific points and then ensure that the machine lands or stops in the right at the finish.
Navigation
Now there is still one big issue I haven't yet talked about and that is navigation. Ensuring these machines can wander around the environment is a real challenge, given this large variety of unique enemy types are all different sizes and shapes. So they need to be able to move through terrain in a way that makes sense for them. But also, they need to be able to recognise changes in local geometry and adapt to that or simply ignore it depending on their type. Now this requires a commonly used AI tool known as a navigation mesh. A nav mesh stores information about how a given character can move across the map based on what are perceived to be obstacles in the world. While you can calculate it at runtime, they're often built or baked before the game is released and loaded into memory when necessary.
Given Horizon Zero Dawn has such a large map and only specific segments of it are relevant at a given point in time - since the AI are only active and moving around if they're near you - the navigation mesh is built at runtime but only around the immediate region of the player. But thing is, there isn't just one nav mesh, there's six of them! Four of them cater for character movement based on the size of the object: small, medium, large and extra-large. Hence humans can move around on the small mesh alongside watchers while the likes of the Thunderjaw has a nav mesh pretty much all to itself. Plus the two extra nav meshes: one for swimming machines such as the Snapmaw as well as a unique mesh that ensures machines stand in good locations should the player be trying to mount them.
In each case, obstacles can block or alter a navigation meshes structure and the system recomputes changes in real-time such that moving obstacles (and even other machines) can impact the ability to move around the space. What's interesting about this is that obstacles have differing properties and can either prove to be completely impassable or simply undesirable to walk across - but much of that is dependant on the state of the machine's AI behaviour. As mentioned in part 1, machine patrols actively avoid stealth vegetation when generated but when investigating local disturbances, while grass is still considered undesirable, it will walk through it if necessary. This same principle actually applies to small rocks and trees: these are impassable obstacles, except for larger machines such as Behemoths, Rockbreakers and Thunderjaws. These beasts can smash rocks apart and uproot trees, but only if they're in an angered state or giving chase to the player. Outside of that behaviour, they'll treat them just like any other obstacle.
Moving Through the Air
While this navigation toolchain caters to land-based machines of all shapes and sizes, it doesn't work at all for those that are based in the air. Non-player characters that move through the air not only have to be aware of the nearby obstacles such that they don't crash into trees or cliff rock, but they also need to be wary of the elevation of nearby geometry. The world of Horizon Zero Dawn is full of rolling hills, forests, rock outcrops and steep mountain climbs. For the two flying machine types: the Glinthawk and the Stormbird, they need to know how to navigate the air such that they can take off, fly a patrol route, land and also swoop down and attack the player when necessary. To achieve this, the game not only has the nav mesh system on land but a completely separate navigation system in the air.
This proved to be a challenge for the AI team on the game. The technique used is known as hierarchical path planning over Mipmaps. Mipmapping is a technique used in computer graphics that aims to minimise the memory overhead of a texture or images by providing a collection of the same image gradually lowering in resolution. It's ideal for managing level of detail in games so that objects hundreds of metres away can be visible but use less texture memory than those directly in front of the player where you need them to be the highest quality possible. This approach was considered given that when a machine is flying a path around the world, it doesn't need to know with complete accuracy the local geometry of where it will be a minute from now, but it really needs to know the lay of the land immediately around it should it decide to land. The path planning system for aerial machines uses Mipmapping for the height map of local geometry - a data structure that tells us the elevation of a given x/y position of the world - with four levels as they move down from top to bottom become increasingly more complicated and realistic. Level 3 is the simple and abstract model, while map level 0 is a pretty accurate height map of the world.
Much like the navigation mesh, the mipmaps are built at runtime when needed, given a machine doesn't need to know the entire worlds elevation data when flying within a fixed region. When they need to fly to a location, the flying navigation AI starts by using the A* search algorithm over the highest level of mipmap, hence it calculates the simplest version of its flight path against a rough version of the geometry. The A* makes flying up and over obstacles more expensive than flying around them, hence you'll see machines glide around mountains and cliff tops more often than fly over them. Each time it calls the A* algorithm it only has a fixed number of iterations, so once the path is completed on the simplest mipmap (known as level 3) it will then take a given segment of the path it calculated and then refine it by borrowing down to mipmap levels 1 and 0 to make it more realistic and respect the geometry more accurately. Plus it then smooths the paths out such that it removes steep slopes or sharp turns and makes them more realistic. This system work really well given that any flying machine that is in the air always has a flight plan - even when it's rubbish - and then it can boil it down to something more practical by repeatedly calling the search algorithm to refine the path to become more and more natural. It's also quite memory efficient, but it does have one caveat, in that given it's based on the maximum height of a given region of the map, they cannot fly under bridges or rock outcrops, but much of the time as a player you won't really notice it.
Between the land-based nav mesh and the air-based mipmaps, flying machines can then coordinate how to attack, land, dive attack and even crash in a way that respects the geometry. Machines that are hovering above the player while attacking are still using the pre-calculated flight plan, only it's not necessarily moving directly forward along that path and plays the corresponding animation. The velocity of the machine is tied to whether it's flying, gliding or hovering and as such they can circle you in the air in a realistic fashion all the while still using the same navigation tool.
Take offs and landings use a separate system that talks between the flight navigation and ground-based navigation mesh: it search for a valid positions on the nav mesh it can land on, typically points that are slightly higher off the ground than the local average and then adjusts angles and velocity accordingly. Once it has landed, it's now using the corresponding nav mesh based upon the machines size. This same principle actually applies to when they crash too, except this time the only valid landing positions are based on the machines current heading and while it might look less graceful, it's fundamentally using the same tools. The specifically programmed edge case for this is the Stormbird's dive attack. Stormbird's will circle the player, then come crashing down to towards you crashing into the ground at your current position. It's using the same systems, but in a much more dramatic fashion. However one added caveat is when circling the player, the Stormbird will often wait until it blocks out the sun before making the attack. You might have noticed this when playing the game yourself and it is intentional. During testing of the Stormbird AI, the QA team noticed that it would periodically block the sun based on where you were standing and this made the attack all the more disorienting as the light shifted and blinded you during the dive. At that time it was purely accidental, but afterwards the AI team went out there way to ensure it does it more frequently and deliberately.
Closing
Horizon Zero Dawn delivers an experience unlike anything seen before, as a world teeming with mechanical life plays host to the tales of Aloy and the mysteries of her past. The AI and gameplay systems of the machines are critical in building this apocalyptic future for players to explore.
As we've seen over throughout, this was a tremendous effort by a team of around 10 people through several years of development. Delivering AI systems at this scale that works well in massive-open world games is only becoming increasingly more difficult as games continue to increase in scale hence it's vital for game development community that these good practices are shared with the wider world such that we can learn from one another.
Plus it's fun to learn about how games work and to appreciate the efforts of those who worked so hard to bring you giant mechanical Tyrannosaurs you can fight while riding on robotic horseback. I mean how cool is that?
References
Julian Berteling, 2018. “Beyond Killzone: Creating New AI Systems for Horizon Zero Dawn”, GDC 2018.
Arjen Beij, 2017. “The AI of Horizon Zero Dawn”, Game AI North 2017.
Wouter Josemans, 2017. “Putting the AI back into Air: Navigating the Air Space of Horizon Zero Dawn”, Game AI North 2017