The AI of BioShock Infinite’s Elizabeth

BioShock is but one of many video games franchises invested in their rich and dynamic storytelling. The ability to put the player in scenarios unlike anything seen in the real world, but still relate to the perils faced by your character. In order to cement this ideal, many games require intimate relationships to be established between the player and other characters. But it’s often difficult to give a non-player character any sense of life or agency in a game that is both fixed and linear. The outcome is always pre-determined, but we want that experience to have value on a personal level. To tell that story in a way that is unique to us, but continue to have value in subsequent playthroughs.

This is but one of the many immenseley challenging areas that AI continues to face in video game development. To build intelligent and responsive non-player characters that interact and engage with the player: allowing us to invest in their relationships, to witness their response to the events around you. But how do you keep that from feeling overtly scripted, in an experience with a tightly constrained plot? How do you give life to a character who exists solely to further your adventure yet – given the existential framing of video game exposition – only expresses their agency when in the players line of sight? In this video we’re taking a look at one of the industries best efforts to-date: the character of Elizabeth Comstock from Irrational Games 2013 release, BioShock Infinite.

Companions

Companion characters have become increasingly more popular in AAA video games as the technology and resources have grown. Allowing for characters that respond to your performance, react to your decisions and engage on a more emotional level.

However it’s not easy giving these characters any sense of life or agency in a game where narrative is paramount: they’re ultimately slaves to the story and their job is to ensure you not only continue to participate within that narrative, but sometimes actually remind you what the story actually is! Even our most beloved of companions ranging from Half Life 2, to Dragon Age and The Last of Us fall short in many regards. This is often driven by three issues we see repeated in a number of games:

  • From a design perspective: companions are typically subordinate. They do what you tell them to do or simply do that which you’re already doing, though arguably not as well.
  • From a narrative perspective: companions are seldom important. They may become a conduit for expression of emotion but ultimately we either risk their lives frequently to establish stakes or just kill them off in order to make the player feel something.
  • From a technical perspective: companions are difficult to build well. A non-player character can have some base agency, but the quality bar for exceptional companion behaviour is incredibly high. We’re going to be with these characters for several hours of the core experience. They need to feel like interesting people who react to the world around them and prove valuable as an asset during the core gameplay loops. But also maintain this illusion effectively by being able to do some pretty basic stuff for human beings: like avoiding obstacles, navigate the world in a smooth and effective fashion and perhaps most importantly not freak the player out.

Elizabeth

Elizabeth arose from a desire within irrational to build an interesting and meaningful relationship between the player and the character. Previous character interactions by Irrational such as System Shock 2‘s Shodan and the likes of Andrew Ryan or Atlas in the original BioShock were constrained given they existed largely via radio communication and cut scenes. However, the one tangible and interesting relationship players could see in BioShock was between the Little Sisters and the Big Daddies: a sense of comraderie and support for one another that would even manifest systemically during gameplay.

Elizabeth is unique in many respects from other characters: one of the few deutranoginists in video games that carries any weight or substance. While this video and my channel is dedicated to talking about artificial intelligence, it would be disengenuous to not take a moment to address how multi-faceted her creation is from a technical perspective. Elizabeth came to life thanks to the ‘Liz Squad’: a team of designers, artists, animators and programmers whose job it was to ensure the central pillar of the entire BioShock: Infinite experience held true.
This required significant collaboration between character art team, the level designers and gameplay programmers to link many disparate elements together to ensure her decisions, movement, behavioural quirks and more held a consistent level of quality as we’ll see later.

In addition, all of this still required the character to feel like an emotive and realised human being, resulting in a need for performance capture to grab the emotion required to give it life. With Courtnee Draper and Heather Gordon acting as the voice and performance capture artists respectively. Given that the game wasn’t aiming for realistic performances, Heather’s motion capture was used as a baseplate from which the animation team built the final behaviours. In addition, Courtnee’s facial expressions were used from her recording sessions as a reference for completely hand-built animation. Despite this, her final facial features were taken from Russian cosplay artist Anna Moleva: whose own attempts at reproducing Elizabeth in the real world resulted in her being asked to become the official human face of the character not just in-game but in a variety of marketing material.

Elizabeth is built to interact with the world around her and to be able to express not just her ’emotions’ to the player throughout play, but ensure that the player actually sees this happening around them. This manifests largely through animation and talking to the player’s character Booker DeWitt. But just having these interactions isn’t enough: it needs to feal like these interactions are meaningful and to reinforce that they are partners in this journey, rather than treating Liz as a subordinate. Perhaps more importantly, they want you to actually enjoy being in her presence, not infuriated at her inability to interact with the world around her as expected.

And while there is plenty of AI behaviour that drives this concept, there are actually two specific areas outside of games that had a tremendous impact on the design and implementation of the character: theatre and football/soccer.

The Art of Theater

As part of the player-facing philosophy that drives much of AI non-player character development, it’s important that the player see and hear everything that characters do. Given it’s the only way we can infer what that character is meant to be thinking at that time. This is largely achieved through use of more theatrical behavior: exagerrated gestures and specific forms of movement and improvisation. We’ve seen this before in the likes of Far Cry 4 and Alien: Isolation in that a characters behaviour is overtly expressed and Elizabeth’s character and story helps reinforce this. She’s a young woman seeing the world for the first time and reacting to it as only someone new to it all can. This innocence and naievete – brought on by her imprisonment in the tower in Columbia – allows for Liz to react with joy, confusion or disgust to many of the things she finds around her and the story elements that play out. Naturally we see this in her physical responses but what is equally important is how she communicates thanks to Courtnee Draper’s voicework to enourage the player to observe her responses. She’s not telling you to watch her, but her infectious enthusiasm and exuberance draw players to her performance. What’s interesting is that this was an idea that – initially – wasn’t considered: with the earliest design meetings for Elizabeth questioning whether she should even have the ability to speak. The need for voice acting was reinforced by early prototypes indicating how disruptive the experience would be if Elizabeth repeatedly grabbed the player in an effort to get their attention both in passive and combat-driven sequences.

Despite this, it still requires the player to focus on finding Elizabeth rather than merely observing her during their exploration of the world. To minimise the need for the player to turn and face her, the Liz Squad adopted the principal of blocking: a process in theatre that dicates how a character positions and moves themselves within the world. However, the problem with that is in traditional theatre we have but one stage and the audience is always facing towards it. In BioShock: Infinite the entirety of Columbia is the theatre stage, so how do we ensure the audience is always facing the performance?

This drove the AI team to adopted the principle of ‘goal-side’: a defensive strategy in football that reinforces a need for players to stand between the ball and the goal whenever they lose possession. In this context however, the goal is always the next objective in the game and the player always has possession of the ball. This means that Liz aims to be goal-side at all times and will interact with items of interest that are nearby; meaning that the blocking principle is upheld throughout the gameplay experience: if she’s always trying to be standing between you and your goal, then the stage is always in the right place for you to observe the performance.

The area that Liz can interact within while remaining goal-side is referred to by the Liz Squad as the Golden Path. This path itself isn’t necessarily in the direction the player is facing, given you might not actually be looking in the direction of your objective, instead it concerns itself largely with the remaining area between player and objective. This ensures that Elizabeth’s interactions and behaviour are not only more likely to be observed, but also that the player will not become frustrated waiting for her to catch up because she’s too busy buying cotton candy or admiring a flower arrangement to participate in your experience – a notion that personally I find rather amusing.

Furthermore, the golden path itself however is not necessarily a direct line: given it would mean she could only interact with items on a very rigid path. In actuality the path is more of a cone shape. Where the radius of the cone is larger the farther away from the objective the player is standing. This gives Liz a lot more freedom to explore the world around her. But as the player gets closer to the objective, the radius of that cone tightens up. Meaning she’s more likely to simply keep pace with the player and only interact with objects just ahead of them as they move together towards the objective.

Despite all of this, Elizabeth does not adhere strictly to golden paths rules given this is a very difficult balancing act to achieve. Players are not the most predictable (or co-operative) participants in creating these kind of dramatic experiences. As such, Liz will periodically teleport between interaction points and wait to pause the interaction on the first frame of its animation or move to a useful position near the player provided it’s outside of the players range of vision.

Smart Terrain

So how do all of these interactions take place? This is achieved through the programmers and level designers working collaboratively on an element known as ‘Smart Terrain’. Smart Terrain are items that could be found sitting somewhere on the game-worlds navigation mesh that would trigger specific animation and audio cues for non-player characters. Some of these items can be used by a variety of NPCs to bring Columbia to life, but they are by default constructed to allow for Elizabeth to engage with the world around her. These ultimately boil down to highly contextual animations and audio that allows for Elizabeth to express a variety of emotions, such as get excited by cotton candy, be repulsed by cigarette smoke or be confused by vending machines. This also extends to small and passive behaviours, such as sitting on benches and leaning against a wall.

The smart terrain elements are hand placed by the level designers themselves and tested to ensure the desired outcomes are correct. Many smart terrain interactions have a limited number of executions, with some referred to as ‘Golden Moments‘: more poignant behaviours that – once triggered – Elizabeth will not repeat for the entirety of the campaign. In addition, some smart terrain will only execute based upon Liz’s current emotional state.

Emotions

These emotional aspects are actually modelled as part of the animation system too. So while she expresses herself through interactions, emotional states are also used to dictate the animations selected and blended throughout the game. This allows Elizabeth to be confused, excited, angry, upset and remorseful throughout the campaign and change how she interacts with smart terrain as well as Booker himself during the game. While they are labelled as emotions, it’d be more accurate to consider them as forms of reaction. Given the system actually stores Cotton Candy as an emotion. In addition, new emotion states and animations were added to Elizabeth to enable her to behave in a rather different capacity in the Burial at Sea DLC.

One final element to mention about Liz’s interactions is how she looks at the world around her. Many smart terrain items carry specific points on which she is meant to look at them. And she is able to look at multiple points in an effort to make her look more human-like as she observes objects from different points and angles. This extends to Booker himself, although she does not hold her gaze for long, otherwise it would make the behaviour look a lot less realistic.

Combat

So with this in mind, let us now consider how Elizabeth handles herself in combat. This is where many of the core tenets of the Liz system are broken, given that we don’t really want her in front of us during the combat sequences. The Liz Squad focussed on how to keep her part of the fight, whilst also being as far away from it as possible.

This starts at the navigation level – with the navigation mesh in the game world aiming to stop Liz from walking into areas where enemies are standing. In addition, there is also an effort to ensure she does not walk directly into the players line of fire. After all we don’t want her looking like she’s trying to get herself killed. At this path the blocking rules on the golden path are broken, given we want Elizabeth to stay safe an get to cover. However, the developers still wanted her to be part of the combat and be a useful asset during these more intense sequences. This led to the introduction of the Booker-Catch mechanic, where Liz throws useful objects to the player during combat. By holding down a button, the player will turn to face Liz who immediately throws weapons or health packs to the player.

This mechanic actually went through multiple iterations, with the original version focussed a lot more on Liz finding useful items for the player, as well as the need to turn and face her before she would initiate the throw. This largely operated within the confines of the existing systems, but felt dysfunctional given that players would have to stop fighting to look for Liz in order to get an item they needed. In time many of the non-combat rules were thrown out, to the point that in-combat, Liz teleports to locations that enable a clean throwing arc and will conjure items out of thin air that are deemed of relevance. These changes, alongside the instant turn and catch behaviour, resulted in the final mechanic acting a lot more like a glorified reload. But on the plus side, it creates these moments of systemic play where we run out of ammo just as enemies are approaching, only to be saved at the last minute by Elizabeth throwing a gun to you.

The emotion system is relevant in this context, given that a post-combat emotion is added to her behaviour to make Elizabeth seem unsettled after combat. This actually arose from testing given that Liz would originally revert to whatever pre-combat emotion state she was in. This would lead to some rather tone-deaf or awkward behaviour as she returned to being excited or giddy mere seconds after the player had violently killed a dozen enemies, which would be rather out of character,

Closing

Elizabeth is but the first-step in a much longer journey in creating dynamic and engaging non-player characters that can be interesting companions as we explore the worlds within our games. The solutions found by the Liz Squad address some really interesting problems on how to communicate to players and make them aware they are not the sole agent of change or narrative within the world they’re exploring. What’s interesting is that the AI systems at play here are not terribly complex nor do they invest in any nuanced decision making, instead she simply reacts to things that she is allowed to, but how that decision is made is rather novel. It reinforces the need to ensure NPCs are placed in front of the player in order for them to be actually noticed and the lengths that need to be take to ensure that it is consistent and of the highest quality.

Bibliography

Enjoying AI and Games? Please support me on Patreon!
Tommy Thompson Written by:

Tommy is the writer and producer of AI and Games. He's a senior lecturer in computer science and researcher in artificial intelligence with applications in video games. He's also an indie video game developer with Table Flip Games. Because y'know... fella's gotta keep himself busy.