Analysing Player Habits in Battlefield 3
Performance Modelling in EA's Popular Shooter
AI and Games is made possible thanks to crowdfunding on Patreon as well as right here with paid subscriptions on Substack.
Support the show to have your name in video credits, contribute to future episode topics, watch content in early access and receive exclusive supporters-only content
If you'd like to work with us on your own games projects, please check out our consulting services provided at AI and Games. To sponsor our content, please visit our dedicated sponsorship page.
Note: This episode was originally written in 2015 and published on our old website.
In modern game development, it’s common for AI techniques to be deployed in ways that are not solely for gameplay. We can gain immense value from understanding how players operate within a game by gathering data about their in-game performance and run analysis upon it.
We can then use this information in a number of ways: such as creating AI systems that are driven by this knowledge or to learn something previously unknown about the problem domain that we are working within. This leads into this articles focus on player modelling in games, looking at research in Battlefield 3 and how personality is reflected in player performance.
Player Modelling: An Overview
Now it’s important to acknowledge at this early stage that the term player modelling is very broad. In time, the term began to mean different things to different people within the games-research community, to the point that a research paper (Smith et al., 2011b) was published in an effort to recognise each strand of player modelling research. We take this opportunity to give a brief overview of what this paper discussed. However, this is by no means a complete analysis and we would advise those interested to read the technical report that this paper is based upon (Smith et al., 2011a).
The taxonomy established in (Smith et al., 2011) came about because even among the authors, there was no consensus on what player modelling actually meant. The authors broke down how and why player modelling is conducted in games and identified four particular facets that were relevant:
Domain: Are we modelling the actions in the game, or the players reactions to in-game activity?
Purpose: Player modelling often aims either to replace the player in some fashion, or to generate information (linguistic descriptions) of the game itself.
Scope: The process can focus on one, a subset or all players of a given game.
Source: How is the player modelling achieved with respect to the data accrued? It’s important to acknowledge that this can distinguish between models that capture directly from recorded data to achieve/replicate a human effect, versus those where more abstract knowledge is hoping to be gained that is not coupled directly to in-game sequences .
The real focus of this work was to allow for player modelling researchers to have a means by which to communicate: if one researcher has a given player model and the other has their own, how do they compare or contrast to one another? We can conduct player modelling with the aim to completely replicate a human behaviour in a game using the same inputs, to modelling how players actually conduct themselves within a game with respect to its design – which opens up the realm of game analytics. In short, this is a massive research area that is only now being explored fully.
Player Modelling in Battlefield 3
The focus of this piece is a research project conducted largely by Shoshannah Tekofsky at Tilburg University in the Netherlands. The project was interested not so much in modelling player behaviour for AI to replicate, but rather to use games as means to quantify personality assessment results. Based upon our introduction to player modelling given above, we typically expect that modelling tasks conducted in games is focussed upon learning something with respect to what players did either in game or out. What made this project rather rare was that the analysis was conducted to compare in-game activity against a players personality. As described in (Tekofsky et al., 2013a), the aim of the project to answer the following:
“Does the statistically trackable play style of a player significantly correlate to his personality?”
In other words, does your personality relate to how you may behave when placed within a video game? To answer this question, Tekofsky drew upon a rather unlikely source: DICE’s first person shooter Battlefield 3. It’s not exactly a game that many would associate with expressing our personality within, given it is a gritty, aggressive and realistic military shooter. However, Battlefield is a feature-rich game that allows players to conduct a range of useful interactions beyond combat. Furthermore, there is a tremendous amount of information that can be mined about a given players behaviour which reflects their performance over time. Numerous websites such as BF3Stats and DICE’s own BattleLog service allow players to analyse a tremendous amount of data which quantifies their performance in-game. This data goes beyond basic win and kill statistics, it also expresses how these players interact with the game in terms of their selected equipment, vehicle usage, gameplay modes and co-operative teamplay. It is this data, combined with the use of a standardised personality assessment, that drew some rather interesting conclusions.
Battlefield – An Overview
For the sake of completeness and to aid those who may not be familiar with the Battlefield series, we provide a (very) brief overview of the games mechanics and design.
The Objective
The Battlefield series is an objective-based first person shooter (FPS) where teams of players are placed in a map with the aim of completing a given task. Unlike it’s popular cousin, Call of Duty, the series is focussed on capturing and defending objectives, with far less emphasis on the numbers of kills scored on each team.
While Battlefield does provide the standard Team Deathmatch game type, it’s two most popular modes are rather different given kills have little impact on the end result should players not be focussing on the objective. Each game type is focussed on the use of ‘tickets’, which are the number of lives a given team can use to complete their objective. In Team Deathmatch, the task is simply to deplete the tickets of the enemy team first. However, it becomes more complicated in other game modes:
Conquest: Key strategic points are established in the map and teams must capture and hold them. Once a team is controlling the majority (>50%) of control points, then the other team will start to ‘bleed’ tickets in addition to those lost in combat.
Rush: Each ‘wave’ of rush is focussed upon two MCOM control points: the attacking team aims to destroy them before their respawn tickets run out. Meanwhile the defending team – who have an infinite supply of tickets – must ensure the attackers do not destroy these objectives. Should the attacking team destroy the two MCOMs, the defending team will fall back to another position that once again has two control points. Meanwhile the attacking team receives a fresh supply of respawn tickets. Once the defending team has been forced to fall back several times (the number varies per map), then the attacking team is declared winner.
Classes & Squads
While each player in the game is given at least two guns (one primary and a sidearm), the range of available weapons combined with additional resources and perks are consolidated into individual classes . These classes embody particular types of gameplay: enabling use of specific mechanics or equipment that, when used effectively, can ensure success. Below we summarise each class that exists in Battlefield 3.
Assault: Armed with an assault rifle, this class is allowed to carry health packs, with the aim to heal teammates in combat. In addition, they have the option to carry a defibrillator that can revive dead team-mates within a short period after their death. This effectively prevents a spawn ticket being lost by the team.
Engineer: Responsible for maintaining the health of vehicles by using the blowtorch tool. In addition to their standard carbine weapon, they can carry anti-vehicular weaponry such as anti-tank mines, stinger missiles and rocket launchers.
Support: A machine-gun-wielding class that also carries ammunition packs to resupply teammates. Often carries C4 explosives for added damage to vehicles or mortars to attack ground troops from a distance.
Recon: A sniper class that often participates in combat from a distance. Recon can also deploy local-range motion trackers and micro air vehicles (MAV’s) to spot distant targets and assist squads as they attack buildings.
In addition, groups of up to four players can deploy into squads, which receive extra score by completing objectives together rather than individually. Squads allow for players to spawn into the game near a fellow squad mate. In addition, one member of the squad can act as the leader and identify which objective the squad should be attacking. Ideally, a squad should have a fair distribution of the soldier classes such that everyone has a particular role to play.
While playing Battlefield, each class has a variety of options the player can unlock by continuing to be successful in their gameplay.
Maps are typically large in scale, to the point that it would often take a player anywhere from 5-10 minutes to navigate across it. This often results in the inclusion of vehicles for players to use. These include dirtbikes, jeeps, armoured trucks, tanks, helicopters and attack jets; each with their own range of utilities and weapons that you need to unlock over time.
Player Score
Outside of the progress towards completing objectives to win the match, players can also earn score which translates into experience points for unlocking new equipment. The simplest way to gain experience points in Battlefield is to kill an opposing player. However, there are a multitude of other means by which a player can accrue these such as:
Vehicles: Disabling, destroying and repairing vehicles.
Objectives: Capturing locations, destroying MCOMs, defending captured locations from enemy attack.
Medical: Healing team-mates, reviving team-mates upon death.
Support in Combat: Providing ammunition, spotting targets.
The PsyOps Project
The idea of using video games as means to achieve a personality assessment was not new at the time of this project, with some existing research already conducted that use video game data combined with a personality assessment. While these had proven successful, like many academic research projects they were of very limited scope with a small number of participants. Despite the confidence in the results, it’s best to have a much larger sample size such that real statistical significance can be established. As such, Tekofsky’s ambitions were aimed much higher and sought to set about player modelling in Battlefield 3 by establishing the PsyOps project: an internet campaign whereby players of the game could sign up to become part of the PsyOps project, and allow for the researchers to pull down their statistical data, combined with the completion of a personality assessment.
This was achieved largely due to the time and effort spent with the Battlefield community to convince them this was a worthwhile venture. Given that at the time, Tekofsky was a YouTube content creator in the Battlefield community, this eased the transition but made the work no less challenging. A full campaign, complete with a website, Facebook page and Youtube campaign, aimed at convincing players to participate in an effort to help be a part of a larger scientific project. In addition players would receive some feedback about their individual personality assessments as part of the project, while also being kept up to date on the results accrued form the larger dataset. The PsyOps campaign proved highly successful: with over 13,000 Battlefield 3 players contributing to the project.
The Study
For a period of six weeks, participants would sign up to the PsyOps project. During this process, the participant would give permission for the anonymous use of their gameplay data, in addition to providing some basic information about themselves and their Battlefield activities.
Player name/gamer tag
Age
The gaming platform they participated via (PC, PS3, Xbox 360)
Country of residence
Whether they wished to receive credit for their participation in the study
Finally, each participant completed a personality questionnaire, based on the International Personality Item Pool (IPIP) collection. IPIP is an effort by researchers to provide a public domain repository for measurements and scales that can be used in personality assessments. These measures can be used to identify particular traits in a participant, such as emotional detachment, cheerfulness, expressiveness and humility. The IPIP collection is fairly vast, with 302 different scales that associate with one of 230 different personality traits. The personality questionnaire contained 100 different items from the IPIP pool. As noted, the player also gives permission to access the players in-game statistics. The collection of statistics that are available for each player is extensive and you can learn a fair amount about a player’s performance and habits by looking at this data.
This data can be used to to describe the main dimensions of human personality, known as the ‘Big Five’: Openness, Conscientiousness, Extroversion, Agreeableness and Neuroticism. Existing research has shown that these dimensions can be identified through the likes of questionnaires and observations.
In-Game Data
The data was filtered with respect to a number of factors, firstly only the first 9368 samples could actually be used in the study. This was due to the format of the data changing on the statistics websites. This was caused by the dataset adapting to changes in gameplay caused by a new DLC expansion to Battlefield 3. In addition, data had to be excluded in the event that the participant did not wish for their data to be used in the study or due to age (the study focusses on participants aged between 12 and 65). Furthermore, a number of sub-samples were established by partitioning based on gaming platform and whether the participant was a native English speaker.
If we return to the original question: does the statistically trackable play style of a player significantly correlate to his personality? The answer is yes. The analysis revealed the three key correlational themes of the study:
Conscientiousness Vs. Speed: The strongest correlation found within the dataset, established that more conscientious players have a slower play style. It was noted that each player has much lower number of actions per time unit. While this might sound negative, being more conscientious does not necessarily mean people are poorer players. What is established is that while more conscientious players score less frequently, they also die less frequently, prefer to play longer game modes such as Conquest and are also more liable to use tanks as their vehicle of choice.
Conscientious and Extraversive Players vs. Unlock Score: Conscientiousness and extraversion are negatively correlated to the unlock score: the process of acquiring new weapons, attachments and perks. Meaning that those who are slower in pace and also arguably less selfish, take longer to unlock new components in the game. This is to be expected, given that the pace of unlocks is attached to the kills achieved using weapons or vehicles.
Worth Ethic Vs. Performance: Players who often state arguably ‘negative’ qualities in the IPIP questionnaire relating to work ethic – “I shirk my duties” etc. – interestingly does not subsequently correspond with a poorer performance in the game. This is interesting given that Battlefield is reliant upon players working well with others in squads or even towards playing objectives.
There was also further evidence, though not significant statistical correlation, that suggested players performance does not correlate to their total play time. As such, work ethic proved more relevant to their performance and this seldom changed the longer they actually played the game. In other words, unless the player actually made a significant effort to change their attitudes, their performance in the game did not improve over time.
As part of the original analysis, an effort was made to establish whether a players age also impacts any aspect of game behaviour and it seems that age actually correlates more strongly with play style than personality. It was determined that players become more conscientious as they age and score slower as they age and which fits with existing research in personality and performance with the increase of age. The triad of correlations between play style, personality and age is shown below.
Older Players Invest in Battlefield: Older players continue to play Battlefield for longer than the youth. Interestingly, further analysis of this correlation shows that older players do actually improve in scoring and killing the longer the play despite the correlations found in point 1.
Age Impacts Vehicle Selection: Perhaps as a combination of the fidelity of control, or the shift in priorities, older players prefer using tanks over helicopters or jets.
Age Impacts Class Selection: Older players have a tendency to lean towards Support and Engineer classes, with younger players preferring Assault and Recon. The rationale for that is open to discussion. Arguably Recon is favoured by younger players given they are not as focussed on objective play and instead prefer the gratification of long distance kills. Support and Engineer are largely supportive roles for players who use vehicles. As such, it could be argued that many older players would defer to the Engineer given the use of tanks as vehicle of choice and Support because they can damage vehicles using C4. It is interesting to note that the Assault class is not picked given it is a very supportive role in healing and reviving other players.
The PsyOps project concluded with a final study published in (Tekosfky et al., 2015), where the previous findings related to age where then observed over time. In this instance, the emphasis was on establishing whether a players performance has some correlation with the neurocognitive effects of aging. While there is certainly significant evidence found within medical literature of this impact, studies in gaming that identify a correlation with in-game performance are rare. This final publication identified that there is a correlation between in-game performance and ageing: with player performance gradually degrading as you age. In addition, it is recognised that player performance typically peaks at the age of 20. So even if you buy the game as soon as your able (which would be 17 in the US given it is a M rating), you would only have a couple of years in your ‘prime’ before things go downhill.
Closing
Given these two studies, a number of interesting facts were established in terms of how people play Battlefield 3 with respect to the personality and ages of players. There is much to gain from modelling the behaviour of players in video games. The real impact of this can be explored as part of creating bots to play games, or learn whether games are balanced accordingly - an idea that has since became more popular long after this study concluded.
Though sometimes, you just need to know whether you can trust the (virtual) soldier beside you and will help you win the game. The PsyOps project gives us some insight how player modelling can indicate how personality and age impact our performance in a video game, even Battlefield!
References
Smith, A. M., Lewis, C., Hullett, K., Smith, G., & Sullivan, A. 2011(a). An inclusive taxonomy of player modeling. University of California, Santa Cruz, Tech. Rep. UCSC-SOE-11-13.
Smith, A. M., Lewis, C., Hullet, K., Smith, G. & Sullivan, A., 2011(b). An inclusive view of player modeling. In Proceedings of the 6th International Conference on Foundations of Digital Games(pp. 301-303). ACM.
Tekofsky, S., Spronck, P., Plaat, Aske, Van den Herik, J. and Broersen, J, 2013(a). PsyOps: Personality Assessment Through Gaming Behaviour. Proceedings of the Foundations of Digital Games conference, FDG 2013.
Tekofsky, S., Spronck, P., Plaat, Aske, Van den Herik, J. and Broersen, J, 2013(b). Play Style: Showing Your Age. Proceedings of the IEEE Conference on Computational Intelligence in Games (IEEE CIG) 2013.
Tekofsky, S, Spronck, P., Goudbeek, M., Plaat, A. and Van den Herik. J., 2015 Past Our Prime: A Study of Age & Play Style Development in Battlefield 3. IEEE Transactions on Computational and Artificial Intelligence in Games (IEEE TCIAIG) 7(1) 2015.