AI and Games Summer School Round-Up | AI and Games Newsletter 26/06/24
Recapping a week of presentations from experts across academic and industry
The AI and Games Newsletter brings concise and informative discussion on artificial intelligence for video games each and every week. Plus summarising all of our content released across various channels, from YouTube videos to episodes of our podcast ‘Branching Factor’ and in-person events.
You can subscribe to and support AI and Games right here on Substack, with weekly editions appearing in your inbox. The newsletter is also shared with our audience on LinkedIn. If you’d like to sponsor this newsletter, and get your name in front of over 3000 weekly readers, please visit our sponsorship enquiries page.
Hello and welcome to this weeks AI and Games Newsletter. Greetings from Schloss Dagstuhl: Leibniz Center for Informatics, a research venue in the rural corners of south-west Germany. This week I am participating in a seminar on Computational Creativity for Game Development. An event that I am grateful to be extended an invitation to, where a collection of experts in AI for games across academia and industry discuss current and future challenges, and how they may be addressed. I hope to be able to report back on this event right here in the newsletter in the coming weeks.
But in the meantime, this weeks newsletter has me report back on the event I was at last week: the 2024 AI and Games Summer School. An event that despite its name is not owned or coordinated by me and AI and Games, but it is hosted by long-time friends and collaborators within the space, and invites speakers from across academia and industry to share their experiences with a collection of students of various skill levels and backgrounds.
So let’s jump through our weekly announcements, and then we discuss the summer school in more detail!
Weekly Announcements
Last week I was invited onto the
podcast by to talk about the state of generative AI in the video games industry. You can catch our conversation over on the XR AI Spotify feed.
I appeared in a panel for the GamesIndustry.biz Sprint event yesterday, in which I discuss the use of AI for automation in game development, alongside Sean Cooper, Aleena Chia, Lauren Maslen and Lucie Migne. Our conversation is now live on the dedicated GI Sprint site and via the video below!
Anyone attending Develop:Brighton and looking to arrange a meeting to discuss life, the universe, and everything - or more importantly potential collaborations and sponsoring my upcoming Goal State project - can find me on the Meet to Match site, as well as on my Calendly page.
The 2024 AI and Games Summer School
So I’m writing this fresh from my attendance at the 6th AI and Games Summer School, hosted at the University of Malta. This event is designed to be a series of presentations and talks from academic and industry partners to help bring new people interested in AI for games into the space. It is heavily attended by students, predominantly in latter stages of undergraduate and postgraduate, but also with young industry professionals as well.
The event is hosted by Professor Georgios Yannakakis of the University of Malta, and Professor Julian Togelius from New York University, with Antonios Liapis and David Melhart (University of Malta) also part of the organising team. Plus in 2024 the event is sponsored both by modl.ai and Keywords Studios, with Stephen Peacock acting as Keywords’ representative. The summer school originated from the textbook that the team originally wrote and published back in 2018 (with an updated 2nd edition on the way) , which aimed to give a high-level overview of the variety of approaches that have emerged from academia in AI for games. Full disclosure, Georgios and Julian are old industry friends of mine, who I have known since I was joined the games/AI research community while in grad school starting in 2006. So I attended as an invited speaker, a panel moderator and all-round professional nuisance.
The event runs in three parts: two days of academic and research-focussed talks, a day of industry presentations (which I contributed to) and lastly two days of game jam that run across the Thursday and Friday. So it allows for people to learn a lot about the space, what’s emerging in research corners, the practical work in commercial games, and also go and make something themselves within the timeframe.
Some of you may recall that I attended this event last year when it ran at Microsoft Research in Cambridge. The format largely stays the same across years, but the talk content evolves, the industry speakers change, and naturally the attendees change as well. For more on that experience, be sure to check the vlog I released last year on AI and Games Plus. There will be a similar video later this summer once I actually get back to my office!
As stated already, this year it is running in Malta, which for those unfamiliar is a (very) small island nation in the south Mediterranean, right between the island of Sicily (Italy), and Libya and Tunisia in North Africa. It’s an interesting country for a number of reasons: a former British territory that also has huge influences from its Mediterranean brethren. With this being my first visit it’s been a wonderful place to spend a few quiet days prior to the main event. Certainly more fun than Cambridge (but then I live in England, so y’know, I’m biased).
Plus, it’s worth mentioning that this year the event is sponsored by Keywords Studios, with their head of Game AI Stephen Peacock playing a big part in the events organisation.
The Academic Overview
The first two days are predominantly focussed on more academic works, largely with Togelius and Yannakakis recounting key parts of the relevant ‘Artificial Intelligence for Games’ textbook. This includes aspects of AI algorithms that learn to play games - including the likes of evolutionary algorithms, reinforcement learning, deep learning and more - as well as a deep dive into AI that models aspects of gameplay, and how you can use it in a variety of approaches. The closing part of the textbook overview, is with procedural generation algorithms and how they can be used to build content such as characters, stories, levels and more.
While many of these elements are fairly consistent from one year of the summer school to the next, the field itself is continuing to evolve and it’s important the school reflects that. Hence some aspects of the material gets updated to reflect current trends, most notably how use of things like large language models can in fact be used to generate levels for games. It’s not an easy process to adopt, but it is in fact attainable!
The Invited Speakers
But of course, it’s not just Togelius and Yannakakis talking, we get a collection of speakers from across academia and later industry providing their own insights.
Ahmed Khalifa, my colleague on the GDC AI Summit advisory board, provided an overview on popular techniques for what are known as ‘constructive’ content generation. This is largely what most indie developers use for content generation such as Binary Space Partitioning like in Caves of Qud, Cellular Automata in Vagabond, Stochastic Agent methods used in Nuclear Throne, and template methods like Spelunky.
We had David Renaudie, a lead data scientist at Massive Entertainment - the Ubisoft studio behind Tom Clancy’s The Division, Avatar: Frontiers of Pandora, and the upcoming Star Wars: Outlaws - talk about their work in data science at the studio. It was a really insightful talk, as David highlighted how AI was used to train models that are reflective of expected performance of the current in-game state on specific hardware profiles (e.g. Xbox Series S/X, PlayStation 5). This was actually of benefit to end-users (i.e. the game developers) working with the tool as they built specific games and were able to immediately see in engine a rough estimate of the potential GPU costs of the current game state on different platforms without having to deploy the build to the dedicated hardware.
Yuqian Sun, who not only previously appeared in the panel I moderated at the London Dev Conference (see the article below), but was also a student at the 2023 school, came to discuss their work in building 1001 Nights: a game built upon text generation language models, and text-to-image models where players tell stories as mechanisms to generate weapons to help them survive a fight against an adversary. You can find it on Steam.
Maxim Gumin, largely known for their work in developing the wave function collapse algorithm, gave an overview of their current work in which they’re building algorithms that can design entire 3D scenes, complete with sets of objects. The relies on the use of large language models, whereby a user can give text input on the type of scene they’d like to see designed. The output provides not just a list of objects placed within the scene, but also the constraints - and even Python code - that can be imposed upon those objects such that everything appears smoothly in the final asset.
Shyam Shinde from Helpshift, a Keywords Studio took to the stage to talk about LLM applications for conversational bots. A lot of chatbots are required to help with first-phase customer support, and historically it has relied on expert systems and types of information search.
Melanie Lopez Malet of Ubisoft came to talk about their experiences in applying Natural Language Process (NLP) to video game productions. Crucially nowadays NLP is largely dominated by generative AI and LLMs, but this hasn’t made applying this knowledge in game development any easier. Malet was one of the team behind Ubisoft’s NEO project which I talked about earlier this year, given I was invited by the team to try out the demo at GDC 2024.
Perhaps the most fun part of this presentation, was the focus on asking why AI should be used? And provided we come up with smart, sensible, and empathetic perspectives on why, then we can start to explore how: how should it be used in ways that make sense for developers and their goals? This is highly relevant given the current situation we exist within, as generative AI is often applied in situations that are unsafe, impractical, unethical and immoral. Undoubtedly one of my favourite presentations of the week.
Sicong Li of Arm attended to give a presentation on the use of LLMs, but critically about running them on mobile. After all models like GPT are simply too big to host on individual devices, and hence we rely on internet connections. But is it possible to have engaging and practical deployments of language models on mobile? Be it trying to run smaller LLMs like Llama 2 on the device (which they demonstrated at GDC 2024), or on the edge.
Wes Kerr from Riot Games shared the studios work on using imitation learning in tactical FPS games by using the data collected from sensors attached to player’s avatars when playing the game. This was a collaboration between Riot and Modl.ai where they used machine learning to train bots that could potentially be used for the live service shooter Valorant. However, given that Valorant is updated fortnightly and has a number of technical restraints upon it, the project sought to build the bots in a way that isn’t disruptive of the overall production of the game. Hence the team worked on a 2 vs 2 version of Unreal’s Lyra FPS demo that was modded to be a ‘Valorant-like’. Allowing for the team to build bots in a game with a similar style to Riot’s shooter. This idea has not bee deployed into production (so no, it’s not in Valorant), but is indicative of how they may start to approach these challenges in future.
Anna Winterstein, who is a game designer at Lively, a Keywords Studio, presented their perspective on working on Project AVA. You may recall a few weeks back I provided a summary of AVA: a game developed as an R&D project internally within Keywords where the goal was to try and build as much as possible using generative AI tools.
One thing Anna brought to this conversation was the value that was found in using GPT as a design tool. Rather than having it be precise, be intrictate and provide detialed information, but use it as a mechanism to generate lots of ideas, and the pick, choose, and refine them. And later using the Azure API to plug in pre-prompt information and examples to ease the process of providing relevant information. Critically, I enjoyed the closing comments explaining what game designers actually need from AI tools, rather than what has been presented. And it largely fell under four key aspects: for generative AI to be trustworthy and accurate, to be consistent in understanding and delivering information (plus consistent prompting methods), diverse corpora so that it’s less cliché-laden, and finally ethical such that it is transparent, accountable, and developed with clearly defined fields of application.
Oh and of course there was my work! I was the opening speaker on the Wednesday morning giving an overview of the history of AI for games, and then moderated a panel in the afternoon as well!
Wrapping Up
It was a fantastic event to attend and I am very grateful to the summer school team for inviting me once again. It’s always great to meet new people joining the game AI community, and also to connect with people across the industry who I have not had a chance to engage with before.
In the coming weeks for the newsletter I’ll be summarising the Dagstuhl seminar I am currently attending. Plus we will have a round-up of the recent Foundation of Digital Games conference and all of the interesting research projects showcased at the event.
Thanks as always for checking out the newsletter, and I’ll be back with more game/AI musings next week!