The 2024 AI Summit, Generative AI at GDC, Ubisoft's NEO NPC | AI and Games Newsletter
The industry's opinion on Generative AI is still rather mixed...
Hello all and welcome to this week’s AI and Games newsletter. Tommy here as I slowly recover from my past week at the Game Developer’s Conference (GDC) which takes place over in San Francisco.
Welcome to GDC: Hope You Survive the Experience
GDC is a hotpot of game developers, business managers, tech providers, industry advocates, academics, and more all smushed into the downtown area of the city.
It’s the Soulsborne of games conferences; a twisted and often maddening concoction that transforms several city blocks into an area teeming with talks, tech demos, business presentations, roundtables, award ceremonies, sponsored sessions, social gatherings and more. This is only my second time at GDC, and given my responsibilities as AI Summit advisor and business needs as an independent consultant, it transforms the event into an endurance race as you fight your way from meeting to meeting, or talk to talk - all the while trying to find a coffee and sandwich worth consuming that doesn’t set you back $30 in the process.
Nonetheless, it was a wonderful time to reconnect with so many people from across the sector, and the bulk of people I interacted with were friendly, energetic and enthusiastic. But of course, it’s a little difficult to be too enthusiastic about the games industry right now. It’s suffering from tremendous disruption, be it from short-sighted corporate reactions leading to thousands of developers losing their jobs, to poorly-designed and aggressively-advertised generative AI tools trying to replace the very humans kicked to the kerb. So this week I’m unpacking the main events or core themes I noticed throughout my time at GDC this year.
I will have a separate video vlog of my experiences which will be ready next week, but here’s a broader assessment of the trends and conversations I observed. In the meantime, if you’re curious what a week in the life of GDC is actually like, check out my vlog from last year.
The AI Summit
First things first, we have to talk about the Artificial Intelligence summit. One of several summits that run on the first two days of the week (Mon, Tues) at GDC. I have had the pleasure to be part of the organisational committee for the summit since 2021; a task that requires us to review talks submitted and then ensure we deliver an exciting programme.
This year was a challenge for a number of reasons, given we had a significant increase in the number of submissions to the summit compared to 2023. A reflection largely in the interest in generative AI for games. Though it’s worth saying we rejected many a talk given they were not aimed at current and aspiring engineers, designers and system architects - who are our primary audience.
The final programme was an interesting mixture of different areas of AI in the context of games. We had a handful of talks focussed on generative AI, most notably large language models (LLMs), but the talk by Hilary Mason from Hidden Door was more in line with expectations of building narrative and stories, almost all the LLM conversation was about building something that wasn’t text, with efforts to generate levels by Julian Togelius of NYU and modl.ai, or even design behaviour tree AI using LLMs by Trevor Santarra and Pierre-Arjun Dalaya from Unity.
Meanwhile we had some great talks on machine learning integrations for games, most notably Transitional Forms’ Dante Camarena and Nick Counter talking about their game ‘Little Learning Machines’ - a game that uses reinforcement learning as a mechanic - and then Gabriel Robert from Ubisoft La Forge on their work in building ML bots for quality assurance (QA) and other purposes in titles such as For Honor and Tom Clancy’s Rainbow Six Siege. We’re still finding new ways to integrate machine learning into games, and new ideas are still being explored across the industry. There were even more talks along these lines at the ML summit, but I haven’t had to chance to dig into those as yet.
But it’s important to highlight that despite the big push for generative AI, and even machine learning, much of the summit was very much focussed on ‘classic’ game AI techniques. Largely because those are the tools and standards used across the industry in a myriad of circumstances when crafting gameplay. Interestingly, one of the biggest topics that emerged in these game AI talks, was navigation. Talks about Hunt: Showdown, Warframe, and Suicide Squad Kill the Justice League all focussed on challenges with navigation. It highlights that while there are many new and exciting opportunities in AI for games, that by and large we’re still trying to solve the same game design problems we were facing 20 years ago, only now they’re on a much larger scale. Generative AI is all well and good, but it doesn’t help with the things that AI programmers in games are actually stuck on. More on that in a moment.
Cause and Effect
While I was late to the expo floor in the central parts of the Moscone Centre, it was interesting to observe how things change within a year. The expo floor is where companies go to peddle their wares: an updated version of a game engine, a new plugin or tool to facilitate parts of the broader development pipeline. It ranges from the big players like Epic Games and Meta, to platforms like Discord and TikTok, and many a smaller company seeking to get eyeballs on their products.
Perhaps the most noticeable shift was the the number of companies that either didn’t attend, or had a more subdued presentation. After all, the last 12 months have not been kind on the games industry as a whole. It was interesting to observe that Microsoft’s Xbox had a very minor presence, while I am only aware of what Sony PlayStation were up to from outside the convention centre. Meanwhile Unity ran a much smaller booth this year when compared to 2023 - with even Godot’s booth being larger. Technically, Epic Games’ booth was smaller, but it still took the crown for largest dedicated space to one single entity on the floor.
Now all of these companies have one thing in common: massive layoffs in the past twelve months. Conversations I had with some folks connected to these companies suggested that their upper management was sensitive to that reality. That showing up here with a booth that would require hundreds of thousands of dollars to coordinate and put together would be seen as insensitive. Though I say this while wearing my free Unreal engine t-shirt and reminiscing about the free beer, cookies and popcorn (yes, you read that correctly), that Epic Games were dishing out. So clearly not everyone got that memo.
Meanwhile, last year saw the continuing downhill trend with regards to Web3 and NFT technologies: what had taken up significant space in the expo floor in 2022 is seeing fewer and fewer square meters of real estate dedicated towards it. Conversely, much of that was now being pushed towards AI. Many AI-related middleware companies now had stalls in the expo, with arguably Inworld (whose work I’ve covered previously) having the largest space amongst this group of AI-driven businesses.
I sadly failed to get a headcount of the number of AI-related businesses with stands on the floor this year, given I’d be curious to see how many of them are still around 12 months from now.
And this is a critical talking point. A prevailing conversation I had throughout the week was whether or not the generative AI hype was all just noise? Is the tech actually heading anywhere? Given my role not just as a consultant and contractor in AI for games, but also as one of the organisers for the AI Summit, I had a lot of people raising this question.
The Generative AI Situation
It seems like whenever I’m at GDC, generative AI becomes a talking point - so I guess if you’re looking for someone to blame, I’m right here? Last year GPT 4 dominated the conversation, as many developers and business execs wondered what impact it would have games and AI as a whole. This year, the conversation has evolved somewhat, and what I present is more or less a summary of talking points I’ve had with many people across the sector.
It’s Useful in Some Areas, Useless in Others
It’s fair to say that any and all games studios of a particular size and calibre are exploring generative AI tools for game development. Whether that just be using GPT to help write parts of design documents and Copilot for writing code, or they’re investing in their own self-trained, self-developed image and text generation models. Everyone is looking into it.
But the common refrain that generative AI ‘solves’ so many game development processes has largely been proven false, and it’s becoming increasingly apparent what effort is required to accomodate these tools into a production pipeline. Some developers tried investing further in specific tools, only to find them substandard and reverted back to their original processes. This was a topic that arose in a talk by Keyword Studios, in which they sought to build an entire game using generative AI tools. While it worked well in areas of code and core game art, they had to fall back on existing practice in areas of animation, sound design and UI/UX given what is on the market was largely substandard for their needs.
Many developers talk about its use in early-phase ideation. A mechanism to just get something into a game: the first pass on a new mechanic, a rough art asset to be indicative of intent. With these first drafts complete, it can then nurture conversation among the dev team on how to move forward. But critically, as they move forward the team largely abandons most AI tools if they’re aiming to release the final product.
Everyone is Interested, Nobody is Committing
As stated both here - and in the GDC industry report not that long ago - everyone is exploring the technology, but nobody is willing to take the leap and make entire games using it given they’re aware of legal risks, and the blowback they will no doubt receive as a result of the ethical ambiguity that surrounds it. Even in the aforementioned Keywords Studios talk they indicated they have zero intent to release the game they made given they’re aware of the problems they’d had even trying to get it onto platforms such as Steam. The purpose was to run it as an experiment to see if it was even possible. The question of commercial viability was largely left unanswered.
From conversations across the board I hear a consensus: we wait. Sure, many a company will dabble in the technology, using it in small parts like we’ve seen in games like Foamstars and The Finals. But nobody I’ve spoken to is looking to embrace it for mass adoption. Among the big concerns is how legal and regulatory frameworks will evolve in the coming years. Policies on generative AI are still largely incomplete across most platform holders (with only Steam providing public guidelines at this time), and governments - most notably the EU - are rapidly moving towards regulation. But also there’s a lack of confidence in the companies themselves.
Sure, many a generative AI company has had money thrown at them hand over fist, with tens of millions being given to a lot of companies to get up and running. But that $20/30/40 million investment is often only enough to have them last for a couple of years. They need to build a product, and start selling it. After all, most generative AI tools are incredibly expensive to train and deploy (for context: GPT-4 costs around $700,000 to run per day). This cost overhead raises two distinct problems for game development:
Most generative AI solutions are charged on a ‘Credit’ style pay-as-you-go model, given the need to bill users for usage of the infrastructure. This doesn’t fit most games studios needs, who prefer the use of a ‘enterprise’ license, which are typically flat fees that can be more easily budgeted for.
More critically, these overheads introduce a critical question: will the AI company survive long enough for the game to hit the market?
This second point is a real dealbreaker: even if the game isn’t going to be using ‘live’ generated AI content (e.g. conversation dialogue), the tool still needs to work throughout the lifetime of the production. Most AAA games (and even AAAA games) have a production cycle of 3-5 years. For a game coming out next year it’s arguably too late to add it to the project without significant headaches, and for any game starting production the big concern is the AI tool’s creators won’t exist by the time the game reaches storefronts.
All in all, the technology is risky, and that is a big detractor for games companies.
Risky Technology Doesn’t Mesh with Games
It’s something of an old adage that the games industry is incredibly risk averse. It’s a space that likes to double down on known commodities: franchises reign supreme across all major publishers and developers. This equally applies to the technology used in the games themselves.
New game engines and middleware need to cut their teeth and prove their worth before mass adoption appears likely. By that same token, an increase in risk will have many a studio turn their backs on what was once considered a staple of their production pipeline. The past 12 months has seen the company behind the Unity game engine come under intense scrutiny for its attempts to gauge existing customers while continuing to provide an increasingly unstable and inferior product when compared to Epic games’ Unreal engine. All of this applies to generative AI as well.
The long and short of it is that for many, there’s simply too much risk to invest fully. Nobody is willing to commit. For now it’s a case of trying to keep current with the technology, while also watching what their competitors are doing.
It’s sort of a chicken and egg problem at this point: everyone is wanting to see how generative AI can change how games are made, but they’re all waiting for someone else to show how it could be done.
Ubisoft’s NEO NPC
Well speaking of how it could be done, last week Ubisoft made headlines with their latest foray into the adoption of generative AI technologies in games. Their NEO NPC is an extension of existing work by Inworld to try and make in-game characters not just be capable of smart conversation, but be able to more successfully interface with the game world around them. I was invited by the NEO team to come and check out the demo at an event during GDC.
The innovation of NEO is to make conversations with NPCs have value, and while it is (very) early days, I think it largely achieves that.
Having tried playing with both Inworld and Convai’s technology in videos released last year, I had a recurring problem with both of them: by making characters capable of conversation, it highlights how limited their intelligence is in every other aspect. If I can have a chat with a character, why do they have zero spatial awareness? If they can tell me the backstory of the world, why can’t they help me with my objective? Conversation is just one part of a much larger number of challenges faced in building successful world cohesion in game design, and by elevating one part of it, it exposes the problems everywhere else.
NEO is, as the team at Ubisoft put it, a very early phase production on how to build more intelligent characters in games. While it uses the Inworld tools for conversations, the system has a ‘scaffolding’ of sorts around it to prop up the system, feeding in a lot of knowledge from the world around it by annotating in-game objects, as well as feeding knowledge of the game world written by narrative designers. In addition, characters are much more objective focussed, and their interactions with you will change as they get to know you better.
I will be releasing my footage of the demo soon alongside a discussion of the experience, but I came away from it impressed. This is what generative AI has the potential to achieve, if tools are built to support game designers in their adoption, rather than trying to replace them.
Closing
Well, that’s far more words than I originally planned on releasing today. I’m still suffering from the jetlag from flying back from San Francisco (and sadly also a small cold too). But fear not, we should be returning to normal operations in the coming weeks.