NAIL122 AI for Games�Machine Learning in Games��Peter Guba
Faculty of Mathematics and Physics
Charles University
Industry response to ML boom�
Unity and Unreal have both introduced machine learning toolkits for their engines
Sony, Ubisoft, EA and other big names have established AI research divisions
Machine learning in gameplay�
2 categories
Machine learning in gameplay�
Agents that learn during gameplay
Not new - Creatures (1996), Black & White (2001)
Machine learning in gameplay�
Agents that learn during gameplay
Not new - Black & White (2001), Creatures (1996-2001)
Both games contain characters whose behaviours can be modified using rewards and punishments
Unless this is the core gameplay mechanic, it’s probably too tedious to implement
Machine learning in gameplay�
Machine learning in gameplay�
Agents that learn during gameplay
More modern example – Little Learning Machines (2024)
Aims to teach the basics of reinforcement learning
You teach robots to solve problems by creating training environments for them and letting them learn
Machine learning in gameplay�
Agents that are pre-trained to behave in a certain way
Some utilisation in RTS games, most notably Age of Empires IV (AoE IV) (2021)
Not one agent – several subsystems, one or more of them created using ML, combine to create a single RTS-playing agent
Machine learning in gameplay�
Agents that are pre-trained to behave in a certain way
2 examples outside RTS games – drivatars in the Forza series (2005-) and Sophy in Gran Turismo (first release in 1997, Sophy introduced in 2023)
Machine learning in gameplay�
Agents that are pre-trained to behave in a certain way
2 examples outside RTS games – drivatars in the Forza series (2005-) and Sophy in Gran Turismo (first release in 1997, Sophy introduced in 2023)
Drivatars meant to mimic other players, Sophy meant to just be a good player
Machine learning in gameplay�
Why just driving games?
Clear and clearly achievable goal
Few ways of interacting with the world
Issue of difficulty – we want agents that are just right for the player, not superhuman
May become a more viable option for other genres in the future
Machine learning in gameplay�
Machine learning in gameplay�
Related example – Killer Instinct
Trains an opponent called a
shadow on player data
The shadow is supposed to model the player’s behaviour
Uses case-based reasoning (related to machine learning, but mostly considered distinct)
Clearly achievable goal and small number of actions, like driving games
Machine learning in gameplay�
Agents that are pre-trained to behave in a certain way
Possibly a more viable approach in the near future
Fortnite is already experimenting with this (Learning Agents in Unreal – link to talk not yet available)
Machine learning in gameplay�
LLMs and generative AI
Machine learning in gameplay�
LLMs and generative AI
Machine learning in gameplay�
LLMs and generative AI
Machine learning in gameplay�
LLMs and generative AI
inworld’s Origins – sci-fi detective game with NPCs powered by LLMs with which you communicate by speaking out loud
They are mainly a company that creates AI-based gamedev tools
Origins is just a demo meant to showcase those, so not too polished
Machine learning in gameplay�
LLMs and generative AI
Narrative-Driven Generation: Story to Game World using Large Language Models (link not available yet)
Escaping the Infinite Mid (link not available yet)
Machine learning in gameplay�
LLMs and generative AI
Hidden Door – a company creating online role-playing games based on popular stories where you can interact with the world and characters
Machine learning in gameplay�
LLMs and generative AI
Hidden Door – a company creating online role-playing games based on popular stories where you can interact with the world and characters
Characters are simulated by LLMs
There is some internal state representation being used to make sure the story doesn’t get derailed
Machine learning in gameplay�
Machine learning in gameplay�
LLMs and generative AI
Google’s Unbounded – a framework in the making that is supposed to allow players to create characters and put them in generated worlds
Main contribution – keeping the characters appearance consistent across different worlds
Machine learning in gameplay�
Machine learning in gameplay�
Practical example of current capabilities – Retail Mage
Machine learning in gameplay�
Revolutionary in some ways, not so much in others
ML still doesn’t lend itself well to the needs of game designers, due to lack of control over the result
NPCs are probably still going to mostly be modelled using classic techniques like FSMs and behaviour trees
LLMs will probably be increasingly be used to create more immersive and free-flowing game experiences
Machine learning behind the scenes�
At runtime
Improved matchmaking, cheat detection, and graphics (thanks to deep learning super sampling)
Providing chat support
During production
Texture upscaling, testing, animation blend generation, analyzing player data
AI newsletter�
Other newsletters�