Overview
There is a growing realisation that to understand the richness of human behaviour and its neural underpinnings, we need to embrace more naturalistic paradigms. Naturalistic neuroimaging has proven highly successful in the domain of perception: recent studies of natural scene perception, movie watching, and storybook listening have yielded novel insights into how the brain supports complex, real-world sensory processing. At present, however, few naturalistic datasets exist for settings in which the participant generates rich and complex behaviour.
In this round-table workshop, we will discuss video games as a potential testbed for comparing biological and artificially intelligent behaviour. Video games capture much of the complexity of real-world decision tasks, such as vast state spaces, multi-step action sequences, interactions with objects and agents, and dynamic interleaving of planning and execution. Yet crucially, they present an experimentally and computationally tractable testbed which allows for experimental manipulations, and comparison of human vs. machine behaviour and internal computations. We will have short talks from researchers currently using video games as a tool for understanding human and artificial cognition, and one central aim will be to identify robust, reliable benchmarks with which human and artificial agents can be compared.
The main outcome of this workshop, if successful, would be a co-designed project that is shaped by input from across the CCN community, where resulting data analysis/modelling is shared across a number of labs.
Outline
We will hold a 2-hour interactive workshop and round-table discussion, hosted during the CCN2025 conference in Amsterdam. The provisional date for this in-person event will be Wednesday 13th August.
The approximate structure for our Community Event will be:
- First ~30 minutes: Pitch talks, approx 4-5-minutes each, from current researchers using video games as a basis for understanding cognition, neuroscience, and/or AI
- Next ~1 hour: Breakout sessions in which people are randomly assigned into groups of ~6 people each, discussing one (or more) of these three questions:
- 'what constitutes a useful benchmark against which to evaluate human behavioural and/or neural data during naturalistic gameplay?'
- ‘what unique questions in cognitive science/neuroscience/AI might be addressed using naturalistic games that are difficult to address using traditional experimental design?’
- ‘what computational models are most appropriate for comparison with human behavioural and neural data in studying naturalistic behaviour with games?’
- Final ~30 minutes: Collective discussion in which we integrate the discussions from the round table breakouts, and summarise how this might inform a future, co-designed data collection project with members of the workshop
Outcomes
Our ideal outcomes for the workshop would be to identify benchmarks, research questions, and models that are useful for future work in the comparison of artificial and biological agents during gameplay. Our ultimate ambition, inspired by similar projects in vision science (Natural Scenes Dataset), would be to develop a large-scale neuroimaging dataset with naturalistic behaviour that bridges cognitive science, computational neuroscience, and AI. We anticipate the outcomes of this workshop will be informative for the development of that longer-term project, and this will ultimately deliver a widely-used resource for the CCN community.
Community Event Registration
Although registration is not mandatory, we invite attendees for this year's CCN to indicate their interest in this community event below (nb you will need to register for CCN in order to attend the workshop). We also invite people who are not attending CCN to use the form to register their interest in this project, so that we can contact them in future.
For further details or questions about the community event, please contact laurence.hunt@psy.ox.ac.uk or jascha.achterberg@dpag.ox.ac.uk
