Request edit access
Summer school "Practical Approaches to Human Multimodal Behavior: From Acquiring and Aligning Data to Quantifying Patterns"
Date: 4-8 September (Mo-Fri) Location: Radboud University Nijmegen (in-person)
How can we make sense of the complex array of multimodal behaviors that people use to communicate? How do speech, gesture and facial expression dynamically interrelate to produce meaning, convey our intentions, and influence others? In this course, we provide an introduction to a set of methodological approaches that address these questions.
 Human communication is multimodal, utilizing different articulators, such as the head, facial muscles, and hands. These multimodal signals are dynamically linked to one another, both due to the mechanical influences of being produced by a single body, as well as on the message level, where people seem to dynamically adjust the entire array of signals depending on the intended meaning, context, or other constraints. As such, to understand human communicative behavior, we need methods that allow us to look beyond single modalities. Looking beyond single modalities can sometimes be challenging, as one needs to become familiar with several different unimodal measurement approaches that are already difficult to learn on their own, let alone together.
 The purpose of this course is to bring together a set of methodological approaches that provide a starting point for analyzing multimodal data. We will cover quantitative approaches that focus on capturing and analyzing multimodal interactions and patterns. We will also dovetail these approaches with more classical methods in this fields, using ELAN for manual annotation of video data. For the quantitative approaches we will utilize Python and R.
 Each day of the course will provide lectures to theoretically ground the approaches, including their assumptions, requirements and what types of questions they can answer, as well as hands-on training in applying these methods. Hands-on sessions will be also be done in small groups, to ensure that everyone is able to gain direct experience working with these techniques. Outside of feedback and support during the hands-on sessions, there will also be time each day for Question and Answer sessions.

Example of things covered in the course:
- introduction to motion tracking
- building your own low-cost 3D markerless motion tracking + audio recording system
- masking identities of participants in video
- data-wrangling: aligning multiple data streams, interpolation, smoothing
- kinematic feature analyses
- kinematic pattern analyses 
- automatic gesture-speech synchrony analyses
- basic introduction to virtual avatar design

Guest speakers will cover
- introduction to machine learning classification
- tracking of speech articulators
- combining qualitative and and quantitative analyses

The exact registration opens on the 20th of June. So that everyone (hopefully) can register on a first come first serve notice.

Follow this link on 20 June (first come first serve): https://docs.google.com/forms/d/1_xKL8nNysaCFS76JqDw9UTYpKHlY_vHM8AQecykEubc
Sign in to Google to save your progress. Learn more
Please indicate your interest so we can make an estimate of the potential group of attendees (this does not count as a registration)
Clear selection
Submit
Clear form
This content is neither created nor endorsed by Google. - Terms of Service - Privacy Policy

Does this form look suspicious? Report

Google Forms