1 of 36

A guide to online experiments using Pavlovia

Giorgio Papitto papitto@cbs.mpg.de

Department of Neuropsychology, Max Planck Institute for

Human Cognitive and Brain Sciences, Leipzig, Germany

Max Planck Institute for Human Cognitive and Brain Sciences

2 of 36

Who am I?

Current position

PhD student at MPI for Human Cognitive and Brain Sciences, department of Neuropsychology.
Currently investigating the relationship between structural properties of action and language.

Max Planck Institute for Human Cognitive and Brain Sciences

3 of 36

Web-Based Research

Advantages

Ability to collect data quickly and from a large participant pool (more diverse samples).
Most findings point to a comparable effects from online and in-lab samples (even for response time data).
Costs for research are reduced.

Slide adapted from An Intro to Web-Based Experiments

Disadvantages

Inability to personally monitor participants and the environment.
Data can be noisier, fear of “bots”.
Limited to particular questions (almost exclusively behavioral research).

Max Planck Institute for Human Cognitive and Brain Sciences

4 of 36

Data quality: The web versus the lab

Some tips (Crump et al., 2013):

Make experiment somewhat engaging & short.
Set good data exclusion criteria (before starting!), e.g. short reading time of instructions.
Include “catch” trials that indicate whether subjects payed attention.
Make instructions short and ask questions to check understanding.
If possible, give participants feedback on their performance.
Ask questions about surroundings and motivation.
If possible, compare your measure to outcomes in previous research.

For more tips on how to improve a web-based study, see Kochari (2019)

Max Planck Institute for Human Cognitive and Brain Sciences

5 of 36

The steps for creating an online experiment

Once you have a clear structure of the experiment in your mind:

(1) Program the task

(2) Host the task to a server

(3) Recruit participants

Max Planck Institute for Human Cognitive and Brain Sciences

6 of 36

PsychoPy

PsychoPy is a free and open source python installation that will run on MacOS, Windows, and Linux.

Visit the PsychoPy page to install (https://www.psychopy.org/).

There are many ways to install, choose your favourite.

Max Planck Institute for Human Cognitive and Brain Sciences

7 of 36

PsychoPy

PsychoPy was developed firstly in 2003 at the University of Nottingham by Jonathan Peirce.
Between 2003 and 2005 it was made available online.
In 2007 and 2009, two papers by Peirce introduced the software to the scientific community.

Peirce 2007, 2009

Max Planck Institute for Human Cognitive and Brain Sciences

8 of 36

Lexical decision task

way

woy

Fixation Cross

Text

press right key for words

press left key for pseudo-words

Time

Word

Psuedoword

Let’s look at the list of all trials

Max Planck Institute for Human Cognitive and Brain Sciences

9 of 36

The Builder

PsychoPy Builder is the main interface for programming experiments.

Max Planck Institute for Human Cognitive and Brain Sciences

10 of 36

Elements in Builder

Text components

Text

Insert Routine (Trial) with two text components (one for fixation cross; one for the lexical item)

This is a $variable!

Max Planck Institute for Human Cognitive and Brain Sciences

11 of 36

Elements in Builder

Participants response

Another variable!

Insert Keyboard (Responses components) in trial

Max Planck Institute for Human Cognitive and Brain Sciences

12 of 36

Defining trials and variables

To make an example, let's create 8 trials. All we need is an excel file structured as follows:

One thing missing: how do we link these values to the experiment?

WITH A LOOP

These are the variables we have been using

Max Planck Institute for Human Cognitive and Brain Sciences

13 of 36

Sending trials and variables to the Builder

Loops

Your experiment is now ready to run

Insert a loop that starts before Trial and ends after it

Max Planck Institute for Human Cognitive and Brain Sciences

14 of 36

Can we also code in PsychoPy?

Insert code component above fix_cross

Make duration of the fixation cross variable in time.

Max Planck Institute for Human Cognitive and Brain Sciences

15 of 36

Can we also code in PsychoPy?

Import the Python package for randomizations

Define the variable every time the routine starts

Max Planck Institute for Human Cognitive and Brain Sciences

16 of 36

Can we also code in PsychoPy?

Import statements are not supported by PsychoJS, we need to find another solution

First delete this

Max Planck Institute for Human Cognitive and Brain Sciences

17 of 36

Can we also code in PsychoPy?

Click on fix_cross to open its properties

Insert the variable name (with $) to call it

Max Planck Institute for Human Cognitive and Brain Sciences

18 of 36

Testing your study

Once the experiment is over, PsychoPy will create a data folder with your participant’s results.

Press run

Participant pressed right for a word (correct trial)

Participant pressed left for a pseudo-word (correct trial)

Participant pressed left for a word (wrong trial)

Participant pressed right for a pseudo-word (wrong trial)

Max Planck Institute for Human Cognitive and Brain Sciences

19 of 36

Sharing the experiment

You can access the experiment we just created at the following link: https://github.com/giorgio-pap/Lexical_Decision_Task
To make it run download first of all the relevant files.
Then, open Psychopy and open the file lexical_decision_task.psyexp.

Max Planck Institute for Human Cognitive and Brain Sciences

20 of 36

The steps for creating an online experiment

(2) Host the task to a server

Max Planck Institute for Human Cognitive and Brain Sciences

21 of 36

Pavlovia

Work began in January 2018 as part of a 2-year project funded by the Wellcome Trust and generously supported by the University of Nottingham.
Pavlovia is currently hosting over 9,500 experiments from nearly 5,000 designers over 5 continents.

It contains also 23 demo experiments that you can try to run right now (digit span task, Stroop task, lexical decision task, and more).

Max Planck Institute for Human Cognitive and Brain Sciences

22 of 36

Pavlovia and PsychoJS

PsychoJS is a JavaScript library that mirrors the PsychoPy Python library classes and functions.

Max Planck Institute for Human Cognitive and Brain Sciences

23 of 36

JavaScript and GitLab

Each experiment corresponds to a GitLab project. Using GitLab you can:

Keep track of changes over time
Work collaboratively

�

You don’t need to know JavaScript:

If you use builder components, no scripting is required.
Psychopy will attempt an accurate translation from Python to JavaScript.
You can find the translation in the .js file in GitLab.

GOOD NEWS

Max Planck Institute for Human Cognitive and Brain Sciences

24 of 36

PsychoPy, PsychoJS and Pavlovia

Pavlovia is an experiment server, allowing you to run your study.

Max Planck Institute for Human Cognitive and Brain Sciences

25 of 36

Pavlovia

Pavlovia is an online platform that uses GitLab to host experiments, launch and run studies, and store data securely.
Experiments can run in any browser, play proprietary media, and most studies run on a mobile OS.

Visit Pavlovia and create an account.

Max Planck Institute for Human Cognitive and Brain Sciences

26 of 36

Create your online study

PsychoPy Builder can automatically generate the Javascript and HTML files.

Many of the existing Builder experiments should "just work", subject to the (Standard) Components being currently supported by PsychoJS (next slide).

Standard components

Custom/Code components

Max Planck Institute for Human Cognitive and Brain Sciences

27 of 36

Code Components

Code components can be automatically translated by Psychopy:

Or you can attempt your own translation:

Automatic translation

Custom translation

Max Planck Institute for Human Cognitive and Brain Sciences

28 of 36

Additional features (and their links)

Timing does not depend on internet connection: it might take some time for your experiment to start but then the resources are stored on your local machine;
You can redirect to a specific web page after the experiment is over.
You can have a personalized dialog box with mandatory fields and dropdown menus for participants info.
You can give/receive stars: the more people like your experiment the more it will be accessed and shared within the community.
You can look at other people experiments and code, you can also fork it from GitLab.

Max Planck Institute for Human Cognitive and Brain Sciences

29 of 36

Our online Lexical Decision task

Open PsychoPy and check that you are properly logged in:

Click on 4 (log-in)

Open the .psyexp file with the lab-based version of the experiment

Create your project repository by syncing your project with the server:

Click on 1 (synch).

Max Planck Institute for Human Cognitive and Brain Sciences

30 of 36

Our online Lexical Decision task

Name: The name of your project on Pavlovia.
Group/Owner: The user/group to upload the project.
Local folder: The (local) project path on your computer.
Description: Describe your experiment.
Tags (comma separated): The tag will be used to filter and search for experiments by key words.
Public: Tick this box if you would like to make your repository public, for anyone to see.

lexical_decision_online

Max Planck Institute for Human Cognitive and Brain Sciences

31 of 36

Our online Lexical Decision task

Once we clicked on Sync and Create a project (remember to make it public), this will show up:

On GitHub and GitLab, saved changes are called commits.

Each commit has an associated commit message, a description explaining why a particular change was made.
Commit messages capture the history of your changes.
One day you might want to restore an old version of your script/experiment and you´ll be able to find it reading the changes you made.

Max Planck Institute for Human Cognitive and Brain Sciences

32 of 36

Our online Lexical Decision task

Change the status to either Piloting or Running

Your experiment is now in Pavlovia!

Go to Pavlovia
Sign in
Go to the dashboard

lexical_decision_online

Max Planck Institute for Human Cognitive and Brain Sciences

33 of 36

The steps for creating an experiment

(3) Recruit participants

Max Planck Institute for Human Cognitive and Brain Sciences

34 of 36

Pavlovia

Pavlovia can connect to a range of other online systems as well, some of which are helpful in recruiting participants:

https://run.pavlovia.org/yourUsername/yourStudyName/index.html

Or you can just share the experiment link via mail:

In the URL you can already set some variables, e.g. participant number/group:

https://run.pavlovia.org/yourUsername/yourStudyName/index.html?participant=1&group=A

Max Planck Institute for Human Cognitive and Brain Sciences

35 of 36

The ultimate guide

If you want to learn something more about PsychoPy, there is a nice book that guides you through all the features of the software

Here you can find the complete manual

Here you can find some help on the forum

Max Planck Institute for Human Cognitive and Brain Sciences

36 of 36

Time to say goodbye

If you have any question concerning Python, PsychoPy or Pavlovia, you can always reach me at papitto@cbs.mpg.de.

I hope you had fun!

Max Planck Institute for Human Cognitive and Brain Sciences