ROSHAMBO17: DAVIS Rock-paper-scissors (RoShamBo) Dataset 2017
Iulia-Alexandra Lungu, Tobi Delbruck
October 2017
Web: This document | analytics
- For questions related to the use of this dataset or the associated tools, please write iulialexandra@ini.uzh.ch or tobi@ini.uzh.ch.
- ROSHAMBO17 was developed by the Sensors Group of the Inst. of Neuroinformatics, Univ. of Zurich and ETH Zurich. See more datasets and tools from the Sensors Group.
Change history
- October 23, 2017 - created
- Jan 17 2018 - improved, added resilosync link, added screenshots
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Dataset description
Roshambo is a dataset of rock, paper, scissors and background images [Fig 1] recorded using an event-driven camera, the DAVIS240C. The DAVIS outputs conventional global shutter gray scale image frames and dynamic vision sensor (DVS) events. The DAVIS also includes its own inertial measurement unit (IMU) that measures camera rotation and acceleration.
Sample images produced from DAVIS DVS data; each image consists of full-rectified 2k events subsampled to 64x64 pixel images
Contents
Citation
Publications using this data should cite the following paper:
I.-A. Lungu, F. Corradi, and T. Delbruck, Live Demonstration: Convolutional Neural Network Driven by Dynamic Vision Sensor Playing RoShamBo, in 2017 IEEE Symposium on Circuits and Systems (ISCAS 2017), Baltimore, MD, USA, 2017 [Online]. Available: https://drive.google.com/file/d/0BzvXOhBHjRheYjNWZGYtNFpVRkU/view?usp=sharing
More about the dataset
20 people were recorded over continuous videos for each of the three rock, paper and scissors symbols, amounting to around 5 hours of total recording time. A fourth background category was also recorded in order to capture non-hand data necessary for classification tasks.
The raw data format is AEDAT. More details about this file format can be found here.
The AEDAT files were converted to AVI movies by creating event histograms using a variable number of events (500, 1000, 2000, 4000 events per frame), using the tool https://github.com/SensorsINI/jaer/blob/master/dvs-slice-avi-writer.sh (see folder for windows .exe version). The usage for dvs-slice-avi-writer are
$ dvs-slice-avi-writer.exe
java DvsSliceAviWriter [-aechip=aechipclassname (fully qualified class name, e.g. eu.seebetter.ini.chips.davis.DAVIS240C)] [-dimx=36] [-dimy=36] [-quality=.9] [-format=PNG|JPG|RLE|RAW] [-framerate=30] [-grayscale=200] [-writedvssliceonapsframe=false] [-writetimecodefile=true] [-numevents=2000] [-rectify=false] [-normalize=true] [-showoutput=true] [-maxframes=0] inputFile.aedat [outputfile.avi]
Note arguments values are assigned with =, not space
If outputfile is not provided its name is generated from the input file with appended .avi
The AVI movies were cut into frames with aedat_to_avi.sh and compiled in two LMDB databases with avi_to_lmdb.py, one for training and another one for testing. For more details about the LMDB format, check this Wikipedia entry.
Data processing included full event rectification (all events are considered positive), 0-1 rescaling and image resizing from 240x160 pixels to 64x64 pixels.
Dataset contents
The folder has the following structure :
- recordings
- AEDAT recordings
Each recording contains a single symbol and is labelled with the corresponding symbol name.
- AVI movies
There is a folder for each number of events that was used to generate the frames.
- lmdb_train
It contains training and validation datasets in LMDB format, created using the avi_to_lmdb.py script provided in the utils folder.
- utils
- aedat_to_avi.sh is a bash script that takes all the .aedat recordings in the specified folder and converts them to .avi using jAER. The LIST_OF_EVENTS specifies what number of events you want to accumulate to produce one frame.
- avi_to_lmdb.py takes all available .avi files and compiles them in train and validation LMDBs.
- architecture.txt specifies the architecture of the network used for training
- caffe_files contains all the files necessary to train and test the network in Caffe:
- LP_Roshambo_NullHop_solver.prototxt: solver prototxt used for training
- LP_Roshambo_NullHop_train.prototxt: prototxt file used for training
- roshambo_test.prototxt: prototxt file used for testing
- train_nullhop.sh: bash script for automatizing the training process
- roshambo.caffemodel: pretrained weights file
Note: remember to change the file paths in all the necessary scripts before attempting to use them.
Downloading the dataset
Use Resilio Sync (a private bittorrent protocol) to get the data at this link:
The complete dataset download is currently about 80GB. Clicking on the link above will pop up this result, assuming you already have resilio sync installed:
Resilio Sync link result
Once you install and run Resilio Sync, you can select the option to “Selective Sync” to synchronize only part of the data.
On linux, you should copy the link above, and paste into the web GUI interface for sync by selecting the + button and selecting the “Enter a key or link” item.
How to extend the dataset
- Install jAER: Follow the iniLabs guide to installing and running jAER: https://inilabs.com/support/software/jaer/
- Record new .aedat files, each one with a particular class.
- Convert to .avi format with tool dvs-slice-avi-writer
- Use the provided script (aedat_to_avi.sh) to convert .aedat files to .avi files.
- Create LMDB: Use the provided script (avi_to_lmdb.py) to create the LMDBs. Both train and test LMDBs are shuffled. The data is split into 90% for the training LMDB and 10% for the test LMDB.