Published using Google Docs
PRED18: Predator/Prey DAVIS Dataset
Updated automatically every 5 minutes

PRED18: Predator/Prey DAVIS Dataset

Authors: Diederik Moeys diederikmoeys@live.com 

   Tobi Delbruck tobi@ini.uzh.ch

(link to source document for this web page)

Note: this dataset was previously called PRED16. It has been enhanced to include prey size labeling. Changes compared to PRED16 are highlighted in red.

PRED18  was developed by the Sensors Group of the Inst. of Neuroinformatics, Univ. of Zurich and ETH Zurich. See more datasets and tools from the Sensors Group.

PRED18 was used to develop the first CNN driven by DVS data, as shown in this YouTube video Predator Prey Robot Synchronized Video.

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Citation

Publications using this data should cite the following work:

  1. D. P. Moeys, F. Corradi, E. Kerr, P. Vance, G. Das, D. Neil, D. Kerr, and T. Delbrück, “Steering a predator robot using a mixed frame/event-driven convolutional neural network,” in 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), 2016, pp. 1–8.
  2. Moeys, Diederik Paul, Neil, D., Corradi, F., Kerr, E., Vance, P., Das, G., et al. (2018). PRED18: Dataset and further experiments with DAVIS event camera in predator-prey robot chasing. in EBCCSP 2018, (submitted). Available at: https://www.dropbox.com/s/yn7sbqe8my9mqse/MoeysEBCCSP2018.pdf?dl=0.

They can also cite the seminal DVS and DAVIS papers

  1. Lichtsteiner, P., Posch, C., and Delbruck, T. (2008). A 128 x 128 120 dB 15 µs Latency Asynchronous Temporal Contrast Vision Sensor. IEEE Journal of Solid-State Circuits 43, 566–576. doi:10.1109/JSSC.2007.914337.
  2. Brandli, C., Berner, R., Yang, M., Liu, S.-C., and Delbruck, T. (2014). A 240x180 130 dB 3 us Latency Global Shutter Spatiotemporal Vision Sensor. IEEE Journal of Solid-State Circuits 49, 2333–2341. doi:10.1109/JSSC.2014.2342715.

See also

Download

Download DHP19 by using this link and follow the "Access the files" link.

Total size is about 307 GB.

The folder contains the following subfolders:

  1. Recording 1-20: the folders containing the total 1.25h of recordings.
  2. Trial run 1-8: the folders containing the resulting trial run recordings (each folder name mentions the speed of the robot used: 0.5-2 m/s).
  3. All LMDB Datasets: the LMDB dataset extracted from all recordings 1-20.
  4. Runtime networks: the folder containing the runtime networks used in the 2 publications.
  5. Demo code: this folder contains the zipped code used for the trial run demos.

Table of contents

Citation        1

See also        1

Download        1

Table of contents        2

Recording 1-20:        2

Trial run 1-8:        3

All LMDB Datasets:        4

Runtime networks:        5

Notes:        5

Recording 1-20:

In each subfolder Recording 1-20 the following files can be found:

  1. The AEDAT file: (example DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1.aedat), the raw recording file which can be opened with jAER.
  2. The AVI file of the APS frames: (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1_APS.avi).
  3. The corresponding APS-timecode file: (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1_APS-timecode.txt), the .txt consists of the APS frame number and its timestamp.
  4. The 3 AVI file of the DVS histograms (36x36/54x54/72x72): (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1_DVS_5000ev_raw200gs_36x36.avi), 5000 events accumulated with a grayscale of 200.
  5. The corresponding 3 DVS-timecode files (36x36/54x54/72x72): (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1_DVS_5000ev_raw200gs_36x36-timecode.txt), the .txt consists of the DVS frame number and its timestamp.
  6. The 3 AVI file of the background-filtered DVS histograms (36x36/54x54/72x72): (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1_DVS_5000ev_raw200gs_36x36_bkgnd_5000.avi), 5000 events accumulated with a grayscale of 200.
  7. The corresponding 3 background-filtered DVS-timecode files (36x36/54x54/72x72): (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1_DVS_5000ev_raw200gs_36x36_bkgnd_5000-timecode.txt), the .txt consists of the DVS frame number and its timestamp.
  8. The target location only in the 240x180 field of view: (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1-targets.txt), the .txt is formatted as follow: frameNumber timestamp x y.
  9. PRED18: The target location and size in the 240x180 field of view: (example: DAVIS240C-2016-01-11T15-43-32+0000-04010058-0_recording_1-targets-resized-targets.txt), the .txt is formatted as follow: frameNumber timestamp x y targetTypeID width height. The width and height are in pixels in the original image.

Trial run 1-8:

In each subfolder Trial run 1-8 the following files can be found:

  1. The AEDAT file: (example DAVIS240C-2016-03-11T10-49-55+0000-04010058-0.aedat), the raw recording file which can be opened with jAER.
  2. The target location only in the 240x180 field of view: (example: DAVIS240C-2016-03-11T10-49-55+0000-04010058-0-targets.txt), the .txt is formatted as follow: frameNumber timestamp x y.
  3. PRED18: The target location and size in the 240x180 field of view: (example: DAVIS240C-2016-03-11T10-49-55+0000-04010058-0-targets-resized-targets.txt), the .txt is formatted as follow: frameNumber timestamp x y targetTypeID width height. The width and height are in pixels in the original image.
  4. The corresponding decisions (for LCRN trials): (example: DAVIS240C-2016-03-11T10-49-55+0000-04010058-0-decisions.txt), the file containing the timestamped decisions obtained in the trial runs. The .txt is formatted as follows: system.currentTimeMillis lastTimestampUs decisionLCRN.
  5. Constraints used in the postprocessing of the network decisions in the LCRN trial run (Constraints used.txt).

All LMDB Datasets:

In the subfolder without_size/ the following subfolders can be found:

Each contains train and test subfolders (LMDBs are inside). Train contains the first 80% of the recording (shuffled). Test contains the second 20% of the recording (unshuffled). All data is labelled 0 1 2 3 (L C R N).

  1. LMDB1-20: (all images unfiltered from videos 1-20), currently 55% of it is non-visible (APS/DVS ratio is almost 45:55%).
  2. LMDB1-20ignore: (ignoring where the target is within 1 pixel of the subsampled image around the boundary region, the data is regenerated without these ambiguous images marked by Ignore_0_ in their name); Data is still unbalanced in non-visible;
  3. LMDB1-20equal: (same as LMDB1-20ignore but this time all data is made equal percentage 25% by taking randomly for each class the number of elements which the class which contains the least element has; example original LCRN dataset has 4 5 8 3 elements per class? new dataset is 3 3 3 3 elements for each class).
  4. LMDB1-20APS and DVS are the LMDB1-20 sorted by APS and DVS type.
  5. LMDBrec20DVS is the LMDB of DVS type only of recording 20.

The subfolder with_size/ the following subfolders can be found:

Each contains train and test subfolders (LMDBs are inside). Train contains the first 80% of the recording (shuffled). Test contains the second 20% of the recording (unshuffled). All data is labelled 0 1 2 3 4 5 6 7 8 9 (L:S L:M L:XL C:S C:M C:XL R:S R:M R:XL N).

  1. Dataset_36x36/54x54/72x72: LMDB of all recordings 1-20, unfiltered DVS.
  2. Dataset_36x36/54x54/72x72_bkgndfiltered: LMDB of all recordings 1-20, background-filtered DVS.
  3. Dataset_36x36/54x54/72x72_lessexp: LMDB of all recordings 1-20, unfiltered DVS with less under/overexposure (used in the paper).
  4. Dataset_36x36/54x54/72x72_lessexp_bkgndfiltered: LMDB of all recordings 1-20, background-filtered DVS with less under/overexposure (used in the paper).

Runtime networks:

  1. The subfolder without_size contains the runtime network with 4 outputs of the EBCSSP paper in xml format.
  2. The subfolder with_size contains the runtime network with 10 outputs in .xml and caffe formats.

Notes:

-1 position is for non-visible, sometimes NaN is found too.