1 of 45

Taking control of your DICOM data: ReproIn/HeuDiConv tools

ReproNim Monthly Webinar, December 6, 2019

Note: This Webinar is being recorded...

ReproSpeaker:

Dr. Yaroslav O. Halchenko, http://centerforopenneuroscience.org, PBS, Dartmouth College

ReproInformation:

Website: repronim.org

EMail: info@repronim.org

Twitter: @ReproNim

ReproInstitutions:

UMass Medical, MIT, Dartmouth, McGill, UC Irvine, UC San Diego, TCG, Inc

ReproFunding:

NIBIB P41 EB019936 “Center for Reproducible Neuroimaging Computation - CRNC”

2 of 45

Acknowledgments

James

Haxby

James

Dobson

Terry

Sackett

Satrajit

Ghosh

Mathias

Goncalves

Chandana

Kodiweera

NIH #1P41EB019936-01A1 (ReproNim)

NSF #1429999 (DataLad)

Michael

Hanke

Matteo

Visconti di Oleggio

Castello

3 of 45

We’re so busy.

How do we do reproducible neuroimaging efficiently?

There’s a problem

4 of 45

5 of 45

We’re so busy.

Datasets came in a variety of layouts,

We had no time to “understand” data of others

There was a problem

6 of 45

Bootstrapped by OpenNeuro&INCF,

community came up with a solution:

7 of 45

http://bids.neuroimaging.io

├── sub-sid000005

│ ├── ses-famfirst

│ │ ├── anat

│ │ │ ├── sub-sid000005_ses-famfirst_acq-MPRAGE_T1w.json

│ │ │ └── sub-sid000005_ses-famfirst_acq-MPRAGE_T1w.nii.gz

│ │ ├── fmap

│ │ │ ├── sub-sid000005_ses-famfirst_acq-25mm_magnitude1.json

│ │ │ ├── sub-sid000005_ses-famfirst_acq-25mm_magnitude1.nii.gz

│ │ │ ├── . . .

│ │ ├── func

│ │ │ ├── sub-sid000005_ses-famfirst_task-fam1back_run-01_bold.json

│ │ │ ├── sub-sid000005_ses-famfirst_task-fam1back_run-01_bold.nii.gz

│ │ │ ├── sub-sid000005_ses-famfirst_task-fam1back_run-01_events.tsv

│ │ │ ├── . . .

. . .

8 of 45

Benefits of BIDS

You have seen one BIDS dataset – you have seen them all!

BIDS is both human- and machine- friendly
BIDS compliance could be automatically verified using bids-validator
PyBIDS, bids-matlab, etc. can assist in scripted use of BIDS datasets
BIDS-apps provide a turnkey solution for BIDS datasets

9 of 45

10 of 45

We’re so busy.

Who wants to deal with naming file structures?

Solution brought a problem

11 of 45

We needed a solution

12 of 45

Automation

13 of 45

https://xkcd.com/1319

14 of 45

ReproIn Goals/Principles

Minimize manual entry (humans make more mistakes than machines)
(Optionally) Place BIDS datasets in a logical hierarchy (for a center or lab)
As much information (subject, session, canceled runs, …) �as possible should come from DICOMs
Be flexible to accomodate any type of neuroimaging study

15 of 45

Open by Design: Join/Contribute

Do not reinvent the wheel - use and contribute to existing projects!

Learn better coding and testing practices
(re)Use already established development, release, and deployment infrastructure
Share/offload “user-support'”
Get credit faster (Zenodo, duecredit)
Find new friends/collaborators/passion

16 of 45

ReproIn

reproin.repronim.org

HeuDiConv

DataLad

dcm2niix

17 of 45

reproin.repronim.org

18 of 45

2(+1) ReproIn Recipe Ingredients

Location of exam card on scanner console (optional)� defines naming and placement of datasets in a hierarchy
ReproIn specification (BIDS-like) of protocol names� defines naming and placement of files within BIDS-dataset
ReproIn heuristic (code) for HeuDiConv implementing 1. + 2.

19 of 45

Hierarchy

20 of 45

Hierarchy (Siemens Prisma 3T Walkthrough)

https://github.com/ReproNim/reproin/blob/master/docs/walkthrough-1.md

21 of 45

Hierarchy (Siemens Prisma 3T Walkthrough)

https://github.com/ReproNim/reproin/blob/master/docs/walkthrough-1.md

22 of 45

Hierarchy (Siemens Prisma 3T Walkthrough)

https://github.com/ReproNim/reproin/blob/master/docs/walkthrough-1.md

23 of 45

Hierarchy

By default ReproIn would place datasets into hierarchy according to �(auto-filled) Study Description field
This behavior could be overridden by providing �--locator "" �option to heudiconv

24 of 45

BIDS

│

...

├── Gobbini

│ ├── Matteo

│ │ ├── 1002-faceangles

│ │ ├── 1017_famface-angles

│ │ ├── 1037_budapest

│ │ └── 1038_hyperface

│ └── Vassiki

├── Haxby

│ ├── Feilong

│ └── Sam

...

Center/PI/Researcher/Study

25 of 45

2. Specification

26 of 45

2. Specification

27 of 45

28 of 45

29 of 45

Subject info (Siemens Prisma 3T Walkthrough)

https://github.com/ReproNim/reproin/blob/master/docs/walkthrough-1.md

30 of 45

dcm2niix

31 of 45

HeuDiConv/ReproIn HOWTO

32 of 45

Features

BIDS JSON files automatically generated (dcm2niix)
original DICOMs stored under sourcedata/
placeholders for BIDS files such as � README, task-*.json, *_events.tsv, dataset_description.json, etc.
incremental, version-controlled updates (due to --datalad)

fake git commit dates (1 sec interval starting Apr 7 22:13:15 2005)

modular datasets (datalad subdatasets)

33 of 45

https://github.com/ReproNim/containers

34 of 45

35 of 45

36 of 45

dcm2niix

37 of 45

BIDS

│

...

├── Gobbini

│ ├── Matteo

│ │ ├── 1002-faceangles

│ │ ├── 1017_famface-angles

│ │ ├── 1037_budapest

│ │ └── 1038_hyperface

│ └── Vassiki

├── Haxby

│ ├── Feilong

│ └── Sam

...

Center/PI/Researcher/Study

38 of 45

famface_angles/data (master) $ git shortlog

DBIC BIDS Team (43):

[DATALAD] added content

Converted subject a000234, 2 sequences, 168 dicoms

Converted subject sid000013, session famfirst, 19 sequences, 3635 dicoms

Converted subject sid000013, session strfirst, 18 sequences, 3462 dicoms

...

Remove bad conversion of sub-sid000014, ses-famfirst

[DATALAD] Recorded existing changes

[DATALAD] added content

Converted subject sid000014, session famfirst, 20 sequences, 3931 dicoms

Matteo Visconti dOC (89):

[DATALAD] new dataset

Remove sub-a000234

Remove duplicate runs

Remove unused scans file for sub-sid000013

Rename task names for some subjects.

Remove low resolution T1 for sub-sid000013

Add script to inject intended for in fieldmap

[DATALAD RUNCMD] python scripts/add_intendedfor.py sub-si...

Rename second fmap for sub-sid000014

...

39 of 45

famface_angles/data (master) $ git shortlog

DBIC BIDS Team (43):

[DATALAD] added content

Converted subject a000234, 2 sequences, 168 dicoms

Converted subject sid000013, session famfirst, 19 sequences, 3635 dicoms

Converted subject sid000013, session strfirst, 18 sequences, 3462 dicoms

...

Remove bad conversion of sub-sid000014, ses-famfirst

[DATALAD] Recorded existing changes

[DATALAD] added content

Converted subject sid000014, session famfirst, 20 sequences, 3931 dicoms

Matteo Visconti dOC (89):

[DATALAD] new dataset

Remove sub-a000234

Remove duplicate runs

Remove unused scans file for sub-sid000013

Rename task names for some subjects.

Remove low resolution T1 for sub-sid000013

Add script to inject intended for in fieldmap

[DATALAD RUNCMD] python scripts/add_intendedfor.py sub-si...

Rename second fmap for sub-sid000014

...

40 of 45

famface_angles/data (master) $ git shortlog

DBIC BIDS Team (43):

[DATALAD] added content

Converted subject a000234, 2 sequences, 168 dicoms

Converted subject sid000013, session famfirst, 19 sequences, 3635 dicoms

Converted subject sid000013, session strfirst, 18 sequences, 3462 dicoms

...

Remove bad conversion of sub-sid000014, ses-famfirst

[DATALAD] Recorded existing changes

[DATALAD] added content

Converted subject sid000014, session famfirst, 20 sequences, 3931 dicoms

Matteo Visconti dOC (89):

[DATALAD] new dataset

Remove sub-a000234

Remove duplicate runs

Remove unused scans file for sub-sid000013

Rename task names for some subjects.

Remove low resolution T1 for sub-sid000013

Add script to inject intended for in fieldmap

[DATALAD RUNCMD] python scripts/add_intendedfor.py sub-si...

Rename second fmap for sub-sid000014

...

41 of 45

famface_angles/data (master) $ git shortlog

DBIC BIDS Team (43):

[DATALAD] added content

Converted subject a000234, 2 sequences, 168 dicoms

Converted subject sid000013, session famfirst, 19 sequences, 3635 dicoms

Converted subject sid000013, session strfirst, 18 sequences, 3462 dicoms

...

Remove bad conversion of sub-sid000014, ses-famfirst

[DATALAD] Recorded existing changes

[DATALAD] added content

Converted subject sid000014, session famfirst, 20 sequences, 3931 dicoms

Matteo Visconti dOC (89):

[DATALAD] new dataset

Remove sub-a000234

Remove duplicate runs

Remove unused scans file for sub-sid000013

Rename task names for some subjects.

Remove low resolution T1 for sub-sid000013

Add script to inject intended for in fieldmap

[DATALAD RUNCMD] python scripts/add_intendedfor.py sub-si...

Rename second fmap for sub-sid000014

...

42 of 45

HeuDiConv/ReproIn HOWTO Summary

Adopt our naming specification at the scanner

with or without hierarchy�

Use HeuDiConv with reproin heuristic file

possible to use even with already collected (in DICOMs) data (HOWTO is coming)�

Docker and Singularity images available

included in ReproNim/containers DataLad dataset with ready to use �Singularity containers of BIDS-Apps, ReproNim products, etc.

Contribute to

43 of 45

Planned work (contribute/join!)

Configurable customizations
Extend to other modalities (M/EEG)
Automatic generation of *_events.tsv files
Automatic collection and “slicing” of

audio/video stimuli
physiological data

https://searchjobs.dartmouth.edu/postings/52587

44 of 45

Related References

Other Apps for conversion of DICOMs to BIDS

visit https://bids.neuroimaging.io/#benefits

DataLad HIRNI (from CP5) - web-UI driven framework

http://docs.datalad.org/projects/hirni/en/latest/
for flexible automated conversion of (any) raw data into target layouts
built also on top of HeuDiConv and DataLad, can be used with ReproIn

Flywheel Gear for HeuDiConv (and thus ReproIn)

https://github.com/PennBBL/fw-heudiconv

45 of 45

Thank you for your attention!

Questions?

issues/recommendations - report on https://github.com/repronim/reproin/issues