Open Data Science and Reproducibility

Mikolaj A. Pawlak MD PhD

mpawlak@ump.edu.pl

Department of Neurology and Cerebrovascular Disorders

Poznan University of Medical Sciences

Research Methodology Conference, February 3rd, 2018

Topics

• The Problem
• Solutions
• Your Role in Clinical Science
• Ideas for practical implementation

The Problem

• Lack of confidence in science
• Irreproducible results
• Inconclusive research
• Abnormal structure of scientific community
• Publication bias

​

Irreproducibility of preclinical cancer research

• Problem starts when the data has to be translated into operational research and products
• Preclinical work should led to Phase I & II clinical trials
• Success rate falling to 18%
• Health outcomes depend on the results of published research

Believe it or not

There is no single cause of irreproducibility

• poor experimental design
• inappropriate analysis
• questionable research practices
• Cultural factors:
• highly competitive research environment
• high value placed on novelty
• publication in high-profile journals

Scale and the consequences

The idea that the same experiments always get the same results, no matter who performs them, is one of the cornerstones of science’s claim to objective truth.

Solutions

• Raw data sharing
• Open data projects
• Open Access publishing
• Reproducible research

Measures that might improve reproducibility

• Greater openness and transparency
• Better use of input and advice from other experts
• Reporting guidelines
• Post-publication peer review
• Pre-registration of protocols and plans for analysis
• Better use of standards and quality control measures

Swedish system for monitoring multiple sclerosis

Swedish system for monitoring multiple sclerosis

Examples of open data

• NIH Pediatric database
• ADNI
• 1000 functional connectomes
• INDI
• Human Connectome project
• Brain Development Database
• Nationale Kohorte

National Database for Autism Research

• The National Database for Autism Research (NDAR) is an NIH-funded research data repository
• aims to accelerate progress in autism spectrum disorders (ASD) research through data sharing, data harmonization, and the reporting of research results
• serves as a scientific community platform and portal to multiple other research repositories, allowing for aggregation and secondary analysis of data

​

​

​

​

https://ndar.nih.gov/

1000 Functional Connectomes

Rockland sample

ADHD 200

• International project aimed at finding imaging features of ADHD
• Machine learning competition based on brain imaging data

Autism Brain Image Database ABIDE

• 539 ASD
• 579 NC
• Structural brain data and resting state BOLD fMRI
• behavioral data
• 17 sites

NIH Pediatric database

• healthy kids 5-18 y.o.
• Learning about the dynamics of brain structure change in healthy population

Brain Development Database

Brain Development Database IXI

IXI - Information eXtraction from Images (EPSRC GR/S21533/02)

The images in NIFTI format can be downloaded from here:

• T1 images (all images)
• T2 images (all images)
• PD images (all images)
• MRA images (all images)
• DTI images (all images, bvecs.txt, bvals.txt)
• Demographic information (spreadsheet)

​

Alzheimer Disease Neuroimaging Initiative

• Largest project oriented at data acquisition and sharing in Alzheimer disease
• Initial budget ~$60M
• Currently 3rd iteration

ADNI perspectives

Dementia is not just a neurological symptom, it is a population challenge

Parkinson Progression Markers Initiative

• database for brain samples
• the aim is to create a reference repository for assessment of clinical progression in parkinson disease

The Open Access Series of Imaging Studies (OASIS)

• cross-sectional collection of 416 subjects aged 18 to 96
• For each subject, 3 or 4 individual T1-weighted MRI scans obtained in single scan sessions are included
• longitudinal collection of 150 subjects aged 60 to 96. Each subject was scanned on two or more visits, separated by at least one year for a total of 373 imaging sessions

Human Connectome Project

Human Connectome Project

S - 1200 subjects

S - HQ data 3T(Skyra HCP) i 7T

W - not everything can be done in one go

O - two day neuropsych evaluation

T - multiple sites interested

​

Big Data - computing needs

• cloud-based tools
• nitrc type repositories
• Virtual research units

Your Role in Clinical Science

• Question scientific results
• Learn methods
• Get the raw data and analyze it yourself
• Replicate previous findings and publish it

R

• Open and free www.r-project.org
• Numerous solutions to common and uncommon biomedical problems
• Collaborative learning (Coursera & Edx)
• Packages with examples
• Publishable scripts enable reproducible research and literate programming

Sweave

Survival analysis

Survival data is all around you

Neurodebian

• Virtual computing environment for brain image processing
• Multiple software packages
• OS independent
• Data-rich
• Practical tutorials

Ideas for Practical Implementation

• Get involved in reproducibility project
• Learn open science skills (stats, image processing)
• Focus on clinical problems (hard outcomes - survival, incidence)
• Publish Open Access paper for free (Springer Open Choice)

Springer Open-Choice for Poland

Number of open access publications

BMC Medicine 2012, 10:124  doi 10.1186/1741-7015-10-124

​

Nationale Kohorte

Open Science Prize

Summary

• Active participation in science is critical for Clinician Scientists
• Tools are open and data is free
• Ideas for research come from real clinical problems
• Intro to Research elective course

mpawlak@ump.edu.pl

References

References

References

References