1 of 13

Good Enough Practices for Scientific Computing

Pankaj Pandey

​

SciPy.in 2016

IIT Bombay

Enthought

2 of 13

Overview

  • Tools and techniques for accessibility and efficiency of research
  • Tried and tested by various researchers and continued adoption
  • Software Carpentry, Data Carpentry

3 of 13

Why?

  • Good enough practices (not Best?)
  • For future self and others
  • Quality and reproducibility

4 of 13

Aspects

  • Handling data
  • Software practices
  • Collaboration
  • Organizing the work
  • Version control
  • Report and paper manuscripts

5 of 13

Data Handling

  • Save Raw Data
  • Understandable: variable names, file format, filename
  • Analysis friendly data
  • Record data processing: script/document
  • Share data

6 of 13

Software Practices

  • Document files and programs
  • Write code in functions (no global variabes)
  • Remove duplication/reuse existing software
  • Meaningful names
  • List dependencies (requirements.txt)
  • Use control flow not comments
  • Example running code
  • Share code

7 of 13

Collaboration

  • Overview of project
  • TODO, project plan, roadmap
  • License it for reuse
  • Make it citable
  • Collaboration tools
    • software repositories
    • document collaboration

8 of 13

Project organization

  • One project one directory
  • Separate text, raw data, results, code, external scripts and programs
  • Name files appropriately

9 of 13

Tracking Changes

  • Version control: git, github
  • Documents: Google drive, Dropbox
  • Frequent small changes
  • Share frequently: Avoid emails with attachments
  • Frequent backups of everything

10 of 13

Manuscripts and Reports

  • Single master copy
  • Rich text editing: Google docs, MS office
  • Git for plain text formats: Latex, markdown
  • Publish supplementary materials like tables and figures other than just embedded pdf formats

11 of 13

Summary

  • Share everything needed for reproducing
  • Track changes
  • Tools to make collaboration easier
  • Make your work reproducible

12 of 13

Further reading

  • http://software-carpentry.org/lessons/
  • http://datacarpentry.org/
  • https://swcarpentry.github.io/good-enough-practices-in-scientific-computing/
  • Best Practices: http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001745

13 of 13

Thank you!

​

Questions?

@pankaj_tweet