Introduction to
CCP-EM
Colin Palmer
S2C2 CCP-EM workshop
10 Nov 2020
Collaborative Computational Project for Electron cryo-Microscopy
Support users and developers in computational aspects of biological EM
Hosted by STFC Rutherford Appleton Laboratory
Alongside CCP4 core team – shared expertise between projects
What is CCP-EM?
CCP-EM & CCP4 | RCaH
eBIC | DLS
Tom
Burnley
Martyn
Winn
Colin
Palmer
Agnel
Joseph
Jola
Mirecka
Matt Iadanza
Collaborative Computational Project for cryo-EM
Thanks to the many people helping to teach this workshop!
CCP-EM training workshops
Previously…
CCP-EM training workshops
Now
CCP-EM training workshops
Now
Our first virtual workshop
Please be patient!
At our in-person workshops we normally teach about 20 people. Now we have 250…
Great to reach so many people, but we don’t have capacity to give individual attention
Setup for tutorial participants
Software
Should all be installed now
If problems, raise in Q&A during tutorials
Data
Please download from https://www.ccpem.ac.uk/training/s2c2_workshop_2020/s2c2_workshop_2020.php
Suite of utilities for EM data processing
Common Python framework
Uses some CCP4 programs
Download from ccpem.ac.uk
Linux & Mac
Free for academic use, fee for commercial
Bugs & requests:
ccpem@stfc.ac.uk
CCP-EM software suite
CCP-EM website
Documentation
Tutorials and lectures
CCP-EM mailing list
Support for CCP-EM and RELION
CCP-EM software – more information
CCP-EM Spring Symposium
Annual UK cryo-EM conference
Talks on all aspects of cryo-EM including software
Recordings on YouTube (search “CCP-EM”)
Proceedings in Acta Cryst D
CCP-EM software – more information
SBGrid webinars
See SBGrid website or YouTube
CCP-EM papers
Collaborative Computational Project for Electron cryo-Microscopy. Acta Cryst. D71:123–126 (2015)
Recent developments in the CCP-EM software suite. Acta Cryst. D73:469–477 (2017)
CCP-EM software – more information
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
This workshop
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
State-of-the-art software for Single Particle Reconstruction
CCP-EM includes pre-compiled RELION binaries for Linux and Mac
CUDA GPU support on Linux
v3.1 in CCP-EM 1.5
Plan to integrate RELION more closely with the rest of the suite in future
RELION
Sjors Scheres
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
Maps from cryo-EM look like maps from X-ray crystallography...
EM maps are different
EM 3.3Å
MX 3.4Å
20S proteasome
Li et al, Nature Methods, 2013
Maps from cryo-EM look like maps from X-ray crystallography...
...but some important differences:
Crystallography software (e.g. CCP4) can often handle cryo-EM maps, but needs care
EM maps are different
MRC to MTZ
Global map sharpening / blurring
Try an array of sharpening factors
Map sharpening
MRC to MTZ
Global map sharpening / blurring
Try an array of sharpening factors
Map sharpening
Garib Murshudov
MRC to MTZ
Global map sharpening / blurring
Try an array of sharpening factors
Visualise multiple sharpened / blurred maps in Coot
Expect local variation...
Map sharpening
Rob Nicholls
Locally adaptive map sharpening
Fits experimental map to local amplitude profile from atomic model B-factors
Requires a refined model (for now!)
Iterative process of model building and map improvement
LocScale
Arjen
Jakobi
Jakobi et al. eLife (2017) 6:e27131
LocScale
Arjen
Jakobi
Jakobi et al. eLife (2017) 6:e27131
Locally adaptive map sharpening
Fits experimental map to local amplitude profile from atomic model B-factors
Requires a refined model (for now!)
Iterative process of model building and map improvement
Local Agreement Filter for Transmission EM Reconstructions
Compares band-passed half maps to identify locally-shared features
Preserves shared signal, suppresses noise
LAFTER
Ramlaul, Palmer & Aylett (2019) J. Struct. Biol 205:30–40
Chris Aylett
Local Agreement Filter for Transmission EM Reconstructions
Compares band-passed half maps to identify locally-shared features
Preserves shared signal, suppresses noise
High contour: strong features remain similar
Low contour: weak noise features are removed
LAFTER
Ramlaul, Palmer & Aylett (2019) J. Struct. Biol 205:30–40
Chris Aylett
High contour
Low contour
Original EMD-2847
LAFTER filtered
Applies multiple hypothesis testing to cryo-EM maps
p-values adjusted for control of False Discovery Rate
Voxel values give a measure of confidence that we can discriminate signal from noise
At a threshold of 0.99 (1% FDR), at least 99% of the voxels truly indicate positive density signal in the map
Confidence Maps
Beckers, Jakobi & Sachse (2019)
Max Beckers
Paper in Acta D
Practical guidance for use and interpretation
doi:10.1107/S2059798320002995
Confidence Maps
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
Rigid-body fitting
Alexei
Vagin
MOLREP
Fast docking of molecular models
High resolution EM maps
Find multiple copies
Reference to target sequence correction
Rigid-body fitting
Alan Roseman
Dock-EM
Docking atomic models at medium to low resolution
Exhaustive 6D rigid body search
Target region of interest
Solutions ranked by cross-correlation coefficient (CCC)
View best hits in Chimera
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
Automated model building in high-resolution maps (<4Å)
Buccaneer: amino acid
Nautilus: nucleic acid
Requires map and sequence
Buccaneer & Nautilus
Kevin Cowtan
Scott Hoh
Find Cα seed positions
Grow chain fragments
Join overlapping fragments
Link adjacent fragments, assign & correct sequence, filter poor-quality fragments
Prune inconsistent fragments
Rebuild side chains
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
Flexible fitting
Agnel Joseph
Flex-EM
Model refinement at medium resolutions
Flexible fitting of rigid body domains to EM maps
Real space MD refinement
Rigid bodies detected based on clusters of secondary structure elements
Maya Topf
REFMAC
Garib Murshudov
Model refinement at high resolution (<~5Å)
Aims:
Automatic handling of EM maps:
Global or local refinement modes
Additional restraints
Oleg Kovalevskiy
Rob Nicholls
+
Data
Atomic model
Fit and refine
REFMAC
More information:
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
CCP-EM Coot built from “refinement” branch – v0.9.2
Now on both Mac and Linux
New features for cryo-EM
Coot
Paul Emsley
CCP-EM workflow
Single Particle Reconstruction
Map Optimisation
Docking /�Model Building
Automated Refinement
Buccaneer
Flex-EM
Molrep
Coot
Chimera
MRC to MTZ
LocScale
Dock-EM
RELION
Confidence Maps
Interactive Refinement
LAFTER
Refmac
Validation
Model Validation
What leads to overfitting?
Need for validation
Garib Murshudov
Another observation common to almost all the deposited models based on high-resolution maps is that they seem to lack the final quality control. The presence of very doubtful multiple conformations of the side chains, poor geometry of the model in comparatively clear regions of the maps, location of the side chains outside of the clear density, or the occurrence of interatomic clashes may indicate the difficulty of manual inspection of these very large structures....
Nevertheless, more attention needs to be paid to such problems that are not easily solved by purely automated means.
Need for validation
Wlodawer et al., (2017) Structure 25:1589
Validation for cryo-EM is still immature and developing rapidly
We need better metrics and better education – but the situation is improving
Beware of errors in models from public databases!
Check important parts yourself
Please deposit your data!
Automated REFMAC half map validation pipeline
Checks for over-fitting in refinement
Requires 3 input maps:
Refinement protocol performed twice:
Half-map Validation
Half-map Validation
FSCwork: model refined against half map 1; compared to half map 1
FSCfree: model refined against half map 1; compared to half map 2
Not over-fitted
Over-fitted
CCP-EM task to run multiple validation metrics:
To-do list of residues to check
Problems grouped in clusters
Model Validation
Agnel Joseph
WT Validation Symposium: 18 – 20th Nov
Other recent additions to CCP-EM | ||
cryoEF v1.1
| Quantify spread of particle angle distribution, recommends tilt angles to minimize bias | Naydenova K & Russo CJ. Nat Commun (2017) |
SIDESPLITTER | Binary and wrapper to work with RELION 3.1, Local filtering approach to mitigate local overfitting | Ramlaul K et al. J Struct Biol (2020) |
Haruspex | Deep learning approach to identify secondary structures in maps | Thorn A et al. Angewandte Chemie (2020) |
Difference maps (TEMPy and LocScale)
| Global/local scaling based map-model difference | Joseph AP et al. JCIM (2019) |
EMDA
| Python library for interpretation of multiple maps and models, Local correlation and difference calculation | Rangana W and Garib M, MRC-LMB |
Recent developments
SIDESPLITTER
Modification of LAFTER algorithm to maintain strict half-set separation
Reduces local over-fitting in refinement
Can help a lot when local resolution is highly variable
See Chris Aylett’s talk from the Spring Symposium
Published recently: doi:10.1016/j.jsb.2020.107545
Ramlaul, Palmer & Aylett (2019) bioRxiv
Difference Maps
Difference calculation with amplitude scaling
Published recently: doi:10.1021/acs.jcim.9b01103
Joseph et al. (2020) JCIM
Haruspex
Annotation of secondary structures in maps by deep learning
See Andrea Thorn’s lecture
Published recently: doi:10.1002/anie.202000421
Mostosi et al. (2020) Angew. Chem. Int. Ed. 59:2–10
CCP-EM core team
CCP4 core team
STFC SCD
DLS / eBIC staff
Birkbeck
Acknowledgements
Imperial College London
Francis Crick Institute
EBI
MRC-LMB
University of York
University of Manchester
TU Delft
EMBL / FZ Jülich
University of Würzburg
Setup for tutorial participants
Reminder
Please download the tutorial data from https://www.ccpem.ac.uk/training/s2c2_workshop_2020/s2c2_workshop_2020.php