1 of 32

XBrainLab

1

The Cognitive Engineering and Computational Neuroscience Lab (CECNL)

National Yang Ming Chiao Tung University (NYCU), Taiwan

23/12/16 NeurIPS AI4Science workshop

2 of 32

About

Github: link

OpenReview: link

Contact us: cecnl@nctu.edu.tw

2

3 of 32

XBrainLab

  • EEG decoding using ML with graphical user interface & code scripting
    • Set up customized experiment setting
    • Train with built-in models or insert your own models
    • Code-free visualization

​

  • Schematic Flow of XBrainLab

​

3

4 of 32

XBrainLab

  • Dashboard overview:
    • Dataset Info
    • Preprocess History
    • Training Scheme
    • Training Setting
    • Training Status

4

5 of 32

Use case demo

  • BCI Competition IV 2a
    • motor imagery EEG
    • 4-class classification (left hand, right hand, feet, tongue)
    • 22 EEG channels and 3 EOG channels, 9 subjects w/ 2 sessions each
  • Built-in CNN model
    • SCCNet

5

C. -S. Wei, T. Koike-Akino and Y. Wang, "Spatial Component-wise Convolutional Network (SCCNet) for Motor-Imagery EEG Classification," 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), 2019, pp. 328-331, doi: 10.1109/NER.2019.8716937.

Has a square activation layer: maybe try normalizing your dataset after loading if the values are small

6 of 32

XBrainLab

  • Import Data
    • Import SET file
    • Import MAT file
    • Import EDF/EDF+/GDF file
    • Import CNT file
    • Import NPY/NPZ file

​

6

Regular Expression for parsing subject/session from filename

In this case: A0(?P<subject>[1-9])(?P<session>[T|E]).gdf

7 of 32

XBrainLab

  • Import Data
    • Import SET file
    • Import MAT file
    • Import EDF/EDF+/GDF file
    • Import CNT file
    • Import NPY/NPZ file

​

​

7

Load label files for each data loaded

(Currently has to be done manually on GUI)

8 of 32

XBrainLab

  • Import Data
    • Import SET file
    • Import MAT file
    • Import EDF/EDF+/GDF file
    • Import CNT file
    • Import NPY/NPZ file

​

​

8

After importing data

9 of 32

XBrainLab

  • Preprocessing
    • Channel Selection
    • Normalization
    • Filtering
    • Resample
    • Time/Window Epoch
    • Edit Event Name/Ids
    • Export
    • Reset

​

9

Select 22 EEG channels

10 of 32

XBrainLab

  • Preprocessing
    • Channel Selection
    • Normalization
    • Filtering
    • Resample
    • Time/Window Epoch
    • Edit Event Name/Ids
    • Export
    • Reset

​

10

11 of 32

XBrainLab

  • Preprocessing
    • Channel Selection
    • Normalization
    • Filtering
    • Resample
    • Time/Window Epoch
    • Edit Event Name/Ids
    • Export
    • Reset

​

11

Downsample to 125 Hz

12 of 32

XBrainLab

  • Preprocessing
    • Channel Selection
    • Normalization
    • Filtering
    • Resample
    • Time/Window Epoch
      • 783
      • 769, 770, 771, 772
    • Edit Event Name/Ids
    • Export
    • Reset

12

13 of 32

XBrainLab

  • Preprocessing
    • Channel Selection
    • Normalization
    • Filtering
    • Resample
    • Time/Window Epoch
    • Edit Event Name/Ids
      • 1: left, 2: right, 3: feet, 4: tongue
    • Export
    • Reset

13

Edit Event names

14 of 32

XBrainLab

  • Preprocessing
    • Channel Selection
    • Normalization
    • Filtering
    • Resample
    • Time/Window Epoch
    • Edit Event Name/Ids
    • Export
      • export as .mat files
    • Reset

​

14

After preprocessing

15 of 32

XBrainLab

  • Training
    • Data Spliting
      • Training type
        • within/cross subject
      • By ….. (independent)
        • make the split uncorrelated to the rest, should be selected in the second step
    • Model Selection
    • Training / Test Only
    • Generate Training Plan
    • Training Manager

15

Select the data splitting method

16 of 32

XBrainLab

  • Training
    • Data Spliting
    • Model Selection
    • Training / Test Only
    • Generate Training Plan
    • Training Manager

​

16

Divide by ratio/number/manual input

(Invalid message on incomplete intermediate inputs is currently normal)

17 of 32

XBrainLab

  • Training
    • Data Spliting
    • Model Selection
    • Training / Test Only
    • Generate Training Plan
    • Training Manager

​

17

“select” split result: currently under development

select a split result in the table and check for dataset contents (useful for checking class imbalance)

18 of 32

XBrainLab

  • Training
    • Data Spliting
    • Model Selection
    • Training / Test Only
    • Generate Training Plan
    • Training Manager

​

18

Load your trained weights

EEGNet: Lawhern, Vernon J., et al. "EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces." Journal of neural engineering 15.5 (2018): 056013.

SCCNet: Wei, Chun-Shu, Toshiaki Koike-Akino, and Ye Wang. "Spatial component-wise convolutional network (SCCNet) for motor-imagery EEG classification." 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, 2019.

ShallowConvNet: Schirrmeister, Robin Tibor, et al. "Deep learning with convolutional neural networks for EEG decoding and visualization." Human brain mapping 38.11 (2017): 5391-5420.

19 of 32

XBrainLab

  • Training
    • Data Spliting
    • Model Selection
    • Training / Test Only
    • Generate Training Plan
    • Training Manager

​

19

If you redo the training, previous results will be automatically backed up in the same location

20 of 32

XBrainLab

  • Training
    • Data Spliting
    • Model Selection
    • Training / Test Only
    • Generate Training Plan
    • Training Manager

​

20

21 of 32

XBrainLab

  • Training
    • Data Spliting
    • Model Selection
    • Training / Test Only
    • Generate Training Plan
    • Training Manager

​

21

Click to plot loss, accuracy, AUC or learning rate curve

The plots shows after selecting plan-repeat

22 of 32

XBrainLab

  • Evaluation
    • Confusion Matrix
    • Performance Table
    • Export Model Output (.csv)

​

22

23 of 32

XBrainLab

  • Evaluation
    • Confusion Matrix
    • Performance Table
    • Export Model Output (.csv)

​

23

24 of 32

XBrainLab

  • Evaluation
    • Confusion Matrix
    • Performance Table
    • Export Model Output (.csv)

​

24

Export model output

Export csv

25 of 32

XBrainLab

  • Visualization
    • Set Montage
    • Saliency Map
    • Saliency Topographic Map
    • Saliency spectrogram
    • 3D Saliency Plot
    • Model Summary
    • Clean Plots

​

25

Selected channels from the left is recorded in the right

26 of 32

XBrainLab

  • Visualization
    • Set Montage
    • Saliency Map
      • channel by time
    • Saliency Topographic Map
    • Saliency Spectrogram
    • 3D Saliency Plot
    • Model Summary
    • Clean Plots

​

26

27 of 32

XBrainLab

  • Visualization
    • Set Montage
    • Saliency Map
    • Saliency Topographic Map
      • channels
    • Saliency Spectrogram
    • 3D Saliency Plot
    • Model Summary
    • Clean Plots

​

27

28 of 32

XBrainLab

  • Visualization
    • Set Montage
    • Saliency Map
    • Saliency Topographic Map
    • Saliency Spectrogram
    • 3D Saliency Plot
    • Model Summary
    • Clean Plots

​

28

29 of 32

XBrainLab

  • Visualization
    • Set Montage
    • Saliency Map
    • Saliency Topographic Map
    • Saliency Spectrogram
    • 3D Saliency Plot
    • Model Summary
    • Clean Plots

​

29

Spin the head around and toggle timestamp bar for observation

plots 3d topo

for a specific plan-repeat-class

30 of 32

XBrainLab

  • Visualization
    • Set Montage
    • Saliency Map
    • Saliency Topographic Map
    • Saliency Spectrogram
    • 3D Saliency Plot
    • Model Summary
    • Clean Plots

​

30

31 of 32

XBrainLab

  • Script
    • Show Command Script
    • Show UI Script
    • Show All Script
    • Clear Script

​

31

Save or copy the script

* Add “lab.show_ui()” at the end of the code, to show GUI window for the finished running script.

32 of 32

XBrainLab

  • Customize XBrainLab GUI with your own DL model
    • put .py file in the model_base folder
      • from “pip install –upgrade git+...”: ~\anaconda3\envs\<env_name>\Lib\site-packages\XBrainLab\model_base
      • from git clone: ~\XBrainLab\model_base
    • common input arguments (Hidden in GUI)
      • n_classes, channels, samples, sfreq
    • custom input arguments (Entries appear in GUI)
      • EEGNet for example: F1, F2, D

32