Review of HW&SW Deep-Learning projects in RTC-Seville

​

Alejandro Linares Barranco

Neuromorphic Processor Project (NPP)

2015-2018 phase1, 2018-2020 phase2

NPP Project Goals^*

1. To develop the theory and practice of training reduced-precision ANNs (analog deep neural networks) and accurate SNNs (spiking neural networks) with lower computational costs�
2. To develop ANN and SNN hardware architectures aimed for SoC integration�
3. To develop live demonstrations of visual and auditory processing by the ANN and SNN processors running on FPGAs

In NPP, ANNs and SNNs have been combined to implement convolutional and recurrent neural network theory and hardware that uses sparse and change-driven computing

* From kickoff meeting May 2015

Y Bengio, M Courbariaux

J Seo

R Manohar

T Delbruck SC Liu, �G Indiveri

F Corradi

ES Shim, SJ Suh, JH Lee

B Barranco, A Linares

NPP Team

MNIST on Stratix-V DE5 platform with Altera OpenCL SDK

Implementation and Performance study

​

Alejandro (USE) & Jae-sun Seo (ASU)

MNIST ConvNet architecture under evaluation. Similar to Le-Net proposed in 1998 by Lecun.

28x28 20@24x24 20@12x12 50@8x8 50@4x4 500 10

MNIST ConvNet architecture from Caffe http://caffe.berkeleyvision.org/

Caffe exports OpenCL for GP-GPUs. Easy to adapt to FPGAs.

Developed with AOCL in 2015

Today OpenVINO Toolkit works with Caffe & Tensorflow

Demo with MNIST and Altera DE5 platform

• ~34ms / frame (~11ms on FPGA)

​

NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps

Roshambo demonstration on Xilinx PSoC

​

Alejandro Linares-Barranco

Antonio Rios-Navarro

IEEE-TNN early access: https://doi.org/10.1109/TNNLS.2018.2852335

RoShamBo training images

RoShamBo CNN architecture

CNN Hw accelerator: NullHop main features

1. Compressed layers are stored using sparsity map

4. Zero pixel MACs are completely skipped

2. Pixels are loaded only once per 128 output feature maps

5. Kernels for layer are loaded to MAC SRAM banks

7. Output maps are computed in parallel (up to 128)

8. 2x2 max pooling “on the fly” cuts external DRAM writes by 4X

9. Compressed layer is written out ready for being streamed back

6. Controllers cluster MAC units for 16-128 output maps/pass

3. Pixel & channel ordering scales to arbitrary image size

FPGA infrastructure for NullHop testing and demonstration

Xilinx Zynq-7100 PSoC from a MMP AVnet module. Motherboard developed by RTC

Demo latencies of Roshambo CNN on NullHop. NIPS-2016

Deep-Learning applied to diagnostic assistance on Prostate Cancer

Tejido sano

(TS)

Gleason Grado 3

(G3)

Gleason Grado 4

(G4)

Gleason Grado 5

(G5)

WSI

Whole Slide Tissue Image

CNN-based methodology

DataSet collected from real tissue from Valme Hospital:

Dataset

​

WSI

​

​

​

I. Rotations

II. Brightness changes

III. Focus changes

TRAIN

X processed images

Labels file

TEST

Y processed images (from different sources of those of X)

Labels file

Always in process!

​

Artificial image processing for dataset increment

​

Resize

RoI extractions (TS, G3, G4 y G5)

CNN (LeNet, AlexNet, ResNet…)

Preliminary results

Other on-going projects:

• Apply FPGA CNN accelerators to mobile robots navigation
• Breast Cancer: Modified published CNN (ResNet-50 and others)
• Brain Glioma: SPP-Net: Spatial Pyramid Pooling, with Fuzzy C-means pre-pr.
• Polyps’ detection in colonoscopy: Faster-RCNN, MICCAI’18 challenge’s DataSet
• Glaucoma: U-NET, public DataSet