Server Storage

Fabrics

Networks

Arrays

Discs

1

Connecting Servers to Fast Storage

TECH SAVVY PRODUCTIONS

Created by:

Lowell Vanderpool & Nathan Vanderpool

Source for much of this presentation came from https:\\www.SNIA.org

Server Storage take aways

• The more advanced the storage the more complex
• There is no cheap server storage solutions, very $$$$$$
• InfiniBand with NVMe is the king of speed
• Latency is the enemy of storage
• NVMe is radically forcing changes in enterprise/data center storage
• HPE and Dell and many smaller storage vendors offer incredible storage options at mind numbing prices
• Storage technology has a complex set of connectors, cables, terminology making a difficult subject to learn
• www.SNIA.org is the go-to source for all storage

​

3

Most Enterprise apps demand storage speed

4

5

Enterprise Applications driving faster and ever-expanding storage

• Enterprise Resource Planning (ERP)
• Databases
• Customer Relationship Management (CRM)
• Real Time Analytics
• Productivity application
• Collaboration Apps

Two massive problems facing server admins

Massive data storage needs (Petabytes)

Fast storage hardware subsystem (as fast as DAS)

6

7

When it comes to server storage

Get ready to spend some serious cash

8

Hardware subsystems

PC/Laptop/Server

9

IU rack storage holds 34 3.5inch 14TB drives = 476TB��Google=Gmail/Drive/Photos�Facebook�Microsoft=Outlook.com

10

Today’s 2U rack servers

• 2 AMD EPYC CPU each with 64 cores
• 128GB of RAM
• SAS RAID controller
• 24 1.8TB 10K rpm storage disks
• $48,000.00
• Powerful compute device

11

Expanding DAS

• Using 12-24Gbs SAS external connectors you can expand your server’s DAS storage but becomes a serious single point of failure for applications running on server

12

Shared Storage, failover and redundancy�

13

Local server storage: many problems for the admin

14

Server local storage

• DAS Direct Attached Storage
• Fast PCIe 3.0-4.0
• Low latency (ex: NVMe 80usec)

• Most server vendors provide expandable DAS storage
• Single point of failure
• Expensive to design failover clusters

15

Shared or Pooled Storage

SAN

16

When storage is remote

• Design your network carefully
• Standard Ethernet Switches will add too much latency
• 25-50Gbe to storage arrays

17

NVMe storage arrays demands network bandwidth

18

Fast Remote Storage requires thoughtful design

19

HBA: What is an HBA?

20

HBA/HCA Host Bus (Channel) Adapters

• HBA is simply a storage controller, most PCs/Laptops/Servers have a storage controller built-in to the motherboard
• Servers using DAS you will have to purchase an optional storage controllers which will have RAID functionally
• Broadcom MegaRAID 9500 series $1,014
• Servers connecting to a FC SAN will simply have a FC storage controller
• Dell SANblade 2 port controller $1,490.00
• Servers connecting to a Infiniband SAN will have a HCA controller
• Mellanox 200Gb PCIe 4.0 x16 $1,652.00

​

21

HBA/HCA Host Bus (Channel) Adapters

• Servers connecting to iSCSI storage an IO iSCSI controller adapter.
• A Dell 10G iSCSI controller $3,012.30
• Windows 10-11 has a free iSCSI controller built-in
• Servers connecting to NSA storage do not need a controller but just a NIC
• The storage controller is built into the NSA storage array

​

22

Remote server storage

• Expandable
• Flexible (FC/ethernet/InfiniBand)
• Can be shared between servers (SAN)
• Simplest definition for Storage Area Network
• Resilient to failure

• Can add latency
• Potential slow bandwidth
• More complex to manage (storage management software)
• Very expensive

23

Popular networks between servers and storage:

• InfiniBand
• Fibre Channel
• Ethernet

​

• Storage admins call these networks: fabrics

24

HP’s ethernet storage fabric

25

Storage Fabrics include protocols

26

InfiniBand fabric

27

InfiniBand: Best bandwidth low latency

28

Source SNAI.org

Fibre Channel

29

Fibre Channel: Enterprise workhorse

• Well established in the enterprise
• Fast
• Requires skilled admins
• Complex
• Typically, special switches for redundancy
• SAN commonly use FC

30

Who supports Fibre channel?

31

Connecting to storage

Ethernet

32

Ethernet: Fast, cheap, but latency

• Ethernet is growing in use
• Fast
• iSCSI, FCoE, NMVe/TCP, RoCE (RDMA over Converged Ethernet)
• Cheaper
• When passing through layer 2-3 switches latency can be unacceptable (“store and forward switches” can add too much latency)

33

Use of Ethernet by storage

• iSCSI SCSI commands are encapsulated in ethernet frames
• routable
• FCoE the FC frame is encapsulated in the payload of ethernet
• Not routable, works well in a LAN
• CNA converged network adapters work well
• Used in Data Center SAN
• NVMe/TCP much like
• FC-NVMe

​

34

Types of remote storage

• NAS typically slower and less expandable
• Dependent on Linux based file systems and RAID
• SAN shared storage
• Complex
• Failover
• Resilient
• Expensive
• Expandable
• Storage as a Service (Google, AWS, Azure, and many more)
• Expensive, higher latency

​

​

​

​

35

What is RDMA

Pushing storage performance

36

What is RDMA?

• Remote Direct Memory Access - provides the ability to perform a direct memory access (DMA) from one computer into to another without involving either one's OS/CPU.
• created in 1999 (implementations: infiniband, RoCE, iWARP)
• Main characteristics:
• High Bandwidth
• Low latency
• Zero copy (CPU offload) – Hardware based data transfers
• Kernel bypass – Direct access to HW for user-level applications
• QoS
• Asynchronous transactions

37

RDMA across the IP = iWARP

38

Server Storage media

• SATA Drives
• Cheap, slow, large storage space, very good archive storage, needs either hardware RAID or software RAID, 3.5 inch is preferred form factor
• Think of SATA as a 2 lane highway
• SAS Drives
• Expensive, medium performance, medium storage capacity, 15Krpm, heat and power dissipation high, needs either hardware RAID or software RAID, 2.5 inch is preferred form factor
• Think of SAS as a 4 lane highway

​

39

Need a refresher on RAID?

40

Server Storage media

• NVMe
• Expensive, fast, low storage capacity, low power and heat dissipation, RAID can boost performance almost to RAM speeds
• Many enterprise form factors: M.2 consumer
• Think of NVMe as a 12 lane highway

​

41

42

43

Source SNAI.org

Media compared

44

Enterprise NVMe: many form factors

45

Server Storage media

• Ethernet SSDs
• Directly connects to 10Ge switch, Open-Source storage management
• No single point of failure, expandable, massive storage capability

​

46

Ethernet SSDs

47

This chassis provides power and cooling and for each SSD a direct connection to a 10G ethernet switch port

Persistent Memory

An exploding trend in enterprise application for fast storage

48

When flash reaches speeds near DRAM chips

49

• Server based applications love RAM
• Turning flash storage into RAM is called persistent memory

​

How to store data? Blocks, Files vs Objects

• Block storage
• Blocks of data
• Databases
• SAN storage

​

​

50

How to store data? Blocks, Files vs Objects

• File storage
• Very familiar to most of us
• File systems
• Hierarchical structure
• folders
• Network Attached Storage (NAS)

​

​

​

51

How to store data? Blocks, Files vs Objects

• A growing trend in container applications is to use object storage
• Azure calls object storage: Blobs
• Developers can use a simple HTTP API, with clients available for all major operating systems and programming languages
• A cost structure that means you only pay for what you use
• Using an object storage service means you don’t have to maintain hard drives and RAID arrays, as that’s handled by Google/AWS/Azure

​

52

Microsoft’s Storage Technologies

• Storage Spaces Direct: driving down the costs of expensive SAN or NAS arrays
• Shipped with Server 2016
• The storage is:
• Highly available
• Scalable (add more local storage to any server or simply add another server)
• Software defined storage
• At a fraction of the cost of typical SAN or NAS arrays

53

How does it work?

• Allows local storage on two or more servers to be clustered, shared and defined by Storage Spaces

​

54

• All the servers will see and use the shared storage in the cluster
• The key to success is very fast ethernet and low latency between servers

​

55

• Server Admin can create volumes and assign to VM or containers running on any of the servers

​

56

Want to grow?

57

Expand more local storage

Add more servers to the cluster

CONTACT US�mrvanderpool@techsavvyproductions.com�nathan@techsavvyproductions.com�

SOCIAL MEDIA

• YouTube: https://www.youtube.com/user/vanderl2796/featured

​

• Twitter: @_TechSavvyTeam
• Facebook: https://www.facebook.com/Tech-Savvy-Productions-105287381500897
• Follow on Instagram: techsavvyproductions
• https://www.instagram.com/techsavvyproductions/

​

• Mr.V: https://www.linkedin.com/in/lowell-vanderpool-57970623/
• Nathan: https://www.linkedin.com/in/nathan-vanderpool-50a27822/

CREDITS

• Social media logos and “Tech Savvy Productions” teaser created by The 11th Hour: https://www.youtube.com/user/The11thH...

​

58

Become a member

59

Check out our website: �https://www.techsavvyproductions.com

60

Want an easy and free way to support this Channel?�Please Subscribe!�80% of the individuals who watch our content do not subscribe.�

61

We have subtitles for the following languages:

We translate subtitles on our videos into the following languages: عربى, българскиB, 简体中文）, 中國傳統的）, Nederlands, Suomalainen, Pilipino, français, Deutsche, हिंदी , Magyar, bahasa Indonesia, 日本語, 한국어, norsk, Polskie, português, Română, русский, Española, Kiswahili, Svenska, and Tiếng Việt

​

62

�A BIG THANK YOU TO ALL OF OUR VIEWERS AND SUBSCRIBERS!�

63