1 of 57

VMware Backup: �Fast, No load, Direct

Transparent Snapshots

Daniel.Olkowski@dell.com

Video describing fast backup of VMware:�https://youtu.be/LETxuJZXnuA �Article with this presentation:

http://backuprecoveryguy.blogspot.com/2022/07/fast-backup-of-files.html

© Copyright 2020 Dell Inc.

1

of 20

Internal Use - Confidential

2 of 57

Agenda

  • VMware backup problems / VADP architecture
  • Transparent Snapshots
  • Architecture
  • Value
  • Requirements
  • Comparison
  • Licensing
  • Demo

© Copyright 2020 Dell Inc.

2

of 20

Internal Use - Confidential

3 of 57

VMware backup problems�VADP architecture

© Copyright 2020 Dell Inc.

3

of 20

Internal Use - Confidential

4 of 57

VMware backup is based on VADP (vStorage API’s for Data Protection) interface since 2009

Used in almost all backup/recovery solutions

Generates lots of problems

Those problems were tried to bypass by many applications (RP4VM)

© Copyright 2020 Dell Inc.

4

of 20

Internal Use - Confidential

5 of 57

VADP Process

Backup Solution

VMDK

  1. Backup Solution tells vCenter to Snapshot VM
  2. vCenter tells ESXi Host to Snapshot VM
  3. vSphere Quiesces VM with VMTools (optional, quiesces Guest OS if possible before snapshot)
  4. Host Snapshots VM, creating Differencing Disk
  5. CBT Information is transmitted to Backup Solution (optional due to additional time for CBT info to transmit)
  6. Backup is transferred from VMware environment to Backup Solution
  7. Backup Application signals completion to vCenter
  8. vSphere consolidates snapshot, merging differencing disk back into main VMDK

​

Backup Complete!

​

①

②

③

④ 

 ⑤ 

Diff. Disk

④ 

⑥ 

⑦

⑧

© Copyright 2020 Dell Inc.

5

of 20

Internal Use - Confidential

6 of 57

Current issues of VMware backup (VADP interface)

  • Slow determination of deltas in the VM dataset�Halt time for an application longer than desired at each backup.
  • Undesirable need for resource-intensive external agents to move VM snapshot data to secondary storage
  • Longer-than-acceptable application read and write latencies during backup
  • Overly impactful application IOPS reduction during backup

© Copyright 2020 Dell Inc.

6

of 20

Internal Use - Confidential

7 of 57

Current issues of VMware backup (VADP interface)

  • Slow determination of deltas in the VM dataset�Halt time for an application longer than desired at each backup.
  • Undesirable need for resource-intensive external agents to move VM snapshot data to secondary storage
  • Longer-than-acceptable application read and write latencies during backup
  • Overly impactful application IOPS reduction during backup

Slow business appliacations�Problems of users

© Copyright 2020 Dell Inc.

7

of 20

Internal Use - Confidential

8 of 57

Current issues of VMware backup (VADP interface)

  • Slow determination of deltas in the VM dataset�Halt time for an application longer than desired at each backup.
  • Undesirable need for resource-intensive external agents to move VM snapshot data to secondary storage
  • Longer-than-acceptable application read and write latencies during backup
  • Overly impactful application IOPS reduction during backup

VADP tried to solve some issues (f.e. CBT) but VADP-based solutions continue to struggle with latency and IOPS issues due to their need for external data movers, outside the ESX host.

© Copyright 2020 Dell Inc.

8

of 20

Internal Use - Confidential

9 of 57

© Copyright 2020 Dell Inc.

9

of 20

Internal Use - Confidential

10 of 57

RP4VM materials

  • Video: https://youtu.be/Dqyqabfg3iM

© Copyright 2020 Dell Inc.

10

of 20

Internal Use - Confidential

11 of 57

RP4VM materiały

© Copyright 2020 Dell Inc.

11

of 20

Internal Use - Confidential

12 of 57

Transparent Snapshots

© Copyright 2020 Dell Inc.

12

of 20

Internal Use - Confidential

13 of 57

Always on

Simple

Transparent Snapshots

Near zero time

Near zero load

No VMware snapshots

No application pause

© Copyright 2020 Dell Inc.

13

of 20

Internal Use - Confidential

14 of 57

Always on

Simple

Transparent Snapshots

Near zero time

Near zero load

No VMware snapshots

No application pause

No proxies,

No additional hardware,

No cost

© Copyright 2020 Dell Inc.

14

of 20

Internal Use - Confidential

15 of 57

Fast Backup / Restore

VMware stability

Transparent Snapshots

Less resources

Lower cost of infrastructure

© Copyright 2020 Dell Inc.

15

of 20

Internal Use - Confidential

16 of 57

Architecture

© Copyright 2020 Dell Inc.

16

of 20

Internal Use - Confidential

17 of 57

LWD

ESX / Virtual Machines

0

1

1

0

0

Datastore

© Copyright 2020 Dell Inc.

17

of 20

Internal Use - Confidential

18 of 57

LWD

ESX / Virtual Machines

0

1

1

0

0

  • In memory of ESX server
  • Observers write IOs
  • Tracks changes in memory bitmap

LWD�LightWeight Delta Filter

Datastore

© Copyright 2020 Dell Inc.

18

of 20

Internal Use - Confidential

19 of 57

LWD

ESX / Virtual Machines

0

1

1

0

0

This bitmap is new type of snapshot

– Always on

– Transparent to the guest

– When backup is triggered, we know which block shall be sent to backup media

Datastore

© Copyright 2020 Dell Inc.

19

of 20

Internal Use - Confidential

20 of 57

LWD

ESX / Virtual Machines

Datastore

0

1

1

0

0

TSDM

TSDM�Transparent Snapshot Data Mover

1%

data transferred

© Copyright 2020 Dell Inc.

20

of 20

Internal Use - Confidential

21 of 57

LWD

ESX / Virtual Machines

Datastore

VMDK

0

1

1

0

0

TSDM

TSDM�Transparent Snapshot Data Mover

  • Tiny software daemon (instead of massive proxy/servers used to transport data)
  • Requires less than 700MB memory in ESX vs 8GB memory of any proxy in any backup software
  • One per ESX host regardless of VMs on the host
  • Sends data directly from ESX server to backup media (Data Domain) in just single step
  • Super fast
  • Performs source de-duplication - unique blocks are not send as whole once but only this part of unique VMware blocks that do not exist on backup media
  • We have full backup

© Copyright 2020 Dell Inc.

21

of 20

Internal Use - Confidential

22 of 57

LWD

ESX / Virtual Machines

Datastore

VMDK

0

1

1

0

0

TSDM

TSDM�Transparent Snapshot Data Mover

ESX host / datastore no more down by backup processes

© Copyright 2020 Dell Inc.

22

of 20

Internal Use - Confidential

23 of 57

LWD

ESX / Virtual Machines

Datastore

VMDK

0

1

1

0

0

TSDM

VIB (VMware Installation Bundle)

– Already signed / approved by VMware

– Automatically installed when backup policy is configured for certain VMs (ESX hosts)

VIB

© Copyright 2020 Dell Inc.

23

of 20

Internal Use - Confidential

24 of 57

. . .

. . .

. . .

. . .

PowerProtect

Data Manager

ESX / Virtual Machines

ESX / Virtual Machines

ESX / Virtual Machines

1% data transferred

PowerProtect Data Manager is single point of control and management

Datastore

Datastore

VMDE - VM Direct Engine

© Copyright 2020 Dell Inc.

24

of 20

Internal Use - Confidential

25 of 57

. . .

. . .

. . .

. . .

PowerProtect

Data Manager

ESX / Virtual Machines

ESX / Virtual Machines

ESX / Virtual Machines

1% data transferred

PowerProtect Data Manager is single point of control and management

Datastore

Datastore

  • Discover assets
  • Creates backup policies
  • Verifies requirements for Transparent Snapshots
  • Automatically deploys all required components
  • Selects mode of backup
  • Manages VIBs
  • Management of backup is performed by VMDE – VM Direct Engine
  • Single VMDE manages thousands of VMs
  • Possibility to perform traditional VADP proxy backups if requirements are fulfilled – we need traditional scaling

© Copyright 2020 Dell Inc.

25

of 20

Internal Use - Confidential

26 of 57

. . .

. . .

. . .

. . .

PowerProtect

Data Manager

ESX / Virtual Machines

ESX / Virtual Machines

ESX / Virtual Machines

1% data transferred

PowerProtect Data Manager is single point of control and management

Datastore

Datastore

VMDE - VM Direct Engine

No proxies!�Huge savings!

© Copyright 2020 Dell Inc.

26

of 20

Internal Use - Confidential

27 of 57

Value

© Copyright 2020 Dell Inc.

27

of 20

Internal Use - Confidential

28 of 57

Full resource usage

Scale? Just backup next VM

Easy! Policy can have 1 or 1000 VMS

Reduce infrastrcuture cost – no proxy

No backup limitations

Speed

Almost zero production latency

Eliminates backup windows

© Copyright 2020 Dell Inc.

28

of 20

Internal Use - Confidential

29 of 57

Hardware independent

Performance / easy

for any platform

No issues with VMware / snapshots

© Copyright 2020 Dell Inc.

29

of 20

Internal Use - Confidential

30 of 57

Requirements

© Copyright 2020 Dell Inc.

30

of 20

Internal Use - Confidential

31 of 57

. . .

. . .

. . .

. . .

PowerProtect

Data Manager 19.9+

ESXi 7.0 U3+

ESX / Virtual Machines

ESX / Virtual Machines

1% data transferred

Datastore

Datastore

VMDE - VM Direct Engine

© Copyright 2020 Dell Inc.

31

of 20

Internal Use - Confidential

32 of 57

. . .

. . .

. . .

. . .

PowerProtect

Data Manager 19.9+

ESXi 7.0 U3+

ESX / Virtual Machines

ESX / Virtual Machines

1% data transferred

Datastore

Datastore

VMDE - VM Direct Engine

Makes sense to update!

© Copyright 2020 Dell Inc.

32

of 20

Internal Use - Confidential

33 of 57

Comparison

© Copyright 2020 Dell Inc.

33

of 20

Internal Use - Confidential

34 of 57

VADP Performance impact on VM

© Copyright 2021 Dell Inc.

34

of 97

Internal Use - Confidential

35 of 57

Transparent Snapshot – Results

© Copyright 2021 Dell Inc.

35

of 97

Internal Use - Confidential

36 of 57

Licensing

© Copyright 2020 Dell Inc.

36

of 20

Internal Use - Confidential

37 of 57

Example of environment�

30 VMs

20 VMs

1 standalone

server

1 server

with VMs

Site A

Site B

1 server

with VMs

Dell - Internal Use - Confidential

37

of Y

Internal Use - Confidential

38 of 57

Example of environment�

30 VMs

20 VMs

1 standalone

server

1 server

with VMs

Site A (5TB)

Site B (1TB)

4 physical

processors

2 physical�processors

1 server

with VMs

Dell - Internal Use - Confidential

38

of Y

Internal Use - Confidential

39 of 57

Example of environment��

30 VMs

20 VMs

1 standalone

server

1 server

with VMs

Site A (5TB)

Site B (1TB)

4 physical

processors

2 physical�processors

1 server

with VMs

Disaster Recovery

1% data transfer

PPDM

vProtect

Data Domain

Virtual Edition

DDVE (6TB)

Data Domain

Virtual Edition

DDVE (6TB)

POWERPROTECT DATA MANAGER ESSENTIALS�6 physical processors

Software components to protect the environment

  • PowerProtect Data Manager (unlimited)
  • RecoverPoint for Virtual Machines (unlimited)
  • vProtect (unlimited)
  • Cloud Snapshot Recovery
  • Data Domain Virtual Edition (6TB)
  • Direct backup to Data Domain (unlimited)
  • Boost FS
  • Cyber Disaster Recovery
  • Storage Direct

Dell - Internal Use - Confidential

39

of Y

Internal Use - Confidential

40 of 57

© Copyright 2020 Dell Inc.

40

of 20

Internal Use - Confidential

41 of 57

PowerProtect Data Manager Essentials offers the same scope of capabilities as PowerProtect Data Manager Plus. Capacity is limited to 50TB or 50 sockets maximum, with aggressive license pricing. Customers who have a need for more than 50TB or sockets will be able to purchase PowerProtect Data Manager Plus using upgrade pricing at no surcharge

© Copyright 2020 Dell Inc.

41

of 20

Internal Use - Confidential

42 of 57

Demo

© Copyright 2020 Dell Inc.

42

of 20

Internal Use - Confidential

43 of 57

© Copyright 2020 Dell Inc.

43

of 20

Internal Use - Confidential

44 of 57

© Copyright 2020 Dell Inc.

44

of 20

Internal Use - Confidential

45 of 57

© Copyright 2020 Dell Inc.

45

of 20

Internal Use - Confidential

46 of 57

© Copyright 2020 Dell Inc.

46

of 20

Internal Use - Confidential

47 of 57

© Copyright 2020 Dell Inc.

47

of 20

Internal Use - Confidential

48 of 57

© Copyright 2020 Dell Inc.

48

of 20

Internal Use - Confidential

49 of 57

© Copyright 2020 Dell Inc.

49

of 20

Internal Use - Confidential

50 of 57

Materials�Data Domain

© Copyright 2020 Dell Inc.

50

of 20

Internal Use - Confidential

51 of 57

Links public

© Copyright 2020 Dell Inc.

51

of 20

Internal Use - Confidential

52 of 57

Links public

© Copyright 2020 Dell Inc.

52

of 20

Internal Use - Confidential

53 of 57

Links public

  • Performance in backup – public video: https://youtu.be/05fFtGH7YCQ

© Copyright 2020 Dell Inc.

53

of 20

Internal Use - Confidential

54 of 57

Links public

© Copyright 2020 Dell Inc.

54

of 20

Internal Use - Confidential

55 of 57

DD6400

​

​

© Copyright 2020 Dell Inc.

55

of 20

Internal Use - Confidential

56 of 57

Links public

© Copyright 2020 Dell Inc.

56

of 20

Internal Use - Confidential

57 of 57

Daniel.Olkowski@dell.com

Questions

© Copyright 2020 Dell Inc.

57

of 20

Internal Use - Confidential