2 of 41

Topics from the syllabus

IP SAN and FCoE : iSCSI, FCIP

Network Attached Storage: General-Purpose Servers versus NAS Devices, Benefits of NAS, File Systems and Network File Sharing, Components of NAS, NAS I/O Operation, NAS Implementations, NAS File-Sharing Protocols, Factors Affecting NAS Performance

3 of 41

Expected Questions from this module

With a neat diagram, explain Fiber Channel protocol stack
Explain FCIP Protocol stack
What is NAS? Explain the different components of NAS.
Write a note on iSCSI
Explain I/O consolidation using FCOE
With a neat diagram, explain the different components of FC-SAN
Wia neat diagram, explain various iSCSI topologies
Write a short note on FCOE
Explain different types of FC connectivity options in FC SAN
With a neat diagram, explain iSCSI implementation
What is NAS? Explain NAS implementation in detail
What is NAS? Explain the benefits of NAS

4 of 41

Question: Write a short note on iSCSI

iSCSI is an IP based protocol that establishes and manages connections between host and storage over IP, as shown in the figure given below.

Figure: iSCSI

5 of 41

iSCSI:

iSCSI encapsulates SCSI commands and data into an IP packet and transports them using TCP/IP.
iSCSI is widely adopted for connecting servers to storage because it is relatively inexpensive and easy to implement, especially in environments in which an FC SAN does not exist.

Components of iSCSI:

An initiator (host), target (storage or iSCSI gateway), and an IP-based network are the key iSCSI components.
If an iSCSI-capable storage array is deployed, then a host with the iSCSI initiator can directly communicate with the storage array over an IP network.
However, in an implementation that uses an existing FC array for iSCSI communication, an iSCSI gateway is used.
These devices perform the translation of IP packets to FC frames and vice versa, thereby bridging the connectivity between the IP and FC environments.

6 of 41

Question: Explain different options of iSCSI Host Connectivity

The three iSCSI host connectivity options are:

A standard NIC with software iSCSI initiator,
A TCP offload engine (TOE) NIC with software iSCSI initiator
An iSCSI HBA
A standard NIC with a software iSCSI initiator is the simplest and least expensive connectivity option. It is easy to implement because most servers come with at least one, and in many cases two, embedded NICs. It requires only a software initiator for iSCSI functionality. Because NICs provide standard IP function, encapsulation of SCSI into IP packets and decapsulation are carried out by the host CPU. This places additional overhead on the host CPU. If a standard NIC is used in heavy I/O load situations, the host CPU might become a bottleneck. TOE NIC helps reduce this burden.

7 of 41

A TOE NIC offloads TCP management functions from the host and leaves only the iSCSI functionality to the host processor. The host passes the iSCSI information to the TOE card, and the TOE card sends the information to the destination using TCP/IP. Although this solution improves performance, the iSCSI functionality is still handled by a software initiator that requires host CPU cycles.

An iSCSI HBA is capable of providing performance benefits because it offloads the entire iSCSI and TCP/IP processing from the host processor. The use of an iSCSI HBA is also the simplest way to boot hosts from a SAN environment via iSCSI. If there is no iSCSI HBA, modifications must be made to the basic operating system to boot a host from the storage devices because the NIC needs to obtain an IP address before the operating system loads. The functionality of an iSCSI HBA is similar to the functionality of an FC HBA.

8 of 41

Question: Explain different iSCSI Topologies**

Two topologies of iSCSI implementations are native and bridged.

Native topology does not have FC components. The initiators may be either directly attached to targets or connected through the IP network.
Bridged topology enables the coexistence of FC with IP by providing iSCSI-to-FC bridging functionality. For example, the initiators can exist in an IP environment while the storage remains in an FC environment.

Native iSCSI Connectivity:

FC components are not required for iSCSI connectivity if an iSCSI-enabled array is deployed.

9 of 41

In, the array has one or more iSCSI ports configured with an IP address and is connected to a standard Ethernet switch.
After an initiator is logged on to the network, it can access the available LUNs on the storage array.
A single array port can service multiple hosts or initiators as long as the array port can handle the amount of storage traffic that the hosts generate.

Bridged iSCSI Connectivity:

A bridged iSCSI implementation includes FC components in its configuration.
Figure given below illustrates iSCSI host connectivity to an FC storage array.

10 of 41

In this case, the array does not have any iSCSI ports. Therefore, an external device, called a gateway or a multiprotocol router, must be used to facilitate the communication between the iSCSI host and FC storage.
The gateway converts IP packets to FC frames and vice versa.
The bridge devices contain both FC and Ethernet ports to facilitate the communication between the FC and IP environments.
In a bridged iSCSI implementation, the iSCSI initiator is configured with the gateway’s IP address as its target destination.
On the other side, the gateway is configured as an FC initiator to the storage array.

Combining FC and Native iSCSI Connectivity:

The most common topology is a combination of FC and native iSCSI. Typically, a storage array comes with both FC and iSCSI ports that enable iSCSI and FC connectivity in the same environment, as shown in the figure given below.

11 of 41

Question: Explain iSCSI Protocol Stack

Figure given below displays a model of the iSCSI protocol layers and depicts the

encapsulation order of the SCSI commands for their delivery through a physical

carrier.

12 of 41

Figure: iSCSI Protocol Stack

13 of 41

SCSI is the command protocol that works at the application layer of the Open System Interconnection (OSI) model.
The initiators and targets use SCSI commands and responses to talk to each other.
The SCSI command descriptor blocks, data, and status messages are encapsulated into TCP/IP and transmitted across the network between the initiators and targets.
iSCSI is the session-layer protocol that initiates a reliable session between devices that recognize SCSI commands and TCP/IP.
The iSCSI session-layer interface is responsible for handling login, authentication, target discovery, and session management.
TCP is used with iSCSI at the transport layer to provide reliable transmission.
TCP controls message flow, windowing, error recovery, and retransmission.
It relies upon the network layer of the OSI model to provide global addressing and connectivity.
The Layer 2 protocols at the data link layer of this model enable node-to-node communication through a physical network.

14 of 41

iSCSI PDU:

A protocol data unit (PDU) is the basic “information unit” in the iSCSI environment.
The iSCSI initiators and targets communicate with each other using iSCSI PDUs. This communication includes establishing iSCSI connections and iSCSI sessions, performing iSCSI discovery, sending SCSI commands and data, and receiving SCSI status.
All iSCSI PDUs contain one or more header segments followed by zero or more data segments.
The PDU is then encapsulated into an IP packet to facilitate the transport.
A PDU includes the components shown in Fig 2.23.
The IP header provides packet-routing information to move the packet across a network.
The TCP header contains the information required to guarantee the packet delivery to the target.

15 of 41

The iSCSI header (basic header segment) describes how to extract SCSI commands and data for the target. iSCSI adds an optional CRC, known as the digest, to ensure datagram integrity. This is in addition to TCP checksum and Ethernet CRC.
The header and the data digests are optionally used in the PDU to validate integrity and data placement.

Figure:iSCSI PDU encapsulated in an IP packet

16 of 41

The iSCSI header (basic header segment) describes how to extract SCSI commands and data for the target. iSCSI adds an optional CRC, known as the digest, to ensure datagram integrity. This is in addition to TCP checksum and Ethernet CRC.
The header and the data digests are optionally used in the PDU to validate integrity and data placement.

iSCSI Session:

An iSCSI session is established between an initiator and a target, as shown in Fig. below.

17 of 41

A session is identified by a session ID (SSID), which includes part of an initiator ID and a target ID. The session can be intended for one of the following:
The discovery of the available targets by the initiators and the location of a specific target on a network
The normal operation of iSCSI (transferring data between initiators and targets)
There might be one or more TCP connections within each session. Each TCP connection within the session has a unique connection ID (CID).
An iSCSI session is established via the iSCSI login process. The login process is started when the initiator establishes a TCP connection with the required target either via the well-known port 3260 or a specified target port.
During the login phase, the initiator and the target authenticate each other and negotiate on various parameters.
After the login phase is successfully completed, the iSCSI session enters the full-feature phase for normal SCSI transactions. In this phase, the initiator may send SCSI commands and data to the various LUNs on the target.

18 of 41

A session is identified by a session ID (SSID), which includes part of an initiator ID The final phase of the iSCSI session is the connection termination phase, which is referred to as the logout procedure.
The initiator is responsible for commencing the logout procedure; however, the target may also prompt termination by sending an iSCSI message, indicating the occurrence of an internal error condition.
After the logout request is sent from the initiator and accepted by the target, no further request and response can be sent on that connection.

FCIP (Fibre channel over IP):

FCIP is a IP-based protocol that is used to connect distributed FC-SAN islands.
Creates virtual FC links over existing IP network that is used to transport FC data between different FC SANS.
It encapsulates FC frames into IP packet.
It provides disaster recovery solution.

19 of 41

Question: With a neat diagram, explain FCIP Protocol Stack

The FCIP protocol stack is shown in the below given figure.

20 of 41

Applications generate SCSI commands and data, which are processed by various layers of the protocol stack.
The upper layer protocol SCSI includes the SCSI driver program that executes the read-and-write commands.
Below the SCSI layer is the Fibre Channel Protocol (FCP) layer, which is simply a Fibre Channel frame whose payload is SCSI.
The FCP layer rides on top of the Fibre Channel transport layer. This enables the FC frames to run natively within a SAN fabric environment. In addition, the FC frames can be encapsulated into the IP packet and sent to a remote SAN over the IP.
The FCIP layer encapsulates the Fibre Channel frames onto the IP payload and passes them to the TCP layer (see Fig 2.28). TCP and IP are used for transporting the encapsulated information across Ethernet, wireless, or other media that support the TCP/IP traffic.
Encapsulation of FC frame into an IP packet could cause the IP packet to be fragmented when the data link cannot support the maximum transmission unit (MTU) size of an IP packet.

21 of 41

When an IP packet is fragmented, the required parts of the header must be copied by all fragments.
When a TCP packet is segmented, normal TCP operations are responsible for receiving and re-sequencing the data prior to passing it on to the FC processing portion of the device.

Question: Explain FCIP Topology

In an FCIP environment, an FCIP gateway is connected to each fabric via a standard FC connection.

22 of 41

The FCIP gateway at one end of the IP network encapsulates the FC frames into IP packets.

The gateway at the other end removes the IP wrapper and sends the FC data to the layer 2 fabric.

The fabric treats these gateways as layer 2 fabric switches.

An IP address is assigned to the port on the gateway, which is connected to an IP network. After the IP connectivity is established, the nodes in the two independent fabrics can communicate with each other.

Question: Write a short note on FCoE (Fibre Channel over Ethernet)

Data centers typically have multiple networks to handle various types of I/O traffic — for example, an Ethernet network for TCP/IP communication and an FC network for FC communication.
TCP/IP is typically used for client-server communication, data backup, infrastructure management communication, and so on.
FC is typically used for moving block-level data between storage and servers.

23 of 41

To support multiple networks, servers in a data center are equipped with multiple redundant physical network interfaces for example, multiple Ethernet and FC cards/adapters. In addition, to enable the communication, different types of networking switches and physical cabling infrastructure are implemented in data centers.
The need for two different kinds of physical network infrastructure increases the overall cost and complexity of data center operation.
Fibre Channel over Ethernet (FCoE) protocol provides consolidation of LAN and SAN traffic over a single physical interface infrastructure.
FCoE helps organizations address the challenges of having multiple discrete network infrastructures.
FCoE uses the Converged Enhanced Ethernet (CEE) link (10 Gigabit Ethernet) to send FC frames over Ethernet.

24 of 41

Figure: SAn Before before using FCOE

25 of 41

Figure: SAN after using FCOE

26 of 41

Question: Explain different Components of FCOE

The key components of FCOE are :

Converged Network Adaptors(CNA)
Cables
FCOE Switches

Converged Network Adaptors(CNA)

A CNA provides the functionality of both a standard NIC and an FC HBA in a single adapter and consolidates both types of traffic. CNA eliminates the need to deploy separate adapters and cables for FC and Ethernet communications, thereby reducing the required number of server slots and switch ports.
As shown in the following figure, a CNA contains separate modules for 10 Gigabit Ethernet, Fibre Channel, and FCoE Application Specific Integrated Circuits (ASICs). The FCoE ASIC encapsulates FC frames into Ethernet frames. One end of this ASIC is connected to 10GbE and FC ASICs for server connectivity, while the other end provides a 10GbE interface to connect to an FCoE switch.

27 of 41

Cables

There are two options available for FCoE cabling:

1. Copper based Twinax

2. standard fiber optical cables.

A Twinax cable is composed of two pairs of copper cables covered with a shielded casing. The Twinax cable can transmit data at the speed of 10 Gbps over shorter distances up to 10 meters. Twinax cables require less power and are less expensive than fi ber optic cables.

28 of 41

The Small Form Factor Pluggable Plus (SFP+) connector is the primary connector used for FCoE links and can be used with both optical and copper cables.

FCoE Switches

An FCoE switch has both Ethernet switch and Fibre Channel switch functionalities.

As shown in Fig 2.32, FCoE switch consists of:

1. Fibre Channel Forwarder (FCF),

2. Ethernet Bridge,

3. set of Ethernet ports

4. optional FC ports

The function of the FCF is to encapsulate the FC frames, received from the FC port, into the FCoE frames and also to de-encapsulate the FCoE frames, received from the Ethernet Bridge, to the FC frames.

Upon receiving the incoming traffic, the FCoE switch inspects the Ethertype (used to indicate which protocol is encapsulated in the payload of an Ethernet frame) of the incoming frames and uses that to determine the destination.

29 of 41

If the Ethertype of the frame is FCoE, the switch recognizes that the frame contains an FC payload and forwards it to the FCF. From there, the FC is extracted from the FCoE frame and transmitted to FC SAN over the FC ports.
If the Ethertype is not FCoE, the switch handles the traffic as usual Ethernet traffic and forwards it over the Ethernet ports.

Figure:FCoE switch generic architecture

30 of 41

If the Ethertype of the frame is FCoE, the switch recognizes that the frame contains an FC payload and forwards it to the FCF. From there, the FC is extracted from the FCoE frame and transmitted to FC SAN over the FC ports.
If the Ethertype is not FCoE, the switch handles the traffic as usual Ethernet traffic and forwards it over the Ethernet ports.

31 of 41

What is NAS?
NAS is an IP based dedicated, high-performance file sharing and storage device.
Enables NAS clients to share files over an IP network.
Uses network and file-sharing protocols to provide access to the file data.
Ex: Common Internet File System (CIFS) and Network File System (NFS).
Enables both UNIX and Microsoft Windows users to share the same data seamlessly.
NAS device uses its own operating system and integrated hardware and software components to meet specific file-service needs.
Its operating system is optimized for file I/O which performs better than a general-purpose server.
A NAS device can serve more clients than general-purpose servers and provide the benefit of server consolidation.

32 of 41

Components of NAS:

NAS device has two key components (as shown in Fig 2.33): NAS head and storage.
In some NAS implementations, the storage could be external to the NAS device and shared with other hosts.
NAS head includes the following components:
CPU and memory
One or more network interface cards (NICs), which provide connectivity to the client network.
An optimized operating system for managing the NAS functionality. It translates file-level requests into block-storage requests and further converts the data supplied at the block level to file data
NFS, CIFS, and other protocols for file sharing
Industry-standard storage protocols and ports to connect and manage physical disk resources
The NAS environment includes clients accessing a NAS device over an IP network using file-sharing protocols.

33 of 41

Question: Explain NAS I/O Operation

NAS provides file-level data access to its clients. File I/O is a high-level request that specifies the file to be accessed.
Eg: a client may request a file by specifying its name, location, or other attributes. The NAS operating system keeps track of the location of files on the disk volume and converts client file I/O into block-level I/O to retrieve data.

34 of 41

The process of handling I/Os in a NAS environment is as follows:

1. The requestor (client) packages an I/O request into TCP/IP and forwards it through the network stack. The NAS device receives this request from the network.

2. The NAS device converts the I/O request into an appropriate physical storage request, which is a block-level I/O, and then performs the operation on the physical storage.

NAS I/O Operation

35 of 41

3. When the NAS device receives data from the storage, it processes and repackages the data into an appropriate file protocol response.

4. The NAS device packages this response into TCP/IP again and forwards it to the client through the network.

Question: Explain different NAS File Sharing Protocols:

NAS devices support multiple file-service protocols to handle file I/O requests

Two common NAS file sharing protocols are:

Common Internet File System (CIFS)
Network File System (NFS)

NAS devices enable users to share file data across different operating environments
It provides a means for users to migrate transparently from one operating system to another.

Network File System (NFS):

NFS is a client-server protocol for file sharing that is commonly used on UNIX systems.

NFS was originally based on the connectionless User Datagram Protocol (UDP).

36 of 41

It uses Remote Procedure Call (RPC) as a method of inter-process communication between two computers.
The NFS protocol provides a set of RPCs to access a remote file system for the following operations:

Searching files and directories
Opening, reading, writing to, and closing a file
Changing file attributes
Modifying file links and directories

NFS creates a connection between the client and the remote system to transfer data.
It does not maintain any kind of table to store information about open files and associated pointers. Each call provides a full set of arguments - a file handle, a particular position to read or write, and the versions of NFS - to access files on the server .

Currently, three versions of NFS are in use:

1. NFS version 2 (NFSv2): Uses UDP to provide a stateless network connection between a client and a server. Features, such as locking, are handled outside the protocol.

37 of 41

2. NFS version 3 (NFSv3): Uses UDP or TCP, and is based on the stateless protocol design. It includes some new features, such as a 64-bit file size, asynchronous writes, and additional file attributes to reduce refetching.

3. NFS version 4 (NFSv4): Uses TCP and is based on a stateful protocol design. It offers enhanced security. The latest NFS version 4.1 is the enhancement of NFSv4 and includes some new features, such as session model, parallel NFS (pNFS), and data retention.

Common Internet File System (CIFS):

CIFS is a client-server application protocol
It enables clients to access files and services on remote computers over TCP/IP.
It is a public, or open, variation of Server Message Block (SMB) protocol.
It provides following features to ensure data integrity:

It uses file and record locking to prevent users from overwriting the work of another user on a file or a record.
It supports fault tolerance and can automatically restore connections and reopen files that were open prior to an interruption. This feature depends on whether an application is written to take advantage of this.

38 of 41

CIFS is a stateful protocol because the CIFS server maintains connection information regarding every connected client. If a network failure or CIFS server failure occurs, the client receives a disconnection notification. User disruption is minimized if the application has the embedded intelligence to restore the connection. However, if the embedded intelligence is missing, the user must take steps to reestablish the CIFS connection.

Question: List and explain Factors Affecting NAS Performance:

NAS uses IP network; therefore, bandwidth and latency issues associated with IP affect NAS performance. Network congestion is one of the most significant sources of latency in a NAS environment. Following figure demonstrates the causes of latency.

39 of 41

Other factors that affect NAS performance at different levels follow:

1. Number of hops: A large number of hops can increase latency because IP processing is required at each hop, adding to the delay caused at the router.

2. Authentication with a directory service such as Active Directory: The authentication service must be available on the network with enough resources to accommodate the authentication load. Otherwise, a large number of authentication requests can increase latency.

3. Retransmission: Link errors and buffer overflows can result in retransmission. This causes packets that have not reached the specified destination to be re-sent. Care must be taken to match both speed and duplex settings on the network devices and the NAS heads. Improper configuration might result in errors and retransmission, adding to latency.

4. Overutilized routers and switches: The amount of time that an overutilized device in a network takes to respond is always more than the response time of an optimally utilized or underutilized device. Network administrators can view utilization statistics to determine the optimum utilization of switches and routers in a network. Additional devices should be added if the current devices are overutilized.

40 of 41

5. File system lookup and metadata requests: NAS clients access files on NAS devices. The processing required to reach the appropriate file or directory can cause delays. Sometimes a delay is caused by deep directory structures and can be resolved by flattening the directory structure. Poor file system layout and an overutilized disk system can also degrade performance.

6. Over utilized NAS devices: Clients accessing multiple files can cause high utilization levels on a NAS device, which can be determined by viewing utilization statistics. High memory, CPU, or disk subsystem utilization levels can be caused by a poor file system structure or insufficient resources in a storage subsystem.

7. Over utilized clients: The client accessing CIFS or NFS data might also be over utilized. An overutilized client requires a longer time to process the requests and responses. Specific performance-monitoring tools are available for various operating systems to help determine the utilization of client resources.

1 of 41