Windows Knowledge Base

[IBM DS] Loops and switches on the DS6000 and DS8000

The DS6000 and DS8000 series uses the term loops when describing the backend disk subsystem within which all the data is held. At the same time, the disk subsystem is described as being a switched network or even as a switched fabric. This may appear to be a mixed usage of terms.

The disk subsystem is made up of three components:

1) Fibre Channel Switches

Each storage enclosure in a DS6000 or DS8000 has two fibre channel switches. Each of these fibre channel switches offers 20 ports. Sixteen ports are used to attach to up to 16 Disk Drive Modules (DDMs) that can be placed into each enclosure. The remaining four are used to attach to other switches within other storage enclosures, or to the device adapters. These are genuine switches in that every connection through the switch is a switched connection. It is definitely not a hub or some sort of loop device.

Regarding these switches, there are two very important points to understand:
+ These switches do not require any user management. This includes zoning or microcode management. All management of the switches is performed by the internal microcode of the DS6000 or DS8000.
+ These switches do not and will not ever become part of any client SAN fabric. They remain purely internal to the machine. Do not think that as an end-user, and because each storage enclosure contains two switches that your SAN fabric will grow because of this. In fact, if you look at the DS6000 or DS8000 you will be hard pressed to even see the switches. They are really internal components.

2) Device adapters

The device adapters are RAID controllers that manage all disk access for the processors within the DS6000 or DS8000. They perform all RAID functions, such as array rebuilds and parity generation. The device adapters are connected to the switches in the first storage enclosure.

3) Disk Drive Modules (DDMs)

The DDMs used in the DS6000 and DS8000 series use Fibre Channel Arbitrated Loop (FC-AL) addressing to communicate with the device adapters. This means each disk and each device adapter port has to have an FC-AL address. This address is given to the DDM by the switch and is determined by the identifier of the enclosure in which the switch is located, and the physical position of the disk within the enclosure. Each time the device adapter connects to a disk, it does so using FC-AL protocol, but the connection itself is done through the switched network. This means the connection is a genuine switched connection. Data does not flow through any of the other disks for instance; it is a point-to-point connection.

In normal SAN terminology, when two switches are connected together, they form a fabric, which is effectively a network of switches. However, throughout either redbook you will see the term loops to describe this switched network. So why use the term loops? Mainly because FC-AL addressing is used. This means the maximum number of DDMs that can be connected in one switched network is limited by FC-AL address limitations. This limits us to a maximum of 112 disks per loop, or seven enclosures. Eight enclosures could contain up to 128 disks, which exceeds the 126 addresses available for a loop. So from an addressing point of view, the switched network is a loop.

In addition, the DS6000 and DS8000 are evolved from the ESS 2105, which used Serial Storage Architecture (SSA) technology, where the disks really were on loops. Consequently there is a tendency to use the terms loops, even if it is not necessarily relevant anymore. An example is the term Arrays Across Loops (AAL). One of the features that was carried across from the ESS 2105 is the use of AAL, which is detailed in the IBM TotalStorage DS800 Series: Concepts and Architecture Redbook, SG24-6452. To use the term Arrays Across Switched Networks does not have the same ring, so the term loops may be with us for some time to come.