Although it was not developed with networking in mind, Serial Attached SCSI (SAS) offers a wealth of benefits to the enterprise server and enclosure customer, including high reliability and performance, mixed enterprise/desktop drive support, and improved economies of scale. Even so, many end customers are badgering OEMs with requests for SAS Storage Area Networks (SANs). With modest future extensions to the SAS standard, SAS could encroach on the established technologies in the storage networking landscape. Sam explores how SAS could evolve into a viable protocol for storage networking in the future.

Understanding SAS
SAS is the evolutionary follow-on to the parallel SCSI interface. Like Serial ATA (SATA), Fibre Channel, and other serial technologies for storage, SAS was originally envisioned as a point-to-point drive connection mechanism only, but it has become much more. In its simplest configuration, SAS provides a physical connection between a host controller and some number of targets. Figure 1 graphically illustrates this connection type.

As the standard gained momentum, it became apparent that OEMs needed a more robust expanded connection construct for supporting large storage topologies. Thus, the concept of an expander was born. Like Fibre Channel switches, the expander provides a switching matrix for connecting multiple devices with a SAS domain such as host controllers (initiators), hard disk drives (targets), and other expanders.

Large topologies with up to 16,384 devices in a single domain can be built through expander cascades and different connection routing mechanisms including direct, subtractive, and table routed. Figure 2 illustrates a large SAS topology using expanders.

Expander types
The two types of SAS expanders defined by the specification include the edge expander and the fan-out expander. Each type provides the connection mechanisms necessary to connect multiple targets to a single host or multiple hosts (one connection at a time). Both have common and unique attributes, with the greatest difference being whether or not a given type can be used in a cascaded fashion. While fan-out expanders can be cascaded, edge expanders are limited in their cascadability.

In general, edge expanders are best suited to those designs where cost is a significant consideration and storage scalability requirements are restricted. With the expander building blocks inherent to SAS, taking the next step of defining SAS as a fabric is a reasonable advancement of this technology.

The makings of a networking interconnect or fabric
Loosely defined, a fabric is a pathway on computing, networking, or storage devices that provides chip-to-chip, adapter-to-adapter, or device-to-device connections for transferring information within computing, networking, or storage systems/subsystems. In essence, a fabric is a switch or cooperative switching facility much like an expander. A fabric can be viewed as a network and vice versa, at least a limited one.

SAS could probably become a fabric; however, as a point-to-point protocol, it was not originally intended to be a storage networking technology. The following new features would most likely be required for SAS to play the role of a fabric or interconnect technology.

Connection-oriented transfer
SAS is a connection-oriented protocol, meaning that a connection between two SAS devices must exist before data transfer can occur. By evolving the SAS protocol to support a connectionless, albeit reliable, transfer scheme, the problems of low link utilization, poor long-haul transfer performance, and SATA/STP host starvation/lockout could be avoided.

Physical connection enhancements
Unlike its network-capable counterpart Fibre Channel, SAS does not currently sport an optical interface. For most intra-data center connections (shelf-to-shelf, rack-to-rack, or box-to-box), standard four-wide SAS cables are more than sufficient. To support greater distances, an optical interface for SAS and its unusual out-of-bound signaling must be defined.

Routing and address virtualization
The routing structures in SAS were originally designed with direct-attach and limited topology sizes in mind. Top-level (fan-out) expanders today require complete knowledge of their connected domain, limiting the effective size of a storage system. By adding a routing summarization feature, no one expander in the domain would be required to maintain knowledge of the entire domain, allowing topologies of arbitrarily large sizes to be built.  

[ad]Similarly, an address virtualization scheme is needed for efficient routing. Each SAS device has a set of hard-coded addresses (the SAS address) that identifies the device to the rest of the system. These addresses are either burned in at the factory or assigned by firmware at system boot time. A mechanism must be put into place that allows the OEM to remap these physical addresses to more logical ones. An address resolution protocol would provide the basis for mapping hardware addresses to virtual ones.

Intelligent expanders
Today, SAS expanders are essentially circuit switches with a great deal of supporting logic for making connections between SAS initiators and SAS targets. Most implementations are based on a cut-through type architecture, meaning that it does not provide buffering of any frame with routing Protocol Data Units (PDUs). Since expanders are the logical basis for making SAS fabric switches, the types of architectures described in the following discussion could feasibly enable SAS expander technology to evolve into tomorrow’s fabric of choice.

Constructing a switch
Fabric switches come in many different sizes and flavors. Loosely defined, a fabric switch is a traffic director, routing PDUs from an input port to an output port based on some combination of criteria. The switch must also resolve any contention resulting from the simultaneous arrival of PDUs at a common output port.

Most switches are based on one of a number of internal architectures: shared memory, shared bus (also known as shared medium), crosspoint matrix, and ring. With all their similarities, the underlying architectures are distinguished from each other principally on the basis of their buffer (queue) servicing policies.

Regardless of the architecture employed, a SAS switch, like an expander, will ultimately consist of the following key elements illustrated in Figure 3.

Connection manager

• Maps a destination SAS address in a connection request to a destination PHY using direct, subtractive, or table routing
• Arbitrates and assigns or denies path resources for connection requests following SAS rules for arbitration and pathway recovery
• Configures the connection router

Connection router

• Routes signals between pairs of PHYs (initiators and targets) as configured by the connection manager
• Provides routing resources needed to support connections

Broadcast processor

• Routes topology messages to appropriate devices (for example, topology changes)

Device interfaces

• Provide physical layer device interfaces to internal and external components
• Arbitrate and route frames between PHYs
• Provide physical and link-layer connections to auxiliary I/O (for example, 10/100 Ethernet, General Purpose I/O, and so on)

Innovation key to SAS’ progression
By improving the existing expander building blocks, adding an optical interface, and augmenting the transport protocol in key areas such as reliable, connectionless transfer, routing summarization, and address virtualization, SAS can and will evolve. Figure 4 shows the SCSI Trade Association’s SAS roadmap as of January 2007.

The future of storage and storage networking is contingent upon the evolution of SAN and network attached storage architectures, the distribution model for storage, and advances in transparent protocol communication techniques. As with any new technology development, whether revolutionary or evolutionary, one size will never fit all. Complementary technologies will address different market segments, and the proper solution will differ by application, connectivity requirements, scalability, performance, and price sensitivity. Only time will tell what this means for SAS’ future as a storage networking technology.

Sam Barnett is the product line director responsible for SAS and SATA storage/storage networking products in the Storage Products Division at Vitesse Semiconductor in Colorado Springs, Colorado. Prior to joining Vitesse, Sam spent five years at MCI, most recently as a strategic architect and consulting engineer for high-speed data transports. He holds a BS in Business from Louisiana Tech University and an MS in Computer Science from Colorado Tech University.

Vitesse Semiconductor
719-867-4481
[email protected]
www.vitesse.com