Intelligent Storage Processors

Abstract
This paper provides information about storage processors for the Microsoft®
Windows® family of operating systems. It provides guidelines for designers to build
next generation storage systems which provide a level of performance which is
unattainable using general purpose processors and more importantly provide a
hardware platform which is manageable through storage services embedded in
Microsoft operating systems.
Contents
Introduction .................................................................................................................................................3
Overview..................................................................................................................................................3
History of Network Processors................................................................................................................4
What’s Different About Storage?.............................................................................................................5
A Closer Look at Virtualization.................................................................................................................7
Variations and Combinations.............................................................................................................8
Why Storage Processors?.......................................................................................................................9
Logical Targets.................................................................................................................................10
Embedded Initiators.........................................................................................................................10
Advanced Storage Functions...........................................................................................................10
Summary of Benefits........................................................................................................................11
Intelligent Storage Processor Architecture............................................................................................11
FibreSlice™ and Windows NT Storage Architecture.............................................................................12
Conclusion.............................................................................................................................................13
 Intelligent Storage Processors - 2

Windows Hardware Engineering Conference

WinHEC Sponsors’ Disclaimer: The contents of this document have not been authored or confirmed by
Microsoft or the WinHEC conference co-sponsors (hereinafter “WinHEC Sponsors”). Accordingly, the
information contained in this document does not necessarily represent the views of the WinHEC
Sponsors and the WinHEC Sponsors cannot make any representation concerning its accuracy. THE
WinHEC SPONSORS MAKE NO WARRANTIES, EXPRESS OR IMPLIED, WITH RESPECT TO THIS
INFORMATION.

Microsoft, Windows, Windows Server, and Windows NT are trademarks or registered trademarks of
Microsoft Corporation in the United States and/or other countries. Other product and company names
mentioned herein may be the trademarks of their respective owners.

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 3

Introduction
This paper provides information about storage processors, a new class of silicon
device dedicated to accelerating storage commands and providing management
capabilities which are not possible with general purpose processors. It compares
and contrasts storage processors with network processors, and further illustrates
how not all storage processors are equal. It also explores the concepts of storage
virtualization with respect to storage processors, and how they are a key component
for designing RAID and Virtualization frameworks. Finally, it delves into how storage
processor technology can be leveraged with Microsoft Enterprise Storage features
to differentiate and add value to products designed with both.

Overview
Until recently, designing storage frameworks included the use of discreet
components such as off-the-shelf generic processors and firmware to meet
competitive standards of performance and manageability. With the advent of new
drive design, packaging and management software, this traditional approach is
being supplanted with a new class of storage processor to meet the new demands
of a network centric storage model.
Many new storage processor devices are now becoming available on the market,
but not all storage processors are created equal. Most of these emerging products
have a traditional network centric view of storage and tend to look at storage
networks as a symmetric network. In reality, storage networks are asymmetric.
Hosts and disks are not peers, as their behavior is very different. To meet the
evolving demands of storage and the adoption of storage virtualization, a new class
of Intelligent Storage Processor (ISP) has appeared as a fundamental building
block. Unlike their cousins in Local Area Network switches and routers, however,
these devices are optimized for dealing with the complexities of a Storage Area
Network (SAN) environment. To achieve maximum benefits in a storage network, a
storage processor must be able to do much more than lookup destinations, forward
frames, probe packets, manipulate data and gather statistics.
In a traditional sense, storage controllers accept read and write commands sent to a
logical volume, then handle the details of mapping those requests to physical
storage devices. This RAID functionality is in essence the simplest form of
virtualization.
When we talk about storage virtualization today, we're extending the RAID
functionality and LUN management functionality found in storage controllers to
operate within network environments. Storage virtualization is a powerful tool for
exploiting the storage architectures of the future. Storage or server assets can be
added, moved, changed and modified without affecting the entire system. Storage
resources can then behave more like a true utility. By representing resources
logically, they can be pooled, provisioned, and deployed with greater flexibility.
Mirroring, Snapshots, and replication can be used to eliminate single points of
failure.
This paper highlights the unique requirements of storage virtualization, explores
some of the various applications that can benefit from virtualization technology, and
outlines how the Aristos Logic FibreSlice architecture can be uniquely leveraged by
the Window NT® storage architecture to provide enhanced functionality such as:

• Hardware acceleration of storage applications

• Real time data collection

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 4

• Real time eventing

• Exceptional performance
• Exceptional manageability

History of Network Processors

Network processors first appeared in high-end Local Area Network (LAN) switches,
where their greater performance and lower cost made them the clear successor to
dedicated microprocessors running custom firmware, or proprietary Application-
Specific Integrated Circuits (ASICs). In fact, it was the ability of network processors
to combine the speed and performance of ASICs with the programming flexibility of
microprocessors that made them so successful.
Network processors were able to achieve this desirable balance because the
computational tasks required of switch ports in a LAN environment are well
understood. LAN network processors usually have microengines dedicated to the
core LAN switching functions – Classification, Lookup, Data Manipulation, Queue
Management, Filter/Forward, and Control/Management. These basic operations
are performed on each arriving packet, using pipelined architectures to achieve
desired performance. In Figure 1, network processors and a fast switching fabric
are all that is necessary to construct a high performance layer1/2/3/4 LAN switch.
The network processors are programmed to provide hardware assists, and
sometimes complete offloads, for functions such as IP Security (IPSec), Virtual
Local Area Networks (VLAN), Differentiated Services (DiffServ), Multi-Protocol Label
Switching (MLPS), and many others.

Figure 1: Typical Network processor Based LAN Switch Architecture

Figure 2 shows the inner structure of a LAN Network processor, including the
streaming packet and memory interfaces.

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 5

Figure 2: Functional Blocks and Interfaces in a LAN Network processor

As you can see from Figure 2, a LAN network processor is optimized for supporting
the high-overhead LAN services associated with security, classification, filtering,
and routing at wire-speed frame forwarding rates.

What’s Different About Storage?

It is tempting to imagine that all of the sophistication and advantages that have gone
into LAN network processors can be directly applied to storage area networks. At
first glance, it certainly seems possible. Both networks use PHYs running at roughly
the same speeds, both use switches, both require destination lookup and frame
forwarding, both have requirements for packet probing, manipulation, and the
gathering of statistical data. In fact, many of the functions are transferable, however
a LAN network processor is simply not equipped to deal efficiently with the unique
requirements of the storage data path. This requires something new: an Intelligent
Storage Processor (ISP). An Intelligent Storage Processor offloads a generic CPU
from processing performance critical read/write commands, accelerates storage
applications and provides manageability features. The FibreSliceTM from Aristos
Logic is the first Intelligent Storage Processor in the industry.
To appreciate how storage imposes new requirements on the functional blocks
involved in data transfer, compare the component-level diagram of typical
networked storage architecture in Figure 3 with the LAN equivalent in Figure 1. In
the LAN network processor, it was possible to identify classifier, lookup and
scheduler functions because a LAN switch performs these functions on every port.
It is completely symmetrical, and from the perspective of the LAN switch, its role is
to optimize the efficiency of the frame forwarding function between ports of
equivalent peers.
In the storage architecture, ports support either servers (shown as Host Bus
Adapters, or HBAs) or storage (shown as the combination of a storage controller
and associated physical disk drives). The servers and storage are not peers. In fact
their relationship is an asymmetrical relationship of initiators (the HBAs) and targets
(the Storage), where behaviors and system roles vary widely.

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 6

Figure 3: Typical Networked Storage (SAN) Architecture

When projecting the requirements of a storage processor on this model, it becomes
apparent that the features and optimizations necessary to provide analogous
benefits to the LAN architectures will vary depending on (1) where in the data path
the ISP is placed, and (2) what storage-specific roles the ISP will be called on to
offload and assist.
In modern RAID architectures, for example, the storage controller is responsible for
a high degree of storage-specific intelligence called command termination and
redirection. Command termination means the storage controller acts as a virtual
target, literally intercepting read and write commands as though it were a disk drive.
The storage controller then creates one or more new commands based on the "real"
commands it has terminated. When the storage controller redirects the new
commands (sends them to a physical device), it is acting as a virtual initiator.
Command termination is necessary to support virtualization functions where the
performance, protection, and redundancy provided through striping and mirroring is
desired. Once a storage controller capable of RAID functionality terminates a
command, it must then apply the appropriate RAID algorithm, based on the type of
command and RAID architecture in effect. For example, a write command to a RAID
1 (mirrored) LUN will result in the termination of the write command, followed by the
redirection of two new write commands, each targeted at one of the volumes in the
mirrored set. The original write command will then be acknowledged by the storage
controller, but only after each physical drive in the mirrored set has acknowledged
their "logical initiators" in the storage controller.
In addition to command termination and RAID functionality, the storage controller
often provides enhanced performance through intelligent cache management. A
cache is implemented in fast memory at the virtual volume level. The storage
controller then uses the virtual initiator capability to populate to cache with data that
is statistically most likely to be requested by the real initiators. A well-designed
caching algorithm can often place the required data from a physical device in the
storage controller cache memory before it is requested, eliminating the need to re-
source new commands to the slower physical devices. Writes can also be
acknowledged faster if the write updates a block locked in cache. To protect against
data corruption, cache memories on storage controllers must take precautions to

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 7

assure data persistence, even in the event of power loss to volatile memory. This
could be accomplished through battery back-up, or write-through to the physical
device, with the cached copy marked as dirty until an acknowledgement has been
received.
It should be apparent from these examples that simply classifying and moving
frames in and out of the storage controller at high speeds is not enough to duplicate
these sophisticated functions.
Moving even closer to the physical drives, spindle management becomes the most
important optimization function. Disk drive rotational latencies and seek times are so
large relative to command processing overhead that there are opportunities for
“predictive reads” on individual drives, and striping related data across multiple
drives can multiply the sustainable I/O Operations Per Second (IOPS) by
aggregating many physical drives into single logical units.
Initiators on the HBA side have quite different optimizations. Here, cache would be a
liability since servers do not typically back up their outstanding writes with
redundant battery-based power. Instead, an initiator needs to have hardware
assisted Direct Memory Access (DMA) to quickly map outstanding I/O contexts. A
“context” is a set of saved pointers, necessary to keep track of where in-host
memory, in-bound data and status need to be placed. Without the pipelining made
possible by this approach, host processors would be bogged down processing the
constant interrupts and context switching associated with hundreds of outstanding
storage I/Os.

A Closer Look at Virtualization

Virtualization is nothing more than a storage service that presents a logical view of
storage instead of a device-bound physical view. To understand why this concept is
important, we reference the Microsoft Virtual Disk Services Model (Figure 4).
The core of any virtualization service is the block aggregation function. This
function can be thought of as "blocks in, blocks out." Physical devices such as disk
drives and storage controllers provide the physical persistent storage (blocks in),
and a mapping function creates a new linear array of blocks, which may be
accessed in the same manner, using the same protocol, as a physical storage
device. Since the new "device" is actually only a termination point and redirection
service for storage access protocols, the term "Virtualization" has come into popular
usage to describe the function.

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 8

Management Apps

Virtual Disk Service

Software Hardware
Provider Provider

Management Console

Virtual Disk Service

Software Hardware
Provider Provider
Target Machine

Figure 4: The Microsoft Virtual Disk Services Model

Variations and Combinations

In the shared storage model, virtualization can be performed in the host (by a
software based volume manager or RAID-enabled HBA), in the network, or in the
device. A storage controller is an example of device-based virtualization.
Interesting variations on these generic approaches are possible. For example, an
embeddable software-based volume manager can be hosted in a fabric. Similarly, a
fabric can embed a storage controller or server blades. Host bus adapters may
perform some functions of a host-based volume manager application (e.g. LUN
Masking).
No matter where virtualization is performed, an Intelligent Storage Processor-based
solution must be designed to support:
The fundamental storage services

• Mapping
• Pooling
The performance and availability services

• Read and Write caching

• Command termination
• Redirection
Additionally, support should include intelligent assists for the more advanced
storage data services applications:

• Snapshot
• Migration

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 9

• Remote copy
These services and features are the fundamental elements of all
sophisticated storage implementations, from RAID to virtualization in a SAN.

Why Storage Processors?

Recall that a LAN network processor created value by identifying and optimizing the
LAN functions that sat as potential bottlenecks in the network critical data path. The
storage-critical function of virtualization, however, is not technically in the data path.
Through command termination and redirection, it both terminates one path and
then initiates one or more different paths. This requires a completely different set of
dedicated functions, and a different conceptual framework for the designer.
At this point, we have contrasted the functional blocks in a LAN network processor
with the different optimization requirements present in the storage data path. We
have also shown that virtualization is the layer (or layers) providing a logical view of
storage, and we have noted that the data processing requirements of a networked
storage architecture are highly dependent on where in the system the processing
element is placed. Those requirements are highly differentiated for efficient
initiators, network elements, and target controllers.
In the last section, we identified the building blocks that are distributed or shared
among all storage processing functions, mapped those functions to RAID
controllers, and showed that RAID is simply a specialized form of virtualization. It is
precisely these functions that should be optimized in the design of an Intelligent
Storage Processor.
A generic Intelligent Storage Processor must be flexible enough to handle multiple
storage applications, but specialized enough to provide the functional "building
blocks" necessary to offload critical storage applications and services.
Figure 5 shows a generalized ISP architecture, which may be contrasted and
compared with the LAN network processor architecture in figure 1. Reflecting the
asymmetrical nature of storage networks, each port on an ISP may be routed (or in
some designs, hardwired) to either logical targets or an embedded initiator.

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 10

Figure 5: Intelligent Storage Processor (ISP) Architecture

Logical Targets
Servers access logical volumes through a host bus adapter, which, in the language
of storage protocol, is an Initiator. Initiators are responsible for "initiating" I/O
operations such as read and write, and maintaining current state information until
the operation is successfully completed. The burden of error recovery and retrying
commands is also an initiator function. At the ISP, an initiator will always be routed
to a port processor running Target emulation code. Targets are devices that
respond to initiator requests with either data or status. Targets operate
asynchronously, and are not required to maintain current state information for the
duration of the I/O operation. Target Emulation has the effect of making the port
respond to requests for data and status exactly as a physical storage device would.
The emulated logical volume (often referred to by its Logical Unit Number, or LUN)
may map directly to a physical device, or may aggregate multiple physical devices.
Logical volumes may also be supported by cache memory, RAID configurations,
and other functions designed to enhance the performance or reliability of the logical
volume. The logical-to-physical mapping structures and services available to the
resulting logical volume define and bound the benefits available from virtualization.

Embedded Initiators
Another core function in the ISP is the Embedded Initiator. This functional block
has the same protocol requirements as a host bus adapter - it must initiate I/O
operations to external targets and track them to completion. The embedded initiator
supports the "back end" of the virtualization function, redirecting write operations or
requesting the physical blocks from external devices once the logical to physical
mapping has been resolved. All initiators are not created equal. The process of
"reverse mapping" the physical to logical data flow may be supported by specialized
hardware engines capable of fast context switching and DMA, or slower software
parsing in buffer memory may be used. Initiators may also be responsible for
Discovery of new block storage devices, recognition of device moves, failover path
management, error detection, and command retries.

Advanced Storage Functions

Just as a LAN network processor may provide benefits to the designer by optimizing
advanced functions such as IPSec and packet classification, ISPs can provide

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 11

similar functional blocks geared towards the unique requirements of storage

networking.
The Storage equivalent of the advanced functions found in LAN network processors
includes caching, snapshot, migration, remote copy, storage pooling through RAID
mapping, and RAID redundancy. ISPs like FibreSlice can even support cluster
management with designed-in support for bound, redundant controllers.
Together, these features take the guesswork out of designing advanced storage
products. For example, one of the trickiest disciplines of storage controller design is
cache management. Well-designed and properly optimized cache controllers can
significantly increase storage performance, while a poorly designed caching
algorithm may actually degrade performance.
With zero-tolerance for data loss increasingly becoming a standard requirement,
and with the availability of low-cost disk drives, RAID protection of logical volumes is
no longer a system luxury. With RAID quickly becoming a check-off item, the system
designer can accelerate time-to-market and improve system performance by
selecting an ISP that includes flexible RAID functionality. Things to consider include
hardware support for the XOR function, RAID levels supported, and rebuild assists.
ISPs may also include advanced services for management and monitoring of
storage resources. With storage provisioning moving towards a policy-driven model,
the ability to gather statistical and real-time storage metrics from physical devices,
logical volumes, and ports will become important considerations.

Summary of Benefits
By choosing to leverage an Intelligent Storage Processor in your network storage
product family, you get optimization of sophisticated storage functions without the
need for specialized in-house expertise. Designers can focus on value and
differentiators, while achieving lower system cost, tighter integration, and higher
performance than discrete designs.

Intelligent Storage Processor Architecture

Aristos Logic’s proprietary Intelligent Storage Processor Technology, is a
scalable fabric of programmable processing elements and memory data structure
controllers that allows flexible configuration and re-purposing of a single architecture
for multiple SAN product implementations.
Figure 6 shows the storage datapath through FibreSlice, from fabric-attached hosts
to external storage, with all system interfaces. By comparing figure 6 to the LAN
network processor in Figure 2, you can see that FibreSlice is architected to provide
dedicated support for the functions most critical to high-performance logical volume
virtualization, while still providing the external interfaces necessary to support a
wide variety of system designs.

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 12

Figure 6: Functional Blocks and Interfaces in FibreSlice

FibreSlice™ and Windows NT Storage Architecture

Aristos Logic is working to leverage the enterprise storage related features in
Windows Server™ 2003 to ensure that all the hardware features of FibreSlice are
easily leveraged by Windows storage applications. The range of work being done
includes:

• Leveraging the Storport driver model to ensure that the software can drive I/O to
obtain a maximum throughput and leverage all the hardware acceleration
features of the FibreSlice architecture
• Leveraging the extensibility provided by Windows Management Initiative – WMI
allows an IHV to provide product differentiation by surfacing additional
instrumentation including data and events classes. These dynamically created
classes can be discovered and utilized by WMI aware systems management
applications
• Leveraging the Volume Shadow Copy Service (VSS) to provide storage and
storage management applications platform integrated access to the snapshot
features of the FibreSlice architecture
• Leveraging the Virtual Disk Service (VDS) to provide storage and storage
management applications integrated platform access to storage in an
application meaningful way (Define a RAID 5 volume of 10GB size; allocate a
RAID 1 volume of 20 GB size; etc)
• Leveraging the power of Multi Path I/O Development kit to utilize the multi path
capabilities of Fibre Slice and provide

WinHEC 2003
Microsoft Windows Hardware Engineering Conference
 Intelligent Storage Processors - 13

Conclusion
This paper explored the differences between network processors and storage
processors and also detailed some important concepts of storage virtualization.
Furthermore, it went on to explore how the Microsoft VDS and VSS architectures fit
into this world of storage processors and how Aristos Logic is using the
Windows NT enterprise storage features to differentiate their FibreSlice device.

WinHEC 2003
Microsoft Windows Hardware Engineering Conference