White Paper

UNCOMPRESSED HIGH-RESOLUTION
WORKFLOWS WITH DELL EMC ISILON
UNDERSTANDING THE REQUIREMENTS AND CHALLENGES
OF STORAGE SYSTEM SPECIFICATION AND DESIGN

Abstract
This document examines the challenges of specifying and implementing
storage systems to support high-performance creative applications. High-
performance creative applications are defined as being those that are used to
manipulate multiple streams of high resolution video in uncompressed
formats, or high quality compressed formats.
It is intended that architects and system integrators use this document as a
guide to both understanding the challenges of working with high-performance
creative applications, and to designing solutions built with Dell EMC products
to deliver against those challenges.
H16818 November, 2017

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
EXECUTIVE SUMMARY AND SCOPE ...............................................................................................................3
OVERVIEW OF APPLICATION DEMANDS ........................................................................................................4
PERFORMANCE ............................................................................................................................................................... 4
COLLABORATIVE WORKING........................................................................................................................................... 4
OVERVIEW OF FILE TYPES ..............................................................................................................................5
METADATA FILES ............................................................................................................................................................. 5
MEDIA FILES ..................................................................................................................................................................... 5
FRAME SIZE AND COLOUR DEPTH ............................................................................................................................... 6
VIDEO ENCODING ............................................................................................................................................................ 8
IMAGE SEQUENCE ...................................................................................................................................................... 8
CLIP BASED MEDIA.................................................................................................................................................... 10
COMPRESSION .......................................................................................................................................................... 10
COMPARATIVE BENEFITS ........................................................................................................................................ 12
PERFORMANCE REFERENCE ...................................................................................................................................... 12
PERFORMANCE METRICS ..............................................................................................................................13
THROUGHPUT ................................................................................................................................................................ 13
LATENCY ......................................................................................................................................................................... 14
COLD DATA ..................................................................................................................................................................... 15
CONFIGURATION GUIDELINES ......................................................................................................................16
WORKSTATION SPECIFICATION .................................................................................................................................. 16
DELL EMC ISILON AND ONEFS VERSION ................................................................................................................... 16
NETWORK CONNECTIVITY ........................................................................................................................................... 16
FILESHARING CONFIGURATION .................................................................................................................................. 16
ADVANCED CONFIGURATION AND OPTIMISATIONS ............................................................................................ 17
ONEFS OPTIMISATIONS ................................................................................................................................................ 17
METADATA READ/WRITE ACCELERATION ............................................................................................................. 17
STREAMING MODE .................................................................................................................................................... 17
FILENAME PREFETCH ............................................................................................................................................... 18
MORE INFORMATION ................................................................................................................................................ 19
CLIENT AND APPLICATION OPTIMISATIONS .............................................................................................................. 19
SUMMARY AND CONCLUSION .......................................................................................................................20

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
EXECUTIVE SUMMARY AND SCOPE
This document examines the challenges of specifying and implementing storage systems to support high-performance
creative applications. Creative applications from different vendors are generally characterised by a number of similar
requirements:

• Relatively high throughput single stream performance.

• Relatively low latency response.

In the context of this document, high-performance creative applications are defined as being those that are used to
manipulate multiple streams of high resolution video in uncompressed formats, or high quality compressed formats.

As well as the performance demands made by creative applications of storage systems, each application will make
particular demands to support the overhead of content creation - for example media management, metadata and other
small files, and sharing. These requirements are not covered by the scope of this document.

It is intended that architects and system integrators use this document as a guide to both understanding the challenges of
working with high-performance creative applications, and to designing solutions built with Dell EMC products to deliver
against those challenges.

In particular, it is assumed that a system will be designed around clustered storage using Dell EMC Isilon Scale-Out NAS
and its underlying system: OneFS, and Dell Precision Workstations.

The requirements of high-performance workflows may be satisfied by using Dell EMC Isilon H600 or F800 all-flash (or
later variants), with OneFS 8.1 or later.

It is recommended that the reader refer to other documents published by Dell EMC regarding optimisations for specific
creative applications and client operating systems. Available documents are listed at:

• https://www.emc.com/industry/communications-media-entertainment/big-data-media-entertainment.htm

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
OVERVIEW OF APPLICATION DEMANDS
Creative applications include those that deal with editing, compositing, 3D effects and finishing; and the demands of
these applications are high - particularly at high frame rates and resolutions. Consequently, these kinds of applications
require a highly performant workstation, networking and storage infrastructure.

PERFORMANCE
When designing a system, architects will need to consider both quantitative and subjective requirements.

• Throughput
• Latency
• Demands of different codecs
• Scrubbing
• Multi-track
• Subjective “feel”

Though hardest to quantify and measure, the subjective “feel” of an application as it is used by a Creative Artist is
arguably the most important measure of a successful system design. An Artist may describe a system that responds well
and is easy to use as “snappy”, whilst a system that does not respond well and is difficult to use may be described as
“sluggish”.

For a system to be “snappy”, the storage component must deliver data to the application in a way that doesn’t interrupt
the creative process.

COLLABORATIVE WORKING
A primary reason for selecting a shared storage solution is to support a collaborative environment. A fundamental
requirement of collaborative working is that Artists should be able to read and write files to the storage, and that they
should be able to access the files of other Artists in a controlled manner.

Additional requirements may include:

• NFS and SMB multi-protocol support.

• Read-whilst-write – across accessing clients.
• Active Directory binding, and complex ACL implementation.

In some environments, the creative application or the user themselves will manage the files on the shared storage. In
other environments (in particular, larger or more complex environments), a separate Media Asset Management layer will
be configured. Media Asset Management is outside of the scope of this document.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
OVERVIEW OF FILE TYPES
Most creative applications work with a number of different file types:

• Metadata:

− Action Only Formats

− Sequence Exchange Formats

• Media:

− Image Sequences
− Clip Based Media

o QuickTime clips
o Media Exchange Format (MXF) clips
o MP4 clips
o Other Streaming Codecs

Each file type presents a different challenge to each part of the system, and the file types being used by the required
workflow must be considered.

METADATA FILES
Metadata and sequence information files are typically small in size and frequently updated. These files contain
information about the sequences and media being created or manipulated. For example, such a file could contain
information about which clips are used in a sequence, or information about how the colours should be corrected.

Metadata and sequence information files are relatively small, and whilst they should not present any meaningful
contribution to overall throughput requirements, it is important to consider that the files will be frequently accessed and
updated. However, this document focuses on the performance requirements of creative applications, and so designing a
storage system to accommodate the requirements of metadata and sequence information is outside of its scope.

MEDIA FILES
Because of their large size, media files are normally considered to represent the most important challenge to an Architect
when designing systems to support creative applications. There are three important factors that influence the size of
media files, and that impact the demands of supporting systems:

• Frame Size and Colour Depth

• Encoding Method
• Compression Method (if any)

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
FRAME SIZE AND COLOUR DEPTH
A key factor that impacts the size of encoded video files, is the number of pixels in each frame. There are a number of
common sizes, with the most common being:

Common Name Application Horizontal Size (pixels) Vertical Size Total number of
(pixels) Pixels

HD Broadcast Television 1920 1080 2073600

2K (DCI) Cinema 2048 1080 2211840

UHD Broadcast Television 3840 2160 8294400

4K (DCI) Cinema 4096 2160 8847360

Table 1. Common formats, resolutions and pixel counts

• HD: High Definition – typically used in television

• UHD: Ultra High Definition – typically used in television
• DCI: Digital Cinema Initiatives – typically used in digital cinema

The colour of each pixel is represented with a number of bits according to the required colour depth. High dynamic range
media requires encoding at a greater colour depth.

Colour Depth Application

10-bit Standard Dynamic Range

12-bit High Dynamic Range (HDR)

16-bit High Dynamic Range (HDR)

Table 2. Colour depth and normal application

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
Figure 1. Relative Video Frame Size and Colour Depth

The uncompressed byte size of each frame can be approximated by multiplying the number of pixels in the frame by the
colour bit depth, and then again by the number of channels. For example: a typical UHD frame has 8294400 pixels (3840
x 2160), is encoded using three channels (red, green and blue – RGB) and at a colour depth of 10 bits; giving a total of
248832000 bits (or 29.66MBytes).

(3840 x 2160) x 10 x 3 / (8 x 1024 x 1024) = 29.66MBytes

To calculate a more accurate byte size, it is necessary to consider how the data is encoded in the underlying data stream
– which is built on 32-bit words. With 10-bit encoding, the data for three pixels will fit in a single word – leaving two bits
remaining. To improve performance, most applications arrange to start the bitstream of each pixel at a boundary of a 32-
bit word in the underlying stream, so those final two bits are not used – and instead are padded with null data.

Our calculation for the uncompressed byte size of each frame should therefore be slightly modified:

(3840 x 2160) x ((10 x 3) + 2) / (8 x 1024 x 1024) = 31.64MBytes

Most applications will also add additional header and payload information, so the byte size of each frame can be even
higher.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
Common Name Resolution Total number of Bit Depth Frame
Pixels Byte Size

HD 1920x1080 2073600 10 7.91MBytes

2K (DCI) 2048X1080 2211840 10 8.44 MBytes

UHD 3840x2160 8294400 10 31.64 MBytes

4K (DCI) 4096x2160 8847360 10 33.75 MBytes

Table 3. Uncompressed Frame Byte Size

The strategy for data padding to match 32-bit word boundaries varies with the application, but it should be noted that
image sequences encoded at a colour depth of 16-bit will not require any padding, whilst sequences encoded at 12-bit
may require a complex strategy to avoid wasteful padding.

VIDEO ENCODING
A sequence of video frames can be encoded for digital storage in one of two ways:

• Image Sequence
• Clip Based Media

IMAGE SEQUENCE
An image sequence (such as a DPX sequence) represents each frame of video with a separate file. The files for a
sequence are stored in the same directory, have a fixed base name with a decimal or hexadecimal incrementing suffix,
and a .dpx extension. Different applications use different naming structures and standards within this form, but some
illustrative examples would be:

filebasename_scene-01_00056.dpx
filebasename_scene-01_00057.dpx
filebasename_scene-01_00058.dpx

filebasename_scene-05_0FC09.dpx
filebasename_scene-05_0FC0A.dpx
filebasename_scene-08_0FC0B.dpx

Image sequences can be more challenging for a storage system, because it can be difficult to ensure that file read-ahead
and caching strategies yield a benefit. OneFS can be optimised to deliver increased throughput performance for image
sequences using the FileName PreFetch option - details of which are covered later in this document.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
Image sequences are particularly suited to VFX (visual effects) workflows, because artists very often work on one frame
video frame at a time. Most creative applications work with a number of different image sequence formats, including:

• DPX
• Raw
• Open EXR

DPX OVERVIEW
The picture information of each image in a DPX sequence is encoded as a bit map. Each pixel is recorded as an integer
value. Consequently, the file size and data rate for a .dpx sequence is steady and predictable.

Because of the simple encoding method, a DPX sequence makes the lowest demands of the application workstation.
However, the larger file size leads to a relatively high demand on storage and networking systems.

RAW OVERVIEW
Modern single sensor cameras capture image information using a technique developed by Bryce Bayer in the 1970s. An
arrangement of coloured masks is overlaid on the pixels of the camera sensor – with twice as many green filters as either
red or blue. The data that is captured is referred to as a RAW image, and must be processed by the application using a
process termed “de-Bayering” before it can be displayed.

Consequently, a RAW image is smaller in size than a DPX image, and may be considered to be compressed. The de-
Bayering of the image requires processing power of the host workstation.

For more information about Bayer Filtering, check

• https://en.wikipedia.org/wiki/Bayer_filter.

Different cameras implement Bayer Filtering in different ways, with a number of proprietary approaches. For a list, check

• https://en.wikipedia.org/wiki/Raw_image_format.

OPEN EXR OVERVIEW

Open EXR is a more complex format. Pixel values are represented using floating-point code values rather than integer
code values (0-1023 in 10-bit). This is separate from the colour depth encoding of each pixel. An Open EXR image may
have a colour depth of 10 bits, 12 bits, or 16 bits per RGB channel. Files may be uncompressed, or compressed using
one of a number of algorithms; and each file may contain multiple additional layers of information:

• Geometry (vector normals)

• Arbitrarily number of extra channels (in addition to RGB):

− Specular Highlights
− Bump Maps
− Alpha Mattes
− Texture Maps (multiple images in the same file).

The Open EXR format is often used to encode HDR (High Dynamic Range) image sequences. A high dynamic range
image contains more colour and luminance depth information – put simply, more and smaller steps between complete
Uncompressed High-Resolution Workflows with Dell EMC Isilon
© 2017 Dell Inc. or its subsidiaries.
black and complete white. Compared to an image encoded using a format with a standard dynamic range, an HDR image
shows much more detail in very dark and very light parts of the scene.

Because of the variety of possible file elements and encoding strategies, it can be difficult to predict the data rate of an
Open EXR image sequence. Care should be taken to understand the exact requirements of any proposed format. It
should also be noted that Open EXR image sequences may make a relatively high demand of the application
workstation.

MORE INFORMATION
More information about DPX, ARRI Raw and Open EXR can be found at:

• https://en.wikipedia.org/wiki/Digital_Picture_Exchange
• http://www.arri.com/camera/alexa/workflow/working_with_arriraw/arriraw/format/
• http://www.openexr.com/
• https://en.wikipedia.org/wiki/OpenEXR

CLIP BASED MEDIA

A Clip based media file represents an entire sequence of video frames (and additionally very often interleaved audio,
timecode and metadata) in a single file on the storage. Because an entire sequence of video frames is stored in a single
file, it’s more straight-forward for conventional caching and read-ahead algorithms to yield benefit. For example, when in
Streaming Mode, OneFS will read-ahead and stage files as they are accessed by the client.

Most creative applications work with a number of different clip-based codecs, including:

• RED
• XAVC
• ProRes
• DNX

More information about these formats - including their respective strengths and use-cases - can be found at:

• http://www.red.com/learn/red-101/redcode-file-format
• https://en.wikipedia.org/wiki/XAVC
• https://en.wikipedia.org/wiki/Apple_ProRes
• https://support.apple.com/HT202410
• https://en.wikipedia.org/wiki/Avid_DNxHD
• http://www.avid.com/products/avid-dnxhr-and-dnxhd

COMPRESSION
Both image sequences and clip-based media can be compressed. Compression can yield benefits in required storage
capacity and throughput, but can generate computational overhead and reduce the perceived visual and audio quality of
the media.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
There are two ways of compressing media files in Media and Entertainment workflows: Lossy, and loss-less. Both
methods can reduce the amount of data requiring to be stored, but different approaches are taken in the underlying
algorithms. The terms lossy and loss-less refer to the visible (or audible) representation of the images being compressed,
and can be characterised by our ability to reverse the compression.

LOSS-LESS COMPRESSION
Loss-less compression is completely reversible - it is possible to recreate exactly the original images (or sounds). This
type of compression will be the most familiar to data scientists who are used to using compression utilities to reduce the
size of many types of file - and then re-inflating them to their exact original state when required.

LOSSY COMPRESSION
Lossy compression is not reversible. It is not possible to recreate exactly the original source - because some data is
discarded in the compression process. Lossy compression is built on algorithms that exploit the characteristics and
limitations of human perception.

For example, perhaps the most fundamental type of lossy compression exploits the fact that humans can perceive spatial
resolution (in other words, the number of dots on the screen) in colour at only about half of that in luminance. The upshot
is that compression algorithms can encode the colour parts of an image at only half the spatial resolution of the
luminance parts - and we can’t tell the difference.

More complex lossy compression (for example, JPEG and similar) is built on algorithms that divide images into
segments, and then describe those segments using patterns or waveforms; or on algorithms that identify parts of an
audio scene that cannot be heard because they are “masked” by other elements.

Depending on the aggressiveness of the algorithms being used in lossy compression, humans may be able to perceive
sound or image degradation. The term “Visually Loss-less” is used when it’s impossible or nearly impossible to discern
the difference between the original and a compressed image - even though some original data has been discarded in the
compression process.

I-FRAME AND LONG GOP ENCODING

• Frames in an image sequence may be individually compressed - using either loss-less or lossy algorithms.
• Files in an OpenEXR image sequence are often compressed using a loss-less algorithm - for example run-length
encoding or scan-line zip.
• Files in a JPEG2000 image sequence may be compressed using a lossy (or visually loss-less) algorithm based on
wavelet transforms. (Note that JPEG2000 uses a very different encoding algorithm to JPEG).
• Clip based media is normally compressed using a lossy algorithm, using one of two strategies:

− I-Frame (Intra-Frame) clip-based codecs perform compression only in the spatial domain. Each frame of video is
compressed individually using algorithms that are similar to those used when compressing still images - making it
relatively simple to cut and splice material at frame boundaries. Creation workflows typically perform best when
using I-Frame codecs.
− Long GOP (Group of Pictures) clip-based codecs additionally perform temporal compression. Each frame of
video in a sequence is compared to adjacent and near-by frames, and additional compression is achieved by
storing only the differences between those frames, rather than each entire frame. Long GOP codecs are more
Uncompressed High-Resolution Workflows with Dell EMC Isilon
© 2017 Dell Inc. or its subsidiaries.
 efficient on storage and throughput, but are more computationally expensive to create and consume. They are
best suited to delivery workflows.

COMPARATIVE BENEFITS
It follows that different codecs have different benefits, and present different challenges to each part of a system.
Fundamentally, there are three key parameters - and a system can be optimised for any two, at the expense of the third.

• Data Rate
• Encode and Decode Overhead
• Image Quality

For example, an uncompressed DPX sequence is very high quality and has a very low decode overhead, but it has a
relatively high data rate. A high quality XAVC clip will have a relatively low data rate, but a much higher decode overhead.

Figure 2. Quality, Data Rate and Decode Overhead Comparison

PERFORMANCE REFERENCE
Some Artists - particularly those dealing with visual effects - normally require to work with image sequences (rather than
clip based media), and very often prefer to work with uncompressed material. Consequently, it’s important that Architects
design a full system that is able to support the high-performance demands of these workflows.

This document focusses on the configuration and optimisations to deliver the best performance when working with
uncompressed DPX image sequences at UHD resolution.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
PERFORMANCE METRICS
Creative applications make the highest demands of workstations, networking and storage in any implementation, and so
systems must be architected to accommodate those performance requirements.

There are two key system performance metrics that architects must consider when specifying storage solutions for a
creative application:

• Throughput
• Latency

Throughput performance is important to ensure that the infrastructure can sustain the streaming requirements of the
application. The higher the throughput that can be achieved, the better. Latency performance is important to ensure that
the throughout performance is consistent, and achieved quickly. The lower the latency that can be achieved, the better.
The responsiveness of the application—and its usability in real-world workflows is highly dependent on both throughput
and latency performance.

If the infrastructure can’t support the required throughput, users are likely to report that video playback is “stuttering” or
that it freezes. If the infrastructure can’t support the required low latency, users are likely to report that the application is
“sluggish” or unresponsive - particularly when scrubbing on a timeline, or rapidly selecting and manipulating content.

THROUGHPUT
Throughput performance can be considered as a measure of the required sustained data transfer rate. To sustain
playback at the application layer, the system must be able to support at least the required throughput over an
indeterminate duration.

Throughput for uncompressed image sequences can be calculated by multiplying the frame byte size by the number of
frames per second according to the sequence. The throughput for a selection of 10-bit RGB frame resolutions and rates
is outlined in the table below.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
Common Name Hres Vres Frame Rate Frame Byte Size Throughput
MBytes MBytes per
second

HD 1920 1080 24fps 7.91 190

2K (DCI) 2048 1080 24fps 8.44 203

UHD 3840 2160 24fps 31.64 759

4K (DCI) 4096 2160 24fps 33.75 810

HD 1920 1080 60fps 7.91 475

2K (DCI) 2048 1080 60fps 8.44 506

UHD 3840 2160 60fps 31.64 1898

4K (DCI) 4096 2160 60fps 33.75 2025

Table 4. Codec, frame size and throughput.

LATENCY
Latency performance can be considered to be a measure of the the required instantaneous data transfer rate. To offer a
responsive experience at the application layer, the system must be able to deliver frames in a timely manner after they
are requested. One measure of latency is the time to first byte - how long it takes data to start flowing after it is requested
by the application.

For image sequence workflows (where each frame of video is stored as a separate file), it is assumed that the application
normally will request each frame sequentially whilst the preceding frame is being displayed. It follows therefore that for
steady and sustained playback, each file must be requested and retrieved in less than the time for which each frame is
displayed.

Perceived performance can be increased (perceived latency reduced) by configuring intelligent read-ahead (or pre-
caching) at the application, client operating system or storage layer.

• An application that is configured to read-ahead (or pre-cache) will intelligently request frames or files in advance of
their being required, and will typically stage them in client RAM.
• OneFS can be configured to aggressively pre-fetch and stage media requests by setting the SmartPools policy for
the required directories to Streaming. Note that in Streaming Mode, other optimisations are also applied at the cluster
- including details regarding the layout of data across the underlying storage units.
• OneFS can be optimised to deliver increased throughput performance for image sequences using the FileName
PreFetch option.
• Client operating systems may implement their own intelligent caching algorithms.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
Configuring any part of a system to intelligently read-ahead (or pre-cache) should be done with care and consideration.
No matter how intelligent the algorithm used, there is still a possibility that it will pre-fetch or stage media that will not ever
be required by the application - thereby unnecessarily consuming system resources. A simple example of such a false
fetch would be where a user halts media playback part way through a sequence.

A summary of file sizes and frame durations for a number of 10-bit image sequence formats helps quantify the required
latency targets

Use Case Frame Rate Common Name Hres Vres Frame Size Frame Duration
MBytes

Cinema 24fps 4K (DCI) 4096 2160 33.75 42ms

PAL* 25fps UHD 3840 2160 31.64 40ms

PAL* 50fps UHD 3840 2160 31.64 20ms

NTSC* 30fps UHD 3840 2160 31.64 33ms

NTSC* 60fps UHD 3840 2160 31.64 17ms

Table 5.

*Note: The references to PAL and NTSC are used to indicate a loose alignment to legacy television standards and
regional variations, and do not refer to any form of colour encoding, number of TV lines, or refresh rate.

COLD DATA
Whilst the optimisations of Streaming Mode and FileName PreFetch can enable performance and usability
improvements, care must be taken to consider overall system performance when dealing with Cold Data.

Cold Data is data or media that is requested by the application but that is not cached or pre-staged in any part of the
environment by the application, the client operating system, or the storage. It is data that must be retrieved from the
underlying storage units - normally assumed to be the component spinning disc or solid-state devices - and processed by
the application within the required time.

Scenarios where a creative application may generate a request for Cold Data include:

• Playing media instantaneously, without first cueing.

• Scrubbing backwards-and-forwards along a timeline in an unpredictable way.
• Jumping to, or skipping over a part of a sequence.
• Playing a sequence that is comprised of a number of other unrelated sequences – which typically occurs at an edit
point in a sequence.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
Figure 3. Cold Data at Edit Point on Timeline

Architects must consider a number of strategies to deliver performance when Cold Data is requested:

• Raw storage performance.

• Intelligent application read-ahead (or pre-cache).

The challenges of Cold Data exist for both image sequence and clip-based media formats.

CONFIGURATION GUIDELINES

WORKSTATION SPECIFICATION
Dell EMC recommends a suitably specified Dell Precision Workstation or similar for use with most high-performance
creative applications. Specification and configuration of the workstation is outside of the scope of this document.

DELL EMC ISILON AND ONEFS VERSION

It is assumed that architects will specify Dell EMC Isilon H600 and F800 all-flash and OneFS version 8.1 or later to
support the highest performance creative applications.

NETWORK CONNECTIVITY
In order to achieve the throughputs required for typical high-performance workflows, it is required to configure a full
40GbE network - including Dell EMC Isilon storage, network infrastructure and client interface.

The choice of Network Interface Card, and recommendations regarding network interface optimisation are dependent on
the specific environment of each installation, and are therefore outside of the scope of this document.

FILESHARING CONFIGURATION
Most creative applications are hosted on a Linux, Windows or macOS platform, using NFS or SMB filesharing from the
Isilon cluster.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
The creation of an NFS Export or SMB share on the Isilon storage cluster can be achieved using the administrative web
interface of OneFS. Details are available at

• https://www.emc.com/collateral/TechnicalDocument/docu84277.pdf

NFS and SMB optimisation guidelines for connections from different operating systems are dependent on the specific
environment of each installation, and are therefore outside of the scope of this document.

ADVANCED CONFIGURATION AND OPTIMISATIONS

When working with high-performance applications, it is important to consider optimisations for the entire system, and the
mutual interactions of those optimisations. Attention in particular should be given to optimisations for:

• OneFS - The Isilon storage cluster.

• Application - The creative application itself.

ONEFS OPTIMISATIONS
Architects should consider three key optimisations for OneFS when deploying in high-performance creative application
environments.

• Metadata Read/Write Acceleration

• Streaming Mode
• FileName PreFetch

METADATA READ/WRITE ACCELERATION

OVERVIEW
File access performance can in some workflows be optimised by enabling Metadata read/write acceleration. When
enabled, this optimisation sees OneFS use SSDs for reading and writing filesystem metadata - which can improve
access time and reduce latency. Where all underlying storage units are SSD (as in the case of Dell EMC Isilon F800 all-
flash), this is not a meaningful or required optimisation.

ENABLING
Metadata read/write optimisation can be enabled as a File Pool Policy at the OneFS Web User Interface. More
information and guidance is available in published documentation:

• http://doc.isilon.com/onefs/8.1.0/help/en-us/index.html#ifs_c_ssd_pools.html

STREAMING MODE

OVERVIEW
Streaming performance of OneFS can be optimised by enabling Streaming Mode at the file pool, or at the directory level.
Streaming performance is important when playing media linearly - for example when playing a video sequence in a time
line.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
Consideration should be given to other workflows hosted in the same pool or directory before enabling Streaming mode.
Best performance for all workflows on a single cluster might be achieved by selectively enabling Streaming Mode on a
limited number of directories.

OPERATION
Streaming Mode optimises two behaviours of OneFS to deliver increased streaming performance:

• Data is striped across more underlying storage units (disc drives or SSDs).
• Data is pre-fetched aggressively.

ENABLING
Streaming mode can be enabled as a File Pool Policy at the OneFS Web User Interface. More information and guidance
is available in published documentation:

• http://doc.isilon.com/onefs/8.1.0/help/en-us/index.html - ifs_t_configure_default_io_optimization_settings.html
• http://doc.isilon.com/onefs/8.1.0/help/en-us/index.html - ifs_t_modify_file_and_directory_properties.html

FILENAME PREFETCH

OVERVIEW
Cluster-wide FileName PreFetch can be enabled to optimise the streaming performance of OneFS when working with
image sequences. Consideration should be given to other workflows hosted on the same cluster, and the impact of
possible false fetches (where resources are used pre-fetching files that are not ever requested by the client) should be
evaluated.

OPERATION
FileName PreFetch enables a performance optimisation by detecting when image sequences are requested by a client.
When a sequence is detected, OneFS will pre-fetch files from the underlying storage units (disc drives or SSDs) before
they are requested.

Figure 4. FileName PreFetch conceptual overview

The FileName PreFetch algorithm is optimised to detect image sequences with either decimal or hexadecimal filename
numerical increments.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
ENABLING
FileName PreFetch can be enabled across the cluster at the command line of OneFS, authenticated as Root. Working at
the command line of OneFS - particularly when authenticated as Root - should be attempted only by authorised and
qualified System Administrators. Extreme care should be exercised. Mistakes can cause data loss.

To enable FileName PreFetch:

isi_for_array sysctl isi.access.streaming.fnprefetch.fn_enabled=1

isi_for_array sysctl isi.access.streaming.PreFetch.PreFetch_by_cluster=1

MORE INFORMATION
For more information about Streaming Mode and FileName PreFetch, check:

• https://community.emc.com/community/products/isilon/blog/2017/04/05/smartpools-data-layout-PreFetch

For detailed guidance on File Pool Policies and data layout and access strategies, check:

• https://www.emc.com/collateral/hardware/white-papers/h10719-isilon-onefs-technical-overview-wp.pdf

Guidance for using the command line to make administrative changes is available at:

• https://www.emc.com/collateral/TechnicalDocument/docu84280.pdf

Guidance for using the web interface to make administrative changes is available at:

• https://www.emc.com/collateral/TechnicalDocument/docu84277.pdf

CLIENT AND APPLICATION OPTIMISATIONS

Overall performance, and the subjective quality of the user experience often can be enhanced by optimising the
application to take full advantage of the power of Isilon and OneFS. Specific application optimisations are covered in
separate documents.

Uncompressed High-Resolution Workflows with Dell EMC Isilon

© 2017 Dell Inc. or its subsidiaries.
 SUMMARY AND CONCLUSION
Accommodating uncompressed high-resolution workflows can represent a challenge to Architects and System
Integrators. This document aims to outline the nature of the performance required by creative applications, and to direct
the reader to other resources in order to design systems to fulfil those requirements.

Learn more about Dell

EMC Isilon Contact a Dell EMC Expert

© 2017 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC and other trademarks are
trademarks of Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective
owners.Reference Number: XXXXXXXXX