Optimising the storage infrastructure for 4K workflows is challenging, because of the size and quantity of files, as well as the variety of formats involved, explains Dave Frederick, senior director, media and entertainment of Quantum
4K workflows present much higher demands, with much greater variability, than the 2K and HD workflows now commonplace in the media and entertainment industry. Post-production facilities moving into 4K must deal not only with a dramatic jump in data but also with a corresponding increase in the bandwidth and processing power needed to work effectively with the higher-resolution content. While the sheer size and quantity of files can be challenging to storage infrastructure, the variety of formats — different types of HD, 2K, 4K and soon 8K — makes for wide variances in file sizes, as well. Requirements for data, capacity and performance are influenced by factors ranging from spiking demand for ‘anywhere, anytime’ content to shrinking deadlines, as well as the usual vagaries of a post project. By looking closely at key aspects of workflow storage, however, post-production facilities can reduce the number of unknowns and plan effectively for their 4K future. In addition to considering storage performance and scalability, they must also evaluate their content accessibility needs and preferred approach to content archiving.
Storage system performance is largely dependent on three factors: drive management, network management and file system capabilities. Understanding how each of these elements affects the workflow makes it easier to identify the solution that will best address the post facility’s format (data rate), capacity and stream count (number of simultaneous users) requirements.
Within a 4K-ready storage infrastructure, the aggregate performance of underlying drives must be managed to provide the gigabyte-per-second speeds required for high-resolution workflows. What does this mean for drive choice? Well, consider that the bandwidth of individual spinning hard disk drives (HDDs) ranges from 75 MB/s to 150 MB/s, and that solid-state drives (SSDs) can deliver up to 1900 MB/s. Consider also that one minute of uncompressed 4K requires about 50 GB of storage capacity, and that the most costly enterprise SSDs can accommodate up to 2 TB, or about 40 minutes, per drive.
Ultimately, while the speed of SSDs is quite compelling, the costs add up quickly in the face of 4K capacity requirements. Though prices for SSDs are falling, HDDs continue to be far less costly in per-gigabyte terms. HDDs with RAID 6 protection offer the best price-to-performance benefits, particularly when most disk operations are sequential reads and writes of individual frames and files. In other scenarios, such as animation and VFX work, SSDs, or an SSD/HDD mix with intelligent data movement across storage media, can provide the best combination of performance and cost.
The way in which network connections attach to storage infrastructure has a direct impact on performance. Rather than engage in a debate over the superiority of network attached storage (NAS) versus storage area network (SAN), the better course today is to determine a combination of the two that can best provide the workstation and client connectivity essential to support 4K workflow.
Delivering fixed, predictable bandwidth between workstations and shared storage, Fibre Channel SAN connections address those elements of post-production that demand fast, reliable access. Ethernet-based NAS connections are more affordable but require time-consuming processing overhead, making them appropriate for the many other operations largely unaffected by the small delay. This unified model offers the greatest benefits with both SAN and NAS connectivity extended across Linux, Windows and Mac operating systems.
A sophisticated file system can enable multiple Windows, Linux and Mac workstations to access shared storage over a Fibre Channel network, allowing them to read and write to the same storage volume at the same time without network delays. As a result, content creators working in the 4K realm can enjoy the immediate, shared access to media files that is key to effective collaboration. When storage extends across multiple tiers, sometimes including cloud storage, the file system can not only simplify the presentation of files to applications and users but also leverage policies to ensure that content is in the right place at the right time.
In any operation experiencing growth, continuous productivity demands the ability to add storage capacity without the need for file migration or downtime. The ability to tune performance to changing requirements, whether due to format type or volume of work, is likewise essential to high productivity. When performance and capacity can be scaled independently of one another, the post facility can achieve the ideal balance for a given workflow, or even to suit the roles and requirements of different individuals within the workflow.
Keeping completed high-resolution projects on speedy production storage is expensive — but it is also critical for rapid turnaround of multiplatform deliverables. Possessing lengthy drive rebuild times, RAID-based NAS systems become an increasingly risky alternative as operating hours and capacity rise. Object storage, on the other hand, offers self-healing capabilities that ensure the integrity and availability of files and associated metadata, stored as objects, across many drives. With 4K content driving up capacity requirements, the uptime guaranteed by object storage makes it a compelling addition to storage infrastructure.
In fact, object storage is a popular choice for both private and public cloud storage, which serves as economical, space-saving offsite repositories for content. Some aspects of high-resolution work, such as uncompressed editorial and VFX, aren’t a natural fit for the cloud, but cloud-based services can simplify or reduce the cost of common tasks such as review and approval, transcoding, rendering and distribution.
Content Preservation and Archiving
Long-term storage of 4K project files and masters calls for a cost-effective and durable storage medium, and LTO (linear tape open) technology fits the bill. Tape cartridges last longer than mechanical drives and use a fraction of the energy that spinning disks require. The current generation technology, LTO-7, provides 6 TB of uncompressed data storage that can be written at 300 MB/s at less than a penny per gigabyte; the next-generation LTO-8 promises impressive speed gains and even greater savings.
The continued performance improvements that make tape a candidate for nearline storage are also giving post facilities much faster access to their archived content. This is a valuable improvement, particularly with increasing consumer demand for content making a readily accessible archive easier to monetize.
Although public cloud-based archive services offer easy and affordable long-term storage, delayed access times and slower download speeds make them optimal for off-site storage within a larger backup strategy.
Establishing a 4K-ready storage infrastructure naturally involves some speculation with respect to the performance and capacity needed to support higher-resolution workflows today and tomorrow, as well as the influence of an evolving and expanding media marketplace. With the right combination of drive, network and file management in a system that affords high reliability, continuous scalability, appropriate levels of availability and economical content preservation, post facilities will be fully prepared to accommodate the intense and often unpredictable requirements of working in the high-resolution 4K realm.