Our Storage Server

GNETICS Film Group is in the process of acquiring a brand new storage server for use in our advanced digital media program. This server will allow students to efficiently work from any computer in the school, without worrying about uploading large files to the web or carrying them on an external hard drive.

Our Current Problem

We live in an era of high-resolution video. It's no longer enough to just publish for DVD or even television - the internet makes streaming very high-resolution video easy for people with access to a fast enough connection.

Traditional television broadcasts can transmit at up to 1080i quality, or 1920 pixels vertically by 1080 pixels horizontally. Each one of those pixels carries an instruction of what color it should be (and in some cases, even how bright the pixel should be), and the tight composition of all these pixels creates the image you see on your screen. But nowadays, 4K video, or four 1080p images, is becoming more and more mainstream. YouTube even supports uploading videos at 8K quality! But why do we need to shoot in such high-resolution?

4K quality video comes with a number of benefits. For one, all of those extra pixels make the image look clearer and sharper, so grass looks more vivid and small details like text on a chalkboard are easier to read for viewers. 4K video is also helpful in post-production, as having approximately 4x the resolution as 1080p allows us to digitally upscale images by up to 4x with minimal/no reduction in quality. This allows our editors to re-frame images, meaning that they can zoom in on a part of the video and change what part of the footage the viewer sees all without having to re-shoot anything. In addition to that, having extra pixels makes a huge difference when we produce visual effects, since our software has more pixels to analyze when we perform tasks like motion tracking and chroma keying.

In the coming years GNETICS plans to produce more long-form, high-resolution documentaries. We've already wrapped up production on our next film, all about conservation efforts on the Galapagos Islands. But if we want to store everything we shoot in high-resolution AND allow multiple students to edit a project simultaneously, we need something better than a single external hard drive. That's where our new storage server comes into play.

What is a Storage Server?

The problem is that 4K video brings with it enormous file sizes, especially when we film at high frame rates. To put this into perspective, our entire BME documentary was shot at 4K 24 frames per second at a bitrate of 100 Mbit per second. All 25+ interviews, as well as our B-roll, consumed over 2.6 Terabytes (approx 2662 Gigabytes) of storage on our hard drive! That's the equivalent of forty-one 64 GB iPhones, completely filled up!

What's even worse is that this makes collaboration very difficult. When we edited our BME documentary, we had one student hold the entire project (the Final Cut Pro library) on his 6 TB external hard drive. The drive was big enough for the project, but this meant that no one else could work on editing the video because the files were on his hard drive, which could only be connected to one computer at a time. In order to share interviews for review, he had to upload the files as private YouTube videos. In addition to this, the hard drive operated a maximum speed of 220 MB/s, and editing became difficult when lots of videos were stacked on top of each other in our editing software.

A storage server is basically a fancy computer with one single purpose - to serve files to multiple users simultaneously. In this case, the files are high-resolution video clips and 3D animations created by our students. Servers are network-connected, meaning that the files can be accessed by any student using any computer in the building, whether that's a laptop in the lunch room or a fast workstation PC in the digital media classroom. Our new school building was built with a network infrastructure fast enough to support transferring huge files over the network, even over Wifi.

Both the server and our school's network support the 10 Gigabit (10/100/1000/10000) networking standard, allowing us to transfer files at a maximum of 1.25 GB per second - much faster than our 220 MB per second hard drive. Faster file transfers mean we'll see fluent timeline performance when editing on our Retina 5K iMacs and Dell Precision workstation PCs, even when working with very large timelines with lots of effects. And because we're hosting our project files and all of our media on the server, one student can start a project on one computer and another student can pick up right where they left off from another, all without having to share the files over AirDrop or wait for them to transfer to a hard drive.

Our New Server

We decided to purchase a server from 45Drives because our CEO had heard of them and their reputable reputation. 45Drives produces the Storinator, a highly configurable storage server used by MIT, Lockheed Martin, deadmau5, Stanford University, and many more. After a short call with a representative, we configured a Storinator Q30 that would allow us to improve our workflow now and ensure that we can store everything we shoot for years to come.

Our Configuration:

Motherboard: X10SRL (DDR4)

The motherboard is like the backbone of the server. It connects the hard drives and every other component so that power and data can flow throughout the entire system. This motherboard supports a fast server processor and RAM.

Processor: E5-2620 v4

With 22 cores and 44 threads, this Intel Xeon processor acts as the server's brain, running the operating system and any necessary calculations needed for data storage and transfer.

RAM (Random-Access Memory): 64 GB ECC

Plenty of error-correcting RAM will help to run the server's operating system, and will ensure that any bit-level memory errors will not result in data loss or corruption.

Boot Drive: Single + Redundant 120 GB SSD

These high-speed solid-state boot drives will host the server's operating system and all of it's required files. We are looking into using additional SSD storage drives for use as a media cache, so our most commonly accessed files see the best read/write performance.

HBA (Host Bus Adapter): 2 x LSI 9305 12 Gb/s

This HBA interfaces the hard drives to the motherboard. The two adapters in our Q30 unity support up to 30 separate hard drives.

NIC (Network Interface Card): Copper NIC

This network card uses traditional copper RJ-45 connectors that allows our server to connect to our network switch (which connects it to all of the computers)

Storage: 10 x 15 TB WD Gold HDD

Providing us with 150 TB of storage, these high-performance Western Digital Gold hard drives are an excellent choice for high-performance network storage.

Operating System: FreeNAS

Despite our CEO's love of unRAID, we chose FreeNAS for our server's operating system on the recommendation of a representative from 45Drives. FreeNAS supports the open source ZFS filesystem, in addition to many other features such as remote web administration, snapshots, and replication. It's got everything we need in a network-attached storage operating system.

Benefits to Our Workflow

Editors see better timeline performance with a 1.25 GB/s theoretical network transfer speed
150 TB of persistent storage allows us to archive years worth of 4K video, so old B-roll shots and interviews can be utilized in new projects in the future
Saving both the project files and their associated media allows us to re-edit old projects in the future
VFX editors can transcode native H.264 content to a codec like ProRes and store the large files on the server to see better performance in Adobe After Effects
Entire classes of stuents can work on their projects simultaneously, without worrying about someone else using the computer their project is stored on
Teams can assign students to different post-production tasks, allowing for simultaneous footage review and select pulling
The server can be configured to automatically back itself up, allowing for it to retain all of it's data in the event a hard drive fails
The server can be configured to allow secure access outside of the local network, allowing for a student to start a project in school and finish it from home.
Teams editing in Adobe Premiere Pro, Davinci Resolve, and eventually Avid Media Composer can simulataneously edit the same video project at the same time. This will be supported in Final Cut Pro if Apple chooses to update the software with this feature.