October 22nd, 2012 by Guest Author
Real World Applications for SSD Caching: Visual Effects and Post Production Video
Visual effects (commonly shortened to Visual F/X or VFX) have become a core part of any modern video production, including films, commercials, or online videos. Visual effects include the various processes to integrate realistic computer-generated imagery with live-action footage. Visual effects and computer generated scenes are usually created (rendered) through multiple iterations from mathematical models.
Video rendering is a very resource intensive process which can take multiple hours to create a video scene. Every time the producer makes a change, the video needs to be rendered again. So, when you sum up the multiple rendering instances required for the edits, a typical movie, video, commercial, or video game takes days or weeks of rendering to complete. Because production cannot move forward until rendering is complete, accelerating the rendering speed means less waiting time and a shorter video production process.
Because visual effects are known to take a lot of processing power, video rendering systems are designed with powerful CPUs: typical visual effects production companies have dozens systems with 12-24 CPU cores, 100+GB of RAM, and advanced storage systems (e.g. Bluarc, Avere, etc.) But with the focus on processing, many people overlook the I/O-bound activities that can have a significant impact on the performance of the overall rendering process. One example of an I/O-bound sub-process that can slow rendering down is using disk as scratch space for temporarily storing data. Removing the I/O bottleneck can substantially increase rendering speed. For example, VeloBit customers were able to accelerate the performance of the video rendering application by 3x and reduce the time it take to render a frame from two hours to under 45 minutes (with SSD caching). The resulting business impact is tremendous! A 300% increase in productivity means that producers can now double or triple the number of iterations they can make in a given time frame, which would greatly enhance the quality of the end production, or can triple the projects they can work on (and, ideally, double or triple the revenue while keeping infrastructure costs essentially the same).
Let’s explore the use of scratch space in more detail: a typical 8 bit frame with HD resolution takes 8 MB. A 3 minute video at 30 frames/second will be 42 GB. However, because multiple interim scenes will be rendered before the final video is created, the process may use up to 2TB of storage for scratch space. Scratch data (small chunks of data used to construct a frame) is scattered throughout the available scratch disk and is read and re-written multiple times until the final frame is constructed. Every read and every write introduces latency and slows the overall video rendering process.
SSD caching can improve rendering performance by 3x or more. Caching software places frequently used data blocks on SSD (which has an order of magnitude higher I/O performance than HDD) which improves application performance. Since large portions of the frame data is identical, you can increase performance with a relatively small SSD size (which keeps your costs low). Our customers have increased rendering performance by 3x or more using SSD that is only 20% of the overall data set. The best part is that you can keep your existing primary storage system and you don’t have to change anything in your application or process to take advantage of this performance boost. You just plug an SSD into an available slot in your server and install the SSD caching software. Everything else happens automatically and transparently to the rendering application.
“SSD caching provides a great opportunity to improve the rendering process with minimal costs and no disruption,” said Todd Dauherty, VFX Tactician at Very Heavy, a VFX and Post Production consultancy. “I have boosted performance and reduced rendering time by 3x for more than a few customers, while enabling them to keep their existing storage infrastructure investments and processes.”
Many of the vendors, including VeloBit, offer free trial of their SSD caching software. Setup is easy and quick: you can install and configure your SSD cache in just a few minutes. You will need an SSD (some vendors will now even loan you an SSD for the duration of the trial), and a few GB of RAM on your server. Run your application for a couple of hours and you will be able to measure the performance increase.
About the Author
VeloBit provides plug & play SSD caching software that dramatically accelerates applications at a remarkably low cost. The software installs seamlessly in 60 seconds and automatically tunes for fastest application speed. Visit www.velobit.com for details.