Supporting media workflows on an advanced cloud object store platform

Abstract

The media industry faces daily challenges storing, managing and facilitating collaboration for vast amounts of video, photos, text, audio, and social media. This, together with increasing quality factors like high frame rates and Ultra-High resolution, drives the need for more dependable storage and scalable computation. As a solution to tackle these challenges, we describe the Active Media Store (AMS), which extends OpenStack Swift with storlets and content-centric access. Computation units called "Storlets" run typical media workflow functions, such as transcoding, metadata extraction and quality checks, close to the stored data, thereby reducing data transfer bandwidth and latency typically associated with cloud storage. Powerful search capabilities over rich user-defined metadata enable content-centric access, the ability to find stored media objects based on their content and relationships. Media workflows are easily implemented and managed through the Media Bridge, middleware running over the AMS and adapting its low level APIs. We describe an example workflow for file staging over this infrastructure in order to illustrate its power.