Redundant Arrays of Independent Libraries (RAIL): The StarFish tertiary storage system

Abstract

Increased computer networking has sparked a resurgence of the 'on-line' revolution of the 1970's, making ever larger amounts of data available on a world wide basis and placing greater demands on the performance and availability of tertiary storage systems. In this paper, we argue for a new approach to tertiary storage system architecture that is obtained by coupling multiple small and inexpensive 'building block' libraries (or jukeboxes) together to create larger tertiary storage systems. We call the resulting system a RAIL and show that it has performance and availability characteristics superior to conventional tertiary storage systems, for almost the same dollar/megabyte cost. A RAIL system is the tertiary storage equivalent of a fixed magnetic disk RAID storage system, but with several additional features that enable the ideas of data striping and redundancy to function efficiently on dismountable media and robotic media mounting systems. We present the architecture of such a system called StarFish I and describe the implementation of a prototype. We also introduce the idea of creating a log-structured library array (LSLA) on lop of a RAIL architecture (StarFish II) and show how it can have write performance equivalent to that of secondary storage, and improved read performance along with other advantages such as easier compression and the elimination of the 4 X RAID/RAIL write penalty. © 1998 Elsevier Science B.V.