Revisiting Data Lakes: The Metadata Lake

Abstract

We argue that emerging federated data management architectures require a means of gathering, linking, curating and enriching metadata in a graph. We call the system that supports these tasks a metadata lake. We explain the underlying architectural principles that are required to achieve such a system and describe our current implementation. We show how our metadata lake is used to achieve certain advanced capabilities and report on its performance.