A distributed job scheduling and flow management system

Abstract

Grid computing, as a specific model of distributed systems, has sparked recent interest in managing job execution among distributed resource domains. Introduction of the meta-scheduler is a key feature in grid evolution, and the next step is to achieve collaborative interactions between meta-schedulers within and external to organizational boundaries to achieve scalability, balanced resource utilization, and location transparency to job submitters. This paper details a distributed system design that consists of a collaborative meta-scheduling framework, and an expanded resource model with schedulers and data as resources. With this framework, we also explore job scheduling and data management issues, and investigate job flow and meta-scheduling interactions for new applications that require job execution beyond simple sequential and conditional control.