A fundamental goal of computational “Grids” is to allow flexible, secure sharing of resources distributed across different administrative domains. To realize this vision, a key challenge that must be addressed by Grid middleware is the provisioning of execution environments that have flexible, customizable configurations and allow for secure execution of untrusted code from Grid users. Such environments can be delivered by architectures that combine “classic” virtual machines (VMs) and middleware for dynamic instantiation of VM instances on a per-user basis. Efficient instantiation of VMs across distributed resources requires middleware support for transfer of large VM state files (e.g. memory, disk) and thus poses challenges to data management infrastructures. This paper shows that a solution for efficient and secure transfer of VM state across domains can be implemented by means of extensions to a user-level distributed file system virtualization layer.

Mechanisms that present in existing middleware can be utilized to support this functionality by treating VM-based computing sessions as processes to be scheduled (VM monitors) and data to be transferred (VM state). In order to fully exploit the benefits of a VM-based model of Grid computing, data management is key: without middleware support for transfer of VM state, computation is tied to the end-resources that have a copy of a user’s VM; without support for the transfer of application data, computation is tied to the end-resources that have local access to a user’s files. However, with appropriate data management support, the components of a Grid VM computing session can be distributed across three different logical entities: the “state server”, which stores VM state; the “compute server”, which provides the capability of instantiating VMs; and the “data server”, which stores user data.

The proposed VM state provisioning solution is constructed upon a user-level distributed file system virtualization layerwhich leverages the NFS de-facto distributed file system standard and provides a basis for establishing dynamic Grid Virtual File System (GVFS) sessions. GVFS extends upon the virtualization infrastructure at the user level to support on-demand, secure and high-performance access to Grid VM state. It leverages SSH tunneling and session-key based cross-domain authentication to provide private file system channels, and addresses performance limitations associated with typical NFS setups in wide-area environments (such as buffer caches with limited storage capacity and write-through policies) by allowing for user-level disk caching. It also supports application-driven meta-data at the file system level to allow for data requests being satisfied using partial- or full-file transfer selectively to efficiently handle VM state files. These mechanisms are implemented transparently to the kernels and applications, and hence support unmodified VM technologies, such as VMware, UML and Xen, which use the file system to store VM state.

Services support for VM-based grid computing