Multi Deployment And Multi Snapshotting On Clouds

Published Date: 02 Nov 2017

ABSTRACT

In recent years, Infrastructure as a Service cloud computing has emerged as a viable alternative to the acquisition and management of physical resources. With Infrastructure as a Service, users can lease storage and computation time from large data centers. Leasing of computation time is accomplished by allowing users to deploy virtual machines on the data center’s resources. Since the user has complete control over the configuration of the Virtual Machines using on demand deployments, Infrastructure as a Service leasing is equivalent to purchasing dedicated hardware but without the long-term commitment and cost. The on-demand nature of Infrastructure as a Service is critical to making such leases attractive, since it enables users to expand or shrink their resources according to their computational needs, by using external resources to complement their local resource base. This emerging model leads to new challenges relating to the design and development of Infrastructure as a Service systems. One of the commonly occurring patterns in the operation of Infrastructure as a Service is the need to deploy a large number of Virtual Machines on many nodes of a data center at the same time, starting from a set of Virtual Machine images previously stored in a persistent fashion. For example, this pattern occurs when the user wants to deploy a virtual cluster that executes a distributed application or a set of environments to support a workflow. We refer to this pattern as Multideployment. Such a large deployment of many Virtual Machines at once can take a long time. This problem is particularly acute for Virtual Machine images used in scientific computing where image sizes are large (from a few gigabytes up to more than 10 GB). A typical deployment consists of hundreds or even thousands of such images. Conventional deployment techniques broadcast the images to the nodes before starting the Virtual Machine instances, a process that can take tens of minutes to hours, not counting the time to boot the operating system itself. This can make the response time of the Infrastructure as a Service installation much longer than acceptable and erase the on-demand benefits of cloud computing. Once the Virtual Machine instances are running, a similar challenge applies to snapshot ting the deployment: many Virtual Machine images that were locally modified need to be concurrently transferred to stable storage with the purpose of capturing the Virtual Machine state for later use (e.g., for check pointing or off-line migration to another cluster or cloud). We refer to this pattern as Multisnapshotting.

RELATEDWORK

Multideployment that relies on ful lbroad cast-based pre-propagation is a widely used technique [28, 31, 14]. While this technique avoids read contention to the repository, it can incur a high overhead in both network traffic and ex-ecution time, as presented in Section 5.2. Furthermore, since the VM images are fully copied locally on the compute nodes, multisnapshotting becomes infeasible:large amounts of data are unnecessarily duplicated and cause unacceptable transfer delays, not to mention huge storage space and net-work traffic utilization.

In order to alleviate this problem, many hypervisors pro-inria-00570682, version 1 - 23 Mar 2011 IEEE/ACM CLOUD COMPUTING June 2011 vide native copy-on-write support by defining custom VM image file formats [12, 26] specifically designed to efficiently store incremental differences. Much like our approach, this allows base images to be used as read-only templates for multiple logical instances which store per-instance modifica-tions. However, lack of standardization and the generation of many interdependent new files limit the portability and manageability of the resulting VM image snapshots. A different approach to instantiate a large number of VMs from the same initial state is proposed in [18]. The authors introduce a new cloud abstraction: VM FORK. Essentially this is the equivalent of the fork call on UNIX operating sys-tems, instantaneously cloning a VM into multiple replicas running on different hosts. While this is similar to CLONE

followed by COMMIT in our approach, the focus is on mini-mizing the time and network traffic to spawn and run, on the fly, new remote VM instances that share the same state of an already running VM. Local modifications are assumed to be ephemeral, and no support to store the state persistently is provided. Closer to our approach is Lithium [13], a fork-consistent replication system for virtual disks. Lithium supports in-stant volume creation with lazy space allocation and instant creation of writable snapshots. Unlike our approach, which is based on segment trees, Lithium is based on log struc-turing [29], which can potentially degrade read performance when increasing the number of consecutive snapshots for the same image: the log of incremental differences starts grow-ing, making it more expensive to reconstruct the image. Cluster volume managers for virtual disks such as Paral-

lax [22] enable compute nodes to share access to a single, globally visible block device, which is collaboratively man-aged to present individual virtual disk images to the VMs. While this enables efficient frequent snapshotting, unlike our approach, sharing of images is intentionally not supported in order to eliminate the need for a distributed lock manager, which is claimed to dramatically simplify the design. Several storage systems, such as Amazon S3 [6] (backed

objec-tives; hence, we believe our work is a welcome by Dynamo [11]), have been specifically designed as highly available key-value repositories for cloud infrastructures. They can be valuable building blocks for block-level storage vol-umes [1] that host virtual machine images; however, they are not optimized for snapshotting. Our approach is intended to complement existing cloud computing platforms, both from industry (Amazon Elastic Compute Cloud: EC2 [5]) and from academia (Nimbus [3,17, 16], OpenNebula [4]). While the details for EC2 are not publicly available, it is widely acknowledged that all these platforms rely on several of the techniques presented

above. Claims to instantiate multiple VMs in "minutes," however, are insufficient for meeting our performance addition in

this context.

Existing System:

Multi-deployment that relies on full broadcast-based pre-propagation is a widely used technique. While this technique avoids read contention to the repository, it can incur a high overhead in both network traffic and execution time. Furthermore, since the VM images are fully copied locally on the compute nodes, multi-snapshotting becomes infeasible: large amounts of data are unnecessarily duplicated and cause unacceptable transfer delays, not to mention huge storage space and network traffic utilization.

Disadvantages:

Lack of standardization.

Portability and manageability of the resulting VM image snapshots.

High network Traffic.

High execution time.

Proposed System:

This paper addresses these challenges by proposing a virtual file system specifically optimized for virtual machine image storage. It is based on a lazy transfer scheme coupled with object versioning that handles snapshotting transparently in a hypervisor-independent fashion, ensuring

high portability for different configurations. Large-scale experiments on hundreds of nodes demonstrate excellent performance results: speedup for concurrent VM deployments ranges from a factor of 2 up to 25, with a reduction in bandwidth utilization of as much as 90%.

Our contributions are can be summarized as follows:

• We introduce a series of design principles that optimize multi-deployment and multi-snapshotting patterns and describe how our design can be integrated with IaaS infrastructures.

• We show how to realize these design principles by building a virtual file system that leverages versioning-based distributed storage services. To illustrate this point, we describe an implementation on top of BlobSeer, a versioning storage service specifically designed for high

throughput under concurrency.

• We evaluate our approach in a series of experiments, each conducted on hundreds of nodes provisioned on the Grid’5000 testbed, using both synthetic traces and real-life applications.

Advantages:

Enable efficient concurrent deployment and snapshotting.

Ensure a maximum compatibility with different configurations.