Virtualization And Grid Computing Heading

Published Date: 02 Nov 2017

Virtualization became a common technique for IT systems in many application environments. The idea was there in the days of mainframe computing but gained increased popularity in recent years. The availability of powerful virtualization solutions gained special interest in managing commercial and academic data centres. The main goal of these activities is a server and/or storage consolidation for better system utilization and lower cost of ownership. However, virtualization also provides new and powerful features in light of service-based infrastructures or Grids.

Virtualization is often associated with virtual machines and corresponding CPU abstraction. However, the idea and current trends show a broader meaning that also includes different kinds of resources. The majority of current applications are in the area of CPU virtualization, storage virtualization and network virtualization. Generally, virtualization hides the physical characteristics of resources from the resource consumers, which can be end users or applications. Virtualization is used in many different contexts, which can be grouped into two main types:

platform virtualization, involving the simulation of whole computers and

resource virtualization, involving the simulation of combined, fragmented, or simplified resources.

In the area of Grid Computing, virtualization gained more and more interest, but less in terms of service consolidation and/or increasing server utilization. Instead, virtualization allows addressing multiple problems in Grid systems, such as coping with the heterogeneity of Grid resources, the difference in software stacks, and enhanced features in resource management such as more general check pointing or migration models. Adopting virtualization in smart ways gets us closer to real Grid computing with more flexibility in the type of applications and the resources to use.

2.1 Virtualization in Grid

Grid Computing

The primary focus in Grid Computing lies in secure resource sharing in terms of access to computers, software and data in a dynamic environment. Sharing of those resources has to be fine grained and highly controlled. Moreover, Foster proposed a three point checklist which characterizes a Grid more in detail:

delivery of nontrivial qualities of service;

usage of standard, open, general-purpose protocols and interfaces e.g. for inter-communication;

Coordination of resources that are not subject to centralized control.

Development was in the early days mostly driven by specific communities requesting amounts of computational power and storage capacities. E.g. CERN was key player concerning in the use of Grid Computing Middleware. Middleware’s are deployed in about 250 sites in all European countries and allows access to over 50,000 compute nodes. Example middleware’s are gLite, UNICORE and Globus. Grid Middleware development started in 1997 with the German UNICORE project. It was extended and renamed to UNICORE Plus and reached a mature production state. The goal of UNICORE Plus was to develop a grid infrastructure together with a computing portal for engineers and scientists to access supercomputer centres from anywhere on the Internet.

Virtualization and Grid computing heading in similar directions

Virtualization is not a complete solution to how enterprises manage their resources. However, it does provide great capabilities in managing and moving operating systems onto different hardware resources. From a technical standpoint, virtualization does two things extremely well. First, it allows us to run multiple workloads on a single machine with great isolation between those workloads. By providing this hardware level abstraction and strong isolation between multiple host operating systems, if one workload crashes, the other can continue to run unobstructed. The second great value of virtualization is that it's great at suspending, resuming and migrating images around an IT environment, in run-time. Without even shutting down an image, you can move jobs to new machines without any sort of disruption in performance.

Like Grid, virtualization is a trend that's being driven by economics. Rather than having to overprovision on the hardware side to meet peak demands, organizations can use virtualization approaches to get better utilization out of existing (underutilized) hardware.

It's also worth noting that virtualization is possibly on its way to becoming a mainstream approach to managing network resources. IBM Internet Standards & Technology, Program Director, David Martin said, "In the next generation of Grids, applications will not necessarily be designed to run on a certain piece of hardware or on a certain network, but will be written to consume certain types of resources, which could be provided anywhere on the network."

Using Virtualization in Grid Computing

Integration of virtualization and Grid Computing can be impelled at different levels.

Grid Computing Level

The gLite Middleware, which is used by the LHC (Large Hadron Collider) Computing Grid High Energy Physics Virtual Organizations (VO), has strong dependencies to Scientific Linux as operating system. Getting gLite to work with other operating systems is a complex task. By encapsulating the Grid Middleware into a virtual appliance, resource providers can support the VOs with minimal effort. The only requirement is support of a virtualization platform like XEN or VMware. Like the gLite Middleware, also other Grid Middleware’s can be packaged as virtual appliances. Then resource providers no longer need to commit to a designated Grid Middleware, but can setup middleware’s on demand. An approach more preferable is the user-based dynamic deployment of the Middleware services. This makes the resource providers’ intervention in the deployment process superfluous and he no longer has to deal with software management and application requirements.

LRMS Level

Integrating virtualization technology and all its features into the local resource management system (LRMS) layer is one of the key research areas in Grid Computing. Typically an LRMS supports job suspension and checkpointing out of the box, whereas virtualization offers the same features not for jobs, but for virtual machines. As a matter of fact, virtualization exhibits with live and deferred migration of virtual machines even one more feature.

Combining the aforementioned LRMS features (suspension, checkpointing) with migration helps the LRMS in dynamically changing the current resource allocation. Furthermore adjustments of resources (e.g. number of CPUs, RAM) assigned to a virtual machine are possible and allow dynamic updates in service quality. Depending on the virtualization platform this can be done on-the-fly or by suspending and resuming.

Virtual Machines

Among other techniques VMs can ensure the correct execution of the application by encapsulating software configurations in a "well-known" environment. VMs are a powerful tool for Grid sysadmins also. They can easily isolate and partition their systems, so deciding the amount of resources they want to put in the Grid.

Specific VO environments can be prepared following an "install once deploy many" approach, so reducing the operational cost of the infrastructure. These kind of scientific virtual appliances (VA) are being packed and distributed in the form of repositories (for example Bioinformatic VA or physics simulation VA).

A description...

Fig 14: Dynamic Provisioning of Computational Services in Grid

Figure 14 shows that the architecture, developed for dynamic provisioning of computational services in grid. The system leverages virtualization technologies to provide flexible support for different VOs. A given site is able to balance the physical resources allocated to each VO it is supporting. This is achieved by balancing the number of virtual worker nodes allocated to each VO. The system deploys on-the-fly VO-specific (familiar) worker nodes to execute their applications. So the costs to change grid based applications and to operate the infrastructure are significantly reduced.

Figure 14 explains that how computational services can be deployed virtually in Grid. The architecture consists of i) Physical machines, ii) Physical worker nodes, iii) Infrastructure manager, iv) GridWay, v) Virtual Workspace Service (VWS) and vi) Grid middleware components such as Monitoring and Discovery of Services (MDS), Grid Resource Allocation Manager (GRAM) and GridFTP. Basically, users submit their requests to GridWay (meta-scheduler). Depending on a predefined set of policies and the actual Grid load, an Infrastructure Manager deploys VO-specific worker nodes in the Grid computing elements. The deployment of the VMs supporting the worker node is done through the Virtual Wokspace Service (VWS). Once the worker node is up and running it registers in the computing element and this information is pushed to the Information Service (MDS). Then GridWay is able to detect the new slot and submit jobs to it through GRAM. Thus by using Virtual Machines, Grids can be turned in a very effective resource provisioning platform.

2.2 Virtualization in Cloud

Virtualization is a systems admin and data management tool that has many technical uses most of which have nothing to do with the cloud. This technology allows enterprises to use a single piece of physical hardware, to perform the work of many. Multiple operating system instances running on one hardware device are far more economical than a single piece of hardware for every server task. Still, there is no direct link to the cloud from pure virtualization.

Cloud computing, on the other hand, is access through the Internet to business applications running in a non-local environment. Cloud computing can certainly take advantage of virtualization but cloud computing can be accomplished without the use of virtualization.

To describe virtualization, imagine that you are standing between two mirrors and now you see dozens of copies of yourself, each copy was capable of doing tasks independent of the other copies. All the copies are "on" one piece of hardware (the real you). That is virtualization. To bring the two together you would take all those virtualized copies of you and beam them out over many different links to an individual(s) who would control and interact with them remotely. Now, you have cloud computing leveraging virtualization!

Cloud computing and virtualization may be modernizing today’s IT business environment. Together, they are ushering in a new era where companies are granted freedom to run their workspaces without having to conduct non-strategic IT functions such as patches, updates and backups. The risk of not knowing the difference between virtualization and cloud computing can be a costly one as hype around these two technologies reaches a steady increase.

Virtualization and cloud computing go hand-in-hand and virtualizing servers is just the tip of the iceberg. The trend to virtualize everything from servers to processing power to software offerings actually started years ago in the personal sector. In the past, it was common for individuals within major organizations to use virtualized services or cloud computing when at home, but at work they weren't using those services at all. Because corporate IT didn't trust the lack of security of the cloud. When talking about virtualization, cloud computing is a natural component. Cloud computing, which refers to companies using remote servers that can store data and allow users to access information from anywhere, takes three different evolutionary forms.

The first is a public cloud. This could be something like Google docs, where we store the data, or something like Flicker, where photos are stored and can be accessed the items from any device at any time as long as we have an Internet connection.

The second form of cloud computing, which is a private cloud, is emerging rapidly. A private cloud exists when a company added security with cloud computing, yet they still want their people to have access to their bigger files and bigger databases from any device anywhere. Since it is private, it's secure and the public does not have access to it. Companies are now beginning to establish private clouds.

The third iteration is evolution of cloud computing i.e. private/public cloud, also called a hybrid cloud. In this configuration, users have a private part of their corporate cloud that is secure and only accessible by employees, but also they have a part of the cloud that is public where strategic partners, vendors and customers can access limited content.

Virtualization can take many forms aside from servers. For example, one can virtualize a desktop, meaning the desktop is stored virtually in the clouds and can access it from anywhere. We can virtualize an operating system, that means we can use a Mac, yet running the latest Windows operating system on the Mac, or can have a PC and have three different operating systems running all at the same time. That's the power of virtualization.

In terms of implementing virtualization and cloud computing options, organizations are now starting to move quickly. Virtualization received a big push in 2009 and 2010 because of the recession, which prompted many companies to cut their IT budget. Companies realized that one way to save money is through virtualization. For example, virtual desktops alone lower costs by 15%.

Now, in 2013, the factors that are increasing an organization's interest in virtualization are speed and agility. Virtualization enables us to do things faster, thus making the company more agile. Instead of delivering a new service in two months, companies are able to do it in two days. As virtualization and cloud computing become more prevalent, companies are going to need to form new strategic relationships because existing relationships may not have the core competencies needed to drive the fundamental changes that will be needed. Cloud computing is inclusive of virtualization and a way to implement it. However cloud can be implemented without virtualization as well. Cloud and Virtualization both help to deliver optimized resources, on-demand utilization, flexibility and scalability. Areas of differentiation may be the areas of self-provisioning, granular billing/chargeback and APIs.

Cloud was implemented more of a outsourced/hosted model first and then slowly being adopted within the enterprise firewall as an architecture. Virtualization on the other hand was started within the boundaries of enterprise firewall and then was utilized in hosted environments. Even if there are differences and similarities, many in the industry use them interchangeably.

Difference between Cloud Computing from virtualization and the WWW

Cloud Computing and Virtualization are two different technologies, these technologies will work independently. Although, cloud computing is better utilized if desktop Virtualization is done first, since most Cloud computing requires multiple virtual servers and storage devices to work or what we call multitenancy. Virtualization on the other hand, enables a business or an individual to save on their computing needs and resources by making virtualized version of a physical resource in the same resource. For example server A is only utilized by 30% most of the time and server B is utilized only by 50%. These two can be combined together by making a virtual server A in server B and therefore not physically using Server A anymore. This means that you save money and resources by eliminating server A from the equation. Below table 1 shows the similarities among Cloud Computing and Virtualization.

Table 1: Similarities among Cloud Computing and Virtualization

 

Cloud Computing

  Virtualization

Location of Virtual Machine

On any host

On a specific host

VM/Instance Storage

Shortly Lived

Persistent

VM resource (CPU, RAM etc.)

Standard

Customizable

Resource Changes

Spin Up new Instance

Resize VM itself

Recovery from Failures

Discard Instance Spin up New one

Attempt to recover failed VM

Relationship between cloud computing, virtualization and the Internet

Virtualization breaks the bond between processing and physical machines, cloud computing enables the use of scalable processing facilities offered by online providers, through virtualization. From an architectural standpoint the question of what should run, where and a clear understanding of the relative cost of processing is essential. Figure 15 shows the cloud computing adoption model. Cloud computing adaptation model consists of layers such as i) hyper cloud where dynamic sharing and self service application provisioning are done, ii) cloud foundations where cloud environments, deployments ways, load balancing are done, iii) cloud exploitation where foundations for scalable application architecture are carried out, iv) cloud experimentations where architectures experimented and v) virtualization where infrastructure and applications are virtualized.

A description...

Fig 15: Cloud Computing Adoption Model

Virtualization as an element of cloud computing

Cloud computing is as much a methodology as it is a technology. We cannot plan any single element without considering the effect on the others. Also we have to add in practices and policies that govern chargeback, monitoring, procurement and many other facets of our IT infrastructure.

Private cloud computing does not center on virtualization or any one technology. It uses a set of technologies that have been aligned to be highly flexible and provide a wide range of services. This approach does not require virtualization, but virtualization does lend well to the core concepts of cloud computing.

Virtualization and cloud computing are also so closely connected because the major hypervisor vendors, VMware, Microsoft and Citrix Systems are putting a lot of emphasis on the cloud. They have closely aligned their products with tools and complementary technologies that promote the adoption of private cloud computing.

Cloud computing is a rapidly evolving discipline and one that will reshape organization charts as fast as it will change data center layouts. It closely aligns with virtualization, but it takes many technologies to be successful.

2.3 Need of Virtualization in Cloud Computing

The Benefits of Virtualization and Cloud Computing

Cloud computing is essentially the ability to acquire or deliver a resource on demand, configured however the users chooses and paid for according to consumption.  From a supplier's perspective, including both internal IT groups and service providers, it means being able to deliver and manage resource pools and applications in a multi-tenancy environment, to deliver the user an on-demand, pay-per-use service. A cloud service can be infrastructure for hosting applications or data storage, a development platform, or even an application that you can get on-demand, either off-site at a provider.

But even more importantly, we have seen new technologies evolve over the past decade that are essential to the notion of the cloud.  The key technology is virtualization. In addition to some amazing cost savings and goodness for the environment, virtualization's ability to separate the OS and application from the hardware give it ideal properties to best deliver these on-demand cloud services. Without virtualization there is no cloud that's what enabled the emergence of this new, sustainable industry.

In other words, users are able to look at the compute power as a centralized resource that they can now allocate to business units on demand, while still maintaining control and operational excellence. Leveraging virtualization to better serve users give organization the obvious lower TCO, but also allows for accountability of usage, simplifies and meets the needs of on-demand infrastructure requests and allows ability to serve, control and manage SLAs. Hence, virtualization has played and will continue to play a huge role in cloud computing. It is the technology that has allowed service providers to deliver lower-cost hosting environments to businesses of all sizes today. Just as virtualization has enabled us to consolidate the servers and do more with less hardware, it also lets us to support more users per piece of hardware and deliver applications and the servers on which they run faster to those users.

The Important Role of Virtualization in Securing the Cloud

Security in the cloud is better as a result of virtualization, makes the cloud even more secure than your physical datacenter. Because virtualization consolidates multiple physical components so that they can be managed in one place, it mitigates the complexity of monitoring these components across both internal and external infrastructure. For example, through virtualization, IT teams can standardize VM images and create back up versions of critical VMs more frequently than in the past, simplifying recovery. By virtualizing the company’s infrastructure, IT admin’s can also create trust zones around information, applications and endpoints that can be adapted to follow workloads through the cloud. Automated policies can then assess risk and immediately initiate remediation with security problems arise. In short, virtualization enables organizations to have greater control and better visibility into their infrastructure, simplifying security management for the cloud.