|
UNIT – 3 |
||||||||||||||||
|
Cloud Management
and Virtualization Technology |
||||||||||||||||
|
Unit-03/Lecture-01 |
||||||||||||||||
|
Cloud
Resiliency: Resilient
computing is a form of failover that distributes redundant implementations of
IT resources across physical locations. IT resources can be pre-configured so
that if one becomes deficient, processing is automatically handed over to
another redundant IT resource. Within cloud computing, the characteristic of
Resiliency can refer to redundant IT resources within the same cloud (but in
different physical locations) or across multiple clouds. Cloud consumers can
increase the reliability and availability of their applications by leveraging
the resiliency of cloud-based IT resources (Figure 1). Figure 1 How
can the cloud provide Resilience: ·
The cloud
is about more than just data storage and ‘anytime/anywhere’ file access; it
is increasingly being used as a disaster recovery tool. ·
With
sensitive data, it’s important to plan for the best but prepare for the
worst. Companies who need to avoid data loss – that is to say, almost every
business – are recognising the value of cloud solutions. ·
Cloud
solutions are capable of both protecting against and dealing with a potential
catastrophe, but it’s important to know exactly what plans are in place and
how the measures taken will cope. ·
It is
important to remember that most organisations will have a mix of physical and
virtual systems and need to account for both in their disaster recovery plan.
In addition, there are many options available and businesses have to select
what will suit them the best, be that physical like-for-like replication for
business-critical production workloads, or using the cloud. These choices can
be far from straightforward. ·
For
those with a mix of physical and virtual infrastructure, using the cloud for
disaster recovery is not a case of simply replicating data; it largely
depends on the size and scope of the production workloads to be protected,
and selecting the disaster recovery solution that is the most suitable for
its replication. ·
It is
essential that these issues are addressed early on to ensure the
infrastructures work together. This then also dictates the selection of
appropriate third party cloud vendors, as organisations cannot simply cherry
pick a provider. The storage in use by both parties needs to be compatible. ·
Cloud
computing can also provide separation between production systems and their
disaster recovery counterparts. A suitable geographical distance helps avoid
knock-on effects in the event of a localised disaster. ·
While
the cloud certainly has its place in today’s disaster recovery strategies it
is essential to stress the importance of proper planning – and testing. In an
ideal situation the entire production workload would be replicated. However,
in reality businesses have to accommodate their budget and prioritise
critical systems that cannot tolerate downtime. It is also essential that the
disaster recovery systems are kept up to date with the evolution of systems
and services and has regular attention. This means regular testing. This can
present a challenge, especially for businesses operating 24/7 as testing
cannot disrupt normal operations. ·
Overall,
when considering the cloud for disaster recovery it is critical that
businesses work with a trusted provider with the expertise to design a
disaster recovery plan that is specific – not only to the organisation’s
needs but also addresses the service levels it has with its own customers.
With the correct approach, disaster recovery in the cloud can be the perfect
solution and often at a much lower cost point when compared to traditional
models. Cloud
Provisioning: When a cloud provider
accepts a request from a customer, it must create the appropriate number of
virtual machines (VMs) and allocate resources to support them. Cloud provisioning refers
to the processes for the deployment and integration of cloud computing
services within an enterprise IT infrastructure. This is a broad term that
incorporates the policies, procedures and an enterprise’s objective in sourcing
cloud services and solutions from a cloud service provider. Cloud provisioning
primarily defines how, what and when an organization will provision cloud
services. These services can be internal, public or hybrid cloud products and
solutions. The process is conducted in
several different ways: advance provisioning, dynamic provisioning and user
self-provisioning. In this context, the term provisioning simply means “to provide."
With advance provisioning, the customer
contracts with the provider for services and the provider prepares the
appropriate resources in advance of start of service. The customer is charged
a flat fee or is billed on a monthly basis. With dynamic provisioning, the provider
allocates more resources as they are needed and removes them when they are
not. The customer is billed on a pay-per-use basis. When dynamic provisioning
is used to create a hybrid cloud, it is sometimes referred to as cloud
bursting. With
user self-provisioning (also known as
cloud self-service), the customer purchases resources from the cloud
provider through a web form, creating a customer account and paying for
resources with a credit card. The provider's resources are available for
customer use within hours, if not minutes. From a
provider’s standpoint, cloud provisioning can include the supply and
assignment of required cloud resources to the customer. For example, the
creation of virtual machines, the allocation of storage capacity and/or
granting access to cloud software.
------------------------------------------------REFERENCE {Internet link:http://searchcloudprovider.techtarget.com/definition/cloud-provisioning}
Unit-03/Lecture-02 Concepts of Map Reduce The Map-Reduce programming model: Map Reduce is a programming
platform Google introduced for processing large quantities of data. It
expresses the computational logic of an application in two simple functions: Map and reduce. Data transfer and
management are completely handled by the distributed storage infrastructure
(i.e., the Google File System), which is in charge of providing access to
data, replicating files, and eventually moving them where needed. Therefore,
developers no longer have to handle these issues and are provided with an
interface that presents data at a higher level: as a collection of key-value pairs.
The computation of Map Reduce applications is then organized into a workflow
of map and reduce operations that is entirely controlled by the runtime
system; developers need only specify how the map and reduce functions operate
on the key-value pairs. More precisely, the MapReduce
model is expressed in the form of the two functions, which are defined as
follows: map (k1,v1) ŕ list (k2,v2)
reduce (k2,list(v2)) ŕ list(v2) ·
The Map function reads a key-value
pair and produces a list of key-value pairs of different types. The reduce
function reads a pair composed of a key and a list of values and produces a
list of values of the same type. ·
The types (k1,v1,k2,kv2) used in
the expression of the two functions provide hints as to how these two
functions are connected and are executed to carry out the computation of a
MapReduce job: The output of map tasks is aggregated together by grouping the
values according to their corresponding keys and constitutes the input of
reduce tasks that, for each of the keys found, reduces the list of attached
values to a single value. Therefore, the input of a MapReduce computation is
expressed as a collection of key-value pairs <k1,v1> and the final output is represented by a
list of values: list(v2). ·
Figure depicts a reference
workflow characterizing MapReduce computations. As shown, the user submits a collection
of files that are expressed in the form of a list of <k1,v1> pairs and
specifies the map and reduce functions. ·
These files are entered into the
distributed file system that supports MapReduce and, if necessary,
partitioned in order to be the input of map tasks. Map tasks generate
intermediate files that store collections of,k2, list(v2).pairs, and these
files are saved into the distributed file system. The MapReduce runtime might
eventually aggregate the values corresponding to the same keys. ·
These files constitute the input
of reduce tasks, which finally produce output files in the form of list (v2).
The operation performed by reduce tasks is generally expressed as an
aggregation of all the values that are mapped by a specific key. The number
of map and reduce tasks to create, the way files are partitioned with respect
to these tasks, and the number of map tasks connected to a single reduce task
are the responsibilities of the MapReduce runtime. In addition, the way files
are stored and moved is the responsibility of the distributed file system
that supports MapReduce. Figure: MapReduce
computation workflow. The computation model expressed
by MapReduce is very straight forward and allows greater productivity for
people who have to code the algorithms for processing huge quantities of
data. This model has proven successful
in the case of Google, where the majority of the information that needs to be
processed is stored in textual form and is represented by Web pages or log
files. Some of the examples that show the flexibility of MapReduce are the
following: ·
Distributed grep. The grep operation, which
performs the recognition of patterns within text streams, is performed across
a wide set of files. MapReduce is leveraged to provide a parallel and faster
execution of this operation. In this case, the input file is a plain text
file, and the map function emits a line into the output each time it
recognizes the given pattern. The reduce task aggregates all the lines
emitted by the map tasks into a single file. ·
Count of URL-access frequency MapReduce is used to distribute
the execution of Web server log parsing. In this case, the Map function takes as input the
log of a Web server and emits into the output file a key-value pair <URL,1>
for each page access recorded in the log. The reduce function aggregates all
these lines by the corresponding URL, thus summing the single accesses, and
outputs a <URL, total-count> pair. ·
Reverse Web-link graph. The Reverse Web-link graph keeps
track of all the possible Web pages that might lead to a given link. In this
case input files are simple HTML pages that are scanned by map tasks emitting
<target, source> pairs for each of the links found in the Web page
source. The reduce task will collate all the pairs that have the same target
into a <target, list (source)>. pair. The final result is given one or
more files containing these mappings. ·
Term vector per host. A term vector recaps the most
important words occurring in a set of documents in the form of list
(<word, frequency>) , where the number of occurrences of a word is
taken as a measure of its importance. MapReduce is used to provide a mapping
between the origin of a set of document, obtained as the host component of
the URL of a document, and the corresponding term vector. In this case, the
map task creates a pair <host, term-vector> For each text document
retrieved, and the reduce task aggregates the term vectors corresponding to
documents retrieved from the same host. ·
Inverted index. The inverted index contains
information about the presence of words in documents. This information is
useful to allow fast full-text searches compared to direct document scans. In
this case, the map task takes as input a document, and for each document it emits
a collection of <word, document-id>. The reduce function aggregates the
occurrences of the same word, producing a pair <word, list (document-id)>. ·
Distributed sort. In this case, MapReduce is used
to parallelize the execution of a sort operation over a large number of
records. This application mostly relies on the properties of the MapReduce
runtime, which sorts and creates partitions of the intermediate files, rather
than in the operations, performed in the map and reduce tasks. Indeed, these
are very simple: The map task extracts the key from a record and emits a <key,
record> pair for each record; the reduce task will simply copy through all
the pairs. The actual sorting process is performed by the MapReduce runtime,
which will emit and partition the key-value pair by ordering them according
to the key. Hadoop enables resilient,
distributed processing of massive unstructured data sets across commodity
computer clusters, in which each node of the cluster includes its own
storage. MapReduce serves two essential functions: It parcels out work to
various nodes within the cluster or map, and it organizes and reduces the
results from each node into a cohesive answer to a query. MapReduce is composed of several
components, including:
To distribute input data and collate results, MapReduce
operates in parallel across massive cluster sizes. Because cluster size
doesn't affect a processing job's final results, jobs can be split across
almost any number of servers. Therefore, MapReduce and the overall Hadoop
framework simplify software development. MapReduce is available in several
languages, including C, C++, Java, Ruby, Perl and Python. Programmers can use
MapReduce libraries to create tasks without dealing with communication or
coordination between nodes. MapReduce is also fault-tolerant, with each node periodically
reporting its status to a master node. If a node doesn't
respond as expected, the master node re-assigns that piece of the job to
other available nodes in the cluster. This creates resiliency and makes it
practical for MapReduce to run on inexpensive commodity servers. Map Reduce in Action: For example, users can list and
count the number of times every word appears in a novel as a single server
application, but that is time consuming. By contrast, users can split the
task among 26 people, so each takes a page, writes a word on a separate sheet
of paper and takes a new page when they're finished. This is the map aspect
of MapReduce. And if a person leaves, another person takes his place. This
exemplifies MapReduce's fault-tolerant element. When all pages are processed,
users sort their single-word pages into 26 boxes, which represent the first
letter of each word. Each user takes a box and sorts each word in the stack
alphabetically. The number of pages with the same word is an example of the
reduce aspect of MapReduce. ------------------------------------REFERENCE {book: Buyya, page number:
269-273} Unit-03/Lecture-03 Cloud Governance, High Availability
and Disaster Recovery Cloud Governance: Cloud services governance is a general
term for applying specific policies or principles to the use of cloud
computing services. The goal of cloud services governance is to secure
applications and data when they are located remotely. In many ways, cloud services governance can be viewed as an
extension of SOA governance, although the unique properties of public cloud
architecture -- such as multi-tenancy -- present different slightly different
concerns. Ideally, cloud services governance complements or is integrated
into existing governance processes and is viewed as an ongoing process, not a
product. Companies (and individuals) often
choose to mitigate risks by buying insurance. This doesn’t keep bad
things from happening, but it reduces risk and helps you return to a steady
state. In the cloud, governance is a kind of insurance; it provides a
structure for managing multiple projects, avoiding redundant work, and
allowing your organization to exploit investments in services by managing
your service portfolio. But planning and implementing
governance is often incremental. In fact, most cloud usage starts with
no governance at all (except that SOA governance may already be in
place). Early cloud usage is often associated with projects at the
departmental level; it’s hard to justify or “sell” governance, which
represents an additional expense, before the deployment of additional
projects makes it clearer what benefits governance offers. (Of course,
if you can convince cloud users to start with governance, or provide
governance as an IT-funded service, so much the better.) Governance can help manage
consumer/provider relationships. It can manage contracts for SLAs and
charge-backs and manage the provisioning of services such as check approval
and credit card processing. But to consume these fine-grained services,
governance is required; otherwise, each time the service is used it will
probably be written again. If SOA governance is already in place
(and in many large enterprises it will be) governance for cloud services may
be implemented on top of it; otherwise, you will want to implement SOA
governance first. SOA governance defines essential governance and
management processes for consumer/provider environments including portfolio
management, project management, service management, and policy
management. Cloud governance requires extensions of SOA service
provisioning to support business models that are more contract-driven. As services (such as HIPPA compliance)
begin to become available from industry services providers, governance will
enable consuming and integrating these services. But buyers need
to be aware of what they’re buying; standards are still in their infancy and
providers may be using de facto standards rather than broader ones, opening
buyers to the possibility of vendor lock-in. The cloud will drive a significant
acceleration in change, making it more important than ever to be able to
consume and re-consume services, rather than writing and rewriting custom
code. Governance allows this to happen. Already it is apparent
that well-governed organizations (on every level from IT governance to SOA
governance to cloud governance) perform better because they are dynamically
better at adapting to change. ------------REFERENCE {Internet link: http://searchsoa.techtarget.com/definition/cloud-governance}
High Availability and Disaster
Recovery: ·
High availability is a system design approach and
associated service implementation that ensures a prearranged level of
operational performance will be met during a contractual measurement period. ·
High availability is
described through service level agreements and achieved through an
architecture that focuses on constant availability even in the face of
failures at any level of the system. While load balancing introduces
redundancy, it’s not a strategy that alone can provide high availability.
Servers sitting behind a load balancer may be running, but that doesn’t mean
that they are available. ·
Availability addresses the
ability to withstand failure from all angles including the network, storage,
and even the data center itself. Enterprise cloud services like those from
CenturyLink Cloud are built on a highly available architecture that uses
redundancy at all levels to ensure that no single component failure in a data
center impacts overall system availability. This includes “passive”
redundancy built into data centers to overcome power or internet provider
failures, as well as “active” redundancy that leverages sophisticated
monitoring to detect issues and initiate failover procedures. ·
All of our customers get platform-level high
availability when they use the CenturyLink Cloud cloud “out of the box.” ·
That means that you can rely on us for your
workloads knowing that our architecture is well-designed and highly
redundant. However – back to the introductory paragraph – it’s the customer’s
responsibility to design a highly-available application architecture. ·
Simply deploying an application to our cloud
doesn’t make it highly available. For example, if you deploy a single
Microsoft SQL Server instance in the CenturyLink Cloud cloud, you do not have
a highly available database. If that database server goes offline or network
access is interrupted, your application’s availability will be impacted. To
design a highly available Microsoft SQL Server solution, you have multiple
options. One choice is to create a cluster of database servers (where all
nodes are active at the same time, or, nodes sit passively by waiting to be
engaged) that access data from a shared disk. When a failure in the active
node is detected, the alternate node is automatically called into action. Disaster
Recovery: ·
Disaster recovery (DR) is the process, policies
and procedures that are related to preparing for recovery or continuation of
technology infrastructure which are vital to an organization after a natural
or human-induced disaster. Disaster recovery is a subset of business
continuity. While business continuity involves planning for keeping all
aspects of a business functioning in the midst of disruptive events, disaster
recovery focuses on the IT or technology systems that support business
functions. ·
DR is all about how you handle unexpected events.
Typically, your cloud provider has to declare a disaster before explicitly
initiating DR procedures. A brief network outage or storage failure in a data
center is usually not enough to trigger a disaster response. There are two
phrases that you often hear when defining a DR plan. A recovery point
objective (RPO) describes the maximum window of data that can be lost because
of a disaster. ·
For example, an RPO of 12 hours means that it is
possible that when you get back online after a disaster, you may have lost
the most recent 12 hours of data collected by your systems. A recovery time
objective (RTO) identifies how long the IT systems (and processes) can be
offline before being restored. For example, an RTO of 48 hours means that it
may take two days before the systems lost in the disaster are brought back
online and becoming usable again. High availability (HA) and disaster recovery (DR) are often thought of
as synonymous with each other. A highly available infrastructure component or
IT system is described as “fault tolerant” or having the ability to “fail
over”. An example of high availability at the component level is adding
redundant power supplies. At the datacenter level adding dual UPS (A/B power)
adds high(er) availability to power systems. To some, this implies the system
is resilient enough to survive a disaster. Implementing high availability on
its own, however, does not achieve disaster recovery. So what is the
difference between High Availability and Disaster Recovery? Here are a couple of definitions. IEEE defines high availability as,
“…the availability of resources in a computer system, in the wake of
component failures in the system.” While the Disaster Recovery Journal
defines disaster recovery as, “Resources and activities to re-establish
information technology services (including components such as infrastructure,
telecommunications, systems, applications and data) at an alternate site
following a disruption of IT services” There are several key differences between the two concepts.
-------------------------------REFERENCE
{Internetlink:http://searchdisasterrecovery.techtarget.com/feature/Disaster-recovery-in-the-cloud-explained}
|
|
Unit-03/Lecture-04 Virtualization: Introduction Virtualization
technology is one of the fundamental components of cloud computing,
especially in regard to infrastructure-based services. Virtualization allows
the creation of a secure, customizable, and isolated execution environment
for running applications, even if they are untrusted, without affecting other
users’ applications. The
basis of this technology is the ability of a computer program—or a
combination of software and hardware—to emulate an executing environment
separate from the one that hosts such programs. For example, we can run
Windows OS on top of a virtual machine, which itself is running on Linux OS.
Virtualization provides a great opportunity to build elastically scalable
systems that can provision additional capability with minimum costs.
Therefore, virtualization is widely used to deliver customizable computing
environments on demand. Virtualization is a large
umbrella of technologies and concepts that are meant to provide an abstract
environment—whether virtual hardware or an operating system—to run
applications. The term virtualization
is often synonymous with hardware virtualization, which plays a fundamental
role in efficiently delivering Infrastructure-as-a-Service (IaaS) solutions
for cloud computing. In fact, virtualization technologies have a long trail
in the history of computer science and have been available in many flavors by
providing virtual environments at the operating system level, the programming
language level, and the application level. Moreover, virtualization
technologies provide a virtual environment for not only executing
applications but also for storage, memory, and networking. Virtualization
technologies have gained renewed interested recently due to the confluence of
several phenomena: • Increased performance and computing capacity. Nowadays, the
average end-user desktop PC is powerful enough to meet almost all the needs
of everyday computing, with extra capacity that is rarely used. Almost all
these PCs have resources enough to host a virtual machine manager and execute
a virtual machine with by far acceptable performance. The same consideration
applies to the high-end side of the PC market, where supercomputers can
provide immense compute power that can accommodate the execution of hundreds
or thousands of virtual machines. • Underutilized hardware and software resources. Hardware and
software underutilization is occurring due to (1) increased performance and
computing capacity, and (2) the effect of limited or sporadic use of
resources. Computers today are so powerful that in most cases only a fraction
of their capacity is used by an application or the system. Moreover, if we
consider the IT infrastructure of an enterprise, many computers are only
partially utilized whereas they could be used without interruption on a
24/7/365 basis. For example, desktop PCs mostly devoted to office automation tasks
and used by administrative staff are only used during work hours, remaining
completely unused overnight. Using these resources for other purposes after
hours could improve the efficiency of the IT infrastructure. To transparently
provide such a service, it would be necessary to deploy a completely separate
environment, which can be achieved through virtualization. • Lack of space. The continuous need for additional capacity,
whether storage or compute power, makes data centers grow quickly. Companies such
as Google and Microsoft expand their infrastructures by building data centers
as large as football fields that are able to host thousands of nodes.
Although this is viable for IT giants, in most cases enterprises cannot
afford to build another data center to accommodate additional resource
capacity. This condition, along with hardware underutilization, has led to
the diffusion of a technique called server consolidation, 1 for which
virtualization technologies are fundamental. • Greening initiatives. Recently, companies are increasingly
looking for ways to reduce the amount of energy they consume and to reduce
their carbon footprint. Data centers are one of the major power consumers;
they contribute consistently to the impact that a company has on the environment.
Maintaining a data center operation not only involves keeping servers on, but
a great deal of energy is also consumed in keeping them cool. Infrastructures
for cooling have a significant impact on the carbon footprint of a data
center. Hence, reducing the number of servers through server consolidation
will definitely reduce the impact of cooling and power consumption of a data
center. Virtualization technologies can provide an efficient way of
consolidating servers. • Rise
of administrative costs. Power consumption and cooling costs have now
become higher than the cost of IT equipment. Moreover, the increased demand
for additional capacity, which translates into more servers in a data center,
is also responsible for a significant increment in administrative costs.
Computers—in particular, servers—do not operate all on their own, but they
require care and feeding from system administrators. Common system
administration tasks include hardware monitoring, defective hardware
replacement, server setup and updates, server resources monitoring, and
backups. These are labor-intensive operations, and the higher the number of
servers that have to be managed, the higher the administrative costs.
Virtualization can help reduce the number of required servers for a given
workload, thus reducing the cost of the administrative personnel. ------------------------------------REFERENCE {book: Buyya, page number: 71-73}
|
|
Unit-03/Lecture-05 Fundamental concepts of compute
storage, networking, desktop and application virtualization Other types of virtualization: Other than execution virtualization, other
types of virtualization provide an abstract environment to interact with.
These mainly cover storage, networking, and client/server interaction. ·
Storage virtualization: Storage virtualization is a system
administration practice that allows decoupling the physical organization of
the hardware from its logical representation. Using this technique, users do
not have to be worried about the specific location of their data, which can
be identified using a logical path. Storage virtualization allows us to
harness a wide range of storage facilities and represent them under a single
logical file system. There are different techniques for storage
virtualization, one of the most popular being network-based virtualization by
means of storage area networks (SANs). SANs use a network-accessible device
through a large bandwidth connection to provide storage facilities. ·
Network virtualization: Network
virtualization combines hardware appliances and specific software for the
creation and management of a virtual network. Network virtualization can
aggregate different physical networks into a single logical network (external
network virtualization) or provide network-like functionality to an operating
system partition (internal network virtualization). The result of external
network virtualization is generally a virtual LAN (VLAN). A VLAN is an
aggregation of hosts that communicate with each other as though they were
located under the same broadcasting domain. Internal network virtualization
is generally applied together with hardware and operating system-level
virtualization, in which the guests obtain a virtual network interface to
communicate with. There are several options for implementing internal network
virtualization: The guest can share the same network interface of the host
and use Network Address Translation (NAT) to access the network; the virtual
machine manager can emulate, and install on the host, an additional network
device, together with the driver; or the guest can have a private network
only with the guest. ·
Desktop virtualization: Desktop
virtualization abstracts the desktop environment available on a personal
computer in order to provide access to it using a client/server approach.
Desktop virtualization provides the same outcome of hardware virtualization
but serves a different purpose. Similarly to hardware virtualization, desktop
virtualization makes accessible a different system as though it were natively
installed on the host, but this system is remotely stored on a different host
and accessed through a network connection. Moreover, desktop virtualization
addresses the problem of making the same desktop environment accessible from
everywhere. Although the term desktop virtualization strictly refers to the
ability to remotely access a desktop environment, generally the desktop
environment is stored in a remote server or a data center that provides a
high-availability infrastructure and ensures the accessibility and
persistence of the data. In this scenario, an infrastructure supporting hardware virtualization
is fundamental to provide access to multiple desktop environments hosted on
the same server; a specific desktop environment is stored in a virtual
machine image that is loaded and started on demand when a client connects to
the desktop environment. This is a typical cloud computing scenario in which
the user leverages the virtual infrastructure for performing the daily tasks
on his computer. The advantages of desktop virtualization are high
availability, persistence, accessibility, and ease of management. The basic services for
remotely accessing a desktop environment are implemented in software components such as
Windows Remote Services, VNC, and X Server. Infrastructures for desktop
virtualization based on cloud computing solutions include Sun Virtual Desktop
Infrastructure (VDI), Parallels Virtual Desktop Infrastructure (VDI), Citrix
XenDesktop, and others. ·
Application server virtualization Application server virtualization abstracts a collection of application
servers that provide the same services as a single virtual application server
by using load-balancing strategies and providing a high-availability
infrastructure for the services hosted in the application server. This is a
particular form of virtualization and serves the same purpose of storage
virtualization: providing a better quality of service rather than emulating a
different environment. ----------------------------------------------------REFERENCE {book: Buyya, page number: 22-26}
Unit-03/Lecture-06 Virtualization Benefits Virtualization is a
broad concept that refers to the creation of a virtual version of something,
whether hardware, a software environment, storage, or a network. In a
virtualized environment there are three major components: guest, host, and
virtualization layer. The guest represents the system component that
interacts with the virtualization layer rather than with the host, as would
normally happen. The host represents the original environment where the guest
is supposed to be managed. The virtualization layer is responsible for
recreating the same or a different environment where the guest will operate. Such a general
abstraction finds different applications and then implementations of the
virtualization technology. The most intuitive and popular is represented by
hardware virtualization, which also constitutes the original realization of
the virtualization concept. FIGURE: Virtualization reference model In the case of hardware virtualization, the
guest is represented by a system image comprising an operating system and
installed applications. These are installed on top of virtual hardware that
is controlled and managed by the virtualization layer, also called the
virtual machine manager. The host is instead
represented by the physical hardware, and in some cases the operating system,
that defines the environment where the virtual machine manager is running. In
the case of virtual storage, the guest might be client applications or users
that interact with the virtual storage management software deployed on top of
the real storage system. The case of virtual networking is also similar: The guest—
applications and users—interacts with a virtual network, such as a virtual
private network (VPN), which is managed by specific software (VPN client)
using the physical network available on the node. VPNs are useful for
creating the illusion of being within a different physical network and thus
accessing the resources in it, which would otherwise not be available. The main common
characteristic of all these different implementations is the fact that the
virtual environment is created by means of a software program. The ability to
use software to emulate such a wide variety of environments creates a lot of
opportunities, previously less attractive because of excessive overhead
introduced by the virtualization layer. The technologies of
today allow profitable use of virtualization and make it possible to fully
exploit the advantages that come with it. Such advantages have always been
characteristics of virtualized solutions. ·
Increased security: The ability to control the
execution of a guest in a completely transparent manner opens new
possibilities for delivering a secure, controlled execution environment. The
virtual machine represents an emulated environment in which the guest is executed.
All the operations of the guest are generally performed against the virtual
machine, which then translates and applies them to the host. This level of
indirection allows the virtual machine manager to control and filter the
activity of the guest, thus preventing some harmful operations from being
performed. Resources exposed by the host can then be hidden or simply
protected from the guest. Moreover, sensitive information that is contained
in the host can be naturally hidden without the need to install complex
security policies. Increased security is a requirement when dealing with
untrusted code. For example, applets downloaded from the Internet run in a
sandboxed3 version of the Java Virtual Machine (JVM), which provides them
with limited access to the hosting operating system resources. Both the JVM
and the .NET runtime provide extensive security policies for customizing the
execution environment of applications. Hardware virtualization solutions such
as VMware Desktop, VirtualBox, and Parallels provide the ability to create a
virtual computer with customized virtual hardware on top of which a new
operating system can be installed. By default, the file system exposed by the
virtual computer is completely separated from the one of the host machine. This
becomes the perfect environment for running applications without affecting
other users in the environment. ·
Managed execution Virtualization of the execution
environment not only allows increased security, but a wider range of features
also can be implemented. In particular, sharing, aggregation, emulation, and
isolation are the most relevant features: • Sharing. Virtualization allows the creation of
separate computing environments within the same host. In this way it is
possible to fully exploit the capabilities of a powerful guest, which would
otherwise be underutilized. As we will see in later chapters, sharing is a
particularly important feature in virtualized data centers, where this basic
feature is used to reduce the number of active servers and limit power
consumption. • Aggregation. Not only is it possible to share physical resource
among several guests, but virtualization also allows aggregation, which is
the opposite process. A group of separate hosts can be tied together and represented
to guests as a single virtual host. This function is naturally implemented in
middleware for distributed computing, with a classical example represented by
cluster management software, which harnesses the physical resources of a
homogeneous group of machines and represents them as a single resource. ·
Emulation. Guest programs are executed within an
environment that is controlled by the virtualization layer, which ultimately
is a program. This allows for controlling and tuning the environment that is
exposed to guests. For instance, a completely different environment with
respect to the host can be emulated, thus allowing the execution of guest
programs requiring specific characteristics that are not present in the
physical host. This feature becomes very useful for testing purposes, where a
specific guest has to be validated against different platforms or
architectures and the wide range of options is not easily accessible during
development. Again, hardware virtualization solutions are able to provide
virtual hardware and emulate a particular kind of device such as Small
Computer System Interface (SCSI) devices for file I/O, without the hosting
machine having such hardware installed. Old and legacy software that does not
meet the requirements of current systems can be run on emulated hardware
without any need to change the code. This is possible either by emulating the
required hardware architecture or within a specific operating system sandbox,
such as the MS-DOS mode in Windows 95/98. Another example of emulation is an
arcade-game emulator that allows us to play arcade games on a normal personal
computer. FIGURE: Functions enabled by managed
execution. ·
Isolation. Virtualization allows providing
guests—whether they are operating systems, applications, or other
entities—with a completely separate environment, in which they are executed.
The guest program performs its activity by interacting with an abstraction
layer, which provides access to the underlying resources. Isolation brings
several benefits; for example, it allows multiple guests to run on the same
host without interfering with each other. Second, it provides a separation
between the host and the guest. The virtual machine can filter the activity
of the guest and prevent harmful operations against the host. Portability: The concept of
portability applies in different ways according to the specific type of
virtualization considered. In the case of a hardware virtualization solution,
the guest is packaged into a virtual image that, in most cases, can be safely
moved and executed on top of different virtual machines. Except for the file
size, this happens with the same simplicity with which we can display a
picture image in different computers. Virtual images are
generally proprietary formats that require a specific virtual machine manager
to be executed. In the case of programming-level virtualization, as
implemented by the JVM or the .NET runtime, the binary code representing
application components (jars or assemblies) can be run without any
recompilation on any implementation of the corresponding virtual machine.
This makes the application development cycle more flexible and application
deployment very straightforward: One version of the application, in most
cases, is able to run on different platforms with no changes. Finally,
portability allows having your own system always with you and ready to use as
long as the required virtual machine manager is available Finally, by means of
virtualization it is possible to achieve a more efficient use of resources.
Multiple systems can securely coexist and share the resources of the
underlying host, without interfering with each other. This is a prerequisite
for server consolidation, which allows adjusting the number of active
physical resources dynamically according to the current load of the system,
thus creating the opportunity to save in terms of energy consumption and to
be less impacting on the environment. ------------------------------------REFERENCE {book: buyya, page number: 73-77}
|
|
Unit-03/Lecture-07 Block and File level Storage
Virtualization, Hypervisor management software The two most popular storage system
technologies are file level storage and block level storage. File level storage is seen and deployed in Network
Attached Storage (NAS) systems. Block level storage is seen and
deployed in Storage Area Network (SAN) storage. File Level Storage -
This storage technology is most commonly used for storage systems, which is
found in hard drives, NAS systems and so on. In this File Level storage, the
storage disk is configured with a protocol such as NFS or SMB/CIFS and the
files are stored and accessed from it in bulk.
Block Level Storage -
In this block level storage, raw volumes of storage are created and each
block can be controlled as an individual hard drive. These Blocks are
controlled by server based operating systems and each block can be
individually formatted with the required file system.
-------------------------REFERENCE {internet link:
http://www.iscsi.com/resources/File-Level-Storage-vs-Block-Level-Storage.asp}
|
||||||||||||
|
Hypervisor
management software Hardware-level
virtualization: Hardware-level virtualization is a virtualization
technique that provides an abstract execution environment in terms of
computer hardware on top of which a guest operating system can be run. In
this model, the guest is represented by the operating system, the host by the
physical computer hardware, the virtual machine by its emulation, and the
virtual machine manager by the hypervisor. The hypervisor is generally a
program or a combination of software and hardware that allows the abstraction
of the underlying physical hardware. Hardware-level virtualization is also
called system virtualization, since it provides ISA to virtual machines,
which is the representation of the hardware interface of a system. This is to
differentiate it from process virtual machines, which expose ABI to virtual
machines. Hypervisors A fundamental element of hardware virtualization is the
hypervisor, or virtual machine manager (VMM). It recreates a hardware
environment in which guest operating systems are installed. There are two
major types of hypervisor: Type I and Type II. • Type I hypervisors
run directly on top of the hardware. Therefore, they take the place of the
operating systems and interact directly with the ISA interface exposed by the
underlying hardware, and they emulate this interface in order to allow the
management of guest operating systems. This type of hypervisor is also called
a native virtual machine since it runs natively on hardware. • Type II hypervisors require the support of an operating
system to provide virtualization services. This means that they are programs
managed by the operating system, which interact with it through the ABI and
emulate the ISA of virtual hardware for guest operating systems. This type of
hypervisor is also called a hosted virtual machine since it is hosted within
an operating system. Conceptually, a virtual machine manager is internally
organized as described in Figure. Three main modules, dispatcher, allocator,
and interpreter, coordinate their activity in order to emulate the underlying
hardware. FIGURE: Hosted
(left) and native (right) virtual machines. This figure provides a graphical
representation of the two types of hypervisors. The dispatcher constitutes the entry point of the monitor
and reroutes the instructions issued by the virtual machine instance to one
of the two other modules. The allocator is responsible for deciding the
system resources to be provided to the VM: whenever a virtual machine tries
to execute an instruction that results in changing the machine resources
associated with that VM, the allocator is invoked by the dispatcher. The
interpreter module consists of interpreter routines. These are executed
whenever a virtual machine executes a privileged instruction: a trap is
triggered and the corresponding routine is executed. The design and architecture of a virtual machine manager,
together with the underlying hardware design of the host machine, determine
the full realization of hardware virtualization, where a guest operating
system can be transparently executed on top of a VMM as though it were run on
the underlying hardware. The criteria that need to be met by a virtual
machine manager to efficiently support virtualization were established by
Goldberg and Popek in 1974 [23]. Three properties have to be satisfied: • Equivalence. A
guest running under the control of a virtual machine manager should exhibit
the same behavior as when it is executed directly on the physical host. • Resource control.
The virtual machine manager should be in complete control of virtualized
resources. ·
Efficiency. A statistically
dominant fraction of the machine instructions should be executed without
intervention from the virtual machine manager. FIGURE:A hypervisor
reference architecture. ------------------------------------REFERENCE {book: buyya, page number: 81-83}
|
|
Unit-01/Lecture-08 Virtual LAN and Virtual SAN Virtual LAN: A virtual local area network (VLAN) is a
logical group of workstations, servers and network devices that appear to be
on the same LAN despite their geographical distribution. A VLAN allows a
network of computers and users to communicate in a simulated environment as
if they exist in a single LAN and are sharing a single broadcast and multicast
domain. VLANs are implemented to achieve scalability, security and ease of
network management and can quickly adapt to change in network requirements
and relocation of workstations and server nodes. Computer networks can be segmented into local area
networks (LAN) and wide area networks (WAN). Network devices such as
switches, hubs, bridges, workstations and servers connected to each other in
the same network at a specific location are generally known as LANs. An LAN
is also considered a broadcast domain.
VLANs also have some disadvantages and limitations
as listed below:
---------------REFERENCE {internet link :
https://www.techopedia.com/definition/4804/virtual-local-area-network-vlan}
Virtual SAN: A virtual storage area network (VSAN) is a logical partition in a
storage area network (SAN). VSANs allow traffic to be isolated within
specific portions of a storage area network. The use of multiple VSANs can make a system
easier to configure and scale out. Subscribers can be added or relocated
without the need for changing the physical layout. If a problem occurs in one
VSAN, that problem can be handled with a minimum of disruption to the rest of
the network. Because the independence of VSANs minimizes the total system's
vulnerability, security is improved. VSANs also offer the possibility of data
redundancy, minimizing the risk of catastrophic data loss. In computer networking, a virtual
storage area network (VSAN) is a collection
of ports from a set of connected Fibre Channel switches, that
form a virtual fabric. Ports within a single switch can be partitioned
into multiple VSANs, despite sharing hardware resources. Conversely, multiple
switches can join a number of ports to form a single VSAN. A virtual storage area network (VSAN) is a logical
partitioning created within a physical storage area network. This
implementation model of a storage virtualization technique divides and
allocates some or an entire storage area network into one or more logical
SANs to be used by internal or external IT services and solutions. A virtual storage area network is primarily implemented in
cloud computing and virtualization environments. A VSAN allows end users and
organizations to provision a logical storage area network on top of the
physical SAN through storage virtualization. The virtualized SAN can be used
to build a virtual storage pool for multiple services; however, it is
generally provisioned to be integrated with virtual machines and virtual
servers. ---------------REFERENCE {internet link : http://searchstorage.techtarget.com/definition/virtual-storage-area-network}
|
|
||||||||||||||||||||||||