Read an Excerpt
IT Virtualization Best Practices
A Lean, Green Virtualized Data Center Approach
By Mickey Iqbal, Mithkal Smadi, Chris Molly, Jim Rymarczy
MC PressCopyright © 2010 Mickey Iqbal, Mithkal Smadi, Chris Molloy, and Jim Rymarczyk
All rights reserved.
How Virtualization Will Save IT
Over the past dozen years, IT organizations have been fighting a losing battle against server sprawl — the essentially irresistible proliferation of commodity servers that are cheap and easy to deploy, but complex and costly to manage in substantial numbers. The cost of managing these distributed servers has grown from a small fraction of the IT budget to the dominant IT cost component, far exceeding the cost of the servers themselves! This growing cost, coupled with tightened IT budgets because of the recent worldwide financial downturn, increases the urgency to find an alternative methodology for IT resource management.
While facing these problems of growing resource management costs and shrinking budgets, IT groups are being asked to do more than ever before. New pressures have come from four major directions:
Exploding volumes of data, information, and new applications. Digital data, both raw and processed, is being generated at a dramatically increasing rate. As our world becomes highly instrumented and interconnected, data is also coming from a growing number of distributed sources (including such diverse sources as supermarket checkout stations, Web-click tracking software, and highway traffic sensors). By the end of 2011, one estimate indicates that the worldwide amount of digital data will be doubling every 11 hours! Accompanying this sea of new data, highly valuable new applications continue to emerge, many of which were previously impractical. In a growing number of situations, the new applications are being used to control complex distributed systems, evolving our world toward the promise of a smarter planet. This broad expansion in the role of IT is driving a huge increase in the need for storage, network, and computing resources — all of which add to the IT resource management challenge.
Growing requirements for business resilience and security. As IT has become a critical underpinning of most businesses and government agencies, even partial IT outages can have traumatic consequences. While this dependence on IT continues to increase, contemporary IT infrastructures are fraught with critical vulnerabilities. The threats to IT are grave and increasing. Some threats come from within the IT organization itself, whereas new threats arise from distant groups, including terrorists and agents of political cyberwar. The magnitude of this problem is reflected in the landslide of security and business resilience regulations that must be followed.
New mandates to reduce energy consumption. Until recently, energy efficiency was a low priority on most IT requirements lists (generally a third-tier requirement). Now, it has risen to the top. In part, this is due to rises in both energy cost and energy demand. In some situations, the energy required for additional IT resources is simply unavailable from existing energy suppliers. Power and thermal issues have also begun to inhibit IT operations — the addition and configuration of hardware resources is no longer just a matter of fitting into the available space. It must also take into account complex power and cooling constraints. Additionally, IT providers face environmental compliance and governance requirements, including the growing societal pressure to be "green."
The accelerating pace of technological innovations. Technological innovations are occurring at an increasing rate, and many of these innovations have compelling IT value. Examples include the rapidly improving virtualization technologies, converged networks, solid-state storage, IT appliances, storage de-duplication, the increasing numbers of cores and threads per chip, low-cost high-bandwidth fiber optics, petaflop supercomputers, cloud computing services, real-time data streams, cloud computing architectures, and distributed sensors/actuators (as used in transportation grids, power grids, and other real-time control systems). The best products available at any point will soon become the old, deficient products that lack many valuable functions and features. As a result, most IT infrastructures will consist of a diversity of product types and generations, rather than becoming homogeneous.
This book opens the door to solutions for all these problems. It explains how to plan and implement advances in virtualization technologies and management software to enable a vital breakthrough in IT effectiveness, efficiency, resiliency, and security.
This chapter explains the various types of technologies that fit under the rubric of virtualization, the advances in virtualization that have historically changed IT centers, and emerging technologies that continue to work major transformations in administrators' jobs and resources. You will also learn how emerging forms of cloud computing will leverage virtualization and the Web to play a valuable and expanding role in the implementation of the IT infrastructures of the future.
The Current IT Crisis
The strains in data centers large and small spring from a number of pressures that we'll explore in this section.
IT Infrastructures Have Become Too Labor-Intensive
Figure 1.1 illustrates a key problem faced by most IT providers. At the top of the figure is a depiction of the business processes whose implementation is the main purpose of IT. Unfortunately, the cost of managing the underlying IT resources has grown to the point where it consumes the lion's share of the IT budget, leaving limited funds to spend on business processes and applications.
Studies have shown that IT operational overhead has grown to account for 70 percent of the enterprise IT budget, leaving precious few resources for new initiatives. Much of this problem stems from the fact that physical configurations, unless well virtualized, have many hard constraints that are impediments to routine operational tasks such as the deployment of new applications or the adjustment of workload resource levels. The amount of resource provided per server is usually fixed when the server is deployed and rarely matches the actual needs of the assigned workload, which may vary dynamically. Industry-wide studies of Windows® and Linux® systems have indicated that their average server utilization is well below 10 percent, which results in a significant waste of data center hardware, floor space, power, and cooling.
Figure 1.2 depicts the non-virtualized resource configuration for a medium-sized business. This figure illustrates the problems that need to be overcome:
Fixed resources per server
Low server utilization
Wasted energy and floor space
Hardware changes affecting software assets
Servers managed individually
Non-virtualized IT environments also typically require substantial labor for software maintenance. The ongoing installation of new hardware generations requires corresponding upgrades to operating system software, which has a ripple effect in driving the need to upgrade middleware and to recertify applications prematurely.
Furthermore, without virtualization, the IT staff must deal with each physical server individually. Workloads must be assigned to specific servers, and achieving balanced resource utilization is a difficult and never-ending task. Whenever a given server malfunctions or needs to be taken offline for service, the IT staff must deal with it manually via a complex but temporary redistribution of workloads.
Many IT Infrastructures Violate Engineering Principles
For large enterprises, IT management is especially difficult due to the numbers of applications deployed and the complexity of their ad hoc configuration methods. Figure 1.3 illustrates the application topology at one large enterprise — and this is just one of two pages! Note that the diagram illustrates the application topology level only; it does not even extend to the underlying physical and virtual resources. It is not hard to appreciate the difficulty of managing such a configuration, especially when changes are needed or when problems must be found and fixed. This configuration is not atypical.
In fact, the feedback from some IT executives indicates that this diagram understates their complexity problems and is more indicative of their desired IT simplification goal! As one IT executive remarked, "If only my application diagram would fit on two pages!"
Other types of large systems, such as aircraft and automobiles, are arguably even more complex than IT data centers, but such systems are generally much simpler to maintain and use. The historical difficulty with IT data centers is that most are unique, and each typically has an extensive history of piecewise evolution, rather than a "fresh sheet of paper" design that complies with well-established engineering design principles and best practices. Significant IT complexity results from the coupling of numerous independent IT components in ways that are overly rigid and ad hoc.
The coming advances in IT resource virtualization and its management will lead to improved IT architectures that exhibit greater modularity and reduced complexity. Moreover, the transition to these future virtualized IT architectures can be done incrementally, yielding positive returns on investment along the way to a full IT transformation.
Prevailing Trends Will Increase IT Complexity
Looking beyond these existing IT conditions, a consideration of IT trends leads us to believe that the situation will get progressively worse unless actions are taken to transform the approach to IT. In particular, four noteworthy trends will increase IT's complexity and cost:
The numbers of systems deployed will continue to increase, driven largely by new applications (such as Web-based, surveillance, and operational asset management apps) and improving hardware price/performance. In the near term, the worldwide financial downturn and the initial wave of server consolidations have combined to put a temporary halt to the historically increasing numbers of servers deployed. However, all indications are that the deployment of new applications and systems will resume a high rate of growth.
The diversity of IT products will increase as competing suppliers continue to introduce new applications, systems, and management software. Today's innovation soon becomes tomorrow's legacy. It almost goes without saying that the IT products you buy today will, in just a few years, lack important capabilities available on the newest products.
The coupling of IT components is extensive and increasing, driven by application tiering, advances in high-performance standard networks, and so on. A growing range of distributed devices are becoming intelligent and networked.
While it has compelling benefits, the virtualization of resources will affect nearly all existing IT processes and can lead to virtual server sprawl. Introducing virtualization can have significant "hidden costs" and requires considerable skills, planning, and discipline. You will learn later in this book how to alleviate these potential downsides of virtualization.
The Power of Virtualization
The term "virtualization" is widely used in the computer industry and is applied to many different things. Virtual memory, virtual machines, and virtual disks have become common IT technology elements, but in each case the process used and result produced are very different. Just what does it mean to be "virtual," and why is this beneficial?
Conceptually, virtualization is a process of substitution. It takes real resources that have well-defined interfaces and functions and creates substitutes for those resources that have the same interfaces and functions. Figure 1.4 shows the relationship between the real resources and the resources presented to the users. To a user, a virtual resource looks and behaves just like its real counterpart. However, a virtual resource can have different attributes — such as size, cost, or adjustability — that make this substitution worthwhile.
Virtualization is often applied to hardware resources, such as servers or storage devices, but any resource with well-defined interfaces and functions can be virtualized. It is increasingly common to find virtualization being applied to software resources, such as operating systems, middleware, and applications.
Virtualization is very powerful, providing four major capabilities that are interrelated and often coexist:
Sharing — Virtualization may be used to create multiple virtual resources from a single real resource. Users of the virtual resources may be unaware of the sharing. In addition to increasing the utilization of real resources, this provides the ability to control and adjust how much of the shared real resource each user consumes. Additional benefits are the ability to create and destroy virtual resources on the fly and to isolate users from one another more securely (although not as well as if each user had his or her own private, real resource). This use of virtualization for resource sharing has become so widespread that it is sometimes considered the only definition of virtualization.
Aggregation — Virtualization also allows us to go in the opposite direction, making multiple real resources look like one resource to users. A classic example of this is a virtual disk. The user sees a single logical disk and doesn't need to worry about the numbers and sizes of the underlying physical disks. This can greatly simplify life for both end users and storage resource managers. Such aggregation also allows a virtual resource to scale well beyond an underlying hardware resource and to survive unplanned and planned hardware outages (e.g., via RAID or clustering techniques).
Emulation — Virtualization may be used to create virtual resources whose interfaces and functions differ from those of the underlying real resources. Major benefits include compatibility with older real resources and the protection of a software investment. Architecture emulators have been widely used since the 1960s; for example, microcode-based emulators enabled IBM® System/360 computers to run the same binary programs that were written for earlier computer generations. Other examples include iSCSI, which provides a virtual SCSI bus on an IP network; virtual tape built upon disk drives; and the new virtual Fibre Channel networks over Ethernet.
Insulation — Virtualization introduces a layer of abstraction and indirection that decouples virtual resources from the underlying real resources. As a result, virtual resources and their users need not be affected by changes to real resources. This enables administrators to dynamically replace, upgrade, and service the real resources in a manner that is transparent to the virtual resources. More importantly, it can insulate software stacks, which are the most valuable and labor-intensive IT components, from the ongoing stream of hardware changes. A noteworthy special case of insulation is encapsulation, in which virtualization is used to insulate the users of a virtual resource from its internals.
By decoupling and insulating software assets from their underlying hardware, then, virtualization can overcome many problems. IT administrators benefit because they can deploy and manage applications more dynamically and easily. The IT personnel who manage and maintain the underlying hardware also benefit because hardware resources can be taken offline non-disruptively, making hardware/firmware changes easier and more timely — and avoiding the need to beg IT users for hardware "change windows." Figure 1.6 shows the distinction between common configurations in non-virtualized environments and the improvements possible in a well-planned virtualization project. As the figure illustrates, virtualization technologies and emerging management software will significantly improve the efficiency and effectiveness of IT data centers.
Progressive Advances in IT Virtualization
Originally introduced on mainframe computers, virtualization technologies have been employed since the 1960s as a means for sharing resources such as servers and storage. These technologies have evolved over the years, and in the past decade they have proliferated to systems and resources of all types, including high-volume distributed systems. The widespread availability and increasing maturity of these technologies, together with the surge in pressure to cut IT costs, has led to the widespread use of virtualization, especially in server consolidation scenarios.
Excerpted from IT Virtualization Best Practices by Mickey Iqbal, Mithkal Smadi, Chris Molly, Jim Rymarczy. Copyright © 2010 Mickey Iqbal, Mithkal Smadi, Chris Molloy, and Jim Rymarczyk. Excerpted by permission of MC Press.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.