The Real Deal
The Government Printing Office retired 42 servers over the past two years, replacing them with four VMware ESX servers. It also swapped out 45 desktop systems for virtualized thin clients that it installed in its typesetting operation.
There were clear cost and environmental benefits: The consolidation saves GPO $56,000 in annual electricity costs, which translates into a reduction of 176 tons of carbon dioxide emissions. But the move to a virtualized environment also required the congressional agency’s IT shop to adjust its management focus, says Byron Blocker, director of GPO’s Systems Integration Division.
GPO’s realization that it would need a new monitoring approach for its VMs is an example of what’s transpiring in IT organizations across government and in the private sector as well, says Ronni Colville, a vice president and distinguished analyst at Gartner of Stamford, Conn.
It used to take hours, days, even months to set up physical servers, but because it now takes just minutes to provision VMs, IT shops have to set strict policies to prevent VM sprawl. IT managers also must be sure their configuration management tools can function with the new virtual servers and VMs.
“With 15 discrete servers, we could see the traffic and respond,” says Blocker. “Now, with roughly 15 virtual machines on one host, monitoring the traffic became challenging. If one virtual machine gets compromised, it could be attacking the main server and we might not see it.”
Good Advice
Blocker says GPO sought advice from VMware and also talked to a consultant about how to most effectively manage its virtual machines. VMware pointed out that its solution has built-in tools that let IT shops manage the hypervisor, which is the core software in a virtual environment that lets multiple virtual machines run on a single server. Blocker says if the host goes down for any reason, VMware can detect it and reproduce VMs on other hosts in the virtual environment.
The consultant suggested Tripwire to manage disk space, CPU usage and application files, but Blocker says GPO found that it could use an existing management product from Computer Associates to do many of the same functions. GPO is also testing software to handle patch management.
“We use the Computer Associates product to manage memory usage, disk space and whether our applications are up or down,” says Blocker.
“Our preference is to use the products we have. It saves the taxpayers money and simplifies the management environment,” he says, pointing out that it’s always best for network administrators to use as few tools as possible.
The Defense Information Systems Agency also primarily uses existing tools to manage about 750 virtual environments across 13 data centers. DISA uses Tivoli for basic management functions and the Virtual Center tool in VMware to manage the hypervisor.
DISA has a capacity-on-demand arrangement with Hewlett-Packard. “We add servers as we need them,” says Alfred Rivera, DISA’s director for computing services. “HP manages the ESX layer, and our staff manages the virtual environments.”
53%
IT operations staff who do not ask IT security people to get involved in virtualization evaluation and deployment
SOURCE: Gartner
Whatever course you take, Gartner’s Colville recommends that if your agency doesn’t already have configuration management control for its physical servers, start developing them now in tandem with provisioning virtual servers.
Ideally, development of configuration processes and controls should occur before an agency makes significant product implementations of physical or virtual servers, Colville says.
“When implementing virtual servers, develop the same governance and controls for these new assets that you have for your physical environment, making note of where modifications need to be made,” she adds.
Similar Tactics
Given the tough budget climate, other agencies are taking an approach similar to GPO’s, using existing tools where possible to manage their virtual environments while tracking the development of emerging products and tools.
The Pacific Northwest National Laboratory in Richland, Wash., runs 225 VMware VMs on 30 physical servers. PNNL, an Energy Department laboratory with 4,200 employees and an annual budget of $850 million, uses an open-source tool to determine uptime or downtime of its VMs. Microsoft’s System Center Operations Manager tracks alerts for disk space and keeps tabs on any SQL Server database or Microsoft Exchange e-mail issues. For patch management of ESX servers, PNNL uses the tools within VMware.
“We realized when we started that we have to patch ESX servers just like anything else,” says Daryl Anderson, the lab’s hosting portfolio manager. “We also have a process in place to control the provisioning of VMs to prevent server sprawl.”
At NASA, the IT team has ambitious plans to begin developing a virtual infrastructure, kicking off by virtualizing 80 percent of the servers at Ames Research Center in Mountain View, Calif.
“Virtualization will be a key element in NASA’s IT strategy over the next few years,” says Ames CIO Chris C. Kemp. “Virtualization offers NASA increased flexibility as we provision infrastructure to meet the sometimes unpredictable needs of our organizations.”
Kemp says that instead of spending all of the center’s server budget at the beginning of a project, Ames’ ability to rapidly provision VMs means it can use only the servers it needs during the life of a project. If the project is scrapped and the physical boxes are no longer necessary, the servers and the VMs in them can be repurposed for another project.
As dramatic as virtualization’s benefits can be, the virtual environment must be managed. NASA also uses mostly existing tools, much like GPO. Kemp says NASA runs Sun Identity Manager to manage user provisioning and Windows Group Policy to handle security password policies, as well as password expirations and user privileges. Lumension Security automates software patches and runs reports on the status of licenses and on the software installed on the servers. Tools within VMware manage the hypervisor.
“Too often,” Kemp says. “Securing virtual environments is overlooked, but I can tell you that security is a forethought with us.”
Smart Process
At the IRS, it’s not OK “to spin up new VMs just because
it’s easy to do,” the agency’s Brian Bahlert says.
Security and management are also priorities at the IRS, which has an automated process in place to provision each new VM. The IRS deployed VMware proto-types at 12 locations nationwide, prototypes it intends to phase out this year. Today, it has 550 virtual servers and 200 virtual desktops that reside on 40 physical servers.
In the next year, the agency plans to retire 1,600 physical servers. The IRS estimates it will wind up with around 200 physical VMware hosts.
“We treat the VMs the same way we would the physical servers,” says Brian Bahlert, who handles server standards and configuration at the Enterprise Computing Operations Division of IRS. “We don’t let people spin up new VMs just because it’s easy to do,” he adds.
The IRS also uses existing management tools to manage security for the virtual servers. HP OpenView does systems monitoring, and tools within VMware manage the hypervisor.
“We’re very happy with VMware,” Bahlert says. “It lets us automate the virtual environment and put it in a maintenance mode.”
