There's no such thing as downtime at the Federal Aviation Administration's Logistics Center. More than 45,000 air traffic control facilities around the world count on it for emergency equipment and support.
So when the center had to move its systems management hub from one building to another on the Mike Monroney Aeronautical Center campus in Oklahoma City, the stakes were high.
Business relocations are always complex, but coordinating the logistics behind a move involving information technology — especially when it comes to mission-critical equipment — can be daunting. It's not an impossible task, however. Seasoned veterans of IT moves by agencies point to chief success factors: taking time to do adequate planning before a single system comes offline and then allocating resources accordingly; making smart use of contractors; and setting contingency plans to deal with worst-case scenarios that might arise.
But even if all the prep work is done to perfection, the most experienced team leader can't anticipate every obstacle that might crop up.
For FAA Project Manager Steven Gregus, the unthinkable took the form of Hurricane Katrina, which battered the Gulf Coast less than two weeks before the scheduled move. Because the FAA Logistics Center was providing mission-critical support in Louisiana, some equipment had to be moved Friday night and come back online by Saturday morning. If that wasn't enough, the vendor coordinating the move, Prism Pointe Technologies, was in Jackson, Miss., smack-dab in the path of destruction.
"A lot of things were thrown in the mix," says CDW Government Account Manager Tom Alex, who helped manage the project with Gregus and Prism Pointe's Joe Williams.
To deal with the weather challenges, FAA rescheduled the move from Sept. 9 to Sept. 16, but Gregus couldn't delay it any further. "We pushed it out a week, but we had a hard target date," he says, explaining that construction crews were already lined up to renovate the old building. "We had to have it done before the end of the month. There was a domino effect here."
Despite the hurdles, the move — which included servers, storage equipment and a tape library that support 600 employees at the logistics center — went smoothly, and Gregus credits its success to the flexibility and close communications among project team members. The varied experience of the team members at FAA, Prism Pointe and CDWÂG also proved invaluable, he says.
"We formed a partnership, we had a common goal, and we worked together," Gregus says. "That was key." As a result, he adds, "It was a quick weekend — in and out."
In the Beginning
The Army is planning to consolidate its IT operations by moving local services to centralized processing centers. The goals include simplifying network management, boosting security, providing greater agility to respond quickly to global missions and reducing the total cost of IT ownership, says Robert Ringdahl, director of strategic concepts and integration for the Army.
Lowering real estate costs, consolidating equipment and boosting operational efficiencies can all be good reasons for moving IT equipment, but, warns the Government Accountability Office's David Powner, such moves can be expensive and the savings don't always add up.
Powner, GAO's director for IT management issues, advises agencies to analyze potential costs and returns and to treat IT relocation projects like any other project competing for limited agency resources. "I think the key is to approach this from an investment point of view," he says.
A big expense that must be factored in is staffing, he points out. Will the agency have to pay to move employees or hire and train new ones? Will it need to hire contractors? Agencies also need to gauge equipment cost and setup and, in some cases, data transition or conversion, which can suck up time and money.
There are other factors to consider, too. If a relocation makes sense, an agency should read the fine print of its moving insurance to ensure that damaged equipment will be replaced in kind, advises John Long, a facilities relocation consultant with Robbins-Gioia of Alexandria, Va. He has helped coordinate major IT relocations at the Veterans Benefits Administration and within the military.
Another cost is what will be spent on vendors hired to plan and coordinate the move. Even if agencies have IT relocation expertise in-house, they may need additional help from their vendors to keep warranties intact, says CDWÂG's Alex. Some manufacturers' warranties require that equipment be packed a certain way or moved by certified professionals or the agreements are voided.
Hiring contractors may also be the only way an agency can meet a challenging deadline. "If you've got to get it done and you don't have the resources to meet the deadline, you go outside," FAA's Gregus says.
One of the biggest costs of any IT relocation is the business interruption.
The FAA weekend move was an extraordinary exception. Most agencies need to factor in some degree of rolling downtime, running anywhere from two weeks to six months for a large IT center, warns Robbins-Gioia relocation consultant John La Jeunesse, who worked with Long on the move by the Veterans Benefits Administration.
"When you are moving computers, you are moving your whole database and your customer base," he says. "You need to pack up the equipment, transport it, unpack it, reassemble the cables and get everything back online. You can't just put it on a dolly and move it next door."
IT network and security systems consist of tightly integrated elements, and moving them around can be risky. Will the new facility accommodate the Internet Protocol addresses of all its new tenants? Will the equipment, especially aging legacy systems, communicate with state-of-the-art firewalls in the new facility?
And the plans must go way beyond the systems themselves. For its area processing centers, the Army will need uninterruptible power, communications, heating and cooling, and raised floor space, Ringdahl says.
Are electrical systems isolated so air conditioners don't drain power from mission-critical systems? Will the fire-suppression system sprinkle water on costly servers? Agencies moving IT operations must map out these issues and detail the plans and backup plans for every contingency, Le Jeunesse says.
The FAA Logistics Center's relocation is a case in point. While the actual move took place over a weekend, the work that went into planning it took several months, Gregus says. From July until September, the team outlined and coordinated network connectivity needs, power requirements and security issues.
"We looked at every scenario so we were able to anticipate problems ahead of time," Gregus says.
A successful IT relocation starts with a realistic schedule, one that anticipates obstacles, La Jeunesse says. Fan motors for heating systems need to be back-ordered, for instance, because a move inevitably leads to dust build-up that requires twice-daily sweepings. Be prepared for inspectors to turn up unexpected problems that stall the move — for a few hours or a few days. The schedule needs to build in contingencies and "be extremely detailed," he says.
Phase at a Time
La Jeunesse suggests dividing project schedules into phases. Phase one is the contracting phase. Next comes the planning and design phase in which all team members — in-house and contract staff — must participate. That's followed by the construction phase, which includes the prep work on permits and inspections.
In addition to planning the physical move, the team needs to anticipate any procedural changes that must take place because of the move. For instance, one of the most significant challenges for the Army project was changing the way the centers handle the payments for and delivery of IT services, Ringdahl says. Because most of the resources for services are divvied up locally, the Army needs to develop a new strategy for delivering IT services in a consolidated environment.
Despite what seems like a lot of hurdles, Ringdahl says that in the end his advice to other agencies about moving IT is simple and straightforward: "Employ good people. Set well-defined and realizable goals. Ensure you have the right skill set. And monitor progress."