To ensure continuity of operations amid or following disasters (both natural and man-made), agencies are building redundancy into their IT infrastructure, using data backup and recovery software and secondary data centers, and deploying extra network equipment.
Officials whose workplaces have been through fire and flood provide advice on how to prepare for the worst.
1. Practice, Practice, Practice Continuity of Operations Plans
Creating a continuity of operations plan (COOP) — including detailed evacuation procedures or step-by-step instructions to relaunch applications at a backup data center — is important, but does no good unless agencies practice those plans.
“We try to think about potential scenarios and run exercises to make sure people know what they are supposed to do in an emergency,” says National Weather Service COO John Murphy. “So when a crisis happens, it’s more of a muscle memory thing, and they aren’t running around in a panic trying to find a plan off the shelf.”
The Federal Aviation Administration, for example, annually participates in major COOP exercises with other federal agencies, but it also practices on its own once a year by failing over from one data center to another, says FAA acting CIO Sean Torpey.
2. Put a Succession Plan in Place for IT Leadership
Agencies should identify IT administrators at other locations who can take over their IT operations management if the executive management team at headquarters is unavailable after a disaster, Torpey says.
For example, if FAA headquarters in Washington, D.C., is damaged, Torpey will pass the baton to an IT leader at the FAA’s aeronautical center in Oklahoma City. Once Torpey reaches either of two backup office locations, he can retake control of the organization.
“If I’m down, someone needs to lead the ship. We have people who will take over the responsibility in other areas of the country, so IT can still be running,” he says.
3. Rely on Agency Leadership to Determine IT Priorities After Disasters
After a March flood displaced 3,200 military personnel at Offutt Air Force Base in Nebraska, the 55th Communications Squadron was inundated with requests for computer and network access.
“In users’ eyes, they were the No. 1 priority, so we had to figure out a way to manage those requests,” recalls Maj. Mike Scott.
The squadron turned to base leadership; because leadership was already dealing with finding office space for displaced personnel, it was natural for them to also decide who got IT services first.
“That made it easier for us when we received a request,” Scott says. “We said, ‘We will put it on the task list, but you need to go to base leadership to get on their priority list.’”
For more on federal disaster recovery plans, check out, “How Disaster Recovery Plans Keep Agencies Running.”