Dec 31 2009

On Guard

With the hurricane season sweeping in shortly, a look at Katrina by the Coast Guard offers insights that agencies can apply to emergency response planning.

Photo: Forrest MacCormack
Tabletop exercises don’t begin to provide the real-world answers that attempting to execute a plan can provide an agency, Lt. Cmdr. Keith LaPlant says.

In a few weeks, the hurricane season returns. At the Coast Guard’s District Office in Miami, Lt. Cmdr. Keith LaPlant is dusting off emergency plans that he drafted for the center a few years ago and revamping them based on what he learned restoring communications and systems for Coast Guard units in the wake of Hurricane Katrina.

He recently returned to Miami after a year-long tour at Coast Guard headquarters in Washington. A disaster systems specialist, LaPlant deploys during crises with the service’s incident management assist teams.

The Coast Guard has a top-down approach to emergency management and continuity of operations planning: All units and facilities update plans yearly. But the harsh reality of the post-Katrina weeks made the disaster-hardened LaPlant realize that more can be done to ensure that the government’s first-response teams and its systems can withstand future crises.

Like No Other

“We’d never had an incident of this magnitude that took down the Coast Guard infrastructure to this extent,” he says. “Until you really have to do this, you just don’t know. … Everybody’s disaster plan is going to be better for it.”

Katrina was like nothing for which LaPlant had handled triage and systems work. Typically, his responsibility is to establish communications and systems for a Coast Guard incident command post. But the destruction Katrina wrought multiplied his tasks.

Agencies across government have focused on improving COOP planning since the 2001 terrorist attacks; the 2005 hurricane season created live events that put those preparations to the test. The Coast Guard’s plans worked well in some cases and failed in others. The result is that LaPlant now points to six things agencies should do beyond merely creating plans:

Set a strategy for how to keep track of all your IT assets.

“In the first two weeks following Katrina, the number of emergency needs boggled the mind,” he says. So many problems occurred simultaneously that LaPlant had to manage a constant shuffling and redeployment of systems and communications services. Knowing who was doing what with what, and where, was a major challenge, he says.

Institute a daily reality check with the support team.

Initially, given the lack of a network link, online communications ground to a halt.

Every day, LaPlant and others at Coast Guard data centers and tech support shops nationwide held a teleconference to discuss the day’s priorities. “I wish I could take the credit, but someone else actually came up with the idea,” he says.

Make sure there’s no single point of failure.

For the Coast Guard, the loss of its central node at the New Orleans Integrated Support Command offices was a catastrophic blow. The offices housed a computer center providing the link to the service’s backbone comm system, the Coast Guard Data Network, for units and organizations throughout the region.

The command offices survived the storm relatively unscathed, but the lack of power in the city and the water that submerged much of downtown crippled the center’s systems. That had a domino effect, pulling Coast Guard office after office in the Gulf Coast area offline as well.

Require multiple backups and approaches to tapping into agency data.

Everyone knows backups are necessary, but in a crisis, it’s driven home how crucial quick and ready access to data is for continuing regular work and dealing with the emergency at hand, LaPlant notes.

Do contingency planning early.

“Do your homework and plan for the worst well in advance,” he says. This includes figuring out all the sources of support and extra gear that the agency can muster on a moment’s notice. The Coast Guard had a strategy for re-establishing comm services and the authority to request support from the public phone companies, but that plan was useless in New Orleans.

“It was really an infrastructure issue brought on by the flooding and the lack of access,” he says. Basic comm service was nonexistent because flooding and winds obliterated the phone companies’ private branch exchanges and cellular towers.

Ultimately, the Coast Guard leased new Voice over Internet Protocol satellite service with F4W, a VoIP and first-response IT services vendor in Lake Mary, Fla.

Do more than talk about the plan.

“You’ve got to do more than get around a table and talk about how ‘we’re going to do this; we’re going to do that,’ ” LaPlant says. Tabletop exercises do not begin to provide the real-world answers that attempting to execute a plan can provide an agency — only then do failure points come to light, he says.

Regaining Control

LaPlant arrived in Louisiana the day that Katrina came ashore. Within hours, it became clear that in addition to creating a systems setup for the command post, he and the New Orleans Electronic Systems Support Unit needed to restore network service for Coast Guard organizations across the Gulf Coast, provide wide-ranging communications services and help support the IT needs of search-and-rescue crews in the field.

“Our central node being down took the whole network down,” LaPlant says. “We had almost no command and control capability.”

And that immediate C2 need for the command post to be able to support Coast Guard search-and-rescue was a paramount issue. Sector New Orleans had set up its command post, before the hurricane hit, at the convention center in Alexandria, La., some 200 miles inland from New Orleans. Meanwhile, the service was running search-and-rescue teams from the Zephyr Field AAA baseball stadium in Metairie, La.

Ultimately, LaPlant took an approach that wasn’t anywhere in the New Orleans emergency or COOP plans. Tapping funds specifically set aside for disaster response, he leased turnkey deployable satellite and VoIP services from F4W. The Coast Guard already had pulled in all its Iridium phones — roughly 50 to 60 of the direct-to-satellite-feed phones — but there were hundreds of “coasties” in the field. At the Alexandria command post alone, more than 400 of the service’s members had relocated from New Orleans.

F4W provided voice, data and network IT services from its TWEB product line and also supplied support staff.

The company makes contained first-responder wireless networking units that let users set up instantaneous self-forming VoIP networks supported by multiple comm platforms, says Matthew Riley, director of the company’s government channel. F4W also helped support the Coast Guard during the ensuing Rita and Wilma hurricanes.

Within 72 hours, the Coast Guard re-established basic telephone service and Internet connectivity to major command and control nodes regionwide, LaPlant says.

Like the Coast Guard, F4W also learned how to better meet first-response needs during its deployments — which involved sending 20 employees to the Gulf Coast during last year’s hurricane season for stints ranging from 14 to 50 days, Riley says. “We’re now more mobile, faster to deploy and offer middleware necessary to bridge the interoperability issues we encountered between multiple agencies that needed to share information and files but did not want [other agencies’ users] operating on their networks.”

For Katrina, the cost for each F4W unit included setup and operation and a technician to run it 24/7. LaPlant recalled some days when there were as many as seven units deployed across the region, supporting operations with as many as 200 Coast Guard employees and as few as 40.

“I think they earned every penny every day,” LaPlant says, noting the incredibly trying environmental conditions and that the service required F4W to continuously deploy and redeploy systems for nearly three months.

More to Do

All told, LaPlant spent almost 30 days in and around New Orleans. LaPlant says he’s itching to get to work applying the lessons he learned during Katrina to the Miami district office’s emergency plan. And there’s also plenty of work to do to bridge the Coast Guard’s services during emergencies with those of other first responders, he says.

“In all honesty, we’re still a long way from the police officer on the street being able to pick up his radio and talk directly to a Coast Guard vessel or aircraft running a search-and-rescue mission,” LaPlant says.

Close

Become an Insider

Unlock white papers, personalized recommendations and other premium content for an in-depth look at evolving IT