Computer Simulations Help Federal Agencies Pinpoint Weaknesses

A growing number of federal agencies use simulation technology to test scenarios, improve training, and more.

Visualize a balmy spring day in Washington,
D.C., with the Capitol grounds and city
streets thick with the hundreds of thousands
of tourists and locals enjoying the annual
Cherry Blossom festival. Thousands more
sprawl on blankets or play Frisbee on the
National Mall, stroll along the Reflecting Pool, or spoon ice
cream in Lafayette Square, a few hundred feet from the White
House. A few blocks away, Metropolitan Police monitor a
demonstration.

Now imagine what would happen if the air were suddenly
split by the thunder of an explosion ripping through a nearby
subway station.

Questions would rain down hard and
fast: Is this a terrorist attack? Are chemical
or biological weapons involved? How
should casualties be handled? Should the
fire department be called? Which law
enforcement agencies should be
contacted? Should federal buildings be
evacuated? Should resources be diverted
from the demonstration? Who should
notify the media, local hospitals and
neighboring jurisdictions?

It is precisely such a scenario that the
Federal Protective Service (FPS) had in
mind when it sponsored an Emergency
Preparedness Incident Command
Simulation (EPiCS) exercise in March.

The exercise was not unusual. FPS,
which is responsible for protecting many
federal buildings and employees, regularly
conducts simulations to improve training;
clarify communications among federal,
state and local agencies; validate planning;
and pinpoint weaknesses. The agency
has used tabletop exercises, computer
simulations and even full-scale live events
to simulate attacks during terrorist trials,
suicide bombings and chemical releases.

"Our scenarios test multiple agencies
based on actual threats, capabilities and
responses without disrupting normal
operations or compromising operational
security," says Joseph Trindal, director of
FPS's National Capital Region.

Testing People, Machines, Processes

Frequently used for training, testing
and policy analysis, simulations reflect
models of actual systems, which can
include elements ranging from factories
and computer networks to highway
systems and national economies.
Simulation technology may involve
people to test their reactions or decision-making prowess, or it may run statistical
permutations of a scenario to determine
best choices or likely outcomes.

Long used by the military, simulations
are increasingly being used by such federal
agencies as the Defense Threat Reduction
Agency (DTRA), Federal Aviation
Administration and Internal Revenue
Service to determine optimal reactions or
consequence mitigation.

According to the research firm Input
based in Reston, Va., federal agencies and
the military have about $20 billion of
simulation projects in the pipeline.

"Simulation technology can test
people, machinery and processes at an
early stage to reduce risk and drive better
outcomes," says Addison Snell, research
director for high-performance computing
at International Data Corp., a market
intelligence firm in Framingham, Mass.

Successful simulations depend on two
factors, says Ralph Keyser, program lead for
the National Infrastructure Simulation and
Analysis Center (NISAC), which searches
for national infrastructure weaknesses,
primarily for the Homeland Security
Department. First, the most important
requirement is to obtain accurate
infrastructure data, such as the location of
underground gas lines and the traffic
capacity of highways.

Second, it's essential to identify all
interdependencies—between everything
from financial services and transportation
to emergency services and agriculture—that could lead to what Keyser calls
"cascading consequences."

Here's an example: "Natural gas, which
powers some utilities, has an added
element so that leaks can be smelled,"
Keyser explains. "But the machine that
adds this odor runs on electricity from
those same utilities. Of course, there are
backup generators, but they run on
diesel supplied by trucks. If traffic lights
stop working because of no electricity,
these trucks will have difficulty
supplying the generators."

Most government simulations run on
PCs and rely on commercial software,
such as geographic information systems.
More complex simulations, such as
microeconomic models, may use custom
code or cluster computing. Agent-based
modeling may require supercomputers.

Evaluating Scenarios

Like the Federal Protective Service, the
Defense Threat Reduction Agency uses
simulation technology to model possible
consequences of chemical, biological and
nuclear attacks. One PC simulation tool,
called Hazard Prediction and Assessment
Capability (HPAC), incorporates weather,
geographic and other data to evaluate the
impact of toxic releases.

"One scenario we've looked at is a
chemical rail-car explosion," says Todd
Hann, deputy chief of DTRA's consequence
assessment branch. Effects are overlaid on
maps indicating airports, hospitals and
other infrastructure, and exercises unfold
on the same approximate timetable as an
actual event.

Real-time scenario consequences, such
as casualties, reflect ongoing decisions.
Exercises can last from three to five
days—from the time of an explosion
through environmental cleanup to
postmortem analysis.

Other simulations deal with more
commonplace scenarios. Over the
course of a decade, the Federal Aviation
Administration (FAA) used more than
20 simulations to determine whether
instrument landings would still be safe if
dual, triple or even quadruple parallel
runways were located as much as 1,300 feet
closer together than are today's runways.

"Closer runways could increase airport
capacity up to 40 percent, reduce fuel costs
and give airports—especially those in
dense urban areas—new options for
expansion," explains Richard Ozmore,
manager of simulation and analysis at FAA.

All the simulations used an air traffic
control lab with high-resolution displays
and the computational capabilities of an
approach control facility connected to
several cockpit flight simulators via
dedicated data lines. Often, simulations
were of worst-case scenarios, such as an
airplane veering toward another during
parallel approaches. Some simulations
were "human-in-the-loop," and others
were "fast-time" to model various levels of
air traffic and the potential risks and
responses resulting from more than
100,000 potential blunders.

The simulations demonstrated the
safety of parallel landings if steps, such as
turning off autopilots during parallel
landings, are taken. Parallel landings have
been implemented at Dallas-Fort Worth,
Minneapolis-St. Paul and other airports.
The Minneapolis-St. Paul airport even
implemented triple dual parallel landings,
with a resulting savings of an estimated
3,182 hours of flight time.

But effective simulation requires more
than executing a "what if" scenario on a
PC. Individuals who are experienced in
running simulations and analyzing their
results emphasize the importance of using
information from everyone involved,
including the decision-makers, systems
analysts and designers.

"You can make a computer do anything
you want," says DTRA's Hann. "So it's critical
to know the issues that are important to the
decision-makers. You also need to make the
decision-makers aware of the assumptions,
limitations and uncertainties associated with
the simulation."

The value of simulation exercises can
be measured in more than improved
decision-making or contingency planning
skills, says FPS's Trindal. For example, he
points out that after a bombing at the
World Trade Center in 1993, regular
evacuation drills were instituted. As a
result, when warnings sounded on
September 11, more than 25,000 people
safely evacuated the twin towers.

Regardless of the type of scenario
simulation technology is used for—training, communications, planning or
shoring up weaknesses—the strategy can
be summed up in two words: Be prepared.

RANKING THE HURDLES

Rating the significance of each obstacle to the adoption of
technology-based simulations

Expense

Very significant obstacle:
48%

Significant obstacle:

34%

Somewhat of an obstacle:
13%

Lack of infrastructure

Very significant obstacle:
28%

Significant obstacle:
36%

Somewhat of an obstacle:
22%

Lack of management
commitment/understanding


Very significant obstacle:
21%

Significant obstacle:
30%

Somewhat of an obstacle:
26%

Lack of quality simulation
tools and services

Very significant obstacle:
17%


Significant obstacle:
35%

Somewhat of an obstacle:
27%

Lack of instructional or
business value

Very significant obstacle:
7%
Significant obstacle:
20%

Somewhat of an obstacle:
28%

Difficulty in measuring
performance


Very significant obstacle:
6%
Significant obstacle:
19%

Somewhat of an obstacle:
28%

Lack of employee
motivation


Very significant obstacle:
6%
Significant obstacle:

13%

Somewhat of an obstacle:
31%

Respondents: 362 senior training professionals at large and midsize organizations in the U.S.

Sources: IDC and Chief Learning Officer magazine, May 2004

DIRECTOR'S TIP

Considering using simulation for hiring? "Be sure to identify all the
costs associated with the current methodology so you can later
determine the ROI from simulation," says Julia Cronin, director of the
IRS Talent and Technology Management Division.

TYPICAL WORK SCENARIOS AT THE IRS

Not all simulations require a galaxy of computing power. For
example, at the Internal Revenue Service (IRS), the telephone is the
only technology used to test job applicants for customer service,
collection and other positions.

The agency simulates typical work scenarios: Applicants study
materials for 30 minutes, and then answer a telephone call from an
assessor, who puts the applicant through the paces of a typical
on-the-job call for 30 to 45 minutes.

This simulation strategy is a huge improvement over the former
method, in which management panels used structured interviews,
says Julia Cronin, director of the IRS Talent and Technology
Management Division. "That method took a lot of resources and time
to interview the candidates and document why one was chosen over
another," she explains. "Simulation reduces the burden on managers
and provides a more realistic preview of the position, as well as an
assessment of actual applicant skills."

In the simulation, the assessor can evaluate through a phone call
how persuasive a revenue officer applicant might be in getting a
taxpayer to agree to a payment schedule for back taxes. Other
skills that can be evaluated with this approach include oral
communication, planning and organizing, and decision-making. The
IRS also uses this strategy at some sites to assess bilingual skills.

The simulation method is paying off, Cronin says. In a cost-benefit
study, the workplace simulations reduced turnover and led to
better-qualified hires.