Ever Vigilant

“We’ve always done monitoring through logs by systems administrators at various data centers, but when you have thousands of systems and logs to look through, there’s only so much they can review in a particular day,” says Col. Michael Jones, deputy director of cyber emerging technologies in the Army’s CIO Office.

Now, to provide greater protection against cyberattacks, the Army has joined other agencies on the front lines to make continuous monitoring of federal networks a reality. The service is piloting an application first developed by the State Department that regularly monitors the Army’s PCs and servers for security risks and vulnerabilities.

“With continuous monitoring, we know where the problems are and know what we need to do. It’s an enterprise approach to monitoring that’s faster, better and cheaper.”

Specifically, the application aggregates security information already captured by the Army’s security and management tools, such as whether antivirus definitions and software patches are applied to every device. It quickly analyzes the combined data and provides IT administrators a comprehensive view of the organization’s security landscape, including a ranking of the most severe security vulnerabilities.

That’s essentially continuous monitoring in a nutshell. In addition to monitoring PCs and servers, the government’s goal is to keep an eye on the entire IT infrastructure in near real time, and that includes networks, software applications and mobile devices. Continuous monitoring automates what used to be a manual process, Jones says.

“It allows us to focus on what the risks and vulnerabilities are today and work on mitigating those risks,” he says, “as opposed to using our time and energy producing reports in three-ring binders that sit on the shelf.”

Agencies Unite Against Threats

Continuous monitoring became a priority this spring when the Office of Management and Budget gave new guidance on the Federal Information Security Management Act that requires agencies to begin submitting real-time security data about their IT infrastructure.

Previously, agencies spent a lot of time and money producing annual or tri-annual FISMA reports that proved that they met federal security compliance regulations, such as security awareness training. Essentially, the new guidance asks agencies to focus on defending their networks, agency IT leaders say.

The number of cyberattacks on government systems is skyrocketing as hackers, criminals, terrorist organizations and adversarial nations try to access sensitive data and disrupt critical systems and services, says Ron Ross, project leader of the FISMA Implementation Project at the National Institute of Standards and Technology.

In addition, agencies have increased the adoption of new technologies in recent years, such as smartphones, e-government services, blogging and other Web 2.0 tools, and that has increased the number of targets for hackers as well as increased the systems and applications that the government has to safeguard, says John Streufert, the State Department’s deputy CIO for information assurance.

In fact, State has seen the number of security incident reports triple in the past three years. “The security challenge has never been greater,” Streufert says.

Four out of five cyberattacks stem from previously known vulnerabilities or wrongly configured device settings, so continuous monitoring will help eliminate most threats, he adds. “If we concentrate our efforts on the problems we know about, it gives us the best chance to lower the potential for attacks.”

Federal agencies — including NIST and the Homeland Security, State, Justice and Defense departments — are all collaborating to develop the requirements, processes and standards that are necessary for continuous monitoring to be implemented en masse.

But because continuous monitoring is an emerging concept, it is still being defined for the purposes of implementation. For example, while many agencies, such as Army and State, are focusing on near-real-time scanning of systems infrastructure, NIST is releasing guidance this fall that gives continuous monitoring a broader definition that includes two additional layers: the organization level, meaning how senior leaders establish an information security and risk governance structure within their organizations; and the mission level, where core missions, business processes and enterprise architectures are defined.

Photo: Nicholas McIntosh

“The more we can get agencies to get started on automating their security programs, the safer government will be,” State CISO John Streufert says.

Continuous monitoring also doesn’t mean scans in real time or every second, the Army’s Jones says. Some systems need to be monitored only once a day, once a week or even just once a month, depending on risk factors.

“We may be more interested in some aspects of the network. Some data points we want to see daily, or maybe three times a day or hourly as bandwidth allows. For other areas that are less of a threat, once a month should suffice,” he says.

State Leads the Way

In 2008, the State Department began testing a new version of a homegrown application called iPost, which monitors the department’s 100,000 PCs and servers worldwide every two to 15 days. In just a year, it reduced known security vulnerabilities by 89 percent.

The iPost application takes advantage of the vulnerability data already captured by State’s handful of security and management tools. For example, Microsoft’s System Center Configuration Manager provides data on whether devices have the required patches and antivirus definitions; Active Directory has data on whether users have regularly updated their passwords; and Tenable Network Security software scans devices for vulnerabilities, wrongly configured security settings, as well as missed vulnerability or security compliance scans.

The iPost application automatically aggregates the security data into a Microsoft SQL Server database, uses a grading system — called the Risk Score Advisor — that assigns values to vulnerabilities, prioritizes them and then displays the top vulnerabilities that IT administrators need to fix immediately.

“There may be 10,000 things that need to be fixed, but some problems can do more substantial damage,” Streufert says. “With the way we display the data on the dashboard, the worst problems are highlighted first.”

Beginning in 2006, State’s 11 security organizations collaborated for two years to develop a common strategy to scan systems frequently and measure progress. iPost, created in 2003, initially focused on the operational performance of Microsoft products. But it was updated in 2008 with the risk-scoring module. Implementation of the new version of iPost was completed across 24 time zones with IT staff members in each office having no direct contact and no extra budget.

Two years into the implementation, the department’s continuous monitoring process has proved itself time and time again. When a major virus or attack occurs, iPost alerts IT security personnel so they can quickly respond.

When the Operation Aurora attacks targeted Google and others, State in April was able to patch systems in 84 percent of its 260 embassies and 140 other organizations worldwide in just seven days. When Microsoft Security Bulletin MS10-042 was released, 93 percent of the offices installed the patch within 30 days.

In contrast, most agencies typically resolve 65 percent of critical vulnerabilities in three to four months, says Alan Paller, research director at the SANS Institute.

By giving the Aurora attack a high vulnerability rating, iPost alerted IT staffers in each office or embassy that they needed to patch the problem right away, Streufert says. “This gave us a rapid approach to patching. It was fast and well organized and allowed us to do it at a pace much quicker than others.”

In addition to patching, iPost evaluates every embassy and office on how well they are able to resolve security risks overall. Each office is assigned a letter grade, from A through F, and those results are shared with not only IT staff but with each department head.

Making the grades public motivates different groups in the department to do better, but it also lets them collaborate and share best practices, Streufert says. This April, about 250 State organizations received A+ grades, so Streufert changed the grading standards to make it three times more challenging. Why? To push each organization to improve its security controls even further, he says.

“It’s a commitment to continuously improve,” he says.

With hardware protected, Streufert is looking to the future, with plans to speed data collection to every 36 to 72 hours. The department’s IT staffs are also starting work to expand near-real-time monitoring to the rest of the IT infrastructure, including wireless, mobile devices, software applications, firewalls and routers. Fully automating the processes across the rest of the IT infrastructure could take several more years, he says.

Streufert also plans to build security controls into the early-design phase for new applications and infrastructure refreshes, rather than bolt them on afterward. “If we know how 80 percent of attacks can occur, we should be building the best engineering solutions rather than chasing problems after they occur.”

In the meantime, State is sharing its iPost app with other agencies, including the Army, so they can try to match State’s success.

The department has provided a proof of concept, and it’s important for larger agencies, such as the Army, which is 10 times the size of State, to prove that continuous monitoring can also be implemented on that scale, says Paller of the SANS Institute.

“When the first couple of agencies prove that what State has accomplished can be transferred to other agencies, you will see the dam break, and 70 percent of the agencies will come across and start implementing,” he predicts.

Army Pilots Continuous Monitoring

At the Army, Jones received approval from his superiors in February to pilot continuous monitoring.

The Office of the Assistant Secretary of Defense for Network Information and Integration (ASD/NII) is watching closely how the Army uses the risk-scoring module, a subset of the State iPost application, to monitor PCs and servers and determine the top vulnerabilities.

Over the summer, an IT team installed iPost in a Defense Information Systems Agency data center. The Army’s 9th Signal Command is providing the vulnerability data and has already started to send their data to DISA.

“We had to sort out how to transport the data, and move the data from an Army aggregate server into an enterprise data bus, so the data could be provided to DISA,” he says. “As you can imagine, we had to set the correct firewall settings and use the right protocols.”

The data will come from a variety of security and management tools, including Microsoft’s SCCM, McAfee and CA Unicenter. The Army is formatting the data feeds using NIST’s Security Content Automation Protocol standard.

After running the pilot for a month, Jones plans to write a report on the lessons learned. Beginning in March and through the end of 2011, Jones hopes to develop a plan on how to deploy continuous monitoring throughout the rest of the Army and how to implement a full-blown version of iPost, or an iPost-like application, with a full grading system.

Jones praises State’s grading system because it motivates people to improve their security posture. “You get competition and collaboration going on. An agency that gets an F may wonder how another agency got an A, and they can collaborate,” he says.

If the initial pilot and planning process goes well, Jones says it’s possible that the Army will begin a full implementation of continuous monitoring throughout the Army in fiscal 2012. It will be a challenge, but it’s doable, he says.

“The State Department has proved that we can do it,” he says. “Their design is smaller than the Army’s. We need to get a lot of data passed through firewalls, for example, but I’m convinced this will dramatically improve the security posture of our network and that we can sort through the hurdles.”