GAO Criticizes DHS’ Efforts to Track Effectiveness of Cybersecurity Outreach
The Department of Homeland Security is not doing enough to measure the effectiveness of its outreach efforts to promote cybersecurity for critical infrastructure in the United States, according to a government watchdog.
The report, from the Government Accountability Office (GAO), found that although DHS has taken steps to promote the National Institute of Standards and Technology’s (NIST) framework for improving critical infrastructure cybersecurity, it is not using specific metrics to track how effectively those efforts are taking root.
Adopting the NIST Framework
The NIST framework, developed in 2014, is designed to “provide a flexible and risk-based approach for entities within the nation's 16 critical infrastructure sectors to protect their vital assets from cyber-based threats,” according to the GAO. Those sectors are chemical, commercial facilities, communications, critical manufacturing, dams, the defense industrial base, emergency services, energy, financial services, food and agriculture, government facilities, healthcare and public health, IT, nuclear reactors and materials, transportation systems, and water and wastewater systems.
DHS is one of several sector-specific agencies that help protect these critical infrastructure sectors. “Sector-specific agencies (SSA) are federal departments or agencies with responsibility for providing institutional knowledge and specialized expertise as well as leading, facilitating, or supporting the security and resilience programs and associated activities of their designated critical infrastructure sector in the all-hazards environment,” as the report explains.
The GAO does praise DHS for creating “the Critical Infrastructure Cyber Community Voluntary Program (C3) to encourage adoption of the framework” and for its activities to support the program. DHS has offered private-sector entities in the critical infrastructure areas guidance and tools on how to adopt the NIST framework, the watchdog agency also points out. As FierceGovernmentIT reported, DHS “provided critical infrastructure participants with downloadable toolkits to set up the framework, held webinars to educate stakeholders, and stood up a central website for partners to find resources.”
DHS Not Properly Measuring its Efforts
Despite those actions, “DHS has not developed metrics to measure the success of its activities and programs,” the GAO report says. “Accordingly, DHS does not know if its efforts are effectively encouraging adoption of the framework.”
Officials of the DHS C3 Voluntary Program “tracked the number of times resources were accessed on the program website, DHS tools were downloaded, and in-person meetings were conducted to promote the framework,” the report states. “For example, according to DHS officials, since the website launch, it had been viewed over 117,000 times and over 22,000 resources had been downloaded as of October 2015.
“However, none of these metrics indicate the effectiveness of the program’s efforts to promote adoption of the framework, and program officials are not otherwise measuring or tracking how effective those efforts or materials are in encouraging individuals and organizations to voluntarily adopt the framework,” the report continues. “For example, they are not tracking what percentage of an individual sector they have promoted to or how effective their efforts and guidance are at encouraging the use of the framework by entities within a critical infrastructure sector.”
DHS program officials say they have not yet set up those metrics or ways to monitor them because they have been “focused on getting as much information and resources out as possible,” according to the report.
The GAO criticizes this logic, observing that “without understanding whether its promotional efforts are effective, the C3 Voluntary Program may not be able to tailor its products and guidance to effectively encourage adoption of the framework to sector stakeholders. As a result, sectors may not fully benefit from the cybersecurity principles and practices embedded in the framework to mitigate their cybersecurity risk.”
Laying out its recommendations, the GAO says the DHS secretary should instruct program officials to establish measures to evaluate how successful their efforts have been in marketing the framework to organizations within critical infrastructure sectors. What’s more, the agency advises DHS as well as the General Services Administration to “set a time frame for determining the need for sector-specific guidance to implement the framework in the government facilities sector.”
