NIST Tool Boosts Software Security

The National Institute of Standards and Technology, NIST, is building a repository of software bugs to help application developers find and eradicate weaknesses in their programming code.

Called the SAMATE Reference Dataset (SRD), the repository is a free online tool that assists software developers in fortifying their creations against hackers. Specifically, the fully searchable repository provides known bugs for the required reference sets in so-called “static analyzer” weakness-checking programs.

SAMATE, which stands for Software Assurance Metrics and Tool Evaluation, is a NIST project with the goal of minimizing errors that leave software open to attack. The SRD is one fruit of the project’s labor.

Now in Version 4.0, the latest SRD release contains more than 30 times the number of instances as the previous version. Each is comparable to a significant grammatical error, explains Dr. Paul E. Black, SAMATE project leader for NIST.

Take the classic example of “let’s eat Grandpa” versus “let’s eat, Grandpa,” where the comma “is as important as it is in a line of code,” Black says.

Industry experts such as Al Hilwa of IDC concur. “This type of tool is priceless in the right hands and can be essential for security engineers who analyze software vulnerabilities,” says Hilwa, IDC’s program director for application and development software.

As Black points out, without the SRD, it can take a developer months to build a bug repository. “One organization cut a two-year project in half by using the SRD,” he says. “They’d planned a whole year just for collecting bugs.”

In addition to providing a data set for static analyzers, the SRD serves an educational purpose. “It can be used to train programmers by showing them what bad code looks like,” Black says.

Despite the current size of the data set, Black says there’s always room for more. “Federal agencies put a lot of time and energy into creating test sets,” he says. “Rather than putting them on the shelf, we’d love to have them to benefit others for years to come.”

Benefitting and Contributing

The Intelligence Advanced Research Projects Activity (IARPA), in the Office of the Director of National Intelligence, is both a user of the SRD and a contributor to the data set. It uses the SRD to reduce costs and improve the effectiveness of the Securely Taking on New Executable Software of Uncertain Provenance (STONESOUP) program, which develops tools to find and automatically mitigate security vulnerabilities in software.

“A data set with sufficient volume and diversity to meet our research requirements is very time-consuming and costly to produce,” explains Konrad Vesey, STONESOUP program manager.

Enter NIST’s SRD. “Using the SRD test suites for internal testing and evaluation allows our researchers to gain insight into how their technology fares against a wide range of vulnerabilities,” Vesey says.

Further, IARPA’s work becomes part of a virtuous cycle via contributions to the SRD. “It’s a very convenient and unique vehicle to disseminate the STONESOUP test data for the public’s benefit,” Vesey says.