Agencies Finally Have the Resources to Scale DevSecOps

Why CISA Is Adopting Automation

At the Cybersecurity and Infrastructure Security Agency, one of the top benefits of DevSecOps has been a significant reduction in revisions, says Eugene Heim, a chief engineer in the agency’s cybersecurity division.

“Before we moved to a DevSecOps approach, it used to take a considerable amount of time for teams to go through the process of validating the security for any given product before we could promote it to production,” Heim says.

“Without fail, the process of navigating through those security gates caused significant friction, delay and frequent revisions to address any gaps or vulnerabilities that had made their way through the development pipeline,” he adds. “Sometimes, the validation process would last about as long as the development process for new features. By delivering in smaller batches, with constant security visibility for the application as it’s being developed, we’re in a much better place to deliver quickly.”

EXPLORE: How DevSecOps can help your federal agency modernize.

Heim says that CISA is taking an organizationwide approach in shifting to DevSecOps, adopting tools that offer automation, orchestration and containerization capabilities.

Ongoing training has been important in cementing the cultural change required for DevSecOps, but Heim says that automation tools also help reinforce this cultural shift.

“The automated processes take the human out of the loop,” he says. “If you attempt to deploy code, it must go through the requisite checks before it can be released. That automated process becomes our gatekeeper, rather than having to make sure that a person remembers to follow the process.”

As a result of these integrated, automated processes, Heim says, “the feedback loop is much faster,” allowing the agency to improve security while also speeding up deployment.

“Those are very powerful benefits,” he says.

How the DOE Is Scaling for DevSecOps

Technology leaders at civilian agencies have been exploring DevSecOps for nearly a decade, says Department of Energy CIO Ann Dunkin. Yet, only in recent years have these agencies secured the resources needed to scale their efforts, she says.

“We’d been talking about DevSecOps for years, but the Department of Defense was the first to figure out how to scale it up,” Dunkin says. “That was a useful opportunity for us to evaluate the landscape and learn from how they created their DevSecOps pipelines.”

The department has invested in low-code development platforms to support its DevSecOps efforts, but more important, the department has promoted cultural change throughout its organization, Dunkin says.

“We want to make this the path of least resistance,” she says. “The overall goal is to integrate our processes in a way that promotes agility. We need to be able to deliver small pieces of capability more frequently, instead of being forced to go through a big process for every change.”

“We can’t have our customers waiting months or years for new capabilities,” Dunkin adds.

DevSecOps has allowed the department’s development teams to more quickly navigate the Authority to Operate security authorization process. “ATO now takes maybe a quarter of the time it would have taken in the past, and we’d like to get that even lower,” Dunkin says.

The approach helped DOE teams prepare for the Bipartisan Infrastructure Law — which gives the agency funding to stand up at least 60 new projects in the next five years — by creating a tracking system for employment applicants in a matter of weeks.

“DevSecOps allows you to be fast and agile,” Dunkin says. “You’re testing and changing a very small subset of your attack surface at once, so you’re also reducing risk, and you’re more productive overall because people aren’t rewriting the fundamental code.”

DevSecOps Is Supporting the VA’s Mission

Daniel McCune, deputy CIO for software product management at the Department of Veterans Affairs, says that DevSecOps is not only an IT strategy but also a way to help meet the VA’s mission.

“Our passion is improving the lives of our heroes,” McCune says. “That means giving them access to the services they’ve earned and giving them confidence that we’re protecting their data. Given our size and complexity, the only way to do that is through software automation.”

McCune says that three technologies have powered the department’s shift toward DevSecOps.

First, cloud resources procured through Azure and AWS have helped the VA to rapidly scale up applications when needed, including telehealth offerings during the COVID-19 pandemic.

Next, application performance monitoring through tools like Dynatrace have had a “transformational” impact on application uptime, McCune says, resulting in a 40 percent improvement in one year.

DISCOVER: Why the U.S. military is embracing telehealth for service members and veterans.

Finally, continuous integration/continuous delivery (CI/CD) capabilities such as those provided by GitHub promote standardization and consistency within the DevSecOps pipeline.

DevSecOps tools and practices helped the VA to respond quickly during the COVID-19 crisis, adapting to changes in how veterans were able to use their education benefits. The department now releases new code for its education benefits portal twice a month.

As recently as 2018, it took the VA nearly 17 months to release new software, but 80 percent of applications now see a new release at least once every 90 days, McCune says. Much like other agencies, the VA is seeing security improvements even as it accelerates development.

“We run automated testing on every iteration,” he says. “Now, when we go to production, we have confidence that our code is secure.”

“All of this helps us to improve life for our heroes,” McCune adds. “If we’re not doing that, we’re doing the wrong thing.”

How the GSA Is Adjusting its DevSecOps Mindset

At the General Services Administration, some teams are further along than others in their implementation of DevSecOps, says Acting Director of Security Engineering Brian Turnau.

“Some teams are advanced in terms of adopting a DevSecOps culture,” says Turnau. “They have a solid software delivery pipeline, and everything is automated as much as possible. Some teams are in the early stages of experimenting with DevSecOps practices, and other teams are in between.”

The agency formally launched its DevSecOps program in late 2019, Turnau says, with hopes of breaking down silos between teams, increasing deployment frequency, decreasing mean time to recovery, and improving security visibility and control.

Although Turnau describes DevSecOps as an “ongoing effort” at GSA, the agency has made strides toward these goals, helped by investments in visibility and collaboration tools and CI/CD solutions.

“We’ve definitely seen a change in the mindset of our teams,” Turnau says. “In the past, you had the development team doing all this development, testing to make sure everything worked, and then they would throw the application over the wall to operations for deployment. Security came pretty late in the game, and those other teams viewed us more as an outsider or a roadblock.”

“With DevSecOps, the development and operations teams realize that security is not their enemy,” Turnau adds. “We’re not even a separate part of the organization. We’re all on the same team, working together to deliver the mission. We all have the same ultimate goal.”