Federal Agencies Look to Flash Storage to Maximize Efficiency

Federal agencies have been pushing to make their data centers more efficient to meet the requirements set out in last year’s Data Center Optimization Initiative. Flash storage is increasingly seen as a viable option for agencies that need to get access to mission-critical data and make rapid decisions.

While many federal agencies haven’t yet jumped on the bandwagon, scientists at the Department of Energy’s National Energy Research Scientific Computing Center, based in Berkeley, Calif., have seen the power of flash firsthand. NERSC currently uses flash storage from NetApp.

As the principal provider of high-performance computing services for the DOE’s Office of Science programs, NERSC supports 6,000 scientists working on 700 projects. The facility’s computer, storage and web systems support complex workflows that run thousands of simulations of protein molecule dynamics.

“We have quite a lot of scientists whose code is limited by the input/output,” says Debbie Bard, a Big Data architect in NERSC’s Data Analytics Services Group. “That means their code might waste a lot of time trying to write data to disk and trying to read data from disks.”

SIGN UP: Get more news from the FedTech newsletter in your inbox every two weeks!

The Benefits of Flash Storage

Many IT professionals once considered flash storage a luxury reserved for only the most demanding workloads, but flash has come a long way, and the perception of it is changing. 

In recent years, solid-state drives have quadrupled in capacity while dropping steadily in price. Along the way, they’ve become a necessity in the data center.

When SSDs first appeared in the 1990s, they were much faster and significantly more expensive than their magnetic-storage competitors. In the decades since, flash has typically been used to provide durable storage for mobile devices or in performance-intensive applications such as financial trading, where increasing the speed of transactions can have a dramatic effect on the bottom line.  

“One of the places SSD caught on in a big way was with Visa and its automated transaction process,” says Jim Handy, a semiconductor market analyst with Objective Analysis. “Adding SSD allowed them to process 50 to 100 percent more transactions in the same amount of time. It was easy to measure the revenue gained versus the cost of the storage.”

But as SSD technology steadily improved over time and manufacturing volumes ramped up, the cost differential between flash and hard-disk drives (HDDs) began to shrink. In 2016, that gap pretty much disappeared, says Dan Cobb, vice president of media strategy for Dell EMC.

Even though flash storage devices have always had a number of advantages over mechanical drives — such as faster performance and lower energy consumption — the raw acquisition costs were always higher, Cobb says.

Last year that began to change. 

As a result, flash has become the storage medium of choice to consolidate general-purpose applications such as e-commerce databases, risk management for credit card transactions and real-time security analytics, says Cobb.

“Now, in nearly every single metric, enterprise flash has an advantage over hard drives,” he says. “Customers are no longer asking, why flash? Instead, they’re asking, why not flash?”

How to Get the Biggest ROI for Flash Investments

With flash, getting things done faster is the name of the game, says Greg Schulz, founder and senior advisory analyst at StorageIO.

When agencies purchase flash, “they’re really buying productivity,” Schulz says. “When people ask me how much solid-state storage they should get, I tell them ‘As much as your budget will allow.’” 

Despite its drop in cost in recent years, flash won’t entirely replace mechanical discs in many agencies. Even major cloud service providers are likely to use mechanical discs and possibly tape drives for less active data, such as bulk storage, archives and backups, says Schulz. 

Data centers that can’t yet afford to go all in on SSD for more active data may opt for hybrid arrays that combine mechanical discs with a solid-state cache, offering a compromise between cost and performance, says Schulz. 

But over time, hybrid systems will gradually be supplanted by all-flash arrays, notes Vito Ferrante, CDW’s EMC team lead and a flash storage technology evangelist. As with virtualization before it, going with flash is rapidly becoming an easy decision for data center managers, he says. 

“People used to ask, ‘Should we virtualize this workload?’ Now they say ‘We’re virtualizing everything, and you need to make an argument not to,’” Ferrante says. “Later this year and next, I think arrays will be all flash, unless you can make a good argument to use a mechanical disc instead.” 

Among the workloads for which flash storage provides clear advantages are Internet of Things applications and cloud services. IoT systems generate massive amounts of data that must be stored and analyzed. The speed and efficiency of flash enables many advanced features and capabilities of IoT, such as capturing and analyzing data in real time.

Many agencies are migrating mission-critical applications to private clouds. When these workloads demand greater performance than HDDs can deliver, flash becomes an excellent option. 

Ferrante adds that flash is also making strong inroads with hyperconverged appliances — integrated boxes that combine hardware, software, networking and power and that can be easily plugged into a rack, simplifying data center management. 

“If I can walk a customer through a data center and say, ‘I’m going to remove this piece of hardware and that network switch — and by the way, we’re going to increase performance 300 percent’ — I look like a hero,” he says.

The real challenge for data center operators then becomes how to get the biggest bang from their SSD investments.

“How do you know how much solid state you need, or where to put it?” asks Schulz.

“You need discovery tools that can look at your environment and say, here is a bottleneck; here is where a little bit of flash can address a problem. And if you use a little more, you can get an even bigger bang.”

Still, if you’re an IT manager, these are good problems to have.

For more on data center solutions, visit CDWG.com/datacenter.