Cloud

Multicloud Environments Help Store Data from Space

NGA, NASA and NOAA use a variety of solutions to manage the boom in data coming from satellites.

Melissa Delaney is a freelance journalist who specializes in business technology. She is a frequent contributor to the CDW family of technology magazines.

In 1960, U.S. intelligence gathered initial photos of the Soviet Union — more than U-2 spy plane missions had generated in five years — from the Corona satellite. But it wasn’t easy.

The 30,000-pound satellite, flying at 17,500 mph, dropped an 84-pound film canister equipped with a parachute, and a military transport plane called a “flying boxcar” caught it midair. It took six days to ship it first to Rochester, N.Y., for processing, then to Washington, D.C., so President Dwight D. Eisenhower could view the images.

Agencies across the federal government still rely on satellite imagery, but they now turn to commercial satellites and small satellites to expand their reach. This shift means lower costs and more frequent launches, producing unprecedented volumes of data.

To manage these growing stores of data, an increasing number of agencies are moving to multicloud solutions. Rather than struggle to house data on-premises or get locked in with a single cloud vendor, they’re working with a mix of commercial cloud providers to meet their unique needs.

“You really are looking not just for the value but for the best functionality,” explains National Geospatial-Intelligence Agency (NGA) CTO Mark Munsell.

Intelligence Agencies Prep for a Multicloud Strategy

There are basic building blocks of the cloud, such as storage and compute, but providers differentiate themselves through next-level services such as artificial intelligence, content management, office automation and worldwide distribution. NGA, for instance, selected one provider for a particular application because of its computer vision application for character recognition, Munsell says.

NGA moved to the cloud in 2014 but is preparing to shift to a multicloud solution if the intelligence community’s Commercial Cloud Enterprise contract is finalized. Transitioning from one to multiple cloud providers presents opportunities and challenges, Munsell explains.

“The benefit is that you have cloud service providers competing for your data storage, and that’s good. Your data storage costs could go down based on competition,” he says.

“The challenge, of course, is that a lot of the benefit of moving to the cloud is that you’re centralizing and simplifying your data access. Having your data stored in multiple clouds could complicate things.”

NGA, for example, stores its large volumes of data centrally to enable activities such as machine learning and AI, and distributing that across multiple cloud vendors is a challenge, Munsell says.

But as demand for multicloud environments grows, so do solutions and strategies for managing them. “The good news is they’ll have more choices,” Munsell says. “The bad news is they’ll have more choices.”

NASA Shifts to Commercial Cloud Environments

By the end of fiscal year 2019, NASA’s Earth Observing System Data and Information System (EOSDIS) maintained 33.6 petabytes of data.

“We have a lot of data, and we’re about to get a lot more with new missions that will launch in the next couple of years,” says Kevin Murphy, program executive for Earth Science Data Systems at NASA. “We’ll have around 250PB of data by the end of 2025.”

Back to the article Autoplay Full Screen Grid View Exit Full Screen

Back to the article Autoplay Full Screen Grid View Exit Full Screen

To glean greater insights, NASA makes its data easily accessible to millions of users who want to use it for any reason. The agency determined that moving from on-premises to a commercial cloud environment was the best way to do that. “When we have this much growth in data volume, we have to be substantially more efficient to deal with that,” Murphy says.

ESDS last year began transitioning NASA’s Distributed Active Archive Centers — facilities throughout the U.S. that manage its mission data — to its Earthdata Cloud architecture, using a variety of cloud services.

“We’re currently investigating how to use other commercial cloud environments, but there’s a long process for accreditation and security approvals that goes with that,” Murphy says.

Since beginning the move, ESDS has experienced substantial performance improvements of its data search engine while the complexity of engineering tasks, such as networking and system redundancy, has decreased, Murphy says.

Cloud Gives Agencies Better Access to Innovation

Commercial clouds also provide agencies with unprecedented access to large investments in developing new capabilities. “As we move our data into them, we have much quicker access to new computational techniques, like machine learning, artificial intelligence and deep learning,” Murphy says.

“Previously, it was hard to bring those kinds of technologies into our organization because they weren’t configured properly or required long procurements,” he explains. “But the innovation that happens in the commercial world and how that can be applied to Big Data problems is another benefit to commercial cloud environments.”

The National Oceanic and Atmospheric Administration began transitioning to a commercial multicloud solution, aiming to satisfy as many NOAA information users as possible.

Cloud service providers offer business intelligence tools, machine learning, AI and advanced analytics. “They allow our scientists to leap ahead in certain areas,” says Jonathan O’Neil, director of NOAA’s Big Data Program.

You really are looking not just for the value but for the best functionality.”

Mark Munsell CTO, National Geospatial-Intelligence Agency

“Rather than having to develop these analytical tools for themselves, the Software as a Service offerings of the cloud providers offer NOAA many tools that we can apply to further our science mission,” he says.

With the added capabilities of multicloud solutions come new management challenges. NGA has always designed its cloud applications knowing that it may not always use the same cloud provider. “We were very cognizant of that, and we built our applications in a way that could be easily transported from one cloud vendor to the other,” Munsell says.

It achieved that by using an abstracted layer above the infrastructure, at the platform level. “We built platforms so that we could take the platform and move our applications lock, stock and barrel to other cloud providers if necessary,” he says.

Cloud Tools Help Bring Data to Life

Data dissemination on the cloud is new for NOAA, so there was concern about how the multicloud solution would work, how it would be funded and how NOAA would govern it internally.

“That concern has been replaced by optimism and excitement as NOAA scientists see the opportunity to have data distributed to more users,” O’Neil says.

Now, he says, more scientists are using the Next Generation Radar archive to explore bird population decline or Microsoft’s AI platform to protect ice seals and beluga whales.

Gaining efficiencies from the cloud, however, often requires rethinking traditional IT processes, O’Neil adds. For example, with cloud toolsets, the agency has the option to make data available in a visualization tool or a data query tool.

“Often data in their raw form are hard to draw insights from," O'Neil says. “By using the toolsets provided by our cloud providers, data come to life in maps, analyses, machine learning models and more.”

Updating Security Strategies for Satellite Data

As agencies move their data to commercial cloud services, they are customizing ways to meet security challenges.

NASA secures its satellite transmissions down to the ground stations, and all of its traffic is encrypted. Public data products are available to anyone.

The National Oceanic and Atmospheric Administration’s cloud providers store information in data buckets that use their respective security architectures. The buckets are public but read-only, with the exception of NOAA’s data broker, the Cooperative Institute for Satellite Earth System Studies.

The Defense Department turned to a space-based cloud Data Security as a Service infrastructure that stores, transports and secures data on a highly secure network of 10 low-Earth-orbit satellites.

GOES Satellite