Managing the Growing Mountain of Government Data
The U.S. government’s use of big data continues to grow, stemming from the Obama administration’s 2012 commitment to invest $200 million in big data capabilities, as well as the expected growth of agency spending on enabling technologies such as cloud infrastructure.
Recognizing the opportunity that big data presents, agencies are working hard to incorporate it into their operations to serve residents to the best of their ability. Yet the complexity and diversity of this data — from machines and sensors, structured and unstructured — makes using a single tool to accomplish this task next to impossible.
In addition to complexity, legacy systems are preventing IT decision-makers from harnessing available innovative technology that could be used to store and manage data more efficiently, especially as older workers are replaced with younger staff with little legacy technology experience. With the big data revolution inevitable, every industry must challenge its IT environment to make big data business as usual.
Innovative new data management solutions can solve agencies' data challenges by doing two things: categorizing data based on its necessary level of security and keeping this information online for years without having to worry about data migration. I dive into these solutions below:
Software-Defined Everything
Software-defined technology is on the rise in federal agencies. Agencies such as DISA are working to improve the Defense Department’s cyber defenses by transitioning to software-defined networks and virtualization to manage the greater demand big data is placing on infrastructure.
Through SDN and network function virtualization (NFV), an agency can automate a number of tasks that are currently being done manually — a must for big data management. Such automation allows agencies to optimize their data center operations and categorize data, meaning they can put information into tiers, accommodate changes in information traffic patterns and classify information based on the level of security and accessibility.
In October, Federal Chief Information Officer Tony Scott published the Cybersecurity Strategy and Implementation Plan (CSIP), which backed and emphasized this necessary move to an automated software-defined architecture. By integrating the capacity of virtualization, mobility and networking, agencies can use innovative technology to create a faster, more secure and interoperable government.
SDN enhances the government’s legacy systems instead of “ripping and replacing,” while also opening the door for more effective use of emerging technologies, like cloud and mobility. Deployed correctly, SDN can simplify and modernize the heterogeneous infrastructure and use automated big data management to reduce costs, and standardize data protection capabilities.
Tiered Storage Solutions
Many agencies have multiple siloed databases. These data silos reside on everything from tier one high-speed disk or flash storage to magnetic tape. Tapes can degrade over time or during data migration, causing issues when agencies see an increase in data that needs to be organized, moved or migrated. This process requires a more automated IT architecture.
An automated IT architecture, such as a tiered solution, can solve this challenge and improve performance by driving intelligent big data placement at a lower cost. By using a converged architecture, government agencies can apply IT solutions and organize data with technology more innovatively.
Tiered data storage benefits federal agencies by easing migration and increasing automation. This strategy allows agencies to keep the most recent and most used information on a high-speed medium while keeping older but still prescient data accessible on slower and less expensive media. The automation and effectiveness of these storage settings ease federal chief technology officers’ and chief information officers’ jobs by synthesizing databases through analytics, thus easing the challenge big data presents.
Increasing Capabilities Without Busting Your Budget
Flash storage has been around for more than 25 years, but agencies are still finding new ways to use this storage function by switching from hard disk drives to flash alternatives. Flash storage brings agencies an unparalleled speed, low-cost performance level that is eons ahead of its spinning disk counterpart. This speed-to-cost ratio offers a crucial advantage to agencies striving to keep pace with burgeoning amounts of data.
While these new technology options are providing government with multiple ways to innovate, the conversation about the benefits and risks of a transition to SDN, hybrid cloud or an all-flash array is ever-present and will continue to be one of federal IT’s top priorities this year.
