What Is AI-Capable Infrastructure? What Do Agencies Need to Know?

What exactly does an AI-capable infrastructure include?

“In its simplest sense, it is the infrastructure that you need to create your AI and to deploy it,” says Terry Halvorsen, vice president of federal client development at IBM and former Defense Department CIO. “It’s made up of both hardware and software.”

Significant processing power is required.

“You need enough compute to handle the volume of the data that you’re going to process,” Halvorsen says. “Then, you need the right kind of storage environment that allows you to store this data that you’re working on so it can be operated on and then distributed in a way that gets out in mission time.”

Agencies need processors that can handle a vast number of computations quickly. With AI “you need a much higher degree of speed and performance in your infrastructure,” Halvorsen says. “It’s a combination of memory and the speed of your servers. You need enough memory so you don’t bring the system to a stop, and you also need servers capable of processing at mission speed.”

AI-capable infrastructure includes a data management capability as well.

“You need a system or a collection of systems that will let you access the right pieces of data,” Halvorsen says. “Then, you need a system that lets you deploy that data, often to an edge environment.”

How Does AI-Capable Infrastructure Work?

At a high level, AI-capable infrastructure allows agencies to use the technology on systems quickly and responsibly. The right mix of capabilities ensures that infrastructure has “the computing power and resilience to run AI and is secure, accessible and responsible,” the GSA spokesperson says.

If the Department of Agriculture were to use AI to help farmers optimize water use, it would need to process vast amounts of data, “and the compute is needed to be able to do those analytics,” Halvorsen says.

The IT team would likely build machine learning models to drive this effort, “and you want to store the model capabilities,” Halvorsen says. Robust storage also houses all of the data within the model so that it can be adjusted as factors change.

For AI-capable infrastructure to work effectively, there needs to be a constant focus on managing the data that drives model outputs.

“That infrastructure is only as good as the data that it’s supporting,” Halvorsen says.

Modernizing infrastructure can create unexpected challenges. For example, IT teams often take for granted the power needed to support their computing needs.

“The advanced computing and advanced storage that you need generally drive more demand for power,” Halvorsen says. “That is something that IT organizations maybe haven’t had to spend much time on, but with AI and other advanced technologies, you need a robust power infrastructure to go along with your more classic IT infrastructure.”

The Technology Modernization Fund Helps Agencies Update

The Technology Modernization Fund provides incremental funding, technical assistance and oversight to agencies looking to upgrade their IT, and it can help pay for a move to AI-capable infrastructures.

“The TMF serves as a catalyst to accelerate modernization projects, such as those looking to responsibly implement AI,” the GSA spokesperson says. It can help by “funding projects in advance of their planned budget and appropriations to meet delivery needs now.”

“TMF meets agencies where they are in their modernization journey by investing in projects that range from core data infrastructure to expanding agency AI pilots,” the spokesperson said.

Still, the TMF is not limitless, and agencies will need to make a strong case to get their slice of that pie.

“First, you have to define your budget and your end objective,” Halvorsen says. “Then, you want to choose the right hardware and software.”

UP NEXT: How agencies can improve artificial intelligence implementation.

Agencies should consider whether they will buy or lease from the commercial sector, go with a public or private network or some combination of the two, and use cloud or on-premises solutions. If an agency goes the cloud route, who manages it?

To make the case for TMF money, agencies should also be looking to document their compliance efforts in ramping up AI applications.

“AI is going to be a highly regulated area of innovation, so you need to establish that,” Halvorsen says.

Finally, IT teams will need to budget not just for the initial ramp-up but for the long-term care and feeding of their AI-capable infrastructure.

“The last thing you want to do is go out and build an AI-capable infrastructure without also making sure that you have funded the future maintenance on those systems,” Halvorsen says.