Technology Readiness Levels: When Is Technology Safe for Adoption in Federal IT?

What Is Technology Readiness?

A technology readiness assessment , as the GAO guide notes, “is a systematic, evidence-based process that evaluates the maturity of technologies (hardware, software, and processes) critical to the performance of a larger system or the fulfillment of the key objectives of an acquisition program, including cost and schedule.”

TRAs are used to test the technical maturity of a technology, and while they do not eliminate technology risk, when done well “they can illuminate concerns and serve as the basis for realistic discussions on how to address potential risks” as technologies move from research and development to deployment. TRAs also “help legislators, government officials, and the public hold government programs accountable for achieving technology performance goals,” the GAO says.

While the R in TRA stands for readiness, Persons says a TRA is essentially a risk assessment process for truly new or novel technologies that are being used in new ways for the first time.

“It’s important that you manage the risk of that particular insertion of that technology or the development of it for the success of these big capital programs,” Persons says.

TRAs can help agencies “parse the ampersand” between R&D, Persons says, noting that the difference between researching a technology and developing it to work in the field is quite different.

“Just because I am successful at the lab level doesn’t mean I can say my technology can be field tested and ruggedized and it will work in the same way,” he says.

An essential part of a TRA is recognizing the true critical technology elements an agency needs for a specific program. “What needs to be identified, how much development and testing and evaluation do I need to do before I go live and risk missions and lives?” Persons asks. “In most cases, these are very high-consequence systems where, if it’s not there, it would be game over for the U.S.”

Oversight bodies, such as those with department or agency acquisition officials or government auditors, can use the GAO guide to “evaluate whether the fundamental processes and best practices of effective TRAs have been followed and whether a TRA demonstrates the characteristics (credibility, objectivity, reliability, and usefulness) of a high-quality assessment,” the guide states.

Brian Bothwell, an assistant director at GAO who worked on the guide, says that when the GAO looks at an agency’s TRA process, it wants to see if it met all of those characteristics. “If it doesn’t rise to a certain level for all of them, we have a finding that is not high-quality,” he says.

VIDEO: What are the seldomly asked questions around emerging tech? 

What Are Technology Readiness Levels?

Technology Readiness Levels, or TRLs, are “the most common measure for systematically communicating the readiness of new technologies or new applications of existing technologies (sometimes referred to as heritage technologies) to be incorporated into a system or program,” the guide notes.

TRLs are a collection of characteristics that describe increasing levels of technical maturity based on demonstrated, or tested, capabilities. “The performance of a technology is compared to levels of maturity (numbered 1-9) based on demonstrations of increasing fidelity and complexity,” the GAO notes.

In general, TRLs are measured on a scale of 1 to 9, in which TRL 1 generally indicates that there are paper or research studies of the basic concept of a technology, moving to laboratory demonstrations around TRL 4, and ending at TRL 9, when an actual system has been proved through successful mission operations.

TRLs can be generalized across most if not all technologies, Persons says. A TRA is a risk assessment process, and a TRL is a numerical outcome. However, “just having a number before you doesn’t tell you how much risk there is” in the technology system, he says.

Then “the art comes in,” and the staff who are making the decisions about whether to move forward on a technology, at the project management office and in the office of the secretary, need to determine what the real risks are.

MORE FROM FEDTECH: Discover how to address legacy IT and achieve your modernization goals. 

What Is a Mature Technology?

While a TRA uses TRLs as a key measure for evaluating critical technologies, an assessment of a technology’s maturity “is more than just a single number at single points in time,” the guide says.

TRAs are a compilation of lower-level assessments that may last across several years, depending on the program schedule for the technology and complexity of the development, according to the GAO.

“Assessments can help gauge the progress of a technology’s development, inform project plans, and identify potential concerns for decision makers throughout acquisitions,” the guide states. “Conducting TRAs periodically and during the earlier phases of development can identify potential concerns before risks are carried into the later and more expensive stages of system development.”

A TRA report is generated at the end of the assessment process and is used to inform agency decision-makers about “whether a prescribed TRL goal has been met, or identify potential areas of concern or risk, among other purposes.”

The report identifies the actions to take for technologies assessed as immature, “such as considering an alternate or backup technology, developing a technology maturation plan, updating the program risk management plan, or updating the cost and schedule risk assessments.”

Program management offices need leeway to contextualize the TRL score, Persons says; for example, if NASA is testing a new rocket and it works well in a controlled lab setting, but the technology is too immature and will likely break in a real-world setting. “The real world is very harsh,” Persons says.

The GAO guide includes a section on technology maturation plans (TMPs), which are management planning tools to mature technologies that have been assessed as immature. “The TMP lays out the steps, actions, and resources needed to mature and uses the TRA report findings and other key information to establish a road map with the necessary engineering activities to mature” the technologies.

Beyond the DOD and NASA, Persons says, there are numerous agencies that can use TRAs to enhance their missions. The Transportation Department, for example, will want to know whether autonomous-vehicle technology is mature enough before it is allowed on the road in certain areas of the country. Or the Federal Emergency Management Agency may be testing out the use of drones in a novel way, Persons says.