5 Facts About the VA’s IBM Watson Pilot

IBM’s Watson technology could be a game changer for physicians at the Veterans Affairs Department and for the millions of patients they treat.

In December, FedTech reported on the VA’s $16 million deal with IBM to use Watson technology in a two-year pilot. The powerful cognitive system could help to speed medical decision-making for VA physicians and enhance treatment for veterans with post-traumatic stress disorder. We followed up with both the federal department and the company to better understand how this technology will empower physicians and what this means for patient care at the VA.

“Watson is all about being a tool that allows physicians [and] practitioners to have access to data immediately,” Andy Maner, managing partner for IBM U.S. Federal, said about VA’s pilot. “It’s not meant to replace the physician.”

Maner said people often draw connections between Watson and Google, but it is much more than that. “Google returns data you can go read. Watson returns the answer or set of answers,” he said.

For now, the department will use the technology in what a VA spokeswoman called a “safe-harbor environment,” running simulated patient encounters using real clinical data.

We asked five questions about the project, and here’s what we found out from the VA representative:

1. What sparked VA's interest in using Watson technology?

VA monitors emerging health technologies and continuously seeks to identify new technologies that might support our mission and strategic goals. Our interest in Watson technology dates back to 2010, when we became aware of the Watson work IBM was doing in the area of cognitive computing. The team felt the ability to find answers using natural language processing and present these along with the associated evidence was very compelling and might potentially offer great value within a clinical setting.

2. How will the use of this technology change or improve how VA physicians do their jobs?

A key part of our assessment will focus exactly on your question. We hope to discover the ways in which clinical-reasoning technology can best offer value and assist our physicians in caring for our veterans. Two areas of particular interest we will be looking closely at are the ability of the system to perform "clinical semantic searches," both searching the electronic medical record and also the clinical literature, and, secondly, the ability to generate what is known as a "problem list" from the patient’s medical record. These two capabilities are expected to be of tremendous value to a clinician.

Problem lists are commonly used by physicians to document and track problems or conditions that are relevant to a patient’s medical care and, importantly, any that may require evaluation or treatment. They are frequently found in “old school” paper charts and are an integral part of most every electronic medical record (EMR) systems, including VA’s VistA/CPRS [Veterans Health Information Systems and Technology Architecture Computerized Patient Record System] EMR system. The lists are typically maintained manually by the physician.

The Watson clinical reasoning system will be assessed, in part, on its ability to analyze a patient’s complete electronic medical record and independently construct a tentative up-to-date problem list. For each problem, the system will be expected to make available the evidence it found to support its determination. That evidence includes information provided by the patient, diagnostic testing and previous diagnoses.

3. What will be the scope of the pilot?

At this point, it is too early to make this technology available in our facilities. Among other things, it will be important to access the accuracy of system responses. The pilot will take place in a "safe-harbor" environment and no patients will be actively treated during the assessment. We will, however, be using real clinical data. During the testing, we will be conducting realistic but simulated patient encounters. We have assembled subject matter experts, including primary care physicians, Post Traumatic Stress Disorder specialists, clinical informatics [experts], engineers and others from around the country as part of the assessment team.

4. What metrics will be measured to determine success of the pilot?

There are a vast number of metrics and reports that will be collected and produced throughout the major pilot phases of installation, training and testing. System responses will be evaluated by the assessment team for a wide range of factors like efficiency, effectiveness, and accuracy. Other considerations such as the level of effort needed to train the system for particular clinical diseases or disorders will be of interest.

5. How might this technology be used in other areas at the VA?

There are a number of other areas where this technology might be of great value. The primary care environment is only one component of the health care enterprise. Reasoning technology could likely be extended to specialty care, emergency departments, in-patient and other clinical areas. Our technology scans have also shown there may be opportunities in the area of administrative and financial functions.