Software

How Computer Vision Can Help Agencies Spot Critical Needs

Training computers to "see" images may add capabilities in emergency situations in the near-future.

Adam Stone writes on technology trends from Annapolis, Md., with a focus on government IT, military and first-responder technologies.

An earthquake hits. Communications are down, and the airport is offline. Emergency responders need to know how many aircraft on the tarmac are undamaged and still able to fly. An aerial snapshot might provide clues, but it could take a human analyst hours to pore over the image.

What if a computer could read that image and instantly report back on the status of resources? Computer vision would enable machines to identify objects in still pictures or activity in video footage in seconds, saving hours of human labor.

“We are pushing the frontiers of analysis in overhead imagery,” says Sam Salinas, program manager for artificial intelligence and machine learning at the Defense Innovation Unit (DIU).

Air Force, NGA Embrace Computer Vision

A team from the Air Force Research Laboratory at Wright-Patterson Air Force Base won its division at the 2017 International Conference on Computer Vision in Venice, Italy, where organizers asked competitors to teach a computer to describe in words the action in a four- or five-second movie clip.

Such capability could be applied to the thousands of hours of drone footage and other imagery that occupy human analysts. By identifying potential high-value imagery, the technology could free them to focus on more valuable intel.

3,400GB

Amount of RAM used in the DIUx xView 2018 Detection Challenge

Source: Defense Innovation Unit

To that end, DIU teamed with the National Geospatial-Intelligence Agency to host the DIUx xView 2018 Detection Challenge. Teams trained computers to identify 60 types of objects, including damaged buildings, vehicle lots and utility trucks.

In a disaster, “you are looking at lines of communication, roads and rails, perhaps airports, to ascertain, is the runway even intact? Are bridges intact?” says Buzz Roberts, director of artificial intelligence, automation and augmentation in the NGA research directorate.

Computer vision could quickly compare recent imagery to archival footage or generate new imagery specific to an event. It might also prevent human error that comes from operator fatigue.

State Department Can Use Computer Vision to Enhance Mission

The State Department could leverage the capability in at least one new way: “We are a big supporter of terrain and other types of information for humanitarian assistance, disaster relief, food safety and crops,” Roberts says.

Computer vision could help officials assess the situation on the ground in the developing world, and augment sparse local imaging resources.

It could also enhance crowd control efforts. Someone on the ground might have a limited view of an evolving situation, while a computer looking at multiple video feeds could assemble a comprehensive understanding of events.

“On a crowded street, I can tell you how many people are walking here, their speed, their trajectory and their appearance, what they are wearing,” says Jenq-Neng Hwang, a professor in the University of Washington department of electrical engineering.

A few bugs remain. Another UW team fooled computer vision by inserting an image of a car into a video about animals once every two seconds. They convinced the computer that the video clip was about an Audi instead.

The technology also requires an extreme level of computing resources. As of August, the DIU challenge had used the equivalent of 450 computers running for over a half million hours.

“These are very hungry algorithms,” Salinas says.

Alexandra Draghici/Getty Images