Security

Video Surveillance May Be Enhanced by Research into Overcoming Occlusions

DHS is looking into ways to employ computer vision to remove obstructions from first responders’ view.

Brian T. Horowitz is a seasoned tech journalist and copy editor with 25 years of experience.

First responders and law enforcement officials often rely on video surveillance to help them scan emergency scenes, and they often find that occlusions can obstruct the view.

Occlusions can include permanent fixtures, such as trees and buildings, as well as temporary objects, such as cars and dumpsters, says Honoré Nyuyse, physical and cybersecurity technology manager for the U.S. Department of Homeland Security’s Science and Technology Directorate (DHS S&T).

In a recent solicitation, DHS S&T challenged companies to create ways to mitigate occlusions and other threats toward soft targets and reduce attacks in crowded places.

New technologies — including applications using computer vision, machine learning (ML) and drones, developed in response to the directorate’s solicitations — should be able to integrate with existing video management systems, says Melissa Oh, managing director at DHS S&T’s Silicon Valley Innovation Program.

DHS S&T is providing up to $2 million in funding per startup to address the Securing Soft Targets challenge, Oh says.

Click on the banner below and learn how to become an Insider.

First Step: Identify the Occlusion

Video analytics platforms, computer vision and image processing can potentially overcome occlusions. Many organizations are performing research to identify objects in a video frame, including bicycles or other vehicles, Nyuyse says.

“When you’re running a video management system, that algorithm can automatically identify an object and classify the object based on what it has been trained to identify,” Nyuyse says.

For ML to help first responders overcome occlusions, they will need to use data sets based on supervised learning, Nyuyse says. “With that type of data set, you can come up with applications or models that will help train the system to start conducting some automated functions.”

Nyuyse describes how NVIDIA is working with Audi, which offers a night vision tool that detects occlusions such as people or animals within 20 yards of a vehicle. NVIDIA is providing 3D segmentation and modeling.

“I think technologies like that have a huge impact in trying to solve some of the challenges we have, especially with occlusion,” Nyuyse says.

Smart Cameras Help Interpret Occlusion-Blocked Events

First responders can use automated video processing to alert them to suspicious behavior. If a person walks behind shrubbery and takes longer than 5 seconds to emerge, they may have passed out, fallen down or been attacked, says Jim Davis, professor of artificial intelligence and machine learning and director of the computer vision lab at The Ohio State University.

Smart cameras can pick up these cues, which are changes from normal behavior, such as a delay in the typical time it takes to emerge from behind an occlusion. A location-aware camera enhanced with AI and ML technology can provide high-level activity alerts to first responders, Davis says.

Another strategy is to use a camera handoff feature to follow a person or group of people walking through a public facility, such as a sports arena, airport or train station, as they pass in and out of view from individual cameras. Operators can coordinate pan-tilt-zoom cameras to track people moving through an area.

DISCOVER: How the intelligence community is investing in IT.

“The fact that you can stitch observability of an individual, a crowd or a group of people across multiple cameras is one way to help combat the occlusion problem,” Davis says.

For the future, Davis suggests that mini drones could help first responders track emergencies at large gatherings, such as sporting events. Just as performers such as Lady Gaga have used drones for entertainment at stadiums, first responders could use drones in these outdoor venues to overcome occlusions.

In addition, 3D scene modeling and behavioral analysis will help first responders overcome occlusions in the same way that smart cars use 3D models, Davis predicts.

“The deeper it gets into 3D geometry, along with 3D behavioral analysis, the better it’s going to enable the AI to understand the environment, particularly in terms of these occlusions,” Davis says.

nuttiwut rodbangpong/Getty Images