First Step: Identify the Occlusion
Video analytics platforms, computer vision and image processing can potentially overcome occlusions. Many organizations are performing research to identify objects in a video frame, including bicycles or other vehicles, Nyuyse says.
“When you’re running a video management system, that algorithm can automatically identify an object and classify the object based on what it has been trained to identify,” Nyuyse says.
For ML to help first responders overcome occlusions, they will need to use data sets based on supervised learning, Nyuyse says. “With that type of data set, you can come up with applications or models that will help train the system to start conducting some automated functions.”
Nyuyse describes how NVIDIA is working with Audi, which offers a night vision tool that detects occlusions such as people or animals within 20 yards of a vehicle. NVIDIA is providing 3D segmentation and modeling.
“I think technologies like that have a huge impact in trying to solve some of the challenges we have, especially with occlusion,” Nyuyse says.
Smart Cameras Help Interpret Occlusion-Blocked Events
First responders can use automated video processing to alert them to suspicious behavior. If a person walks behind shrubbery and takes longer than 5 seconds to emerge, they may have passed out, fallen down or been attacked, says Jim Davis, professor of artificial intelligence and machine learning and director of the computer vision lab at The Ohio State University.
Smart cameras can pick up these cues, which are changes from normal behavior, such as a delay in the typical time it takes to emerge from behind an occlusion. A location-aware camera enhanced with AI and ML technology can provide high-level activity alerts to first responders, Davis says.
Another strategy is to use a camera handoff feature to follow a person or group of people walking through a public facility, such as a sports arena, airport or train station, as they pass in and out of view from individual cameras. Operators can coordinate pan-tilt-zoom cameras to track people moving through an area.
“The fact that you can stitch observability of an individual, a crowd or a group of people across multiple cameras is one way to help combat the occlusion problem,” Davis says.
For the future, Davis suggests that mini drones could help first responders track emergencies at large gatherings, such as sporting events. Just as performers such as Lady Gaga have used drones for entertainment at stadiums, first responders could use drones in these outdoor venues to overcome occlusions.
In addition, 3D scene modeling and behavioral analysis will help first responders overcome occlusions in the same way that smart cars use 3D models, Davis predicts.
“The deeper it gets into 3D geometry, along with 3D behavioral analysis, the better it’s going to enable the AI to understand the environment, particularly in terms of these occlusions,” Davis says.