Tracking the Occluded Indoor Target with Scattered Millimeter Wave Signal

The popularity of mobile robots in factories, warehouses, and hospitals has raised safety concerns about human-machine collisions, particularly in nonline-of-sight (NLoS) scenarios such as corners. Developing a robot capable of locating and tracking humans behind the corners will greatly mitigate risk. However, most of them cannot work in complex environments or require a costly infrastructure. This article introduces a solution that uses the reflected and diffracted millimeter wave (mmWave) radio signals to detect and locate targets behind the corner. Central to this solution is a localization convolutional neural network (L-CNN), which takes the angle-delay heatmap of the mmWave sensor as input and infers the potential target position. Furthermore, a Kalman filter is applied after L-CNN to improve the accuracy and robustness of estimated locations. A red-green-blue-depth (RGB-D) camera is attached to the mmWave sensor as the annotation system to provide accurate position labels. The results of the experimental evaluation demonstrate that our data-driven approach can achieve remarkable positioning accuracy at the 10-cm level without extensive infrastructure. In particular, the approach effectively mitigates the adverse effects of diffraction and multibounce phenomena, making the system more resilient.