3D Point Cloud Generation with Millimeter-Wave Radar

hello everyone in this video i will introduce our work 3d point cloud generation with millimeter wave radar a wide range of sensing modalities are already available on vehicles such as cameras lidar and radar visual sensors like cameras and lidar are vulnerable to adverse weather conditions such as rain snow and fog in contrast radar can penetrate small water particles and can make autonomous driving more robust against bad weather however vehicle radar has two main drawbacks first the radar has low spatial resolution due to its small form factor while visual sensors reveal fine grained shapes of targets

rita only generates intensity maps with strong reflection peaks second vehicle radar typically emits millimeter wave signals which are specularly reflected by most objects and may not be captured by the radar these figures show the line camera images and radar intensity maps of different vehicles the reader can only see parts of the vehicles with strong reflections towards radar recent innovations in millimeter wave sensing non-coherent refuses environments along the radar's moving trajectory while to some extent elevating the specular reflecting problem by illuminating the target from diverse locations their image resolution is too limited by the physical sense

of the antenna ray and cannot be fundamentally improved through spatial sampling to solve this problem we propose melee point which enables synthetic actual radar imaging with low and vehicle radars millipoint coherently combines measurements of the radar to generate dense and high resolution 3d point clouds the first challenge is that star requires accurate tracking of the reader the figures below show two cases where the radar moves along the cross-range direction at a constant speed and the accelerating speed without knowledge of the reader's location the image of the object can be highly distorted to solve this challenge

we propose a self-cross-run tracking method we let the radar estimate its instantaneous beat we exploit our observation that different transmit receive antenna pairs of the moving radar experience similar channel responses with a time delay an intuitive example is shown in left figure the reader has equally spaced four antennas as the reader moves the first antenna pair at time t1 coincidence with the second antenna pair at time t 0 so their measurements become highly correlated with the time delay of t1 minus t0 meanwhile the radar moves by one antenna spacing so the speed of the radar

can be calculated in general for any two antenna pairs the cross range moment is the average of the spacing between the transmit antennas and that between the receiver antennas the elapsed time is the delay time of the measurements of the two antenna pairs with the highest correlation values the second challenge is that sar requires focusing on the center of the target the figures below show the scenario where a metal surface is 3 meters away and has an incident angle of 135 degrees with a cross-range direction if the server focuses on the broadside direction then the

target disappears as in the left figure only when the cell focuses on the direction of 45 degrees the target is imaged as in the right figure the root cause of this phenomenon is the mismatch of the physical aperture of the radar and the effective apertures of the targets the physical actual represents the physical trajectory of the radar the effective aperture of the target is a segment of the trajectory where the target's reflection is received as shown in the figure below due to spectral reflection the effective aperture of the metal surface is short-flat segment while that

of the wall is the blue segment and the same as the physical aperture of the radar in the paper we proved that the focusing center can be represented by the parameters l and zeta which can be identified in the spatial frequency spectrum on this basis we propose a multi-focusing algorithm to overcome the status reflection problem and image all objects in the environment the final challenges i saw only generates 2d identity maps the figures below show an indoor scenario where images of objects at different heights overlap with each other to solve this challenge we exploit multiple

antenna pairs along the vertical direction for each antenna pair we generate one star image and find pixels with high intensity in the paper we proved that the complex pixel values do encode the elevation information so we combine co-registered pixels of all star images to form a virtual pixel array and estimate the elevation of the pixels the right figures show an indoor scenario captured by a stereo camera and radar point cloud generated by millipoint we conduct experiments using a tti cascade radar with six transmit antennas and eight received antennas but said stereo camera is used to

capture the ground tools we compare millipoint with non-coherent imaging and the imaging with a static radar in different scenarios with different objects such as cars pedestrians bikes and trash bins millipoint generates the densest point clouds and points are accurately aligned with objects and reflect their shapes in conclusion in this paper we propose melee point which is the first step to enable high resolution high density 3d point cloud generation on low end vehicle radar based on sar we envision middle point as a new type of sensor fusion modality thank you and i'd like to take any

questions