Multi-robot multi-modal sensor fusion and outdoor lidar localization
Two Overland AI robots are using lidar and camera sensors to move through unmapped terrain without using GPS. The first robot navigates its way around obstacles as it moves to each new objective point. The second robot follows, looking out for its own hazards, some that weren’t in the path of the first robot. The visualization shows multi-modal sensor fusion from both robots. The bottom view shows each robot as a white body-frame bounding box (if you look very closely, the pose of more than 20 sensors are illustrated), plus the robot's trajectory (leader is blue, follower is red). Lidar scans (two per robot) from both robots are shown. Leader is painted with the winter colormap, follower is autumn. The top portion of the visualization shows lidar scans (from both lidars from both robots) reprojected into the second robot's fisheye cameras. It's possible to see features painted by all four lidars in some of the camera frames, and of course the lidar scans perfectly align with the camera images. This multi-robot multi-modal sensor fusion only works with an accurate sensor calibration and high-quality odometric pose. We use the calibration to time-compensate the various sensors and to enable multi-sensor reprojection. Odom is used to motion-compensate the lidar scans. Pose is used to show trajectories and to blend all the sensor data together.