Multi-robot multi-modal sensor fusion and outdoor lidar localization

Two Overland AI robots are using lidar and camera sensors to move through unmapped terrain without using GPS. The first robot navigates its way around obstacles as it moves to each new objective point. The second robot follows, looking out for its own hazards, some that weren’t in the path of the first robot.

The visualization shows multi-modal sensor fusion from both robots. Each robot is wrapped in a white body-frame bounding box (if you look very closely, the pose of more than 20 sensors are illustrated), plus the robot's trajectory (leader is blue, follower is red). Lidar scans (two per robot) from both robots are shown. Leader is painted with the winter colormap, follower is autumn.

This multi-robot multi-modal sensor fusion only works with an accurate sensor calibration and high-quality odometric pose. We use the calibration to time-compensate the various sensors and to enable multi-sensor reprojection. Odom is used to motion-compensate the lidar scans. Pose is used to show trajectories and to blend all the sensor data together.