Main Street Autonomy

Calibration results

Calibration Anywhere generates a calibration result that consists of either:

  • A zip file containing sensor extrinsics, intrinsics, and time offsets
  • A text file explaining the failure

Successful calibration output

A successful calibration output consists of a zip file containing:

  • Sensor extrinsics
  • Camera intrinsics
  • Wheel intrinsics (if encoders or wheel speeds were included)
  • Time offsets (if sensors have a similar shared clock)
  • Ground detection (if the robot moves in a plane on flat ground)
  • Metadata

Sensor extrinsics

Filename Description
extrinsics.urdf NVIDIA Isaac Perceptor compatible URDF
extrinsics.yaml YAML describing 6DoF extrinsics

The YAML includes transforms between the reference frame and the 6DoF pose of each sensor. The transforms are specified as positions [x,y,z] in meters with quaternions [x,y,z,w].

Extrinsics example

# Calibration Anywhere by Main Street Autonomy
# 
# Position xyz; Quaternions xyzw
# [ x_m, y_m, z_m, qx, qy, qz, qw]
back_2d_lidar:
  parent: "base_link"
  child: "back_2d_lidar"
  value: [-0.494047, 0.006522, 0.426849, -0.001075, 0.001938, -0.017212, 0.999849]
front_2d_lidar:
  parent: "base_link"
  child: "front_2d_lidar"
  value: [0.021299, -0.003307, 0.424852, 0.001939, 0.001071, 0.999878, 0.015468]
front_3d_lidar:
  parent: "base_link"
  child: "front_3d_lidar"
  value: [-0.234892, -0.006363, 0.527931, -0.001696, -0.000016, -0.001686, 0.999997]
chassis_imu:
  parent: "base_link"
  child: "chassis_imu"
  value: [-0.216158, 0.012443, 0.164176, -0.000206, 0.001893, 0.705067, 0.709138]
right_stereo_camera:
  parent: "base_link"
  child: "right_stereo_camera"
  value: [-0.286952, -0.166885, 0.352829, 0.005047, 0.003323, 0.707693, -0.706494]
left_stereo_camera:
  parent: "base_link"
  child: "left_stereo_camera"
  value: [-0.435905, 0.148248, 0.352080, -0.000206, 0.001893, 0.705067, 0.709138]
front_stereo_camera:
  parent: "base_link"
  child: "front_stereo_camera"
  value: [0.102732, 0.063245, 0.351200, -0.002547, -0.008137, -0.004329, 0.999954]
front_fisheye_camera:
  parent: "base_link"
  child: "front_fisheye_camera"
  value: [0.110473, -0.011563, 0.380919, -0.001953, 0.010184, -0.004450, 0.999936]

Using extrinsics

ROS robot_state_publisher publishes static transforms on /tf_static, making the calibrated sensor poses available to any node that queries the TF tree. Load extrinsics.urdf as follows.

ROS2:

ros2 run robot_state_publisher robot_state_publisher --ros-args \
  -p robot_description:="$(cat extrinsics.urdf)"

ROS1:

rosparam set /robot_description "$(cat extrinsics.urdf)"
rosrun robot_state_publisher robot_state_publisher

Camera intrinsics

Filename Description
<sensor_name>.intrinsics.yaml OpenCV-compatible camera intrinsics model

Each imaging sensor has a corresponding YAML file that includes a projection matrix, distortion model, and other components of an OpenCV camera model, plus rolling shutter time.

Calibration Anywhere supports fisheye, equidistant, ftheta3, rational polynomial, and plumbob (linear) models.

Intrinsics example

# Calibration Anywhere by Main Street Autonomy
#
{
    image_width: 1920, 
    image_height: 1200, 
    camera_name: front_fisheye_camera, 
    camera_matrix: {rows: 3, cols: 3, data: [452.05569820004172, 0, 943.94114653517386, 0, 451.71918139761948, 600.59251441034132, 0, 0, 1]}, 
    distortion_model: equidistant, 
    distortion_coefficients: {rows: 1, cols: 5, data: [0.057062094961652564, 0.013893343356487644, -0.0020329006883655913, -0.00080049913664872227, 0]}, 
    rectification_matrix: {rows: 3, cols: 3, data: [1, 0, 0, 0, 1, 0, 0, 0, 1]}, 
    readout_time_sec: 0.015505231518380849, 
    projection_matrix: {rows: 3, cols: 4, data: [452.05569820004172, 0, 943.94114653517386, 0, 0, 451.71918139761948, 600.59251441034132, 0, 0, 0, 1, 0]}
}

OpenCV field mapping

Each camera’s <sensor_name>.intrinsics.yaml is in OpenCV format, compatible with sensor_msgs/CameraInfo. The fields map as follows:

Intrinsics YAML field CameraInfo field
camera_matrix K (3×3 intrinsic matrix)
distortion_model distortion_model
distortion_coefficients D
rectification_matrix R
projection_matrix P
image_width, image_height width, height

Rolling shutter timing

Readout time is included for rolling shutter cameras. readout_time_sec is the total time for all rows in the image to be exposed and read out. A negative value indicates that the sensor reads out from bottom to top.

To compute the exposure time for any given pixel row, combine the readout time with the time offset from time_offsets.yaml:

# units are seconds in system time
readout_time_sec      # total time for image exposure
center_of_exposure    = header_time + time_offset
start_of_exposure     = center_of_exposure - 0.5 * readout_time_sec
exposure_time_for_row = start_of_exposure + readout_time_sec * row_px / image_height_px

Using camera intrinsics

To publish sensor_msgs/CameraInfo with the camera’s intrinsic parameters, load the file using the ROS camera_info_manager or configure it in your camera driver’s camera_info_url parameter.

Wheel speed / encoder intrinsics

Filename Description
wheels_cal.yaml YAML describing wheel intrinsics

The YAML includes an axle track (distance between drive wheels) estimate in meters, plus corrective gain factors for either left and right wheel speeds or meters-per-tick.

Wheel speed / encoder example

# Calibration Anywhere by Main Street Autonomy
#
axle_track_m: 0.4856914300973283
left_speed_gain: 1.030691271066441622
right_speed_gain: 1.030696325397954808

Using wheel speed / encoder intrinsics

corrected_left_speed_mps = raw_left_speed * left_speed_gain corrected_right_speed_mps = raw_right_speed * right_speed_gain

Time offsets

Filename Description
time_offsets.yaml YAML describing time offsets

The YAML lists time of validity offsets for all sensors which were timestamped in the same time domain in the input data. For example, if all the sensors report timestamps in Unix time, or if all of the sensors report timestamps in seconds since turn-on, there will be an entry for every sensor.

The time of validity offset is the difference between the capture time determined by Calibration Anywhere and the reported capture time (typically from the message header). Adding the time of validity offset to each sensor’s timestamps brings each sensor into time alignment.

Time offset example

# Calibration Anywhere by Main Street Autonomy
# Time offsets are listed in seconds
#
/chassis/imu: 0
/back_2d_lidar/scan: -0.022429021
/front_2d_lidar/scan: -0.002225964
/front_3d_lidar/lidar_points: -0.17906949900000002
/front_stereo_camera/image_raw: 0.0072620950000000005
/left_stereo_camera/image_raw: 0.0076680940000000003
/right_stereo_camera/image_raw: 0.0058531150000000007
/front_fisheye_camera/image_raw: 0.0059962550000000007

Ground detection

Filename Description
ground.yaml YAML specifying transforms between the ground plane and each sensor

Example:

# Calibration Anywhere by Main Street Autonomy
# 
ground_relative_to_sensors:
  - sensor_type: lidar
    sensor_name: front_3d_lidar
    origin_sensor_frame:
      - 0.23484351741359347
      - 0.0089428245755625857
      - -0.52791281978551674
    direction_sensor_frame:
      - 3.6959477739646857e-05
      - -0.003392711107378672
      - 0.99999424405610404
  - sensor_type: camera
    sensor_name: front_stereo_camera
    origin_sensor_frame:
      - 0.062374165673735785
      - 0.34980619953554176
      - -0.10789840095531419
    direction_sensor_frame:
      - 0.0050233246870731718
      - -0.99985460047339636
      - 0.016295524578032849
  - sensor_type: camera
    sensor_name: front_fisheye_camera
    origin_sensor_frame:
      - -0.012102494474520616
      - 0.38304527129333032
      - -0.10280394618481999
    direction_sensor_frame:
      - 0.0039954112451021095
      - -0.99978496675938267
      - -0.020348388902378112

Metadata

Filename Description
msa_metadata.tar Reflected metadata file (if provided in upload)

If a file named msa_metadata.tar was included in the log, it will be included in the output zip file. This file is not unpacked or analyzed by Calibration Anywhere.