How to capture sensor data

• Do not modify sensor data
• Configure sensors to capture timestamps
• Store data in a convenient format
• Follow recomendations for specific sensor types

Raw sensor data

MSA software runs best on raw unmodified sensor data:

• Do not motion compensate or transform lidar point clouds, these data are unusable
• Do not resample or filter or post-process IMU or GNSS/GPS data
• Do not crop or rectify or apply filters or correct white balance or demosaic images

One exception: please compress camera images! JPEG compression is just fine.

Sensor data timestamps

MSA software benefits from sensor data with timestamps associated with each individual measurement. Precise hardware time synchronization (PTP) is not necessary.

If possible, sensors should be configured to timestamp data at the time of measurement:

• Cameras: Each image timestamped at time of exposure
• Depth cameras: Each depth image timestamped at time of source image exposure
• Lidars: Each individual point timestamped at time of capture (recording raw lidar packets is often the easiest way to ensure this)
• Radars: Each return timestamped at time of capture
• IMUs: Each sample timestamped at time of capture
• Encoders: Each tick or speed sample timestamped at time of measurement
• GPS/GNSS: Each sample timestamped at time of measurement
• INS: Each output timestamped at time of estimation

Note: If your system cannot meet these recommendations, please send the data anyway. We regularly work with imperfect timestamping setups and can advise on next steps after reviewing your data.

Sensor data storage format

ROS data format (preferred)

Storage: Save data in ROS1 .bag files or ROS2 .mcap or .db3 files.

Message types: Use standard ROS message types whenever possible. See Wheel Encoders message type recommendation.

Rosbag requirements: Split bag files are acceptable. If multiple ROS bags are submitted, they should not contain duplicates of the same message (this will happen if multiple instances of rosbag record are subscribed to the same topic).

Non-ROS Data Format

Storage: Save all the data in a single .tar file.

Camera data: Store images in .jpg format in a directory named after the sensor, where the image filename is the exposure timestamp. For example: camera_front/1234567890.jpg.

Lidar data: Store lidar data (raw UDP packets collected with tcpdump) in a .pcap file named after the sensor. For example: lidar_front.pcap. Include relevant metadata, like metadata.json, that has information about manufacturer intrinsic calibration of lidar beam angles.

IMU data: Store IMU data in a .csv file named after the sensor. For example: main_imu.csv. Include the following header and data in these columns: send_time_s, time_s, ax_mpss, ay_mpss, az_mpss, gx_rps, gy_rps, gz_rps, where send_time_s is the recording time, and time_s is the signal capture time.

Encoder data: Store wheel encoder or wheel speed data in a .csv file named after the sensor. For example: wheel_speeds.csv. Include the following header and data in these columns: send_time_s, time_s, wheel_left_mps, wheel_right_mps, where send_time_s is the recording time, and time_s is the signal capture time.

Sensor configuration recommendations

Cameras

Image rate: Preferably >5 Hz unless your robot moves very slowly.

Image format: JPEG or PNG compressed images (ROS CompressedImage message) are preferred. Do not rectify or otherwise process images.

Image resolution: Provide images at the resolution used by your autonomy stack. Minimum resolution is 600x400 pixels.

Depth Cameras

Dot projection off: Turn off dot projection for depth camera images.

Range awareness: Depth cameras produce the best data within a short range, so it’s very important that they observe tall static structure within 1-2 meters during the data collection.

Lidars

Unmodified data: Store raw unmodified data as reported by the sensor. Do not transform, rotate, or apply motion compensation to lidar data.

Native output: If your lidar produces packets, record and send those if possible. Also include any topics or files like metadata.json that contain lidar configuration or manufacturer beam-angle intrinsics information.

Per-point timestamps: Record individual timestamps for each lidar point. All major lidar manufacturers support per-point timestamping. The easiest way to get per-point timestamps is to record lidar packets. To store per-point timestamps with individual points, you may need to adjust the lidar driver configuration.

IMUs

Use a ROS Imu message with linear_acceleration populated with values in m/s^2 and angular_velocity populated with values in rad/s, plus header timestamps.

Unmodified data: Do not remove gravity from acceleration data. Try to provide raw, unfiltered measurements without any preprocessing.

Wheel Encoders

Use a ROS JointState message with position or velocity populated, plus header timestamps, for each encoder.

INS

Use a ROS TransformStamped message with transform populated with the 6-degree-of-freedom transform between the sensor and a global frame (e.g., ECEF or UTM).

We do not require any data about the uncertainty of the INS system’s pose fix. Please exclude unreliable measurements; for example, if the system is not yet in it’s steady state of localization that you would use online, do not include those messages.

Quality Check

Before submitting data:

• Check that each sensor is represented by a topic
• Check that each topic contains messages at the expected rate

If you are using ROS, rosbag info is useful for quality checks.