Construction-Site-Traversability: Robot Loops With Camera, Range and Lidar in FiftyOne

Most robot driving datasets are one pass through a place. Construction-Site-Traversability is a tracked robot driving closed loops through two active construction sites, so the same ground shows up again on later passes. That revisiting is what makes the runs useful for loop closure and for traversability work.

What is in it

The recordings come from the construction-site ROS 2 release. The Voxel51 version, Voxel51/Construction-Site-Traversability, rewrites them from ROS 2 MCAP into native multimodal MCAP episodes that the FiftyOne App reads directly.

The robot carries an OAK-D RGB-D camera, a Livox 3D lidar, two inertial units and a u-blox GNSS receiver. A lidar-inertial odometry estimate and the wheel encoders are recorded alongside them. There are 4 sessions, 1 hour 45 minutes of driving, and 9,760 m of ground.

Session	Site	Colour frames	Fixes	Duration	Path
site1_session01	site1	28,219	1,850	47m 02s	3166 m
site1_session02	site1	8,854	884	14m 45s	1203 m
site2_session01	site2	17,226	1,382	28m 43s	4852 m
site2_session02	site2	8,696	870	14m 30s	538 m

Across the whole set that is 62,995 colour frames, 62,931 range frames, 62,995 lidar sweeps holding 0.82 billion points, 1,888,884 inertial samples, 556,459 pose estimates and 4,986 satellite fixes.

Each episode has these streams:

Stream	What it holds
/camera	OAK-D colour camera, 640x480
/depth	OAK-D range image, 640x400
/lidar-points	Livox sweeps with x, y, z, intensity, tag, line and time_offset
/imu.plot, /livox-imu.plot	Body and lidar inertial units, 100 Hz and 200 Hz
/odometry, /odometry.plot	Lidar-inertial pose estimate, with position and velocity
/wheel-odometry.plot	Wheel encoder estimate
/gnss	Position fixes
/camera-calibration, /tf	Colour camera intrinsics, and where the camera and lidar sit on the robot
/session	The run name

Episode fields include site, session, duration, num_lidar_points, has_gnss and odometry_path_m.

Scrub three views of the same ground

Load it and launch the App:

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/Construction-Site-Traversability",
    name="Construction-Site-Traversability",
    persistent=True,
)
fo.launch_app(dataset)

The colour camera and the lidar both run at 10 Hz. The range camera runs at 20 Hz and every second frame is kept, so the visual and 3D streams in an episode share one clock. Open an episode and you can scrub the colour frame, the range image and the point cloud together, then check where the robot was on the GNSS track.

To start with the longest run, sort by duration. To compare the two sites, filter on the site field:

from fiftyone import ViewField as F

view = dataset.sort_by("duration", reverse=True)
fo.launch_app(dataset, view=view)

site2 = dataset.match(F("site") == "site2")
print(site2.count())

Read the notes before you trust a stream

• Range is not depth. The range camera publishes an 8-bit single-channel image rather than metric depth. The two cameras differ in resolution, so the frames are not pixel-aligned. The range camera has no published extrinsic and stays in its own frame.
• Path length is decimated. odometry_path_m is measured from the pose stream decimated to 10 Hz. The estimate is published at 100 Hz, and its per-tick noise is the same order as the robot’s travel between ticks, so summing it at full rate reads several kilometres of travel across a site tens of metres across.
• Odometry has no named body. The estimate stays on /odometry rather than becoming a moving transform. The calibration does not say which body it tracks, and its height over a run matches neither the robot’s base nor the scanner.
• GNSS is sparse. Only GGA sentences are carried, and sentences reporting no fix are dropped. Fixes arrive at about 1 Hz against a much busier source topic. They place both sites near Bangalore, India.
• Sweeps hold returns only. The lidar driver writes a miss as a point at the origin, and those points are dropped. What is left runs from 9,313 to 15,449 points per sweep, depending on how open the ground was.
• Not everything from the source is here. The TF and static transform streams of the source recording and the receiver’s raw correction input are not reproduced.

The recordings were made on working sites and contain site personnel and vehicles in the camera streams. They are republished unchanged from a public release.

License and related work

The source dataset is released under CC BY-NC 4.0, and this conversion uses the same license. Use is limited to non-commercial purposes and attribution is required. The authors publish their annotated frames, camera calibration and a trained segmentation model separately at manojkarnekar/construction-traversability-dataset.

For another mobile robot with several sensors on one timeline, see SEW-Multimodal-AMR, which adds thermal and 4D radar to RGB.

For more of my FiftyOne dataset notes, see my Voxel51 author page, and to curate your own robot data in the same tool, try Voxel51.

Frequently asked questions

What is in the Construction-Site-Traversability dataset?

4 sessions from a tracked mobile robot, 1 hour 45 minutes of driving over 9,760 m of ground, split 2 at site1 and 2 at site2. Each episode has an OAK-D colour camera, an OAK-D range image, Livox lidar sweeps, two inertial units, a lidar-inertial odometry estimate, wheel odometry and GNSS fixes.

Why are these recordings useful for loop closure and traversability work?

The robot drives closed loops, so the runs revisit the same ground. Seeing the same place on different passes is what loop closure needs, and it gives traversability work repeated views of one terrain.

How do I load Construction-Site-Traversability in FiftyOne?

Call load_from_hub with Voxel51/Construction-Site-Traversability and persistent=True, then launch the App. Sort by duration to start with the longest run.

Is the range image metric depth?

No. The range camera publishes an 8-bit single-channel image rather than metric depth. The colour and range cameras also differ in resolution, so their frames are not pixel-aligned.

Can I use Construction-Site-Traversability commercially?

No. The source dataset and this conversion are released under CC BY-NC 4.0, which limits use to non-commercial purposes and requires attribution. The recordings also show site personnel and vehicles in the camera streams.