TaF-Tactile-Force: Tactile Images, Pressure Maps and Six-Axis Force on One Clock

A tactile camera shows a gel deforming. It does not tell you how many newtons did it. TaF-Tactile-Force puts the force on the same clock as the image, so for any contact you can see the gel and read the load.

## What is paired

Each frame pairs what a tactile sensor sees with what a force sensor measures. A vision-based tactile sensor records the gel deforming, a 12x12 piezoelectric array records where the contact lands, and an ATI six-axis sensor records the wrench that caused it. All three share one 30 fps clock. Contacts reach 90.8 N and 1.58 Nm.

The set has 3,594 episodes across 408 sequences, 10,057,742 frames and 93.1 hours. It is a conversion of jiamig/taf-dataset, which accompanies the paper TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulation.

Stream	What it holds
/tactile-camera	The tactile image
/pressure-map	The 12x12 array as a heatmap
/force-torque	The six-axis wrench, with /force-torque.plot carrying fx, fy, fz, tx, ty and tz as a timeline
/pressure.plot	Summed pressure, the peak cell, and how many cells are in contact
/instruction	The sensor configuration

Six sensor configurations are represented:

Configuration	Episodes
GelSight Mini with markers	1,103
GelSight Mini without markers	810
Custom sensor without markers	657
Custom sensor, 6x6 markers	650
Custom sensor, 8x8 markers	285
Custom sensor, 7x7 markers	89

Peak force runs to a median of 25.0 N and a maximum of 90.8 N. Peak pressure runs to a median of 884 and a maximum of 4052.

Find the hardest presses

Episodes carry peak_force, peak_fz, peak_torque and peak_pressure, so you can select on how hard the contact got:

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/TaF-Tactile-Force",
    name="TaF-Tactile-Force",
    persistent=True,
)

view = dataset.match({"peak_force": {"$gt": 50}})
fo.launch_app(dataset, view=view)

Open one of those episodes. The tactile image shows the gel’s deformation, and the force-torque plot carries the load that produced it, on the same clock.

Reading it correctly

The pressure heatmap is scaled per episode against that episode’s 99.5th percentile, so a light touch and a hard press both stay readable. That means heatmap colours are not comparable across episodes. Use /pressure.plot for the unscaled values.

Tactile image resolution varies by sequence. The tactile video was AV1 in the source, which the viewer does not decode, so it is re-encoded to H.264.

Three configurations spell their marker grid with an asterisk in the sequence field, as in custom_designed_6*6_mark_obj1. Directories on disk carry 6x6, 7x7 and 8x8 instead, because the asterisk cannot appear in a Windows path.

TaF-Dataset is released under the MIT license, and the conversion uses the same license. The data is at Voxel51/TaF-Tactile-Force.

For what a gel image can and cannot show about touch more broadly, see Robots Can See. They Still Can’t Feel: tactile LeRobot datasets in FiftyOne. For more of my FiftyOne dataset notes, see my Voxel51 author page, and to inspect and curate your own multimodal data in the same tool, try Voxel51.

Frequently asked questions

What does TaF-Tactile-Force pair together?

A vision-based tactile image of the gel deforming, a 12x12 piezoelectric pressure array, and an ATI six-axis sensor that records the force and torque, all on one 30 fps clock.

How big is it and how hard do the contacts get?

3,594 episodes across 408 sequences, 10,057,742 frames and 93.1 hours. Contacts reach 90.8 N and 1.58 Nm, and the median peak force is 25.0 N.

How do I find the firmest contacts?

Each episode carries peak_force, peak_fz, peak_torque and peak_pressure fields. Match on peak_force greater than 50 to select the firmest contacts.

Which sensors are covered?

Six configurations: GelSight Mini with and without markers, and four custom sensors (no markers, and 6x6, 8x8 and 7x7 marker grids). GelSight Mini with markers has the most episodes, 1,103.

Do the counts match the source card?

Not exactly. This conversion holds 3,594 episodes and 10,057,742 frames, against the 3,592 and 10,053,265 the source card states, across the same 408 sequences.