Replaying robot logs is not reproducing an experiment

Evidence and interpretation

rosbag2 documents recording, playback ordering and inspection of bag metadata. Maha’s proposed package complements those functions with task and version bindings. Record the time basis, data gaps and dependencies before using replay as evidence. Do not assume that replaying identical messages into a stateful system recreates its initial state or thread scheduling. Choose whether the comparison requires exact bytes, a metric tolerance or a qualitative outcome, and justify that choice before looking at the result.

Proposed evidence workflow

1. Retain the recording format, selected streams and time-order convention.
2. Bind replay software, initial state and expected comparison method.
3. Keep a replay result separate from the original observation and record mismatches.

Worked illustration

The toy example uses integer transitions and a fixed traversal order, so exact replay is feasible. Its verifier reruns all planned cases and compares the complete package. A future ROS bag adapter would need its own tests; the prototype does not parse bags or guarantee deterministic robot control.

Limits

Successful replay verifies this implementation’s consistency. It does not authenticate who recorded a real event, establish completeness of a sensor view or verify physical safety.

Sources and review

Continue reading

• Robot evidence needs explicit frames and units
• Compare controller revisions without moving the goalposts