Robotics evidence guide · automated editorial preparation, not expert review
Document how a robot outcome was judged
Ground truth is a reference with its own method and limitations. Identify who or what judged success and whether that judgement is independent of the controller being evaluated.
Evidence and interpretation
NIST’s measurement guidance motivates explicit treatment of the reference estimate. Maha proposes a separate outcome-observer record: instrument or annotation method, applicable calibration, visibility limits and adjudication rule. A controller’s success flag is not an external observation of object placement. Likewise two models agreeing on a video do not automatically supply an independent physical reference. Preserve disagreements rather than overwriting them with a majority label whose meaning was never specified.
Proposed evidence workflow
- Define the outcome reference and its relationship to the controller.
- Record unobservable intervals, annotation disagreements and reference limitations.
- Keep controller-reported completion separate from externally judged completion.
Worked illustration
The grid verifier reads the simulator’s final state. That is an exact reference inside the program but shares its assumptions and potential defects. A physical demonstration filmed by one occluded camera would instead require an unresolved outcome when the release cannot be seen.
Limits
This page does not establish an independent oracle for the toy experiment or validate a physical measurement system.
Sources and review
Locator: §2.1–2.4, including the note distinguishing error and uncertainty
Measurement uncertainty accompanies an estimate; Type A and Type B classify evaluation methods, not simply random versus systematic effects.
Boundary: This section does not supply robot-specific tolerances, a calibration certificate or a complete uncertainty budget.
Inspection and reuse basis
Selected sections inspected 2026-09-15. Original paraphrase and link only; no publication reproduced. This is not independent expert review of this article.