Physical AI · evidence and evaluation

A runnable perception–action fixture, and what it is not

This section ships a small deterministic simulation of a perception–action loop with latency, noise, an uncertainty monitor, intervention and a failure report. It is a teaching fixture: it demonstrates the accounting, not any robot’s capability.

How it works

The fixture runs a one-dimensional tracking task over fixed seeds. Each step produces a noisy observation, the controller acts on an observation delayed by a configurable number of steps, and a monitor may request intervention when the freshest observation disagrees with the running estimate. The report separates autonomous successes, assisted successes, failures and aborted runs, and splits monitor firings into those a deliberately injected disturbance explains and those it does not. Without an injected disturbance every firing is a nuisance alarm by construction, which is the cheapest way to see what a badly set threshold costs. Three counterexamples ship with it: latency alone changing the outcome, a threshold tightened past the noise floor, and a real disturbance that the monitor does catch.

A concrete case

With seed 7, no delay and the monitor held off, the tracker reaches tolerance in four steps and the run is scored autonomous. With three steps of delay and everything else identical, the same controller diverges and the run is scored a failure with a final error near 5, the last observation and command retained. Tightening the monitor to 0.02 instead produces fourteen interventions on pure noise: the task still succeeds, but it is now scored assisted rather than autonomous.

What this establishes

That latency and monitoring policy change closed-loop outcomes independently of perception accuracy, within this simulation.

What it does not

A synthetic one-dimensional simulation with no physics, no contact, no hardware and no people. Nothing it produces is evidence about a real robot’s capability or safety, and its numbers are properties of the fixture.

Questions worth asking

  • Run it with delay 0 and delay 3 and compare the reports.
  • Move the monitor threshold and watch nuisance interventions rise while task success does not.
  • Try to make it report success it did not achieve; the scoring refuses unrecorded outcomes.

Sources

This page proposes a method rather than reporting a finding about the world, so it cites no external source. Where it describes something published, that page carries the citation.

Continue

Maha Strategies publishes explanation and evaluation method. We build no robots, run no physical experiments, and report no benchmark results of our own. Hardware, safety and evidence-intake questions live in the robotics section.