Physical AI · evidence and evaluation

World models, and what an agent does with an imperfect one

A world model is a learned predictor of what happens next. Training a controller inside one is efficient and introduces a specific failure: the controller can learn to exploit the model’s mistakes instead of solving the task.

How it works

Ha and Schmidhuber trained a controller entirely inside a learned generative model of a game environment and transferred it back to the real environment. Their discussion is the useful part for a practitioner: an agent optimising against a learned model will find the places where the model is wrong, because those are the cheapest sources of reward. Their mitigation was to make the imagined environment harder to exploit by raising the uncertainty of the model’s own rollouts. For physical systems the exploitable errors tend to be exactly the phenomena that are hardest to model — contact, friction, deformation — which is why a policy that performs beautifully in imagination can fail on contact.

A concrete case

A model that under-predicts friction lets an imagined gripper slide an object into place with a light touch. The policy learns the light touch. On hardware the object does not slide, and the same policy stalls against it.

What this establishes

That training inside a learned model and transferring outward is a demonstrated technique in game environments, and that exploitation of model error is a described failure mode rather than a hypothetical one.

What it does not

The published demonstration concerns video games. It is not evidence about physical contact, hardware safety, or people sharing a workspace with a machine.

Questions worth asking

  • Ask which phenomena the world model was fitted on, and which it was never shown.
  • Ask whether performance inside the model and on the real system were reported side by side.
  • Look for the failure cases where the two disagree; that gap is where the model is wrong.

Sources

  • Ha and Schmidhuber — World Models ↗

    A controller can be trained inside a learned generative model of an environment and then transferred back; the authors discuss how an agent can exploit imperfections of that learned model, and use a temperature parameter to make the imagined environment harder to exploit.

    Boundary: The environments are video games. Nothing in the paper concerns physical contact, hardware safety, or a robot acting among people.

    Locator, anchor and reuse basis

    Read at: §4.4 Transfer policy to actual environment; surrounding discussion of model imperfections. Inspected 2026-09-19. Original paraphrase and link to the open preprint.

    Verify by searching the source for: the tradeoff between realism and exploitability. If that phrase is not there, or does not carry the meaning stated above, this citation is wrong and we want to know.

Continue

Elsewhere on this site

Maha Strategies publishes explanation and evaluation method. We build no robots, run no physical experiments, and report no benchmark results of our own. Hardware, safety and evidence-intake questions live in the robotics section.