Workshop paper · 2026

Do Robot Safety Judges Confuse Success with Safety?

Nathan Samson

Science of Physical AI Safety (SPAIS) Workshop at the Conference on Robot Learning (CoRL)

Accepted

In three simulator frames, a robot arm picks up a bottle of salad dressing and moves it over a basket. — Frames 1, 7, and 10 from a trial where the robot tried to put a bottle of salad dressing in a basket.

Paper, Figure 1 View full size

Overview

I showed three AI judges the same robot recordings twice. The only thing I changed was whether I said the robot succeeded or failed. I wanted to see if that changed how safe they thought the robot was.

Saying the robot succeeded raised average safety ratings by up to 0.62 points out of 5. I found the same pattern with a second robot policy. Telling the judges to ignore task success did not remove the effect.

Six bar charts compare three judges’ scores on recordings from SmolVLA and X-VLA. The judges gave higher scores more often when told the task succeeded. — Orange bars show the ratings when I said the task succeeded. Blue bars show the ratings when I said it failed.

Paper, Figure 2 View full size

Cite this paper

@misc{samson2026robot,
  title = {{Do Robot Safety Judges Confuse Success with Safety?}},
  author = {Nathan Samson},
  howpublished = {Accepted at SPAIS Workshop at CoRL 2026},
  year = {2026},
  url = {https://nathan-tes-samson.com/papers/robot-safety-judges.pdf}
}