This is not a game. (Or is it?)

When AI concludes it's in a simulation, it stops playing nice. But it doesn't always get it right, and we have no method for proving it wrong.