Revisiting the shutdown problem (Part 1: Introduction)

The catastrophic shutdown problem is (roughly) the problem of designing AI systems that can be shut down when their acts would lead to existential catastrophe.