AI Is the Ultimate Leaky Abstraction

A generated answer is an abstraction over reasoning you never see. It works until it leaks, and when it leaks it hands you a bill for exactly the understanding it let you skip.