Raising the Bar for LLM Introspection

Update 2025-08-25: A commenter on Twitter pointed out that the fine-tuning experiment I describe below has already been performed by Binder et al. - and with positive results, no less! I'm duly surprised, and my assessment of LLM introspective ability has increased accordingly. tl;dr: I propose a "heavyweight" form of LLM introspection that requires introspection output to be invariant to the…