LLMs excel at programming—how can they be so bad at it?

My explanation for the mystery of why LLMs can be both exceptionally good and quite terrible at programming.