Can You Dramatically Improve Results on the Latest Large Language Model Reasoning Benchmark With a Simple Prompt?

tldr: Yes. A recent paper, GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models, makes the claim that their new benchmark demonstrates that state of the art reasoning models have critical limitations when it comes to reasoning. Putting aside the fact that they grouped together tiny 2.7 B parameter models (that could fit [...]