This article explores the phenomenon where Large Language Models (LLMs) can produce less accurate outputs when given extended reasoning time or additional thinking steps. It explains how sequential token generation can lead models to deviate from correct initial conclusions by over-analyzing simple questions or building on erroneous interpretations. The analysis highlights the tension between using extra reasoning to catch mistakes and the risk of drifting away from the intended objective.
Key points
LLMs generate text token-by-token, meaning each subsequent piece of output is conditioned on previous reasoning steps, which can either refine or degrade the result.
Extended reasoning time is beneficial for complex problems where errors can be self-corrected through additional verification steps.
For simple or straightforward questions, forcing an LLM to 'think' more can lead to unnecessary complexity or the introduction of irrelevant details that distort the original, correct answer.
The accuracy of a model is not strictly proportional to the amount of reasoning time spent; it depends on the intrinsic difficulty of the prompt.