Sara Vera Marjanović Arkil Patel Vaibhav Adlakha Milad Aghajohari Parishad BehnamGhader et al

Abstract

Large Reasoning Models like DeepSeek-R1 mark a fundamental shift in how LLMsapproach complex problems. Instead of directly producing an answer for a giveninput, DeepSeek-R1 creates detailed multi-step reasoning chains, seemingly"thinking" about a problem before providing an answer. This reasoning processis publicly available to the user, creating endless opportunities for studyingthe reasoning behaviour of the model and opening up the field of Thoughtology.Starting from a taxonomy of DeepSeek-R1's basic building blocks of reasoning,our analyses on DeepSeek-R1 investigate the impact and controllability ofthought length, management of long or confusing contexts, cultural and safetyconcerns, and the status of DeepSeek-R1 vis-`a-vis cognitive phenomena, suchas human-like language processing and world modelling. Our findings paint anuanced picture. Notably, we show DeepSeek-R1 has a 'sweet spot' of reasoning,where extra inference time can impair model performance. Furthermore, we find atendency for DeepSeek-R1 to persistently ruminate on previously exploredproblem formulations, obstructing further exploration. We also note strongsafety vulnerabilities of DeepSeek-R1 compared to its non-reasoningcounterpart, which can also compromise safety-aligned LLMs.