Abstract

Reasoning is a core capability of large language models, yet understandinghow they learn and perform multi-step reasoning remains an open problem. Inthis study, we explore how different architectures and training methods affectmodel multi-step reasoning capabilities within a cellular automata framework.By training on state sequences generated with random Boolean functions forrandom initial conditions to exclude memorization, we demonstrate that mostneural architectures learn to abstract the underlying rules. While modelsachieve high accuracy in next-state prediction, their performance declinessharply if multi-step reasoning is required. We confirm that increasing modeldepth plays a crucial role for sequential computations. We demonstrate that anextension of the effective model depth with recurrence, memory, and test-timecompute scaling substantially enhances reasoning capabilities.