Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure

摘要

Latent or continuous chain-of-thought methods replace explicit textual rationales with a number of internal latent steps, but these intermediate computations are difficult to evaluate beyond correlation-based probes. In this paper, we view latent chain-of-thought as a manipulable causal process in representation space by modeling latent steps as variables in a structural causal model (SCM) and analyzing their effects through step-wise $\mathrm{do}$-interventions. We study two representative paradigms (i.e., Coconut and CODI) on both mathematical and general reasoning tasks to investigate three key questions: (1) which steps are causally necessary for correctness and when answers become decidable early; (2) how influence propagates across steps and relates to explicit CoT; (3) how to characterize and interpret the influence patterns revealed by (2). Across settings, we find that latent-step budgets should be treated as distinct functionalities rather than homogeneous extra depth, We further show that training/decoding should account for a gap between early output bias and late representational commitment. These results motivate mode-conditional and stability-aware analyses as more reliable tools for interpreting and eventually improving latent reasoning systems.