生成式递归推理
Generative Recursive Reasoning
Junyeob Baek · Mingyu Jo · Minsu Kim · Mengye Ren · Yoshua Bengio · Sungjin Ahn
中文摘要
未来的神经推理系统应如何实现扩展计算？递归推理模型（Recursive Reasoning Models, RRMs）通过共享转移函数进行迭代隐状态精化，为自回归序列扩展提供了一种有前景的替代方案。然而现有 RRMs 大多为确定性模型，仅沿单一隐轨迹并收敛到单一预测。本文提出生成式递归推理模型（Generative Recursive reAsoning Models, GRAM），将递归隐推理转化为概率性多轨迹计算。GRAM 将推理建模为随机隐轨迹，支持多假设、替代求解策略，并可通过递归深度与并行轨迹采样实现推理时扩展。由此得到隐变量生成模型，支持通过 $p_\theta(y \mid x)$ 进行条件推理，并在输入固定或缺失时通过 $p_\theta(x)$ 进行无条件生成。GRAM 采用摊销变分推断训练，在结构化推理与多解约束满足任务上优于确定性循环与递归基线，并展示了无条件生成能力。
关键要点
- 01现有递归推理模型多为确定性，仅沿单一隐轨迹收敛到单一预测
- 02GRAM 将推理建模为随机隐轨迹，支持多假设与替代求解策略
- 03通过递归深度和并行轨迹采样实现推理时扩展
- 04采用摊销变分推断训练，在结构化推理与多解约束满足任务上优于基线
- 05具备无条件生成能力，但摘要未提及其局限
解读
尚无解读。
原始英文摘要
arXiv:2605.19376v5 Announce Type: replace Abstract: How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative latent-state refinement with shared transition functions. Yet existing RRMs are largely deterministic, following a single latent trajectory and converging to a single prediction. We introduce Generative Recursive reAsoning Models (GRAM), a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM models reasoning as a stochastic latent trajectory, enabling multiple hypotheses, alternative solution strategies, and inference-time scaling through both recursive depth and parallel trajectory sampling. This yields a latent-variable generative model supporting conditional reasoning via $p_\theta(y \mid x)$ and, with fixed or absent inputs, unconditional generation via $p_\theta(x)$. Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while demonstrating an unconditional generation capability. https://ahn-ml.github.io/gram-website