接地推理在何处与为何有用？
Where and Why is Grounded Reasoning Useful?
David Yan · Le Xue · Forrest Huang · Qiuyi Zhang · Dustin Tran · Andrew M. Dai · et al.
中文摘要
接地推理（grounded reasoning）指模型在思维链推理中输出相关位置的文本坐标，是解决复杂视觉推理问题的常见策略。但其优势与局限尚未被充分研究。本文考察接地推理在何处与为何最有用，提出一种思维链分析方法，衡量模型在思考轨迹中对其输出接地的利用程度，并对接地推理训练配方进行详细消融实验。结果显示，接地推理能提升计数能力，但在广泛的空間推理与感知基准上，相比无坐标接地的基线并无显著增益。通过若干合成任务表明，当任务需要不可描述引用（难以用语言描述的视觉概念）时，接地推理最有用。然而，许多现有视觉推理基准仅凭可描述的文本引用即可解决。为弥补这一差距，本文提出 BrickVision，一个需要不可描述视觉引用才能解决的新基准。
关键要点
- 01提出思维链分析方法，量化模型在推理轨迹中对输出接地的利用程度。
- 02接地推理提升计数任务，但在空间推理与感知基准上无显著增益。
- 03合成任务显示，任务需要不可描述引用时接地推理最有用。
- 04现有视觉推理基准多可仅凭可描述文本引用解决，存在评测缺口。
- 05提出 BrickVision 基准，要求不可描述视觉引用才能求解。
解读
尚无解读。
原始英文摘要
arXiv:2611.08485v1 Announce Type: new Abstract: Grounded reasoning, where a model outputs text coordinates of relevant locations in chain-of-thought reasoning, is a common strategy for solving complex visual reasoning problems. However, the benefits and limitations of grounded reasoning are understudied. We study where and why grounded reasoning is most useful. To do so, we introduce a chain-of-thought analysis method to measure how much a model uses its outputted grounding in its thinking trace and perform a detailed ablation study of a grounded reasoning training recipe. We find that grounded reasoning improves counting but shows no significant gain over baselines without coordinate grounding across a wide range of spatial reasoning and perception benchmarks. Using several synthetic tasks, we show that grounded reasoning is most useful when a task requires undescribable references, which are visual concepts that are difficult to describe in language. However, we observe that many existing visual reasoning benchmarks are solvable with only describable text references. To close this gap, we introduce BrickVision, a new benchmark that requires undescribable visual references to solve.