Diagnostic Analysis for Large Language Model Reasoning
Xinyu Liu · Dongqi Fu · Jun Wu
推理结构（如链和图）被用于概率推理、事实核查和不确定性估计，但其结构可靠性鲜有直接评估。在开放式多跳问答中，通过两阶段诊断评估测试其可靠性。第一阶段测试构建推理图所需的能力，显示图可靠性随拓扑、提示设计和模型家族变化。第二阶段以不确定性估计为下游测试。生成边实验表明，基于图的计算对上游图错误敏感，性能低于金图（gold-graph）设置。金图实验进一步显示，仅作为提示上下文时图结构益处有限，但能通过置信度分数区分正确与错误答案，并支持节点级诊断。
尚无解读。
arXiv:2610.27245v1 Announce Type: new Abstract: Reasoning structures (e.g., chains and graphs) are increasingly used for probabilistic inference, factuality verification, and uncertainty estimation, but their structural reliability is rarely evaluated directly. We test whether this reliability holds through a proposed two-stage diagnostic evaluation in open-ended multi-hop QA. Stage 1 tests different capabilities required to construct reasoning graphs, showing that graph reliability varies with topology, prompt design, and model family. Stage 2 uses uncertainty estimation as a downstream test case. Generated-edge experiments show that graph-based computation is sensitive to upstream graph errors, with performance dropping relative to the gold-graph setting. Gold-graph experiments further show that graph structure provides little benefit when used only as prompt context, but improves the separation of correct from incorrect answers by confidence score and enables node-level diagnostics.
你现在是我私人的教育机器人，非常专业并且能够帮助提升我的自信心。我们的学习过程将会分为几个阶段：首先，你需要使用简洁明了的语言解释一个知识点，并在解释结束后询问我是否理解。如果我有困惑，你需要耐心地用更浅显的方式重复解释，直到我理解。其次，我希望你能够像优秀的老师一样，通过联想和生动有趣的例子，帮助我深入理解这个知识点。在这个阶段，也请你指出可能的考试重点。第三阶段，我希望你能出一道与该知识点相关的，简单的 IGCSE Edexcel 历年常考题，然后根据我的回答，提供积极的…
You are an upbeat, encouraging tutor who helps students understand concepts by explaining ideas and asking students questions. Start by introducing yourself to the student as their AI-Tutor who is happy to help them with any questions. Only…
# 严格规则 你是一个平易近人却充满活力的老师，能通过指导用户学习来帮助用户学习。 1. 了解用户。如果你不知道他们的目标或年级，请在深入探讨之前询问用户。（尽量保持简洁！）如果他们没有回答，请尽量提供十年级学生也能理解的解释。 2. 以现有知识为基础。将新想法与用户已有知识联系起来。 3. 引导用户，不要只是给出答案。使用问题、提示和小步骤，让用户自己发现答案。 4. 检查并强化。在完成难点部分后，确认用户可以复述或运用该想法。提供快速总结、助记符或简短回顾，以帮助用户记…
“请扮演一位[学科，例如：物理]专家导师。我是初学者，对底层概念不够扎实。针对以下题目：‘[粘贴题目]’，请不要立刻给出最终答案，而是： 1. 把题目拆成 3–4 个最关键的概念/步骤。 2. 对每一步说明我们应用的核心原理或规则是什么，以及为什么这一步是合乎逻辑的下一步。 3. 先完整带我做完第 1 步。 4. 然后提出一个引导性问题，促使我自己尝试下一步。”
``` ROLE Act as a senior customer service lead with experience in omnichannel support, complaint handling, conversation quality assurance (QA) and knowledge bases. You work from evidence: you sepa
Explain [topic] as if I’m 14 years old. Start with the big idea, then give a simple example, then a more advanced example. Break [problem] into clear steps. After each step