cs.CL提交于 2026-10-27已译

回答前先提问：用于上下文学习的问题驱动证据推理架构（QERA）

Ask Before You Answer: A Question-Driven Evidence Reasoning Architecture (QERA) for In-Context Learning

Jinhu Qi · Minda Hu · Wentao Zhang · Zixing Song · Irwin King

中文摘要

AI 助手日益需要依据用户自己的网络记录进行推理，包括群聊、社区帖子、活动日志和修订历史。在真实情境上下文学习基准 CL-bench Life 上，前沿 LLM 完成的任务不足五分之一。研究发现，请求本身并非完整规范：超过 80% 的评分规则与最终请求的内容词重合度低于 60%，且 LLM 直接回答时通过这些隐式规则的比例仅为 67%，其余规则为 81%。QERA 将请求转换为需求清单和分类信息需求，并通过代码过滤以忠实于用户表述；随后从完整对话中作答，再次提出相同问题并核验引用，最后执行受保护的局部修复。该方法无需训练或预设评分规则，每个任务约调用 LLM 四次。在四个前沿 LLM、全部 405 个任务上，QERA 对每个模型均取得最高解决率，平均为 21.0%，高于三种推理时基线中最强的 18.2%及直接回答的 14.6%；汇总结果显著领先所有基线，同时 token 用量少于 PSCI，仅为 Ctx2Skill 的 4–5%。GPT-6-Luna 的消融实验表明，收益在回答前已形成：固定回答协议带来 3.8 个规则分，请求特定问题带来 1.6 个，且收益集中于隐式规则；事后检查主要改进数值需求。

关键要点

01真实情境上下文学习困难，前沿 LLM 在 CL-bench Life 上解决率不足 20%
02多数评分规则是请求中未显式表达的隐式约束，直接回答难以识别
03QERA 先生成需求清单与信息需求，再检索证据、提问核验并局部修复
04四个前沿 LLM、405 个任务上的平均解决率最高达 21.0%
05无需训练或预设规则，每任务约四次 LLM 调用且 token 用量较低

解读

尚无解读。

原始英文摘要

arXiv:2610.35083v1 Announce Type: new Abstract: AI assistants are increasingly asked to reason over a user's own web traces: group chats, community threads, activity logs and revision histories. On CL-bench Life, a benchmark of such real-life context learning, frontier LLMs solve fewer than one task in five. We find that the request is not the specification: over 80% of the benchmark's rubrics share less than 60% of their content words with the final request, as for a rule set two turns earlier or every record that meets the user's criterion, and LLMs answering directly pass only 67% of these implicit rubrics against 81% of the others. We propose QERA, a Question-driven Evidence Reasoning Architecture that asks before it answers. QERA first turns the request into a requirement checklist and typed information needs, filtered by code for faithfulness to the user's words; it then answers from the whole conversation, asks the same questions again with quotes verified by code, and applies a guarded local repair. It needs no training or rubrics and makes about four LLM calls per task. On all 405 tasks with four frontier LLMs, QERA attains the highest solving rate on every LLM (21.0% on average, against 18.2% for the strongest of three inference-time baselines and 14.6% for direct answering) and leads every baseline significantly when pooled across LLMs, while using fewer tokens than PSCI and 4-5% of the tokens of Ctx2Skill. Ablations on GPT-6-Luna show that the gain is settled before the answer, by a fixed answer protocol (+3.8 rubric points) and the request-specific questions (+1.6), and that it is concentrated on implicit rubrics, with checking afterwards refining mainly numeric requirements.