跳到正文
返回论文列表
cs.AI提交于 已译

MemSteer:将用户历史转化为监督信号以优化智能体记忆

MemSteer: Turning User Histories into Supervision for Agent Memory Optimization

Jie Huang · Cuiling Lan · Qize Yu · Xiulian Peng · Yan Lu

中文摘要

有效的智能体记忆系统应适应用户历史与任务需求。人工标注的监督信号可通过提供反馈来评估和改进记忆设计,从而引导这种适应。然而,新用户历史往往缺乏合适标签,手动构建问答对与支撑证据成本高昂。MemSteer 将无标注用户历史转化为带支撑证据的问答(QA)监督,用于记忆设计优化。它构建基于证据的答案计划,明确相关记录中的事实如何共同支撑参考答案;该计划指导问题生成,并通过来源审计与盲答恢复检验标注质量。为研究该监督的效用,作者开发 MemSteer-Opt,将基于 QA 的设计评估与证据引导的失败诊断结合,在不更新模型参数的情况下修改记忆存储、检索与上下文构建。在 ATM-Bench 与 RHELM 上的实验表明,该监督与官方 QA 引导的优化效果相当。相对 Base-BM25,MemSteer-Opt 在 ATM-Bench 的 Question Score 上提升 13.4 分,在 RHELM 上无外部数据与有外部数据时分别提升 4.1 和 6.6 分。该监督在 ATM-Bench 上还使两个现有优化器受益;新用户实验表明,可利用目标历史生成的监督持续优化设计。

关键要点

  1. 01新用户历史缺乏标注,人工构建 QA 与证据成本高,形成监督缺口
  2. 02将无标注用户历史转为带证据的 QA 监督,用答案计划指导问题生成并审计
  3. 03MemSteer-Opt 结合 QA 评估与失败诊断,不更新参数即改记忆存储、检索与上下文
  4. 04ATM-Bench Question Score 较 Base-BM25 提升 13.4 分,RHELM 提升 4.1/6.6 分
  5. 05该监督还使两个现有优化器受益,并支持基于目标历史持续优化设计

解读

尚无解读。

原始英文摘要

arXiv:2610.39276v1 Announce Type: new Abstract: An effective agent memory system should adapt to a user's history and task requirements. Human-annotated supervision can guide this adaptation by providing feedback for evaluating and improving memory designs. Yet suitable labels are rarely available for new user histories, and manually constructing QA pairs and supporting evidence is costly. We introduce MemSteer, which turns unlabeled user histories into question-answer (QA) supervision with supporting evidence for memory-design optimization. MemSteer constructs an evidence-grounded answer plan that specifies how facts across related records jointly support a reference answer. The plan guides question generation, while source-grounded auditing and blind answer recovery check the resulting annotations. To study the utility of this supervision, we develop MemSteer-Opt, which combines QA-based design evaluation with evidence-guided failure diagnosis to revise memory store, retrieval, and context construction without updating model parameters. Experiments on ATM-Bench and RHELM show that this supervision is competitive with official-QA-guided optimization. Relative to Base-BM25, MemSteer-Opt improves ATM-Bench Question Score by 13.4 points and RHELM accuracy by 4.1 and 6.6 points without and with external data, respectively. The supervision also benefits two existing optimizers on ATM-Bench, while new-user experiments demonstrate continued design optimization using supervision generated from the target histories.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 6