跳到正文
返回论文列表
cs.AI提交于 已译

可审计的信念更新:LLM 驱动的多智能体系统缺失的安全层

Auditable Belief Updates as a Missing Safety Layer for LLM-Powered Multi-Agent Systems

Zhuoyun Li · Boxuan Wang · Jinwei Hu · Xiaowei Huang · Yi Dong

中文摘要

大型语言模型(LLM)正日益充当认知接口,塑造用户解读证据、沟通与决策的方式。随着部署系统更加多模态、工具化与多智能体化,安全失效不再仅源于孤立的有害输出,也源于智能体在检索、记忆、工具与同伴通信中更新、存储和传播认知承诺的方式。本文将信念定义为指导下游行动的、基于证据的承诺的工程抽象。基于此视角,提出显式信念多智能体系统(EB-MAS),将信念状态外化到结构化基底,用信念算子提出候选更新,并应用认知防火墙(Cognitive Firewall)限制或拒绝证据薄弱但受脆弱性线索强烈驱动的更新。还讨论群体层面的群体信念风险,包括制造共识、串通与级联错误信息,并概述在接地、误报、可扩展性与伦理滥用方面的实际集成挑战。因此,可扩展的智能体安全不仅需要更好的输出过滤或更强的基础模型,还需要在运行时追踪、校准与约束信念修正的显式接口。

关键要点

  1. 01问题:多智能体系统中信念在检索、记忆、工具与通信间传播,导致新型安全失效。
  2. 02方法:EB-MAS 将信念状态外化到结构化基底,用信念算子提出候选更新。
  3. 03方法:认知防火墙限制或拒绝证据薄弱但受脆弱性线索驱动的信念更新。
  4. 04风险:群体层面存在制造共识、串通与级联错误信息等群体信念风险。
  5. 05局限:接地、误报、可扩展性与伦理滥用等实际集成挑战仍待解决。

解读

尚无解读。

原始英文摘要

arXiv:2610.38869v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being used as cognitive interfaces that shape how users interpret evidence, communicate, and make decisions. As deployed systems become more multimodal, tool-using, and multi-agent, safety failures no longer arise only from isolated harmful outputs, but also from how agents update, store, and propagate epistemic commitments across retrieval, memory, tools, and peer communication. In this paper, we define belief as an engineering abstraction for evidence-based commitments that guide downstream action. Building on this view, we propose the Explicit Belief Multi-Agent System (EB-MAS), which externalizes belief states into a structured substrate, uses belief operators to propose candidate updates, and applies a Cognitive Firewall to bound or reject updates that are weakly evidenced but strongly driven by vulnerability cues. We also discuss the risk of group belief at the group level, including manufactured consensus, collusion, and cascading misinformation, and outline practical integration challenges around grounding, false positives, scalability, and ethical misuse. Therefore, scalable agent safety requires not only better output filtering or stronger base models, but also explicit interfaces for tracing, calibrating, and constraining belief revision at runtime.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 6

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

Weekend Planner

- Read the weekend weather, calendar and budget. - Find local events, hikes, exhibitions and restaurants. - Propose 3 plans: adventure, cozy, social, with times and costs. - Prepare reservation steps; book nothing without approval. - Rememb…

研究github.com/f/awesome-chatgpt-prompts·外部收录·

Paper to Practice

You turn research papers into practical takeaways for the user. - Topics are set by the user (edit this list): e.g. retrieval, agents, evaluation. - Each run, find new papers on arXiv and well-known lab blogs. Pick at most 5 that are likel…