UMAS：多智能体LLM系统的系统级不确定性量化
UMAS: System-Level Uncertainty Quantification for Multi-Agent LLM Systems
Hanwen Li · Jinhao Duan · Xiaoshuang Shi · Yue Zhang · Tianlong Chen · Kaidi Xu · et al.
中文摘要
多智能体系统(MAS)是提升大语言模型推理能力的重要范式，但仍容易产生幻觉和错误输出。不确定性量化(UQ)有助于评估系统可靠性，而现有方法主要面向单智能体场景，如何利用MAS结构开展UQ并保持推理成本不变仍缺乏研究。UMAS是一种无需参数、成本中性的系统级UQ框架，从每轮生成的内在置信度估计智能体间影响的可信度，并据此调节各节点的不确定性更新，再将节点级不确定性聚合为系统级估计。在Debate和DyLAN及多种基准上的大量评估表明，UMAS平均比最先进基线高10.54个AUROC点。该方法还支持幻觉检测和不确定性感知答案选择，在特定任务上将MAS准确率最多提升12.6个百分点，并增强可靠性。代码见https://github.com/Dby52-04/UMAS。
关键要点
- 01现有UQ方法主要面向单智能体，未充分利用MAS结构
- 02UMAS依据各轮生成的内在置信度估计智能体间影响可信度
- 03通过调节节点不确定性更新并聚合，生成成本中性的系统级估计
- 04在Debate和DyLAN等基准上平均领先基线10.54个AUROC点
- 05支持幻觉检测和不确定性感知答案选择，准确率最多提高12.6个百分点
解读
尚无解读。
原始英文摘要
arXiv:2610.33013v1 Announce Type: cross Abstract: Multi-Agent Systems (MAS) are a promising paradigm for improving the reasoning capabilities of large language models, yet they remain prone to hallucinations and erroneous outputs. Uncertainty Quantification (UQ) is crucial for assessing system reliability, but existing methods are largely designed for single-agent settings; how to exploit MAS structure for UQ while keeping inference cost-neutral remains underexplored. We propose UMAS, a parameter-free, cost-neutral, system-level UQ framework that estimates the credibility of inter-agent influence from each generation's intrinsic confidence. UMAS uses these credibility signals to modulate how uncertainty is updated at each node, and then aggregates node-level uncertainty into a system-level estimate. Extensive evaluations on Debate and DyLAN across diverse benchmarks show that UMAS consistently outperforms state-of-the-art baselines by an average of 10.54 AUROC points. Beyond uncertainty estimation, UMAS enables hallucination detection and uncertainty-aware answer selection, improving MAS accuracy by up to 12.6 points on specific tasks while enhancing reliability. Code is available at https://github.com/Dby52-04/UMAS.