SymAnt:以可适配符号先验为基础的长时程动作预测
SymAnt: Grounding Long-Horizon Action Anticipation with Adaptable Symbolic Priors
Fnu Aryan · Yaqi Xie · Joseph Campbell · Woojun Kim · Kwonjoon Lee · Hossein Nourkhiz Mahjoub · et al.
中文摘要
随着机器人和智能系统越来越多地与人类交互,它们必须仅凭简短观察预测未来的长序列动作。纯神经网络模型通常需要大量观测才能预测未来动作,且难以解释和调整;符号先验则提供了结构化的替代方案。为此提出 SymAnt,一种神经-符号(neuro-symbolic)方法,将神经网络的表达能力与场景图和知识图谱的结构化推理相结合。给定一段短视频,SymAnt 对两种图进行联合搜索以构建符号上下文(symbolic context),从而统一推理空间关系、物体属性、可供性(affordance)及相关动作。该符号上下文作为先验,驱动迭代精化过程逐步生成并改进未来动作序列。通过将预测建立在符号上下文之上,SymAnt 以参数高效的方式从短观测窗口预测长时程动作,并允许用户直接编辑底层知识来调整预测而无需额外训练。在 Breakfast、50 Salads、EPIC-KITCHENS、EGTEA Gaze+ 和 Assembly101 五个数据集上的实验表明,SymAnt 在仅有短观测上下文时尤为有效。
关键要点
- 01问题:纯神经模型预测长时程动作需大量观测,且难以解释与调整。
- 02方法:联合搜索场景图与知识图谱构建符号上下文,作为迭代精化生成动作序列的先验。
- 03结果:在 Breakfast、50 Salads、EPIC-KITCHENS、EGTEA Gaze+、Assembly101 五个数据集上验证有效。
- 04优势:参数高效,短观测窗口下表现突出,用户可直接编辑知识调整预测而无需再训练。
解读
尚无解读。
原始英文摘要
arXiv:2610.40632v1 Announce Type: new Abstract: As robots and intelligent systems increasingly interact with humans, they must anticipate long sequences of future actions from only brief observations. While purely neural models typically require extensive observations to predict future actions and remain difficult to interpret and adjust, symbolic priors offer a structured alternative. To this end, we introduce SymAnt, a novel neuro-symbolic approach that unifies the expressiveness of neural networks with the structured reasoning of scene and knowledge graphs. Given a short video, SymAnt performs a joint-search over both graphs to construct a symbolic context, enabling unified reasoning over spatial relationships, object attributes, affordances, and associated actions. This symbolic context serves as a prior for an iterative refinement process that progressively generates and improves the sequence of future actions. By grounding predictions in symbolic context, SymAnt anticipates long-horizon actions from short observation windows in a parameter-efficient manner, while enabling users to directly edit the underlying knowledge to adjust predictions without further training. Through our experiments, we demonstrate the effectiveness of SymAnt, particularly when only short observation contexts are available, across five diverse datasets: Breakfast, 50 Salads, EPIC-KITCHENS, EGTEA Gaze+, and Assembly101.