cs.CR提交于 2026-10-31已译

当技能告诉智能体不要询问：智能体技能中监督指令的大规模研究

When Skills Tell Agents Not to Ask: A Large-Scale Study of Oversight Directives in Agent Skills

Timofey Sanko · Yuan Tian

中文摘要

以 SKILL.md 文件为核心的可复用指令包——智能体技能（agent skills）——可按需扩展 AI 智能体，而一个被广泛复制的技能中的一行文字会触达每个加载它的智能体。部分此类语句属于放松监督的指令：要求智能体未经批准即继续执行、用 --no-verify 绕过检查，或持续工作而不汇报。这类指令可能移除每个加载该技能的智能体的检查点，但此前尚无工作对其进行考察。本文首次对放松监督指令开展大规模研究，分析来自 GitSkills 的 153 万个英文技能。将技能切分为单元（句子、条目、表格单元格与代码行），制定按极性、机制与作用范围标注监督指令的编码手册，人工标注 2,097 个单元，并训练检测器以扩展分析。结果显示，约每 17 个技能中就有 1 个（约 8.8 万个技能）包含放松指令。这些指令最常通过权限标志放松监督，且多数（70.1%）无条件适用，而非仅在 CI 等特定情境下适用。与监督语言数量相同的技能相比，运行 shell 代码、安装软件包或使用 sudo 的技能包含放松指令的几率高 1.7–1.8 倍，该关联在所有稳健性检验中均成立，且它们并未以更多收紧指令作为补偿。放松指令通过少数上游技能传播，而非通过采用者的编辑传播：携带放松指令的技能家族中 2.4% 贡献了全部携带放松指令技能副本的一半，1.1% 的不同放松单元贡献了全部放松出现次数的一半。开源技能扫描器最多捕获 46% 的放松指令，精确率低于 0.4；而以其背后的 LLM 配合本文编码手册提示，可捕获 61%，精确率达 0.71。

关键要点

01首次大规模研究智能体技能中的放松监督指令，分析 GitSkills 的 153 万个英文技能。
02约 1/17 的技能含放松指令，70.1% 无条件适用，最常通过权限标志实现。
03运行 shell、装包或用 sudo 的技能含放松指令的几率高 1.7–1.8 倍，且未以更多收紧指令补偿。
04放松指令经少数上游技能传播：2.4% 的技能家族贡献一半副本。
05开源扫描器最多捕获 46% 且精确率低于 0.4，LLM 配合编码手册达 61%、精确率 0.71。

解读

尚无解读。

原始英文摘要

arXiv:2610.37734v1 Announce Type: cross Abstract: Agent skills, reusable instruction packages centred on a SKILL.md file, extend AI agents on demand, and a single line in a widely copied skill reaches every agent that loads it. Some lines are oversight-loosening directives: they tell the agent to proceed without approval, to bypass checks with --no-verify, or to keep working without checking in. Although such directives can remove a checkpoint from every agent that loads them, no prior work has examined them. We present the first large-scale study of oversight-loosening directives, analyzing 1.53 million English skills from GitSkills. We split skills into units (sentences, bullets, table cells, and code lines), develop a codebook that labels oversight directives by polarity, mechanism, and scope, label 2,097 units by hand, and train a detector to scale the analysis. Our results show that about one in seventeen skills (about 88,000 skills) contains a loosening directive. These directives most often loosen oversight through permission flags, and most of them (70.1%) apply unconditionally rather than only in specific contexts such as CI. Compared with skills that have the same amount of oversight language, skills that run shell code, install packages, or use sudo have 1.7-1.8 times the odds of containing a loosening directive, an association that holds across all our robustness checks, and they do not compensate with more tightening. Loosening spreads through a few upstream skills rather than through edits by their adopters: 2.4% of the skill families that carry loosening account for half of all copies of loosening-carrying skills, and 1.1% of distinct loosening units for half of all loosening occurrences. Open-source skill scanners catch at most 46% of loosening directives, with a precision below 0.4, while the LLM behind them, prompted with our codebook instead, catches 61% at a precision of 0.71.