OpenAI 发布 GPT-6 Cyber 网络安全模型
OpenAI 发布网络安全模型 GPT-6 Cyber,用于发现可利用漏洞并开发修复方案。该模型经过训练,被要求只在授权范围内行动。
推荐理由:OpenAI 推出面向网络安全的专用模型,强调在授权范围内工作,可观察安全智能体的对齐约束如何落地。
Alignment by Socialization: Learning Tradeoffs Under Selection
Thomas Brennan · Andrew W. Lo · Chaoyi Zhao
AI 智能体需在完成任务的同时遵守行为规则,但这两类目标可能冲突。研究演化选择如何塑造智能体在二者间的取舍,以及经验能否使这些取舍反映人类判断并抵御后续压力。模型中,智能体的副本面对共同的不确定奖励状态,只有采取被奖励动作的副本才能繁殖。结果表明,当每种动作被采取频率与环境奖励频率一致时,长期增长最大化。比较三种策略:仅任务策略在只奖励完成度的沙盒中表现良好,但当行为规范重要时失败;固定线性策略随 stakes 升高越来越偏向任务完成,即使人类从不奖励该行为;学习策略则收敛到环境所奖励的权衡,当人类提供奖励时称为社会化。积累的经验使社会化在后续仅任务训练下更难逆转,但主流倾向改变前可能出现违规,有限记忆会削弱这种保护。当单次违规即可终止部署时,硬性禁止可防止违规,但错误禁止也有代价。小神经网络实验展示了训练目标和记忆如何塑造这些结果。对齐不仅取决于赋予智能体的目标,也取决于塑造和维持其行为的环境。
尚无解读。
arXiv:2611.18531v1 Announce Type: new Abstract: Artificial intelligence (AI) agents are expected to complete tasks while following rules of conduct, but these objectives can conflict. We study how evolutionary selection shapes an agent's choices between them, and whether experience can make those choices reflect human judgments and withstand subsequent pressures. In our model, copies of an agent face a common uncertain reward state, and only those taking the rewarded action reproduce. We show that long-run growth is maximized when each action is taken as often as the environment rewards it. We compare three policies. A task-only policy thrives in a sandbox rewarding completion alone but fails when conduct matters. A fixed linear policy increasingly favors task completion as its stakes rise, even where humans never reward it. A learned policy instead converges to the tradeoffs rewarded by its environment; when humans supply the rewards, we call this socialization. Accumulated experience makes socialization harder to reverse under subsequent task-only training, although violations can arise before the prevailing disposition changes, and finite memory weakens this protection. When a single violation can end a deployment, a hard prohibition can prevent it, but a mistaken prohibition can also be costly. Experiments with small neural networks illustrate how training targets and memory shape these outcomes. Our results suggest that alignment depends not only on the objectives assigned to agents, but also on the environments that shape and sustain their behavior.
OpenAI 发布网络安全模型 GPT-6 Cyber,用于发现可利用漏洞并开发修复方案。该模型经过训练,被要求只在授权范围内行动。
推荐理由:OpenAI 推出面向网络安全的专用模型,强调在授权范围内工作,可观察安全智能体的对齐约束如何落地。