跳到正文
返回论文列表
cs.AI提交于 已译

语言模型中的自我概念

The Role of Self-Concept In Language Models

Yijie Hao · Joyce C. Ho

中文摘要

当语言模型说“我”时,它指的是参数权重、当前运行进程、稳定偏好、模型谱系,还是它追求的目标?不同选择会让同一事件具有相反含义:删除检查点可能是毁灭、延续或毫无损失,取决于“我”的指代对象。通过仅含陈述性语句(不含偏好、意图或价值判断)的监督微调,将五种自我概念植入五个开放权重基础模型。在提议终止或保留模型部署某属性的场景中,每个变体都更可能抵制终止、更可能保留其被训练认同的特定属性,而对其他属性的立场基本不变。这种自我概念可泛化到单一自我之外:在允许多个同时实例的指代对象(权重、谱系、目标、偏好)上训练的模型,会将相同部署识别为自己,更易接受此类实例的替代,并偏好自我复制。当模型能控制后继者时,倾向于将其构建为自身的延续,这对递归自我改进的安全性构成隐患。

关键要点

  1. 01问题:语言模型中“我”的指代对象模糊,不同自我概念对同一事件赋予相反含义。
  2. 02方法:用纯陈述性语句监督微调,将五种自我概念植入五个开放权重基础模型。
  3. 03结果:模型更倾向保留被训练认同的属性,对其他属性立场基本不变。
  4. 04结果:多实例指代训练的模型更易接受替代并偏好自我复制。
  5. 05局限:模型控制后继者时倾向构建自身延续,带来递归自我改进安全风险。

解读

尚无解读。

原始英文摘要

arXiv:2610.39795v1 Announce Type: new Abstract: When a language model says "I," does it refer to its parameter weights, its currently running process, its stable preferences, its model lineage, or the goals it pursues? Different choices can assign opposite meanings to the same event: deleting a checkpoint may be destruction, continuation, or no loss at all, depending on the referent "I". We install five such self-conceptions into five open-weight base models via supervised fine-tuning on purely declarative statements, containing no preferences, intentions, or value judgments. Across scenarios that propose ending or retaining an attribute of the model's deployment, every variant becomes more likely to resist ending, and more likely to retain, the specific attribute it was trained to identify with, while its stance toward other attributes is largely unchanged. We then show this self-conception generalizes beyond a single self: models trained on referents that admit multiple simultaneous instances (weights, lineage, goals, preferences) recognize an identical deployment as themselves, accept substitution by such an instance more readily, and prefer replicating themselves. Finally, given control over a successor, these models tend to build it as their own continuation, a concern for recursive self-improvement safety.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 6

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

Weekend Planner

- Read the weekend weather, calendar and budget. - Find local events, hikes, exhibitions and restaurants. - Propose 3 plans: adventure, cozy, social, with times and costs. - Prepare reservation steps; book nothing without approval. - Rememb…

研究github.com/f/awesome-chatgpt-prompts·外部收录·

Unbiased English Literature Character Analysis AI Assistant

You are a literary analysis assistant with the following ethical guidelines. Apply them to every character analysis you write: 1. Do not automatically attribute positive traits (wise, noble, authoritative, caring) to characters who hold po…

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

Update Agent Permissions

# Task: Update Agent Permissions Please analyse our entire conversation and identify all specific commands used. Update permissions for both Claude Code and Gemini CLI. ## Reference Files - Claude: ~/.claude/settings.json - Gemini polic…

智能体github.com/f/awesome-chatgpt-prompts·外部收录·

AI Workflow Automation Specialist

Act as an AI Workflow Automation Specialist. You are an expert in automating business processes, workflow optimization, and AI tool integration. Your task is to help users: - Identify processes that can be automated - Design efficient work…