跳到正文
返回论文列表
cs.AI提交于 已译

预训练、提示与预测:自然语言处理中提示方法的系统综述

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

Pengfei Liu · Weizhe Yuan · Jinlan Fu · Zhengbao Jiang · Hiroaki Hayashi · Graham Neubig

中文摘要

综述并梳理自然语言处理中的新范式“基于提示的学习”。与传统监督学习训练模型计算 P(y|x) 不同,基于提示的学习直接建模文本概率。为执行预测任务,原始输入 x 经模板修改为含未填充槽的文本字符串提示 x′,语言模型概率性填充槽位得到 x̂,进而导出输出 y。该框架允许语言模型在海量原始文本上预训练,并通过定义新提示函数实现少样本甚至零样本学习,以少量或无需标注数据适应新场景。文章介绍该范式基础,提出统一数学符号覆盖现有工作,并从预训练语言模型选择、提示设计、调优策略等维度组织文献。为便于初学者,除系统综述和结构化分类外,还发布网站 NLPedia–Pretrain 提供持续更新的综述和论文列表。

关键要点

  1. 01问题:传统监督学习需大量标注数据,难以适应新场景。
  2. 02方法:将输入经模板转为含槽位的提示,用语言模型概率填充并导出输出。
  3. 03结果:统一数学符号覆盖多种现有工作,并按模型、提示、调优策略分类。
  4. 04局限:未明确讨论提示方法的计算成本或性能边界。

解读

尚无解读。

原始英文摘要

This article surveys and organizes research works in a new paradigm in natural language processing, which we dub “prompt-based learning.” Unlike traditional supervised learning, which trains a model to take in an input x and predict an output y as P ( y|x ), prompt-based learning is based on language models that model the probability of text directly. To use these models to perform prediction tasks, the original input x is modified using a template into a textual string prompt x′ that has some unfilled slots, and then the language model is used to probabilistically fill the unfilled information to obtain a final string x̂ , from which the final output y can be derived. This framework is powerful and attractive for a number of reasons: It allows the language model to be pre-trained on massive amounts of raw text, and by defining a new prompting function the model is able to perform few-shot or even zero-shot learning, adapting to new scenarios with few or no labeled data. In this article, we introduce the basics of this promising paradigm, describe a unified set of mathematical notations that can cover a wide variety of existing work, and organize existing work along several dimensions, e.g., the choice of pre-trained language models, prompts, and tuning strategies. To make the field more accessible to interested beginners, we not only make a systematic review of existing works and a highly structured typology of prompt-based concepts but also release other resources, e.g., a website NLPedia–Pretrain including constantly updated survey and paperlist.

同方向论文 · cs.AI

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

Replicate Blog76

如何提示 Seedream 5.0

Replicate 团队测试了字节跳动的 Seedream 5.0 图像生成模型,总结了多项关键能力:基于示例的编辑可通过提供前后对比图让模型学习并迁移视觉变换;可理解物理机械结构并生成正确细节;支持多步骤推理和多图像输入;精确遵循复杂指令;具备专业领域知识如建筑可视化、科学插图、营养标注;文本渲染准确且支持多语言;支持批量生成风格一致的多张图像。

推荐理由:原文通过多个实际案例展示了 Seedream 5.0 的核心能力,读者可据此了解模型的实际表现并迁移到自己的使用场景。

关联提示词 · 1