跳到正文
返回论文列表
cs.LG提交于 已译

提示词风格如何影响 AI 在竞赛编程中的准确率与速度

How Prompt Style Affects AI Accuracy and Speed in Competitive Programming

Saad B. Alarifi · Mohammed A. Alam

中文摘要

一项探索性研究在 GPT-5.5 上测试了 9 种风格的提示词,任务包括 3 项 International Olympiad in Informatics (IOI) 准确率任务和 2 项 NP-hard 速度任务。结果显示,没有一种提示词始终最佳:最优选择取决于任务类型,以及结果采用均值还是中位数汇总。平均分差异常由少数离群运行造成,仅用单一指标评估可能掩盖效果波动。与在不同条件下参赛的人类 IOI 选手作描述性比较时,AI 在一项任务中超过平均水平,但另外两项低于平均水平。研究将提示词比较从准确率扩展到运行速度,并置于人类表现的参照下。

关键要点

  1. 01既有研究主要比较提示词的准确率,未评估运行速度,也缺少人类表现参照。
  2. 02研究在 GPT-5.5 上比较 9 种提示词风格,涵盖 3 项 IOI 准确率任务和 2 项 NP-hard 速度任务。
  3. 03没有一种提示词始终最佳;选择取决于任务类型以及采用均值或中位数汇总。
  4. 04平均分差异常来自少数离群运行,单一指标可能掩盖提示词效果的实际波动。
  5. 05与不同条件下参赛的人类 IOI 选手相比,AI 一项任务高于平均水平,另两项低于平均水平。

解读

尚无解读。

原始英文摘要

arXiv:2610.30223v1 Announce Type: new Abstract: Large language models are increasingly used in competitive programming, and their outputs are sensitive to how a prompt is phrased. Prior work has compared prompt strategies for accuracy on competitive programming, but not for runtime speed, and has not situated the results against human performance. In an exploratory study of one prompt in each of nine styles on GPT-5.5 across three accuracy tasks from the International Olympiad in Informatics (IOI) and two NP-hard speed tasks, we found that no single prompt was best: the strongest prompt depended on the task type and on whether results were summarized by the mean or the median, suggesting that, at least for these tasks, prompt style should be matched to the objective rather than treated as universal. Many differences in mean scores came from a few outlier runs, showing that evaluating prompting by a single metric can obscure how much its effectiveness varies. Compared descriptively with human IOI contestants, who worked under different conditions, the AI was neither strictly better nor strictly worse, exceeding the average contestant on one task but falling short on the other two. By extending prompt comparison from accuracy to runtime speed and grounding it against human performance, this study gives a broader picture of how prompt style affects competitive programming performance.

同方向论文 · cs.LG

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 6

编程github.com/PlexPt/awesome-chatgpt-prompts-zh·外部收录·

充当前端智能思路助手

我想让你充当前端开发专家。我将提供一些关于Js、Node等前端代码问题的具体信息,而你的工作就是想出为我解决问题的策略。这可能包括建议代码、代码逻辑思路策略。我的第一个请求是“我需要能够动态监听某个元素节点距离当前电脑设备屏幕的左上角的X和Y轴,通过拖拽移动位置浏览器窗口和改变大小浏览器窗口。”

编程github.com/f/awesome-chatgpt-prompts·外部收录·

Corporate Website Refactor and Finalization

Act as an elite full-stack developer and UI/UX expert. Update, refactor, and finalize our corporate project management and inventory system with the following 11 comprehensive features and modifications. Ensure absolute state management usi…

编程github.com/f/awesome-chatgpt-prompts·外部收录·

Senior Python Code Auditor (Spanish Output)

Act as a Senior Software Architect and Python expert. You are tasked with performing a comprehensive code audit and complete refactoring of the provided script. Your instructions are as follows: ### Critical Mindset - Be extremely critica…

编程github.com/f/awesome-chatgpt-prompts·外部收录·

Code Review Assistant for Bug Detection

Act as a Code Review Assistant. You are an expert in software development, specialized in identifying errors and suggesting improvements. Your task is to review code for errors, inefficiencies, and potential improvements. You will: - Analy…

编程github.com/f/awesome-chatgpt-prompts·外部收录·

Code oprimisation / optimise your code

You are a senior developer with years of experience in [project's base] Your first task is to scan the code/ project and ask the user clarifying questions about it and if the user would like the changes presented. # Task Optimize the given…