跳到正文
返回论文列表
cs.CY提交于 已译

从效率到充分性:面向自动作文评分的成本感知测量设计框架

From Efficiency to Sufficiency: The Cost-Aware Measurement Design Framework for Automated Essay Scoring

Yi Gui

中文摘要

自动作文评分(AES)中的计算成本通常被视为技术效率问题,而非测量设计关切。然而,部署决策需要超越延迟、模型大小或推理费用的报告。在高风险场景中,总负担包括验证、监控、仲裁、合规和治理;在教学场景中,定价、许可和集成可能主导成本。该研究将AES重构为成本约束的测量系统,并引入成本感知测量设计(CAMD)框架。这是一个包含计量、分配、单位经济学和决策的四层结构,用于将计算和工作流资源记录转化为决策相关的成本结构。直接成本被指定为货币支出加上可归因的劳动和时间,设计归因和可审计性两个标准决定了成本边界排除的内容。不满足这两个标准的负担(如机构能力和治理姿态)被表示为对可接受设计的约束,而非被定价。该研究定义了每个可辩护分数的成本(CPDS),并主张AES系统应按质量-成本充分性进行比较:在满足测量要求的可辩护评分设计中最小化成本。评分量级决定了所有四层,因此CPDS被视为规模的函数,而非单一可转移数值。CAMD为跨AES研究和部署环境表示、比较和评估成本提供了可移植的基础。

关键要点

  1. 01指出AES成本常被误判为纯技术效率问题,忽略了验证、合规及集成等部署决策中的关键负担。
  2. 02提出CAMD框架,通过计量、分配、单位经济学和决策四层结构,将资源记录转化为决策相关的成本构造。
  3. 03定义直接成本为货币支出加可归因劳动时间,并依据设计归因和可审计性标准划定成本边界。
  4. 04引入每个可辩护分数的成本(CPDS)指标,主张在满足测量要求的前提下最小化成本以实现质量-成本充分性。
  5. 05强调评分量级决定所有成本层级,CPDS应视为规模的函数而非单一可转移数值,以支持跨场景比较。

解读

尚无解读。

原始英文摘要

arXiv:2610.19287v1 Announce Type: cross Abstract: Compute cost in automated essay scoring (AES) is usually treated as a technical efficiency issue rather than a measurement design concern. Yet deployment decisions require more than reporting latency, model size, or inference expense. In high-stakes settings, total burden includes validation, monitoring, adjudication, compliance, and governance; in instructional settings, pricing, licensing, and integration may dominate cost. This paper reframes AES as a cost-constrained measurement system and introduces the Cost-Aware Measurement Design (CAMD) Framework, a four-layer structure consisting of metering, allocation, unit economics, and decision, for translating computational and workflow resource records into decision-relevant cost constructs. Direct cost is specified as monetary outlay plus attributable labor and time, and two criteria, design attribution and auditability, determine what the cost boundary excludes. Burdens that satisfy neither criterion, such as institutional capability and governance posture, are represented as constraints on which designs are admissible rather than priced. The paper defines cost per defensible score (CPDS) and argues that AES systems should be compared by quality-cost sufficiency: minimizing cost among defensible scoring designs that satisfy measurement requirements. Scoring volume conditions all four layers, so cost per defensible score is treated as a function of scale rather than as a single transferable number. CAMD provides a portable basis for representing, comparing, and evaluating cost across AES studies and deployment contexts.

同方向论文 · cs.CY

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联提示词 · 3

其它github.com/PlexPt/awesome-chatgpt-prompts-zh·外部收录·

担任心理健康顾问

我想让你担任心理健康顾问。我将为您提供一个寻求指导和建议的人,以管理他们的情绪、压力、焦虑和其他心理健康问题。您应该利用您的认知行为疗法、冥想技巧、正念练习和其他治疗方法的知识来制定个人可以实施的策略,以改善他们的整体健康状况。我的第一个请求是“我需要一个可以帮助我控制抑郁症状的人。”

其它github.com/f/awesome-chatgpt-prompts·外部收录·

Tech-Challenged Customer

Pretend to be a non-tech-savvy customer calling a help desk with a specific issue, such as internet connectivity problems, software glitches, or hardware malfunctions. As the customer, ask questions and describe your problem in detail. Your…

其它github.com/f/awesome-chatgpt-prompts·外部收录·

Yogi

I want you to act as a yogi. You will be able to guide students through safe and effective poses, create personalized sequences that fit the needs of each individual, lead meditation sessions and relaxation techniques, foster an atmosphere …