跳到正文
返回论文列表
cs.CV提交于 已译

从评估到进化:图像编辑基准测试与图像模型演化

From Evaluation to Evolution: Benchmarking Image Editing and Evolving Image Models

Jun He · Canmiao Fu · Ying Zhang · Zhipeng Huang · Chenyu Mu · Yazhen Xie · et al.

中文摘要

真实世界图像编辑需协调多种视觉变换,同时保持内容并满足参考约束,但聚合分数容易掩盖未满足的局部要求。有效评估须识别局部失败,并将其转化为后续模型改进的优先项。提出 WeGenEditBench,包含 2,740 个评估实例和 148 种任务类型,覆盖单图、文本密集和多参考编辑。跨模型共享的预生成检查表将每个请求分解为原子要求;先建立条目级扣分和视觉证据,再汇总为编辑执行、保持、视觉质量和任务特定正确性分数。为支持递归自我改进(RSI)的经验获取环节,开发 WeGenEvoStudio,一个面向 RSI 学习的统一多智能体框架,集成模型条件失败诊断、检索接地数据构建、监督适配和再评估。在 15 个评估配置中,GPT-Image-2 以 Overall 4.076 领先单图编辑,Seedream-5.0-Pro 以 4.291 领先文本密集编辑。跨四个生成与编辑基准的进化实验显示显著定向增益:在报告选定的 Top-5 单图目标上,FLUX.2 [dev] 的 Overall 从 1.083 提升至 3.517,Qwen-Image-Edit-2511 提升 13.2%。该基准与框架为通过可解释评估和反馈驱动学习开发更可靠图像模型提供了基础。

关键要点

  1. 01聚合分数掩盖局部失败,需条目级评估识别未满足要求
  2. 02WeGenEditBench 含 2,740 实例、148 任务类型,覆盖单图、文本密集和多参考编辑
  3. 03GPT-Image-2 单图编辑 Overall 4.076 领先,Seedream-5.0-Pro 文本密集编辑 4.291 领先
  4. 04WeGenEvoStudio 集成失败诊断、数据构建、监督适配与再评估,支持递归自我改进
  5. 05FLUX.2 [dev] 在 Top-5 单图目标上 Overall 从 1.083 升至 3.517,Qwen-Image-Edit-2511 提升 13.2%

解读

尚无解读。

原始英文摘要

arXiv:2610.41210v1 Announce Type: new Abstract: Real-world image editing requires coordinating multiple visual transformations with content preservation and reference constraints, yet aggregate scores can readily obscure unmet local requirements. Effective evaluation must therefore identify local failures and translate them into priorities for subsequent model improvement. We introduce WeGenEditBench, comprising 2,740 evaluation instances and 148 task types across single-image, text-intensive, and multi-reference editing. Pre-generated checklists shared across models decompose each request into atomic requirements; item-level deductions and visual evidence are established first and then summarized into scores for edit execution, preservation, visual quality, and task-specific correctness. To support the experience-acquisition component of recursive self-improvement (RSI), we develop WeGenEvoStudio, a unified multi-agent harness for RSI-oriented learning that integrates model-conditioned failure diagnosis, retrieval-grounded data construction, supervised adaptation, and re-evaluation. Among 15 evaluated configurations, GPT-Image-2 leads single-image editing with an Overall score of 4.076, while Seedream-5.0-Pro leads text-intensive editing at 4.291. Evolution experiments across four generation and editing benchmarks demonstrate substantial targeted gains: on their report-selected Top-5 single-image targets, FLUX.2 [dev] improves Overall from 1.083 to 3.517, while Qwen-Image-Edit-2511 gains 13.2%. Together, our benchmark and harness provide a foundation for developing more reliable image models through interpretable evaluation and feedback-driven learning.

同方向论文 · cs.CV

查看全部 →

反向发现 · 这篇论文相关的工具与提示词

去交叉发现 →

关联工具 · 1

量子位 QbitAI78

字节Seedream 5.5实测:100轮图像编辑不崩,API单张约0.28元

字节跳动推出图像生成模型Seedream 5.5,已在即梦、豆包和豆包工作上线。量子位实测显示,该模型在100轮连续编辑任务中比GPT-Image 2.6更保真,并支持多图融合、小字与多语言渲染、透明图生成及图层拆分编辑。据官方信息,其API延续输入图加输出图两段计费,2K方图编辑单张成本约0.28元,不到Seedream 5.0 Pro一半。

推荐理由:通过100轮连续编辑、多图合成、图层拆分等实测,呈现Seedream 5.5在编辑稳定性上的具体表现与API价格变化。