微软 CARE-X:结合辅助监督、对齐强化学习和工具增强测量的临床可用胸部 X 光视觉语言模型
微软研究院发布 CARE-X,一个统一胸部 X 光 VLM,结合生成和判别能力,使用 DAPO 强化学习进行临床对齐优化,并配备辅助分类头和接地头提供校准概率和空间定位。
推荐理由:强化学习与辅助监督的结合在医学影像 VLM 中展现了临床价值,工具增强测量在需要定量诊断的条件上提升了平均 F1 达 43.6 个百分点。
Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond
Zhiyuan Ma · Jiabao Wei · Tianyu Wang · Hanqing Wang · Jiadong Pan · Haowen Xu · et al.
扩散模型(DMs)在“预训练-微调”范式下展现出高保真生成能力,但在复杂推理、自主探索及对齐人类目标方面存在局限。受 RLHF 启发,强化学习(RL)成为通过奖励驱动轨迹优化迭代改进 DMs 的关键后训练机制。该综述首次全面梳理 RL 驱动的 DMs,分析其如何重塑扩散过程以实现以经验为中心的目标导向生成。文章构建了涵盖语言、多模态、机器人及科学模型的技术全景,重点探讨扩散轨迹优化与奖励工程。通过多任务基准实验,系统评估代表性方法以展示不同 RL 策略的优势。研究揭示了阻碍实际部署的三大瓶颈:奖励设计脆弱性、高维生成空间中的探索-利用权衡,以及稀疏奖励下的训练不稳定性。最后,提供了将 RL 集成到 DMs 的原则性路线图,并展望了构建鲁棒、可解释的 RL-扩散系统的未来方向。
尚无解读。
arXiv:2610.21539v1 Announce Type: new Abstract: Diffusion models (DMs) have steadily demonstrated outstanding performance in high-fidelity, controllable generation across diverse scenarios, following the "pretraining-finetuning" paradigm established in LLMs. Despite these advances, conventional DMs still face critical limitations in sophisticated reasoning, autonomous exploration of generation pathways, and adaptability to complex, human-aligned objectives. Inspired by RLHF's success in LLMs, RL has recently emerged as a pivotal post-training mechanism to iteratively refine DMs through reward-driven trajectory optimization. This paper presents the first comprehensive review of RL-driven DMs and an empirical analysis of how RL reshapes the diffusion process towards experience-centric, goal-directed generation. We propose a novel technical panorama elucidating RL's multifaceted roles in diffusion-based language, multimodal, robotic, and scientific models, with a granular focus on diffusion trajectory optimization and reward engineering. Through extensive experiments on multiple task-specific benchmarks, we systematically evaluate representative methods to intuitively highlight the advantages of different RL post-training strategies. Furthermore, we reveal and dissect three crucial bottlenecks that currently hinder the real-world deployment of RL-driven DMs: reward design fragility, exploration-exploitation trade-off in high-dimensional generation spaces, and training instability under sparse rewards. Based on these perspectives, we provide a principled roadmap for the integration of RL into DMs and outline promising future research directions for robust, interpretable RL-diffusion systems. We also construct a public repository to continuously track the latest developments in this rapidly evolving field: https://github.com/MAIR-Lab-HUST/Awesome-RL-for-Diffusion-Models.
微软研究院发布 CARE-X,一个统一胸部 X 光 VLM,结合生成和判别能力,使用 DAPO 强化学习进行临床对齐优化,并配备辅助分类头和接地头提供校准概率和空间定位。
推荐理由:强化学习与辅助监督的结合在医学影像 VLM 中展现了临床价值,工具增强测量在需要定量诊断的条件上提升了平均 F1 达 43.6 个百分点。