cs.CL提交于 2026-10-23已译

IQuest-Q1：提升智能体 CLI 系统的基础能力

IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems

A. Jian Yang · An Huang · Bo Dan · Chen Mu · Chengran Yang · Chuan Shi · et al.

中文摘要

IQuest-Q1 是一种稀疏混合专家(Mixture-of-Experts, MoE)智能体基础模型，总参数量为 320B，激活参数量为 15B。训练分为三阶段：首先进行稳定高效预训练，结合 Stable SplitNorm、Muon 与 muP，并采用提高编码数据比例的两阶段课程，同时保持安全与最优学习率之间较宽的稳定裕量；其次开展细粒度中期训练，扩展领域与格式覆盖范围，并逐步延长上下文；最后进行多脚手架、多版本后训练，跨不同脚手架整合 SFT、领域专用 RL 和 MOPD，融合互补训练信号以增强智能体编码能力。此外，通过模型合并融合不同专家及各训练阶段的检查点，同时缓解参数更新冲突。最终模型具备编码与通用任务能力，并以开源形式发布，以支持社区研究和开发。

关键要点

01构建 320B 总参数、15B 激活参数的稀疏 MoE 智能体基础模型
02采用两阶段课程进行稳定高效预训练，并提高编码数据占比
03通过细粒度中期训练扩展领域、格式及上下文长度
04结合 SFT、领域 RL 与 MOPD 进行多脚手架后训练
05利用模型合并融合专家与检查点，并缓解参数更新冲突

解读

尚无解读。

原始英文摘要

arXiv:2610.30949v1 Announce Type: new Abstract: We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (1) stable and efficient pre-training, combining Stable SplitNorm, Muon, and muP with a two-stage curriculum whose second stage increases the coding-data mixture, preserving a wide stability margin between safe and optimal learning rates; (2) fine-grained mid-training, expanding the data mixture across domains and formats and progressively extending the context; and (3) multi-scaffold with multi-version post-training, combining SFT, domain-specialized RL, and MOPD across various scaffolds to integrate complementary training signals and strengthen agentic coding capabilities. Furthermore, model merging combines different experts and checkpoints from different stages while mitigating conflicts among their parameter updates. The resulting IQuest-Q1 demonstrates capabilities in both coding and general tasks. We release IQuest-Q1 as an open-source model to support community research and development.