cs.CL提交于 2026-11-04已译

LLM 智能体能否成为前向部署工程师？企业级 Text-to-SQL 的需求发现与交付

Can LLM Agents Be Forward-Deployed Engineers? Requirement Discovery and Delivery for Enterprise Text-to-SQL

Marianne Menglin Liu · Rongguang Wang · Zheng Wang · Yewen Wang · Leonid Boytsov · Tao Sheng · et al.

中文摘要

LLM 编码智能体已能逐步自动化模型后训练，但其能否充当前向部署工程师（FDE）智能体、发现客户需求并转化为部署决策仍不明确。研究基于真实企业 text-to-SQL 项目：模拟客户以真实用例需求为基础，从模糊请求出发。智能体需在发现阶段识别验收标准、数据权限与硬件约束，在交付阶段规划并执行后训练方案。在 15 个模型中，最强模型发现超过 80% 的隐藏需求，开放权重模型发现 40-50%；交付阶段，八种编码智能体配置中仅 25% 的运行通过全部验收标准。识别出四个瓶颈：(1) 发现不完整，40% 隐藏需求缺失或表述错误；(2) 执行不可靠；(3) 计算预算利用不足，缺乏跨基座模型与后训练方法的系统比较；(4) benchmark 准确率不保证生产就绪。建议人工审查发现的需求、监控执行中的智能体计划，并明确实验环境规范，涵盖硬件、预算范围与生产检查。

关键要点

01研究 LLM 智能体能否承担前向部署工程师角色，完成企业 text-to-SQL 需求发现与交付
02最强模型发现超 80% 隐藏需求，开放权重模型仅 40-50%，40% 需求缺失或表述错误
03八种编码智能体配置中仅 25% 运行通过全部验收标准，执行可靠性不足
04计算预算利用不足，缺乏基座模型与后训练方法的系统比较
05benchmark 准确率不保证生产就绪，建议人工审查需求并监控执行计划

解读

尚无解读。

原始英文摘要

arXiv:2611.03551v1 Announce Type: new Abstract: LLM coding agents can increasingly automate model post-training, but whether they can act as forward-deployed engineer (FDE) agents, discovering customer requirements and turning them into deployment decisions, remains unclear. We study this on a real enterprise text-to-SQL engagement: a simulated customer, grounded in the real use case's requirements, starts with an ambiguous request. An agent must discover acceptance criteria, data permissions, and hardware constraints in the discovery phase, then plan and execute a post-training recipe in the delivery phase. Across 15 models, the strongest discover over 80% of withheld requirements and open-weight models discover 40-50%; in the delivery phase, only 25% of runs across eight coding-agent configurations pass all acceptance criteria. We identify four bottlenecks in this engagement: (1) incomplete discovery, with 40% of withheld requirements missing or misrepresented; (2) unreliable execution; (3) underused compute budgets and a lack of systematic comparisons across base models and post-training methods; and (4) benchmark accuracy does not guarantee production readiness. We suggest human review of discovered requirements, monitoring of the agent's plan during execution, and an explicit experiment-environment specification covering hardware, budget scope, and production checks.