无服务器架构与计费下的CPU语音合成极限优化
Pushing CPU Speech Synthesis to the Wall: Extreme Inference Tuning under Serverless Architecture and Billing
Pakorn Nathong · Kunat Pipatanakul
中文摘要
实例计费的无服务器平台对暖实例生命周期内的CPU和内存收费,使空闲推理状态成为直接的服务成本。我们提出在无服务器CPU上进行计费感知的神经文本到语音(TTS)服务,优化CPU秒和GB秒而非仅优化吞吐量或延迟。传统运行时在此场景下表现不佳:每请求并行在并发下导致CPU争用,而暖实例保留着数GB的可计费推理状态和页缓存。我们通过请求级并发推理解决这些成本问题,它可限制每请求CPU并行度;并通过可回收实例生命周期,在空闲期释放推理状态和页缓存内存,同时保留服务器进程和编译缓存。在Kokoro-82M上,我们的系统实现每CPU秒2.71音频秒,而ONNX Runtime默认仅为0.89;每音频小时成本从PyTorch的0.0631美元降至0.0153美元,降低4.1倍。空闲计费内存从8.7GB降至1.33GB,而恢复到首音频仅需2.2s,PyTorch冷启动需7.7s。在突发流量下,生命周期回收对于将推理效率转化为更低的serverless成本至关重要。
关键要点
- 01问题:实例计费无服务器平台对CPU和内存按暖实例生命周期收费,空闲推理状态成为直接成本,传统运行时在并发下有CPU争用问题
- 02方法:请求级并发推理限制每请求CPU并行度,可回收实例生命周期在空闲期释放推理和页缓存内存但保留进程和编译缓存
- 03结果: Kokoro-82M上实现每CPU秒2.71音频秒,相比ONNX Runtime默认提升3倍,每音频小时成本从$0.0631降至$0.0153,降低4.1倍
- 04结果:空闲计费内存从8.7GB降至1.33GB,恢复时间从PyTorch冷启动7.7s降至2.2s
- 05局限:生命周期回收对突发流量下将推理效率转化为更低serverless成本至关重要,但未提及在更低配置或更大模型上的表现
解读
尚无解读。
原始英文摘要
arXiv:2610.00063v1 Announce Type: cross Abstract: Instance-billed serverless platforms charge for CPU and memory over the lifetime of a warm instance, making idle inference state a direct serving cost. We present billing-aware neural text-to-speech (TTS) serving on serverless CPUs, optimizing CPU-seconds and GB-seconds rather than throughput or latency alone. Conventional runtimes are poorly suited to this setting: per-request parallelism causes CPU contention under concurrency, while warm instances retain gigabytes of billable inference and page-cache state. We address these costs with request-sized concurrent inference, which bounds per-request CPU parallelism, and a reclaimable instance lifecycle, which releases inference state and page-cache memory after idle periods while retaining the server process and compile cache. On Kokoro-82M, our system achieves 2.71 audio-seconds per CPU-second versus 0.89 with ONNX Runtime defaults and reduces cost per audio-hour from $0.0631 with PyTorch to $0.0153, a 4.1x reduction. Idle billed memory falls from 8.7 GB to 1.33 GB, while restoration reaches first audio in 2.2 s versus 7.7 s for a PyTorch cold start. Under bursty traffic, lifecycle reclamation is essential for translating inference efficiency into lower serverless cost.