Reddit · r/MachineLearning· /u/AlarmingTrouble5261·· 7 小时前AI 评分16
15M 参数决策模型对比：线性循环与 Transformer 注意力在长上下文下的成本权衡
I trained a 15M "decision-only" model two ways — linear recurrence vs Transformer attention. Accuracy was a coin flip; latency and training cost weren't.[R]
AI 导读
作者在 Colab T4 上用相同预算训练了两个约 15.09M 参数的"仅做决策"模型，分别采用线性循环（Gated Segmented State Space）骨干和标准 Transformer 注意力骨干。
来源：Reddit · r/MachineLearning · reddit.com