cs.LG提交于 2026-10-08已译

从 Prompt 到树:面向少样本表格分类的 LLM 引导决策树生成

From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

Yue Qiu · Zekang Du · Yiqun Diao · Bingsheng He · Qinbin Li

中文摘要

大语言模型(LLM)具备丰富的世界知识与出色的泛化能力,但直接将其应用于表格数据分类时,受限于高昂的推理成本和较弱的可解释性。决策树推理快、透明性好,却在低数据场景下表现欠佳。该工作提出一种新框架,在少样本学习设定下,将 LLM 的知识蒸馏到可解释的决策树中,从而连接这两种范式。区别于直接提示 LLM 生成完整决策树(这种方式往往不稳定且低效),该方法采用三阶段范式,先提示 LLM 生成规则,再将这些规则组织成决策树。在多个真实表格数据集上的实验表明,相比现有基线,该方法在显著降低 Prompt 开销的同时,取得了更高的准确性与更好的可解释性。

关键要点

01问题定义:LLM 直接用于表格分类推理成本高、可解释性差,而决策树虽快且可解释却在少样本下表现不足
02方法:采用三阶段范式,先用 Prompt 让 LLM 生成规则,再将规则组织成可解释的决策树,实现知识蒸馏
03结果:在多个真实表格数据集上,该方法以更低 Prompt 开销获得比现有基线更高的准确性和可解释性
04局限:摘要未讨论具体失败案例或适用边界

解读

尚无解读。

原始英文摘要

arXiv:2610.10227v1 Announce Type: new Abstract: While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs and limited interpretability. In contrast, decision trees are fast and transparent but often underperform in low-data regimes. In this work, we propose a novel framework that bridges these paradigms by distilling LLM knowledge into interpretable decision trees under a few-shot learning setting. Instead of directly prompting the LLM to generate full trees, which is often unstable and inefficient, we develop a three-stage paradigm that prompts the LLM to generate rules and organize the rules into a tree. Experiments on multiple real-world tabular datasets demonstrate that our method achieves superior accuracy and interpretability with significantly lower prompting overhead compared to existing baselines.