跳到正文
返回论文列表
cs.GR提交于 已译

GALA: 面向可控物体摆放的几何感知语言模型

GALA: Geometry-Aware Language Model for Controllable Object Arrangement

Chucheng Xiang · Runze Wang · Zhi Deng · Ruchao Bao · Yuanwei Zhang · Yuxuan Xie · et al.

中文摘要

针对精细放置约束(如支撑和包含)的推理对真实 3D 场景创建至关重要,现有依赖有向包围盒(OBB)等简化表示的方法因忽略详细几何结构而面临挑战。GALA 引入显式点云感知的几何感知语言模型,通过自回归方式在场景上下文和放置指令条件下逐个预测目标物体的 6D 位姿。模型采用两阶段训练策略:第一阶段通过预训练任务对齐几何与文本模态,第二阶段针对放置任务进行微调。研究团队构建并开源了涵盖多种房间类型和精细家具布局的多源场景数据集。基准实验表明,GALA 显著优于无显式几何推理的基线模型。此外,开发了多阶段“规划-放置-验证”智能体流水线,利用该模型实现文本驱动的场景生成。

关键要点

  1. 01现有方法依赖有向包围盒(OBB)等简化表示,无法感知详细几何结构,难以处理支撑和包含等精细放置约束。
  2. 02提出 GALA 模型,具备显式点云感知能力,通过自回归方式在场景上下文和指令条件下逐个预测目标物体的 6D 位姿。
  3. 03采用两阶段训练策略,先通过预训练对齐几何与文本模态,再针对放置任务进行微调,并构建开源多源场景数据集。
  4. 04基准实验显示 GALA 显著优于无显式几何推理的基线模型,并开发了多阶段智能体流水线实现文本驱动的场景生成。

解读

尚无解读。

原始英文摘要

arXiv:2610.28280v1 Announce Type: cross Abstract: Reasoning about fine-grained placement constraints, such as support and containment, is essential for realistic 3D scene creation. It remains a significant challenge for existing methods that rely on simplified object representations like oriented bounding box (OBB), which are blind to the detailed geometry. To address this limitation, we introduce GALA, a geometry-aware language model with explicit point cloud perception for controllable object placement. Our model performs autoregressively by predicting the 6D pose of one target object at a time conditioned on the scene context and placement instruction. We utilize a two-stage training strategy: the first stage aligns geometric and textual modalities through pretraining tasks, and the second stage fine-tunes the model for placement. To facilitate this task, we also curate a multi-source scene dataset covering diverse room types and fine-grained furniture arrangements, which will be open-sourced for community benefit. Experiments on our benchmark show our model significantly outperforms baselines without explicit geometric reasoning. We further develop a multi-stage plan-place-verify agent pipeline that uses the model for text-driven scene generation. Project Page: https://xccelephant.github.io/paper/gala/

同方向论文 · cs.GR

查看全部 →