基于多模型检索融合与交叉编码器重排序的合成到真实文本人员异常搜索
Synthetic-to-Real Text-Based Person Anomaly Search via Multi-Model Retrieval Fusion and Cross-Encoder Reranking
Eugene Denteh · Blessing Agyei Kyem · Andrews Danyo · Joshua Kofi Asamoah · Richard Dzinyela · Solomon Owusu-Ansah · et al.
中文摘要
文本人员异常搜索要求模型根据人物外观与行为的自然语言描述，从大型图像库中检索对应人员。由于训练依赖合成图文对，而评估使用真实监控图像，任务面临域差异、近重复帧及大量视觉相似干扰项等挑战。研究提出多模型检索与重排序框架，以改善合成到真实的迁移能力。首先通过双向对比学习、显式负采样、假负样本掩码和域随机化，使 Qwen3-VL-Embedding-8B 适应合成人物描述；随后结合四个视角的图库编码，融合六个互补 Qwen3-VL 检查点与冻结 SigLIP2 模型的归一化分数。融合检索分数筛选候选，并使用 Qwen3-VL-Reranker-8B 联合评估查询与候选，再混合交叉编码器及检索分数生成最终排序。还构建了完全使用合成数据的干扰项匹配验证协议，用于选择检查点权重、融合系数、候选深度及重排序参数，而不访问官方测试图像。最终系统在 AI City Challenge 2026 Track 4 官方测试设置下取得 94.7815% mAP 和 90.8493% R@1，并位列公开排行榜第八名。
关键要点
- 01合成训练与真实监控评估之间存在域差异和相似干扰项挑战
- 02以双向对比学习、负采样和域随机化适配人物描述
- 03融合六个 Qwen3-VL 检查点与 SigLIP2 的四视角检索分数
- 04利用 Qwen3-VL-Reranker-8B 对候选进行交叉编码器重排序
- 05纯合成验证协议避免访问测试图像并完成参数选择
解读
尚无解读。
原始英文摘要
arXiv:2610.31930v1 Announce Type: new Abstract: Text-based person anomaly search requires a model to retrieve individuals from a large image gallery using natural-language descriptions of both appearance and behavior. This task is challenging because model training relies on synthetic image-text pairs, whereas evaluation is performed on real surveillance images containing domain differences, near-duplicate frames, and many visually similar distractors. We present a multi-model retrieval and reranking framework designed to improve synthetic-to-real transfer. The framework first adapts Qwen3-VL-Embedding-8B to synthetic person descriptions using bidirectional contrastive learning, explicit negative sampling, false-negative masking, and domain randomization. It then combines the normalized scores of six complementary Qwen3-VL checkpoints with those of a frozen SigLIP2 model using four-view gallery encoding. The fused retrieval scores identify the strongest candidates, which are jointly evaluated with the query using Qwen3-VL-Reranker-8B. The cross-encoder and retrieval scores are subsequently blended to produce the final ranking. We also construct a distractor-matched validation protocol entirely from synthetic data to select checkpoint weights, fusion coefficients, candidate depths, and reranking parameters without accessing the official test images. The final system achieves 94.7815% mAP and 90.8493% R@1 under the official AI City Challenge 2026 Track 4 test setting, with a reported eighth-place finish on the public leaderboard.