MemoCare:面向自动化认知筛查的交互式多模态移动系统
MemoCare: An Interactive Multimodal Mobile System for Automated Cognitive Screening
Duy-Cat Can · Mau Minh Phuc Le · Tuan-Khoa Hoang · Hai-Dang Nguyen · Trung-Hieu Do · Dang Minh Ly · et al.
中文摘要
MemoCare 是一个用于自动化多模态(multi-modal)认知筛查的交互式移动系统。基于 React Native 开发的应用在完整的英语和越南语工作流中整合了语音应答、时间与空间定向、触屏动作以及视觉构建(visuoconstruction)四类任务。语音经由 Google Speech-to-Text 转写,并由本地确定性任务专属自然语言处理规则进行评分;GPS 坐标由 MemoCare 空间模块解析后与答案进行匹配;触屏任务依据交互事件评分;绘画任务则采用三模型卷积神经网络(convolutional neural network,CNN)集成共识,并辅以独立的视觉解读。软件测试中,语音/语言、空间作答、触屏交互评分三类共 151 项预定义用例全部通过(151/151),空间回归在四国坐标解析用例上通过 48/48。绘画模块中,经验证筛选的 ShuffleNetV2 x1.5 在锁定的 71 张测试集上取得 91.33% 的平均平衡准确率与 78.87% 的三项准则完全准确率。四位临床医生共同作者对端到端工作流进行了检视,在八项标准上整体中位数评分为 4/5,各题项中位数介于 3 至 4.5 之间。在 MMM 会议现场,参会者可直接试用精简版多模态筛查工作流,并查看自动生成的题项级与总分评分。
关键要点
- 01问题:传统认知筛查依赖单一模态,缺乏面向英语与越南语双语的移动端自动化多模态筛查工具。
- 02方法:基于 React Native 构建 MemoCare 系统,集成语音(GCP 转写 + 本地确定性 NLP 规则)、GPS 空间定向、触屏交互与绘画任务,绘画模块采用三模型 CNN 共识结合独立视觉解读。
- 03结果:软件测试 151/151 通过,空间回归 48/48 通过;绘画模块 ShuffleNetV2 x1.5 在 71 张锁定测试集上达到 91.33% 平均平衡准确率。
- 04局限:临床评估仅由四位共同作者在八项标准上给出 3-4.5 中位数评分,样本量小且存在利益相关偏差;测试图像集规模仅 71 张。
- 05应用:在 MMM 会议提供可交互的精简版多模态筛查流程供与会者试用并查看自动评分。
解读
尚无解读。
原始英文摘要
arXiv:2610.10448v1 Announce Type: cross Abstract: MemoCare is an interactive mobile system for automated multimodal cognitive screening. A React Native application combines spoken responses, temporal and spatial orientation, touchscreen actions, and visuoconstruction in complete English and Vietnamese workflows. Speech is transcribed by Google Speech-to-Text and scored locally with deterministic task-specific natural language processing rules; GPS coordinates are resolved by the MemoCare spatial module before answer matching; touch tasks are scored from interaction events; and the drawing task uses a three-model convolutional neural network consensus with separate visual interpretation. Software tests pass 151/151 predefined cases across speech/language, spatial-answer, and touch-interaction scoring, while spatial regression passes 48/48 four-country coordinate-resolution cases. For the drawing module, validation-selected ShuffleNetV2 x1.5 achieved 91.33% mean balanced accuracy and 78.87% exact three-criterion accuracy on a locked 71-image test set. Four clinician co-authors additionally inspected the end-to-end workflow, yielding a pooled median rating of 4/5 across eight criteria, with item-level medians ranging from 3 to 4.5. At MMM, attendees can directly try a shortened multimodal screening workflow and inspect automatic item-level and total scoring.