IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 33, pp. 1449-1459, 2025,

A Multimodal Approach for Early Identification of Mild Cognitive Impairment and Alzheimer’s Disease With Fusion Network Using Eye Movements and Speech

Authors: Hasnain Ali Shah; Sami Andberg; Anne M. Koivisto; Roman Bednark
Publication: IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 33, pp. 1449-1459, 2025,
Date: April 15, 2025
Tags: Alzheimer's Disease,Mild Cognitive Impairment,

Abstract

Detecting Alzheimer’s disease (AD) in its earliest stages, particularly during an onset of Mild Cognitive Impairment (MCI), remains challenging due to the overlap of initial symptoms with normal aging processes. Given that no cure exists and current medications only slow the disease’s progression, early identification of at-risk individuals is crucial. The combination of eye-tracking and speech analysis offers a promising diagnostic solution by providing a non-invasive method to examine differences between healthy controls and individuals with MCI, who may progress to develop AD. In this study, we analyzed a multimodal clinical eye-tracking and speech dataset collected from 78 participants (37 controls, 20 MCI, and 21 AD) during the King-Devick test and a reading task to classify and diagnose MCI/AD versus healthy controls. To that end, we developed a Fusion Neural Network, a deep learning-based classification model that integrates gaze and speech-derived features, including pupil size variations, fixation duration, saccadic movements, and speech delay, to improve MCI diagnosis performance. We achieved an average classification accuracy of 79.2% for MCI diagnosis and 82% for AD. Our findings indicate that features related to pupil size and eye-speech temporal dynamics are strong indicators for detection tasks. Moreover, the results indicate that using multimodal data (gaze + speech) significantly improves classification accuracy compared to unimodal data from speech or gaze alone.

Summary Points:

  • We analyzed multimodal data comprising eye movements and speech from 78 participants (37 controls, 20 MCI, and 21 AD) using both the KD test and a newly introduced reading task to investigate key features related to MCI/AD that the KD test might have overlooked.
  •  

    Our findings show that AD exhibits reduced reading speed and a higher error rate relative to both controls and MCI subjects. The KD test provides robust differentiation in both timing and error measures, particularly between the AD and control groups.

Read More