CVAE-guided triage and modular classifiers for multimodal ASD detection.

Abstract

Autism spectrum disorder (ASD) is a heterogeneous neurodevelopmental condition, and diagnosis remains challenging when relevant evidence is distributed across behavioural, neuroimaging, physiological, and visual domains. In this work, we propose a screening-guided modular framework for ASD assessment. A Conditional Variational Autoencoder-guided Quantitative Checklist for Autism in Toddlers (Q-CHAT) module estimates ASD risk under incomplete or noisy questionnaire responses and identifies high-risk cases for further analysis. For such cases, five modality-specific classifiers are designed for: (i) structural and resting-state functional Magnetic Resonance Imaging, (ii) facial expression images, (iii) gastrointestinal endoscopy, (iv) synchronized eye-tracking with functional Near Infrared Spectroscopy (fNIRS), and (v) Activities of Daily Living (ADL) motion signals. In the MRI branch, a 3D CNN learns structural representations, a Graph Convolutional Network (GCN) models-based functional connectivity graphs, a parallel temporal 3D CNN captures resting-state functional Magnetic Resonance Imaging (rs-fMRI) dynamics, and phenotypic metadata are encoded using dense layers before fusion-based classification. Facial assessment uses MediaPipe Face Mesh region masks with an attention-based CNN. The ADL branch applies a BiLSTM with attention to handcrafted temporal features. The GI branch fuses visual embeddings with BioGPT-derived class prompts using cross-attention. The eye-tracking and fNIRS branch uses a dual-branch multilayer perceptron that combines gaze statistics with haemoglobin-based features. Because currently available public datasets do not provide all modalities for the same subjects, the downstream branches are trained and evaluated independently on modality-specific cohorts. Accordingly, the final fusion stage is presented as a conceptual late-fusion decision strategy for future patient-level multimodal deployment rather than a fully validated subject-level multimodal experiment. Across their respective benchmark datasets, the individual branches achieved strong performance, including approximately 99.05% for Q-CHAT screening, 95% for MRI-based assessment, 95% for facial analysis, 96% for eye-tracking with fNIRS, 94% for GI-based classification, and 93% for ADL-based assessment, demonstrating the feasibility of the proposed modular framework as a scalable and clinically relevant blueprint for future multimodal ASD studies.