Multimodal Fusion With Reference: Searching for Joint Neuromakers of Working Memory Deficits in Schizophrenia
Author Information:
Shile Qi1,2, Vince D. Calhoun3,4,5, Theo G. M. van Erp6, Juan Bustillo6, Eswar Damaraju3, Jessica A. Turner3,6, Yuhui Du3, Jian Yang7, Jiayu Chen3, Qingbao Yu3, Daniel H. Mathalon8,9, Judith M. Ford8,9, James Voyvodic10, Bryon A. Mueller11, Aysenil Belger12, Sarah McEwen13, Steven G. Potkin6, Adrian Preda6, Tianzi Jiang1,2,14 and Jing Sui*1,2,3,14
1Brainnetome Center and National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
2University of Chinese Academy of Sciences, Beijing, China
3Mind Research Network, Albuquerque, NM, USA
4Department of Psychiatry, University of New Mexico, Albuquerque, NM, USA
5Department of ECE, University of New Mexico, Albuquerque, NM, USA
6Department of Psychiatry and Human Behavior, University of California Irvine, Irvine, CA, USA
7Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Electronics, Beijing Institute of Technology, Beijing, China
8Department of Psychiatry, University of California, San Francisco, San Francisco, CA, USA
9San Francisco VA Medical Center, San Francisco, CA, USA
10Department of Radiology, Brain Imaging and Analysis Center, Duke University, Durham, NC, USA
11Department of Psychiatry, University of Minnesota, Minneapolis, MN, USA
12Department of Psychiatry, University of North Carolina School of Medicine, Chapel Hill, NC, USA
13Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA
14CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China
Abstract:
By exploiting cross-information among multiple imaging data, multimodal fusion has often been used to better understand brain diseases. However,most current fusion approaches are blind, without adopting any priorinformation. There is increasing interest to uncover the neurocognitive mapping of specific clinical measurements on enriched brain imaging data; hence, a supervised, goaldirected model that employs prior information as a reference to guide multimodal data fusion is much needed and becomes a natural option. Here, we proposed a fusion with reference model called “multi-site canonical correlation analysis with reference + joint-independent component analysis” (MCCAR+jICA), which can precisely identify co-varying multimodal imaging patterns closely related to the reference, such as cognitive scores. In a three-way fusion simulation, the proposedmethodwas comparedwith its alternatives on multiple facets; MCCAR+jICA outperforms others with higher estimation precision and high accuracy on identifying a target component with the right correspondence. In human imaging data, working memory performance was utilized as a reference to investigate the co-varying working memory-associated brain patterns among three modalities and how they are impaired in schizophrenia. Two independent cohorts (294 and 83 subjects respectively) were used. Similar brain maps were identified between the two cohorts along with substantial overlaps in the central executive network in fMRI, salience network in sMRI, and major white matter tracts in dMRI. These regions have been linked with working memory deficits in schizophrenia in multiple reports and MCCAR+jICA further verified them in a repeatable, joint manner, demonstrating the ability of the proposed method to identify potential neuromarkers for mental disorders.
Keywords:Multimodal fusion with reference, MCCAR, supervised learning, brain modeling, psychiatry, working memory, ICA, MCCB, CMINDS
