Advanced Signal Processing and Machine Learning Techniques for Unraveling Relations Between Various Functional Brain Imaging Modalities

Multi-modal approaches and in particular combined electrophysiological measures with fMRI offer an opportunity to observe neurophysiological events in high temporal and spatial resolution. Numerous attempts of developing methods for solving EEG (and MEG) inverse problems based on the apriori information extracted from the fMRI were made. All the related approaches that try to answer this question still rely on the use of handcrafted features and simple models. However, considering the specific characteristics of EEG-fMRI data should lead to the development of a more sophisticated exploratory tool based on the latest achievements in the field of machine learning. We demonstrate for the first time the use of the deep learning for mapping a multichannel EEG to a BOLD signal. We employ end-to-end convolutional neural network based architecture and train it to predict denoised fMRI volume from the raw EEG resting state signal.