目录
深度学习(DL)在应用于自然图像分析时非常成功。相比之下,分析神经成像数据提出了一些独特的挑战,包括:
神经成像是一个强大的工具,正在被用来为健康和紊乱的人类大脑提供重要的见解。它也有潜力将发现和技术进步转化为大脑疾病.
❝In contrast to natural images, which are collected under natural light, neuroimaging data consist mostly of raiological images. Because of this, thie noise distribution of neuroimaging varies depending on the acquisition used.
与在自然光下收集的自然图像相比,神经影像学数据主要由图像组成。正因为如此,神经影像学的噪声分布因所使用的采集而异。
❝Rician noise in MRI, quantum noise in computed tomography (CT)
MRI中的Rician噪声,计算机断层扫描(CT)中的量子噪声等等。
❝As showns in Table 1, neuroimaging data come with many other additional unique aspects, including the number of modalities, high dimensionality, low signal-to-noise ratio, and small sample sizes compared to natural image data.
如表1所示,与自然图像数据相比,神经影像数据具有许多其他独特的方面,包括模态数量、高维数、低信噪比和小样本量。
image.png
MRI作为a noninvasive technique with high spatiotemporal resolution, 是当前研究最广泛的neuroimaging modality.
image.png
❝Advanced neuroimaging analysis approaches are essential for linking brain function and structure to network and behavior.
先进的神经影像学分析方法对于将大脑功能和结构与网络和行为联系起来至关重要。
❝Linear models and, in particular, flexible matrix decomposition approaches have contributed a lot to our current understanding. For instance, group independent component analysis (ICA), as a purely data-driven algorithm that reveals large-scale networks by making group inferences from funcional MRI (fMRI), is particularly useful for data fusion of multiple modalities, such as genome-wide single-nucleotide polymorphism (SNP) data or event-related potentials.
线性模型,特别是灵活的矩阵分解方法,对我们目前的理解做出了很大贡献。例如,组独立成分分析(ICA)作为一种纯粹的数据驱动算法,通过从功能MRI(fMRI)进行组推断来揭示大规模网络,对于多种模式的数据融合特别有用,例如全基因组单核苷酸多态性(SNP)数据或事件相关的风险。
此外,还有standard machine learning (SML)标准机器学习的方法来分析。SML方法通常需要相当多的领域专业知识来设计特征提取器。这些特征提取器将原始数据转换为合适的内部表示或特征向量。
然后就是深度学习DL的方法。但是复杂的模型容易受到黑盒的影响。
在这篇review综述中,four interrelated topics are covered:
分类和回归是两个被广泛研究的监督学习任务。广义上说,两个任务的目标都是把x(神经成像数据映射到y(诊断,治疗反应和行为)。尽管神经成像数据高度多样化,还是可以分成两大类:structural imaging and functional imaging.
structural neuroimaging data 结构成像,例如structural MRI(sMRI)和diffusion MRI(dMRI弥散MRI),reflect voxel tissue density/volume or structural connectivity.反应了体素组织密度、体积或结构连通性
结构研究的主要目的是为了揭示anatomical relationships解剖关系 in the brain,这也可以用于预测。
functional neuroimaging data关注于大脑活动或连通性的动态变化。由于MRI等身成像的高纬度、低信噪比、高效的特征处理对于减少建模前的冗余是非常重要的。例如,functional MRI的时间过程通常采用atlas-based or data-driven approaches来降低维度,例如ICA。然后将得到的时间签名用于研究时间依赖性,如functional network connectivity (FNC)或者dynamic FNC。在这里我们总结了流行的DL模型的基本机制,并就其相应的神经成像提出了建议
通过简单的梯度下降训练的多层感知器MLP是第一个提出可训练的多层特征呢工程的解决方案【3】.
image.png
MLP最适合低纬和较少冗余的输入,如FNC向量。
❝Desprite the great success of CNNs, the non-Euclidean characteristic of graph features such as those obtained from FNC makes the general convolutionand no as well defined as on natural images.
❝Similarly, a graph convolutional network GCN is a type of neural network architecture that can capture the graph structure and aggregate node information from the neighborhoods in a convolutional fashion with fewer learnable parameters. GCNs are useful in medical or biochemical applications with graph data such as FNC。
❝Compared to classical linear machine learning models, such as a hidden Markov model, an RNN models the long-term nonlinear mechanisms of the sequential data.
就是生成模型。
❝The use of an attention module was proposed to increase the representation power and improve interpretability by focusing on important brain regions and suppressing unnecessary ones, which is often combined with other DL models for interpretation, allowing the model to dynamically emphasize certain parts of input.
简单的说,就是可解释性。
❝DL models designed for 3D and 4D neuroimaging data often consist of millions of parameters that require many samples for optimization.
❝The characterization of brain activity and connectivity dynamics (e.g., the chronnectome) is crucial for out understanding of brain function. However, uncovering relevant transient patterns in brain function is challenging because of the lack of computational tools that can effectively capture nonlinear dynamics from high-dimensional data.
大脑活动和连接动力学(例如,connectome)的特征对于理解大脑功能至关重要。然而,揭示大脑功能中的相关瞬态模式具有挑战性,因为缺乏能够有效地从高维数据中捕获非线性动力学的计算工具。
❝Recent studies show that DL models, especially RNN-based networks, have the potential to capture whole-brain dynamic information and utilize the time-varing functional connectivity state profiles to expand our understanding of brain function and disorder.
最近的研究表明,深度学习模型,特别是基于RNN的网络,有可能捕获全脑动态信息,并利用时间变化的功能连接状态图来扩展我们对大脑功能和紊乱的理解。
image.png
传统的神经成像分类方法以functional networks connectivity or spatial maps作为输入特征,忽略了时间动态信息。DL模型具有良好的特征表示学习能力,为直接捕捉时空信息提供了一个潜在的工具。
特别是RNN在序列建模方面取得了巨大的成功,目前广泛应用于brain disorder diagnosis, brain decoding and temporally dynamic functional state translation detection. dFNC是一种从功能核磁共振成像数据中识别time-varing patterns的方法.
后面是对rnn,biLSTM啥的模型,相关文献:
image.png
为了方便发现神经丞相数据中的动态信息,DL可以与研究充分的数据驱动的机器学习方法混合,如ICA。这也可以提高结果的可解释性。有一篇论文提出了一种新的大脑分割时空网络,该网络将3D的CNN和ICA相结合,使得该框架可以探索5D的大脑动力学。
image.png
image.png
此外RNN-ICA提出了结合RNN和ICA使用方案,which can explicitly optimize linear generative models to model temporal dynamics and infer intrinsic networks from time-series observations (the network structure and identified spatial maps are shown in RNN leverages ICA in figure 3)
image.png
基于RNN建模挺好的,但是现有工作中,动态特征通常是基于窗口的相关性的,因此窗口大小是一个影响dFNC特征的超参数,具有较短窗口的不能捕捉长时间的相关性,而较长的窗口降低了对快速变化的敏感性。
什么是窗口,在学习dFNC的时候可以了解到。
神经成像通常包括多种模式,比如sMRI,fMRI,dMRI,他们为观察和分析大脑提供了多种视图。为了利用不同模式的互补表征,因此使用多模态融合。
❝Despite the variety of available models, most multimodal fusion strategies fall into the following two categories: prefusion and postfusion.
除了基于concatenation-based postfusion,还有更先进的方法,考虑交叉模态的关系。Multimodal reconstruction,deep canonical correlation analysis (DCCA) and knowledge-transfer-based fusion are three popular multimodal fusion methods.
更先进的3中多模态融合的方法:
image.png
这个的参考文献可以学一下:
image.png
流行的可视化方法可以分成四类:
5. gradient-based methods 基于梯度的方法 6. layer-wise relevance propagation 分层相关性传播
相关文献:
image.png
通过tsne和聚类来可视化,看看能不能发现新的亚型