Q&A with OHBM Multi-Modal Imaging Task Force members

By AmanPreet Badhwar, Nils Muhlert & Jeanette Mumford

At its best, multi-modal imaging offers rich insight into a many aspects of brain structure & function. At the same time, its development has been thwarted by challenges, for example simultaneous EEG-fMRI has additional safety concerns, and the EEG data requires extra analysis steps to account for artifacts from the magnetic field and rapidly changing field gradients. Despite these issues, there is increasing attention to the merits of this approach, with high profile journals dedicating special issues to multi-modal data fusion.

To find out about the promises and pitfalls of multi-modal imaging, we sent a series of questions to members of the OHBM Multi-Modal Imaging Task Force. This team is comprised of experts in different imaging domains, and aims to promote and develop multi-modal imaging. We found out the state of the field from Alain Dagher, neurologist and PET/fMRI expert in the Montreal Neurological Institute, Urs Ribary, cognitive neuroscientist and EEG/fMRI expert in British Columbia, Gitte Knudsen, neurologist and translational neurobiologist at Copenhagen University, and Shella Keilholz, physicist and fMRI expert at Emory University and Georgia Tech.

From left to right: Shella Keilholz, Alain Dagher, Gitte Knudsen and Urs Ribary

OHBM: First, what advice would you give to those who are keen to get into multi-modal imaging?

Alain Dagher (AD): Make sure you have a strong grasp of both methods.

Urs Ribary (UR): First, focus on understanding the neurophysiological and biochemical aspects of the brain; then learn individual methods (MRI-fMRI, MEG/EEG, PET, or others…); finally, learn the additional technologies and techniques that will allow you to integrate these different sources of information.

Gitte Knudsen (GK): You need to train at a site where there is high-level expertise in both modalities, and preferentially integrated. If you cannot readily become attached to an academic site that masters true multimodality, do your master thesis/PhD in a centre where they master one or two of the modalities and then move on to another site with the complementary expertise.

Shella Keilholz (SK): Well I would tell them that if they want to do it, just go for it! It’s a great way to increase the impact of your research, especially when the additional modality allows you to make inferences about causality or fundamental mechanisms that you can’t obtain with a single methodology. Sometimes it seems overwhelmingly difficult to add another modality but we have always been able to find collaborators who generously help us get started.

OHBM: It seems the tools for collecting the data are more readily available (e.g. MRI compatible EEG setups). What is the biggest remaining hurdle in conducting multimodal studies? Is data-fusion between modalities improving?

AD: The increased cost and complexity is generally what holds this back. [Further note from Jean Chen, OHBM blogteam member: “For example, an integrated PET/MRI system is more costly than a regular PET or MRI system. Whilst it may not be as expensive as buying a PET and an MRI system separately, new money is often required to get into multi-modal imaging.”].

GK: The biggest hurdle is, first, to master more than one tool to perfection and second, to ask the right scientific questions that can only be addressed using a multimodality approach. Data-fusion between modalities is a challenge, but slowly improving.

UR: Yes, data fusion is improving, but not so much the underlying knowledge of neurophysiology (why to integrate). There are also clearly issues with money (more recordings are more expensive) and with time (it requires more knowledge and work, and everybody wants to publish quickly). On the other hand, data fusion is not something that has to be done alone, and can be done efficiently in collaborations.

SK: One of the biggest challenges in multimodal research is designing experiments and analyses that maximize the use of the information obtained from both modalities. It requires thinking beyond the conventional paradigms for each of the modalities involved.

OHBM: The increased use of simultaneous PET-MR scanners has clear advantages for cancer imaging. What benefits do you feel it may hold for other areas of neuroimaging?

UR: A clear benefit would be the ability to combine biochemical information with information about brain structure, function and dynamics.

AD: There are many benefits. For example if you take the combination of BOLD and neurotransmitter imaging, since neurotransmitter signalling fluctuates, simultaneous measurement of, for example, dopamine signalling and task-related BOLD has great potential. This then also allows powerful task designs with pharmacological manipulations.

GK: It allows us to measure neurotransmitter release and receptor occupancies and hemodynamic responses simultaneously. We can then use this with pharmacological, physiological or other stimuli. Another great advantage is that it saves time (becoming a one stop shop) for patients with neurological or psychiatric disorders, and so can be useful for those who are not able to tolerate multiple scanning sessions. Unfortunately, despite saving time and possibly resources, the simultaneous acquisition of these different types of information has not yet been truly exploited.


OHBM: The last decades have seen the development of a number of new radioligands for imaging tau and amyloid pathology, microglial activation with translocator protein, phosphodiesterases, and other exciting clinical markers. Are these helping drive multi-modal imaging research? Which emerging PET tracers are you most excited about and why?

AD: For me, the most exciting tracers have been those used to image tau and amyloid, providing otherwise unavailable information about neurodegenerative diseases. Previously we only had brain atrophy as a proxy of disease.

GK: If we’re still talking about hybrid scanners, then we are most interested in developing tracers that target components in the brain that are under rapid regulation. In these cases the methodology can capture these regulations and relate them to, for example, the hemodynamic responses. I’m currently excited about radioligands that are sensitive to neurotransmitter release, as well as emerging PET tracers that are informative of brain processes key to many different types of functions/pathologies. For example, tracers that indicate neuroplasticity or stem cells.

UR: Everything helps! I’ve been impressed with recent research relating imaging of neurotransmitters to cognitive functions in health and disease. In addition, the ability to image GABA as an inhibitory substance has been fascinating to see how it may contribute to, and even control, brain development and dynamic network functions. Last, it’s helped us understand the brain as a fine-tuned electrochemical system which controls all brain functions.

OHBM: Simultaneous EEG-fMRI offers high spatial and temporal precision - but how have labs coped with the challenge of integrating and analysing this wealth of data?

AD: This has been especially problematic for EEG. What we need is good open-source processing software for integrating this information, along with online tutorials and courses to teach people how to use them.

UR: I believe that there’s still not enough work in this area. We need to have a much greater understanding of how structure, overall function and brain dynamics integrate in order to understand how typical/atypical brain networks function. Here the question is not so much about using information from different methods to prove each other but instead to complement each other.

OHBM: EEG-fMRI has clear benefits in conditions like epilepsy, for identifying seizure focus and spread. What applications has it had in other conditions - and what do researchers hope to achieve with it?

AD: Cognitive neuroscience can certainly benefit from the combination of higher spatial and temporal resolution in brain mapping.

GK: EEG-fMRI also has promise for use in sleep physiology, sleep disorders and coma.

UR: Any typical cognitive functions and any pathology which are ALL based on structure, function and dynamics....

OHBM: What do you think are the main strengths of multi-modal MRI work? Do you feel it offers hope for developing valid and reliable MR-biomarkers?

UR: Absolutely! Science is not a mystery, the more complementary information we have, the better we understand the human brain. It will help us to diagnose/monitor sub-types of pathologies and give much greater precision when tracking the effects of interventions....

AD: I do believe using multiple MR measures makes sense for biomarker development and understanding pathophysiology. Pathological processes (e.g. in Alzheimer’s Disease) can affect the brain in multiple but likely stereotyped ways. We can also Increase our power to detect pathology (e.g. inflammation, white and grey matter tissue loss, connectivity information) by combining multiple measures.

OHBM: What additional challenges do animal studies have in terms of sequence development or protocol considerations? How do you find these studies enrich those in humans?

AD: Clearly a major issue is the small size of animal brains. We also have to account for the animals typically being anaesthetised when scanned, which has implications for physiology.

GK: Sometimes data from preclinical studies can help optimize a project to be conducted later in humans.

UR: The real benefit of these preclinical studies is that it allows us to perform complementary invasive studies not possible on humans, such as MRI-histology studies. We do however need to continue developing better, or more realistic, settings in animal research in order to better correlate those findings with human brain research.

SK: One of the challenges that we’ve found is that tools that are available on human MRI systems (simultaneous multislice EPI, for example) are not easily implemented on animal systems due to hardware limitations. As Alain says, the other main issue is the use of anesthesia in animals, a special challenge for functional neuroimaging studies, as discussed in our review. Luckily, many of the basic properties of the brain remain relatively intact under light anesthesia, which has been critical of the validation of human neuroimaging methods against “ground truth” modalities like microelectrode recording. People talk of animal research as preclinical or translational, but we like to think of it more as circular. For example, one can take a neuroimaging finding in humans (e.g., fMRI response to tactile stimulation) and look at its basis in the rat using MRI and electrophysiology. Then perhaps one sees that this response is altered in human patients with a particular disorder (maybe stroke). One can then go back to a rat model of stroke and see if the same alteration is present, which helps to validate the stroke model. Then one can look for the neural basis of the alteration using MRI and electrophysiology and identify specific alterations in patients that may be detectable with EEG…etc, etc. We think that human and animal neuroimaging work should inform each other.

OHBM: Thanks all for your insight! We look forward to the multi-modal imaging symposium at OHBM 2018 in Singapore.