In this episode, Peter Bandettini meets with Tom Nichols, Remi Gau and Jack Van Horn to discuss the motivation for a set of best reporting and analysis practices. This provides insight into how the COBIDAS (Committee on Best Practice in Data Analysis and Sharing) in OHBM started. We talk about the reproducibility crisis in fMRI and how it is being addressed. We discuss how the culture of fMRI has changed from isolated scientists doing N=20 studies to a connected web of researchers collecting and contributing to fMRI databases of high quality data for the purpose of revealing ever more subtle information. Through this work, the field aims to achieve robust biomarkers that are clinically useful in diagnosing and treating diseases. We also discuss many of the issues and decisions made in analysis, and how this may contribute to irreproducible results. Last, we consider the ongoing and future global efforts to increase data transparency to make fMRI a more effective tool.
Remi Gau, Ph.D. is currently a postdoc at the Catholic University of Louvain in Belgium. He received his PhD. in 2010 in neurosciences from the University of Pierre and Marie Curie in Paris, and has studied fMRI methodology at Max Planck Institute in Tuebingen and University of Birmingham, UK. He has been active over the years focusing on the infrastructure of imaging data collection and sharing as well as more widely on the culture of neuroimaging, and most recently, created the COBIDAS (Committee on Best Practice in Data Analysis and Sharing) checklist in 2019 as well as eCOBIDAS. He also does neuroscience research, focusing on laminar fMRI to explore how the brain integrates and uses information.
Tom Nichols, Ph.D. is the Professor of Neuroimaging Statistics and a Wellcome Trust Senior Research Fellow in Basic Biomedical Science. He is a statistician with a solitary focus on modelling and inference methods for brain imaging research. He has a unique background, with both industrial and academic experience, and diverse training including computer science, cognitive neuroscience and statistics. He received his Ph.D. in Statistics from Carnegie Mellon University in 2001. After serving on the faculty of University of Michigan's Department of Biostatistics (2000-2006) he became the Director Modelling and Genetics at GlaxoSmithKline's Clinical Imaging Centre, London. He returned to academia in 2009 moving to the University of Warwick, taking a joint position between the Department of Statistics and the Warwick Manufacturing Group. Finally in 2017, he joined the Big Data Institute at Oxford. The focus of Dr. Nichols work is developing modelling and inference methods for brain image data. His current research involves meta-analysis of neuroimaging studies and informatics tools to make data sharing easy and pervasive.
Jack Van Horn, Ph.D. received his Ph.D. in Psychology from the University of London, and then received his Masters of Science and Engineering from the University of Maryland. He is currently a professor in the department of Psychology at the University of Virginia. He was a staff fellow at the NIH until 2000. He moved to Dartmouth College and while there - until 2006 - was instrumental in starting their databasing and data sharing efforts. In 2006 he moved to UCLA and contributed in a large way to their data repository efforts. In 2014 he moved to USC, and finally in 2020, moved to the University of Virginia. He has been an active member of OHBM and a proponent of data sharing since the very early days.
The Neurosalience production team consists of Anastasia Brovkin, Katie Moran, Nils Muhlert, Kevin Sitek, and Rachael Stickland.
by Roselyne Chauvin & Valentina Borghesani
We’ve freshened up!
After two years of existence as an official OHBM Special Interest Group (SIG), the BrainArt SIG has now proudly released its website, created by Anastasia Brovkin and Désirée Lussier, following brainstorming by all SIG officials. You can browse through all previous competitions and exhibits, as well as submit your pieces for the 2021 edition!
You can find out more about our SIG by checking out previous posts on how we came of age and how we consolidated our role within OHBM, but also about our prehistory and history. And we highly recommend having a listen to Neurosalience episode #8, where we had a blast chatting with Peter Bandettini.
We’re preparing a great BrainArt Exhibit for you!
The BrainArt SIG is busy preparing the annual brainart exhibit for OHBM 2021 meeting attendees. The SIG has confirmed artists from all over the world and...well, we don’t want to spoil your surprise but you are in for a treat! We will offer a broad representation of artists ranging from full time scientists, full time artists, and all creative souls in-between. Our 2021 theme is “Big Data & Me”. We wish to celebrate the achievements of Big Data neuroimaging projects, while acknowledging the suffering of individuals affected with brain disorders. Honoring the trees as well as the forest.
First, we will dive into large datasets while keeping in focus inclusivity, diversity, and the representation of populations.
Second, from big N to small N, what about the personal suffering of individuals? After all, we started the field with case study. We will explore the dimension of brain illnesses such as schizophrenia, depression, age-related neurodegenerative diseases, epilepsy, multiple sclerosis and autism in a single subject looking glass.
Finally, will host the pioneering ideas linking different levels of observation, as interpreted by artists who are encouraged to explore the reciprocal interactions between Big Data research and personalized treatment, i.e. breaking of the barrier between research findings and treatment.
We’re ready for your artworks!
So don’t wait another minute! Please go explore our website and the archives and get inspired. It’s time to create and participate in the BrainArt competition 2021 - now open to accept your masterpieces. On the website you will find a form to submit your art for one of the following five award categories:
This competition highlights an ongoing aim of the BrainArt SIG, which is to foster the dialog between artists and scientists, blurring the line between Arts and Science. We believe that the exchange of ideas and tools between these two disciplines encourages the development of novel approaches to scientific data visualization, and promotes the exploration of different perspectives on human brain structures and functions. Researchers, scientists, and everyone in between: you are all encouraged to submit your original work(s)! There are no limits to the number of submissions per participant, and both team and single-person entries are welcomed.
The Submission Deadline is 11:59 PM CDT, Saturday, June 6, 2021 and the award notification will take place during the Annual Meeting. For additional details, please check out our website.
This year, following the success of our campaign to provide a logo for Aperture, there is also a very special 6th category added to the BrainArt competition:
We hope that this teaser and the exploration of the new OHBM BrainArt SIG website will encourage you to participate or enjoy this year's BrainArt Competition & Exhibit.
With contributions from the BrainArt SIG:
In this episode Peter Bandettini meets Carolina Makowski, Michele Veldsman and Alex Fornito to discuss the OHBM Student–Postdoc special interest group (SIG), with particular emphasis on their mentoring scheme and meeting-related workshops. Carolina is a current member of the SIG, Michele previously served as its Chair, and Alex has been an active mentor to several junior OHBM members over the years through this group. They discuss the mentorship program, the workshops at the meeting, what good mentorship is, and why it’s needed more than ever, as the stresses and demands of students and postdocs increases within an ever more demanding professional climate.
Carolina Makowski, Ph.D. is the Career Development and Mentorship Director–Elect of the Student–Postdoc Special Interest Group. Dr. Makowski completed her PhD in neuroscience at McGill University and is currently a postdoctoral fellow at the University of California San Diego with Dr. Anders Dale and Dr. Chi-Hua Chen, with funding from the Canadian Institutes of Health Research, Fonds de Recherche du Quebec - Santé, and the Kavli Institute for Brain and Mind.
Michele Veldsman, Ph.D. is a previous Chair of the Student-Postdoc Student Interest Group and is currently a Postdoctoral Research Scientist in Cognitive Neurology, University of Oxford.
Alex Fornito, Ph.D. is the Sylvia and Charles Viertel Foundation Senior Research Fellow, Professor of Psychological Sciences, and Head of the Brain Mapping and Modelling Research Program at the Turner Institute for Brain and Mental Health. He leads his Neural Systems and Behavior Lab and has actively participated in the student-postdoc SIG.
The Neurosalience production team consists of Anastasia Brovkin, Katie Moran, Nils Muhlert, Kevin Sitek, and Rachael Stickland.
“It is precisely our plasticity, our long childhood, that prevents a slavish adherence to genetically programmed behavior in human beings more than in any other species.”
― Carl Sagan, Dragons of Eden: Speculations on the Evolution of Human Intelligence
I first learned about Ted Satterthwaite’s work when I started teaching about resting state fMRI and motion artifacts. His research showed how motion affects resting state connectivity measures, and I was thrilled that his group also compared the variety of effects with different preprocessing pipelines. In Mexico, every year we host a Neuroimaging Meeting where we invite neuroimaging researchers to visit the city of Guanajuato, [binge] eat Mexican food and talk to students, and so we were delighted to invite Ted to our 2019 meeting.
From our time together there, I got to know more about Ted and his research program. He is currently an Associate Professor in the Department of Psychiatry at the University of Pennsylvania Perelman School of Medicine, and the Director of the Lifespan Informatics & Neuroimaging Center. As a psychiatrist, he is highly interested in human development and building huge development datasets.
When I was asked to do this interview I knew it was going to be difficult to focus on a topic, but we managed to come up with a coherent chat, which I hope we can soon repeat with some beer and mezcal.
Eduardo A. Garza-Villarreal (EG): How did a psychiatrist end up writing these influential methodological papers such as the effect of movement on BOLD signal? And how did you go from psychiatry to methods? What was your career path?
Ted Satterthwaite (TS): I am a psychiatrist, and the reason I got into research was to try to develop tools that could be useful for the diagnosis and treatment of mental illness. That being said, I quickly learned if we ignore the methods, we probably can’t make progress on the ultimate clinical problems that we're interested in. I've primarily been working on large scale datasets, because to me, the only question in psychiatry is clinical heterogeneity. For example: someone comes to see me in the clinic and I diagnose them with depression. But clearly, depression is not one “thing” – depression is almost certainly many different things that we call one thing. It is pretty clear that to parse heterogeneity, we need large studies, because your ability to parse heterogeneity will be determined by both the noise in your signal and the size of your sample. Since we have noisy signals, we probably need very large samples. However, when having very large samples, although you get a lot of statistical power, you also become incredibly sensitive to confounding signals. There's a history in psychiatric imaging of being worried about medication confounds as well as other types of confounds. When we started doing developmental studies, that's right around the time my twins were born. Now, you don't need to scan thousands of kids to recognize that they don't sit still, you just need to come to dinner at my house. And it brought up this obvious question of “is movement going to impact our measures?”, and we started thinking about it because we could see the artifact in the time series. We were just very surprised. We assumed that this had already been solved. However, although there were papers from Karl Friston and others from a decade ago on motion in task-fMRI, there was nothing for functional connectivity. At that point it was just a practical question because we wanted to study brain development, we wanted to study psychopathology, and we know both age and illness are associated with in-scanner motion.
EG: Why are you so interested in development? And why do you think it's an important topic of research in psychiatry?
TS: The dominant paradigm in psychiatry now is that, when you see someone with their first onset symptoms like a severe mood disorder or psychosis, it's not like something went wrong in their brain right there that caused them to have the symptoms. It’s not like a switch that was flipped. Rather, there is accruing evidence for many years now suggesting that most mental illnesses are neurodevelopmental in origin. So, the goal is to understand how the brain develops normally, and then understand also how abnormal patterns of brain development are associated with different sorts of psychopathology. If you think about other fields of medicine, the way they've made clinical progress is by getting there earlier and unpacking heterogeneity. Think about cancer. We used to diagnose cancer only when we found huge tumors that had spread widely, and they were diagnosed on physical examination, like palpating the abdomen and saying, “you have a tumor”. Advances in oncology and in other fields, they have been predicated on both getting there earlier and unpacking heterogeneity; saying it's not just a tumor, but that it is a malignancy from this tissue, with this receptor profile and that genetics, and as a result it's going to be sensitive to this treatment. I think we're still at that “it's a tumor” phase in psychiatry. My hope is that by better understanding patterns of brain development and heterogeneity within the disorders we treat, we can get there earlier and ultimately achieve better outcomes for patients.
EG: As you say, psychiatric disorders are very heterogeneous, and there's a lot of overlap especially in symptoms. What do you think about the current disorder classification, about the Research Domain Criteria (RDoC), and about this overlap between disorders?
TS: There are two sides of the same coin: heterogeneity within disorders, and non-specificity across disorders. The RDoC framework is trying to map symptom profiles to brain systems, and I think that's a totally laudable approach. The one challenge of all this, though, is we don't have a ground truth. We don't have anything like postmortem pathology in neurodegenerative diseases, and so the challenge right now, for example in machine learning techniques, is that we don't know what the labels should be. You can have the best engineers and the best pattern recognition algorithms, but a lot of advances in machine learning have been based on supervised learning, and right now, we just don't have the best labels. Without good labels, it's kind of garbage in, garbage out. What I see is that the biggest challenge is using biological data like images to help us understand what those labels should be. I think it's a challenge and we're all still grappling with it.
EG: Do you think machine learning will have some influence on the future of psychiatry, or you think it's one more tool in the bag?
TS: I think both. Machine learning is going to have a big influence, especially as we have larger and larger training datasets, and the ability to generalize things across samples. However, part of the problem right now is that we don't have a lot of datasets where we can link multivariate patterns from machine learning tools to outcomes of interest, which I think is an essential step. Datasets like the Adolescent Brain Cognitive Development (ABCD) Study that follow 10,000 kids over 10 years are actually really important starting points because longitudinal prognosis is a very important clinical outcome to be able to predict. However, there is still a lot of room to make advances in terms of incorporating these methods using health system data where we have medical records on medications and hospitalizations, but we have a long way to go. Also, I think machine learning won't solve everything. Dani Bassett, one of my closest collaborators, makes a very cogent case that machine learning, while great, alone will not do it; you need a combination of good machine learning and sound theory. I agree. We should not forget that we know a lot about the brain from decades of basic neuroscience – we need to incorporate that knowledge as priors to inform these tools and help interpret the results.
EG: To you, what is your lab’s most interesting project and why?
TS: Right now, the one I am most excited about is a super ambitious project that I lead together with Michael Milham at the Child Mind Institute. We're trying to get a lot of the larger studies of brain development - around 11,000 samples in total – and make sure they are pristinely curated, processed, and QA’d ahead of a public data release on the International Neuroimaging Data-Sharing Initiative (INDI). The project is called the RBC - “Reproducible Brain Charts” – project. One thing that I've learned is that a lot of the sexy stuff in science is actually easier than the non-sexy stuff like data organization, curation, the really low-level things that are necessary for reproducible neuroscience. These things are often really time consuming. What makes me excited about this project with Mike and his team is that if we can do those things well, the dataset will be much more useful to everyone else, and everyone can just run faster by not having to recreate the wheel. I think that's super exciting. But it's been a huge challenge, because it's very heterogeneous data from different studies, which meant we've had to build new tools to handle it.
EG: Can you tell us a bit about the tools you're developing?
TS: Sure-- there’s a couple, which are at different stages. Sydney Covitz and Matt Cieslak have a really cool tool that is being presented as a OHBM poster at the meeting called “Cu-BIDS” for “Curation of BIDS”. It is a tool for curating Brain Imaging Data Structure (BIDS) formatted data at scale, designed for very large heterogeneous datasets -- we needed it to be able to handle the RBC data. I wish I had a time machine and we had that a couple years ago, because it makes life so much easier. And then once you get that data curated into BIDS, and this is all building off from Russ Poldrack, Chris Gorgolewski and Oscar Esteban’s work with BIDS and fMRIrep, we're focusing on running everything in containerized pipelines. For example, xcpEngine is our containerized post processing pipeline that was developed in the lab by Azeez Adebimpe (now) and Rastko Ciric (originally); it consumes fMRIPrep output to produce derived measures for studies of functional connectivity. Now, a lot of efforts are about moving beyond fMRI. For example, I am super excited about Matt Cieslak’s works to build QSIPrep, which is a fully containerized and highly generalizable BIDS-app for diffusion images. Lastly, Azeez is just finishing up ASLPrep for ASL MRI data. In the end, the goal of all this is to make it all super reproducible and open. If you talk to other neuroscientists, one of the biggest reservations they have about imaging is that it's so complicated, and there's so much data processing... that they just don't believe it. Over time, I’ve come to really agree with other people who have been doing open science for many more years than me -- I think the only answer that will get people to believe us is to just show them everything; sunlight really is the best disinfectant.
EG: In terms of your research achievements, what are you most proud of?
TS: You know, it's funny. What makes me most excited is not the individual projects, but the people I work with. I've just been super lucky to have amazing students who do really incredible work. This year’s keynote leverages work from many awesome people in the lab. The title of the talk is from a review from Valerie Sydnor, one of my grad students, who just dug into complex literature and produced a work of incredible scholarship. Similarly, some of the developmental data I am most excited about is from the hard work of another grad student, Adam Pines. What I'm most proud of is the trainees and how much heart they put into their work, and how much they teach me. That's the fun part for me, the people, not any individual finding.
EG: To me one key discovery was about motion artifacts and how it affects the signal. Even right now there is a huge debate on things like the global signal. I don’t think we have it figured out.
TS: Yes, it remains contentious. I think studies of motion artifacts however is a great example of how science should work. It was awesome – we had that paper in March of 2012, but in January of 2012, Koene Van Dijk and Randy Buchner published almost the same finding in an independent dataset. In February 2012 you got Jonathan Power and Steve Petersen with the same finding and also a method for handling it. So, three different labs, working independently with different data sets – all coming to the same conclusion -- producing three papers in NeuroImage at almost the same time. I think that's a great example of how science can work to provide convergent evidence.
EG: What do you think psychiatry should go for in the future? Where do you think is going? You mentioned looking for an actual ground truth, do you think we're going to get it at some point?
TS: People ask that all the time, “when is this stuff going to be useful?” And I think the first answer is: it's not useful now, to be honest. But that doesn't mean it's not a super important problem. If we do make progress, this sort of work could be incredibly impactful because psychiatric disorders are among the most common afflictions that humans get. But in the end, to really show something, we're going to need clinical trials and outcomes that matter. Some people who have been starting towards this, like Nicholas Koutsouleris and the PRONIA consortium – they are doing some really cool work. In the end we will not convince practicing physicians that this brain imaging in psychiatry matters until we show real results in clinical trials. And that's a challenge—perhaps a 10- or 20-year challenge. But I think we'll get there.
EG: Thank you Ted for taking the time to sit down for this interview. Looking forward to your keynote at OHBM 2021.
In this conversation, Peter Bandettini meets members of the BrainArt SIG to discuss its history from the NeuroBureau to its current formal SIG status. They discuss what brain art (or more generally science art) is, consider what the best features of brain art are, and how, essentially, any scientist trying to convey the essence of their findings can be considered an artist. You’ll discover the planned competitions and directions of the BrainArt SIG. The discussion also considers why diversity in this SIG, the field of Brain Mapping, and science in general is so important.
In the episode you’ll hear about the ‘Dream Catchers’ exhibit from OHBM2017 in Vancouver, and how those with dementia can discover new artistic creativity. You can also see some highlights from the OHBM 2020 exhibits below:
By Kevin Sitek
OHBM’s Annual Meeting is virtual again in 2021, following in the footsteps of 2020’s conference—but don’t expect it to look the same.
2020 was a year marked by challenges. For the Organization for Human Brain Mapping, that included quickly transitioning from the final stages of planning an in-person conference to putting together an entirely new format for its virtual meeting. In many ways, the 2020 Annual Meeting was a huge success. Understandably, though—given the short timeframe for creating and executing a brand-new conference format—not every part of the conference went off without a hitch.
For 2021, the OHBM Council sought to build the Annual Meeting on three core pillars of the OHBM community: Openness, Interactivity, and Accessibility. After months of deliberation by a dedicated task force, the OHBM 2021 Annual Meeting will run on a fully customized, open source platform designed and engineered by the Sparkle team. To help make this decision, Council created the OHBM Technology Task Force (TTF) in September of 2020. In an effort to ensure representation across the entire OHBM community, Council invited over twenty OHBM members to join the TTF, including representatives from the Open Science, Student–Postdoc, Sustainability & Environmental Action, and Brain Art Special Interest Groups (SIGs), multiple OHBM committees, and other diverse voices from OHBM’s membership around the globe.
“The goal of this group was to identify areas for improvement for the 2021 Annual Meeting, as well as to identify a virtual event platform which would meet all stakeholders’ needs for this year’s meeting,” says Mike Mullaly, a member of the OHBM Executive Office. “Over the course of several months, this group vetted various platforms via virtual demos.” To learn more about this process, we turned to TTF members themselves to hear directly about their experiences and hopes in selecting this year’s OHBM virtual platform.
The TTF looked not only at feedback regarding last year’s virtual OHBM meeting but also at other conferences. “We discussed which aspects of these meetings worked or did not work and which features we would like to incorporate into the OHBM platform,” says TTF Chair Professor Alex Fornito. “We also evaluated platforms used by other meetings that were considered to be effective by different TTF members. We then shortlisted different platforms and vendors, encompassing a broad range of open source and commercial options and met with reps for several of them.”
And TTF members came with strong, detailed expectations for the platform vendors. According to TTF member and Student–Postdoc SIG Social Chair Dr. Elvisha Dhamala, the ideal platform “enables real-time conversations and reactions to presentations. It has features that facilitate spontaneous and random interactions and conversations. It has an intuitive user interface that is easily manageable and navigable.”
Finding a vendor that could do all of these things (and more!) turned out to be easier said than done. Alex explains, “We quickly learned that there was no single platform that could do everything we wanted to the level that we desired. We had to make trade-offs.” For instance, some of the platforms that were considered had great search functionality and discoverability but couldn’t incorporate social interactions or work seamlessly across time zones. “In the end, we had to focus on identifying a platform that could do a good job of our priority features, while also having the potential to further develop other features in coming years.” Given the uncertainty of the past 18 months, flexibility will be an important feature moving forward.
Ultimately, the TTF settled on the Sparkle platform. Alex acknowledges that, “In many ways, this is a risky choice; Sparkle was originally developed for online concerts and I believe that we are the first scientific conference to take place on the platform.” Yet, the Sparkle team ultimately won over the TTF.
"Having reviewed in detail many different dedicated conference platforms, the TTF was nearly unanimous in their support for Sparkle.” Mike agrees. “We were looking for a vendor that was offering something completely customizable, open-source, and would improve social interaction. Sparkle was by far and away the front runner in this regard.”
There were a few central tenets that the TTF found attractive in the Sparkle platform. Most crucially, Sparkle demonstrated that virtual conferences could still be highly interactive, serendipitous, and fun. “While OHBM 2020 successfully presented the scientific content for the meeting, it lacked the features needed to socially interact with one another,” explains Elvisha, “whether that’s through mini one-on-one conversations about the ongoing presentation or in spontaneously formed small groups during a happy hour.” This year, the Sparkle platform “enables real-time conversations and reactions to presentations. It has features that facilitate spontaneous and random interactions and conversations. It has an intuitive user interface that is easily manageable and navigable.”
An early prototype of the platform's main map, showing various conference locations, sponsor visibility, and chat functionality. Specific stylistic elements and functions will likely be updated before the conference.
Secondly, the Sparkle team is fully dedicated to open source development: OHBM contracted Sparkle to build the conference platform, but the platform’s source code itself is open source. This means that OHBM can use the conference infrastructure beyond OHBM2021 and continue building in new features and technology. Indeed, “the Sparkle team was very willing to work with us to extend the platform and develop many of the essential features we required,” says Alex. “We did not encounter this openness with many other platforms.”
Finally, Sparkle understood our community’s need for accessibility and inclusion, working with the TTF to incorporate automatic text captioning and intuitive design elements. At last year’s conference, TTF member Professor Tilak Ratnanather used his own speech-to-text software during talks and poster presentations, but it was an imperfect solution to a very real problem. “Not having to think about this will make me more relaxed and focus on science.”
A page dedicated to OHBM SIGs is just off the main map. This is an early prototype—specific features unique to each SIG will be added for the conference in June.
However, as the meeting approaches, there are still a few high-priority items that OHBM and the Sparkle team are working on, including global accessibility. Testing is currently under way around the globe and Zoom has been integrated wherever possible. However, individuals concerned about the ability to connect to the Sparkle platform may try connecting via VPN and a list of Zoom links for the meeting will be available from the Executive Office upon request.
In addition, while real-time speech-to-text technology is advancing rapidly (for instance, 2020’s star app, Zoom, recently made live captioning an option for institutional accounts, and Google Chrome can now do live captioning automatically), in practice there are still significant limitations, especially for speakers with accents that the software wasn’t trained on, as well as for fast-paced, jargon-filled presentations. (We’re sure you can remember one or two of these.)
So while the conference platform is still being optimized for the meeting, TTF representatives from across the OHBM community are helping guide the platform’s development. And, according to Mike, OHBM "will be sending out a survey during the meeting (as we always do) looking for areas of improvement and member feedback" to improve the experience for the next meeting—whatever state that will be in.
Overall, excitement about the new platform is palpable across the board. Elvisha sums it up best: “The lack of space constraints and the endless features that Sparkle has really enables us to facilitate multiple activities simultaneously so we can cater to all interests and host a more inclusive social experience. I’m really excited about Sparkle and I can’t wait for the OHBM community to experience all that’s planned for the 2021 conference!”