Search the site...

ORGANIZATION FOR HUMAN BRAIN MAPPING
  • BLOG
  • Tutorials
  • Media
  • Contributors
  • OHBM WEBSITE
  • BLOG
  • Tutorials
  • Media
  • Contributors
  • OHBM WEBSITE

How to share and analyze our MRI data best: COBIDAS Report

4/14/2017

0 Comments

 
SHRUTI GOPAL VIJ
COBIDAS: OHBM Committee on Best Practice in Data Analysis and Sharing 

Neuroimaging researchers study both the structural and functional organization of the brain in health as well as a variety of neuropsychological conditions. Extensive exploration has been conducted over the past few decades using novel experimental and analysis techniques. However, recent evaluations of these techniques have highlighted concerns that published scientific results are less reliable or reproducible primarily due to the lack of transparency in research practices. To address the need for outlining principles of scientific research that will increase transparency and reproducibility in human neuroimaging, the Organization for Human Brain Mapping created the Committee on Best Practices in Data Analysis and Sharing (COBIDAS).

COBIDAS has initiated efforts in distilling best practices for open science in human neuroimaging and has compiled suggestions for specific research practices to support open data and open methodology. These suggestions were composed in a report that was drafted and ratified using community collaboration. While the report itself provides a detailed description of best practices as well as approaches to avoid, a commentary that reviewed the impact of COBIDAS and the challenges ahead was published in February 2017.

The commentary underlines the importance of reproducibility and highlights the different aspects of replicating a study including generalizability over methods, materials and results as shown in Figure 1. One practice that was universally recommended is the transparent and complete reporting of all facets of a study, allowing the critical reader to evaluate the work and fully understand its strengths and limitations. Additionally, thorough reporting will equip the reader/other researchers with a detailed knowledge of how to fully replicate the study.

COBIDAS MRI report presents reporting checklists in Appendix D that researchers can follow while preparing manuscripts. These checklists are highly comprehensive and run the gamut of stages a human neuroimaging study passes through. The report also pinpoints specific bad practices, especially in statistical modeling and inference (Pages 11 & 12), and provides concrete recommendations for avoiding them. COBIDAS also suggest that researchers at all levels be mindful of these considerations while conducting individual studies and compiling manuscripts.
Picture
Figure 1: Different conditions of reproducibility and generalizability of MRI studies. For example, results can be reproduced using the same data, same analysis code and same data analysts. Results can also be generalized by using same analysis and experimental methods but with different analysts and experimenters or subjects. This graph summarizes the different dimensions in which experimental results can be reproduced. Image created by the OHBM Committee on Best Practice in Data Analysis and Sharing (COBIDAS).

Another strong recommendation from COBIDAS is data sharing as highlighted in section 7 (Pages 19-24). While sharing of results was strongly agreed upon by the imaging community and COBIDAS, some concerns were articulated by the community. The main concerns of the neuroimaging community outside of COBIDAS included data ownership, maintaining autonomy over results, concern of uncovering errors in analysis, cost of data sharing, and protection of  subject privacy. COBIDAS posits that there are ways to overcome each of these concerns by not only being personally open to the fallible nature of individual research and welcoming collaborations initiated by data sharing, but also by emphasizing the role of the organization and the lab in making these practices widespread. COBIDAS also suggests that a culture of constructive criticism needs to be imbued in the community to allow researchers to work on reproducibility studies. They credit some studies such as 
Waskom et al, Whitaker et al and Pernet et al as well as the Montreal Neurological Institute that have become the flag bearers of open science by sharing their research data and analysis scripts.


Such recommendations and practices are leading to the development of new data sharing methods and open science tools for neuroimaging. Some of the identified tools for data sharing at various experimental stages include the Brain Imaging Data Structure (BIDS); the CBRAIN web-based analysis service; the COINS service; the LONI pipeline; the Neurovault repository; and the FCP/INDI and ADNI data sharing repositories. The COBIDAS recommends using the many resources available to share data,  analysis scripts, and well-written code, along with unthresholded result maps to allow precise meta-analysis and follow up studies.

Finally, the COBIDAS acknowledges that while many of the practices suggested and recommended require individuals to change the way they approach implementation and reporting of a research study, advancement of these open science initiatives will only bear fruit by institutional support and direction. They posit that universities and research centers, as well as the journals themselves, should require data and code sharing. Foundations and grant agencies should also recognize and fund the explicit costs of data sharing. However, none of these will be able to single-handedly bring about the radical change that open science warrants. COBIDAS seeks a coordinated effort from individual researchers, research centers,universities,grant agencies, and it also invites professional organizations to accelerate the drive towards open science via extensive education and outreach. 

What does this mean for you and me as human brain mappers? We need to be more transparent in what we do. Not just because of the implications this has on clinical applications, but to make human brain mapping a more reliable field. Small changes in our approach to science such as documenting all the analytical methods applied while working through the projects and sharing this document on our personal sites or as supplementary material to manuscripts is a small initiative we can all take. Another simple task would be to share code written for projects on code sharing platforms such as github. At the level of the PI’s, making it a lab requirement to share this information with all the lab members including de-identified participant information and maintaining documentation for all projects: ongoing and completed would be speed up the process. This also makes it easy to follow up on projects once postdocs and graduate students have moved on. 

As they say- “Little drops of water make the mighty ocean”, if we all commit to do our part, we can make open science a reality and very soon.
0 Comments

Your comment will be posted after it is approved.


Leave a Reply.

    BLOG HOME

    ​TUTORIALS

    ​MEDIA

    ​contributors

    ​OHBM WEBSITE

    ​

    OHBM OnDemand 
    ​Education Platform


    RSS Feed

    Archives

    January 2024
    December 2023
    November 2023
    October 2023
    September 2023
    August 2023
    July 2023
    June 2023
    May 2023
    April 2023
    March 2023
    January 2023
    December 2022
    October 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    March 2022
    January 2022
    December 2021
    November 2021
    October 2021
    September 2021
    August 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021
    December 2020
    November 2020
    October 2020
    September 2020
    June 2020
    May 2020
    April 2020
    March 2020
    February 2020
    January 2020
    December 2019
    November 2019
    October 2019
    September 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016

stay connected with ohbm!


become a member

Telephone

952-646-2029

Email ohbm

EMAIL BLOG TEAM
Header image created by Thiebaut de Schotten & Batrancourt  
www.brainconnectivitybehaviour.eu