Variability in the analysis of a single neuroimaging dataset by many teams

Botvinik-Nezer, R.; Holzmeister, F.; Camerer, C. F.; Dreber, A.; Huber, J.; Johannesson, M.; Kirchler, M.; Iwanir, R.; Mumford, J. A.; Adcock, R. A.; Avesani, P.; Baczkowski, B. M.; Bajracharya, A.; Bakst, L.; Ball, S.; Barilari, M.; Bault, N.; Beaton, D.; Beitner, J.; Benoit, R. G.; Berkers, R. M. W. J.; Bhanji, J. P.; Biswal, B. B.; Bobadilla-Suarez, S.; Bortolini, T.; Bottenhorn, K. L.; Bowring, A.; Braem, S.; Brooks, H. R.; Brudner, E. G.; Calderon, C. B.; Camilleri, J. A.; Castrellon, J. J.; Cecchetti, L.; Cieslik, E. C.; Cole, Z. J.; Collignon, O.; Cox, R. W.; Cunningham, W. A.; Czoschke, S.; Dadi, K.; Davis, C. P.; Luca, A. D.; Delgado, M. R.; Demetriou, L.; Dennison, J. B.; Di, X.; Dickie, E. W.; Dobryakova, E.; Donnat, C. L.; Dukart, J.; Duncan, N. W.; Durnez, J.; Eed, A.; Eickhoff, S. B.; Erhart, A.; Fontanesi, L.; Fricke, G. M.; Fu, S.; Galván, A.; Gau, R.; Genon, S.; Glatard, T.; Glerean, E.; Goeman, J. J.; Golowin, S. A. E.; González-García, C.; Gorgolewski, K. J.; Grady, C. L.; Green, M. A.; Guassi Moreira, J. F.; Guest, O.; Hakimi, S.; Hamilton, J. P.; Hancock, R.; Handjaras, G.; Harry, B.B.; Hawco, C.; Herholz, P.; Herman, G.; Heunis, S.; Hoffstaedter, F.; Hogeveen, J.; Holmes, S.; Hu, C. P.; Huettel, S. A.; Hughes, M. E.; Iacovella, V.; Iordan, A. D.; Isager, P. M.; Isik, A. I.; Jahn, Andrew; Johnson, Matthew R.; Johnstone, Tom; Joseph, Michael J. E.; Juliano, Anthony C.; Kable, Joseph W.; Kassinopoulos, Michalis; Koba, Cemal; Kong, Xiang-Zhen; Koscik, Timothy R.; Kucukboyaci, Nuri Erkut; Kuhl, Brice A.; Kupek, Sebastian; Laird, Angela R.; Lamm, Claus; Langner, Robert; Lauharatanahirun, Nina; Lee, Hongmi; Lee, Sangil; Leemans, Alexander; Leo, Andrea; Lesage, Elise; Li, Flora; Li, Monica Y. C.; Lim, Cheng Phui; Lintz, Evan N.; Liphardt, Schuyler W.; Losecaat Vermeer, Annabel B.; Love, Bradley C.; Mack, Michael L.; Malpica, Norberto; Marins, Theo; Maumet, Camille; McDonald, Kelsey; McGuire, Joseph T.; Méndez Leal, Adriana S.; Meyer, Benjamin; Meyer, Kristin N.; Mihai, Glad; Mitsis, Georgios D.; Moll, Jorge; Nielson, Dylan M.; Nilsonne, Gustav; Notter, Michael P.; Olivetti, Emanuele; Onicas, Adrian I.; Papale, Paolo; Patil, Kaustubh R.; Peelle, Jonathan E.; Pérez, Alexandre; Pischedda, Doris; Poline, Jean-Baptiste; Prystauka,Yanina; Ray, Shruti; Reuter-Lorenz, Patricia A.; Reynolds, Richard C.; Ricciardi, Emiliano; Rieck, Jenny R.; Rodriguez-Thompson, Anais M.; Romyn, Anthony; Salo, Taylor; Samanez-Larkin, Gregory R.; Sanz-Morales, Emilio; Schlichting, Margaret L.; Schultz, Douglas H.; Shen, Qiang; Sheridan, Margaret A.; Silvers, Jennifer A.; Skagerlund, Kenny; Smith, Alec; Smith, David V.; Sokol-Hessner, Peter; Steinkamp, Simon R.; Tashjian, Sarah M.; Thirion, Bertrand; Thorp, John N.; Tinghög, Gustav; Tisdall, Loreen; Tompson, Steven H.; Toro-Serey, Claudio; Torre Tresols, Juan Jesus; Tozzi, Leonardo; Truong, Vuong; Turella, Luca; van ‘t Veer, Anna E.; Verguts, Tom; Vettel, Jean M.; Vijayarajah, Sagana; Vo, Khoi; Wall, Matthew B.; Weeda, Wouter D.; Weis, Susanne; White, David J.; Wisniewski, David; Xifra-Porxas, Alba; Yearling, Emily A.; Yoon, Sangsuk; Yuan, Rui; Yuen, Kenneth S. L.; Lei Zhang; Zhang, Xu; Zosky, Joshua E.; Thomas E. Nichols,; Poldrack, Rusell A.; Schonberg, Tom; Melero Carrasco, Helena

Variability in the analysis of a single neuroimaging dataset by many teams

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1....

Descripción completa

Detalles Bibliográficos
Autores:	Botvinik-Nezer, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber, J., Johannesson, M., Kirchler, M., Iwanir, R., Mumford, J. A., Adcock, R. A., Avesani, P., Baczkowski, B. M., Bajracharya, A., Bakst, L., Ball, S., Barilari, M., Bault, N., Beaton, D., Beitner, J., Benoit, R. G., Berkers, R. M. W. J., Bhanji, J. P., Biswal, B. B., Bobadilla-Suarez, S., Bortolini, T., Bottenhorn, K. L., Bowring, A., Braem, S., Brooks, H. R., Brudner, E. G., Calderon, C. B., Camilleri, J. A., Castrellon, J. J., Cecchetti, L., Cieslik, E. C., Cole, Z. J., Collignon, O., Cox, R. W., Cunningham, W. A., Czoschke, S., Dadi, K., Davis, C. P., Luca, A. D., Delgado, M. R., Demetriou, L., Dennison, J. B., Di, X., Dickie, E. W., Dobryakova, E., Donnat, C. L., Dukart, J., Duncan, N. W., Durnez, J., Eed, A., Eickhoff, S. B., Erhart, A., Fontanesi, L., Fricke, G. M., Fu, S., Galván, A., Gau, R., Genon, S., Glatard, T., Glerean, E., Goeman, J. J., Golowin, S. A. E., González-García, C., Gorgolewski, K. J., Grady, C. L., Green, M. A., Guassi Moreira, J. F., Guest, O., Hakimi, S., Hamilton, J. P., Hancock, R., Handjaras, G., Harry, B.B., Hawco, C., Herholz, P., Herman, G., Heunis, S., Hoffstaedter, F., Hogeveen, J., Holmes, S., Hu, C. P., Huettel, S. A., Hughes, M. E., Iacovella, V., Iordan, A. D., Isager, P. M., Isik, A. I., Jahn, Andrew, Johnson, Matthew R., Johnstone, Tom, Joseph, Michael J. E., Juliano, Anthony C., Kable, Joseph W., Kassinopoulos, Michalis, Koba, Cemal, Kong, Xiang-Zhen, Koscik, Timothy R., Kucukboyaci, Nuri Erkut, Kuhl, Brice A., Kupek, Sebastian, Laird, Angela R., Lamm, Claus, Langner, Robert, Lauharatanahirun, Nina, Lee, Hongmi, Lee, Sangil, Leemans, Alexander, Leo, Andrea, Lesage, Elise, Li, Flora, Li, Monica Y. C., Lim, Cheng Phui, Lintz, Evan N., Liphardt, Schuyler W., Losecaat Vermeer, Annabel B., Love, Bradley C., Mack, Michael L., Malpica, Norberto, Marins, Theo, Maumet, Camille, McDonald, Kelsey, McGuire, Joseph T., Méndez Leal, Adriana S., Meyer, Benjamin, Meyer, Kristin N., Mihai, Glad, Mitsis, Georgios D., Moll, Jorge, Nielson, Dylan M., Nilsonne, Gustav, Notter, Michael P., Olivetti, Emanuele, Onicas, Adrian I., Papale, Paolo, Patil, Kaustubh R., Peelle, Jonathan E., Pérez, Alexandre, Pischedda, Doris, Poline, Jean-Baptiste, Prystauka,Yanina, Ray, Shruti, Reuter-Lorenz, Patricia A., Reynolds, Richard C., Ricciardi, Emiliano, Rieck, Jenny R., Rodriguez-Thompson, Anais M., Romyn, Anthony, Salo, Taylor, Samanez-Larkin, Gregory R., Sanz-Morales, Emilio, Schlichting, Margaret L., Schultz, Douglas H., Shen, Qiang, Sheridan, Margaret A., Silvers, Jennifer A., Skagerlund, Kenny, Smith, Alec, Smith, David V., Sokol-Hessner, Peter, Steinkamp, Simon R., Tashjian, Sarah M., Thirion, Bertrand, Thorp, John N., Tinghög, Gustav, Tisdall, Loreen, Tompson, Steven H., Toro-Serey, Claudio, Torre Tresols, Juan Jesus, Tozzi, Leonardo, Truong, Vuong, Turella, Luca, van ‘t Veer, Anna E., Verguts, Tom, Vettel, Jean M., Vijayarajah, Sagana, Vo, Khoi, Wall, Matthew B., Weeda, Wouter D., Weis, Susanne, White, David J., Wisniewski, David, Xifra-Porxas, Alba, Yearling, Emily A., Yoon, Sangsuk, Yuan, Rui, Yuen, Kenneth S. L., Lei Zhang, Zhang, Xu, Zosky, Joshua E., Thomas E. Nichols,, Poldrack, Rusell A., Schonberg, Tom, Melero Carrasco, Helena
Tipo de recurso:	artículo
Fecha de publicación:	2020
País:	España
Institución:	Universidad Complutense de Madrid (UCM)
Repositorio:	Docta Complutense
Idioma:	inglés
OAI Identifier:	oai:docta.ucm.es:20.500.14352/100828
Acceso en línea:	https://hdl.handle.net/20.500.14352/100828
Access Level:	acceso abierto
Palabra clave:	Psicología (Psicología) 6106.01 Actividad Cerebral

id	ES_10d38dfb094c8b82dae832d4eab81138
oai_identifier_str	oai:docta.ucm.es:20.500.14352/100828
network_acronym_str	ES
network_name_str	España
repository_id_str
dc.title.none.fl_str_mv	Variability in the analysis of a single neuroimaging dataset by many teams
title	Variability in the analysis of a single neuroimaging dataset by many teams
spellingShingle	Variability in the analysis of a single neuroimaging dataset by many teams Botvinik-Nezer, R. Psicología (Psicología) 6106.01 Actividad Cerebral
title_short	Variability in the analysis of a single neuroimaging dataset by many teams
title_full	Variability in the analysis of a single neuroimaging dataset by many teams
title_fullStr	Variability in the analysis of a single neuroimaging dataset by many teams
title_full_unstemmed	Variability in the analysis of a single neuroimaging dataset by many teams
title_sort	Variability in the analysis of a single neuroimaging dataset by many teams
dc.creator.none.fl_str_mv	Botvinik-Nezer, R. Holzmeister, F. Camerer, C. F. Dreber, A. Huber, J. Johannesson, M. Kirchler, M. Iwanir, R. Mumford, J. A. Adcock, R. A. Avesani, P. Baczkowski, B. M. Bajracharya, A. Bakst, L. Ball, S. Barilari, M. Bault, N. Beaton, D. Beitner, J. Benoit, R. G. Berkers, R. M. W. J. Bhanji, J. P. Biswal, B. B. Bobadilla-Suarez, S. Bortolini, T. Bottenhorn, K. L. Bowring, A. Braem, S. Brooks, H. R. Brudner, E. G. Calderon, C. B. Camilleri, J. A. Castrellon, J. J. Cecchetti, L. Cieslik, E. C. Cole, Z. J. Collignon, O. Cox, R. W. Cunningham, W. A. Czoschke, S. Dadi, K. Davis, C. P. Luca, A. D. Delgado, M. R. Demetriou, L. Dennison, J. B. Di, X. Dickie, E. W. Dobryakova, E. Donnat, C. L. Dukart, J. Duncan, N. W. Durnez, J. Eed, A. Eickhoff, S. B. Erhart, A. Fontanesi, L. Fricke, G. M. Fu, S. Galván, A. Gau, R. Genon, S. Glatard, T. Glerean, E. Goeman, J. J. Golowin, S. A. E. González-García, C. Gorgolewski, K. J. Grady, C. L. Green, M. A. Guassi Moreira, J. F. Guest, O. Hakimi, S. Hamilton, J. P. Hancock, R. Handjaras, G. Harry, B.B. Hawco, C. Herholz, P. Herman, G. Heunis, S. Hoffstaedter, F. Hogeveen, J. Holmes, S. Hu, C. P. Huettel, S. A. Hughes, M. E. Iacovella, V. Iordan, A. D. Isager, P. M. Isik, A. I. Jahn, Andrew Johnson, Matthew R. Johnstone, Tom Joseph, Michael J. E. Juliano, Anthony C. Kable, Joseph W. Kassinopoulos, Michalis Koba, Cemal Kong, Xiang-Zhen Koscik, Timothy R. Kucukboyaci, Nuri Erkut Kuhl, Brice A. Kupek, Sebastian Laird, Angela R. Lamm, Claus Langner, Robert Lauharatanahirun, Nina Lee, Hongmi Lee, Sangil Leemans, Alexander Leo, Andrea Lesage, Elise Li, Flora Li, Monica Y. C. Lim, Cheng Phui Lintz, Evan N. Liphardt, Schuyler W. Losecaat Vermeer, Annabel B. Love, Bradley C. Mack, Michael L. Malpica, Norberto Marins, Theo Maumet, Camille McDonald, Kelsey McGuire, Joseph T. Méndez Leal, Adriana S. Meyer, Benjamin Meyer, Kristin N. Mihai, Glad Mitsis, Georgios D. Moll, Jorge Nielson, Dylan M. Nilsonne, Gustav Notter, Michael P. Olivetti, Emanuele Onicas, Adrian I. Papale, Paolo Patil, Kaustubh R. Peelle, Jonathan E. Pérez, Alexandre Pischedda, Doris Poline, Jean-Baptiste Prystauka,Yanina Ray, Shruti Reuter-Lorenz, Patricia A. Reynolds, Richard C. Ricciardi, Emiliano Rieck, Jenny R. Rodriguez-Thompson, Anais M. Romyn, Anthony Salo, Taylor Samanez-Larkin, Gregory R. Sanz-Morales, Emilio Schlichting, Margaret L. Schultz, Douglas H. Shen, Qiang Sheridan, Margaret A. Silvers, Jennifer A. Skagerlund, Kenny Smith, Alec Smith, David V. Sokol-Hessner, Peter Steinkamp, Simon R. Tashjian, Sarah M. Thirion, Bertrand Thorp, John N. Tinghög, Gustav Tisdall, Loreen Tompson, Steven H. Toro-Serey, Claudio Torre Tresols, Juan Jesus Tozzi, Leonardo Truong, Vuong Turella, Luca van ‘t Veer, Anna E. Verguts, Tom Vettel, Jean M. Vijayarajah, Sagana Vo, Khoi Wall, Matthew B. Weeda, Wouter D. Weis, Susanne White, David J. Wisniewski, David Xifra-Porxas, Alba Yearling, Emily A. Yoon, Sangsuk Yuan, Rui Yuen, Kenneth S. L. Lei Zhang Zhang, Xu Zosky, Joshua E. Thomas E. Nichols, Poldrack, Rusell A. Schonberg, Tom Melero Carrasco, Helena
author	Botvinik-Nezer, R.
author_facet	Botvinik-Nezer, R. Holzmeister, F. Camerer, C. F. Dreber, A. Huber, J. Johannesson, M. Kirchler, M. Iwanir, R. Mumford, J. A. Adcock, R. A. Avesani, P. Baczkowski, B. M. Bajracharya, A. Bakst, L. Ball, S. Barilari, M. Bault, N. Beaton, D. Beitner, J. Benoit, R. G. Berkers, R. M. W. J. Bhanji, J. P. Biswal, B. B. Bobadilla-Suarez, S. Bortolini, T. Bottenhorn, K. L. Bowring, A. Braem, S. Brooks, H. R. Brudner, E. G. Calderon, C. B. Camilleri, J. A. Castrellon, J. J. Cecchetti, L. Cieslik, E. C. Cole, Z. J. Collignon, O. Cox, R. W. Cunningham, W. A. Czoschke, S. Dadi, K. Davis, C. P. Luca, A. D. Delgado, M. R. Demetriou, L. Dennison, J. B. Di, X. Dickie, E. W. Dobryakova, E. Donnat, C. L. Dukart, J. Duncan, N. W. Durnez, J. Eed, A. Eickhoff, S. B. Erhart, A. Fontanesi, L. Fricke, G. M. Fu, S. Galván, A. Gau, R. Genon, S. Glatard, T. Glerean, E. Goeman, J. J. Golowin, S. A. E. González-García, C. Gorgolewski, K. J. Grady, C. L. Green, M. A. Guassi Moreira, J. F. Guest, O. Hakimi, S. Hamilton, J. P. Hancock, R. Handjaras, G. Harry, B.B. Hawco, C. Herholz, P. Herman, G. Heunis, S. Hoffstaedter, F. Hogeveen, J. Holmes, S. Hu, C. P. Huettel, S. A. Hughes, M. E. Iacovella, V. Iordan, A. D. Isager, P. M. Isik, A. I. Jahn, Andrew Johnson, Matthew R. Johnstone, Tom Joseph, Michael J. E. Juliano, Anthony C. Kable, Joseph W. Kassinopoulos, Michalis Koba, Cemal Kong, Xiang-Zhen Koscik, Timothy R. Kucukboyaci, Nuri Erkut Kuhl, Brice A. Kupek, Sebastian Laird, Angela R. Lamm, Claus Langner, Robert Lauharatanahirun, Nina Lee, Hongmi Lee, Sangil Leemans, Alexander Leo, Andrea Lesage, Elise Li, Flora Li, Monica Y. C. Lim, Cheng Phui Lintz, Evan N. Liphardt, Schuyler W. Losecaat Vermeer, Annabel B. Love, Bradley C. Mack, Michael L. Malpica, Norberto Marins, Theo Maumet, Camille McDonald, Kelsey McGuire, Joseph T. Méndez Leal, Adriana S. Meyer, Benjamin Meyer, Kristin N. Mihai, Glad Mitsis, Georgios D. Moll, Jorge Nielson, Dylan M. Nilsonne, Gustav Notter, Michael P. Olivetti, Emanuele Onicas, Adrian I. Papale, Paolo Patil, Kaustubh R. Peelle, Jonathan E. Pérez, Alexandre Pischedda, Doris Poline, Jean-Baptiste Prystauka,Yanina Ray, Shruti Reuter-Lorenz, Patricia A. Reynolds, Richard C. Ricciardi, Emiliano Rieck, Jenny R. Rodriguez-Thompson, Anais M. Romyn, Anthony Salo, Taylor Samanez-Larkin, Gregory R. Sanz-Morales, Emilio Schlichting, Margaret L. Schultz, Douglas H. Shen, Qiang Sheridan, Margaret A. Silvers, Jennifer A. Skagerlund, Kenny Smith, Alec Smith, David V. Sokol-Hessner, Peter Steinkamp, Simon R. Tashjian, Sarah M. Thirion, Bertrand Thorp, John N. Tinghög, Gustav Tisdall, Loreen Tompson, Steven H. Toro-Serey, Claudio Torre Tresols, Juan Jesus Tozzi, Leonardo Truong, Vuong Turella, Luca van ‘t Veer, Anna E. Verguts, Tom Vettel, Jean M. Vijayarajah, Sagana Vo, Khoi Wall, Matthew B. Weeda, Wouter D. Weis, Susanne White, David J. Wisniewski, David Xifra-Porxas, Alba Yearling, Emily A. Yoon, Sangsuk Yuan, Rui Yuen, Kenneth S. L. Lei Zhang Zhang, Xu Zosky, Joshua E. Thomas E. Nichols, Poldrack, Rusell A. Schonberg, Tom Melero Carrasco, Helena
author_role	author
author2	Holzmeister, F. Camerer, C. F. Dreber, A. Huber, J. Johannesson, M. Kirchler, M. Iwanir, R. Mumford, J. A. Adcock, R. A. Avesani, P. Baczkowski, B. M. Bajracharya, A. Bakst, L. Ball, S. Barilari, M. Bault, N. Beaton, D. Beitner, J. Benoit, R. G. Berkers, R. M. W. J. Bhanji, J. P. Biswal, B. B. Bobadilla-Suarez, S. Bortolini, T. Bottenhorn, K. L. Bowring, A. Braem, S. Brooks, H. R. Brudner, E. G. Calderon, C. B. Camilleri, J. A. Castrellon, J. J. Cecchetti, L. Cieslik, E. C. Cole, Z. J. Collignon, O. Cox, R. W. Cunningham, W. A. Czoschke, S. Dadi, K. Davis, C. P. Luca, A. D. Delgado, M. R. Demetriou, L. Dennison, J. B. Di, X. Dickie, E. W. Dobryakova, E. Donnat, C. L. Dukart, J. Duncan, N. W. Durnez, J. Eed, A. Eickhoff, S. B. Erhart, A. Fontanesi, L. Fricke, G. M. Fu, S. Galván, A. Gau, R. Genon, S. Glatard, T. Glerean, E. Goeman, J. J. Golowin, S. A. E. González-García, C. Gorgolewski, K. J. Grady, C. L. Green, M. A. Guassi Moreira, J. F. Guest, O. Hakimi, S. Hamilton, J. P. Hancock, R. Handjaras, G. Harry, B.B. Hawco, C. Herholz, P. Herman, G. Heunis, S. Hoffstaedter, F. Hogeveen, J. Holmes, S. Hu, C. P. Huettel, S. A. Hughes, M. E. Iacovella, V. Iordan, A. D. Isager, P. M. Isik, A. I. Jahn, Andrew Johnson, Matthew R. Johnstone, Tom Joseph, Michael J. E. Juliano, Anthony C. Kable, Joseph W. Kassinopoulos, Michalis Koba, Cemal Kong, Xiang-Zhen Koscik, Timothy R. Kucukboyaci, Nuri Erkut Kuhl, Brice A. Kupek, Sebastian Laird, Angela R. Lamm, Claus Langner, Robert Lauharatanahirun, Nina Lee, Hongmi Lee, Sangil Leemans, Alexander Leo, Andrea Lesage, Elise Li, Flora Li, Monica Y. C. Lim, Cheng Phui Lintz, Evan N. Liphardt, Schuyler W. Losecaat Vermeer, Annabel B. Love, Bradley C. Mack, Michael L. Malpica, Norberto Marins, Theo Maumet, Camille McDonald, Kelsey McGuire, Joseph T. Méndez Leal, Adriana S. Meyer, Benjamin Meyer, Kristin N. Mihai, Glad Mitsis, Georgios D. Moll, Jorge Nielson, Dylan M. Nilsonne, Gustav Notter, Michael P. Olivetti, Emanuele Onicas, Adrian I. Papale, Paolo Patil, Kaustubh R. Peelle, Jonathan E. Pérez, Alexandre Pischedda, Doris Poline, Jean-Baptiste Prystauka,Yanina Ray, Shruti Reuter-Lorenz, Patricia A. Reynolds, Richard C. Ricciardi, Emiliano Rieck, Jenny R. Rodriguez-Thompson, Anais M. Romyn, Anthony Salo, Taylor Samanez-Larkin, Gregory R. Sanz-Morales, Emilio Schlichting, Margaret L. Schultz, Douglas H. Shen, Qiang Sheridan, Margaret A. Silvers, Jennifer A. Skagerlund, Kenny Smith, Alec Smith, David V. Sokol-Hessner, Peter Steinkamp, Simon R. Tashjian, Sarah M. Thirion, Bertrand Thorp, John N. Tinghög, Gustav Tisdall, Loreen Tompson, Steven H. Toro-Serey, Claudio Torre Tresols, Juan Jesus Tozzi, Leonardo Truong, Vuong Turella, Luca van ‘t Veer, Anna E. Verguts, Tom Vettel, Jean M. Vijayarajah, Sagana Vo, Khoi Wall, Matthew B. Weeda, Wouter D. Weis, Susanne White, David J. Wisniewski, David Xifra-Porxas, Alba Yearling, Emily A. Yoon, Sangsuk Yuan, Rui Yuen, Kenneth S. L. Lei Zhang Zhang, Xu Zosky, Joshua E. Thomas E. Nichols, Poldrack, Rusell A. Schonberg, Tom Melero Carrasco, Helena
author2_role	author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author author
dc.contributor.none.fl_str_mv	Universidad Complutense de Madrid
dc.subject.none.fl_str_mv	Psicología (Psicología) 6106.01 Actividad Cerebral
topic	Psicología (Psicología) 6106.01 Actividad Cerebral
description	Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2–5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.
publishDate	2020
dc.date.none.fl_str_mv	2020 2020-05-20 2020 2020-05-20
dc.type.none.fl_str_mv	journal article http://purl.org/coar/resource_type/c_6501
dc.type.openaire.fl_str_mv	info:eu-repo/semantics/article
format	article
dc.identifier.none.fl_str_mv	https://hdl.handle.net/20.500.14352/100828
url	https://hdl.handle.net/20.500.14352/100828
dc.language.none.fl_str_mv	Inglés eng
language_invalid_str_mv	Inglés
language	eng
dc.rights.none.fl_str_mv	open access http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv	info:eu-repo/semantics/openAccess
rights_invalid_str_mv	open access http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv	openAccess
dc.format.none.fl_str_mv	application/pdf
dc.source.none.fl_str_mv	reponame:Docta Complutense instname:Universidad Complutense de Madrid (UCM)
instname_str	Universidad Complutense de Madrid (UCM)
reponame_str	Docta Complutense
collection	Docta Complutense
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_	1869403531694309376
spelling	Variability in the analysis of a single neuroimaging dataset by many teamsBotvinik-Nezer, R.Holzmeister, F.Camerer, C. F.Dreber, A.Huber, J.Johannesson, M.Kirchler, M.Iwanir, R.Mumford, J. A.Adcock, R. A.Avesani, P.Baczkowski, B. M.Bajracharya, A.Bakst, L.Ball, S.Barilari, M.Bault, N.Beaton, D.Beitner, J.Benoit, R. G.Berkers, R. M. W. J.Bhanji, J. P.Biswal, B. B.Bobadilla-Suarez, S.Bortolini, T.Bottenhorn, K. L.Bowring, A.Braem, S.Brooks, H. R.Brudner, E. G.Calderon, C. B.Camilleri, J. A.Castrellon, J. J.Cecchetti, L.Cieslik, E. C.Cole, Z. J.Collignon, O.Cox, R. W.Cunningham, W. A.Czoschke, S.Dadi, K.Davis, C. P.Luca, A. D.Delgado, M. R.Demetriou, L.Dennison, J. B.Di, X.Dickie, E. W.Dobryakova, E.Donnat, C. L.Dukart, J.Duncan, N. W.Durnez, J.Eed, A.Eickhoff, S. B.Erhart, A.Fontanesi, L.Fricke, G. M.Fu, S.Galván, A.Gau, R.Genon, S.Glatard, T.Glerean, E.Goeman, J. J.Golowin, S. A. E.González-García, C.Gorgolewski, K. J.Grady, C. L.Green, M. A.Guassi Moreira, J. F.Guest, O.Hakimi, S.Hamilton, J. P.Hancock, R.Handjaras, G.Harry, B.B.Hawco, C.Herholz, P.Herman, G.Heunis, S.Hoffstaedter, F.Hogeveen, J.Holmes, S.Hu, C. P.Huettel, S. A.Hughes, M. E.Iacovella, V.Iordan, A. D.Isager, P. M.Isik, A. I.Jahn, AndrewJohnson, Matthew R.Johnstone, TomJoseph, Michael J. E.Juliano, Anthony C.Kable, Joseph W.Kassinopoulos, MichalisKoba, CemalKong, Xiang-ZhenKoscik, Timothy R.Kucukboyaci, Nuri ErkutKuhl, Brice A.Kupek, SebastianLaird, Angela R.Lamm, ClausLangner, RobertLauharatanahirun, NinaLee, HongmiLee, SangilLeemans, AlexanderLeo, AndreaLesage, EliseLi, FloraLi, Monica Y. C.Lim, Cheng PhuiLintz, Evan N.Liphardt, Schuyler W.Losecaat Vermeer, Annabel B.Love, Bradley C.Mack, Michael L.Malpica, NorbertoMarins, TheoMaumet, CamilleMcDonald, KelseyMcGuire, Joseph T.Méndez Leal, Adriana S.Meyer, BenjaminMeyer, Kristin N.Mihai, GladMitsis, Georgios D.Moll, JorgeNielson, Dylan M.Nilsonne, GustavNotter, Michael P.Olivetti, EmanueleOnicas, Adrian I.Papale, PaoloPatil, Kaustubh R.Peelle, Jonathan E.Pérez, AlexandrePischedda, DorisPoline, Jean-BaptistePrystauka,YaninaRay, ShrutiReuter-Lorenz, Patricia A.Reynolds, Richard C.Ricciardi, EmilianoRieck, Jenny R.Rodriguez-Thompson, Anais M.Romyn, AnthonySalo, TaylorSamanez-Larkin, Gregory R.Sanz-Morales, EmilioSchlichting, Margaret L.Schultz, Douglas H.Shen, QiangSheridan, Margaret A.Silvers, Jennifer A.Skagerlund, KennySmith, AlecSmith, David V.Sokol-Hessner, PeterSteinkamp, Simon R.Tashjian, Sarah M.Thirion, BertrandThorp, John N.Tinghög, GustavTisdall, LoreenTompson, Steven H.Toro-Serey, ClaudioTorre Tresols, Juan JesusTozzi, LeonardoTruong, VuongTurella, Lucavan ‘t Veer, Anna E.Verguts, TomVettel, Jean M.Vijayarajah, SaganaVo, KhoiWall, Matthew B.Weeda, Wouter D.Weis, SusanneWhite, David J.Wisniewski, DavidXifra-Porxas, AlbaYearling, Emily A.Yoon, SangsukYuan, RuiYuen, Kenneth S. L.Lei ZhangZhang, XuZosky, Joshua E.Thomas E. Nichols,Poldrack, Rusell A.Schonberg, TomMelero Carrasco, HelenaPsicología (Psicología)6106.01 Actividad CerebralData analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2–5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.Universidad Complutense de Madrid20202020-05-2020202020-05-20journal articlehttp://purl.org/coar/resource_type/c_6501info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/20.500.14352/100828reponame:Docta Complutenseinstname:Universidad Complutense de Madrid (UCM)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:docta.ucm.es:20.500.14352/1008282026-06-02T12:44:21Z
score	15,300724

Variability in the analysis of a single neuroimaging dataset by many teams

Similares en LA Referencia