Nature News: Half of top cancer studies fail high-profile reproducibility effort
The topic of this report is old- every a few years there were special issues in high-profile journals to discuss low replicatability in preclinical studies (both in vitro and in vivo) and call for systemically evaluating. Finally this is done and the results were published in eLife now. These studies confirm, again, the replicatability is low. OK, so what’s next?
I have worked on cancer modeling in mice for two decades, with hands-on experience in most of the time, so please allow me to provide a two-word answer from my experience: infrastructure and standardization. Institutions should invest to build teams and records for:
Mouse breeding and genotyping
Cell line maintenance and QC (such as STR assay, markers, sequencing results)
Therapeutic protocols
Endpoint analysis and criteria
Many institutes do not intend to invest in these infrastructures. Graduate students, postdocs, and research scientists are left alone to do all of these tasks, without help from and could not contribute to the infrastructures. When they leave, everything will be started all over again in Sisyphus style, and the publications are like castles built on sand. People on top of the pyramid try to use manpower to fill gap (“go counting these 500 slides and give me the statistics on Monday”), rather than investing in developing better methods.
To change the situation, we have to learn from "KAIZEN", a strategy employed by Japanese automobile industry, where employees at all levels of a company work together proactively to achieve regular, incremental improvements to the manufacturing process, preventing the tiny singularity events that accumulate to cause collapse. As the very first step, I believe we need to establish databases for research groups to upload published or unpublished preclinical study data, allowing meta-analysis for any given treatment, and examining the effects of different details (mouse strain, age, sex, etc.). This is step 1 in Edward Deming's approach!
I have worked on cancer modeling in mice for two decades, with hands-on experience in most of the time, so please allow me to provide a two-word answer from my experience: infrastructure and standardization. Institutions should invest to build teams and records for:
Mouse breeding and genotyping
Cell line maintenance and QC (such as STR assay, markers, sequencing results)
Therapeutic protocols
Endpoint analysis and criteria
Many institutes do not intend to invest in these infrastructures. Graduate students, postdocs, and research scientists are left alone to do all of these tasks, without help from and could not contribute to the infrastructures. When they leave, everything will be started all over again in Sisyphus style, and the publications are like castles built on sand. People on top of the pyramid try to use manpower to fill gap (“go counting these 500 slides and give me the statistics on Monday”), rather than investing in developing better methods.
To change the situation, we have to learn from "KAIZEN", a strategy employed by Japanese automobile industry, where employees at all levels of a company work together proactively to achieve regular, incremental improvements to the manufacturing process, preventing the tiny singularity events that accumulate to cause collapse. As the very first step, I believe we need to establish databases for research groups to upload published or unpublished preclinical study data, allowing meta-analysis for any given treatment, and examining the effects of different details (mouse strain, age, sex, etc.). This is step 1 in Edward Deming's approach!
Please sign in or register for FREE
If you are a registered user on Research Communities by Springer Nature, please sign in