Updated: Oct 19
On: Checking the robustness of research results in four efficient steps
Michéle is assistant professor at the Meta-Research Center at Tilburg University. She will speak on the topic of 'Checking the robustness of research results in four efficient steps' at the national Symposium on Research Integrity on October 15th.
We asked her to share some of her thoughts on this topic in advance:
1. What made you interested in joining this symposium as a speaker? Why is the topic of research integrity important to you?
In both my research and my teaching, I think a lot about how we can improve the scientific system and improve research practices. Integrity is an important part of that.
2. Regarding the topic of your talk at the symposium:
a. Why is this topic particularly relevant now, to the Dutch research community?
Scientists in the Netherlands seem to be frontrunners in the open science movement. In psychology, my field, a lot of the proposed initiatives are focused on replication: redoing a study in a new sample to see if you find the same (or similar) outcomes. For example, the Dutch funder NWO piloted a grant specifically for replication research. I wholeheartedly agree that replication is important, but I also think we can find ways to make the process more efficient.
b. Can you tell us one of your main take away messages from your talk that we can already highlight?
In my talk, I will argue that before replicating a study, it is important to first check a study’s analytical reproducibility. In other words: if you reanalyze the data, will the same numbers roll out as the ones reported in the paper? This may seem like a given, but in practice many reported results turn out not to be reproducible: they cannot be traced back to the underlying data. Analytical reproducibility is a prerequisite for replication, so I will suggest a four-step robustness check that focuses on analytical verification before replication.
3. The symposium is being organised for researchers across the Netherlands. What do you hope as an outcome for an event where research integrity is being discussed?
I hope that researchers become (or remain) comfortable discussing integrity, especially the “gray areas” and difficult cases. As long as we stay in open communication with each other and are realistic about our flaws as humans, I think we can achieve a lot in making science more robust.
4. Anything else you would like to share that may be interesting for our (online) audience?
Earlier this year, Joeri Tijdink, Serge Horbach, Gareth O’Neill and I wrote a position paper for ZonMw highlighting six main themes that we think need more research in order to promote responsible research practices. The paper can be found here: https://www.zonmw.nl/fileadmin/zonmw/documenten/Fundamenteel/BVO/PositionPaper_Promoting_Responsible_Research_Practices__2019_.pdf