Impact of mood-induction on maladaptive thinking in the vulnerability for depression: pre-registration & data and code sharing

Open Research objectives/practices

Pre-registration of the study design and analysis plan and making data and source code publicly available.

Introduction

As part of my Masters project, I designed a study together with my supervisors Marie-José van Tol and Marieke van Vugt. In this study we wanted to examine the impact of mood-induction on maladaptive thinking in the vulnerability for depression. At the start of the data collection for the study we have pre-registered the study design, sampling plan, variables and analysis plan at Open Science Framework. This helped me think carefully of the design, hypotheses and data analysis already before the start of the data collection.
Yet, preregistration also had its challenges. A year after finishing my Masters project, I started a PhD with Marie-José van Tol and Marieke van Vugt. By that time I learnt about more advanced statistical modelling which made me rethink the analysis that I did in my Masters project. We found that the analysis method that I learned about better fitted the data. Therefore, we decided that we should apply this method to the data. Moreover, we calculated Bayes Factors to test the strength of evidence for either the null hypothesis or the alternative hypothesis. Bayes Factor showed that with the pre-registered sample size of n=20 participants per group, we could not distinguish between the alternative hypothesis and the null hypothesis. Therefore, another study with n=20 participants per group was performed to double the sample size to n=40 per group. As the analysis plan was already pre-registered, we had to update the original registration including our motivation to change the analysis and increase the sample size. Now the paper is finished and ready for submission. Included in the manuscript is the URL to the pre-registration and the data and source code that is publicly available at Open Science Framework.

Motivation

I believe pre-registration is important because it makes research more transparent, which was one of the main reasons to pre-register our project. With the pre-registration we can show others reading the paper once published that we did not change the study design, hypothesis or sampling plan after the data was collected in order to find significant results. It was good to see that we were still able to add a note to our original pre-registration where we could explain why we had to change our analysis plan and sample size and add the adjustments to the plan. Besides, the pre-registration helped me to already think of the hypotheses, outcome variables and analysis plan at an early stage. It allowed me to think critically about the design of the study related to the goals of our research. Furthermore, it helped me when I had to start with the data analysis and writing. Besides the pre-registration, we also made our data and source code publicly available. This allows researchers to replicate the study and check if what we did was correct. This also further benefits transparency in academia. Sharing the data and analysis scripts will also help me at the revision stage of writing.

Lessons learned

I have learned that pre-registration is important for transparency and open research. When we had to change the analysis plan and sample size I saw the pre-registration as a barrier at the beginning but soon after I found that if you show a good motivation and explanation for changing the analysis plan, this is still possible if you have good reasons to do so.

URLs, references and further information

Data and source code is available at: https://osf.io/zab4z/.
The study’s design and analysis plan were preregistered at: https://osf.io/m97pd/.