For my first paper as a PhD student, I wrote a preregistration. This case study "To preregister or not? A case study
about preregistration of a quantitative study using secondary data in the social sciences" will highlight some challenges that we faced with preregistering our analytical strategy. Within a larger project on the role of peer experiences and its consequences for relationships in adolescence and adulthood, my study examined if bullying perpetration (being a bully) of parents would predict bullying behaviours of their children. We used secondary data from two large longitudinal cohorts, which were carried out in England. Prior to preregistration, we had access to information about the items and their frequencies provided by The Centre for Longitudinal Studies, however, we did not have information about correlations between items.

Why did we preregister? A preregistration is a preconceived analytic plan and entails committing to the methodological and analytic steps in that plan, without advance knowledge of the outcomes (Nosek et al., 2018). A preregistration can hereby increase the transparency and reproducibility of a study. For our study, information about the research questions and hypotheses, prior knowledge of the data, the available items and their transformations, exclusion criteria, methods
of dealing with outliers and missing data, a power analysis, and the statistical models and their inference criteria were preregistered before downloading the needed data.

A real challenge with a preregistered analytic strategy is that one has to deal with eventual deviations from the planned and preregistered analytic strategy in the most transparent and honest way. For instance, for our study we planned to estimate six complex structural equation models and we expected that we could estimate all those models and indicate in our paper that we did everything as described in our preregistration. However, not all preregistered analyses were feasible with the models we had planned and, eventually, all of them needed adjusting. We had, for example, planned to use one construct of bullying perpetration as predictor, based upon bullying items assessed by parentsand teacher-reports. Yet, when we estimated the models in this way, the model fits were not acceptable as the parentsand teacher-reports did not work well together as one construct. We therefore decided, after inspecting the correlations more thoroughly, to separate the parentand teacher-items into two constructs. This shows that reality is a bit more unruly and, especially for complex models, it can sometimes be difficult to predict in advance whether a model can be estimated as you have planned. Preregistering these kind of models can thus be challenging because one often needs to deviate from the original plan.

How to deal with those deviations? Firstly, a great benefit of the preregistration is that prior to conducting the study, one already gathered information about the best possible statistical technique(s) for the analyses. If one has to deal with deviations, it is easier to make adjustments within the planned strategy itself than to come up with a new strategy. Furthermore, one of the reasons why preregistration is a transparent method is due to the fact that one must describe in detail why and how one deviated from the preregistered analyses. In this way, adjustments that were made are clarified for everyone.

To conclude, an important benefit of preregistration is that prior conducting a study, methodological and analytic decisions are made, which improves the  transparency and trustworthiness of that study. Even if difficulties with the preregistered analyses are encountered, as some decisions are easier made when the data are known, those difficulties are easier to overcome by staying close to the proposed plans and by detailed documentation of possible deviations.
Preregistering a study is useful in the vast majority of cases and should be an important part of the research process nowadays.

References and links

1. Centre for Longitudinal Studies
2. Nosek, B. A., Ebersole, C. R., DeHaven, A. C., &
   Mellor, D. T. (2018). The preregistration revolution.
   Proceedings of the National Academy of Sciences,
   115(11), 2600–2606.
   https://doi.org/10.1073/pnas.1708274114
3. Preregistration: https://osf.io/kacng/?view_only=f97e7430579d471389c5b6ad87f883e0