Winner 2022 - Preregistering qualitative research: our known unknown

Open Research objectives/practices

• Objective: Using online tools and services to increase the transparency of research processes and methodologies.
• Objective: Making scientific research more reproducible by increasing the amount and quality of information placed on the public record. NOTE: Even though the original objective of the ORA focuses on reproducibility, in qualitative research we prefer trustworthiness over reproducibility.
• Practice: Creating a public pre-registration of a study design

Introduction

Our research project aims to provide a theoretical model that explains how social interactions in childhood shape one’s somatic symptom proneness in adulthood. What better way to form this theoretical model than via a large qualitative study? And as social scientific research such as ours should be verifiable and trustworthy, what better way to achieve this than to preregister this research project? After all, adequate verifiability and trustworthiness requires predefined research questions, transparent hypotheses and methods. Here, however, our big known unknown came into play: we were aware of the practice of preregistering research and already had some experience with this in quantitative research, but preregistering qualitative research was still unknown territory.

Motivation

Qualitative research, especially research following a grounded theory approach, is an iterative process by design. Inherent to the cyclical process is the constant redevelopment and redefinition of the methods of data analysis. Even data collection, the step before data analysis, is arguably an inconsistent process as the researcher functions as an extension of the measurement instrument itself: different interviewers have different styles of interviewing, different triggers, and different associations with the narrative of the interviewee. The goal of qualitative research is to answer the why, what and how of phenomena, as perceived first-handedly by participants. Ultimately hypotheses will be generated, contrasting with quantitative research which often involves confirming or rejecting hypotheses. This flexibility (although some may call it structured chaos) interfered with our idea of preregistration: ‘freezing’ our research question, hypotheses and methods at a certain timepoint ensuring that these are not altered to fit the results found after data analyses (also referred to as HARKing: Hypothesizing After Results are Known). Furthermore, due to the flexible nature of qualitative research, reproducibility is not necessarily an attribute of it, while trustworthiness is.

Lessons learned

Thus, the prevailing forms of preregistration were challenged, since 1) predefined hypotheses are absent in our qualitative research; 2) our qualitative research iteratively combines data analysis and data collection and; 3) the high level of subjectivity in our qualitative research. We were therefore doubtful on how we could, and even whether we should, preregister our research project. Nevertheless, we decided it was something to get our teeth into. A preregistration is more than merely stating hypotheses: it allows for the scientific community to scrutinize our work. Preregistering our work also does not force us to abandon the flexibility of qualitative research, as a clear motivation of deviations from the preregistered protocol does not make our research any less valid. And last, every qualitative researcher has its own lens through which they try to answer the phenomenon’s why, what and how. Although the interpretative and cyclical process of qualitative research cannot be predefined, a preregistration forces researchers to explicate their research paradigm, traditions and lens.

We learned that our preregistration should become a ‘rolling’ document: it should built upon earlier versions, instead of ‘freezing’ our research approach in time. It should provide an audit trail for all to assess if our research deviates from the initial plan. It should further aid in enhancing the credibility and trustworthiness of the research. Therefore, we clearly explain the credibility strategies, such as reflexivity and consensus building among team members, we aim to use to enhance the trustworthiness of the research. Our first version of the preregistration can be found on https://osf.io/a4gn2.

We are constantly learning more about open science, and about how to apply this in our research, both qualitative and quantitative. However, this learning process too, is an iterative process. In this way, we do our best to change our known unknown into a known known.

URLs, references and further information

https://osf.io/a4gn2