Open Research practices in the study of concealed information detection

Open Research objectives/practices

To find reliable ways to detect concealed crime-relevant information has long been a major goal of forensic scientists. A newly developed concealed information test (CIT), with items presented in rapid serial visual presentation (RSVP), is a promising tool to detect concealed information in combination with electroencephalography (EEG). This new method can prevent the use of countermeasures to avoid detection, a deficit of the original CIT. Since EEG is not applicable in practice, we proposed a solution using pupil dilation as a measure that only requires a camera for data collection. During the whole study, we used Open Science Frame to share all of the code and data as well as preregister our analysis plan. We also published the paper with full open access.

Introduction

1. We conducted two experiments, of which all of the materials, code, and data are available online on the Open Science Frame (OSF).
2. We preregistered the analysis plan of the second experiment in order to replicate the result of the first experiment.
3. Additionally, we pre-printed the paper with the outcomes on bioRxiv, an open access preprint repository.
4. Finally, we published the paper in Psychophysiology, with full open access.

Motivation

In practical terms, the OSF is a useful frame for storing experimental materials and data. Our group members were able to share the resources and check updates without sending files via email. We could trace back to the old versions of these resources without worrying about data loss. Once we made the resources available publicly, other researchers were also able to reproduce the results and understand how we collected and analysed data, by using the all of the materials on OSF.

The preregistered analysis plan helped to ensure our findings were robust and reliable. Once the analysis plan was preregistered on OSF, it couldn’t be changed any more. We analysed the data in our second experiment based on the preregistered plan. It avoided any form of data fishing, and thereby made our results more convincing.

Our goal with posting the preprint version was to enable rapid dissemination of our results, and to allow other researchers to get up to speed and build on our findings as soon as possible. Indeed, once our paper was pre-printed and published, we received feedback and inquiries immediately. Open Research practices thus clearly sped up the communication in science.

Lessons learned

Some personal information, such as names of participants, were initially contained in our experimental materials. It is challenging to keep such information confidential when we uploaded the code and data online and made them Open Access. However, we finally managed to maintain confidentiality by deleting the personal data or replacing them of meaningless symbols, without damaging the functions of the code and data.

URLs, references and further information

The first experiment: https://osf.io/9fkpm/
The preregistered analysis plan and the second experiment: https://osf.io/9re75/
The pre-print: https://www.biorxiv.org/content/10.1101/2021.06.18.448944v2
The open access publication: https://onlinelibrary.wiley.com/doi/full/10.1111/psyp.14155