Meta-research: Using shared data to provide practical solutions for current research challenges

Research on research: Finding real solutions to real research problems

Clinical studies face many methodological and practical challenges that sometimes limit the validity of study results or lead to premature study discontinuation or even non-publication. It would be helpful for researchers to have reliable information about the advantages and disadvantages of specific research methods and processes available; however, such evidence is scarce. Research on research (RoR), or meta-research, aims to investigate the research process or research methods themselves in order to create this evidence and provide guidance. It should produce actionable findings and outputs (e.g. tools, recommendations, or new statistical methods) that can be used by actors and stakeholders in the clinical research ecosystem. Close collaboration between evidence producers (e.g. meta-researchers) and evidence users (e.g. CTU staff or clinical researchers) is necessary in order to ensure that pressing problems in research practice are addressed and results are delivered in the most convenient formats. This idea of collaboration between meta-researchers and CTU staff across Switzerland led to the foundation of the STEAM (Swiss clinical Trials Empirical Assessment & Methods) working group in 2019, a bottom-up initiative of meta-researchers working with the Swiss Clinical Trial Organisation (SCTO) and its Clinical Trial Unit (CTU) Network to tackle methodological and practical aspects of clinical studies. The White Paper: Clinical Research, published by the Swiss Academy of Medical Sciences in 2021, mentioned STEAM and the promotion of RoR as part of a roadmap to further strengthen clinical research in Switzerland.¹

In principle, STEAM members take up issues and problems identified by clinical research stakeholders or encountered in actual clinical studies. They then generate the corresponding research questions and devise methodology for RoR projects that address these issues. The results and outputs from STEAM’s RoR projects (e.g. checklists, tools, publications, and guidelines) are fed back into research practice through teaching and training as well as consulting and collaboration. This creates a clinical research learning system for the continuous improvement of the quality, transparency, and value of clinical research (see Figure 1). In addition, STEAM members actively reach out to national stakeholders (e.g. swissethics and the Swiss National Science Foundation), they contribute to international initiatives (e.g. Trial Forge), and they participate in European and international RoR efforts (e.g. the European Cooperation in Science and Technology (COST) Action on evidence-based research). The STEAM working group currently meets twice per year to discuss current projects, recommendations, tools, publications, and priorities and to initiate new RoR projects among members. It welcomes new researchers with an interest in RoR.

Research data sharing: The key to meta-research

Access to research data is key to RoR. The further use of collected data can improve current knowledge and help update recommendations. Data sharing in clinical research has many advantages and faces various challenges that are discussed in other articles of this issue of RA Watch. The following viewpoint focuses exclusively on research data sharing from the RoR perspective.

First, it is important to point out that individual participant data (IPD) from a clinical trial can be useful for investigating trial processes such as participant recruitment or retention. For example, in a current study we are using IPD on the enrolment dates of almost 300 randomised trials to empirically investigate recruitment patterns and to develop and evaluate user-friendly recruitment prediction tools.² Second, apart from IPD, there are metadata in the form of trial protocols, case report forms, or data analysis plans that may be shared to enable meta-researchers to empirically investigate risks for bias (e.g. selective outcome reporting), problems of study conduct (e.g. insufficient recruitment), or non-publication.^3,4,5 Other metadata that would be valuable for RoR are resource use and cost data for various tasks in clinical studies.⁶ An increased availability of shared cost data would help meta-researchers, for instance, to evaluate new study designs such as registry-based randomised trials⁷ or clinical researchers and funding agencies to make more accurate budget estimations and budget approval decisions. A third important aspect of sharing metadata is related to the confidentiality concerns of various stakeholders. Traditionally, concerns about confidentiality have been raised mainly by data producers and ethics committees. However, as patient representatives have become increasingly engaged in clinical research in recent years – a positive development thanks to patient and public involvement (PPI) initiatives – they, too, have started expressing concerns about the risks of privacy breaches. Yet despite these concerns, patients are generally very much in favour of data sharing.^8,9 Such concerns were addressed in STEAM’s past RoR projects through direct mandates from stakeholders (e.g. ethics committees) in combination with signed confidentiality agreements.¹⁰ Finally, sharing IPD itself – as one process in the clinical research enterprise – is a timely topic for RoR.  Unanswered questions about data sharing are, for instance, the following: Which methods of de-identification of participant data are most appropriate in the Swiss context? What is a suitable metadata scheme for data sets from clinical studies to ensure findability in data repositories? What is the best way to monitor and assess the impact of IPD reused from clinical trials?

References

1. Swiss Academy of Medical Sciences (SAMS) (2021) White Paper: Clinical Research. Swiss Academies Communications 16(4). doi: 10.5281/zenodo.4772725
2. Kasenda B et al. (2020) Prediction of RECRUITment in randomized clinical trials (RECRUIT-IT): Rationale and design for an international collaborative study. Trials 21(1):731. doi: 10.1186/s13063-020-04666-8 [includes contributions from STEAM members]
3. Speich B et al. (2022) Non-registration, discontinuation, and non-publication of randomized trials: A repeated meta-research analysis. PloS Med 2022 [Epub ahead of print]. doi: 10.1371/journal.pmed.1003980 [includes contributions from STEAM members]
4. Amstutz A et al. (2017) Discontinuation and non-publication of randomized clinical trials supported by the main public funding body in Switzerland: A retrospective cohort study. BMJ Open 7(7):e016216. doi: 10.1136/bmjopen-2017-016216 [includes contributions from STEAM members]
5. Redmond S et al. (2013) Cohort study of trials submitted to ethics committee identified discrepant reporting of outcomes in publications. Journal of Clinical Epidemiology 66(12):1367–75. doi: 10.1016/j.jclinepi.2013.06.020 
6. Speich B et al. (2018) Systematic review on costs and resource use of randomized clinical trials shows a lack of transparent and comprehensive data. Journal of Clinical Epidemiology 96:1–11. doi: 10.1016/j.jclinepi.2017.12.018 [includes contributions from STEAM members]
7. Karanatsios B et al. (2020) Defining key design elements of registry-based randomised controlled trials: A scoping review. Trials 21, 552. doi: 10.1186/s13063-020-04459-z
8. Chan AW et al. (2006) Research protocols: Waiving confidentiality for the greater good. BMJ 332:1086–9. doi: 10.1136/bmj.332.7549.1086
9. Mello MM, Van Lieou BS, and Goodman SN (2018) Clinical trial participants’ views of the risks and benefits of data sharing. New England Journal of Medicine 378(23):2202–2211. doi: 10.1056/NEJMsa1713258
10. Kasenda B et al. (2014) Prevalence, characteristics, and publication of discontinued randomized trials. JAMA 311(10):1045–1052. doi: 10.1001/jama.2014.1361 [includes contributions from STEAM members]