Speaker
Description
When addressing a particular research question using observational data, many decisions must be made during the conceptualization of the statistical analysis plan. This multiplicity of analysis strategies is a well-known problem that leads to high variation in research findings and associated low replicability, since each decision can lead to different results, even if each decision on its own was scientifically justifiable.
Several approaches for reporting and visualizing this variation of effects have been proposed to address the consequences of the multiplicity of analysis strategies. For example, the social science literature advocates the multi-model approach of Young and Holsteen, which presents a preferred model estimate alongside results from other plausible models. This concept is similar to sensitivity analysis, which is often used in epidemiological research, but it mainly focuses on aspects of the main analysis model, such as variations in the adjustment variables or the definitions of the exposure and outcome variables. Similarly, the vibrations of effects framework of Patel mainly is concerned with changes in the effect measure estimate due to different adjustment sets and offers a neat visual representation in the form of volcano plots. Specification curves by Simonsohn et al. are an alternative suggestion to visualize the variation in results due to different decisions made at various stages of a statistical analysis, including also the data pre-processing phase. Multiverse-style methods by Steegen et al. , which have been developed within the data science community, also aim to neutrally present the results of ‘’all’’ possible analysis decisions.
We exemplify these methods for addressing multiplicity in analysis strategies using a pharmacoepidemiological case study. In particular, we aimed to compare the effect of clopidogrel, ticagrelor, and prasugrel in a cohort of 12 000 stented acute coronary syndrome (ACS) patients regarding time to first ACS‑related readmission or death. The data were supplied by the Austrian Health Insurance Fund, and included basic demographic information as well as information on all hospitalization between 2019 and 2023, individual medical service codes and all drug redemptions in that time period.
We present our preferred and alternative analysis strategies, and visualize and numerically quantify the variation of results according to the approaches mentioned above in addition to reflections regarding their implementation.
75002916929