Speaker
Description
Multiverse analysis offers a powerful framework to assess the robustness of statistical inferences across a spectrum of plausible analytical choices. However, when applied to predictor selection, especially in high-dimensional settings, the issue of multiplicity becomes critical. In this study, we present a comprehensive simulation framework to evaluate the impact of different multiple testing correction strategies—such as Bonferroni, Benjamini-Hochberg, and permutation-based approaches—on the stability and interpretability of predictor selection within multiverse analyses.
We further demonstrate the practical implications of these findings through a case study on lung cancer staging, using data from the SEER (Surveillance, Epidemiology, and End Results) program. Target variables include clinical stage classifications, with predictors drawn from demographic, tumor-specific, and treatment-related features. Our multiverse approach explores variations in preprocessing, model specification, and selection criteria, revealing how multiplicity corrections influence the perceived robustness of key predictors.
Results highlight the trade-offs between false discovery control and model generalizability, emphasizing the need for transparent reporting and principled correction strategies in multiverse workflows. This work contributes to the growing discourse on reproducibility and robustness in biomedical data science, offering practical guidance for researchers navigating complex modeling landscapes.
References
1. Steegen S, Tuerlinckx F, Gelman A, Vanpaemel W. Increasing Transparency Through a Multiverse Analysis. Perspect Psychol Sci. 2016 Sept 1;11(5):702–12.
2. Streiner DL. Best (but oft-forgotten) practices: the multiple problems of multiplicity—whether and how to correct for many statistical tests. Am J Clin Nutr. 2015 Oct;102(4):721–8.
53573505644