Speaker
Description
Animal experiments are often purely exploratory, with little to no data available to support the planning phase. Nonetheless, ethical guidelines demand scientifically sound biometric planning. The experimental designs are typically complex, involving numerous experimental groups and adaptive steps, which complicates statistical planning.
In recent years, statistical aspects of such experimental designs have been increasingly advocated by authorities and the scientific community [1]. However, so far, no statistical approach actually acknowledge the complexity of the experimental designs. Instead, statistical planning usually focuses on a small subset of the design, e.g., a two-groups-comparison, and applies classical biometrical methodology from clinical trials, i.e., 5% type I error rate, 80% power, and a priori estimated effect size. Often, with the argument of the experiment being “exploratory” instead of “confirmatory”, this biometric justification for the sample size of the two-group comparison is extrapolated to the rest of the experiment. Even though it is widely known that effect sizes from animal experiments are strongly biased and suffer from poor replicability and translation to clinical trials [2, 3], little emphasis has been put on this remarkable gap between experimental research and statistical planning.
We demonstrate that common design practices in animal experiments introduce substantial error in effect size estimation, even if properly adjusted for inflated type I error rates and false discovery rates. To address this, we propose a simulation-based approach to quantify the estimation error and to classify its magnitude compared to a reference design, enabling an intuitive assessment of the suitability of a specific experimental design. Additionally, we present and dicuss resampling estimators that improve effect estimation and reduce estimation error even in complex experimental designs, consequently contributing to the reproducibility of preclinical findings.
[1] Piper, Sophie K., et al. "Statistical review of animal trials—A guideline." Biometrical Journal, 65(2): 1-12. 2023.
[2] Kimmelman, Jonathan, Jeffrey S. Mogil, and Ulrich Dirnagl. "Distinguishing between exploratory and confirmatory preclinical research will improve translation." PLoS biology, 12(5): 1-4. 2014.
[3] ter Riet, Gerben, et al. "Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions." PLoS ONE, 7(9): e43404. 2012.
42858803969