Frederik Aust, Samuel Pawel, Eric-Jan Wagenmakers
This repository contains all data nd code to reproduce the results of our paper.
The p-value and the Bayes factor are measures of evidence that are often considered to be philosophically and mathematically incompatible: The p-value quantifies conflict between data and H0 ("surprise"), whereas the Bayes factor quantifies the relative predictive accuracy of H0 versus H1 ("evidence"). We revisit Jeffreys’s Approximate Bayes factor (JAB)---a simple, largely overlooked approximation dating back to the 1930s---which connects these two paradigms for objective hypothesis testing of the existence of an effect. Under a unit-information prior the approximation requires only the p-value and the effective sample size. We clarify the core assumptions and boundary conditions for the application of JAB and show across 704 published t-tests and 39 comparisons of proportions that JAB approximates objective Bayes factors remarkably well. The connection between p-values and JAB has a practical implication: The evidence implied by a p-value depends strongly on effective sample size. Conventional verbal labels for p-values (e.g., "strong surprise" for .001 < p < .01) correspond to similarly graded Bayes factors only around effective sample size of approximately 8; for larger samples the same p-value implies weaker evidence. In moderately sized to large samples, p > .10 can amount to moderate or even strong evidence for H0. JAB offers a cheap, sample-size-sensitive supplement to p-values, computable from routinely reported statistics, that remains valid even under optional stopping.
The manuscript was rendered using Quarto (Version 1.9.38) and R (Version
4.5.3). The computational environment including all R packages is
documented in ./dependencies.md.
@Article{aust_evidence_p_2026,
title = {Extracting Bayesian Evidence from Frequentist $p$ Values},
author = {Aust, Frederik and Pawel, Samuel and Wagenmakers, Eric-Jan},
year = {2026},
doi = {}
}